[shell] Loop through a comma-separated shell variable

Suppose I have a Unix shell variable as below

variable=abc,def,ghij

I want to extract all the values (abc, def and ghij) using a for loop and pass each value into a procedure.

The script should allow extracting arbitrary number of comma-separated values from $variable.

This question is related to shell loops unix for-loop cut

The answer is


You can use the following script to dynamically traverse through your variable, no matter how many fields it has as long as it is only comma separated.

variable=abc,def,ghij
for i in $(echo $variable | sed "s/,/ /g")
do
    # call your procedure/other scripts here below
    echo "$i"
done

Instead of the echo "$i" call above, between the do and done inside the for loop, you can invoke your procedure proc "$i".


Update: The above snippet works if the value of variable does not contain spaces. If you have such a requirement, please use one of the solutions that can change IFS and then parse your variable.


Hope this helps.


Try this one.

#/bin/bash   
testpid="abc,def,ghij" 
count=`echo $testpid | grep -o ',' | wc -l` # this is not a good way
count=`expr $count + 1` 
while [ $count -gt 0 ]  ; do
     echo $testpid | cut -d ',' -f $i
     count=`expr $count - 1 `
done

#/bin/bash   
TESTSTR="abc,def,ghij"

for i in $(echo $TESTSTR | tr ',' '\n')
do
echo $i
done

I prefer to use tr instead of sed, becouse sed have problems with special chars like \r \n in some cases.

other solution is to set IFS to certain separator


Here's my pure bash solution that doesn't change IFS, and can take in a custom regex delimiter.

loop_custom_delimited() {
    local list=$1
    local delimiter=$2
    local item
    if [[ $delimiter != ' ' ]]; then
        list=$(echo $list | sed 's/ /'`echo -e "\010"`'/g' | sed -E "s/$delimiter/ /g")
    fi
    for item in $list; do
        item=$(echo $item | sed 's/'`echo -e "\010"`'/ /g')
        echo "$item"
    done
}

Another solution not using IFS and still preserving the spaces:

$ var="a bc,def,ghij"
$ while read line; do echo line="$line"; done < <(echo "$var" | tr ',' '\n')
line=a bc
line=def
line=ghij

If you set a different field separator, you can directly use a for loop:

IFS=","
for v in $variable
do
   # things with "$v" ...
done

You can also store the values in an array and then loop through it as indicated in How do I split a string on a delimiter in Bash?:

IFS=, read -ra values <<< "$variable"
for v in "${values[@]}"
do
   # things with "$v"
done

Test

$ variable="abc,def,ghij"
$ IFS=","
$ for v in $variable
> do
> echo "var is $v"
> done
var is abc
var is def
var is ghij

You can find a broader approach in this solution to How to iterate through a comma-separated list and execute a command for each entry.

Examples on the second approach:

$ IFS=, read -ra vals <<< "abc,def,ghij"
$ printf "%s\n" "${vals[@]}"
abc
def
ghij
$ for v in "${vals[@]}"; do echo "$v --"; done
abc --
def --
ghij --

Not messing with IFS
Not calling external command

variable=abc,def,ghij
for i in ${variable//,/ }
do
    # call your procedure/other scripts here below
    echo "$i"
done

Using bash string manipulation http://www.tldp.org/LDP/abs/html/string-manipulation.html


Here is an alternative tr based solution that doesn't use echo, expressed as a one-liner.

for v in $(tr ',' '\n' <<< "$var") ; do something_with "$v" ; done

It feels tidier without echo but that is just my personal preference.


Examples related to shell

Comparing a variable with a string python not working when redirecting from bash script Get first line of a shell command's output How to run shell script file using nodejs? Run bash command on jenkins pipeline Way to create multiline comments in Bash? How to do multiline shell script in Ansible How to check if a file exists in a shell script How to check if an environment variable exists and get its value? Curl to return http status code along with the response docker entrypoint running bash script gets "permission denied"

Examples related to loops

How to increment a letter N times per iteration and store in an array? Angular 2 Cannot find control with unspecified name attribute on formArrays What is the difference between i = i + 1 and i += 1 in a 'for' loop? Prime numbers between 1 to 100 in C Programming Language Python Loop: List Index Out of Range JavaScript: Difference between .forEach() and .map() Why does using from __future__ import print_function breaks Python2-style print? Creating an array from a text file in Bash Iterate through dictionary values? C# Wait until condition is true

Examples related to unix

Docker CE on RHEL - Requires: container-selinux >= 2.9 What does `set -x` do? How to find files modified in last x minutes (find -mmin does not work as expected) sudo: npm: command not found How to sort a file in-place How to read a .properties file which contains keys that have a period character using Shell script gpg decryption fails with no secret key error Loop through a comma-separated shell variable Best way to find os name and version in Unix/Linux platform Resource u'tokenizers/punkt/english.pickle' not found

Examples related to for-loop

List append() in for loop Prime numbers between 1 to 100 in C Programming Language Get current index from foreach loop how to loop through each row of dataFrame in pyspark TypeScript for ... of with index / key? Is there a way in Pandas to use previous row value in dataframe.apply when previous value is also calculated in the apply? Python for and if on one line R for loop skip to next iteration ifelse How to append rows in a pandas dataframe in a for loop? What is the difference between ( for... in ) and ( for... of ) statements?

Examples related to cut

Loop through a comma-separated shell variable cat, grep and cut - translated to python How to find the last field using 'cut' Using the grep and cut delimiter command (in bash shell scripting UNIX) - and kind of "reversing" it? Using cut command to remove multiple columns how to remove the first two columns in a file using shell (awk, sed, whatever) How can I remove the extension of a filename in a shell script? bash script use cut command at variable and store result at another variable How to specify more spaces for the delimiter using cut? Copy and paste content from one file to another file in vi