[linux] How to split a list by comma not space

I want to split a text with comma , not space in for foo in list. Suppose I have a CSV file CSV_File with following text inside it:

Hello,World,Questions,Answers,bash shell,script
...

I used following code to split it into several words:

for word in $(cat CSV_File | sed -n 1'p' | tr ',' '\n')
do echo $word
done

It prints:

Hello
World
Questions
Answers
bash
shell
script

But I want it to split the text by commas not spaces:

Hello
World
Questions
Answers
bash shell
script

How can I achieve this in bash?

This question is related to linux bash shell csv split

The answer is


Read: http://linuxmanpages.com/man1/sh.1.php & http://www.gnu.org/s/hello/manual/autoconf/Special-Shell-Variables.html

IFS The Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read builtin command. The default value is ``''.

IFS is a shell environment variable so it will remain unchanged within the context of your Shell script but not otherwise, unless you EXPORT it. ALSO BE AWARE, that IFS will not likely be inherited from your Environment at all: see this gnu post for the reasons and more info on IFS.

You're code written like this:

IFS=","
for word in $(cat tmptest | sed -n 1'p' | tr ',' '\n'); do echo $word; done;

should work, I tested it on command line.

sh-3.2#IFS=","
sh-3.2#for word in $(cat tmptest | sed -n 1'p' | tr ',' '\n'); do echo $word; done;
World
Questions
Answers
bash shell
script

Create a bash function

split_on_commas() {
  local IFS=,
  local WORD_LIST=($1)
  for word in "${WORD_LIST[@]}"; do
    echo "$word"
  done
}

split_on_commas "this,is a,list" | while read item; do
  # Custom logic goes here
  echo Item: ${item}
done

... this generates the following output:

Item: this
Item: is a
Item: list

(Note, this answer has been updated according to some feedback)


You can use:

cat f.csv | sed 's/,/ /g' |  awk '{print $1 " / " $4}'

or

echo "Hello,World,Questions,Answers,bash shell,script" | sed 's/,/ /g' |  awk '{print $1 " / " $4}'

This is the part that replace comma with space

sed 's/,/ /g'

I think the canonical method is:

while IFS=, read field1 field2 field3 field4 field5 field6; do 
  do stuff
done < CSV.file

If you don't know or don't care about how many fields there are:

IFS=,
while read line; do
  # split into an array
  field=( $line )
  for word in "${field[@]}"; do echo "$word"; done

  # or use the positional parameters
  set -- $line
  for word in "$@"; do echo "$word"; done

done < CSV.file

kent$  echo "Hello,World,Questions,Answers,bash shell,script"|awk -F, '{for (i=1;i<=NF;i++)print $i}'
Hello
World
Questions
Answers
bash shell
script

Set IFS to ,:

sorin@sorin:~$ IFS=',' ;for i in `echo "Hello,World,Questions,Answers,bash shell,script"`; do echo $i; done
Hello
World
Questions
Answers
bash shell
script
sorin@sorin:~$ 

Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to shell

Comparing a variable with a string python not working when redirecting from bash script Get first line of a shell command's output How to run shell script file using nodejs? Run bash command on jenkins pipeline Way to create multiline comments in Bash? How to do multiline shell script in Ansible How to check if a file exists in a shell script How to check if an environment variable exists and get its value? Curl to return http status code along with the response docker entrypoint running bash script gets "permission denied"

Examples related to csv

Pandas: ValueError: cannot convert float NaN to integer Export result set on Dbeaver to CSV Convert txt to csv python script How to import an Excel file into SQL Server? "CSV file does not exist" for a filename with embedded quotes Save Dataframe to csv directly to s3 Python Data-frame Object has no Attribute (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape How to write to a CSV line by line? How to check encoding of a CSV file

Examples related to split

Parameter "stratify" from method "train_test_split" (scikit Learn) Pandas split DataFrame by column value How to split large text file in windows? Attribute Error: 'list' object has no attribute 'split' Split function in oracle to comma separated values with automatic sequence How would I get everything before a : in a string Python Split String by delimiter position using oracle SQL JavaScript split String with white space Split a String into an array in Swift? Split pandas dataframe in two if it has more than 10 rows