[linux] How to parse a CSV file in Bash?

I'm working on a long Bash script. I want to read cells from a CSV file into Bash variables. I can parse lines and the first column, but not any other column. Here's my code so far:


  cat myfile.csv|while read line
  do
    read -d, col1 col2 < <(echo $line)
    echo "I got:$col1|$col2"
  done

It's only printing the first column. As an additional test, I tried the following:

read -d, x y < <(echo a,b,)

And $y is empty. So I tried:

read x y < <(echo a b)

And $y is b. Why?

This question is related to linux bash csv

The answer is


We can parse csv files with quoted strings and delimited by say | with following code

while read -r line
do
    field1=$(echo "$line" | awk -F'|' '{printf "%s", $1}' | tr -d '"')
    field2=$(echo "$line" | awk -F'|' '{printf "%s", $2}' | tr -d '"')

    echo "$field1 $field2"
done < "$csvFile"

awk parses the string fields to variables and tr removes the quote.

Slightly slower as awk is executed for each field.


From the man page:

-d delim The first character of delim is used to terminate the input line, rather than newline.

You are using -d, which will terminate the input line on the comma. It will not read the rest of the line. That's why $y is empty.


If you want to read CSV file with some lines, so this the solution.

while IFS=, read -ra line
do 
    test $i -eq 1 && ((i=i+1)) && continue
    for col_val in ${line[@]}
    do
        echo -n "$col_val|"                 
    done
    echo        
done < "$csvFile"

Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to csv

Pandas: ValueError: cannot convert float NaN to integer Export result set on Dbeaver to CSV Convert txt to csv python script How to import an Excel file into SQL Server? "CSV file does not exist" for a filename with embedded quotes Save Dataframe to csv directly to s3 Python Data-frame Object has no Attribute (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape How to write to a CSV line by line? How to check encoding of a CSV file