Reading input files by line using read command in shell scripting skips last line

Question

I usually use the read command to read an input file to the shell script line by line  An example code such as the one below yields a wrong result if a new line isn t inserted at the end of the last line in the input file  blah txt      bin sh  while read line do echo  line done  lt blah txt   So if the input file reads something like -  One  Two Three Four   and I do not hit return after Four  the script fails to read the last line  and prints   One Two Three   Now if I leave an extra blank line after Four  like   One  Two Three Four   blank line   the output prints all the lines  including Four  However  this is not the  case when I read a line using the cat command  all lines including the last get printed without me having to add an extra blank line at the end   Anyone has ideas on why this happens  The scripts I create will mostly be run by others  so it isn t necessary they re going to add an extra blank line at the end of every input file   I ve been trying to figure this out for ages  I d appreciate it if you have any solutions of course  the cat command is one  but I d like to know the reason behind read not working as well

User · Answer

Use while loop like this   while IFS  read -r line      -n   line     do   echo   line  done  lt file   Or using grep with while loop   while IFS  read -r line  do   echo   line  done  lt   lt  grep    file    Using grep   instead of grep    will skip the empty lines   Note     Using IFS  keeps any line indentation intact  You should almost always use the -r option with read  Don t read lines with for File without a newline at the end isn t a standard unix text file

User · Answer

One line answer   IFS    n   for line in   cat file txt   do echo   line    done

User · Answer

DONE false until  DONE do   read line    DONE true   echo  line done  lt  blah txt   How to use  while read   Bash  to read the last line in a file if there   s no newline at the end of the file

User · Answer

Below code with Redirected  while-read  loop works fine for me  while read LINE do       let count         echo   count  LINE   done  lt   FILENAME  echo -e   nTotal  count Lines read

User · Answer

read reads until it finds a newline character or the end of file  and returns a non-zero exit code if it encounters an end-of-file  So it s quite possible for it to both read a line and return a non-zero exit code   Consequently  the following code is not safe if the input might not be terminated by a newline   while read LINE  do     do something with LINE done   because the body of the while won t be executed on the last line   Technically speaking  a file not terminated with a newline is not a text file  and text tools may fail in odd ways on such a file  However  I m always reluctant to fall back on that explanation   One way to solve the problem is to test if what was read is non-empty  -n    while read -r LINE       -n  LINE     do     do something with LINE done   Other solutions include using mapfile to read the file into an array  piping the file through some utility which is guaranteed to terminate the last line properly  grep    for example  if you don t want to deal with blank lines   or doing the iterative processing with a tool like awk  which is usually my preference    Note that -r is almost certainly needed in the read builtin  it causes read to not reinterpret  -sequences in the input

[bash] Reading input files by line using read command in shell scripting skips last line

Examples related to bash

Examples related to parsing

Examples related to shell

Examples related to file-io