How does cat EOF work in bash

Question

I needed to write a script to enter multi-line input to a program  psql    After a bit of googling  I found the following syntax works   cat  lt  lt  EOF   psql ---params BEGIN    pg dump ----something   update table      statement       END  EOF   This correctly constructs the multi-line string  from BEGIN  to END   inclusive  and pipes it as an input to psql   But I have no idea how why it works  can some one please explain   I m referring mainly to cat  lt  lt  EOF  I know  gt  outputs to a file   gt  gt  appends to a file   lt  reads input from file    What does  lt  lt  exactly do   And is there a man page for it

User · Answer

In your case   EOF  is known as a  Here Tag   Basically  lt  lt Here tells the shell that you are going to enter a multiline string until the  tag  Here  You can name this tag as you want  it s often EOF or STOP   Some rules about the Here tags    The tag can be any string  uppercase or lowercase  though most people use uppercase by convention  The tag will not be considered as a Here tag if there are other words in that line  In this case  it will merely be considered part of the string  The tag should be by itself on a separate line  to be considered a tag  The tag should have no leading or trailing spaces in that line to be considered a tag  Otherwise it will be considered as part of the string    example      cat  gt  gt  test  lt  lt HERE  gt  Hello world HERE  lt -- Not by itself on a separate line - gt  not considered end of string  gt  This is a test  gt   HERE  lt -- Leading space  so not considered end of string  gt  and a new line  gt  HERE  lt -- Now we have the end of the string

User · Answer

Long story short  EOF marker but a different literal can be used as well  is a heredoc format that allows you to provide your input as multiline  A lot of confusion comes from how cat actually works it seems  You can use cat with  gt  gt  or  gt  as follows    cat  gt  gt  temp txt line 1 line 2  While cat can be used this way when writing manually into console  it s not convenient if I want to provide the input in a more declarative way so that it can be reused by tools and also to keep indentations  whitespaces  etc  Heredoc allows to define your entire input as if you are not working with stdin but typing in a separate text editor  This is what Wikipedia article means by   it is a section of a source code file that is treated as if it were a separate file

User · Answer

This is called heredoc format to provide a string into stdin  See https   en wikipedia org wiki Here document Unix shells for more details     From man bash      Here Documents      This type of redirection instructs the shell to read input from   the current source until a line   containing only word  with no trailing   blanks   is  seen       All of the lines read up to that point are then used as the   standard input for a command       The format of here-documents is              lt  lt  - word                   here-document           delimiter       No parameter expansion  command substitution  arithmetic expansion  or   pathname expansion is performed on   word   If any characters in word are   quoted  the   delimiter is the result of quote removal on word  and the lines   in the here-document are not expanded    If word is unquoted  all lines of  the   here-document  are  subjected  to  parameter  expansion   command   substitution  and arithmetic   expansion   In the latter case  the   character sequence   lt newline gt  is   ignored  and   must be used to quote the characters       and         If the redirection operator is  lt  lt -  then all leading tab characters   are stripped from input lines and the   line containing delimiter    This    allows  here-documents within shell scripts to be indented in a natural fashion

User · Answer

Worth noting that here docs work in bash loops too   This example shows how-to get the column list of table    export postgres db name  my db  export table name  my table name     start copy  while read -r c  do test -z   c     echo  table name  c     done  lt   lt  cat  lt  lt  EOF   psql -t -q -d  postgres db name -v table name    table name -   SELECT column name FROM information schema columns WHERE 1 1 AND table schema    public  AND table name      table name     EOF     stop copy   now paste straight into the bash shell      output   my table name guid   my table name id   my table name level   my table name seq     or even without the new line   while read -r c  do test -z   c     echo  table name  c     perl -ne   s  n  gm print    done  lt   lt  cat  lt  lt  EOF   psql -t -q -d  postgres db name -v table name    table name -    SELECT column name  FROM information schema columns  WHERE 1 1  AND table schema    public   AND table name      table name      EOF        output  daily issues guid  daily issues id  daily issues level  daily issues seq  daily issues prio  daily issues weight  daily issues status  daily issues category  daily issues name  daily issues description  daily issues type  daily issues owner

User · Answer

POSIX 7  kennytm quoted man bash  but most of that is also POSIX 7  http   pubs opengroup org onlinepubs 9699919799 utilities V3 chap02 html tag 18 07 04       The redirection operators   lt  lt   and   lt  lt -  both allow redirection of lines contained in a shell input file  known as a  here-document   to the input of a command       The here-document shall be treated as a single word that begins after the next  and continues until there is a line containing only the delimiter and a   with no  characters in between  Then the next here-document starts  if there is one  The format is as follows    n  lt  lt word     here-document delimiter       where the optional n represents the file descriptor number  If the number is omitted  the here-document refers to standard input  file descriptor 0        If any character in word is quoted  the delimiter shall be formed by performing quote removal on word  and the here-document lines shall not be expanded  Otherwise  the delimiter shall be the word itself       If no characters in word are quoted  all lines of the here-document shall be expanded for parameter expansion  command substitution  and arithmetic expansion  In this case  the  in the input behaves as the  inside double-quotes  see Double-Quotes   However  the double-quote character         shall not be treated specially within a here-document  except when the double-quote appears within              or             If the redirection symbol is   lt  lt -   all leading  lt tab gt  characters shall be stripped from input lines and the line containing the trailing delimiter  If more than one   lt  lt   or   lt  lt -  operator is specified on a line  the here-document associated with the first operator shall be supplied first by the application and shall be read first by the shell       When a here-document is read from a terminal device and the shell is interactive  it shall write the contents of the variable PS2  processed as described in Shell Variables  to standard error before reading each line of input until the delimiter has been recognized    Examples  Some examples not yet given   Quotes prevent parameter expansion  Without quotes   a 0 cat  lt  lt EOF  a EOF   Output   0   With quotes   a 0 cat  lt  lt  EOF   a EOF   or  ugly but valid    a 0 cat  lt  lt E O F  a EOF   Outputs    a   Hyphen removes leading tabs  Without hyphen   cat  lt  lt EOF  lt tab gt a EOF   where  lt tab gt  is a literal tab  and can be inserted with Ctrl   V  lt tab gt   Output    lt tab gt a   With hyphen   cat  lt  lt -EOF  lt tab gt a  lt tab gt EOF   Output   a   This exists of course so that you can indent your cat like the surrounding code  which is easier to read and maintain  E g     if true  then     cat  lt  lt -EOF     a     EOF fi   Unfortunately  this does not work for space characters  POSIX favored tab indentation here  Yikes

User · Answer

A little extension to the above answers  The trailing  gt  directs the input into the file  overwriting existing content  However  one particularly convenient use is the double arrow  gt  gt  that appends  adding your new content to the end of the file  as in   cat  lt  lt EOF  gt  gt   etc fstab data server  var sharedServer authority cert  var sharedFolder sometin authority cert nfs data server  var sharedServer cert    var sharedFolder sometin vsdc cert nfs EOF   This extends your fstab without you having to worry about accidentally modifying any of its contents

User · Answer

The cat  lt  lt EOF syntax is very useful when working with multi-line text in Bash  eg  when assigning multi-line string to a shell variable  file or a pipe   Examples of cat  lt  lt EOF syntax usage in Bash   1  Assign multi-line string to a shell variable      sql   cat  lt  lt EOF SELECT foo  bar FROM db WHERE foo  baz  EOF     The  sql variable now holds the new-line characters too  You can verify with echo -e   sql    2  Pass multi-line string to a file in Bash    cat  lt  lt EOF  gt  print sh    bin bash echo   PWD echo  PWD EOF   The print sh file now contains      bin bash echo  PWD echo  home user   3  Pass multi-line string to a pipe in Bash    cat  lt  lt EOF   grep  b    tee b txt foo bar baz EOF   The b txt file contains bar and baz lines  The same output is printed to stdout

User · Answer

Using tee instead of cat  Not exactly as an answer to the original question  but I wanted to share this anyway  I had the need to create a config file in a directory that required root rights   The following does not work for that case     sudo cat  lt  lt EOF  gt  etc somedir foo conf   my config file foo bar EOF   because the redirection is handled outside of the sudo context   I ended up using this instead     sudo tee  lt  lt EOF  etc somedir foo conf  gt  dev null   my config file foo bar EOF

User · Answer

This isn t necessarily an answer to the original question  but a sharing of some results from my own testing  This    lt  lt test  gt  print sh    bin bash echo   PWD echo  PWD test   will produce the same file as   cat  lt  lt test  gt  print sh    bin bash echo   PWD echo  PWD test   So  I don t see the point of using the cat command

[linux] How does "cat << EOF" work in bash?

Examples related to linux

Examples related to bash

Examples related to scripting

Examples related to heredoc