How do I use shell variables in an awk script

Question

I found some ways to pass external shell variables to an awk script  but I m confused about   and     First  I tried with a shell script     v 123test   echo  v 123test   echo   v  123test   Then tried awk     awk  BEGIN print    v       123test   awk  BEGIN print    v       123   Why is the difference   Lastly I tried this     awk  BEGIN print     v         123test   awk  BEGIN print     v      awk  cmd  line 1  BEGIN print awk  cmd  line 1                unexpected newline or end of string    I m confused about this

User · Answer

It seems that the good-old ENVIRON awk built-in hash is not mentioned at all  An example of its usage     X Solaris awk  BEGIN print ENVIRON  X    ENVIRON  TERM     Solaris rxvt

User · Answer

You could pass in the command-line option -v with a variable name  v  and a value     of the environment variable     v        awk -vv    v    BEGIN   print v    123test   Or to make it clearer  with far fewer vs      environment variable 123test   awk -vawk variable    environment variable    BEGIN   print awk variable    123test

User · Answer

Getting shell variables into awk      may be done in several ways  Some are better than others  This should cover most of them   If you have a comment  please leave below                                                                                                                                                                         v1 5     Using -v   The best way  most portable   Use the -v option   P S  use a space after -v or it will be less portable  E g   awk -v var  not awk -vvar    variable  line one nline two  awk -v var   variable   BEGIN  print var   line one line two   This should be compatible with most awk  and the variable is available in the BEGIN block as well   If you have multiple variables   awk -v a   var1  -v b   var2   BEGIN  print a b     Warning   As Ed Morton writes  escape sequences will be interpreted so  t becomes a real tab and not  t if that is what you search for  Can be solved by using ENVIRON   or access it via ARGV    PS If you like three vertical bar as separator      it can t be escaped  so use -F                Example on getting data from a program function inn to awk  here date is used    awk -v time    date    F  H  M  -d  -1 minute     BEGIN  print time       Variable after code block  Here we get the variable after the awk code  This will work fine as long as you do not need the variable in the BEGIN block   variable  line one nline two  echo  input data    awk   print var   var    variable   or awk   print var   var    variable   file    Adding multiple variables    awk   print a b  0   a   var1  b   var2  file   In this way we can also set different Field Separator FS for each file    awk  some code  FS     file1 txt FS     file2 ext   Variable after the code block will not work for the BEGIN block    echo  input data    awk  BEGIN  print var   var    variable      Here-string  Variable can also be added to awk using a here-string from shells that support them  including Bash    awk   print  0    lt  lt  lt    variable  test   This is the same as   printf   s    variable    awk   print  0     P S  this treats the variable as a file input     ENVIRON input  As TrueY writes  you can use the ENVIRON to print Environment Variables  Setting a variable before running AWK  you can print it out like this   X MyVar awk  BEGIN print ENVIRON  X   ENVIRON  SHELL     MyVar  bin bash     ARGV input  As Steven Penny writes  you can use ARGV to get the data into awk   v  my data  awk  BEGIN  print ARGV 1      v  my data   To get the data into the code itself  not just the BEGIN   v  my data  echo  test    awk  BEGIN var ARGV 1  ARGV 1       print var   0     v  my data test     Variable within the code  USE WITH CAUTION  You can use a variable within the awk code  but it s messy and hard to read  and as Charles Duffy points out  this version may also be a victim of code injection   If someone adds bad stuff to the variable  it will be executed as part of the awk code   This works by extracting the variable within the code  so it becomes a part of it   If you want to make an awk that changes dynamically with use of variables  you can do it this way  but DO NOT use it for normal variables   variable  line one nline two  awk  BEGIN  print     variable      line one line two   Here is an example of code injection   variable  line one nline two    for  i 1 i lt  1000   i  print i   awk  BEGIN  print     variable      line one line two 1 2 3     1000   You can add lots of commands to awk this way   Even make it crash with non valid commands     Extra info   Use of double quote  It s always good to double quote variable   variable  If not  multiple lines will be added as a long single line   Example   var  Line one This is line two   echo  var Line one This is line two  echo   var  Line one This is line two   Other errors you can get without double quote   variable  line one nline two  awk -v var  variable  BEGIN  print var   awk  cmd  line 1  one nline awk  cmd  line 1       backslash not last character on line awk  cmd  line 1  one nline awk  cmd  line 1       syntax error   And with single quote  it does not expand the value of the variable   awk -v var   variable   BEGIN  print var    variable   More info about AWK and variables  Read this faq

User · Answer

Use either of these depending how you want backslashes in the shell variables handled  avar is an awk variable  svar is a shell variable    awk -v avar   svar       avar      file awk  BEGIN avar ARGV 1  ARGV 1         avar        svar  file   See http   cfajohnson com shell cus-faq-2 html Q24 for details and other options  The first method above is almost always your best option and has the most obvious semantics

User · Answer

I had to insert date at the beginning of the lines of a log file and it s done like below   DATE   date    Y- m- d   awk    print     DATE      0      path to log file log file log   It can be redirect to another file to save

User · Answer

You can utilize ARGV   v 123test awk  BEGIN  print ARGV 1      v    Note that if you are going to continue into the body  you will need to adjust ARGC   awk  BEGIN  ARGC--   print ARGV 2    0   file   v

User · Answer

I just changed  Jotne s answer for  for loop     for i in  seq 11 20   do host myserver- i   awk -v i   i    print  myserver- i     4    done

[bash] How do I use shell variables in an awk script?

Examples related to bash

Examples related to shell

Examples related to awk