Multi-dimensional arrays in Bash

Question

I am planning a script to manage some pieces of my Linux systems and am at the point of deciding if I want to use bash or python   I would prefer to do this as a Bash script simply because the commands are easier  but the real deciding factor is configuration   I need to be able to store a multi-dimensional array in the configuration file to tell the script what to do with itself   Storing simple key value pairs in config files is easy enough with bash  but the only way I can think of to do a multi-dimensional array is a two layer parsing engine  something like  array  amp d1 v1 v2 v3 amp d2 v1 v2 v3   but the marshall unmarshall code could get to be a bear and its far from user friendly for the next poor sap that has to administer this   If i can t do this easily in bash i will simply write the configs to an xml file and write the script in python     Is there an easy way to do this in bash   thanks everyone

User · Answer

After a lot of trial and error i actually find the best, clearest and easiest multidimensional array on bash is to use a regular var. Yep.

Advantages: You don't have to loop through a big array, you can just echo "$var" and use grep/awk/sed. It's easy and clear and you can have as many columns as you like.

Example:

$ var=$(echo -e 'kris hansen oslo\nthomas jonson peru\nbibi abu johnsonville\njohnny lipp peru')

$ echo "$var"
kris hansen oslo
thomas johnson peru
bibi abu johnsonville
johnny lipp peru

If you want to find everyone in peru

$ echo "$var" | grep peru
thomas johnson peru
johnny lipp peru

Only grep(sed) in the third field

$ echo "$var" | sed -n -E '/(.+) (.+) peru/p'
thomas johnson peru
johnny lipp peru

If you only want x field

$ echo "$var" | awk '{print $2}'
hansen
johnson
abu
johnny

Everyone in peru that's called thomas and just return his lastname

$ echo "$var" |grep peru|grep thomas|awk '{print $2}'
johnson

Any query you can think of... supereasy.

To change an item:

$ var=$(echo "$var"|sed "s/thomas/pete/")

To delete a row that contains "x"

$ var=$(echo "$var"|sed "/thomas/d")

To change another field in the same row based on a value from another item

$ var=$(echo "$var"|sed -E "s/(thomas) (.+) (.+)/\1 test \3/")
$ echo "$var"
kris hansen oslo                                                                                                                                             
thomas test peru                                                                                                                                          
bibi abu johnsonville
johnny lipp peru

Of course looping works too if you want to do that

$ for i in "$var"; do echo "$i"; done
kris hansen oslo
thomas jonson peru
bibi abu johnsonville
johnny lipp peru

The only gotcha iv'e found with this is that you must always quote the var(in the example; both var and i) or things will look like this

$ for i in "$var"; do echo $i; done
kris hansen oslo thomas jonson peru bibi abu johnsonville johnny lipp peru

and someone will undoubtedly say it won't work if you have spaces in your input, however that can be fixed by using another delimeter in your input, eg(using an utf8 char now to emphasize that you can choose something your input won't contain, but you can choose whatever ofc):

$ var=$(echo -e 'field one?field two hello?field three yes moin\nfield 1?field 2?field 3 dsdds aq')

$ for i in "$var"; do echo "$i"; done
field one?field two hello?field three yes moin
field 1?field 2?field 3 dsdds aq

$ echo "$var" | awk -F '?' '{print $3}'
field three yes moin
field 3 dsdds aq

$ var=$(echo "$var"|sed -E "s/(field one)?(.+)?(.+)/\1?test?\3/")
$ echo "$var"
field one?test?field three yes moin
field 1?field 2?field 3 dsdds aq

If you want to store newlines in your input, you could convert the newline to something else before input and convert it back again on output(or don't use bash...). Enjoy!

User · Answer

I ve got a pretty simple yet smart workaround  Just define the array with variables in its name  For example   for    i 0   i lt     maxvalue   1     i        do   for    j 0   j lt     maxargument   1     j          do     declare -a array i  j    Your rule     done done   Don t know whether this helps since it s not exactly what you asked for  but it works for me   The same could be achieved just with variables without the array

User · Answer

I do this using associative arrays since bash 4 and setting IFS to a value that can be defined manually   The purpose of this approach is to have arrays as values of associative array keys   In order to set IFS back to default just unset it    unset IFS     This is an example      bin bash  set -euo pipefail    used as value in asscciative array test      x3 x4 x5      associative array declare -A wow       1    test     2    test   echo  default IFS  for w in   wow      do   echo     w  done  IFS   echo  IFS    for w in   wow      do   for t in  w  do     echo     t    done done echo -e   n or n  for w in    wow     do   echo     w    for t in   wow  w     do     echo       t    done done  unset IFS unset w unset t unset wow unset test   The output of the script below is   default IFS   x3 x4 x5   x3 x4 x5 IFS     x3   x4   x5   x3   x4   x5   or    1     x3     x4     x5   2     x3     x4     x5

User · Answer

Independent of the shell being used  sh  ksh  bash       the following approach works pretty well for n-dimensional arrays  the sample covers a 2-dimensional array     In the sample the line-separator  1st dimension  is the space character  For introducing a field separator  2nd dimension  the standard unix tool tr is used  Additional separators for additional dimensions can be used in the same way   Of course the performance of this approach is not very well  but if performance is not a criteria this approach is quite generic and can solve many problems   array2d  1 1 1 2 1 3 2 1 2 2 3 1 3 2 3 3 3 4   function process2ndDimension       for dimension2 in        do         echo -n  dimension2           done     echo    function process1stDimension       for dimension1 in  array2d     do         process2ndDimension  echo  dimension1   tr            done    process1stDimension   The output of that sample looks like this   1 1     1 2     1 3      2 1     2 2      3 1     3 2     3 3     3 4

User · Answer

I am posting the following because it is a very simple and clear way to mimic  at least to some extent  the behavior of a two-dimensional array in Bash  It uses a here-file  see the Bash manual  and read  a Bash builtin command      Store the  quot two-dimensional data quot  in a file     is just the process ID of the shell  to make sure the filename is unique  cat  gt  physicists     lt  lt EOF Wolfgang Pauli 1900 Werner Heisenberg 1901 Albert Einstein 1879 Niels Bohr 1885 EOF nbPhysicists   wc -l physicists      cut -sf 1 -d            Number of lines of the here-file specifying the physicists      Extract the needed data declare -a person       Create an indexed array  necessary for the read command                                                                                    while read -ra person  do     firstName   person 0       familyName   person 1       birthYear   person 2       echo  quot Physicist   firstName    familyName  was born in   birthYear  quot        Do whatever you need with data done  lt  physicists        Remove the temporary file rm physicists     Output   Physicist Wolfgang Pauli was born in 1900 Physicist Werner Heisenberg was born in 1901 Physicist Albert Einstein was born in 1879 Physicist Niels Bohr was born in 1885 The way it works   The lines in the temporary file created play the role of one-dimensional vectors  where the blank spaces  or whatever separation character you choose  see the description of the read command in the Bash manual  separate the elements of these vectors  Then  using the read command with its -a option  we loop over each line of the file  until we reach end of file   For each line  we can assign the desired fields    words  to an array  which we declared just before the loop  The -r option to the read command prevents backslashes from acting as escape characters  in case we typed backslashes in the here-document physicists      In conclusion a file is created as a 2D-array  and its elements are extracted using a loop over each line  and using the ability of the read command to assign words to the elements of an  indexed  array  Slight improvement  In the above code  the file physicists    is given as input to the while loop  so that it is in fact passed to the read command  However  I found that this causes problems when I have another command asking for input inside the while loop  For example  the select command waits for standard input  and if placed inside the while loop  it will take input from physicists     instead of prompting in the command-line for user input  To correct this  I use the -u option of read  which allows to read from a file descriptor  We only have to create a file descriptor  with the exec command  corresponding to physicists    and to give it to the -u option of read  as in the following code     Store the  quot two-dimensional data quot  in a file     is just the process ID of the shell  to make sure the filename is unique  cat  gt  physicists     lt  lt EOF Wolfgang Pauli 1900 Werner Heisenberg 1901 Albert Einstein 1879 Niels Bohr 1885 EOF nbPhysicists   wc -l physicists      cut -sf 1 -d            Number of lines of the here-file specifying the physicists  exec  id file  lt   physicists          Create a file descriptor stored in  id file       Extract the needed data declare -a person       Create an indexed array  necessary for the read command                                                                                    while read -ra person -u  quot   id file  quot   do firstName   person 0   familyName   person 1   birthYear   person 2   echo  quot Physicist   firstName    familyName  was born in   birthYear  quot    Do whatever you need with data done     Close the file descriptor exec  id file  lt  amp -    Remove the temporary file rm physicists     Notice that the file descriptor is closed at the end

User · Answer

This works thanks to 1   quot indirect expansion quot  with   which adds one layer of indirection  and 2   quot substring expansion quot  which behaves differently with arrays and can be used to  quot slice quot  them as described https   stackoverflow com a 1336245 317623   Define each array and then add it to the main one SUB 0   quot name0 quot   quot value 0 quot   SUB 1   quot name1 quot   quot value 1 quot   MAIN ARRAY     SUB 0      SUB 1         Loop and print it   Using offset and length to extract values COUNT    MAIN ARRAY     for   i 0  i lt  COUNT  i     do   NAME    MAIN ARRAY i  0 1    VALUE    MAIN ARRAY i  1 1    echo  quot NAME   NAME  quot    echo  quot VALUE   VALUE  quot  done  It s based off of this answer here

User · Answer

echo  Enter no of terms  read count for i in   seq 1  count  do   t   expr  i - 1     for j in   seq  t -1 0    do     echo -n       done   j   expr  count   1     x   expr  j -  i     for k in   seq 1  x    do     echo -n        done   echo    done

User · Answer

Lots of answers found here for creating multidimensional arrays in bash   And without exception  all are obtuse and difficult to use   If MD arrays are a required criteria  it is time to make a decision   Use a language that supports MD arrays  My preference is Perl   Most would probably choose Python  Either works   Store the data elsewhere  JSON and jq have already been suggested   XML has also been suggested  though for your use JSON and jq would likely be simpler   It would seem though that Bash may not be the best choice for what you need to do   Sometimes the correct question is not  How do I do X in tool Y    but rather  Which tool would be best to do X

User · Answer

Expanding on Paul s answer - here s my version of working with associative sub-arrays in bash   declare -A SUB 1    name1key    name1val    name2key    name2val   declare -A SUB 2    name3key    name3val    name4key    name4val   STRING 1  string1val  STRING 2  string2val  MAIN ARRAY        SUB 1           SUB 2           STRING 1        STRING 2     echo  COUNT       MAIN ARRAY     for key in    MAIN ARRAY      do     IFS     read -a val  lt  lt  lt    MAIN ARRAY  key       echo  VALUE      val         if       val     -gt 1     then         for subkey in    val      do             subval   val  subkey               echo  SUBVALUE      subval          done     fi done   It works with mixed values in the main array - strings arrays assoc  arrays  The key here is to wrap the subarrays in single quotes and use   instead of   when storing a subarray inside the main array so it would get stored as a single  space separated string     SUB 1       Then it makes it easy to parse an array out of that when looping through values with IFS     read -a val  lt  lt  lt    MAIN ARRAY  key    The code above outputs   COUNT   4 VALUE   name1val name2val SUBVALUE   name1val SUBVALUE   name2val VALUE   name4val name3val SUBVALUE   name4val SUBVALUE   name3val VALUE   string1val VALUE   string2val

User · Answer

Bash does not support multidimensional arrays  nor hashes  and it seems that you want a hash that values are arrays  This solution is not very beautiful  a solution with an xml file should be better   array   d1  v1 v2 v3    d2  v1 v2 v3    for elt in  quot   array     quot  do eval  elt done echo  quot d1    d1       d1     quot  echo  quot d2    d2       d2     quot   EDIT  this answer is quite old  since since bash 4 supports hash tables  see also this answer for a solution without eval

User · Answer

Bash does not supports multidimensional array  but we can implement using Associate array  Here the indexes are the key to retrieve the value  Associate array is available in bash version 4      bin bash  declare -A arr2d rows 3 columns 2  for   i 0 i lt rows i     do     for   j 0 j lt columns j     do         arr2d  i  j   i     done done   for   i 0 i lt rows i     do     for   j 0 j lt columns j     do         echo   arr2d  i  j       done done

User · Answer

If you want to use a bash script and keep it easy to read recommend putting the data in structured JSON  and then use lightweight tool jq in your bash command to iterate through the array   For example with the following dataset            specialId   123        specialName   First           specialId   456        specialName   Second           specialId   789        specialName   Third       You can iterate through this data with a bash script and jq like this   function loopOverArray         jq -c       testing json   while read i  do           Do stuff here         echo   i      done    loopOverArray   Outputs      specialId   123   specialName   First     specialId   456   specialName   Second     specialId   789   specialName   Third

User · Answer

Bash doesn t have multi-dimensional array  But you can simulate a somewhat similar effect with associative arrays  The following is an example of associative array pretending to be used as multi-dimensional array   declare -A arr arr 0 0  0 arr 0 1  1 arr 1 0  2 arr 1 1  3 echo    arr 0 0     arr 0 1      will print 0 1   If you don t declare the array as associative  with -A   the above won t work  For example  if you omit the declare -A arr line  the echo will print 2 3 instead of 0 1  because 0 0  1 0 and such will be taken as arithmetic expression and evaluated to 0  the value to the right of the comma operator

[arrays] Multi-dimensional arrays in Bash

If you want to find everyone in peru

Only grep(sed) in the third field

If you only want x field

Everyone in peru that's called thomas and just return his lastname

To change an item:

To delete a row that contains "x"

To change another field in the same row based on a value from another item

Of course looping works too if you want to do that

Examples related to arrays

Examples related to bash

Examples related to shell

Examples related to multidimensional-array