How to merge 2 JSON objects from 2 files using jq

Question

I m using the jq tools  jq-json-processor  in shell script to parse json  I ve got 2 json files and want to merge them into one unique file Here the content of files  file1        quot value1 quot   200       quot timestamp quot   1382461861       quot value quot              quot aaa quot                  quot value1 quot    quot v1 quot                quot value2 quot    quot v2 quot                      quot bbb quot                  quot value1 quot    quot v1 quot                quot value2 quot    quot v2 quot                      quot ccc quot                  quot value1 quot    quot v1 quot                quot value2 quot    quot v2 quot                     file2        quot status quot   200       quot timestamp quot   1382461861       quot value quot              quot aaa quot                  quot value3 quot    quot v3 quot                quot value4 quot   4                     quot bbb quot                  quot value3 quot    quot v3 quot                            quot ddd quot                  quot value3 quot    quot v3 quot                quot value4 quot   4                    expected result        quot value quot              quot aaa quot                  quot value1 quot    quot v1 quot                quot value2 quot    quot v2 quot                quot value3 quot    quot v3 quot                quot value4 quot   4                     quot bbb quot                  quot value1 quot    quot v1 quot                quot value2 quot    quot v2 quot                quot value3 quot    quot v3 quot                      quot ccc quot                  quot value1 quot    quot v1 quot                quot value2 quot    quot v2 quot                      quot ddd quot                  quot value3 quot    quot v3 quot                quot value4 quot   4                    I try a lot of combinations but the only result i get is the following  which is not the expected result       quot ccc quot          quot value2 quot    quot v2 quot        quot value1 quot    quot v1 quot          quot bbb quot          quot value2 quot    quot v2 quot        quot value1 quot    quot v1 quot          quot aaa quot          quot value2 quot    quot v2 quot        quot value1 quot    quot v1 quot             quot ddd quot          quot value4 quot   4       quot value3 quot    quot v3 quot          quot bbb quot          quot value3 quot    quot v3 quot          quot aaa quot          quot value4 quot   4       quot value3 quot    quot v3 quot         Using this command  jq -s      value  file1 file2

User · Answer

First    quot value quot    value  can be abbreviated to just  value   Second  the --argfile option  available in jq 1 4 and jq 1 5  may be of interest as it avoids having to use the --slurp option  Putting these together  the two objects in the two files can be combined in the specified way as follows    jq -n --argfile o1 file1 --argfile o2 file2   o1    o2    value    The  -n  flag tells jq not to read from stdin  since inputs are coming from the --argfile options here  Note on --argfile The jq manual deprecates --argfile because its semantics are non-trivial  if the specified input file contains exactly one JSON entity  then that entity is read as is  otherwise  the items in the stream are wrapped in an array  If you are uncomfortable using --argfile  there are several alternatives you may wish to consider   In doing so  be assured that using --slurpfile does not incur the inefficiencies of the -s command-line option when the latter is used with multiple files

User · Answer

This can be used to merge any number of files specified on the command   jq -rs  reduce     as  item           item   file1 json file2 json file3 json     file10 json  or this for any number of files  jq -rs  reduce     as  item           item       json

User · Answer

Use jq -s add     echo    a   foo   b   bar     c   baz   a  0     jq -s add      a   0     b    bar      c    baz      This reads all JSON texts from stdin into an array  jq -s does that  then it  reduces  them    add is defined as def add  reduce     as  x  null       x    which iterates over the input array s object s values and adds them   Object addition    merge

User · Answer

Since 1 4 this is now possible with the   operator  When given two objects  it will merge them recursively  For example   jq -s    0      1   file1 file2   Important  Note the -s  --slurp  flag  which puts files in the same array   Would get you        value1   200     timestamp   1382461861     value          aaa            value1    v1          value2    v2          value3    v3          value4   4             bbb            value1    v1          value2    v2          value3    v3              ccc            value1    v1          value2    v2              ddd            value3    v3          value4   4               status   200     If you also want to get rid of the other keys  like your expected result   one way to do it is this   jq -s    0      1     value   value   file1 file2   Or the presumably somewhat more efficient  because it doesn t merge any other values    jq -s    0  value     1  value    value      file1 file2

User · Answer

Who knows if you still need it  but here is the solution   Once you get to the --slurp option  it s easy   --slurp -s      Instead of running the filter for each JSON object in the input      read the entire input stream into a large array and run the filter just once    Then the   operator will do what you want   jq -s    0      1   config json config-user json    Note  if you want to merge inner objects instead of just overwriting the left file ones with the right file ones  you will need to do it manually

User · Answer

Here s  a version that works recursively  using    on an arbitrary number of objects  echo    quot A quot     quot a quot   1       quot A quot     quot b quot   2       quot B quot   3        jq --slurp  reduce     as  item           item         quot A quot          quot a quot   1       quot b quot   2         quot B quot   3

[json] How to merge 2 JSON objects from 2 files using jq?

Note on --argfile

Examples related to json

Examples related to shell

Examples related to command-line

Examples related to jq