I'm using the jq tools (jq-json-processor) in shell script to parse json.
I've got 2 json files and want to merge them into one unique file
Here the content of files:
file1
{
"value1": 200,
"timestamp": 1382461861,
"value": {
"aaa": {
"value1": "v1",
"value2": "v2"
},
"bbb": {
"value1": "v1",
"value2": "v2"
},
"ccc": {
"value1": "v1",
"value2": "v2"
}
}
}
file2
{
"status": 200,
"timestamp": 1382461861,
"value": {
"aaa": {
"value3": "v3",
"value4": 4
},
"bbb": {
"value3": "v3"
},
"ddd": {
"value3": "v3",
"value4": 4
}
}
}
expected result
{
"value": {
"aaa": {
"value1": "v1",
"value2": "v2",
"value3": "v3",
"value4": 4
},
"bbb": {
"value1": "v1",
"value2": "v2",
"value3": "v3"
},
"ccc": {
"value1": "v1",
"value2": "v2"
},
"ddd": {
"value3": "v3",
"value4": 4
}
}
}
I try a lot of combinations but the only result i get is the following, which is not the expected result:
{
"ccc": {
"value2": "v2",
"value1": "v1"
},
"bbb": {
"value2": "v2",
"value1": "v1"
},
"aaa": {
"value2": "v2",
"value1": "v1"
}
}
{
"ddd": {
"value4": 4,
"value3": "v3"
},
"bbb": {
"value3": "v3"
},
"aaa": {
"value4": 4,
"value3": "v3"
}
}
Using this command:
jq -s '.[].value' file1 file2
This question is related to
json
shell
command-line
jq
This can be used to merge any number of files specified on the command:
jq -rs 'reduce .[] as $item ({}; . * $item)' file1.json file2.json file3.json ... file10.json
or this for any number of files
jq -rs 'reduce .[] as $item ({}; . * $item)' ./*.json
Use jq -s add
:
$ echo '{"a":"foo","b":"bar"} {"c":"baz","a":0}' | jq -s add
{
"a": 0,
"b": "bar",
"c": "baz"
}
This reads all JSON texts from stdin into an array (jq -s
does that) then it "reduces" them.
(add
is defined as def add: reduce .[] as $x (null; . + $x);
, which iterates over the input array's/object's values and adds them. Object addition == merge.)
Who knows if you still need it, but here is the solution.
Once you get to the --slurp
option, it's easy!
--slurp/-s:
Instead of running the filter for each JSON object in the input,
read the entire input stream into a large array and run the filter just once.
Then the +
operator will do what you want:
jq -s '.[0] + .[1]' config.json config-user.json
(Note: if you want to merge inner objects instead of just overwriting the left file ones with the right file ones, you will need to do it manually)
First, {"value": .value} can be abbreviated to just {value}.
Second, the --argfile option (available in jq 1.4 and jq 1.5) may be of interest as it avoids having to use the --slurp option.
Putting these together, the two objects in the two files can be combined in the specified way as follows:
$ jq -n --argfile o1 file1 --argfile o2 file2 '$o1 * $o2 | {value}'
The '-n' flag tells jq not to read from stdin, since inputs are coming from the --argfile options here.
The jq manual deprecates --argfile
because its semantics are non-trivial: if the specified input file contains exactly one JSON entity, then that entity is read as is; otherwise, the items in the stream are wrapped in an array.
If you are uncomfortable using --argfile, there are several alternatives you may wish to consider. In doing so, be assured that using --slurpfile
does not incur the inefficiencies of the -s
command-line option when the latter is used with multiple files.
Here's a version that works recursively (using *
) on an arbitrary number of objects:
echo '{"A": {"a": 1}}' '{"A": {"b": 2}}' '{"B": 3}' |\
jq --slurp 'reduce .[] as $item ({}; . * $item)'
{
"A": {
"a": 1,
"b": 2
},
"B": 3
}
Source: Stackoverflow.com