How do I combine a stream of partial JSON objects into one JSON object?

← JSON and Data Processing · Ref: Q1412

Pour them into a JSON object with '>'. Pouring into JSON follows the target's shape: an unset or array target collects, an object target merges each streamed object in; anything else - a non-object into an object, or anything into a single value - is invalid, leaves the target unset, and later items do not revive it.

AN OBJECT TARGET MERGES

Each streamed object is merged in by the merge rule (the same rule as ':~:'):
- a key the target lacks, or holds as null, is filled;
- a key the target already holds is kept - the first recorded value wins;
- a nested object under a shared key is merged by this same rule, and a nested array appends.
What the target takes is a copy, so changing a source afterwards does not reach it.

START FROM AN OBJECT

The target must already be an object - an unset JSON collects the items into an array instead (Q1411). JSON().object() is an empty object:

  combined <- JSON().object()
  cat [JSON("name", JSON("Ann")), JSON("name", JSON("Bob"))] > combined
  stdout.println($combined)

See Q1411 for collecting into an array. See Q1413 for a stream that leaves the object unset. See Q192 for ':~:' on JSON.

Example

defines module qa.jsonanddata.jsonpourmergesobjects

  defines program

    JsonPourMergesObjectsDemo()
      stdout <- Stdout()

      //The first recorded value wins on a clashing key
      firstWins <- JSON().object()
      cat [JSON("name", JSON("Ann")), JSON("name", JSON("Bob"))] > firstWins
      stdout.println(`first wins: ${firstWins}`)

      //Nested objects merge by the same rule, nested arrays append, missing keys are filled
      person <- JSON("name", JSON("Ann")) + JSON("address", JSON("city", JSON("Leeds"))) + JSON("tags", $$ ["a"])
      fromTaskOne <- JSON("name", JSON("Bob")) + JSON("address", JSON("city", JSON("York")) + JSON("zip", JSON("LS1")))
      fromTaskTwo <- JSON("tags", $$ ["b"]) + JSON("age", JSON(30))
      cat [fromTaskOne, fromTaskTwo] > person
      stdout.println(`combined: ${person}`)

Coming from another language?

Java: Jackson ObjectNode.setAll() overwrites (last wins); a first-wins deep merge is hand-written. Python: {**a, **b} and dict.update() are shallow and last-wins. JavaScript: Object.assign is shallow and last-wins; lodash.merge is deep but last-wins. EK9: '>' into a JSON object is a deep merge where the first recorded value wins, so partial results gathered from several sources never overwrite each other.

Keywords: pour, partial, nested, merge, combine, pipe, json, first wins, stream, object