SplitJson

Modified on Fri, 30 Jun, 2023 at 8:47 AM

We commonly receive flowfiles with JSON content that is stored within an array. While there are a number of different ways to query the data within the array (QueryRecord, EvaluateJsonPath, etc.) sometimes we want to split the source JSON up into distinct flowfiles for routing of the data. The SplitJson record does just that.

For this example we are using a JSON file retreived from the Star Wars API at https://swapi.dev/api/people. See below for a truncated example of this data:

{
  "count" : 82,
  "next" : "https://swapi.dev/api/people/?page=2",
  "previous" : null,
  "results" : [ {
    "name" : "Luke Skywalker",
    "height" : "172",
    "mass" : "77",
    "hair_color" : "blond",
    "skin_color" : "fair",
    "eye_color" : "blue",
    "birth_year" : "19BBY",
    "gender" : "male",
    "homeworld" : "https://swapi.dev/api/planets/1/",
    "starships" : [ "https://swapi.dev/api/starships/12/", "https://swapi.dev/api/starships/22/" ],

    "url" : "https://swapi.dev/api/people/1/"
  }, {
    "name" : "C-3PO",
    "height" : "167",
    "mass" : "75",
    "hair_color" : "n/a",
    "skin_color" : "gold",
    "eye_color" : "yellow",
    "birth_year" : "112BBY",
    "gender" : "n/a",
    "homeworld" : "https://swapi.dev/api/planets/1/",
    "starships" : [ ],
    "url" : "https://swapi.dev/api/people/2/"
<REMAINING RESULTS TRUNCATED FOR BREVITY>
  } ]
}

As we can see, this JSON document has different characters listed in it within the results array that we'd like to split into separate flowfiles. The actual JSON document had 10 characters in it, but the above example was cut down to 2 to reduce screen real estate.

The SplitJson processor has only 2 properties for configuration:

JsonPath Expression

This property tells the processor which array you would like to split on. In our scenario above, the section we'd like to split on is labeled results. With that knowledge, we we simply put in the JsonPath expression of $.results

Null Value Representation

This allows you to determine how you would like the processor to write data if If the JSON document has values set to null. You can select to have the value set to an empty string "" or you can set the string to 'null'.

Passing the sample JSON document above into the SplitJson processor splits the single incoming flowfile into 2 separate flowfiles that can then be processed elsewhere.

Note: When splitting a JSON document, all data that is NOT within the array being split will be removed from the flowfile content. Using the above example JSON our output flowfiles would not include the count, next, and previous Key-Value pairs.