Home / Stream/ Functions/JSON Unroll

JSON Unroll

The JSON Unroll Function accepts a JSON object string _raw field and unrolls (explodes) an array of objects therein into individual events, while also inheriting top-level fields. See example(s). Cribl highly recommends not using the JSON Unroll Function for certain types of data. Perform the unrolling using an Event Breaker for inputs that support configuring an Event Breaker. Specifying the Event Breaker type JSON Array and toggling the JSON Extract Fields option to Yes will accomplish the same unrolling but more efficiently. We recommend this, for example, for CloudTrail and Office 365 events, which are collected as JSON arrays.

Usage

Filter: Filter expression (JavaScript) that selects data to feed through the Function. Defaults to true, meaning it evaluates all events.

Description: Simple description about this Function. Defaults to empty.

Final: Toggle on to stop feeding data to the downstream Functions. Default is toggled off.

Path: Path to array to unroll, such as foo.0.bar.

New name: The name that the exploded array element will receive in each new event. Leave empty to expand the array element with its original name.

Example(s)

Assume you have an incoming event that has a _raw field as a JSON object string like this:

Sample _raw field
 {"date":"9/25/18 9:10:13.000 PM",
    "name":"Amrit",
    "age":42,
    "allCars": [
        { "name":"Ford", "models":[ "Fiesta", "Focus", "Mustang" ] },
        { "name":"GM", "models":[ "Trans AM", "Oldsmobile", "Cadillac" ] },
        { "name":"Fiat", "models":[ "500", "Panda" ] },
        { "name":"Blackberry", "models":[ "KEY2", "Bold Touch 9900" ] }
    ]
 }

Settings:

Path: allCars New Name: cars

Output Events:

Resulting Events
Event 1:
{"_raw":"{"date":"9/25/18 9:10:13.000 PM","name":"Amrit","age":42,"cars":{"name":"Ford","models":["Fiesta","Focus","Mustang"]}}"}

Event 2:
{"_raw":"{"date":"9/25/18 9:10:13.000 PM","name":"Amrit","age":42,"cars":{"name":"GM","models":["Trans AM","Oldsmobile","Cadillac"]}}"}

Event 3:
{"_raw":"{"date":"9/25/18 9:10:13.000 PM","name":"Amrit","age":42,"cars":{"name":"Fiat","models":["500","Panda"]}}"}

Event 4:
{"_raw":"{"date":"9/25/18 9:10:13.000 PM","name":"Amrit","age":42,"cars":{"name":"Blackberry","models":["KEY2","Bold Touch 9900"]}}"}

Each element under the original allCars array is now placed in a cars field in its own event, inheriting original top level fields; date, name, and age

See Also

  • The Cribl Knowledge Pack provides sample Pipelines that demonstrate converting a JSON string into an object literal, and validating JSON data against a schema.