Cribl LogStream – Docs

Cribl LogStream Documentation

Questions? We'd love to help you! Meet us in #Cribl Community Slack (sign up)
Download entire manual as PDF - v2.3.3

Regex Extract

Description

The Regex Extract Function extracts fields using regex named groups. (In Splunk, these will be index-time fields). Fields that start with __ (double underscore) are special fields in Cribl LogStream. They are ephemeral: they can be used by any Function downstream, but will not be added to events, and will not exit the Pipeline.

Usage

Filter: Filter expression (JS) that selects data to be fed through the Function. Defaults to true, meaning that all events will be evaluated.

Description: Simple description about this Function. Defaults to empty.

Final: If true, stops data from being fed to downstream Functions. Defaults to No.

Regex: Regex literal. Must contain named capturing groups, e.g.: (?<foo>bar). Can contain special _NAME_N and _VALUE_N capturing groups, which extract both the name and value of a field, e.g.: (?<_NAME_0>[^\s=]+)=(?<_VALUE_0>[^\s]+). Defaults to empty. See Examples below.

Additional regex: Click + Add Regex to chain extra regex conditions.

Source field: Field on which to perform regex field extraction. Nested addressing is supported. Defaults to _raw.

Advanced Settings

Max exec: The maximum number of times to apply the Regex to the source field when the global flag is set, or when using _NAME_N and _VALUE_N capturing groups. Named capturing groups will always use a value of 1. Defaults to 100.

Field name format expression: Expression to format field names when _NAME_ capturing groups are used. The original field name is in the global name. E.g., to append XX to all field names: `${name}_XX`. If not specified, names will be sanitized using regex: /^[_0-9]+|[^a-zA-Z0-9_]+/g.

Overwrite existing fields: Whether to overwrite existing event fields with extracted values. If set to No (the default), existing fields will be converted to an array. If toggled to Yes, Regex Extract will create array fields if applied multiple times, or if fields exist. (E.g., if src_ip is extracted in an input Pipeline where it is assigned a value of 10.1.2.2, and is also in a processing Pipeline with a value of 10.2.3.3, then the resulting field is ["10.1.2.2", "10.2.3.3"].)

Examples

Scenario A

Assume a simple event that looks like this: metric1=23 metric2=42 dc=23 abc=xyz

Extract only the metric1 field:

Regex: metric1=(?<metric1>\d+)
Result: metric1:"23"

Scenario B

Use the first line of the sample here:
https://github.com/weeb-cribl/cribl-samples/blob/master/parser/functions/parser/cisco_estreamer.log

Extract all k=v pairs, and add an _XX suffix to the end of the extracted fields:

Regex: (?<_NAME_0>[^\s]+)=(?<_VALUE_0>[^\s]+)
Field Name Format Expression: `${name}_XX`

Result:

📘

For further examples, see Using Cribl to Analyze DNS Logs in Real Time – Part 2.

Updated about a month ago

Regex Extract


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.