Event breakers are regex patterns and timestamp definitions that assist in breaking incoming streams of data into events. The Event Breakers management interface can be found under Knowledge | Event Breakers . Event Breaker rules and ruleset can be edited, added, deleted, searched and tagged as necessary.
Rules define configurations needed to break down a stream of data into discrete events.
Filter Condition: As a stream of data moves into the engine, a rule's filter expression is applied. If it evaluates to
true, the rule configurations are engaged for the entire duration of that stream. Else, the next rule down the line is evaluated.
Event Breaker: After a breaker regex pattern has been selected it will apply on the stream continuously. Breaking will occur at the beginning of the match and the matched content will be consumed/thrown away. If necessary, a positive lookahead regex can be used e.g., (?=pattern) to keep the content. Capturing groups are not allowed to be used anywhere in the event breaker pattern as they will further break the stream. This is often undesirable.
Breaking will also occur if Max Event Bytes has been reached. (See below for default value).
Timestamping: After events are synthesized out streams, timestamping will be attempted. First, a timestamp anchor will be located inside the event. Next, starting there, the engine will: try to scan up to a configurable depth into the event and autotimestamp, OR, timestamp using a manually supplied
strptime format OR timestamp the event with current time.
The closer an anchor is to the timestamp pattern the better the performance and accuracy, especially if multiple timestamps exist within an event. For the manual option, the anchor needs to lead the engine right before the timestamp pattern begins.
Fields: After events have been timestamped one or more fields can be added. Their values can be fully evaluated using JS expressions.
Filter Condition defaults to
Event Breaker to
Timestamp anchor to
Timestamp format to
Auto and a scan depth of
Max Event Bytes to
Default Timezone to
Rule Example: Break on newlines and use Manual timestamping after the sixth comma, as indicated by this pattern:
Rulesets are collections of rules that are associated with Sources. Rules within a ruleset are ordered and evaluated top->down. One ore more rulesets can be associated with a source and they too, are evaluated top->down. First rule that matches goes into effect for a stream from a source.
Ruleset A Rule 1 Rule 2 ... Rule n ... Ruleset B Rule Foo Rule Bar ... Rule FooBarn
Here's an example of 5 rulesets associated with a Source:
Default Rule: there is a system default rule that sits at the bottom of the Ruleset/Rule hierarchy that goes into effect if there are no matching rules. See Defaults above.
Cribl vs. Custom
Event Breaker Ruleset shipped by Cribl will be listed under the Cribl tab while those built by users will be found under Custom. Over time Cribl LogStream will ship more patterns and this distinction allows for both sets to grow independently. In the case of an ID/Name conflict, the Custom pattern takes priority in listings and search.
Updated 2 months ago