As data travels through a Cribl LogStream Pipeline, it is operated on by a series of Functions. Functions are fundamentally JavaScript code.
Functions that ship with Cribl LogStream are configurable via a set of inputs. Some of these configuration options are literals, such as field names, and others can be JavaScript expressions.
Expressions are valid units of code that resolve to a value. Every syntactically valid expression resolves to some value, but conceptually, there are two types of expressions: those that assign value to a variable (a.k.a., with side effects), and those that evaluate to a value.
Assigning a value | Evaluating to a value |
---|---|
|
|
Filters and Value Expressions
Filters
Filters are used in Routes to select a stream of the data flow, and in Functions to scope or narrow down the applicability of a Function. Filters are expressions that must evaluate to either true
(or truthy) or false
(or falsy). Keep this in mind when creating Routes or Functions. For example:
sourcetype=='access_combined' && host.startsWith('web')
source.endsWith('.log') || sourcetype=='aws:cloudwatchlogs:vpcflow'
Truthy | Falsy |
---|---|
|
|
Value Expressions
Value expressions are typically used in Functions to assign a value – for example, to a new field. For example:
Math.floor(_time/3600)
source.replace(/.{3}/, 'XXX')
Considerations and Best Practices for Creating Predictable Expressions
-
In a value expression, ensure that the source variable is not null, undefined, or empty. For example, assume you want to have a field called
len
, to be assigned the length of a second field calledemployeeID
. But you're not sure ifemployeeID
exists. Instead ofemployeeID.length
, you can use a safer shorthand, such as:(employeeID || '').length
. -
If a field does not exist (undefined), and you're doing a comparison with its properties, then the boolean expression will always evaluate to false. For example, if
employeeID
is undefined, then both of these expressions will evaluate to false:employeeID.length > 10
, andemployeeID.length < 10
. -
==
means "equal to," while===
means "equal value and equal type." For example,5 == 5
evaluates to true, while5 === "5"
evaluates to false. -
A ternary operator is a very powerful way to create conditional values. For example, if you wanted to assign either
minor
oradult
to a fieldgroupAge
, based on the value ofage
, you could do:(age >= 18) ? 'adult' : 'minor'
.
Expressions Using Fields with Non-Alphanumeric Characters
If there are fields with non-alphanumeric characters – e.g., @timestamp
or user-agent
or kubernetes.namespace_name
– you can access them using __e['<field-name-here>']
. (Note the single quotes.) More details here. In any other place where the field is referenced – e.g., in the Eval function's field names – you should use a single-quoted literal, of the form: '<field-name-here>'
.
Wildcard Lists
Wildcard Lists are used throughout the product, especially in various Functions, such as Eval, Mask, Publish Metrics, Parser, etc.
Wildcard Lists, as their name implies, accept strings with asterisks (*
) to represent one or more terms. They also accept strings that start with an exclamation mark (!
) to negate one or more terms.
Wildcard Lists are order-sensitive only when negated terms are used. This allows for implementing any combination of whitelists and blacklists.
For example:
Wildcard List | Value | Meaning |
---|---|---|
List 1 |
| All terms that start with foo, except foobar. |
List 2 |
| All terms, except for those that start with foo. |
Updated 2 months ago