Cribl - Docs

Getting started with Cribl LogStream

Questions? We'd love to help you! Meet us in #cribl (sign up)
Download manual as PDF

    Documentation

Introduction

As data travels a Cribl pipeline, it is operated on by a series of functions. Functions are fundamentally Javascript code.
Functions that ship with Cribl are configurable via a set of inputs. Some of these configuration options are literals, such as field names, and others can be Javascript expressions.
Expressions are valid units of code that resolve to a value. Every syntactically valid expression resolves to some value but conceptually, there are two types of expressions: those that assign value to a variable (a.k.a with side effects) and those that evaluate to a value.

Assigning a value
Evaluating to a value

x = 42
newFoo = foo.slice(30)

(Math.random() * 42)
3 + 4
'foobar'
'42'

Filters and Value Expressions


Filters

Filters are used in Routes to select a stream of the data flow, and in Functions to scope or narrow down the applicability of a function. They are expressions that must evaluate to either true (or truthy) or false (or falsy). Keep this in mind when creating routes or functions. For example:

  • sourcetype=='access_combined' && host.startsWith('web')
  • source.endsWith('.log') || sourcetype=='aws:cloudwatchlogs:vpcflow'
Truthy
Falsy

true
42
-42
3.14
"foo"
Infinity
-Infinity

false
null
undefined
0
NaN
''
""

Value Expressions

Values expressions are typically used in Functions to assign a value, for example, to a new field. For example:

  • Math.floor(_time/3600)
  • source.replace(/.{3}/, 'XXX')

Considerations and best practices for creating predictable expressions


  • In a value expression ensure that the source variable is not null, undefined or empty. For example, if you want to have a field called len to be assigned the length of a field called employeeID but you're not sure if employeeID exists, instead of employeeID.length you can use a safer shorthand as such: (employeeID || '').length.
  • If a field does not exist (undefined) and you're doing a comparison with its properties the boolean expression will always evaluate to false. For example, if employeeID is undefined, then both of these expressions employeeID.length > 10, and employeeID.length < 10 will evaluate to false.
  • == means equal to, while === means equal value and equal type.. For example, 5 == 5 evaluates to true, while 5 === "5" evaluates to false.
  • Ternary operator is a very powerful way to create conditional values. For example, if you wanted to assign either minor or adult to a field groupAge based on the value of age you can do: (age >= 18) ? 'adult' : 'minor'

Wildcard Lists


Wilcards Lists are used throughout the product especially in various Functions such as Eval, Mask, Publish Metrics, Parser etc.

Wilcard Lists, as their name implies, accept strings with asterisks (*) to represent one or more term. They also accept strings that start with exclamation mark (!) to negate one or more terms.

Wildcard Lists are order sensitive only when negated terms are used. This allows for implementing any combination of whitelists and blacklists.

For Example:

Wildcard List
Value
Meaning

List 1

!foobar, foo*

All terms that start with foo except foobar.

List 2

!foo*, *

All terms except for those that start with foo.

Introduction


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.