Cribl - Docs

Getting started with Cribl LogStream

Questions? We'd love to help you! Meet us in #cribl (sign up)

Changelog    Guides

Routes

What are Routes

Before incoming data is searched, processed, transformed etc. by a pipeline, Cribl gives users an opportunity to first select a subset and then deliver it to the correct pipeline. This process is done via routes.

How do Routes Work

Routes apply filter expressions on incoming events and then send matching results to the appropriate pipeline . Filters are JS-syntax compatible expressions, e.g., source=='foo.log' && index=='bar', true, etc. that are configured with each route. There can be multiple routes in the system but a route can only be associated with one pipeline.

Route evaluation order matters!

When multiple routes exist, they are evaluated in order, top down.

In this example, incoming events will be evaluated against the route named Route first, then Sensitive Data, then Logs to Metrics and so on. At the end, the Main route serves as a catch-all for any event that does not match any of the other routes. If a route needs to be applied before another, simply drag it on top of it. In addition, you can turn routes On/Off inline as necessary.

The Final Toggle

In most cases, an event that enters the system and matches a route-pipeline pair will either be dropped by a function in that pipeline or optionally transformed and exit the system. This is ensured by the final toggle in route settings. It defaults to Yes and means that matched events will be consumed by that route and not evaluated against any other routes that sit below it.

If the toggle is set to No, clone(s) of the matching events are processed by the configured pipeline and the original events are allowed to continue their trip downstream to be evaluated and/or processed by other route-pipeline pairs.

This is very useful in cases where the same set of events needs to be processed differently and delivered to different destinations. Each clone can be decorated with KV pairs as necessary.

Other Considerations

Depending on your cloning needs you may want to follow a most specific first or most general first processing strategy. The general goal is to minimize the number of filters/routes an event gets evaluated against. For example:

  • If cloning is not needed at all (i.e. all final toggles at default), then it makes sense to start with the broadest expression at the top so as to consume as many events as early as possible.
  • If cloning is needed on a narrow set of events, then it may make sense to do that upfront and follow it with a route that consumes those clones immediately after.