Skip to main content
Version: 3.2

Routes

Before incoming events are transformed by a processing Pipeline, Cribl LogStream uses a set of filters to first select a subset of events to deliver to the correct Pipeline. This selection is normally made via Routes.

Don't Need Routes?

Routes are designed to filter, clone, and cascade incoming data across a related set of Pipelines and Destinations. If all you need are independent connections that link parallel Source/Destination pairs, you can use LogStream's QuickConnect rapid visual configuration tool as an alternative to Routes.

Accessing Routes

Select Routing > Data Routes from LogStream's global top nav (single-instance deployments) or from a Worker Group's top nav (distributed deployments). To configure a new Route, click + Route.

How Do Routes Work

Routes apply filter expressions on incoming events to send matching results to the appropriate Pipeline. Filters are JavaScript-syntax–compatible expressions that are configured with each Route. Examples are:

  • true
  • source=='foo.log' && fieldA=='bar'

There can be multiple Routes in a LogStream deployment, but each Route can be associated with only one Pipeline.

Routes are evaluated in their display order, top‑>down. The stats shown in the Bytes/Events (toggle) column are for the most-recent 15 minutes.

Routes and bytes

In the example above, incoming events will be evaluated first against the Route named speedtest, then against mtr, then against statsd, and so on. At the end, the main Route serves as a catch-all for any event that does not match any of the other Routes.

Above, note the selectors to toggle between displaying Events versus Bytes, and to display In versus Out.

When you condense the Routes page to a narrower viewport, LogStream consolidates the In/Out/Dropped selectors onto an expanded Bytes/Events drop-down menu, as shown below.

Routes and events (combined menu)

Managing the Routes Page

To apply a Route before another, simply drag it vertically. Use the sliders to turn Routes On/Off inline, as necessary, to facilitate development and debugging.

You can press the ] (right-bracket) shortcut key to toggle between the Preview pane and the expanded Routes display shown above. (This works when no field has focus.)

Click a Route's Options (...) menu to display multiple options for inserting, grouping, moving, copying, or deleting Routes, as well as for capturing sample data through the selected Route.

Route > Options menu

Copying a Route displays the confirmation message and the (highlighted) Paste button shown below.

Paste button for copied Route

Pasting creates an exact duplicate of the Route, with a warning indicator to change its duplicate name.

Pasted duplicate Route

Output Destination

You can configure each Route with an Output that defines a Destination to send events to after they're processed by the Pipeline.

If you toggle the Enable expression slider to Yes, the Output field changes to an Output expression field. Here, you can enter a JavaScript expression that LogStream will evaluate as the name of the Destination. Sample expression format: myType:myDest_${C.logStreamEnv}. (This evaluation happens at Route construction time, not per event.)

The Final Toggle

When an event that enters the system and matches a Route-Pipeline pair, it will usually be either:

  • Dropped by a function, or
  • Transformed (optionally) and exit the system.

This behavior is ensured by the Final toggle in Route settings. It defaults to Yes, meaning that matched events will be consumed by that Route, and will not be evaluated against any other Routes that sit below it.

If the Final toggle is set to No, clone(s) of the matching events will be processed by the configured Pipeline, and the original events will be allowed to continue their trip to be evaluated and/or processed by other Route-Pipeline pairs.

Final Flag and Cloning Considerations

Depending on your cloning needs, you might want to follow a most-specific first or a most-general first processing strategy. The general goal is to minimize the number of filters/Routes an event gets evaluated against. For example:

  • If cloning is not needed at all (i.e., all Final toggles stay at default), then it makes sense to start with the broadest expression at the top, so as to consume as many events as early as possible.
  • If cloning is needed on a narrow set of events, then it might make sense to do that upfront, and follow it with a Route that consumes those clones immediately after.

Route Groups

A Route group is a collection of consecutive Routes that can be moved up and down the Route stack together. Groups help with managing long lists of Routes. They are a UI visualization only: While Routes are in a group, those Routes maintain their global position order.

Route groups work much like Function groups, offering similar UI controls and drag-and-drop options.

Unreachable Routes

Routes display an "unreachable" warning indicator (orange triangle) when data can't reach them.

Unreachable Route warning, on hover

This condition will occur when, with your current configuration, any Route higher in the stack matches all three of these conditions:

  • Previous Route is enabled (slider is set to On).
  • Previous Route is final (Final slider is set to Yes).
  • Previous Route's Filter expression evaluates to true, (e.g., true, 1 === 1, etc.).

Note that the third condition above can be triggered intermittently by a randomizing method like Math.random(). This might be included in a previous Route's own Filter expression, or in a Pipeline Function (such as one configured for random data sampling).

Unreachable Route warnings, many

Routing with Output Router

Output Router Destinations offer another way to route data. These function as meta-Destinations, in that they allow selection of actual Destinations based on rules. Rules are evaluated in order, top‑>down, with the first match being the winner.