Subscriptions and Routes

​

From the UI similarity, you’ve probably noticed that Subscriptions look and function much like Routes. There are both similarities and differences – which we’ll consider in reverse order.

Subscriptions/Routes Differences

​

Compared to Routes, Subscriptions present these exceptions:

• Subscriptions lack a Route’s Output or Output Expression field. (Instead, you associate Subscriptions with Projects, where Project editors can process the data and send it to Destinations.
• Subscriptions have no Final flag. So a Subscription is similar to a Route with the Final toggle switched off.
• Subscriptions do not relay backpressure upstream to data senders. (This is by design, to prevent any Subscription from affecting data flow through parallel Subscriptions, Projects, or Routes/Pipelines.) If any backpressure occurs, a Subscription will drop data.

Subscriptions/Routes Similarities

​

A further link to Cribl Stream’s Data Routes interface is that Subscriptions can ingest data only from Sources configured (at the Cribl Stream product level) as Send to Routes. (Subscriptions and Projects cannot ingest data from QuickConnect Sources, despite the similarity of Projects’ visual UI (the Project view).

This also means that Subscriptions tap exactly the same overall flow of inbound data that Data Routes do, in parallel. So there are cases where, after you’ve configured Subscriptions to ingest certain data, you might want to configure a Pipeline at the top of the Routing table to drop the same data. This will prevent Cribl Stream from processing the same events twice.

Usage Tips

​

Compared to sending the same data flow through Routes, adding Subscriptions can have a performance impact, because each Subscription creates a copy of the data that Cribl Stream needs to process. Therefore, your resource requirements could change with the number of Subscriptions you add.

Please work with your Cribl Solutions Engineer to select an architecture that best balances your needs for performance versus segmented data/configuration access.

Cribl recommends that you configure a separate Subscription – and corresponding Pipeline(s) – for each data type you ingest. This enables you to filter by data type in the Subscription’s definition, and again in individual Pipeline Functions.

Accessing Subscriptions

​

1. From your Worker Group’s Manage submenu, select Projects.

2. Select the Subscriptions lower tab.

   Any Subscriptions you’ve already configured will appear here. Expand the accordions to access each Subscription’s configuration.

Configuring a New Subscription

​

1. From the Subscriptions lower tab, click Add Subscription to open the configuration options described below.

2. Assign this Subscription a unique Subscription ID. (This can include letters, numbers, _, and -. IDs are case-sensitive, so entering the same characters with different capitalization will create a separate ID.)

3. Modify the Filter by replacing the default true with a JavaScript expression that selects a subset of events to filter through this Subscription. (Your expression can specify an upstream sender, a data type, the presence of particular fields, etc. For examples, see Cribl Expression Syntax.)

4. Modify (as needed) the Source’s default passthru Pre‑processing Pipeline.

5. Optionally, add a Description of the Subscription’s purpose.

6. Optionally, assign the Subscription to one or more Projects. (You can also import Subscriptions from individual Projects’ configurations, and can control Subscriptions’ visibility in the Project view.)

7. Click Save to save the new Subscription to the Subscriptions table.

8. Commit and deploy your changes.

   A Subscription must specify a Pre‑processing Pipeline, even if this is the default passthtru, which performs no processing.

Adding and Previewing More Subscriptions

​

From the Subscriptions tab, you can click Add Subscription repeatedly to define multiple Subscriptions. These stack on the left, like Routes in Cribl Stream’s Routing table.

Note these further similarities to the Routing table:

• Each Subscription’s blue title bar has a toggle that you can use to disable the Subscription from gathering data, while preserving its configuration.
• On the right, you can capture, import, and save sample data, and. You can use these samples to preview how your Subscriptions transform incoming data.

Editing a Subscription

​

To update an existing Subscription, just click its accordion in the Subscriptions table. This will reopen the same configuration options outlined above.