Notifications

Send Notifications about your scheduled searches.

Cribl Search can alert you when your scheduled searches generate results that match triggering conditions that you specify. You set up alerts in these basic stages:

1. Optionally, configure any custom Notification targets to where you want to send alerts. (Cribl Search comes preconfigured with at least one default System Messages. target, Some plans add a preconfigured system_email.)
2. Save and schedule each search on which you want to trigger alerts.
3. Configure one or more Notifications on each scheduled search.

You can’t configure Notifications on a saved or scheduled search that resides within a Pack.

Notification Targets

​

A Notification target specifies the delivery method for a Notification. Every Notification must have at least one target. Available target types are:

• System Messages: Banner messages in the Cribl Search UI (the default target).
• Email messages.
• Amazon SNS topics.
• PagerDuty accounts.
• Slack channels.
• Webhook connections.

By default, any Notification that you configure will have a System Messages target. Also, your Notification will include this target regardless of any other target that you add. When a Notification condition is triggered, Cribl Search will add an indicator on the top nav’s 🔔 Messages button. Here, select this button to view details in the Messages drawer.

Add a Notification Target

​

To add a new Notification target:

1. In Cribl Search, go to Settings > Search > Notification Targets.
2. Select Add Target to open the New Target modal.
3. Give this target a unique Target ID.
4. Select the desired Target type.
5. Configure the target as outlined on that target’s appropriate page (see Notification Targets).

When you create a Notification target in Cribl Search, Stream, or Edge, it will also be available to you in the other products. For example, if you create a Notification target in Cribl Search, you can also access it in Cribl Stream and Cribl Edge.

Manage Notification Targets

​

You manage existing targets on the same Notification Targets page where you create targets.

Select any existing Notification target to open a modal where you can manage the target, using buttons at the bottom of the modal.

Select Delete Target to delete an existing Notification target.

Select Clone Target to copy an existing target. This refreshes the modal, enabling you to customize the new target.

To edit any target’s definition in a JSON text editor, select Manage as JSON at the bottom of the Notifications Targets modal. You can directly edit multiple values, and you can use the Import and Export buttons to share existing target configurations as .json files.

Configure Notifications

​

To configure how, when, and where to send Notifications:

1. Schedule a search, or open an existing scheduled search.

2. In the search’s configuration modal, open the Notifications left tab.

3. Toggle Send notifications on, and configure the options listed below.

   The Notifications left tab unlocks only after you enable the Schedule tab’s Run on schedule option. To configure further Notifications on the same scheduled search, select Add Notification.

Notification Options

​

As you configure each Notification, the Notifcations tab offers the following options.

• When: Select the type of trigger condition, then define the corresponding trigger. (See Notification Triggers.)

• Send Notification to: Select an existing Notification target to deliver the Notification, or select Create to create a new target. (For target options, see Notification Targets.)

  All Notifications, regardless of their configured target, will also generate a bulletin message in the Cribl Search UI.

• Include search results: To see how each alert type implements this option, follow the links from Notification Targets.

• Message: Message payload template. For details, see Message Options below.

Message Options

​

The message sent with each Notification can contain a maximum of 1,000 characters, and a maximum of 100 events x 20 fields.

In the Message pane, you can customize each Notification’s contents by inserting the template variables listed below. These are placeholders to populate your message with fields generated by the search. Insert a variable between pairs of double braces, in this format: {{savedQueryId}}.

Here’s an example that inserts some literal text followed by a corresponding variable: Date: {{timestamp}}. This is the first string in the default template, shown below.

The following template variables are available.

If you choose to explicitly add the {{resultSet}} variable to the message body, this will have different effects with different targets:

• With a PagerDuty or Slack Notification, you can include `{{resultSet}} to control where inline search results will be displayed within alerts.

• With an Email Notification, including {{resultSet}} will insert placeholder text indicating that the results will appear either “in the table below” (if you’ve selected Send as: Inline table) or “attached” (if you’ve selected Send as: CSV or JSON).

• With an Amazon SNS or Webhook Notification, including {{resultSet}} will send duplicate result sets – one each in the message and on the payload.

Notification Triggers

​

On the Notifications left tab, use the When… controls to define the trigger condition that will send the Notification. The drop-down here provides two options.

• Where: Select this to define a custom condition. See Notification Trigger Examples below.

• Count of Results: Exposes a drop-down where you can select a comparison operator, and a field where you can enter the corresponding threshold number.

How Notification Triggers Work

​

Cribl Search will send a Notification when at least one event satisfies the trigger condition. This can be useful if you want to define a search query that returns multiple results, but want alerts sent only when a specific condition among these results is met.

Notification Trigger Examples

​

For a query like this:

dataset="cribl_internal_logs" error 
| summarize errorCount=count() by message 
| top 10 by errorCount

You could enter this Where condition to alert when Request failed is in the top 10 errors:

message == "Request failed"

For a query like this:

dataset="cribl_search_sample" log_status="OK" 
| summarize count() by action

You could enter this Where condition to alert upon connection errors:

action== "REJECT"