On This Page

Home / Search/ Metrics (Preview)/Manage Monitor Notifications

Manage Monitor Notifications

Preview Feature

Cribl is still developing this Preview feature. We don’t recommend using it in a production environment, because the feature might not be fully tested or optimized for performance, and related documentation could be incomplete.

Please continue to submit feedback through normal Cribl support channels, but assistance might be limited while the feature remains in Preview.

Manage the Targets, Templates, and Policies that deliver Monitor alerts, from one place.


About Monitor Notifications

When a Monitor triggers an alert, a Notification delivers that alert to where your team works, such as email, Slack, or a webhook. The Notifications area centralizes the three building blocks that shape and route those Notifications:

  • A Target is a delivery method, such as a Slack channel, a webhook endpoint, or an email recipient.
  • A Template defines the content of a Notification, using a Handlebars template.
  • A Policy routes an alert to one or more template-and-target pairs, based on conditions that match the alert’s labels.

A Monitor can route its alerts either through Policies or directly to a specific Target. See Notifications on the Monitor page.

Cribl Search has two notification experiences. This page covers Notifications for Monitors. To alert on scheduled searches instead, see Notifications. The two experiences share the same Notification Targets, so a Target you create here is also available to scheduled searches, and the reverse. Templates and Policies apply only to Monitors.

Open Monitor Notifications

On the Cribl.Cloud top bar, select Products > Search > Monitors, then select the Notifications tab. A left sidebar exposes three sub-tabs: Targets, Templates, and Policies.

Each sub-tab has its own read and write permissions. If your role can’t read a sub-tab, Cribl Search shows a “not authorized” message in place of the list. If your role can read but not write, you can view items but the Add, Edit, and Delete controls are disabled, and the row action shows View instead of Edit.

Targets

A Target specifies how and where to deliver a Notification. The Targets tab lists every Target with these columns:

ColumnDescription
Target nameThe Target ID, with a health indicator (Green, Red, or Unknown).
TypeThe Target type.
UsageHow many Policies and Monitors reference this Target.
Last UpdatedThe time of the Target’s most recent health check.
ActionsEdit (or View) and a Delete menu.

Target Types

You can create and edit these Target types:

  • Webhook
  • Slack
  • PagerDuty
  • SNS
  • Email (SMTP)

Cribl Search also manages some Targets automatically. These appear in the list but you can’t create or edit them: Bulletin Message, Default, Router, and Notifications Log. The built-in system_notifications and system_email Targets are view-only, so you can’t edit or delete them.

Add a Target

  1. On the Targets tab, select Add Notification Target.
  2. Give the Target a unique Target ID.
  3. Select the Target type, then configure the fields for that type.
  4. Select Create Target.

To change a Target, select its name or the Edit action, adjust the fields, then select Update Target.

Delete a Target

Select the Delete action on a Target’s row, or select multiple Targets and use the bulk Delete control. Cribl Search prevents deletion in these cases:

  • The Target is a system-managed Target: System targets can’t be deleted.
  • The Target is referenced by a Policy or Monitor: In use by a policy or monitor.

Templates

A Template defines the content of a Notification. The Templates tab lists every Template with these columns: Template name, Type, Usage, Last Updated, Created By, and Actions.

Templates marked with a Default tag are supplied by Cribl Search. You can view and clone them, but you can’t edit or delete them.

Template Fields

  • Template ID: a unique identifier for the Template. Required. You can’t change it after you create the Template.
  • Description: an optional note about the Template.
  • Target type: the Target type this Template formats content for, one of Webhook, Slack, PagerDuty, SNS, or Email (SMTP). Required. You can’t change it after you create the Template.
  • Content: the Notification body, written as a Handlebars template and edited in an inline code editor. Required.

Cribl Search validates the Content field as Handlebars. If the field is empty, you see Template content is required. If the syntax is invalid, you see Invalid Handlebars template syntax.

Add a Template

  1. On the Templates tab, select Add Notification Template.
  2. Enter a Template ID, and optionally a Description.
  3. Select the Target type.
  4. Write the Content in the editor.
  5. Select Create Template.

To reuse an existing Template as a starting point, select its Clone action. To review a Template without changing it, select View.

Delete a Template

Select the Delete action, or select multiple Templates and use the bulk Delete control. Cribl Search prevents deletion in these cases:

  • The Template is a default Template: Default templates can’t be deleted.
  • The Template is referenced by a Policy or Monitor: In use by a policy or monitor.

Policies

A Policy routes an alert to one or more template-and-target pairs when the alert’s labels match the Policy’s conditions. The Policies tab lists every Policy in evaluation order, with these columns:

ColumnDescription
IDThe Policy ID, with a number that shows its position in the evaluation order.
RoutingWhether the Policy is Final. A final Policy ends routing for a matching alert.
DescriptionA summary in the form If <conditions> then send to <target → template pairs>.
ActionsEdit (or View) and a Delete menu.

Policies are evaluated top-down. To change the order, drag a Policy’s handle to a new position.

Conditions

A Policy’s conditions are grouped. Within a group, all conditions must match (AND). A Policy matches if any group matches (OR). A Policy with no conditions matches every alert, shown as all.

Build conditions with these controls:

  • Add Label: add a condition to the current group.
  • Add Label Group: add another OR group.

Each condition has a key, an operator, and a value. The operators are =, !=, =~ (regex), and !~ (regex not) - the same operators as a Monitor’s Query label filters. Cribl Search suggests keys and values from your Monitors’ labels, drawn from Monitor metadata, PromQL label filters, and threshold tags.

Template-and-Target Pairs

Each Policy sends matching alerts to one or more pairs, each of which combines a Template with a Target. The Description column shows each pair as target → template. A pair with a missing Target or Template appears in red.

Add a Policy

  1. On the Policies tab, select Add Notification Policy.
  2. Enter a unique Policy ID.
  3. Define at least one condition group. Each condition requires a key.
  4. Add at least one template-and-target pair. Each pair requires both a Target and a Template.
  5. Optionally turn on Final to stop routing at this Policy for matching alerts.
  6. Select Save.

Preview Policy Matches

To check which Policies a set of labels would match, turn on the Preview toggle. Cribl Search then shows:

  • A label selector where you choose the labels to test.
  • A Generate Labels from Monitor button that fills the selector with labels from one of your Monitors.
  • A Status column that marks each matching Policy, and condition tags that turn green when they match.

Permissions

Targets, Templates, and Policies each have their own read and write permissions:

  • Read controls whether you can open a sub-tab and view its items.
  • Write controls whether you can add, edit, or delete items on that sub-tab.

Read-only users see View in place of Edit and can inspect items, but can’t create, change, delete, or reorder them. Creating or editing a Monitor requires separate Monitor write permission.

Next Steps