On This Page

Home / Search/ Metrics (Preview)/Create Monitors in Cribl Search

Create Monitors in Cribl Search

Preview Feature

Cribl is still developing this Preview feature. We don’t recommend using it in a production environment, because the feature might not be fully tested or optimized for performance, and related documentation could be incomplete.

Please continue to submit feedback through normal Cribl support channels, but assistance might be limited while the feature remains in Preview.

Get notified when a metric moves outside its expected range, using Monitors.


About Monitors

A Monitor watches a metric query and raises an alert when the metric meets a condition you define, such as crossing a threshold or behaving anomalously.

  • A Monitor is what keeps watch - a metric query, its detection method, thresholds, and firing conditions.
  • An alert is the recorded moment a Monitor’s condition was met.
  • A Notification delivers that alert to where your team works, such as email, Slack, or a webhook.

Monitors are part of the Metrics experience. To explore and visualize metrics before you alert on them, see Explore Metrics in Cribl Search.

During Preview, some routing and suppression controls are not yet wired up - the sections below call out what is coming soon.

Monitor Permissions

Viewing a Monitor needs read access to every metrics Dataset that the Monitor evaluates. Creating, editing, and deleting Monitors needs that same read access, plus the Editor or Admin Permission on Cribl Search.

Members with the Admin Permission on Cribl Search can manage Monitors on any metrics Dataset. Members with the Editor Permission can manage Monitors only on the metrics Datasets that a Maintainer has shared with them. Members with the User Permission can view those Monitors and their alerts, but can’t change them. For details, see Metrics Permissions.

Open Monitors

On the Cribl.Cloud top bar, select Products > Search > Monitors.

The list shows only the Monitors whose queries all read metrics Datasets you can access. A Monitor on a Dataset you can’t access is hidden rather than shown as locked, so two Members can see different Monitors in the same Organization.

From the Monitors list, you can:

  • Search and filter your Monitors.
  • Enable or disable a Monitor with its row toggle.
  • Select Monitors for bulk actions.
  • Select Add Monitor to create one, or select an existing Monitor to edit it.

Build a Monitor

The Monitor page has a live preview chart at the top. Expand the sections below the chart to configure your Monitor settings.

  • Turn Simulate on past data on to preview how the Monitor would have behaved against recent data.
  • Use the Enabled toggle in the header to turn the Monitor on or off, then select Save.

Configure the Monitor in these sections:

  1. Detection Method: choose how the Monitor fires.
  2. General: name the Monitor and set its priority and team.
  3. Query: choose the metric and shape the query.
  4. Thresholds: map values to severities.
  5. Metadata: attach key-value labels to alerts.
  6. Firing Conditions: control how long a condition must hold.
  7. Notifications: decide how alerts are delivered.

When you edit an existing Monitor, the page splits into a Configure tab and an Activity tab. See View Alert Activity.

Detection Method

The Detection Method determines how the Monitor decides to fire an alert. Select one:

MethodFires when
ThresholdMetrics cross defined thresholds.
ChangeThe difference between a value N minutes ago and now.
AnomalyA metric behaves anomalously based on historical data.
ForecastA metric is projected to cross a threshold.

General

  • Name: a unique name for the Monitor. Required.
  • Description: a short note on what the Monitor checks and why it matters.
  • Priority: the urgency of the Monitor: P1 (Critical), P2 (High), P3 (Medium), P4 (Low), P5 (Info), or Not defined.
  • Team: the team that owns the Monitor.

Query

The Query section defines what the Monitor evaluates. You can build the query visually or write PromQL directly.

Visual builder:

  • Metric: the base metric to evaluate.
  • Label filters: narrow the series with key-value matchers. Operators are =, !=, =~ (regex), and !~ (regex not).
  • Aggregation: how samples roll up over the window: count, avg, sum, min, or max.
  • Group by: one or more labels to evaluate as separate series.
  • Evaluation window: the rolling lookback, such as 5m or 1h.

PromQL:

  • Enter PromQL directly in the query editor.
  • Use $variable in the query to add template parameters. Cribl Search shows a Resolved Query preview with the parameter values filled in.

You can add more than one query, labeled A through D, and give the Monitor a Units value (such as seconds, bytes, or count) so charts and thresholds read correctly.

When a Monitor has more than one query, or when a query reads a different metrics Dataset than the Monitor’s default, you need read access to every metrics Dataset the Monitor evaluates. Cribl Search rejects a Monitor that reads a metrics Dataset you can’t access, and doesn’t save it. See Monitor Permissions.

Thresholds

Thresholds map evaluated values to severities in a Default Rule. For each row, configure:

  • Severity: Critical, Warning, or Info.
  • Operator: greater than, greater than or equal, less than, less than or equal, equal to, or not equal to.
  • Threshold: the numeric value to compare against.
  • Times Triggered: the number of consecutive evaluations the condition must hold before the alert fires. The minimum is 1.

Select Add severity to add a row (up to six). Turn on Show on Chart to draw the threshold on the preview chart.

Use Included Tags and Excluded Tags to scope the rule to specific series.

Metadata

Add arbitrary key-value pairs, such as env=prod or team=platform. These become labels on every alert the Monitor generates, so Notification policies and templates can use them for routing and formatting.

Firing Conditions

Firing Conditions control how long a condition must hold before an alert opens or clears. This is how Monitors avoid alert flapping when a metric hovers around a threshold.

  • Fire delay: how long the condition must stay true before the alert fires.
  • Clear delay: how long the condition must stay healthy before the alert clears.

Both accept Immediate, 1 minute, 5 minutes, 10 minutes, 15 minutes, 30 minutes, or 1 hour. Set the Clear delay together with Times Triggered to smooth out noisy metrics.

Notifications

Choose how alerts are delivered:

  • Route via Notification Policies: route alerts by matching their labels against shared policies. Policies are evaluated top-down, and each matching policy processes the alert until a final policy ends routing.
  • Select Specific Notification Target: send directly to a chosen target, bypassing policies.

To create and manage the Targets, Templates, and Policies that these options use, see Manage Monitor Notifications.

Mute a Monitor

Muting suppresses a Monitor’s Notifications during a planned window, such as scheduled maintenance, without disabling the Monitor or losing alert history.

During Preview, Add Muting Rule is coming soon on Cribl Search Monitors. The button appears on the Monitor page but is not yet active. For Insights Alerts, see Muting Rules.

View Alert Activity

When you edit a Monitor, the Activity tab shows its history, so you can review when it fired and how it changed over time. The Alerts area also has an Activity tab across all Monitors.

Alert activity follows the same access rules as the Monitors list, so you see activity only for the Monitors you can access. History isn’t available for a deleted Monitor, because Cribl Search no longer has a configuration to check your access against.

Next Steps