On This Page

Home / Guard/Detect Sensitive Data with Cribl Background Detection

Detect Sensitive Data with Cribl Background Detection

Background detection in Cribl Guard is an AI-driven feature that continuously samples data in your Pipelines, scans it for new sensitive data patterns, and surfaces findings. You can then review findings, ignore them, or mitigate them by adding Guard rules.

Background detection workflow at-a-glance:

  1. Enable background detection in AI Settings and on Destinations.
  2. Commit & Deploy. Background detection runs silently, sampling data streams for new patterns.
  3. Review and act on findings from the Guard page (create rules, mark mitigated, or ignore).
  4. Optionally analyze detections and read review output for the selected Worker Group.
  5. Optionally refine scope per Pipeline or disable when needed.

Before You Start

  • Cribl Guard must be enabled for at least one Destination. See Configure Cribl Guard.
  • Environment: Background detection is available only in Cribl Stream, in the following deployment types:
    • Hybrid environments with Cribl-managed Cloud Workers (AWS-hosted Cloud Worker Group).
    • Cribl.Cloud-only environments (AWS-hosted Cloud Worker Group).
    • On-premises deployments.

Enable Background Detection and Select a Model

Use this path to turn on background detection for your Workspace, select a Cribl AI model, and choose which Destinations use it. Default sampling size and interval apply.

  1. In your Cribl Workspace, select Cribl Stream.

  2. Go to Settings > AI Settings.

  3. Find Background Detection and toggle it On to add its tile to the Guard homepage.

  4. Select an AI model from the drop-down. For more details on each model, see About Cribl Guard Models.

  5. In the sidebar, select Guard.

  6. In the Background Detection column, select Enable Detection for each Destination you want to monitor.

    Guard homepage with background detection column and Enable Detection
    Guard homepage with background detection column and Enable Detection

    After you enable a Destination, the Background Detections tile shows total findings and a Review All link.

  7. Select Commit & Deploy to push AI Settings, Guard config, and background detection scope to your Workers.

Background detection is now running. Next: Review and Act on Findings.

About Cribl Guard Models

You can select the Cribl AI model to use with background detection. In AI Settings, there is a drop-down with model options:

  • cribl-privacy-2.0: A balanced default, this is the best option for most users. It contains a balance of speed and depth.
  • cribl-privacy-2.0-fast: A lightweight, speed-optimized model. Use this when compute efficiency, throughput, or footprint is more important than maximum detection depth.

These are not large-language models (LLMs). They are based on Named Entity Recognition (NER) models, which automatically scan text to identify and classify key information. The Cribl Guard AI models do not run data through any configured Custom AI Providers.

Review and Act on Findings

This is the main workflow after background detection is on: see what was found, then create rules, mark as mitigated, or ignore.

  1. Open the Guard homepage, then either:

    • Select Review All in the Background Detections tile, or
    • Select the yellow detections count in the Background Detection column for a specific Pipeline.
  2. Review the type of data found. Select a detection type (for example, CREDIT_CARD_NUMBER) to view sampled events.

    Events viewer with detection type and sampled events
    Events viewer with detection type and sampled events
  3. Take action via the three dots in the Actions column (or from the events viewer):

    • Create Guard Rule: Cribl creates a rule; you add it to a Scanning Ruleset in your Cribl Guard Pipeline and Commit & Deploy so it mitigates data going forward. Successful rules mask those sensitive entities in data.
    • Mark as Mitigated: Labels the finding as mitigated for history; does not change rules or Pipelines. Use this after you create a Guard rule.
    • Ignore Datatype: Cribl ignores future detections for this PII type.
    • Analyze Detections: Use Cribl AI to review detections, provide a summary, and suggest an action.

Analyze Detections

You can use Cribl AI to analyze Pipeline detections. Use Analyze Detections when you want a consolidated view of what was found before you create rules, mark items mitigated, or ignore Datatypes manually. When you select Analyze Detections, Cribl runs a background job that reviews recent background detection findings for the Worker Group you have selected on the Guard page.

If you’re using a custom AI model, Cribl Guard will run any detection analysis through the model you selected. If not, Cribl analyzes your data using the in-house, Cribl-managed model. For more information, see Custom AI Providers.

To analyze detections:

  1. Open the Guard homepage and select the Worker Group whose findings you want to include.
  2. Go to the Findings view (for example, select Review All in the Background Detections tile or open findings from a Pipeline detection count).
  3. Select Analyze Detections.
Analyze detected datatypes
Analyze detected datatypes
  • Analyzing detections reads findings only from the last 24 hours for that Worker Group. When the job completes, results for the job will populate in the results panel.

  • The job output is available only in the product after the job finishes; it is not saved as a separate downloadable report or history entry.

Next, continue with the usual finding actions (Create Guard Rule, Mark as Mitigated, Ignore Datatype) as needed.

Actions to Take on Guard Analysis Results

Depending on the result, you may see one or more of these flags:

Badge TitleDescriptionAction
Create Rule and Add to PipelineThe analysis result is a good candidate for a regex-based Guard rule.Select the Add rule badge under Recommended Action to allow Cribl AI to create a rule for the matching pattern and add it to the Pipeline.
Existing Rule FoundA rule matching the detected data already exists.Add the rule to the Pipeline containing the detected data, so that it will be mitigated.
False PositiveThe analysis result looks like it matches a sensitive data pattern, but the actual data is not sensitive.Mark these detections as ignored.
Manual Review RequiredThe results are ambiguous.Review the detection yourself and make a decision based on your findings.

Optional: Refine the Scope of Background Detection

Use these options if you want to limit where background detection runs or turn it off for specific Pipelines.

Enable Per Pipeline

You can turn background detection on for specific Pipelines in a Worker Group so only certain data streams use it.

  1. Open a Pipeline that contains the Cribl Guard Function.
  2. Expand the Function and scroll to the bottom.
  3. Toggle Background Detection on.
Cribl Guard Function with background detection toggle
Cribl Guard Function with background detection toggle

Turn Off in Specific Pipelines

To disable background detection for Pipelines with highly sensitive or constrained data:

  1. Open the Worker Group, then Processing > Pipelines and the Pipeline that contains the Guard Function.
  2. Toggle Background Detection off for that Pipeline.

Disable Background Detection

You can turn background detection off for a single Pipeline, for a Destination, or for the whole Workspace.

For One Pipeline

  1. Open the Worker Group, then Processing > Pipelines and the Pipeline with the Guard Function.
  2. Expand the Cribl Guard Function and toggle Background Detection off.

For One Destination

  1. Open each Pipeline that sends data to that Destination.
  2. Turn off Background Detection in every Cribl Guard Function in those Pipelines.

For the Whole Workspace

  1. In the Workspace, select Cribl Stream.
  2. Go to Settings > AI Settings.
  3. Set Background Detection to Off.

Then Commit & Deploy so changes apply to your Workers.

When it’s off: If background detection is disabled in a Pipeline, PII in that Pipeline won’t get a Guard mitigation rule and won’t appear as a detection on the Guard page. If it’s off in global AI Settings, PII detected across Pipelines without a Guard rule won’t be shown as a detection on the Guard page.

Why Use Background Detection?

BenefitDescription
Find unknowns in your dataCatches sensitive data that existing Guard rulesets missed (new types, sources, formats) before it reaches Destinations.
Better compliance and reportingOne place to see new findings, open issues, and past remediations.
Faster path to protectionFindings are surfaced automatically; you choose whether to ignore or mitigate (e.g., by creating a Guard rule).

Common Use Cases

Use CaseStream User PersonaDescription
Prove and improve data risk postureCISO & CIOContinuously see where new PII, secrets, or regulated data enter telemetry; show that Guard catches previously unknown patterns and keep an evidence trail after remediation.
Continuous protection for high-risk DestinationsCompliance / ObservabilityUse background detection on high-value Destinations (SIEM, data lake, observability, ticketing) to ensure unintended PII doesn’t reach them as Pipelines change; adjust Guard rules or routing when detections appear.
Monitor schema and data driftOperatorsUse findings as a signal when upstream teams add fields, change formats, or onboard apps; tune Guard rulesets or Pipeline filters instead of manually checking after every change.
Internal reporting and audits (Auditors / Risk)Auditor & Risk ManagementExport or summarize detections to report where sensitive data was found, how fast it was remediated, and what guardrails were added, for risk reviews, third-party assessments, and due diligence.