On This Page

Home / Edge/ Work with Data/ Process Data with AI Assistance/Create Pipelines With Cribl Copilot Editor

Create Pipelines With Cribl Copilot Editor

Cribl Copilot Editor is an interactive, AI-powered assistant that helps you build data Pipelines in Cribl Edge using plain-language prompts. Instead of building a Pipeline manually, you can describe what you want to do with your data and Copilot Editor will generate a working Pipeline for you.

Copilot Editor gives you an easy way to explore what Cribl Edge can do, especially if you’re new to the platform or want help tackling more complex use cases. You stay in control: Copilot Editor’s suggestions are editable, and you can review, modify, or view outputs from the suggestions before using them in production.

Here are a few common ways you can use Copilot Editor:

  • Structure raw data to match a standard schema or convert from one schema to another: The data for many Destinations follow specific schemas. If events are improperly formatted, some Destinations may drop data or store them in ways that make the data difficult to find and use. With Cribl Edge, you can convert your raw data into normalized schemas, using Copilot to do the heavy lifting around configuring Functions. Copilot Editor is optimized for transforming data into the OCSF, UDM, and ECS schemas. You can also upload your own schema objects to convert to other schemas. See Convert Data into a Custom Schema for information about using custom schemas or schemas that aren’t yet officially supported.
  • Apply common Functions to transform or enrich your data: Copilot Editor can apply Cribl Edge’s most frequently used Functions based on your intent. You don’t need to know Regex or JavaScript syntax or which Functions to use up front. Just describe your goals and Copilot Editor builds the appropriate logic for you.
  • Filter and route data dynamically: Use plain-language prompts to define filtering or routing logic without writing Regex or JavaScript expressions manually. Copilot Editor builds the first draft of the filtering and control logic. You can use its suggestions as-is or fine-tune as needed.

How to Use Copilot Editor

This section explains the overall workflow for using Copilot Editor to build Pipelines.

Before You Begin

Before you can start using Copilot Editor, you must first enable Cribl Copilot for your Organization:

  1. From Settings, select the Global tab, and then select AI Settings.

  2. Read the privacy policy and accept the terms and conditions.

See Enable or Disable Cribl AI Features for information about how to enable or disable Cribl Copilot programmatically.

Launch Copilot Editor

You can launch Copilot Editor in three ways:

Select a tab for more detailed instructions about each method for launching Copilot Editor:

QuickConnectPipeline pageAdd Function menu

Build a Pipeline with Copilot Editor

The interactive flow of Copilot Editor follows a few general phases:

  1. Capture sample data: When you first launch Copilot Editor, you see a chat window that prompts you to capture sample data. Choose a method for providing sample data, such as capturing a live sample from a Source, selecting an existing sample file, or pasting raw events directly into the chat.

    Why does it need sample data?

    Copilot Editor uses this data to build, preview, test, and validate the data transformations it creates. It helps you compare inbound and outbound data to ensure the Pipeline produces the expected output.

  2. Choose an objective: After receiving your sample data, Copilot Editor summarizes the data and presents options for transforming it. Select the objective that matches your goal. Copilot Editor prompts you for any additional information it needs.

  3. Review the Pipeline: Copilot Editor generates the Pipeline automatically and presents the output events in the chat window so you can see what the transformed data looks like. Expand the Functions in the editor to explore the settings it applied. Use natural language prompts in the chat window to request changes and continue to iterate until the Pipeline meets your needs.

  4. Apply the Pipeline: After you apply the Pipeline, select Back to QuickConnect at the top of the chat window to return to QuickConnect. Select Save Connection to add the Pipeline to your deployment.

    Take note of the Pipeline ID displayed in the chat window so that you can find this Pipeline later.

  5. Confirm the Pipeline is present: Confirm that the Pipeline you built with Copilot Editor appears on the Pipelines page or in QuickConnect. To find it in QuickConnect, select the connection you used to launch Copilot Editor, then select its Pipeline icon and select Pipeline to view the associated Pipelines.

Edit or Update a Pipeline With Copilot Editor

When you open an existing Pipeline and launch Copilot Editor, the Pipeline loads automatically. You can then choose how to proceed:

  • Describe your edits directly: Tell Copilot Editor what changes you want to make, and it applies them to the loaded Pipeline.
  • Capture sample data first: Optionally collect sample events to understand the Pipeline’s current behavior or test changes before committing to them.

After Copilot Editor applies your edits, it presents the output events in the chat window so you can review the results. Use natural language prompts to continue refining the Pipeline as needed.

When you are satisfied with the changes, Copilot Editor asks how to save them:

  • Save as a new Pipeline: Creates a new Pipeline, leaving the original unchanged. Use this option when you want to test the updated version alongside the original.
  • Overwrite the existing Pipeline: Replaces the original Pipeline with the edited version.

You can edit any existing Pipeline with Copilot Editor, not just Pipelines that were previously generated with Copilot Editor.

To start an edit session, launch Copilot Editor from one of these entry points:

  • From inside a Pipeline, select Add Function, then Edit with Copilot Editor.
  • From QuickConnect, select a connection that has an existing Pipeline, then open Copilot Editor.

Guidelines and Better Practices

When working with Copilot Editor, keep these guidelines in mind:

  • Always review and validate Pipeline suggestions: As with any AI tool, you should always check and confirm that the suggested changes match your expectations. Trust, but verify.
  • Copilot Editor may take different paths to meet an objective: AI tools do not always solve problems in the exact same way every time, which sometimes takes some getting used to. This behavior is by design, but just be aware that the results may vary.

Use effective prompts to ensure the best results from the Copilot Editor. Some general guidelines:

  • Be specific: To receive high-quality results, be clear and explicit with your requests. Provide exact names of fields that you want to reference or create.
  • Provide examples: For instance, if you want to mask a specific piece of text, describe the target text in detail and explain what it should be masked to (for example, Mask all IPv4 IP addresses to X.X.X.X).
  • Ask questions: Use the chat window to answer general questions about Pipelines. For example, you could ask the chatbot, What Pipeline Functions can I use to redact sensitive information like IP addresses?

Feature Support

Cribl does not make any warranty regarding the availability, uptime, performance, or accuracy of Cribl Copilot, including PII Detection and Prevention. Please read the Supplemental Terms for Cribl Copilot for more information.

Requirements

By default, Cribl Copilot Editor requires internet access to reach the Cribl-managed AI backend. If you configure a Custom AI Provider, Copilot Editor instead requires network connectivity to the configured provider endpoint and does not need to reach the Cribl-managed backend.

Supported Deployment Types

Cribl Copilot Editor is available:

  • For Cribl.Cloud or hybrid deployments.
  • For Cribl Edge 4.12.0 and newer.

Supported Schemas

Cribl Copilot Editor specializes in transforming data into these schemas:

SchemaSupported Version(s)
OCSF
Open Cybersecurity Schema Framework
  • 1.6.0
  • 1.5.0
  • 1.4.0
UDM
Unified Data Model
  • 6.0
ECS
Elastic Common Schema
  • 9.0.0 (base version only)

Supported Functions and Integrations

Copilot Editor can build Pipelines using these Functions:

Copilot Editor can support all available Sources and Destinations.

What Data the Copilot Editor Can Access

The Copilot Editor assistant has access to:

  • The natural language query, written by the user.
  • The sample input event selected by the user.

Copilot Editor has access to no other user or Organization data.