These docs are for Cribl Edge 4.12 and are no longer actively maintained.

See the latest version (4.13).

Packs

Packs enable Cribl Edge administrators and developers to pack up and share complex configurations and workflows across multiple Worker Groups, or across organizations.

Benefits of Using Packs

​

Using Packs can help you scale your Cribl Edge deployment easily. You can build and test configurations in one environment or Fleet and then port those configurations to other environments or Fleets as needed.

One example is routing an existing stream of data to additional Destinations (like object storage). You can test the flow end to end before making changes to production environment by adding Sources, Routes, Pipelines, and Destinations to a Pack. This lets you consolidate your dependencies, develop, and test changes before porting them to production instances or deploying them to multiple Fleets.

Another example is consolidating dependencies on Lookups. Where a Pipeline configuration references lookup file(s), Cribl Edge will import the Pipeline only if the Lookups are available in their configured locations. A Pack can make the Pipeline portable across Cribl Edge instances. If you import the Lookups into the Pack, you can develop and test a configuration, and then port it from development to production instances, or readily deploy it to multiple Fleets.

Packs also improve your ability to troubleshoot problems in your environment. You can export and import Packs to other environments to prevent or detect problems that might be caused by configuration changes between environments. You can also use Packs to accelerate troubleshooting when working with Cribl Support, because they facilitate quickly replicating your Cribl Stream environment.

Expanded Pack Capabilities

​

If you’re familiar with Packs from prior versions of Cribl Edge before 4.12, it’s important to know that Packs have evolved. In previous versions, Packs were mainly used to package processing logic and included Pipelines, Routes, Functions, Knowledge Objects, and Sample data files.

Now, Packs can contain complete, end-to-end data flow configurations:

• Sources
• Destinations
• Event Breakers
• Pipelines
• Routes
• Knowledge Objects (such as lookup files)
• Sample data files

Within a Pack, Event Breaker Functions do not support custom rulesets defined in the same Pack. You can use custom Event Breaker rulesets in Sources and Destinations within the Pack. (Known Issue).

See Supported Pack Integrations for a full list of supported Sources and Destinations.

This means you can use Packs to build and share end-to-end ingestion and delivery solutions — not just the processing logic. You can manage full data stream configurations in a single version-controlled unit, making it easier to test, share, and promote solutions across traditional development environments (such as from dev to staging to prod).

These new Packs also supports Git workflows, templating, and CI/CD automation.

Sources and Destinations created in Packs add the Pack name to the __inputId and __outputId fields of their events, for example: datagen:My_Pack.my_datagen_source, or cribl_http:My_Pack.my_http_destination.

Best Practices for Designing Packs with Inputs and Outputs

​

If you use Packs that include inputs and outputs (Sources and Destinations), follow these best practices:

• Be cautious about chaining Sources and Destinations across Packs: It’s generally best to avoid mixing global Sources and Destinations with Packs that define their own Sources and Destinations. While it’s technically possible to use only the Routes and Pipelines from such a Pack, this approach can introduce maintenance challenges and ambiguity. It becomes unclear whether the Pack is intended to serve a global scope (across the entire Organization) or a local scope (self-contained within the Pack itself).
• Keep your Packs modular and focused: Try to keep each Pack scoped to a single purpose or use case. For example, a Pack that collects logs from a cloud provider and routes them to S3 or a Pack that transforms and enriches firewall events before delivery to a SIEM. Avoid mixing too many unrelated Sources or Destinations in one Pack. It increases complexity and makes version control harder.
• Use variables and naming conventions consistently: When Packs include Sources and Destinations, use clear, environment-aware naming (such as src_aws_logs_dev, dst_s3_prod). Consider using Pack-level variables to template environment-specific values.
• Track and tag changes: Use Git integration to track changes to your Pack components, tag known-good versions, and promote Packs safely across environments.
• Document what the Pack does: If storing the Pack in an external Git repository, include README files and comments within components that explain what kind of data the Pack handles, which environment is it intended for, and whether there are any Sources/Destinations shared across Packs.

Get Packs

​

You can add Packs to your deployment by importing them from:

• Packs Dispensary
• File
• URL
• Git repo

Pack Naming on Import

​

Each Pack that you install within a given Fleet (or Single-instance deployment) must have a unique ID. The ID is based on the internal configuration of the Pack – not its container file name, nor on its Display name.

If you import a Pack whose internal ID matches an installed Pack – whether an update, or just a duplicate – you can choose one of two options:

• Assign a unique New Pack ID. This will import the Pack as a new, separate Pack with the new ID.
• Toggle Overwrite on to overwrite the installed Pack, reusing the same ID.

Be careful about enabling the Overwrite option. Doing so will cause the imported Pack to completely overwrite your existing Pack configuration.

Each Pack within a Fleet must have a unique Pack ID, so you cannot share an ID between two (or more) installed Packs.

Add from the Packs Dispensary

​

To add a Pack from the Cribl Packs Dispensary sharing site:

1. Navigate to the Packs page.
2. Select Add Pack at the upper right and select Add from Dispensary.
3. The Packs Dispensary will open in a drawer, as shown in the screenshot above.
4. Using the drawer controls, browse, or search for the Pack(s) you want. (You can use the check boxes at the left to filter by data type, use case, and technology.)
5. Select any Pack tile to display its details page with its README. This will typically outline the Pack’s purpose, compatibility, requirements, and installation.
6. To proceed, select Add Pack on this page.
7. If the Pack you are importing binds variables to any fields, you will see a Configure Variables button in the notification about successful import. Select it to go to the Variables page, where you can configure the values for the variables for your deployment.

Import from a File

​

To import a Pack (.crbl file) from your local filesystem:

1. Navigate to the Packs page.
2. Select Add Pack at the upper right and select Import from File.
3. Select the file to import.
4. Optionally, give the Pack an explicit, unique New Pack ID. (For details about this option, see Upgrade an Existing Pack.)
5. Select Import to confirm the import.
6. If the Pack you are importing binds variables to any fields, you will see a Configure Variables button in the notification about successful import. Select it to go to the Variables page, where you can configure the values for the variables for your deployment.

Import from a URL

​

To import a Pack from a known, public or internal, URL:

1. Navigate to the Packs page.
2. Select Add Pack at the upper right and select Import from URL.
3. Enter a valid URL for the Pack’s source. (This field’s input is validated for URL format, but not for accuracy, before you submit the modal.)
4. Optionally, give the Pack an explicit, unique New Pack ID. (See Upgrade an Existing Pack.)
5. Select Import to confirm the import.
6. If the Pack you are importing binds variables to any fields, you will see a Configure Variables button in the notification about successful import. Select it to go to the Variables page, where you can configure the values for the variables for your deployment.

To import a Pack from a public URL, the Leader Node (or single instance) requires internet access. A Distributed deployment Leader can then deploy the Pack to Workers even if the Workers lack internet access.

Import from Git Repos

​

When you import a Pack from a private Git repository using a personal access token, you can also push changes back to the same branch or repository you imported from. If you don’t provide an access token, Cribl Edge cannot write back to the repository. This means you won’t be able to push changes to a repository without authenticating with a personal access token.

Some additional considerations:

• If you don’t specify a branch during import, Cribl Edge assumes you want to use the repository’s default branch.
• You cannot push changes to a Git tag. Importing from a tag is allowed, but writeback is only supported when importing from a branch.
• If you need to change the token used for Git operations, you’ll need to re-import the Pack using the new token.

To import a Pack from a known public or private Git repo:

1. Navigate to the Packs page.

2. Select Add Pack at the upper right and select Import from Git.

3. Enter the source repo’s valid URL.
   This field’s input is validated for URL format, but not for completeness or accuracy, before you submit the modal. When targeting a private repo, use the format: https://<username>:<token/password>@<repo‑address>. Public repos need only https://<repo‑address>, as shown in the following example.

4. Optionally, give the Pack an explicit, unique New Pack ID. (See Upgrade an Existing Pack.)

5. Optionally, enter a Branch or tag to filter the import source using the repo’s metadata. You can specify a branch (such as master) or a tag (such as a release number: 0.5.1, and so forth).

6. Select Import to confirm the import.

7. If the Pack you are importing binds variables to any fields, you will see a Configure Variables button in the notification about successful import. Select it to go to the Variables page, where you can configure the values for the variables for your deployment.

To import a Pack from a public repo, the Leader Node (or single instance) requires internet access. A Leader in a Distributed deployment can then deploy the Pack to Workers even if the Workers lack internet access.

Prepare Pack for Use

​

After you have imported a Pack, verify that secrets in imported fields have been retained. If not, re-enter them. This may happen if the Pack was exported in merge mode, which deletes all encrypted fields during the export process to ensure security and prevent the accidental sharing of sensitive information.

If you did not configure values for a Pack variable during import, you can do it later. In the imported Pack, go to Knowledge > Variables and select the variable you want to provide the value for.

Pack Repositories

​

There are two places where you can search for useful Packs to import into your deployment: Cribl Packs Dispensary, and the Dispensary GitHub Repo.

The Cribl Packs Dispensary

​

The Cribl Packs Dispensary is a Cribl-hosted resource for you to find and share Packs. Cribl, our partners, and community users develop Packs and submit them to the Dispensary for easy sharing. Cribl tests submissions before publication, to ensure each Pack’s quality, security guardrails, and stability.

You can install Dispensary Packs directly through Cribl Edge’s user interface, as outlined in Add from Packs Dispensary.

Interested in publishing your own Packs on the Cribl Packs Dispensary? See Publishing a Pack.

Dispensary GitHub Repo

​

A particularly useful public repo is the Cribl Pack Dispensary on GitHub. This repo was established prior to the Cribl-hosted Cribl Packs Dispensary site described above, and it is a place to collaborate on developing Packs prior to submitting them to the newer site.

You can install this repo’s Packs directly through Cribl Edge’s user interface, as outlined in Import from Git Repos above. However, if you prefer, you can click through to any Dispensary repo release page, download the corresponding .crbl file, and then upload the file into Cribl Edge.

If you’ve posted completed Packs to our GitHub repo, we encourage you to also submit them to the Cribl Packs Dispensary site. See Publishing a Pack.

Use Packs

​

Version Compatibility

​

Packs created or modified in Cribl Edge 4.0.x cannot be used in any Cribl Edge version older than 4.0. (If you try, you’ll see a should NOT have additional properties error.) To avoid this problem, Cribl recommends that you upgrade to Cribl Edge 4.0.x or newer.

For compatibility questions about any individual Packs, please contact us in the Cribl Community Slack #packs channel.

For Packs that do not contain Sources or Destinations, you can use them in all places where you can reference a Pipeline:

• In Sources, where you attach pre-processing Pipelines.
• In Destinations, where you attach post-processing Pipelines.
• In Routes, in the Routing table’s Pipeline and Destination columns.

This expanded view shows how a Pack can replace a Pipeline in a Route:

Packs are distinguished in the display with a PACK badge, as you can see here in the Routing table:

The PACK badge is also displayed when you click into a resource – shown here on one of the Routes from the above table:

Cribl Edge’s Monitoring page includes a Packs link where you can monitor the throughput of a Pack.

Sample Pack

​

Each Cribl Edge deployment contains a sample Pack called HelloPacks that you can use to learn to work with Packs.

Once loaded, each Pack displays tabs labeled with familiar links: Routes, Pipelines, Knowledge, and Pack Settings in the left pane, along with Sample Data, and Simple Preview on the right.

The left pane tabs give you access to configuration objects specific to this Pack.

Select Pipelines, and you’ll see that the Pack includes devnull, main, and passthru Pipelines, corresponding to the default Pipelines provided at Cribl Edge’s global level. This Pack also includes an Apache-specific sample Pipeline – select it to unpack that, too.

Select Routes, and you’ll see that this Pack also provides both a default and an Apache-specific Route.

Cribl imposes a limit of 200 maximum Pipelines per Pack. This limit helps ensure optimal performance, and prevents potential errors. It helps maintain stability and avoids issues that can arise from managing very large Packs.

Access Packs

​

You access Packs differently, depending on your deployment type.

Single-Instance Deployment

​

In a Single-instance deployment, Packs are global. On the top bar, select Processing > Packs. On the filesystem, Packs (including those that you add) are stored at $CRIBL_HOME/default/.

Distributed Deployment

​

In a Distributed deployment, Packs are associated with (and installed within) Fleets. In the sidebar, select Fleets, and then select the Fleet you want to manage. Next, from that Fleet’ submenu, select Processing > Packs (Stream) or More > Packs (Edge).

Each Group’s Packs are stored at $CRIBL_HOME/groups/<groupName>/default/. (In a typical installation, you’ll find the starter HelloPacks Pack at /opt/cribl/groups/default/default/.)

By design, you can readily share Packs across Fleets by copying them between Groups.

Use Sample Data

​

Within a Pack, the right pane defaults to displaying all sample data files available on your Cribl Edge instance. If you prefer to filter only sample files internal to the Pack, select the In Pack only option.

If you add sample data files via a Pack UI, they will be internal to that Pack. Each sample file here displays its own In Pack toggle on its row, which works as follows:

A light-blue toggle is locked, meaning that this sample file is internal to the Pack. It will export with the Pack. If you want to make this sample available across Cribl Edge, you’ll need to also add it via the global right preview pane (accessed from Routing > Data Routes or Processing > Pipelines).

The toggle indicates whether the sample file is global to Cribl Edge and available to this Pack. Toggle on if you want the sample file to export along with the Pack.

Basically, you can manipulate all the options here as you would work with the equivalent options in the global navigation.

Upgrade an Existing Pack

​

To upgrade an existing Pack, open the Options ••• menu on its row, and then select Upgrade.

If you’ve modified an installed Pack, Cribl Edge will block the overwrite of the Pack, to prevent deletion of your locally created resources.

When upgrading a Pack, Cribl recommends that you:

• Import the updated Pack under a new name that includes the version number (for example, cribl‑syslog‑input‑120). This allows you to review and adjust new functionality against currently–deployed configurations.
• Do a side–by–side comparison of the previous and new versions of the Pack – remember to review all comments in the new Pack. Doing this side-by-side comparison allows you to copy Function expressions and other settings from the current version into the same fields in the new version.
• Enable or disable any Functions in the new Pack as necessary.
• Update any Routes, Pipelines, Sources, or Destinations that use the previous Pack version to reference the new Pack.
• If the Pack includes any user–modified versions of default Cribl Edge Knowledge objects (for example, lookup files): Be sure to copy the modified files locally for safekeeping, before upgrading the Pack. After you install the upgrade, copy those files back to the upgraded Pack, overwriting the default versions in the Pack.
• Test, test, and test!
• Commit and Deploy.

Copy a Pack

​

You can share Packs in a Distributed deployment by copying the Packs between Fleets. You can copy one or more Packs from the Packs page as described here or using the Cribl API.

To copy a single Pack, your first step is to open its Options ••• menu, then select Copy to another Fleet.

To copy multiple Packs in one operation:

1. Select the check boxes for all Packs you want to copy on the Packs page.

2. In the options ••• menu, select Copy selected Packs to another Fleet.

3. Select the Fleets to which you want to copy the Pack.

4. Optionally, select the Overwrite option to replace existing Packs that have the same Pack ID.

   Be careful about enabling the Overwrite option. Doing so will cause the imported Pack to completely overwrite your existing Pack configuration.

5. Select Copy.

A status modal will either display a success message, or list any Packs that failed to copy.

Create a Pack

​

You can create a new Pack from scratch, to consolidate and export multiple Cribl Edge configuration objects:

1. Navigate to the Packs page and select Add Pack.

2. From the submenu, select Create Pack.

3. Enter a unique Pack ID and other details:

   • Each Pack within a Fleet must have a separate Pack ID, but you can assign arbitrary Display names.
   • Version is a required field identifying the Pack’s own versioning.
   • Minimum Edge version is an optional field specifying the lowest compatible version of Cribl Edge software.
   • Description and Author are optional identifiers.
   • Data type, Use cases, and Technologies are optional combo boxes. You can insert one or multiple keywords to help users filter Packs that you post publicly on the Cribl Packs Dispensary.
   • Tags are optional, arbitrary labels that you can use to filter/search and organize Packs.

4. Select Save.

5. On the Packs page, select the new Pack’s row to open the Pack.

6. Use the standard Cribl Edge controls to configure and save the infrastructure you want to pack up. As you save changes in the UI, they’re saved to the Pack.

Pack Variables

​

To ensure that Packs are easily portable between deployments, you can use variables in selected fields.

This lets you create Packs which rely on information that differs between deployments (such as a Redis URL). On import, the user of the Pack provides values for the variables adapted to their own configuration.

You define variables per Pack and they are scoped to the Pack. This means that when you have two variables with the same name, one in a Pack, and one in your deployment’s global variables, those variables will not conflict and you can set distinct values for each of them.

Pack variables are bound to the fields they were added to. This means that, if you change the value of the variable, the field value will update accordingly. However, if you delete a variable, the field will retain its last configured value.

To add a variable to a Pack:

1. Select Variable next to the field you want to configure a variable for.
2. In the Variables modal, either choose an existing variable or select Add Variable.
3. If you are adding a new variable, in the New Variable modal, enter the name for the variable and its value for your current deployment, then save.
4. Confirm the use of the variable for the field by selecting Add.

Now, when another user imports your Pack, they will be prompted to provide their own values for all the variables you configured.

Secrets such as passwords are removed from the variables on export. They will be replaced with a changeme value, prompting users to provide the value for their own deployment.

Modify Pack Settings

​

You can update Pack metadata (such as Version, Description, and Author) and the Pack’s display settings. If you’re developing a new Pack to share, you’ll want to use this interface to populate the README and display logo within the Pack:

1. Open a Pack and select Pack Settings. The left README tab will gain focus.
2. To populate the Pack README file, toggle View to Edit, replace the placeholder markdown content with your actual Pack description, and Save.

3. To update other metadata, select the left Settings tab.

4. You can add a Pack logo on the Settings tab.

   Cribl recommends adding a logo to each custom Pack, to visually distinguish the Pack UI from the surrounding Cribl Edge UI (as well as from other Packs). You can upload a .png or .jpg/.jpeg file, up to a maximum size of 2 MB and 350x350 px. Cribl recommends a transparent image, sized approximately 280x50 px.

Share and Publish Packs

​

To share your Pack with other users, you first need to export it from your deployment.

Then, if you wish, you can publish it to the Cribl Pack Dispensary to make it available for the whole Cribl ecosystem.

Export a Pack

​

You can also use the Cribl API to export a Pack.

To export a newly created or modified Pack from the Packs page, open its Options ••• menu and select Export.

The Export Pack modal provides the following options.

Export Mode

​

Select one of these three buttons to determine the export mode:

• Merge safe: (Deprecated) Attempt to safely merge local modifications into the Pack’s default layer (original configuration), then export.

• Merge: Force-merge local modifications into the Pack’s original configuration, then export. When you export Packs in merge mode, Cribl Edge deletes all encrypted fields from Pipelines. Upon importing these packs, you will need to re-enter the secrets to make them functional again.

• Default only: Export only the Pack’s original configuration, without local modifications.

The (deprecated) Merge safe option is conservative, and will block the export where Cribl Edge can’t readily merge modified contents with the Pack’s original contents, due to conflicts. If you encounter an error like the example shown below, use the Merge or Default only export mode instead.

Exported Pack ID

​

Defaults to the Pack’s current ID, with the version number appended. This field provides an opportunity to change the Pack’s ID, if you’re exporting a new version of the Pack.

Publish a Pack

​

If you’d like to share your Pack with the community of Cribl users, you can publish it on the Cribl Packs Dispensary.

The Cribl Packs Dispensary site is designed for sharing completed Packs. If you want to collaborate with others on iteratively developing a Pack, Cribl recommends relying on our Dispensary GitHub Repo for the development phase.

If you have a Cribl.Cloud account, you can also collaborate there by inviting team members to your Cribl.Cloud Organization. See Inviting and Managing Other Users.

Once your Pack is ready to share, we encourage you to submit it to the Cribl Packs Dispensary site. If you already have completed Packs on our GitHub repo, bring them over here!

Change Pack Owner

​

If you no longer can or wish to maintain a Pack you published to the Cribl Packs Dispensary, you can transfer its ownership to another user. The new owner will have full control over the Pack.

To transfer a Pack to a new owner:

1. Log in to the Cribl Packs Dispensary.
2. Optionally, toggle View only my Packs to Yes to narrow down the list to the Packs you own.
3. Select Configure on your Pack’s tile.
4. At the bottom of the Pack pane, select Change Owner.
5. Enter the email address of the new owner and, optionally, a message you want to send them.
6. Confirm by selecting Change Owner.

The recipient will get an email message with a request to accept ownership of the Pack. They will have 24 hours to respond.

When the new owner either accepts or rejects the request, you will be notified by email. Until the new owner responds, you can withdraw the request. To do it, open the Pack again and select Cancel Ownership Request.

Supported Pack Integrations

​

While most Sources and Destinations are available in Packs, there is a small subset that you can’t include inside a Pack.

Unsupported Sources

​

The following Sources are not supported inside Packs for Cribl Stream:

• Cribl Internal
• System Metrics
• System State
• Collectors:
  • Azure Blob
  • Database
  • File System
  • Google Cloud Storage
  • Health Check
  • REST
  • S3
  • Script
  • Splunk Search
• Collector-based Sources:
  • Office 365 Services
  • Office 365 Activity
  • Office 365 Message Trace
  • Prometheus Scraper
  • Splunk Search
  • Wiz

The following Sources are not supported inside Packs for Cribl Edge:

• Cribl Internal
• K8s Metrics
• K8s Logs
• K8s Events
• System Metrics
• Windows Metrics
• System State
• Windows Event Logs

Unsupported Destinations

​

The following Destinations are not supported inside Packs:

• Amazon Security Lake
• Cribl Lake