Skip to main content
Version: 3.2

Diagnosing Issues

To help diagnose LogStream problems, you can share a diagnostic bundle with Cribl Support. The bundle contains a snapshot of configuration files and logs at the time the bundle was created, and gives troubleshooters insights into how LogStream was configured and operating at that time.

What's in the Diagnostic Bundle

The following subdirectories (and their contents) of $CRIBL_HOME are included:

  • /default/*
  • /local/* – except for /local/cribl/auth/, to exclude sensitive files.
  • /log/*
  • /groups/*
  • /state/jobs/* – includes only the latest 10 task from the latest 10 jobs.

As a security measure, the bundle excludes all .crt, .pem, .cer, and .key files from all $CRIBL_HOME subdirectories.

Creating and Exporting a Diagnostic Bundle

If you're managing your own LogStream deployment (single-instance or distributed), you can create and securely share bundles with Cribl Support either from the UI or from the CLI. In either case, you'll need outbound internet access to https://diag‑upload.cribl.io and a valid Support Case number. That site works only when using the cribl diag command or uploading using the LogStream UI. (So connecting directly to it with your web browser will fail.)

With a LogStream Cloud deployment, contact Cribl Support to gather a diag bundle on your behalf.

Using the UI

To create a bundle, go to global ⚙️ Settings (lower left) > Diagnostics > Diagnostic Bundle and click Create Diagnostic Bundle.

  • To download the bundle locally to your machine, click Export.
  • To share the bundle with Cribl Support, toggle Send to Cribl Support to Yes, enter your case number, and then click Export.

You can create a bundle from individual workers if you have the Worker UI access setting enabled. Go to Workers > <worker-name> > Settings (top right) > Diagnostics > Diagnostic Bundle, and click Create Diagnostic Bundle.

Previously created bundles are stored in $CRIBL_HOME/diag. They're also listed in the UI, where you can re-download them or share them with Cribl Support.

Using the CLI

To create a bundle using the CLI, use the diag command.

# $CRIBL_HOME/bin/cribl diag
Usage: [sub-command] [options] [args]

Commands:
get - List existing Cribl LogStream diagnostic bundles
create - Creates diagnostic bundle for Cribl LogStream
send - Send LogStream diagnostic bundle to Cribl Support, args:
-c <caseNumber> - Cribl Case Number
[-p <path>] - Diagnostic bundle path (if empty, then new bundle will be created)


## Creating a diagnostic bundle
# $CRIBL_HOME/bin/cribl diag create
Created Cribl LogStream diagnostic bundle at /opt/cribl/diag/cribl-logstream-<hostname>-<datetime>.tar.gz.

## Creating and sending a diagnostic bundle
# $CRIBL_HOME/bin/cribl diag send -c 420420
Sent LogStream diagnostic bundle to Cribl Support

## Sending a previously created diagnostic bundle
# $CRIBL_HOME/bin/cribl diag send -p /opt/cribl/diag/cribl-logstream-<hostname>-<datetime>.tar.gz -c 420420
Sent LogStream diagnostic bundle to Cribl Support

Including CPU Profiles

If Cribl Support asks you to grab CPU profiles of Worker Processes, follow these steps:

  1. Use top or htop on the Worker Node to identify Worker PIDs consuming a lot of CPU.
  2. See Sizing & Scaling > CPU Profiling for instructions on accessing the UI's Profile options (for your deployment type), and generating and saving profiles.
  3. Find the Worker Processes matching the PIDs you identified above.
  4. Click Profile on each. Start with the default 10-second Duration.
  5. Once the profile is displayed, save it to a JSON file. (See details at the above link.)
  6. Repeat steps 3–6 for other CPU-intensive Worker Processes.
  7. Upload the profile JSON files to Cribl Support.

On an already CPU-starved Worker Node, profiling might fail with an error message, or just hang. In this case, you might need a few retries to get a successful profile.