These docs are for Cribl Stream 3.1, a product version we no longer actively maintain.
See the latest version (4.5).
Diagnosing Issues
To help diagnose LogStream problems, you can share a diagnostic bundle with Cribl Support. The bundle contains a snapshot of configuration files and logs at the time the bundle was created, and gives troubleshooters insights into how LogStream was configured and operating at that time.
What’s in the Diagnostic Bundle
The following directories (and their contents) off of $CRIBL_HOME
are included:
/default/*
/local/*
/log/*
/groups/*
/state/jobs/*
– will return all jobs if left empty.
Creating and Exporting a Diagnostic Bundle
Users can create and securely share bundles with Cribl Support either from the UI or from the CLI.
In either case, you’ll need outbound internet access to https://diag‑upload.cribl.io and a valid support case number. That site only works when using the cribl diag
command or uploading using the LogStream UI (i.e. connecting directly to it with your web browser will fail).
Using the UI
To create a bundle, go to global ⚙️ Settings (lower left) > Diagnostics > Diagnostic Bundle and click Create Diagnostic Bundle.
- To download the bundle locally to your machine, click Export.
- To share the bundle with Cribl Support, toggle Send to Cribl Support to Yes, enter your case number, and then click Export.
You can create a bundle from individual workers if you have the Worker UI access setting enabled. Go to Workers > <worker-name> > Settings (top right) > Diagnostics > Diagnostic Bundle, and click ** Create Diagnostic Bundle**.
Previously created bundles are stored in $CRIBL_HOME/diag
. They’re also listed in the UI, where you can re-download them or share them with Cribl Support.
Using the CLI
To create a bundle using the CLI, use the diag
command.
Open your support case before sending a diagnostic bundle to Cribl Support. Then, when creating the bundle, be sure to use the case number provided. Case numbers are 8 digits in length and you must include any leading zeros. For example,
00001234
.
# $CRIBL_HOME/bin/cribl diag
Usage: [sub-command] [options] [args]
Commands:
get - List existing Cribl LogStream diagnostic bundles
create - Creates diagnostic bundle for Cribl LogStream
send - Send LogStream diagnostic bundle to Cribl Support, args:
-c <caseNumber> - Cribl Case Number
[-p <path>] - Diagnostic bundle path (if empty, then new bundle will be created)
## Creating a diagnostic bundle
# $CRIBL_HOME/bin/cribl diag create
Created Cribl LogStream diagnostic bundle at /opt/cribl/diag/cribl-logstream-<hostname>-<datetime>.tar.gz.
## Creating and sending a diagnostic bundle
# $CRIBL_HOME/bin/cribl diag send -c <caseNumber>
Sent LogStream diagnostic bundle to Cribl Support
## Sending a previously created diagnostic bundle
# $CRIBL_HOME/bin/cribl diag send -p /opt/cribl/diag/cribl-logstream-<hostname>-<datetime>.tar.gz -c <caseNumber>
Sent LogStream diagnostic bundle to Cribl Support
Including CPU Profiles
If Cribl Support asks you to grab CPU profiles of Worker Processes, follow these steps:
- Use
top
orhtop
on the Worker Node to identify Worker PIDs consuming a lot of CPU. - See Sizing & Scaling > CPU Profiling for instructions on accessing the UI’s Profile options (for your deployment type), and generating and saving profiles.
- Find the Worker Processes matching the PIDs you identified above.
- Click Profile on each. Start with the default 10-second Duration.
- Once the profile is displayed, save it to a JSON file. (See details at the above link.)
- Repeat steps 3–6 for other CPU-intensive Worker Processes.
- Upload the profile JSON files to Cribl Support.
On an already CPU-starved Worker Node, profiling might fail with an error message, or just hang. In this case, you might need a few retries to get a successful profile.