Journal Files Source

The Journal Files Source collects data from systemd’s centralized logging, which is called the journald service. This service is a centralized location for all messages logged by different components in a systemd-enabled Linux system. This includes messages from kernel, boot, syslog, or other services. This Source is currently configured to configured to collect data from the user journal, as system is privileged. This Source is disabled by default.

Type: System | TLS Support: N/A | Event Breaker Support: No

The Journal Files Source is available in Cribl.Cloud on customer-managed hybrid Workers, but not on Cribl-managed Workers in Cribl.Cloud.

Discovering and Filtering Journal Files to Monitor

​

To determine all the names of all journals to read, the Journal Files Source runs a discovery procedure at a configurable Polling interval. The Source then applies an Allowed list to filter the initial list down into its final form. Then, for each journal on the list, the Source compares current state with previously stored state. This comparison determines whether the Journal Files Source will actually stream a given journal for a given polling interval. The Source then runs the files through the Filter Rules which allows you to set additional criteria.

Cribl Edge ships with a single, disabled instance of the Journal Files Source with a default configuration. The search path is configured to point to $CRIBL_EDGE_FS_ROOT/var/log/journal/$MACHINE_ID and defaults to streaming the primary system.journal it finds there. $MACHINE_ID is an environment variable that you can insert into the Source’s search path. If you use the environment variable but don’t assign a value, the Source will look up the local host machine id from /etc/machine-id and use that value.

Currently, this Source does not support all of the compression options that the journald service supports. The parser this Source uses to read the files can handle LZ4 and ZSTD compression. When the parser encounters compressed fields it can’t unpack, an error message emits in the logs. Below is an example of the error message:

Error reading journald event, dropping offset=31678152 evt="Mar 02 21:29:54 goats-xps15 multipassd: undefined"

Also, this Source does not support the new COMPACT binary file format. The logs will contain an error message when the parser encounters this type of file.

Configure a Journal Files Source

​

1. On the top bar, select Products, and then select Cribl Edge. Under Fleets, select a Fleet. Next, you have two options:
   • To configure via QuickConnect, navigate to Routing > QuickConnect (Stream) or Collect (Edge). Select Add Source and select the Source you want from the list, choosing either Select Existing or Add New.
   • To configure via the Routes, select Data > Sources (Stream) or More > Sources (Edge). Select the Source you want. Next, select Add Source.
2. In the New Source modal, configure the following under General Settings:
   • Enabled: Toggle on to enable the Source.
   • Input ID: Enter a unique name to identify this Source configuration. If you clone this Source, Cribl Edge will add -CLONE to the original Input ID.
   • Description: Optionally, enter a description.
   • Search path: Specify the directory to search journals on. Accepts environment variables. The default value for this field varies depending on the product:
   • In Cribl Edge, it defaults to $CRIBL_EDGE_FS_ROOT/var/log/journal/$MACHINE_ID.
   • In Cribl Stream, it defaults to /var/log/journal/$MACHINE_ID.
   • Journal allowlist: The full path of each discovered journal is matched against this wildcard list. Defaults to system.
3. Next, you can configure the following Optional Settings:
   • Polling interval: How often, in seconds, to collect metrics. If not specified, defaults to 10 seconds.
   • Filter Rules: Optionally, specify which journal objects to allow. Events are generated if all the rules’ expressions evaluate to true, or if no rules are specified.
   • Filter expression: JavaScript expression to filter Journal objects. Returns true to include it. For details, see Filter Expressions below.
   • Description: Optional description of the rule.
   • Current boot only: Skip events that are not part of the current boot session.
   • Age duration limit: Optionally, specify a maximum age of journal objects to stream. This Source will filter events with timestamps newer than the configured duration. Enter a duration in a format like 30s, 4h, 3d, or 1w. Defaults to an empty field, which applies no age filters.
   • Tags: Optionally, add tags that you can use to filter and group Sources in Cribl Edge’s UI. These tags aren’t added to processed events. Use a tab or hard return between (arbitrary) tag names.
4. Optionally, you can adjust the Processing and Advanced settings, or Connected Destinations outlined in the sections below.
5. Select Save, then Commit & Deploy.

Filter Expressions

​

The default Filter expression is severity <= 4 with a Description of Allow events having 'emergency', 'alert', 'critical', 'error', or 'warning' priority.

The screenshot below shows other examples you can use:

Processing Settings

​

Fields

​

In this section, you can define new fields or modify existing ones using JavaScript expressions, similar to the Eval function.

• The Field Name can either be a new field (unique within the event) or an existing field name to modify its value.
• The Value is a JavaScript expression (enclosed in quotes or backticks) to compute the field’s value (can be a constant). Select this field’s advanced mode icon (far right) if you’d like to open a modal where you can work with sample data and iterate on results.

This flexibility means you can:

• Add new fields to enrich the event.
• Modify existing fields by overwriting their values.
• Compute logic or transformations using JavaScript expressions.

Pre-Processing

​

In this section’s Pipeline drop-down list, you can select a single existing Pipeline or Pack to process data from this input before the data is sent through the Routes.

Advanced Settings {advanced-tab}

​

Environment: If you’re using GitOps, optionally use this field to specify a single Git branch on which to enable this configuration. If empty, the config will be enabled everywhere.

Connected Destinations

​

Select Send to Routes to enable conditional routing, filtering, and cloning of this Source’s data via the Routing table.

Select QuickConnect to send this Source’s data to one or more Destinations via independent, direct connections.

In Cribl Edge, this Source defaults to QuickConnect. While, in Cribl Stream, it defaults to Send to Routes.