Cribl LogStream ‚Äď Docs

Cribl LogStream Documentation

Questions? We'd love to help you! Meet us in #Cribl Community Slack (sign up here)
Download entire manual as PDF ‚Äď v.3.1.1

Data Preview

LogStream's Sample Data Preview features enable you to visually inspect events as they flow into and out of a Pipeline. Preview helps you shape and control events before they're delivered to a Destination, and helps you troubleshoot Pipeline Functions.

Preview works by taking a set of sample events and passing them through the Pipeline, while displaying the inbound and outbound results in a separate pane. Any time a Function is modified, added, or removed, the Pipeline changes, and so does its displayed output.

Preview optionsPreview options

Preview options

While you're in a Pipeline, you can add samples through one of the supported options: Paste, Attach, or Capture New. The Paste and Attach options work with content that needs to be broken into events, while the Capture New option works with events only.

Adding Sample Data (Using Paste as an Example)

When you click on the corresponding option, you'll be presented with a modal like the one shown below.

Add Sample Data modalAdd Sample Data modal

Add Sample Data modal

Paste Area

This is where the content of the paste (or uploaded file) is displayed.

Event Breaker Settings

An Event Breaker is a regular expression that tells Cribl LogStream how to break the file or pasted content into events. Breaking will occur at the start of the match. Cribl LogStream ships with several common breaker patterns out of the box, but you can also configure custom breakers. The UI here is interactive, and you can iterate until you find the exact pattern.

Capturing Sample Data

The Capture New button opens a slightly different modal ‚Äď it does not require event breaking. In¬†the composite screenshot below, we've already captured some events using the Capture drop-down.

Capture New > Capture Sample Data modalCapture New > Capture Sample Data modal

Capture New > Capture Sample Data modal

Capturing from a Single Source or Destination

To capture data from a single enabled Source or Destination, it's fastest to use the Sources or Destinations UI instead of the Preview pane. You can initiate an immediate capture by clicking the Live button on the Source's or Destination's configuration row.

Source > Live buttonSource > Live button

Source > Live button

You can similarly start an immediate capture from within an enabled Source's or Destination's configuration modal, by clicking the modal's Live Data tab.

Destination modal > Live Data tabDestination modal > Live Data tab

Destination modal > Live Data tab

Controlling Sample Size

Tp prevent in-memory samples from getting unreasonably large, samples input by any means (Capture/Live Data, Attach;/upload, or Paste) are constrained by a limit set at global ‚öôÔłŹ¬†Settings (lower left)¬†> General¬†Settings > Limits > Max¬†sample¬†size. The default limit is 256KB, and you can adjust this upward or downward,

Fields

In the Capture Sample Data and Live Data modals, use the Fields sidebar (at left) to streamline how events are displayed. You can toggle among All fields, None (to reset the display), and check boxes that enable/disable individual fields by name.

Field Type Symbols

Within the right Preview pane, each field's type is indicated by one of these leading symbols:

SymbolMeaning
őĪstring
#numeric
bboolean
mmetric
{}JSON object
[]array

On JSON objects and arrays, you'll also see:

SymbolMeaning
+expandable
-collapsible

Saving Sample Data

The Preview pane's Add Sample Data or Capture Sample Data modal, once you've successfully populated it with data, provides options to save the data as a sample and/or datagen file. Click the appropriate button, accept or modify the default/generated file name and other options, and confirm the save.

Saving sample dataSaving sample data

Saving sample data

IN Tab: Displaying Samples on the Way IN to the Pipeline

The Preview pane offers two display options for events: Event and Table. (You can also download data as JSON or NDJSON, using the Advanced Settings > Save submenu from the top right.) Each format can be useful, depending on the type of data you are previewing.

Event, Table, and Advanced options (composite screenshot)Event, Table, and Advanced options (composite screenshot)

Event, Table, and Advanced options (composite screenshot)

CPU Profiling

The Advanced Settings > CPU Profiling submenu (accessible from the top right) offers Timeout (sec) and Memory (MB) limits. You can increase these controls' defaults to adjust for cases where very large data samples fail to load. For example, you might increase the Timeout (sec) to 30 and the Memory (MB) to 3048.

Accessing and Managing Data Files

As you add more samples to your system, you can easily access them via the Sample data file drop-down.

Selecting an existing sampleSelecting an existing sample

Selecting an existing sample

You can also manage and modify sample files via the Samples tab highlighted below.

Managing sample filesManaging sample files

Managing sample files

Simple Versus Full Preview

Click Simple or Full beside a file name to display its events in the Preview pane. The Preview Simple option enables you to view events on either the IN or the OUT (processed) side of a single Pipeline.

Preview Simple schematicPreview Simple schematic

Preview Simple schematic

The Preview Full option gives you a choice of viewing events on the OUT side of either the processing or post-processing Pipeline. Selecting this option expands the Preview pane's upper controls to include an Exit Point drop-down, where you make this choice.

Preview Full schematicPreview Full schematic

Preview Full schematic

Modifying Sample File Details

With the Preview pane's Sample Data tab selected, click directly on a file name to open the modal shown here, with options to clone the sample, save it as a datagen Source, delete it, associate it with a Pipeline, and set a description, expiration time, and tags.

Options for modifying a sampleOptions for modifying a sample

Options for modifying a sample

OUT Tab: Displaying Samples on the Way OUT of the Pipeline

As data traverses Functions in a Pipeline, events can be modified, and some might be dropped altogether. The OUT tab indicates changes using this color coding:

  • Dropped events: When events are dropped, the OUT tab displays them as grayed-out text, with strikethrough. You can control their display using the Advanced¬†Settings menu's Show¬†Dropped Events slider.

  • Added fields: When LogStream's processing adds new fields, these fields are highlighted green. You can control these fields' display using the Select Fields drop-down.

  • Redacted fields: These fields are highlighted amber.

  • Deleted fields: These fields are highlighted red.

Dropped and added fields in a Pipeline's outputDropped and added fields in a Pipeline's output

Dropped and added fields in a Pipeline's output

Managing the Preview Pane

With the Routes or Pipelines page displayed in the left pane, hover over the pane divider (in the headers row) to display the Collapse/Expand toggle shown in the composite screenshot below.

Collapse / Expand toggle (composite)Collapse / Expand toggle (composite)

Collapse / Expand toggle (composite)

Click Collapse to hide the Preview pane. This allows the Route or Pipeline configuration to expand to your browser's full width. (The Preview pane collapses automatically on narrow viewports.)

Click Expand at your browser's right edge to restore the split view. The pane divider will snap back to wherever you last dragged it.

Updated about a month ago

Data Preview


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.