LogStream's Sample Data Preview features enable you to visually inspect events as they flow into and out of a Pipeline. Preview helps you shape and control events before they're delivered to a Destination, and helps you troubleshoot Pipeline Functions.
Preview works by taking a set of sample events and passing them through the Pipeline, while displaying the inbound and outbound results in a separate pane. Any time a Function is modified, added, or removed, the Pipeline changes, and so does its displayed output.
While you're in a Pipeline, you can add samples through one of the supported options: Paste, Attach, or Capture New. The Paste and Attach options work with content that needs to be broken into events, while the Capture New option works with events only.
When you click on the corresponding option, you'll be presented with a modal like the one shown below.
This is where the content of the paste (or uploaded file) is displayed.
An Event Breaker is a regular expression that tells Cribl LogStream how to break the file or pasted content into events. Breaking will occur at the start of the match. Cribl LogStream ships with several common breaker patterns out of the box, but you can also configure custom breakers. The UI here is interactive, and you can iterate until you find the exact pattern.
The Capture New button opens a slightly different modal – it does not require event breaking. In the composite screenshot below, we've already captured some events using the Capture drop-down.
To capture data from a single enabled Source or Destination, it's fastest to use the Sources or Destinations UI instead of the Preview pane. You can initiate an immediate capture by clicking the Live button on the Source's or Destination's configuration row.
You can similarly start an immediate capture from within an enabled Source's or Destination's configuration modal, by clicking the modal's Live Data tab.
Tp prevent in-memory samples from getting unreasonably large, samples input by any means (Capture/Live Data, Attach;/upload, or Paste) are constrained by a limit set at global ⚙️ Settings (lower left) > General Settings > Limits > Max sample size. The default limit is
256KB, and you can adjust this upward or downward,
In the Capture Sample Data and Live Data modals, use the Fields sidebar (at left) to streamline how events are displayed. You can toggle among All fields, None (to reset the display), and check boxes that enable/disable individual fields by name.
Within the right Preview pane, each field's type is indicated by one of these leading symbols:
On JSON objects and arrays, you'll also see:
The Preview pane's Add Sample Data or Capture Sample Data modal, once you've successfully populated it with data, provides options to save the data as a sample and/or datagen file. Click the appropriate button, accept or modify the default/generated file name and other options, and confirm the save.
The Preview pane offers two display options for events: Event and Table. (You can also download data as JSON or NDJSON, using the Advanced Settings > Save submenu from the top right.) Each format can be useful, depending on the type of data you are previewing.
The Advanced Settings > CPU Profiling submenu (accessible from the top right) offers Timeout (sec) and Memory (MB) limits. You can increase these controls' defaults to adjust for cases where very large data samples fail to load. For example, you might increase the Timeout (sec) to
30 and the Memory (MB) to
As you add more samples to your system, you can easily access them via the Sample data file drop-down.
You can also manage and modify sample files via the Samples tab highlighted below.
Click Simple or Full beside a file name to display its events in the Preview pane. The Preview Simple option enables you to view events on either the IN or the OUT (processed) side of a single Pipeline.
The Preview Full option gives you a choice of viewing events on the OUT side of either the processing or post-processing Pipeline. Selecting this option expands the Preview pane's upper controls to include an Exit Point drop-down, where you make this choice.
Modifying Sample File Details
With the Preview pane's Sample Data tab selected, click directly on a file name to open the modal shown here, with options to clone the sample, save it as a datagen Source, delete it, associate it with a Pipeline, and set a description, expiration time, and tags.
As data traverses Functions in a Pipeline, events can be modified, and some might be dropped altogether. The OUT tab indicates changes using this color coding:
Dropped events: When events are dropped, the OUT tab displays them as grayed-out text, with strikethrough. You can control their display using the Advanced Settings menu's Show Dropped Events slider.
Added fields: When LogStream's processing adds new fields, these fields are highlighted green. You can control these fields' display using the Select Fields drop-down.
Redacted fields: These fields are highlighted amber.
Deleted fields: These fields are highlighted red.
With the Routes or Pipelines page displayed in the left pane, hover over the pane divider (in the headers row) to display the Collapse/Expand toggle shown in the composite screenshot below.
Click Collapse to hide the Preview pane. This allows the Route or Pipeline configuration to expand to your browser's full width. (The Preview pane collapses automatically on narrow viewports.)
Click Expand at your browser's right edge to restore the split view. The pane divider will snap back to wherever you last dragged it.
Updated about a month ago