Sampling
The Sampling Function filters out events, based on an expression and a sampling rate.
Each Worker Process executes this Function independently on its share of events. For details, see Functions and Shared-Nothing Architecture.
Usage
Filter: Filter expression (JavaScript) that selects data to feed through the Function. Defaults to true, meaning it evaluates all events.
Description: Simple description of this Function. Defaults to empty.
Final: Toggle on to stop feeding data to the downstream Functions. Default is toggled off.
Sampling rules: Events matching these rules will be sampled at the rates you specify:
Filter: Filter expression matching events to be sampled. Use
trueto match all.Sampling Rate: Enter an integer
N. (Defaults to1.) Sampling will pick 1/Nevents matching this rule.
How It Works
Setting this Function’s Sampling rate to 30 would mean that only 1 of every 30 events would be kept.
Let’s assume that we save this setting, and then capture data from a datagen Source by selecting Preview > Start a Capture > Capture. In the Capture Sample Data modal, select: 100 seconds, 100 events, and As they come in. Then start the capture, and Save as Sample File.
Next, in the Preview pane, select Simple beside the new file’s name. If you then select the Basic Statistics (chart) button, you should see that we’ve kept about 4 of the original 100 events, or close to 1 in 30.
Examples and Scenarios
For usage examples, see these Better Practices topics:
- Sampling: Sampling verbose or voluminous data at ingest time, to make analysis and troubleshooting more efficient.
- Sample Logs: Sampling event data for efficient onbaording into Cribl Stream.
- Access Logs: Apache, ELB, CDN, S3, Etc.: Sampling voluminous logs from sources like Amazon S3, Amazon Cloudfront, and AWS ELBs.
- Firewall Logs: VPC Flow Logs, Cisco ASA, Etc.: Sampling voluminous logs from firewalls.
See Also
- Dynamic Sampling Function: Filter out events based on an expression, a sample mode, and the volume of events