Sources

Each Cribl Edge Source is a configuration that enables Edge nodes to collect or receive observability data – logs, metrics, application data, etc. – in real time. Edge can receive continuous data input from Splunk, HTTP senders, Elastic Beats, Prometheus, TCP JSON, and many others. Sources can receive data from either IPv4 or IPv6 addresses.

Edge’s UI offers a configuration modal for each type of supported Source. However, you can add multiple instances of each Source type – with each configured to match the parameters of the corresponding sender. E.g., you can have multiple File Monitors and multiple listeners for Syslog, Splunk, Elastic Beats, Prometheus, TCP JSON, and many others.

System and Internal Sources​

Sources that generate data locally at the Edge Node; or monitor resources; or move data among Edge Nodes and/or Stream Workers within your Cribl deployment.

• AppScope
• Cribl Internal
• Cribl HTTP
• Cribl TCP
• Datagen
• Exec
• File Monitor
• Kubernetes Logs
• Kubernetes Metrics
• System Metrics
• System State
• Windows Metrics

PUSH Sources​

Supported data Sources that Cribl Edge fetches data from.

These Sources can continue ingesting data even if no Leader is active:

• Amazon Kinesis Firehose
• Datadog Agent
• Elasticsearch API
• Grafana
• HTTP/S (Bulk API)
• Loki
• Metrics
• Raw HTTP/S
• OpenTelemetry (OTel)
• Prometheus Remote Write
• SNMP Trap
• Splunk HEC
• Splunk TCP
• Syslog
• TCP JSON
• TCP (Raw)
• UDP (Raw)
• Windows Event Forwarder

PULL Sources​

Supported data Sources that Cribl Edge fetches data from.

These Sources can ingest data only if a Leader is active:

• Prometheus Edge Scraper
• Prometheus Scraper (Deprecated)
• Windows Event Logs

Configuring and Managing Sources​

For each Source type, you can create multiple definitions, depending on your requirements.

To configure Sources, from the top nav, click Manage, then select a Fleet to configure. Then, you have two options:

• To access the graphical QuickConnect UI, click Collect. Next, click either Add New or (if displayed) Select Existing.

• To access the Routing UI, click More > Sources. On the resulting Data Sources page’s tiles or left menu, select the desired type, then click Add New.

Capturing Source Data​

To capture data from a single enabled Source, you can bypass the Preview pane, and instead capture directly from a Manage Sources page. Just click the Live button beside the Source you want to capture.

In order to capture live data, you must have Edge Nodes registered to the Fleet for which you’re viewing events. You can view registered Edge Nodes from the Status tab in the Source.

You can also start an immediate capture from within an enabled Source’s configuration modal, by clicking the modal’s Live Data tab.

Monitoring Source Status​

Each Source’s configuration modal offers two tabs for monitoring: Status and Charts.

Status Tab​

The Status tab provides details about the Edge Nodes in the Fleet and their status. An icon shows whether the Edge Node is operating normally.

You can click each Edge Node’s row to see specific information, for example, to identify issues when the Source displays an error. The specific set of information provided depends on the Source type. The data represents only process 0 for each Edge Node.

The content of the Status tab is loaded live when you open it and only displayed when all the data is ready. With a lot of busy Edge Nodes in a group, or nodes located far from the Leader, there may be a delay before you see any information.

The statistics presented are reset when the Edge Node restarts.

Charts Tab​

The Charts tab presents a visualization of the recent activity on the Source. The following data is available:

• Events in
• Thruput in (events per second)
• Bytes in
• Thruput in (bytes per second)

This data (in contrast with the status tab) is read almost instantly and does not reset when restarting an Edge Node.

Preconfigured Sources​

To accelerate your setup, Cribl Edge ships with several common Sources configured but not switched on. Open, clone (if desired), modify, and enable any of these preconfigured Sources to get started quickly:

• System Metrics – Basic Level
• File Monitor > in_file_auto – Auto Discovery Mode
• File Monitor > in_file_varlog – Manual Discovery Mode
• AppScope > in_appscope – Unix Domain Socket listener
• Cribl Internal > CriblLogs – Internal
• Cribl Internal > CriblMetrics – Internal

Cribl University offers a course titled Collecting Data in Edge that provides an illustrated overview. To follow the direct course link, first log into your Cribl University account. (To create an account, click the Sign up link. You’ll need to click through a short Terms & Conditions presentation, with chill music, before proceeding to courses – but Cribl’s training is always free of charge.) Once logged in, check out other useful Cribl Edge courses.