These docs are for Cribl Stream 4.5 and are no longer actively maintained.

See the latest version (4.13).

Prometheus Publisher (deprecated)

This Function was deprecated as of Cribl Stream 3.0, and has been removed from Cribl Stream as of v.4.0. Please instead use the Prometheus Destination to send metrics to Prometheus-compatible endpoints.

The Prometheus Publisher Function allows for metrics to be published to a Prometheus-compatible metrics endpoint. These can be upstream metrics received by Cribl Stream, or metrics derived from the output of Cribl Stream’s Publish Metrics or Aggregation Functions. A Prometheus instance is responsible for collecting the metrics at that endpoint, and for performing its own processing of the metric data.

In the current Cribl Stream version, the endpoint is: http://<worker_node_IP>:<api-port>/metrics. Within Cribl Stream, that endpoint redirects from http://<worker_node_IP>:9000/metrics to http://<worker_node_IP>:9000/api/v1/metrics.

If used, this Function must follow any Publish Metrics or Aggregations Functions within the same Pipeline. This is to ensure that any data not originating from a metrics input is transformed into metrics format.

Usage

​

Filter: Filter expression (JS) that selects data to feed through the Function. Defaults to true, meaning it evaluates all events.

Description: Simple description of this Function. Defaults to empty.

Final: If toggled to Yes, stops feeding data to the downstream Functions. Defaults to No.

Fields to publish: Wildcard list of fields to publish to the Prometheus endpoint.

Advanced Settings

​

Batch write interval: How often, in milliseconds, the contents should be published. Defaults to 5000.

Passthrough mode: If set to No (the default), overrides the Final setting, and suppresses output to downstream Functions’ Destinations. Toggle to Yes to allow events to flow to consumers beyond the Prometheus endpoint. In effect, when previewing the pipeline output what you’ll see is your event fields will have strikethrough font applied to them. This does not mean the Prometheus function is not matching your events but rather indicative of the Passthrough being disabled.

Update mode: On the default No setting, suppresses output to downstream Functions’ Destinations. (This overrides the Final setting.) Toggle to Yes to allow events to flow to consumers beyond the Prometheus endpoint.

Example

​

This example uses the same PANOS sample data as the Publish Metrics Function, and is similarly preceded in a Pipeline by a Parser Function that extracts fields from the PANOS log.

Filter: Set as appropriate. Fields to publish: Set as appropriate. We’ll use the default of * for this example. Advanced settings: Accept defaults.

After committing and deploying changes, you should be able to use a curl command (-L needed to follow the redirect mentioned above) to verify that metrics are being published, just a few seconds after data is ingested on an idle system.

$ curl -L http://<worker_node_IP>:9000/metrics
# TYPE perf_192_168_1_248_bytes_sent counter
metric_192_168_1_248_bytes_sent {destination_ip="160.177.222.249",inbound_interface="ethernet1/2",destination_port="443"} 60

# TYPE perf_192_168_1_248_bytes_rcvd counter
metric_192_168_1_248_bytes_rcvd {destination_ip="160.177.222.249",inbound_interface="ethernet1/2",destination_port="443"} 0

# TYPE perf_192_168_1_248_pkts_sent counter
metric_192_168_1_248_pkts_sent {destination_ip="160.177.222.249",inbound_interface="ethernet1/2",destination_port="443"} 1

# TYPE perf_192_168_1_248_pkts_rcvd counter
metric_192_168_1_248_pkts_rcvd {destination_ip="160.177.222.249",inbound_interface="ethernet1/2",destination_port="443"} 0

Now, we need to have Prometheus scrape the metrics. In this very basic example, you can add the target endpoint to the prometheus.yml file, under the scrape_configs ‑> static_configs section. Specify the endpoint in IP:port syntax, because Prometheus assumes (and requires) /metrics for all endpoints.

Restart Prometheus. Within just a few seconds, you should be able to use its query interface to retrieve metrics published by Cribl Stream.