Metrics Reference

Reference for the metrics available in the custom Monitor query builder.

When you create a custom Monitor in Cribl Insights, the Metric dropdown lists the base metrics you can evaluate. This page is a reference for those metric names, grouped by category, with each metric type, unit, and definition.

Why Your Available Metrics May Differ

The Metric dropdown only shows metrics that have landed in the internal store. If a metric has never been written for your environment, it does not appear in the list.

Several metrics are conditional, so the exact set you see depends on your hardware, configuration, and activity:

Hardware/OS-dependent, per-device metrics. Some metrics are emitted once per block device or network interface, so the metric name carries a device suffix. For example, you might see BlockActiveTime_nbd0 and BlockActiveTime_nbd1 for two different block devices. A server with different disks or NICs produces different suffixes, and GPU metrics only appear when a GPU is present. This is the largest source of real-world variance.
Feature-gated metrics. Some counters emit only after a feature is exercised. For example, certain cache, replication, or S3 metrics stay at zero or absent until they are used.
Cgroup/container metrics. These differ between bare-metal, container, and cloud instance types.
Zero-value suppression. Depending on how Cribl scrapes metrics and whether zero-valued series are written, some metrics may never appear in a quiet environment.

Stream Workers

Metric	Type	Unit	Definition
`stream_workers_total`	gauge	count	Total number of Worker Nodes registered.
`stream_worker_groups_total`	gauge	count	Total number of distinct Worker Groups.
`stream_worker_processes_total`	gauge	count	Worker Processes running on a given Worker Node.
`stream_worker_cpus_total`	gauge	count	CPU cores available on a Worker Node.
`stream_worker_memory_gib`	gauge	GiB	Total physical memory on a Worker Node.
`stream_worker_storage_free_gib`	gauge	GiB	Free disk space on a Worker Node.
`stream_worker_storage_total_gib`	gauge	GiB	Total disk space on a Worker Node.
`stream_worker_disk_utilization_percent`	gauge	percent	Disk used divided by total, times 100, for a Worker Node.
`stream_worker_uptime_seconds`	gauge	seconds	Seconds since the Cribl process of the Worker started.
`stream_worker_disconnected`	gauge	count	`1` if the Worker is disconnected from the Leader, else `0`.

Worker and System Health

Metric	Type	Unit	Definition
`system_load_avg`	gauge	ratio	CPU load average of a Worker Process.
`system_mem_rss`	gauge	bytes	Resident set size (RSS) memory of a Worker Process.
`system_mem_heap_used`	gauge	bytes	V8 heap memory used.
`system_mem_heap_total`	gauge	bytes	V8 heap memory allocated.
`system_free_mem`	gauge	bytes	Free system memory on the Node.
`system_total_mem`	gauge	bytes	Total system memory on the Node.
`system_disk_used`	gauge	bytes	Disk space used on the Node.
`system_total_disk`	gauge	bytes	Total disk space on the Node.
`system_pq_used`	gauge	n/a	Persistent-queue / tempfs disk usage on the Node.
`system_num_workers`	gauge	count	Number of Worker Processes on the Node.
`system_worker_crashes`	counter	count	Worker Process crashes detected.

Cribl Edge

Metric	Type	Unit	Definition
`edge_workers_total`	gauge	count	Total Edge Nodes.
`edge_workers_alive_total`	gauge	count	Edge Nodes currently alive/connected.
`edge_workers_disconnected_total`	gauge	count	Edge Nodes disconnected (global count).
`edge_workers_unhealthy_total`	gauge	count	Edge Nodes reporting unhealthy.
`edge_workers_groups_total`	gauge	count	Number of Edge Fleets.
`edge_workers_conf_versions`	gauge	count	Distinct config versions across Edge Nodes.
`edge_workers_software_versions`	gauge	count	Distinct software versions across Edge Nodes.
`edge_groups_total`	gauge	count	Total Fleets.
`edge_groups_sources_total`	gauge	count	Sources configured across Fleets.
`edge_groups_destinations_total`	gauge	count	Destinations configured across Fleets.
`edge_groups_pipelines_total`	gauge	count	Pipelines configured across Fleets.
`edge_groups_routes_total`	gauge	count	Routes configured across Fleets.

Cribl Search

Metric	Type	Unit	Definition
`search_service_cpu`	gauge	percent	Search service CPU usage.
`search_service_elu`	gauge	percent	Search service event-loop utilization.
`search_service_memory_heapUsed`	gauge	bytes	Search service heap memory used.
`search_searches_errored`	counter	count	Searches that errored.
`search_searches_completed`	counter	count	Searches completed.
`search_searches_dispatched`	counter	count	Searches dispatched to executors.
`search_searches_queued`	gauge	count	Searches waiting in queue.
`search_error`	counter	count	Search errors by type (`kusto`, `limits`, or `queue`).
`search_executors_allocated`	gauge	count	Executors allocated.
`search_executors_dispatched`	gauge	count	Executors dispatched.
`search_job_queue_length`	gauge	count	Length of the search job queue.
`search_healthcheck`	gauge	count	Search service health-check status.
`search_cpu_time_billable`	counter	seconds	Billable CPU time.
`search_cpu_time_coord`	counter	seconds	Coordinator CPU time.
`search_cpu_time_execs`	counter	seconds	Executor CPU time.
`search_cpu_util_coord`	gauge	percent	Coordinator CPU utilization.
`search_runtime`	gauge	ms	Search runtime duration.
`search_time_to_first_byte`	gauge	ms	Time to first byte for a search.
`search_in_bytes`	counter	bytes	Bytes read by search.
`search_out_bytes`	counter	bytes	Bytes written by search.
`search_in_events`	counter	events	Events read by search.
`search_out_events`	counter	events	Events written by search.

Cribl Lake

Metric	Type	Unit	Definition
`lake_failed_lakehouses`	gauge	count	Lakehouses currently in a failed state (dimension: `lake_id`). Feeds the Lake health score.
`lake_byos_metrics_to_generate`	gauge	count	Bring Your Own Storage (BYOS) billing-metric generation backlog: past days of usage still pending metric generation for a storage location (decrements as each day is processed). Dimensions: `lake_id`, `storage_location_id`. Sustained high values mean billing-metric generation is falling behind.
`lake_byos_metrics_generated`	gauge	count	Bring Your Own Storage (BYOS) days of usage for which billing metrics were successfully generated (increments per success). Dimensions: `lake_id`, `storage_location_id`.
`lake_da_http_delayed`	gauge	count	Direct Access (DA) HTTP Sources stuck in the delayed provisioning state (past the 1-hour timeout; escalates to critical at 24 hours).
`lake_da_http_provision_time_ms`	gauge	ms	Wall-clock time to provision the infrastructure of a Direct Access (DA) HTTP Source.

Data Throughput

Metric	Type	Unit	Definition
`total_in_bytes`	counter	bytes	Bytes received, per Source (input). Rate = ingest bytes/sec.
`total_out_bytes`	counter	bytes	Bytes sent, per Destination (output). Rate = egress bytes/sec.
`total_in_events`	counter	events	Events received, per Source.
`total_out_events`	counter	events	Events sent, per Destination.
`service_in_bytes`	counter	bytes	Bytes received by a Source service/listener.
`service_out_bytes`	counter	bytes	Bytes sent by a Destination service.
`service_in_events`	counter	events	Events received by a Source service.
`service_out_events`	counter	events	Events sent by a Destination service.
`in_bytes`	counter	bytes	Bytes into a specific component (Pipeline, Route, or Pack).
`out_bytes`	counter	bytes	Bytes out of a specific component.
`in_events`	counter	events	Events into a specific component.
`out_events`	counter	events	Events out of a specific component.
`pipe_in_bytes`	counter	bytes	Bytes into a Pipeline.
`pipe_out_bytes`	counter	bytes	Bytes out of a Pipeline.
`pack_in_events`	counter	events	Events entering a Pack.
`health_inputs`	gauge	count	Health status per Source (labels: `input`, `input_type`, `__worker_group`).
`health_outputs`	gauge	count	Health status per Destination (label: `output`).
`service_health`	gauge	count	Health status of a service.

Connections

Metric	Type	Unit	Definition
`service_openCxn`	counter	count	New connections opened on a service.
`service_closeCxn`	counter	count	Connections closed on a service.
`service_rejectCxn`	counter	count	Connections rejected by a service.
`service_activeCxn`	gauge	count	Currently open connections on a service.
`total_openCxn`	counter	count	Total connections opened across services.
`total_activeCxn`	gauge	count	Total currently active connections.
`connections_stale_should_shed`	counter	count	Worker-to-Leader connections dropped because the Worker’s heartbeat went stale (no message for more than 5 heartbeat periods). Rising values mean Workers are losing contact with the Leader.
`connections_legacy_should_shed`	counter	count	Worker/Edge connections deliberately bounced off the shared legacy listener so they reconnect to the correct product-specific listener (version 4.14.1 and later). Normal during upgrades; not an error.

Persistent Queue

Metric	Type	Unit	Definition
`pq_buffered_events`	gauge	events	Events currently buffered in persistent queues (a queue is active when this is greater than `0`).
`pq_queue_size`	gauge	count	Current persistent-queue depth.
`pq_in_bytes`	counter	bytes	Bytes written into persistent queues.
`pq_health`	gauge	count	Persistent-queue health status (`0` = unhealthy in the default Monitor).

Errors and Dropped Data

Metric	Type	Unit	Definition
`logged_errors`	counter	count	Processing errors logged by Workers.
`total_dropped_events`	counter	events	Events dropped by a Destination (labels: `output`, `output_type`, `__worker_group`).
`pipe_dropped_events`	counter	events	Events dropped by a Pipeline.
`route_dropped_events`	counter	events	Events dropped by a Route.
`backpressure_outputs`	gauge	count	Outputs currently experiencing backpressure.
`blocked_outputs`	gauge	count	Outputs currently blocked.
`iometrics_failed_requests`	counter	count	Failed I/O requests (labels: `status_code`, `input`, `__worker_group`).

Jobs and Collectors

Metric	Type	Unit	Definition
`jobs_job_done`	counter	count	Completed jobs (label `collector_type`: `rest`, `script`, or `system`).
`jobs_job_init`	counter	count	Initialized jobs.
`jobs_inflight_jobs`	gauge	count	Jobs currently in flight (sustained high values indicate stalled jobs).
`jobs_task_errors`	counter	count	Collector task errors.
`jobs_task_requests`	counter	count	Collector/job task requests issued by the Leader’s dispatcher, per Worker Group.
`collector_cache_size`	gauge	count	Number of entries in the Collector cache (compiled job function modules; 30-minute TTL).

Metrics Pipeline Internals

Metric	Type	Unit	Definition
`metrics_pool_num_metrics`	gauge	count	Distinct metric series currently held in the in-memory metrics pool; a proxy for metric cardinality/memory pressure.
`metrics_pool_string_pool_bytes`	gauge	bytes	Memory used by the metrics string pool (interned label/name strings).
`metrics_pool_array_pool_bytes`	gauge	bytes	Memory used by the metrics array pool (metric value buffers).
`metrics_packet_reducer_packets_accepted`	counter	count	Metric packets taken in by the packet reducer (the stage that collapses duplicate metric packets).
`metrics_packet_reducer_packets_reduced`	counter	count	Packets merged/aggregated into fewer packets (the savings from reduction).
`metrics_packet_reducer_packets_emitted`	counter	count	Packets sent downstream after reduction (labeled by reason: `timer` vs. `max_size` flush).
`metrics_packet_reducer_new_series`	counter	count	New unique metric series first seen by the reducer.

Cluster and Transport Internals

Metric	Type	Unit	Definition
`nodecluster_primary_messages`	counter	count	Inter-process messages sent from the primary (main) process of a Node to its Worker Processes; internal IPC within a single Node (per Worker Node).
`cluster_metrics_sender_inflight`	gauge	count	Metric batches currently in flight from a Worker to the Leader (not yet acknowledged). Sustained high values indicate a metrics backlog.
`packet_stream_max_writable_length`	gauge	count	Peak buffer size of the Worker-to-Leader packet stream during backpressure. Rising values indicate connection congestion.

Git Manager

Metric	Type	Unit	Definition
`gitmgr_serialized_operation_count`	counter	count	Git operations run through the serialized (one-at-a-time) queue. Cribl serializes git writes to avoid conflicts.
`gitmgr_serialized_operation_duration`	gauge	ms	Time taken by serialized git operations. High values indicate slow config commits/deploys.

Metrics Reference ​

Why Your Available Metrics May Differ ​

Stream Workers ​

Worker and System Health ​

Cribl Edge ​

Cribl Search ​

Cribl Lake ​

Data Throughput ​

Connections ​

Persistent Queue ​

Errors and Dropped Data ​

Jobs and Collectors ​

Metrics Pipeline Internals ​

Cluster and Transport Internals ​

Git Manager ​

Common Resources