On This Page

Home / Stream/ Monitor Health and Metrics/ Internal Logs/cribl.log

cribl.log ​

Cribl Stream records its process activity in cribl.log. Several Cribl Stream processes each write their own file named cribl.log, and each process keeps its file in its own directory. Each process’s cribl.log records different activity. For example:

  • The Worker Process cribl.log records data processing, including a _raw stats summary once per minute. It is useful when events are dropped, delayed, or transformed unexpectedly.
  • The API Process cribl.log records communication with the Leader and other API requests, which is useful when a Worker Node cannot communicate with the Leader.
  • The Config Helper cribl.log on the Leader records config maintenance and previews for a Worker Group, which is useful when a config preview fails.

For the path of each cribl.log, see Internal Logs. To search cribl.log files in a built-in Dataset, see Map Log Files to Built-in Datasets.

Example Event ​

The following example shows a typical event in cribl.log. Event fields may vary depending on the process or action that generated the event.

{
  "time": "2026-03-03T19:37:55.764Z",
  "cid": "api",
  "channel": "HBChannelClient",
  "level": "info",
  "message": "metric sender",
  "total": 1283,
  "failed": 0,
  "dropped": 151
}

Event Fields ​

The following table describes fields that may be included in events in this log. Event fields may vary depending on the process or action that generated the event.

FieldDescription
timeUTC time when Cribl Stream wrote the event.
cidProcess that wrote the event. The set of values depends on which processes are running. The API Process uses api. A Worker Process uses w plus its process ID, such as w0, wLB for the load balancer, or wPQWorker for the persistent queue Worker. A Leader service uses service: plus the service name, such as service:connections. Every process for that service uses the same value. A Config Helper uses cfg: plus the Worker Group ID, such as cfg:default.
channelLogger channel for the component that wrote the event.
levelLog level for the event, typically info.
messageEvent category. Values include metric sender and, on Worker Process copies once per minute, _raw stats.
totalOn metric sender events, number of metric packets sent during the interval.
failedOn metric sender events, number of send attempts that failed.
droppedOn metric sender events, number of metric packets dropped before send.
streamErrorsOn some service telemetry events, count of stream errors in the interval.
streamBackpressuresOn some service telemetry events, count of backpressure events in the interval.

_raw stats Events ​

Each Worker Process writes a _raw stats event to its copy of this log once per minute. The event summarizes that process for the interval between starttime and endtime.

{
  "time": "2022-11-17T16:54:05.349Z",
  "cid": "w0",
  "channel": "server",
  "level": "info",
  "message": "_raw stats",
  "inEvents": 307965,
  "outEvents": 495848,
  "inBytes": 52756162,
  "outBytes": 83028013,
  "starttime": 1668703980,
  "endtime": 1668704040,
  "activeCxn": 0,
  "openCxn": 0,
  "closeCxn": 0,
  "rejectCxn": 0,
  "abortCxn": 0,
  "pqInEvents": 62000,
  "pqOutEvents": 114591,
  "pqInBytes": 12163896,
  "pqOutBytes": 22481509,
  "pqTotalBytes": 480467058,
  "droppedEvents": 0,
  "tasksStarted": 6,
  "tasksCompleted": 6,
  "activeEP": 9,
  "blockedEP": 0,
  "cpuPerc": 101.09,
  "eluPerc": 97.81,
  "mem": {
    "heap": 277,
    "heapTotal": 287,
    "ext": 46,
    "rss": 453,
    "buffers": 0
  }
}

When message is _raw stats, the event can include the following fields in addition to the common event fields for cribl.log.

FieldDescription
abortCxnNumber of TCP connections that were aborted.
activeCxnNumber of TCP connections newly opened at the time the _raw stats are logged. This is a gauge when exported in internal metrics, and can otherwise be ignored as an instantaneous measurement. Only some application protocols count toward this. For example, any HTTP-based Source does not count.
activeEPNumber of currently active event processors (EPs). EPs are used to process events through Breakers and Pipelines as the events are received from Sources and sent to Destinations. EPs are typically created per TCP connection (such as for HTTP).
blockedEPNumber of currently blocked event processors (caused by blocking Destinations).
closeCxnNumber of TCP connections that were closed.
cpuPercCPU utilization from the combined user and system activity over the last 60 seconds.
droppedEventsThis is equivalent to the total.dropped_events metric. Drops can occur from Functions in processing Pipelines and Destination post-processing Pipelines, from Source or Destination backpressure, or from other errors. Any event not sent to a Destination is considered dropped.
eluPercEvent loop utilization. Represents the percentage of time over the last 60 seconds that the Node.js runtime spent processing events within its event loop.
endtimeEnd of the one-minute interval for _raw stats.
inBytesNumber of bytes received from all Sources (based only off _raw).
inEventsNumber of events received from all inputs after Event Breakers are applied. This can be larger than outEvents if events are dropped via Drop, Aggregation, Suppression, Sampling, or similar Functions. Compare with outEvents to spot a sudden gap from drops, aggregation, or new events.
mem.buffersMemory allocated for ArrayBuffers and SharedArrayBuffers.
mem.extExternal section of process memory, in MB.
mem.heapUsed heap section of process memory, in MB.
mem.heapTotalTotal heap section of process memory, in MB.
mem.rssResident set size section of process memory, in MB.
openCxnSame as activeCxn, but tracked as a counter rather than a gauge. So openCxn will show all connections newly opened each minute, and is more accurate than using activeCxn.
outBytesNumber of bytes sent to all Destinations (based only off _raw).
outEventsNumber of events sent out to all Destinations. This can be larger than inEvents due to creating event clones or entirely new unique events (such as when using the Aggregation Function).
pqInBytesNumber of bytes that were written to persistent queues, across all Destinations.
pqInEventsNumber of events that were written to persistent queues, across all Destinations.
pqOutBytesNumber of bytes that were flushed from persistent queues, across all Destinations.
pqOutEventsNumber of events that were flushed from persistent queues, across all Destinations.
pqTotalBytesAmount of data currently stored in persistent queues, across all Destinations.
rejectCxnNumber of TCP connections that were rejected.
starttimeBeginning of the one-minute interval for _raw stats.
tasksCompletedThe number of tasks the process completed for all collection jobs for which it was executing tasks during the minute.
tasksStartedThe number of tasks the process started for all collection jobs for which it was executing tasks during the minute. These counters do not identify an individual job. For one collection run, see Collector Job Logs.