On This Page

Home / Edge/ Reference/Linux System Metrics Details

Linux System Metrics Details

Consult the reference for the metrics reported by the System Metrics Source on Linux.


Events generated by the System Metrics Source have metrics metadata to designate dimension and metric fields. The host field contains the hostname, and is included as a dimension in all of them. The collectors include:

In the Source’s configuration modal, you can set the level of detail for each type of metrics:

  • Basic enables minimal metrics, averaged or aggregated.
  • All enables full, detailed metrics, specified for individual CPUs, interfaces, and so on.
  • Custom displays sub-menus and buttons from which you can choose a level of detail (Basic, All, Custom, or Disabled) for each type of event.
  • Disabled means that no metrics will be generated.

Basic and Custom have different meanings depending on event type and will be described under each section below.

The tables outline the metrics emitted for each mode (Basic or Custom) and where applicable, the dimensions (to indicate where the metrics are coming from).

System

With System Metrics enabled, Cribl Edge captures CPU load averages, uptime, and count. The Custom option allows you to include process metrics that reflect the numbers of processes in various states.

Metrics for the overall system include the following:

NameDescriptionTypeDimensionsMode
node_uname_infoLabeled system information as provided by the uname system call.Counterrelease, sysname, versionBasic
node_cpu_countThe number of CPU cores.Gaugerelease, sysname, versionBasic
node_uptime_secondsSystem uptime in seconds.CounterN/ABasic
node_boot_time_secondsNode boot time in Unix time.CounterN/ABasic
node_time_secondsSystem time in seconds.CounterN/ABasic
node_load11m load average.GaugeN/ABasic
node_load55m load average.GaugeN/ABasic
node_load1515m load average.GaugeN/ABasic
node_open_fdsOpen file descriptorsCounterN/ABasic
node_processes_state_allTotal number of processes in different states.GaugestateBasic
node_processes_threadsAllocated threads in system.GaugestateBasic
node_procs_blockedNumber of processes blocked waiting for I/O to complete.GaugestateBasic
node_procs_runningNumber of processes in runnable state.GaugestateBasic
node_processes_stateNumber of processes in each state.GaugestateCustom: Process metrics

CPU

Cribl Edge captures active, user, system, idle, and iowait percentages over all CPUs, with options to add per-CPU metrics and raw time counters for each state.

Metrics for CPUs include the following:

NameDescriptionTypeDimensionsMode
node_cpu_percent_active_allPercent all the CPUs spent in activity.GaugeN/ABasic
node_cpu_seconds_active_all_totalSeconds all the CPUs spent in activity (excluding idle and wait).CounterN/ACustom: CPU time metrics
node_cpu_seconds_active_totalSeconds each CPU spent in activity (excluding idle and wait).CountercpuCustom: CPU time metrics
node_cpu_seconds_all_totalSeconds for all CPUs usage.CountermodeCustom: CPU time metrics
node_cpu_seconds_totalSeconds for each CPU’s usage.Countercpu, modeCustom: CPU time metrics
node_cpu_percent_activePercent each CPU spent in activity.GaugecpuCustom: Per CPU or Detailed metrics
node_cpu_percent_allPercent CPU usage for all.Gaugemode,userCustom: Detailed metrics
node_cpu_percentPercent CPU usage for each CPU.Gaugecpu,mode,userCustom: Per CPU or Detailed metrics
  • The Per CPU option adds metrics with the cpu dimension.
  • The Detailed metrics option adds metrics with mode dimension set to: irq, softirq, steal, guest, guest_nice, and nice.

Memory

With System Metrics enabled, Cribl Edge captures memory metrics including total, used, available, swap_free, and swap_total, with the option to toggle all memory states.

Metrics for memory include the following:

NameDescriptionTypeDimensionsMode
node_memory_MemTotal_bytesMemory information field MemTotal_bytes.GaugeN/ABasic
node_memory_Used_bytesUsed memory in bytes.GaugeN/ABasic
node_memory_Used_percentPercent used memory.GaugeN/ABasic
node_memory_MemAvailable_bytesMemory information field MemAvailable_bytes.GaugeN/ABasic
node_memory_MemAvailable_percentPercent memory available.GaugeN/ABasic
node_memory_SwapFree_bytesMemory information field SwapFree_bytes.GaugeN/ABasic
node_memory_SwapTotal_bytesMemory information field SwapTotal_bytes.GaugeN/ABasic
node_vmstat_oom_kill/proc/vmstat information field oom_kill.GaugeN/ABasic
node_vmstat_pgfault/proc/vmstat information field pgfault.GaugeN/ABasic
node_vmstat_pgmajfault/proc/vmstat information field pgmajfault.GaugeN/ABasic
node_vmstat_pgpgin/proc/vmstat information field pgpgin.GaugeN/ABasic
node_vmstat_pgpgout/proc/vmstat information field pgpgout.GaugeN/ABasic
node_vmstat_pswpin/proc/vmstat information field pswpin.GaugeN/ABasic
node_vmstat_pswpout/proc/vmstat information field pswpout.GaugeN/ABasic
node_memory_Active_bytesMemory information field Active_bytes.GaugeN/ACustom: Detailed metrics
node_memory_Buffers_bytesMemory information field Buffers_bytes.GaugeN/ACustom: Detailed metrics
node_memory_Cached_bytesMemory information field Cached_bytes.GaugeN/ACustom: Detailed metrics
node_memory_Dirty_bytesMemory information field Dirty_bytesGaugeN/ACustom: Detailed metrics
node_memory_MemFree_bytesMemory information field MemFree_bytes.GaugeN/ACustom: Detailed metrics
node_memory_SwapCached_bytesMemory information field SwapCached_bytes.GaugeN/ACustom: Detailed metrics

Network

With System Metrics enabled, Cribl Edge captures bytes, packets, errors, and network connections over all interfaces. The Custom option allows you to filter interfaces, and to decide whether to select per-interface metrics and generate protocol metrics.

Metrics for networks include the following:

NameDescriptionTypeDimensionsMode
node_network_receive_bytes_all_totalNetwork device statistic receive_bytes.CounterN/ABasic
node_network_receive_errs_all_totalNetwork device statistic receive_errs.CounterN/ABasic
node_network_receive_packets_all_totalNetwork device statistic receive_packets.CounterN/ABasic
node_network_transmit_bytes_all_totalNetwork device statistic transmit_bytes.CounterN/ABasic
node_network_transmit_errs_all_totalNetwork device statistic transmit_errs.CounterN/ABasic
node_network_transmit_packets_all_totalNetwork device statistic transmit_packets.CounterN/ABasic
node_socket_tcp_established_totalTCP established connections.CounterN/ABasic
node_network_receive_bytes_totalNetwork device statistic receive_bytes per interface.CounterdeviceCustom: Per Interface
node_network_receive_errs_totalNetwork device statistic receive_errs per interface.CounterdeviceCustom: Per Interface
node_network_receive_packets_totalNetwork device statistic receive_packets per interface.CounterdeviceCustom: Per Interface
node_network_transmit_bytes_totalNetwork device statistic transmit_bytes per interface.CounterdeviceCustom: Per Interface
node_network_transmit_errs_totalNetwork device statistic transmit_errs per interface.CounterdeviceCustom: Per Interface
node_network_transmit_packets_totalNetwork device statistic transmit_packets per interface.CounterdeviceCustom: Per Interface
node_network_receive_drop_all_totalNetwork device statistic receive_drop.CounterN/ACustom: Detailed Metrics
node_network_receive_drop_totalNetwork device statistic receive_drop per interface.CounterdeviceCustom: Detailed Metrics
node_network_transmit_drop_all_totalNetwork device statistic transmit_drop.CounterN/ACustom: Detailed Metrics
node_network_transmit_drop_totalNetwork device statistic transmit_drop per interface.CounterdeviceCustom: Detailed Metrics
node_socket_tcp_syn_sent_totalTCP sent packets total.CounterN/ACustom: Detailed Metrics
node_socket_tcp_syn_recv_totalTCP received packets total.CounterN/ACustom: Detailed Metrics
node_socket_tcp_fin_wait1_totalTotal connections waiting for termination request from remote TCP.CounterN/ACustom: Detailed Metrics
node_socket_tcp_fin_wait2_totalActive TCP connections to be shut down.CounterN/ACustom: Detailed Metrics
node_socket_tcp_time_wait_totalLength of time to pass to be sure the remote TCP received the acknowledgement to terminate.CounterN/ACustom: Detailed Metrics
node_socket_tcp_close_totalTotal TCP sockets with closed connections.CounterN/ACustom: Detailed Metrics
node_socket_tcp_last_ack_totalTotal TCP sockets in state before the TCP connection is closed.CounterN/ACustom: Detailed Metrics
node_socket_tcp_listen_totalTotal TCP sockets waiting for a connection request from any remote TCP/port.CounterN/ACustom: Detailed Metrics
node_socket_tcp_closing_totalTotal TCP sockets waiting for connection termination request acknowledgement.CounterN/ACustom: Detailed Metrics
node_socket_tcp_none_totalNumber of TCP sockets with no connections.CounterN/ACustom: Detailed Metrics
node_socket_udp_totalNumber of UDP sockets in use.CounterN/ACustom: Detailed Metrics

Disk

With System Metrics enabled, Cribl Edge captures disk-used metrics - in percent, bytes read and written, and read and write operations - over all mounted disks. The Custom option allows you to filter devices, mountpoint, and filesystem type, and to decide whether to select per-device metrics and generate detailed metrics.

Metrics for Disk include the following:

NameDescriptionTypeDimensionsMode
node_disk_reads_completed_all_totalTotal number of reads completed successfully.CounterN/ABasic
node_disk_read_bytes_all_totalTotal number of bytes read successfully.CounterN/ABasic
node_disk_writes_completed_all_totalNetwork device statistic receive_packets.CounterN/ABasic
node_disk_written_bytes_all_totalTotal number of bytes written successfully.CounterN/ABasic
node_filesystem_size_bytes_allFilesystem size in bytes.GaugeN/ABasic
node_filesystem_avail_bytes_allFilesystem space available to non-root users in bytes.GaugeN/ABasic
node_filesystem_used_bytes_allFilesystem used space in bytes.GaugeN/ABasic
node_filesystem_used_percent_allPercent filesystem used space.GaugeN/ABasic
node_filesystem_files_free_allFilesystem free file nodes.GaugeN/ABasic
node_filesystem_files_used_allFilesystem total used file nodes.GaugeN/ABasic
node_filesystem_files_used_percent_allPercent Filesystem used in all disks.GaugeN/ABasic
node_filesystem_size_bytes_allFilesystem size in bytes.GaugeN/ABasic
node_filesystem_used_bytesFilesystem used space in bytes per disk.Gaugedevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_filesystem_used_percentPercent Filesystem used per disk.Gaugedevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_disk_reads_completed_totalTotal number of reads completed successfully per disk.CounterdeviceCustom: Per device metrics
node_disk_read_bytes_totalTotal number of bytes read successfully per disk.Counterdevice, fstype, mountpointCustom: Per device metrics
node_disk_writes_completed_totalNetwork device statistic receive_packets per disk.Counterdevice, fstype, mountpointCustom: Per device metrics
node_disk_written_bytes_totalTotal number of bytes written successfully per disk.Counterdevice, fstype, mountpointCustom: Per device metrics
node_disk_discards_completed_all_totalTotal number of discards completed successfully.CounterN/ACustom: Detailed Metrics
node_disk_discards_completed_totalTotal number of discards completed successfully per disk.Counterdevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_disk_discards_merged_all_totalTotal number of discards merged.CounterN/ACustom:Detailed metrics
node_disk_read_time_seconds_totalTotal number of seconds spent by all reads per disk.Counterdevice, fstype, mountpointCustom:Detailed metrics
node_disk_write_time_seconds_all_totalTotal number of seconds spent by all writes.CounterN/ACustom: Detailed Metrics
node_disk_read_time_seconds_all_totalTotal number of seconds spent by all reads.CounterN/ACustom: Detailed Metrics
node_disk_write_time_seconds_totalTotal number of seconds spent by all writes per disk.Counterdevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_disk_reads_merged_all_totalTotal number of reads merged.CounterN/ACustom: Detailed Metrics
node_disk_writes_merged_all_totalTotal number of writes merged.CounterN/ACustom: Detailed metrics
node_disk_reads_merged_totalTotal number of reads merged per disk.Counterdevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_disk_writes_merged_totalTotal number of writes merged per disk.Counterdevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_disk_discards_merged_totalTotal number of discards merged per disk.Counterdevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_disk_io_time_seconds_all_totalTotal seconds spent doing I/Os.CounterN/ACustom: Detailed Metrics
node_disk_io_time_seconds_totalTotal seconds spent doing I/Os per disk.Counterdevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_disk_io_now_allThe number of I/Os currently in progress.GaugeN/ACustom: Detailed Metrics
node_disk_io_nowThe number of I/Os currently in progress per disk.Counterdevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_disk_io_time_weighted_seconds_all_totalWeighted number of seconds spent doing I/Os per device.Counterdevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_filesystem_size_bytesFilesystem size in bytes per disk.Gaugedevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_filesystem_avail_bytesFilesystem space available to non-root users in bytes per disk.Gaugedevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_filesystem_files_freeFilesystem free file nodes per disk.Gaugedevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_filesystem_files_used_percentPercent Filesystem used per disk.Gaugedevice, fstype, mountpointCustom: Per device metrics or Detailed metrics
node_filesystem_size_bytesFilesystem size in bytes per disk.Gaugedevice, fstype, mountpointCustom: Per device metrics or Detailed metrics

Container

With System Metrics enabled, Cribl Edge generates Docker information with CPU, memory, network, and disk metrics for running containers. Optionally, you can customize which containers to generate metrics from.

Metrics for Container include the following:

NameDescriptionTypeDimensionsMode
container_start_time_secondsUnix time (seconds) when the container was started.CounterN/ABasic
container_finish_time_secondsUnix time (seconds) when the container was stopped. Only for non-running containers.CounterN/ABasic
container_fs_reads_bytes_all_totalTotal bytes read for all disk devices.CounterN/ABasic
container_memory_usage_percentPercent of available memory being used.GaugeN/ABasic
container_network_receive_bytes_all_totalTotal bytes received for all network interfaces.CounterN/ABasic
container_network_receive_errors_all_totalTotal number of errors received for all network interfaces.CounterN/ABasic
container_network_receive_packets_all_totalTotal number of packets received for all network interfaces.CounterN/ABasic
container_network_transmit_bytes_all_totalTotal bytes transmitted for all network interfaces.CounterN/ABasic
container_network_transmit_errors_all_totalTotal number of errors transmitted for all network interfaces.CounterN/ABasic
container_network_transmit_packets_all_totalTotal number of packets transmitted for all network interfaces.CounterN/ABasic
container_memory_total_bytesTotal number of memory bytes available.CounterN/ABasic
container_cpu_user_seconds_totalNumber of seconds the container has been on the CPU running user code.CountercpuCustom: Per device metrics or Detailed metrics
container_cpu_system_seconds_totalNumber of seconds the container has been on the CPU running kernel code.CountercpuCustom: Per device metrics or Detailed metrics
container_fs_reads_bytes_totalTotal bytes read per deviceCounterdeviceCustom: Per device metrics or Detailed metrics
container_fs_writes_bytes_all_totalTotal bytes written for all disk devicesCounterN/ACustom: Detailed metrics
container_fs_writes_bytes_totalTotal bytes written per deviceCounterdeviceCustom: Per device metrics or Detailed metrics
container_fs_reads_all_totalTotal number of read operations for all disk devices.CounterN/ACustom: Detailed metrics
container_fs_writes_totalTotal number write operations per device.CounterdeviceCustom: Per device metrics or Detailed metrics
container_memory_mapped_fileTotal bytes writted for all disk devices.CounterN/ACustom: Detailed metrics
container_memory_max_usage_bytesHighest seen value of the container_memory_usage_bytes metric.CounterN/ACustom: Detailed metrics
container_memory_pginTotal number of memory page-in events.CounterN/ACustom: Detailed metrics
container_mem.pgpgoutTotal number of memory pages paged out.CounterN/ACustom: Detailed metrics
container_memory_pgfaultTotal number of major page faults.CounterN/ACustom: Detailed metrics
container_memory_pgmajfaultTotal number of minor page faults.CounterN/ACustom: Detailed metrics
container_memory_usage_bytesNumber of memory bytes used.CounterN/ACustom: Detailed metrics
container_network_receive_dropped_all_totalTotal number of receives dropped for all network interfaces.CounterN/ACustom: Detailed metrics
container_network_transmit_dropped_all_totalTotal number of transmits dropped for all network interfaces.CounterN/ACustom: Detailed metrics

Process Metrics

With Process Metrics enabled, Cribl Edge captures process-specific metrics from Linux servers and reports them as events. This allows you to monitor specific processes on Cribl.Cloud instances. You can generate events for any process object.

For information on how to configure the System Metrics Source to generate process-specific metrics, check out the Process Metrics section of the System Metrics page.

Process-specific metrics are not affected by the Host Metrics detail setting.

Process-specific metrics include the following:

NameDescriptionTypeDimensions
process_num_threadsThe number of threads.Gaugeprocess_cmdline, process_set, process_uid, process_gid, process_service
process_open_filedescThe number of file descriptors.Gaugeprocess_cmdline, process_set, process_uid, process_gid, process_service
process_write_bytesThe number of bytes which this process caused to be sent to the storage layer.Counterprocess_cmdline, process_set, process_uid, process_gid, process_service
process_read_bytesThe number of bytes this process actually fetched from the storage layer. This number is accurate for block-backed filesystems.Counterprocess_cmdline, process_set, process_uid, process_gid, process_service
process_major_page_faultsThe number of major faults for this process that required loading a memory page from disk.Counterprocess_cmdline, process_set, process_uid, process_gid, process_service
process_minor_page_faultsThe number of minor faults for this process that have not required loading a memory page from disk.Counterprocess_cmdline, process_set, process_uid, process_gid, process_service
process_voluntary_context_switchesThe number of voluntary context switches.Counterprocess_cmdline, process_set, process_uid, process_gid, process_service
process_nonvoluntary_context_switchesThe number of involuntary context switches.Counterprocess_cmdline, process_set, process_uid, process_gid, process_service
process_cpu_usageThe process’s CPU usage, expressed as a percentage of total CPU power.Gaugeprocess_cmdline, process_set, process_uid, process_gid, process_service
process_cpu_secondsThe process’s CPU usage, based on user time and system time.Counterprocess_cmdline, process_set, process_uid, process_gid, process_service
process_resident_memory_bytesThe amount of memory used, in bytes. Includes the pages that count toward text, data, or stack space. Does not include pages that haven’t been demand-loaded in or are swapped out.Gaugeprocess_cmdline, process_set, process_uid, process_gid, process_service
process_virtual_memory_bytesThe process’s virtual memory size.Gaugeprocess_cmdline, process_set, process_uid, process_gid, process_service
process_swapped_memory_bytesThe process’s swapped memory size.Gaugeprocess_cmdline, process_set, process_uid, process_gid, process_service
process_memory_bytesThe total amount of memory used by the process, in bytes.Gaugeprocess_cmdline, process_set, process_uid, process_gid, process_service
process_memory_usageThe total amount of memory used by the process, as a percentage of total memory.Gaugeprocess_cmdline, process_set, process_uid, process_gid, process_service
process_start_timeThe time that the process started, derived by adding the start time to the boot time, making it relative to epoch.Gaugeprocess_cmdline, process_set, process_uid, process_gid, process_service

GPU Metrics

With GPU Metrics enabled, Cribl Edge captures temperature, utilization, memory, power, clock, and throttle metrics from Nvidia GPUs.

The All option adds per-GPU events with identifying dimensions (gpu_index, gpu_name, gpu_uuid) and detailed PCIe, ECC, encoder, and power metrics.

In Basic mode, or when Per GPU metrics is disabled in Custom mode, the collector emits a single aggregated metric event for all GPUs during each collection interval. These metrics use the same name with the _all suffix, such as node_gpu_temperature_celsius_all. Across GPUs, those _all values use sum, average, or maximum, depending on how the metric is defined. The node_gpu_count gauge appears only on the aggregated event and does not have an _all equivalent.

NameDescriptionTypeMode
node_gpu_countNumber of NVIDIA GPUs detected (aggregated event only).GaugeBasic
node_gpu_temperature_celsiusGPU core temperature in degrees Celsius.GaugeAll or Custom: Per GPU
node_gpu_temperature_memory_celsiusGPU memory temperature in degrees Celsius.GaugeAll or Custom: Per GPU
node_gpu_fan_speed_percentFan speed as a percent of maximum.GaugeAll or Custom: Per GPU
node_gpu_utilization_gpu_percentPercent utilization of GPU compute.GaugeAll or Custom: Per GPU
node_gpu_utilization_memory_percentPercent utilization of GPU memory.GaugeAll or Custom: Per GPU
node_gpu_utilization_encoder_percentPercent utilization of the hardware video encoder.GaugeAll or Custom: Per GPU
node_gpu_utilization_decoder_percentPercent utilization of the hardware video decoder.GaugeAll or Custom: Per GPU
node_gpu_power_draw_wattsInstantaneous GPU power draw in watts (see also average and instantaneous detail metrics).GaugeAll or Custom: Per GPU
node_gpu_power_limit_wattsConfigured GPU power limit in watts.GaugeAll or Custom: Per GPU
node_gpu_memory_total_bytesTotal GPU framebuffer memory size in bytes.GaugeAll or Custom: Per GPU
node_gpu_memory_used_bytesGPU memory used in bytes.GaugeAll or Custom: Per GPU
node_gpu_memory_free_bytesGPU memory free in bytes.GaugeAll or Custom: Per GPU
node_gpu_memory_reserved_bytesGPU memory reserved but not actively used in bytes.GaugeAll or Custom: Per GPU
node_gpu_clocks_gr_mhzGPU graphics clock in megahertz.GaugeAll or Custom: Per GPU
node_gpu_clocks_sm_mhzStreaming multiprocessor clock in megahertz.GaugeAll or Custom: Per GPU
node_gpu_clocks_mem_mhzGPU memory clock in megahertz.GaugeAll or Custom: Per GPU
node_gpu_clocks_video_mhzVideo clock in megahertz.GaugeAll or Custom: Per GPU
node_gpu_clocks_max_gr_mhzMaximum advertised graphics clock in megahertz.GaugeAll or Custom: Per GPU
node_gpu_clocks_max_mem_mhzMaximum advertised memory clock in megahertz.GaugeAll or Custom: Per GPU
node_gpu_clocks_max_sm_mhzMaximum advertised streaming multiprocessor clock in megahertz.GaugeAll or Custom: Per GPU
node_gpu_throttle_gpu_idleWhether the GPU idle clock throttle reason is active; 1 = active, 0 = not active.GaugeAll or Custom: Per GPU
node_gpu_throttle_applications_clocks_settingWhether the applications clocks-setting throttle reason is active; 1 = active, 0 = not active.GaugeAll or Custom: Per GPU
node_gpu_throttle_sw_power_capWhether the software power cap throttle reason is active; 1 = active, 0 = not active.GaugeAll or Custom: Per GPU
node_gpu_throttle_hw_thermal_slowdownWhether the hardware thermal slowdown throttle reason is active; 1 = active, 0 = not active.GaugeAll or Custom: Per GPU
node_gpu_throttle_hw_power_brake_slowdownWhether the hardware power brake slowdown throttle reason is active; 1 = active, 0 = not active.GaugeAll or Custom: Per GPU
node_gpu_throttle_sw_thermal_slowdownWhether the software thermal slowdown throttle reason is active; 1 = active, 0 = not active.GaugeAll or Custom: Per GPU
node_gpu_throttle_sync_boostWhether the synchronous boost throttle reason is active; 1 = active, 0 = not active.GaugeAll or Custom: Per GPU
node_gpu_throttle_hw_slowdownWhether the hardware slowdown throttle reason is active; 1 = active, 0 = not active.GaugeAll or Custom: Per GPU
node_gpu_pcie_gen_currentCurrent PCIe link generation.GaugeAll or Custom: Per GPU and Detailed
node_gpu_pcie_gen_gpucurrentNegotiated PCIe link generation reported by the GPU.GaugeAll or Custom: Per GPU and Detailed
node_gpu_pcie_gen_maxMaximum PCIe link generation.GaugeAll or Custom: Per GPU and Detailed
node_gpu_pcie_gen_gpumaxMaximum PCIe link generation supported by the GPU.GaugeAll or Custom: Per GPU and Detailed
node_gpu_pcie_gen_hostmaxMaximum PCIe link generation supported by the host.GaugeAll or Custom: Per GPU and Detailed
node_gpu_pcie_width_currentCurrent PCIe link width in lanes.GaugeAll or Custom: Per GPU and Detailed
node_gpu_pcie_width_maxMaximum PCIe link width in lanes.GaugeAll or Custom: Per GPU and Detailed
node_gpu_accounting_buffer_sizeProcess accounting statistics buffer size in KiB.GaugeAll or Custom: Per GPU and Detailed
node_gpu_utilization_jpeg_percentJPEG engine utilization in percent.GaugeAll or Custom: Per GPU and Detailed
node_gpu_utilization_ofa_percentOFA engine utilization in percent.GaugeAll or Custom: Per GPU and Detailed
node_gpu_encoder_session_countNumber of NVENC encoder sessions.GaugeAll or Custom: Per GPU and Detailed
node_gpu_encoder_average_fpsEncoder throughput in average frames per second.GaugeAll or Custom: Per GPU and Detailed
node_gpu_encoder_average_latency_usEncoder latency in microseconds.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_volatile_device_memoryCorrected ECC error count (volatile) for device memory.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_volatile_dramCorrected ECC error count (volatile) for DRAM.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_volatile_register_fileCorrected ECC error count (volatile) for register file.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_volatile_l1_cacheCorrected ECC error count (volatile) for L1 cache.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_volatile_l2_cacheCorrected ECC error count (volatile) for L2 cache.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_volatile_texture_memoryCorrected ECC error count (volatile) for texture memory.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_volatile_cbuCorrected ECC error count (volatile) for CBU.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_volatile_sramCorrected ECC error count (volatile) for SRAM.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_volatile_totalCorrected ECC error count (volatile) for memory (total).GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_aggregate_device_memoryCorrected ECC error count (aggregate lifetime) for device memory.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_aggregate_dramCorrected ECC error count (aggregate lifetime) for DRAM.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_aggregate_register_fileCorrected ECC error count (aggregate lifetime) for register file.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_aggregate_l1_cacheCorrected ECC error count (aggregate lifetime) for L1 cache.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_aggregate_l2_cacheCorrected ECC error count (aggregate lifetime) for L2 cache.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_aggregate_texture_memoryCorrected ECC error count (aggregate lifetime) for texture memory.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_aggregate_cbuCorrected ECC error count (aggregate lifetime) for CBU.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_aggregate_sramCorrected ECC error count (aggregate lifetime) for SRAM.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_corrected_aggregate_totalCorrected ECC error count (aggregate lifetime) for memory (total).GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_volatile_device_memoryUncorrected ECC error count (volatile) for device memory.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_volatile_dramUncorrected ECC error count (volatile) for DRAM.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_volatile_register_fileUncorrected ECC error count (volatile) for register file.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_volatile_l1_cacheUncorrected ECC error count (volatile) for L1 cache.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_volatile_l2_cacheUncorrected ECC error count (volatile) for L2 cache.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_volatile_texture_memoryUncorrected ECC error count (volatile) for texture memory.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_volatile_cbuUncorrected ECC error count (volatile) for CBU.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_volatile_sramUncorrected ECC error count (volatile) for SRAM.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_volatile_totalUncorrected ECC error count (volatile) for memory (total).GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_device_memoryUncorrected ECC error count (aggregate lifetime) for device memory.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_dramUncorrected ECC error count (aggregate lifetime) for DRAM.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_register_fileUncorrected ECC error count (aggregate lifetime) for register file.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_l1_cacheUncorrected ECC error count (aggregate lifetime) for L1 cache.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_l2_cacheUncorrected ECC error count (aggregate lifetime) for L2 cache.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_texture_memoryUncorrected ECC error count (aggregate lifetime) for texture memory.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_cbuUncorrected ECC error count (aggregate lifetime) for CBU.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_sramUncorrected ECC error count (aggregate lifetime) for SRAM.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_totalUncorrected ECC error count (aggregate lifetime) for memory (total).GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_volatile_sram_parityUncorrected ECC error count (volatile) for sram parity.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_volatile_sram_secdedUncorrected ECC error count (volatile) for sram secded.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_sram_parityUncorrected ECC error count (aggregate lifetime) for sram parity.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_sram_secdedUncorrected ECC error count (aggregate lifetime) for sram secded.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_sram_threshold_exceededUncorrected ECC error count (aggregate lifetime) for sram threshold exceeded.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_sram_l2Uncorrected ECC error count (aggregate lifetime) for sram l2.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_sram_smUncorrected ECC error count (aggregate lifetime) for sram sm.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_sram_mcuUncorrected ECC error count (aggregate lifetime) for sram mcu.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_sram_pcieUncorrected ECC error count (aggregate lifetime) for sram pcie.GaugeAll or Custom: Per GPU and Detailed
node_gpu_ecc_uncorrected_aggregate_sram_otherUncorrected ECC error count (aggregate lifetime) for sram other.GaugeAll or Custom: Per GPU and Detailed
node_gpu_retired_pages_sbeNumber of framebuffer pages retired after single-bit errors.GaugeAll or Custom: Per GPU and Detailed
node_gpu_retired_pages_dbeNumber of framebuffer pages retired after double-bit errors.GaugeAll or Custom: Per GPU and Detailed
node_gpu_remapped_rows_correctableNumber of DRAM rows requiring remapping after correctable ECC.GaugeAll or Custom: Per GPU and Detailed
node_gpu_remapped_rows_uncorrectableNumber of DRAM rows requiring remapping after uncorrectable ECC.GaugeAll or Custom: Per GPU and Detailed
node_gpu_remapped_rows_histogram_maxRemapped-row histogram count for worst-case mapping impact.GaugeAll or Custom: Per GPU and Detailed
node_gpu_remapped_rows_histogram_highRemapped-row histogram count for high mapping impact.GaugeAll or Custom: Per GPU and Detailed
node_gpu_remapped_rows_histogram_partialRemapped-row histogram count for partial mapping impact.GaugeAll or Custom: Per GPU and Detailed
node_gpu_remapped_rows_histogram_lowRemapped-row histogram count for low mapping impact.GaugeAll or Custom: Per GPU and Detailed
node_gpu_remapped_rows_histogram_noneRemapped-row histogram count for no measurable mapping impact.GaugeAll or Custom: Per GPU and Detailed
node_gpu_temperature_throttle_celsiusGPU throttle temperature threshold (t limit) in degrees Celsius.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_draw_average_wattsAverage GPU power draw in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_draw_instant_wattsInstantaneous GPU power draw in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_enforced_power_limit_wattsEnforced GPU power limit in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_default_limit_wattsDefault GPU power limit in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_min_limit_wattsMinimum GPU power limit setting in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_max_limit_wattsMaximum GPU power limit setting in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_module_power_draw_average_wattsAverage module-level power draw in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_module_power_draw_instant_wattsInstantaneous module-level power draw in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_module_power_limit_wattsModule-level power limit in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_module_enforced_power_limit_wattsEnforced module-level power limit in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_module_power_default_limit_wattsDefault module-level power limit in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_module_power_min_limit_wattsMinimum module-level power limit setting in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_module_power_max_limit_wattsMaximum module-level power limit setting in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_primary_floor_wattsPower-smoothing primary floor in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_secondary_floor_wattsPower-smoothing secondary floor in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_min_primary_activation_offsetMinimum activation offset for the primary power floor.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_min_primary_activation_pointMinimum activation point for the primary power floor.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_window_multiplierPower-smoothing window multiplier.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_curr_secondary_floor_wattsActive profile secondary floor in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_curr_primary_act_win_multiplierActive profile primary-floor activation-window multiplier.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_curr_primary_tar_win_multiplierActive profile primary-floor target-window multiplier.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_curr_primary_act_offsetPrimary-floor activation offset for the active smoothing profile.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_admin_secondary_floor_wattsAdmin override secondary floor in watts.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_admin_primary_act_win_multiplierAdmin primary activation-window multiplier override.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_admin_primary_tar_win_multiplierAdmin primary target-window multiplier override.GaugeAll or Custom: Per GPU and Detailed
node_gpu_power_smoothing_admin_primary_act_offsetAdmin primary activation offset.GaugeAll or Custom: Per GPU and Detailed
node_gpu_clocks_applications_gr_mhzRequested application-driven graphics clock in megahertz.GaugeAll or Custom: Per GPU and Detailed
node_gpu_clocks_applications_mem_mhzRequested application-driven memory clock in megahertz.GaugeAll or Custom: Per GPU and Detailed
node_gpu_clocks_default_applications_gr_mhzDefault application graphics clock in megahertz.GaugeAll or Custom: Per GPU and Detailed
node_gpu_clocks_default_applications_mem_mhzDefault application memory clock in megahertz.GaugeAll or Custom: Per GPU and Detailed
node_gpu_protected_memory_total_bytesTotal protected VRAM capacity in bytes.GaugeAll or Custom: Per GPU and Detailed
node_gpu_protected_memory_used_bytesProtected VRAM used in bytes.GaugeAll or Custom: Per GPU and Detailed
node_gpu_protected_memory_free_bytesProtected VRAM free in bytes.GaugeAll or Custom: Per GPU and Detailed
node_gpu_throttle_counter_sw_power_capCount of software power-cap clock-throttle events.GaugeAll or Custom: Per GPU and Detailed
node_gpu_throttle_counter_sync_boostCount of synchronous-boost throttle events.GaugeAll or Custom: Per GPU and Detailed
node_gpu_throttle_counter_sw_thermal_slowdownCount of software thermal slowdown events.GaugeAll or Custom: Per GPU and Detailed
node_gpu_throttle_counter_hw_thermal_slowdownCount of hardware thermal slowdown events.GaugeAll or Custom: Per GPU and Detailed
node_gpu_throttle_counter_hw_power_brake_slowdownCount of hardware power-brake slowdown events.GaugeAll or Custom: Per GPU and Detailed
gpu_serialBoard serial number.PropertyAll or Custom: Per GPU
driver_versionInstalled NVIDIA driver version.PropertyAll or Custom: Per GPU
pci_bus_idFull PCI bus identifier in domain:bus:device.function format.PropertyAll or Custom: Per GPU
pci_busPCI bus number.PropertyAll or Custom: Per GPU
pci_devicePCI device number on the bus.PropertyAll or Custom: Per GPU
pci_device_idPCI vendor and device ID.PropertyAll or Custom: Per GPU
persistence_modeWhether persistence mode is enabled.PropertyAll or Custom: Per GPU
pstateCurrent GPU performance state (P0 is maximum performance).PropertyAll or Custom: Per GPU
pci_domainPCI domain number.PropertyAll or Custom: Per GPU and Detailed
pci_base_classPCI base class code.PropertyAll or Custom: Per GPU and Detailed
pci_sub_classPCI sub-class code.PropertyAll or Custom: Per GPU and Detailed
pci_sub_device_idPCI subsystem device ID.PropertyAll or Custom: Per GPU and Detailed
vgpu_driver_cap_heterogenous_multi_vgpuWhether the driver supports heterogeneous multi-vGPU.PropertyAll or Custom: Per GPU and Detailed
vgpu_device_cap_fractional_multi_vgpuWhether the device supports fractional multi-vGPU.PropertyAll or Custom: Per GPU and Detailed
vgpu_device_cap_heterogeneous_time_slice_profileWhether the device supports heterogeneous time-slice profiles.PropertyAll or Custom: Per GPU and Detailed
vgpu_device_cap_heterogeneous_time_slice_sizesWhether the device supports heterogeneous time-slice sizes.PropertyAll or Custom: Per GPU and Detailed
vgpu_device_cap_homogeneous_placementsWhether the device supports homogeneous vGPU placements.PropertyAll or Custom: Per GPU and Detailed
vgpu_device_cap_mig_time_slicingWhether the device supports MIG time-slicing.PropertyAll or Custom: Per GPU and Detailed
vgpu_device_cap_mig_time_slicing_modeCurrent MIG time-slicing mode.PropertyAll or Custom: Per GPU and Detailed
display_modeWhether the display feature is enabled.PropertyAll or Custom: Per GPU and Detailed
display_attachedWhether a display is physically attached to this GPU.PropertyAll or Custom: Per GPU and Detailed
display_activeWhether a display is actively receiving output from this GPU.PropertyAll or Custom: Per GPU and Detailed
addressing_modeMemory addressing mode.PropertyAll or Custom: Per GPU and Detailed
accounting_modeWhether process accounting is enabled.PropertyAll or Custom: Per GPU and Detailed
driver_model_currentActive driver model; typically N/A on Linux.PropertyAll or Custom: Per GPU and Detailed
driver_model_pendingPending driver model after reboot; typically N/A on Linux.PropertyAll or Custom: Per GPU and Detailed
vbios_versionVideo BIOS version string.PropertyAll or Custom: Per GPU and Detailed
inforom_imgInfoROM image version.PropertyAll or Custom: Per GPU and Detailed
inforom_oemInfoROM OEM object version.PropertyAll or Custom: Per GPU and Detailed
inforom_eccInfoROM ECC object version.PropertyAll or Custom: Per GPU and Detailed
inforom_pwrInfoROM power management object version.PropertyAll or Custom: Per GPU and Detailed
inforom_checksum_validationInfoROM data integrity check result.PropertyAll or Custom: Per GPU and Detailed
gpu_recovery_actionRecommended recovery action following a GPU error.PropertyAll or Custom: Per GPU and Detailed
reset_status_reset_requiredWhether a GPU reset is required to clear an error condition.PropertyAll or Custom: Per GPU and Detailed
reset_status_drain_and_reset_recommendedWhether a drain-and-reset is the recommended recovery procedure.PropertyAll or Custom: Per GPU and Detailed
gom_currentCurrent GPU operation mode (All On, Compute, or Low DP).PropertyAll or Custom: Per GPU and Detailed
gom_pendingPending GPU operation mode, applied after the next reboot.PropertyAll or Custom: Per GPU and Detailed
clocks_throttle_reasons_supportedBitmask of clock throttle reasons supported by this GPU.PropertyAll or Custom: Per GPU and Detailed
clocks_throttle_reasons_activeBitmask of clock throttle reasons currently active.PropertyAll or Custom: Per GPU and Detailed
compute_modeCompute access mode (Default, Exclusive Thread, Prohibited, or Exclusive Process).PropertyAll or Custom: Per GPU and Detailed
compute_capCUDA compute capability in major.minor format.PropertyAll or Custom: Per GPU and Detailed
dram_encryption_mode_currentCurrent DRAM encryption mode.PropertyAll or Custom: Per GPU and Detailed
dram_encryption_mode_pendingPending DRAM encryption mode, applied after the next reboot.PropertyAll or Custom: Per GPU and Detailed
ecc_mode_currentCurrent ECC mode.PropertyAll or Custom: Per GPU and Detailed
ecc_mode_pendingPending ECC mode, applied after the next reboot.PropertyAll or Custom: Per GPU and Detailed
retired_pages_pendingWhether pending page retirements require a reboot to take effect.PropertyAll or Custom: Per GPU and Detailed
remapped_rows_pendingWhether pending row remappings require a reboot to take effect.PropertyAll or Custom: Per GPU and Detailed
remapped_rows_failureWhether a row remapping failure has been recorded.PropertyAll or Custom: Per GPU and Detailed
power_managementWhether power management is supported for this GPU.PropertyAll or Custom: Per GPU and Detailed
power_smoothing_supportedWhether delayed power smoothing is supported.PropertyAll or Custom: Per GPU and Detailed
mig_mode_currentCurrent MIG (Multi-Instance GPU) mode.PropertyAll or Custom: Per GPU and Detailed
mig_mode_pendingPending MIG mode, applied after the next reboot.PropertyAll or Custom: Per GPU and Detailed
gsp_mode_currentCurrent GSP firmware mode.PropertyAll or Custom: Per GPU and Detailed
gsp_mode_defaultDefault GSP firmware mode.PropertyAll or Custom: Per GPU and Detailed
c2c_modeCurrent chip-to-chip interconnect (C2C) mode.PropertyAll or Custom: Per GPU and Detailed
fabric_stateNVLink fabric state.PropertyAll or Custom: Per GPU and Detailed
fabric_statusNVLink fabric status.PropertyAll or Custom: Per GPU and Detailed
fabric_clique_idNVLink fabric clique identifier.PropertyAll or Custom: Per GPU and Detailed
fabric_cluster_uuidNVLink fabric cluster unique identifier.PropertyAll or Custom: Per GPU and Detailed
fabric_health_summaryOverall NVLink fabric health summary.PropertyAll or Custom: Per GPU and Detailed
fabric_health_bandwidthNVLink fabric bandwidth health status.PropertyAll or Custom: Per GPU and Detailed
fabric_health_route_recovery_in_progressWhether NVLink fabric route recovery is in progress.PropertyAll or Custom: Per GPU and Detailed
fabric_health_route_unhealthyWhether any NVLink fabric routes are unhealthy.PropertyAll or Custom: Per GPU and Detailed
fabric_health_access_timeout_recoveryWhether access-timeout recovery is in progress on the NVLink fabric.PropertyAll or Custom: Per GPU and Detailed
fabric_health_incorrect_configurationWhether the NVLink fabric has an incorrect configuration.PropertyAll or Custom: Per GPU and Detailed
fabric_health_partition_assignedWhether the NVLink fabric partition has been assigned.PropertyAll or Custom: Per GPU and Detailed
platform_chassis_serial_numberSerial number of the chassis this GPU is installed in.PropertyAll or Custom: Per GPU and Detailed
platform_slot_numberChassis slot number for this GPU module.PropertyAll or Custom: Per GPU and Detailed
platform_tray_indexTray index within the chassis for this GPU module.PropertyAll or Custom: Per GPU and Detailed
platform_host_idHost identifier in the platform topology.PropertyAll or Custom: Per GPU and Detailed
platform_peer_typeType of the platform peer connection.PropertyAll or Custom: Per GPU and Detailed
platform_module_idModule identifier within the platform.PropertyAll or Custom: Per GPU and Detailed
platform_gpu_fabric_guidNVLink fabric GUID assigned to this GPU in the platform topology.PropertyAll or Custom: Per GPU and Detailed
hostnameHostname of the system as reported by nvidia-smi.PropertyAll or Custom: Per GPU and Detailed
timestampCollection timestamp fetched from nvidia-smi. Used internally to maintain CSV column alignment; not emitted on any event.QueryBasic
countTotal number of GPUs returned by nvidia-smi. Produces the node_gpu_count gauge on the aggregated event; not emitted as a separate field.QueryBasic