How the OTel Source Transforms Trace Events

​

This Function can process only events coming from the Cribl Stream OTel Source, which always ingests events in the form of OTLP trace protocol buffers (protobufs). These protobufs are structures that nest objects within other objects. Each OTLP trace protobuf contains top-level ResourceSpans objects that associate an OTLP Resource with one or more spans.

How This Function Batches Traces

​

For OTLP traces, batching means grouping extracted spans that arrive within a specified time window and whose OTLP Resource Attributes are identical. The Function will create separate batches for each unique combination of resource attributes and metadata keys.

Usage

​

Ideally, this Function should process events and/or traces after other Functions have formatted them. Therefore, Cribl recommends that you place this Function last in the Pipeline.

Filter: Filter expression (JS) that selects data to feed through the Function. The default true setting passes all events through the Function.

Description: Simple description of this Function’s purpose in this Pipeline. Defaults to empty.

Final: If toggled to Yes, stops feeding data to the downstream Functions. Defaults to No.

Drop non-trace events: Whether or not to drop any non-trace events. Defaults to No.

OTLP version: Specifies the version of the OTLP protocol used for formatting and sending output events downstream.

• Version 0.10.0 (Default): This version uses the older OTLP format. It is recommended for users who need compatibility with legacy systems.
• Version 1.3.1: This version supports newer OTLP features, including scope_spans instead of instrumentation_library_spans, and additional fields like flags in Link and Span objects. It is recommended for users who want to leverage the latest OTLP features and improvements.

Batch OTLP traces: When set to Yes, the Function batches together extracted spans that have the same resource and (optionally) metadata keys until they reach a flush criteria. When set to No, the Function will emit extracted/unextracted span events that have been normalized, but they will not be combined with other spans - each event passed to the Function will result in one event being returned.

When set to Yes, exposes these additional controls:

• Batch size: The number of spans after which a batch will be sent, regardless of whether the timeout duration has been reached.
• Batch timeout (ms): Time duration after which a batch will be sent regardless of size. Defaults to 200 milliseconds.
• Batch size limit (kb): Maximum batch size in kilobytes. 0 means no limit.
• Batch trace metadata keys: Optionally, add metadata keys that you want the Function to create batcher instances for, in addition to any batching based on resource attributes. When set, this processor will create one batcher instance per distinct combination of values in the metadata.
• Metadata cardinality limit: Limits the number of unique combinations of metadata key values that the batcher will process. Once this limit is reached, events with new combinations of metadata key values will be dropped. Defaults to 1000.

Example

​

This example demonstrates how the OTLP Traces Function processes and batches trace events when the Batch OTLP traces setting is enabled. The settings used in this example are:

• Batch OTLP traces: Yes
• Batch size: 2
• Batch timeout (ms): 200
• Batch size limit (kb): 0 (no limit)
• Batch trace metadata keys: None
• Metadata cardinality limit: 1000
• Drop non-trace events: Yes

Incoming Event 1

​

{
  "resource": {
    "attributes": {
      "service.name": "currencyservice",
      "service.instance.id": "instance-1"
    }
  },
  "scope": {
    "name": "currencyservice",
    "version": "1.0.0"
  },
  "spans": [{
    "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
    "span_id": "00f067aa0ba902b7",
    "name": "GetSupportedCurrencies",
    "start_time_unix_nano": 1616587310000000000,
    "end_time_unix_nano": 1616587311000000000,
    "attributes": {
      "http.method": "GET",
      "http.url": "/currencies"
    }
  }]
}

Incoming Event 2

​

{
  "resource": {
    "attributes": {
      "service.name": "currencyservice",
      "service.instance.id": "instance-1"
    }
  },
  "scope": {
    "name": "currencyservice",
    "version": "1.0.0"
  },
  "spans": [{
    "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
    "span_id": "00f067aa0ba902b8",
    "name": "ConvertCurrency",
    "start_time_unix_nano": 1616587312000000000,
    "end_time_unix_nano": 1616587313000000000,
    "attributes": {
      "http.method": "POST",
      "http.url": "/convert"
    }
  }]
}

Outgoing Batched Event

​

{
  "resource": {
    "attributes": [
      {
        "key": "service.name",
        "value": {
          "string_value": "currencyservice"
        }
      },
      {
        "key": "service.instance.id",
        "value": {
          "string_value": "instance-1"
        }
      }
    ]
  },
  "scope_spans": [
    {
      "scope": {
        "name": "currencyservice",
        "version": "1.0.0"
      },
      "spans": [
        {
          "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
          "span_id": "00f067aa0ba902b7",
          "name": "GetSupportedCurrencies",
          "start_time_unix_nano": 1616587310000000000,
          "end_time_unix_nano": 1616587311000000000,
          "attributes": [
            {
              "key": "http.method",
              "value": {
                "string_value": "GET"
              }
            },
            {
              "key": "http.url",
              "value": {
                "string_value": "/currencies"
              }
            }
          ]
        },
        {
          "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
          "span_id": "00f067aa0ba902b8",
          "name": "ConvertCurrency",
          "start_time_unix_nano": 1616587312000000000,
          "end_time_unix_nano": 1616587313000000000,
          "attributes": [
            {
              "key": "http.method",
              "value": {
                "string_value": "POST"
              }
            },
            {
              "key": "http.url",
              "value": {
                "string_value": "/convert"
              }
            }
          ]
        }
      ]
    }
  ]
}