Monitor StreamAI Usage
StreamAI records usage for each client key. You can see the usage in three places:
| Where | What it shows |
|---|---|
| The Overview page | Tokens, requests, and estimated cost for the whole Workspace or for one key, for a time window of 24 hours, 7 days, 14 days, or 30 days. See View Usage on the Overview Page. |
| The usage API | Usage in 10-minute buckets for 24 hours, or in daily buckets for up to 35 days, with request counts, errors, tokens, estimated cost, and request duration for each key. See Read Usage. |
| Trace export | One OpenTelemetry span for each completed inference, sent to a receiver that you configure. See Export Inference Traces. |
View Usage on the Overview Page
The Overview page shows the StreamAI usage for your Workspace. It reads the same usage data as the usage API. It reports what the StreamAI Workers recorded, and it isn’t a billing record.
To open the page, on the top bar, select Products, and then select StreamAI. When the gateway is ready, the Overview page opens. If you haven’t provisioned the gateway, StreamAI shows the welcome page instead. See Get Started with StreamAI. To return to the page from the Keys page, select Overview in the StreamAI navigation.
The page has these controls in its header:
| Control | Description |
|---|---|
| Keys | Select All Keys, or select one key to show only that key’s usage. |
| Time window | Select 24 hours, 7 days, 14 days, or 30 days. StreamAI remembers your choice in the browser. See Time Windows and Retention. |
| Edit Connection and AI Observability App | Edit Connection opens the connect modal, and AI Observability App opens the App. Both appear only after you connect the App and while the App is installed. See Connect the AI Observability App. |
The page has the following sections.
- KPI cards: Tokens used, Requests, and Total cost (projected) for the selected window. A card also shows the change from the previous period of the same length, when StreamAI has usage data for that period. A 24-hour window and a 30-day window have no previous period to compare with.
- Tokens: Avg. Tokens per request, and a chart of token usage over the window.
- Key: A Top keys ranking that you can order by Cost, Tokens, or Requests, and the Active keys count. The ranking and the count include only the keys that recorded usage in the window.
- Cost: Avg. Cost per Request, and a chart of cumulative spend over the window. For a window of 7 or 14 days, the chart also shows the previous period.
The Cost section starts with an alert that says StreamAI currently supports automatic smart routing at no additional cost for Cribl AI features, Apps, and agents built on the Cribl platform. The costs on the page reflect actual inference costs for visibility, and Cribl doesn’t bill them to your account. See How StreamAI Estimates Cost.
Time Windows and Retention
The Overview page reads System Insights for windows longer than 24 hours, so it offers a window only when the System Insights retention covers it. System Insights keeps 2 days of data by default, so by default only 24 hours is available. The 7 days, 14 days, and 30 days options stay in the list but you can’t select them. In Cribl.Cloud, an Upgrade Insights link appears next to the list when a window is unavailable. The link opens the System Insights tab of your Workspace’s details page in the Cribl.Cloud portal. If the retention becomes shorter than the window that you selected, the page switches to the longest available window.
Connect the AI Observability App
If you haven’t connected the AI Observability App, the Overview page shows the banner Need more data? Connect StreamAI to the AI Observability App to collect full metrics. See Docs for more details. To connect the App:
- In the banner, select Connect AI Observability.
- In the Connect AI Observability App modal, install the AI Observability App in your Workspace from Cribl Marketplace, if you haven’t already. Select AI Observability App in the modal to open the App’s Marketplace page.
- Open the App, and configure it to use OpenTelemetry (OTEL).
- Copy the Destination URL from the App’s setup.
- In the modal, enter the URL in the Destination URL field, and select Connect.
StreamAI saves the URL as the traceReceiverUrl of the trace export settings, so the gateway starts to export a span for every inference to the receiver.
If the AI Observability App is installed, StreamAI shows the message AI Observability App is connected. with an Open AI Observability link, the banner disappears, and Edit Connection and AI Observability App appear in the header. If the App isn’t installed, StreamAI still saves the URL and shows Destination URL saved, but the page keeps the banner. The page treats the App as connected only while the App is installed, even if the saved URL is in the settings.
The modal says that connecting StreamAI to the App lets you review data from the past 30 days, monitor Organization-wide usage, and investigate suspicious activity.
For more information about collecting and analyzing LLM telemetry, see LLM Observability.
Export Inference Traces
The gateway can capture each completed inference and export it as one OpenTelemetry (OTLP) span to a Cribl OpenTelemetry Source that you configure. Each inference is the root span of its own trace. The span includes the request’s system instructions, input messages, and output messages, along with the model and usage details. Because the spans can contain your users’ prompts and the models’ responses, send them only to a receiver that you trust.
To turn on the export, set these properties in the gateway’s trace export settings. For the requests that read and change the settings, see Manage Trace Export Settings. You can also connect the AI Observability App from the Overview page. See View Usage on the Overview Page.
| Property | Description |
|---|---|
traceReceiverUrl | The base URL of the receiver, as an absolute http or https URL. Don’t include the /v1/traces path, which the gateway appends. The URL can’t include credentials, a query string, or a fragment. In Cribl.Cloud, the hostname must resolve to a publicly routable address. Set the value to null to turn the export off. |
sampleRate | The fraction of completed inferences to capture, from 0 (none) to 1 (all). If you omit it, the gateway captures every inference. |
The gateway doesn’t export a span for a request that it refuses with a 401, 403, or 404 response.
The gateway doesn’t send authentication headers to the receiver. Configure the receiver’s OpenTelemetry Source to accept unauthenticated OTLP requests. In Cribl.Cloud, use an address that’s publicly routable.
StreamAI doesn’t test whether the receiver is reachable when you save the setting. If the receiver is blocked or fails, the gateway drops the span. The inference itself isn’t affected. The gateway captures at most 256 KiB of response text and tool arguments for each inference. It leaves out the content of a request body that’s larger than 256 KiB. When the gateway leaves content out, it sets the span’s cribl.stream_ai.content_truncated attribute to true.
What a Span Contains
The gateway sends the spans in OTLP version 1.3.1 protobuf format over HTTP to the /v1/traces path of the receiver. If the receiver uses TLS, the gateway verifies its certificate. Each span has these properties:
| Property | Value |
|---|---|
| Name | chat <model>, where <model> is the model that the request names, such as chat cribl/auto. |
| Kind | Client. |
| Status | Unset for a successful inference. Error for a failed inference, with the error type as the message. |
| Resource | The service.name attribute is stream-ai. |
| Scope | cribl.stream_ai. |
A span has these attributes. A span includes an attribute only when StreamAI has a value for it.
| Group | Attributes |
|---|---|
| Model and usage | gen_ai.operation.name (always chat), gen_ai.provider.name, gen_ai.request.model, gen_ai.response.model, gen_ai.response.finish_reasons, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.usage.cache_read.input_tokens, gen_ai.usage.cache_creation.input_tokens, and gen_ai.usage.reasoning.output_tokens. |
| Key and user | cribl.stream_ai.client_key.id and cribl.stream_ai.client_key.name identify the client key that sent the request. user.id, enduser.id, user.full_name, user.email, and gen_ai.conversation.id come from the identity headers in the next table. user_agent.original is the client’s User-Agent header. |
| Request outcome | http.response.status_code, error.type, cribl.stream_ai.request.id, cribl.stream_ai.upstream.model, cribl.stream_ai.estimated_cost.usd, cribl.stream_ai.estimated_cost.basis, cribl.stream_ai.duration_ms, cribl.stream_ai.ttft_ms, cribl.stream_ai.streaming, cribl.stream_ai.usage_complete, and cribl.stream_ai.content_truncated. |
| Content | gen_ai.system_instructions, gen_ai.tool.definitions, gen_ai.input.messages, and gen_ai.output.messages. |
The cribl.stream_ai.upstream.model attribute is the model name that StreamAI sent the request to. For a cribl/auto request, it identifies the model that answered.
The cribl.stream_ai.estimated_cost.basis attribute says how StreamAI priced the request, or why it didn’t:
list_price: StreamAI priced the request at the list price of the model that answered. Only this value comes withcribl.stream_ai.estimated_cost.usd.usage_unreported: The response didn’t report any token counts, so there’s nothing to price.upstream_model_unreported: StreamAI recorded no model that it sent the request to, so it has no price to look up.upstream_model_unpriced: The model catalog has no list price for the model that StreamAI sent the request to.over_cost_cap: The estimated cost is more than $10,000, so StreamAI leaves it out.
When StreamAI can’t calculate a cost, it leaves the cost out instead of reporting 0. A request without a cost has no cribl.stream_ai.estimated_cost.usd attribute.
A client can send these optional headers to say who a request is for:
| Header | Span attribute |
|---|---|
x-cribl-session-id | gen_ai.conversation.id |
x-cribl-username | user.id and enduser.id |
x-cribl-user-full-name | user.full_name. Percent-encode the value as UTF-8, as encodeURIComponent does, so that a name outside ASCII survives a header. |
x-cribl-user-email | user.email. Percent-encode the value in the same way. |
StreamAI records these values as the client sends them and doesn’t verify them. It ignores a value that’s blank, longer than 256 characters, or that contains a control character. The gateway doesn’t forward these headers to the model provider. When Cribl AI uses StreamAI as its provider, Cribl AI sends the headers with the signed-in user’s identity.
How StreamAI Estimates Cost
The cost that StreamAI reports is an estimate at list prices, not a bill. It appears as the cost counter in the usage API and as the cost on the Overview page. For each request, StreamAI multiplies the token counts that the model provider reports by the list price of the exact model that answered. It uses the prices in the model snapshot that the request was admitted against.
For a request to cribl/auto, StreamAI prices the request at the model that answered, not at cribl/auto. This matters when auto mode falls back to another model.
StreamAI can’t price a request, and leaves its cost out instead of counting it as zero, when any of these is true:
- The request has no upstream model.
- StreamAI has no price for the model that answered, or more than one model shares the model’s upstream ID.
- The model provider reported no token counts.
- The request’s estimated cost exceeds $10,000.
When some requests are unpriced, the cost that StreamAI reports is a minimum. Cached input tokens that the provider includes in the input count are priced once, at the cache-read rate, and not again at the input rate.
What StreamAI Records
StreamAI counts usage for each client key, as the usage API reports it. StreamAI reports usage by key only. It doesn’t report usage for each model. In a usage response, a counter that has no activity in a bucket has the value 0, and a bucket in which every requested counter is 0 is left out.
| Counter | What it counts |
|---|---|
requests | The number of requests that StreamAI recorded a result for. It doesn’t include requests that the gateway rejected because of the key, the request body, or the model. |
errors | The number of requests that failed. A request fails if the client disconnected before the response finished, the response status isn’t 2xx, or the provider reported a failure in a 2xx response. |
tokens_input | The input tokens that the provider reported. |
tokens_output | The output tokens that the provider reported. Reasoning tokens are already part of this count. |
tokens_cache_read | The tokens that the provider read from its prompt cache. |
tokens_cache_write | The tokens that the provider wrote to its prompt cache. |
tokens_total | The provider’s total or, when the provider reports none, the sum of the reported token counts. Per-key token limits use this count, but StreamAI doesn’t enforce per-key limits. |
cost | The estimated cost at list prices, in micro-USD. It isn’t a bill. See How StreamAI Estimates Cost. |
unpriced_requests | The number of requests that reported tokens but have no cost, because StreamAI has no price for the model or the cost is over the cap. While it’s above 0, cost is a lower bound. |
| Request duration | The time from when the gateway starts to handle a request until the response finishes. The usage API reports the sum of the durations and the number of requests, so you can calculate the mean. |
For the query parameters and the response format, see Read Usage.
How StreamAI Collects Usage
- The gateway Workers count each finished request for the key that sent it.
- Each Worker reports its counts to the Cribl.Cloud Leader once a minute.
- The Leader merges the counts into 10-minute buckets. The usage API returns the most recent 24 hours, as 144 buckets.
- System Insights stores the counts for longer. The usage API reads it for the daily history, and for the 24 hours before the most recent 24 hours when a request sets
window=previous. The Overview page reads it only for the 7 days, 14 days, and 30 days windows.
If System Insights isn’t enabled, a usage API request for the daily history or for window=previous returns 503. See Usage History and System Insights Retention.
A Worker tracks up to 1,000 distinct keys. If more keys send requests, the Worker counts the extra requests under __overflow. The usage data is an estimate of what the Workers recorded. It isn’t a billing record.
Limits of Usage Data
- No usage limits: You can’t set a usage limit on a key. Cribl applies a request limit to each Organization, and requests over the limit return a
429error.GET /system/ai-gateway/usage/limitsalways returns503. See Read Usage. - History depends on System Insights retention: The usage API reports daily history for up to 35 days, but only for the days that System Insights still holds, which is 2 days by default. The Overview page offers a window only when the retention covers it. See Time Windows and Retention.
- No usage by model: StreamAI reports usage by key only.