On This Page

Home / StreamAI/StreamAI Troubleshooting

StreamAI Troubleshooting ​

Find out why StreamAI rejected a request, and fix common key, model, usage, and trace export problems. For an overview of StreamAI, see Learn About StreamAI.

Diagnose a Failed Request ​

The gateway checks each request in this order: the key, the request body, and then the model. It returns the first failure, and a rejected request never reaches the model. To find the cause:

  1. Read the HTTP status code of the response.
  2. Read error.type and error.message in the response body.
  3. Find the status in the following table.
Statuserror.typeCauseWhat to do
400invalid_request_errorThe request body isn’t a JSON object, or it doesn’t include a non-empty model string. The same error also happens when the Content-Encoding header names an unsupported encoding or the compressed body is corrupt. It also happens when the body is larger than the limit for the endpoint after decompression.Send a JSON object that includes model. If you compress the body, use gzip, x-gzip, deflate, or br, and make sure that the decompressed size is within the limit. See Request Size Limits.
401invalid_api_key on OpenAI endpoints, authentication_error on /v1/messagesThe key is missing, not valid, revoked, or expired. The response doesn’t say which.Send the key in the Authorization: Bearer <key> or x-api-key header. Then check the key’s status on the Keys page. See Keys and Access Problems.
403permission_denied on OpenAI endpoints, permission_error on /v1/messagesThe request names a model other than cribl/auto. StreamAI rejects any other model.Send cribl/auto as the model. See Auto Mode.
404model_not_found on OpenAI endpoints, not_found_error on /v1/messagesThe request names a model that StreamAI doesn’t serve.Send cribl/auto as the model. See Auto Mode.
404model_not_found on OpenAI endpoints, not_found_error on /v1/messagesThe request names cribl/auto, and StreamAI has no model available for cribl/auto right now.Try the request again later. If the error continues, contact Cribl Support.
404None. The body is {"message": "not found"}.The path isn’t a gateway endpoint.Check the base URL. See Find the Gateway URL.
413ai_gateway_request_too_largeThe request body is larger than the limit for the endpoint.Send a smaller request. See Request Size Limits.
429VariesYour Organization exceeded its request limit.Wait, and then retry.
500ai_gateway_request_errorThe gateway failed to process the request.Try the request again. If the error continues, contact Cribl Support.
501ai_gateway_upstream_endpoint_unavailableThe model provider doesn’t offer this endpoint.Use a different endpoint.
502ai_gateway_upstream_error or ai_gateway_upstream_authThe request to the model provider failed, or the gateway couldn’t get credentials for it.Try the request again. If the error continues, contact Cribl Support.
503ai_gateway_busyThe gateway is holding the maximum amount of request data.Try the request again after a short wait.
504ai_gateway_upstream_timeoutThe model provider sent nothing for the rest of the request’s 120-second time budget. The gateway spends part of the budget on reading the request body, and any fallback to another model in auto mode shares the same budget. The budget is an idle timeout, so a response that keeps sending data isn’t cut off at 120 seconds.Send a smaller request, or use streaming so that the model starts to respond sooner.

Error Response Format ​

When the gateway rejects a request, it returns an HTTP error status and a JSON body in the format that the endpoint’s API uses:

  • /v1/chat/completions and /v1/responses: {"error": {"message": "...", "type": "..."}}
  • /v1/messages: {"type": "error", "error": {"message": "...", "type": "..."}}

/v1/messages wraps the error with "type": "error", except for the 500, 501, and 502 errors and the 504 errors that happen after the gateway has read the request body. Those errors use the same bare error object on every endpoint.

If the model provider stalls after the gateway has sent the response headers, the gateway can’t return a 504 error. It drops the connection, so the client sees a response that’s cut off.

Errors From the Model ​

When the model returns an error, the gateway relays the response to the client with the status code and body that it received. The error can come from the model provider or from Cribl’s AI service. You can therefore see statuses that the table doesn’t list, and a relayed error can have a body that differs from the table’s row for the same status. These errors don’t come from the gateway itself.

For cribl/auto, if a model answers with a 404, 429, or 503 status before the gateway sends anything to the client, the gateway tries the next model. The client sees only the last answer. See Auto Mode.

You can’t set a usage limit on a key. Cribl applies a request limit to each Organization. When your Organization exceeds it, StreamAI returns a 429 error. Wait, and then retry the request. See Limits of Usage Data.

Request Size Limits ​

The gateway reads the whole request body before it sends the request to the model, and it rejects a body that exceeds the limit for the endpoint with a 413 status. The limit applies to the body as the client sends it, so StreamAI checks a compressed body at its compressed size, before it decompresses the body. The decompressed body must also fit within the same limit. A compressed body that exceeds the limit after decompression returns a 400 status, not a 413.

EndpointMaximum request body
/v1/chat/completions1 MiB
/v1/messages4.5 MB
/v1/responses4.5 MB

Cribl’s AI service rejects any request body larger than 4.5 MB, so this is the effective limit for /v1/messages and /v1/responses. The AI service returns a 413 status for a larger body, and the gateway relays that response. The error body then differs from the ai_gateway_request_too_large body that the gateway returns itself. See Errors From the Model.

Fix Common Problems ​

Check That the Gateway Is Running ​

Send a GET request to /health on the gateway host. The endpoint doesn’t need a key. A running gateway returns 200 and {"status":"OK"}.

Don’t use GET /system/ai-gateway as a health check. Its status is always unverified. See Get Gateway Status.

Keys and Access Problems ​

ProblemCauseWhat to do
A new key returns 401.A new key typically starts to authenticate requests within about 30 seconds.Wait, and try again.
A key that worked returns 401.The key expired or someone revoked it. A key expires 30 days after creation unless you set a different expiration.Check the key’s Status on the Keys page. The page shows both an expired key and a revoked key as Revoked. To tell them apart, compare the key’s expiresAt value with the current time. Create a replacement key. See Add a Key.
A revoked key still works for a short time.A revoked key typically stops authenticating requests within about 30 seconds.Wait, and try again. See Revoke a Key.
A request to the key or settings API returns 403.The caller doesn’t have the role on the StreamAI Worker Group that the request needs. Listing keys needs at least the Read Only role. Creating, editing, or revoking a key needs the Editor or Admin role.Ask an administrator to give you the role. See Permissions.
A request to create a key returns 409.The Workspace reached the limit of active keys.Revoke a key that you no longer need, or wait for one to expire. See Get Started with StreamAI.
The enabled property of a key in the REST API doesn’t reflect expiration, so an expired key still shows enabled: true.

Usage and Cost Problems ​

ProblemCauseWhat to do
You can’t select 7 days, 14 days, or 30 days on the Overview page.The Overview page locks a time window that’s longer than the System Insights retention. System Insights keeps 2 days of data by default.Use 24 hours, or extend the System Insights retention. See Time Windows and Retention.
The cost on the Overview page or in the usage API is lower than expected.StreamAI can’t price some or all of the requests. A request without a price adds nothing to the cost.See How StreamAI Estimates Cost.
A client received a 403 or 404, but the key’s errors counter didn’t change.StreamAI doesn’t count requests that the gateway rejects because of the key, the request body, or the model.See What StreamAI Records.
Usage appears under __overflow or __nokey.__overflow holds requests that a Worker could no longer attribute to a key, because it reached its limit on tracked keys. __nokey is normally empty. A request without a client key gets a 401 before StreamAI records usage, so __nokey holds only a usage record that has an empty key ID.See Usage Response.
A recent request is missing from the usage data.Workers report usage once a minute, so a request doesn’t appear right away. The Leader groups usage into 10-minute buckets, and the bucket in progress is incomplete. In the rolling 24-hour view, the last bucket of the current window is incomplete. In the daily history, today’s bucket is incomplete while the day is running. The previous window, and a history range that ends before today, have only complete buckets.Wait, and check again. The partial field of the response lists the incomplete buckets.
A usage request returns 403.The caller needs an administrator role (Organization Admin, Organization Owner, Workspace Admin, or Workspace Owner) and at least the Read Only role on the StreamAI Worker Group.See Permissions.
A usage request returns 503.The usage metrics store failed or didn’t answer in time. For usage/history and window=previous, StreamAI reads System Insights, which might not be enabled, might not answer within 5 seconds, or might return data that StreamAI can’t read. The message is AI Gateway usage history is unavailable.Try the request again later. See Usage Status Codes.
GET /system/ai-gateway/usage/limits returns 503.StreamAI has no consumption ledger. The message is AI Gateway consumption ledger is unavailable.No action. The endpoint always returns this error.
A usage history request returns no data for older days.System Insights keeps data for 2 days by default, and StreamAI omits the days that it no longer holds.See Usage History and System Insights Retention.
A usage request returns 400.fields includes an unknown counter, keyId has more than 100 values, window is unknown or repeated, a date is malformed or out of range, or the request has a model parameter.See Get Rolling 24-Hour Usage and Get Daily Usage History.

Trace Export Problems ​

StreamAI doesn’t test whether the trace receiver is reachable when you save the setting, and the gateway drops a span that it can’t deliver. If the receiver doesn’t get spans, check these items:

  • The traceReceiverUrl value is the base URL of the receiver. It doesn’t include credentials, a query string, a fragment, or the /v1/traces path, which StreamAI appends. In Cribl.Cloud, the hostname must resolve to a publicly routable address.
  • The PATCH request included sampleRate. PATCH replaces the settings, so omitting sampleRate resets it to 1, which captures every inference.
  • The deployment to the StreamAI Worker Group finished. A successful update confirms that StreamAI stored the value and started the deployment. It doesn’t confirm that every Worker applies the setting. A 503 response means that StreamAI couldn’t store the value or start the deployment.
  • The receiver accepts OpenTelemetry (OTLP) traces over HTTP, at the /v1/traces path, without authentication. The gateway doesn’t send authentication headers.
  • The request that you expect a span for wasn’t refused. The gateway doesn’t export spans for requests that it refuses with a 401, 403, or 404 response.

For the settings and their requirements, see Manage Trace Export Settings and Export Inference Traces.

Provisioning Problems ​

Provisioning the AI Gateway can take several minutes. To check progress with the API, send a GET request to /products/stream-ai every 15 to 30 seconds. The status of the gateway is provisioning, ready, or degraded. provisioning means that no Worker has connected yet. ready means that at least one Worker is healthy. degraded means that Workers are connected, but none is healthy. An empty items array means that the gateway doesn’t exist. See Provision by Using the API.