On This Page

Home / Cribl AI/ Custom AI Providers/Configure an OpenAI-Compatible Endpoint for Cribl AI

Configure an OpenAI-Compatible Endpoint for Cribl AI ​

Use this topic to connect any endpoint that implements the OpenAI Chat Completions API as a Bring Your Own Model (BYOM) provider for Cribl AI. This provider type covers self-hosted inference servers (such as vLLM, Ollama with an OpenAI adapter, or FastChat), third-party API gateways, and direct access to the OpenAI API. This page covers:

  • What you need from your endpoint.
  • How to fill in each step of the Cribl configuration wizard.

For the generic wizard flow, prerequisites, and how to edit or delete a configured provider, see Configure Custom AI Providers.

Prerequisites ​

In addition to the general prerequisites, you need:

  • A running endpoint that accepts POST /chat/completions requests in OpenAI format.
  • A Deployment URL for the endpoint (for example, https://vllm.internal.example.com).
  • An API key or bearer token for authentication.
  • The specific model IDs exposed by your endpoint that you plan to assign to the small, frontier, and reasoning tiers.
  • On-prem only: Network connectivity from the Leader to your endpoint.

Model IDs ​

Cribl does not maintain a supported model list for OpenAI-compatible endpoints. You supply the model IDs yourself in the wizard, and they must match what your endpoint expects in the model field of Chat Completions requests.

Examples of valid model IDs depending on your backend:

  • A Hugging Face model name: meta-llama/Llama-3-8B-Instruct
  • An internally assigned alias: small-model, frontier-model
  • An OpenAI model name (if using the OpenAI API directly): gpt-4o, o4-mini

Make sure each model ID you assign is available on your endpoint before testing. Requests sent to an unavailable model will fail.

For details on what each tier is used for, see Model IDs and Feature Routing.

Step 1: Select the Provider Type ​

Navigate to your AI Settings and open the configuration wizard:

  1. Select Use Custom AI Provider (or Try it!).
  2. Select the OpenAI Compatible tile.
  3. Select Continue.

Step 2: Enter Connection Details ​

Provide the credentials Cribl needs to reach your endpoint.

  • ID: Enter a short, unique identifier (for example, vllm-prod). Avoid putting secrets or keys in this field.
  • Description: Enter a human-readable label for the provider card (for example, vLLM cluster - production). This field is optional.
  • Deployment URL: Enter the base URL of your OpenAI-compatible endpoint.

Include the https:// prefix. Do not include trailing slashes or path suffixes such as /chat/completions – Cribl appends these automatically.

  • API Key: Enter the API key or bearer token required by your endpoint. If your endpoint does not enforce authentication, enter a placeholder value such as none.

Select Continue.

Step 3: Assign Models to Tiers and Test ​

Assign one model ID to each tier. For OpenAI-compatible endpoints, the wizard does not restrict or suggest models – you enter any model ID your endpoint accepts.

  1. For the Small tier, enter the model ID to use for lightweight tasks.
  2. For the Frontier tier, enter the model ID to use for multi-step agent tasks.
  3. For the Reasoning tier, enter one or more model IDs for extended-context tasks. The first model in the list serves as the system default. While most features exclusively use this first model, Cribl Search investigations allow you to switch between any listed models via a dropdown menu.
  4. Select Test and Save.

Cribl tests each assigned model by sending a request through your configured endpoint:

  • A results panel shows how many models passed and failed.
  • If any test fails, verify that:
    • The Deployment URL points to your running server and is reachable from the Cribl Leader.
    • The API Key is valid and authorized.
    • Each model ID is available on your endpoint.
  • Select Retry to test again after making corrections.
  • When all models pass, the configuration is saved automatically.

After you save:

  • A Custom AI provider card appears at the top of AI Settings, showing your ID, description, OpenAI-compatible as the provider type, and the number of configured models.
  • Supported Cribl AI capabilities in that Workspace begin routing through your endpoint.

Verify Cribl AI Behavior ​

To confirm that your OpenAI-compatible provider is correctly configured:

  1. In AI Settings, confirm that the Custom AI provider card lists OpenAI-compatible.
  2. Use a supported Cribl AI capability (for example, the KQL assistant or Git commit message suggestions).
  3. Verify that:
    • Requests succeed without provider-related errors.
    • Latency and behavior align with your expectations for the models your endpoint serves.

If you see failures or unexpected behavior:

  • Check your endpoint server logs for errors related to the model IDs or authentication.
  • Select the gear (⚙️) on the provider card and select Edit to reopen the wizard, then use Test and Save to review error details.
  • Confirm that the Cribl Leader has uninterrupted network access to the endpoint.

Change or Stop Using an OpenAI-Compatible Provider ​