On This Page

Home / Cribl AI/ Custom AI Providers/Configure LiteLLM for Cribl AI

Configure LiteLLM for Cribl AI ​

Use this topic to connect a self-hosted LiteLLM proxy as a Bring Your Own Model (BYOM) provider for Cribl AI. LiteLLM is an open-source proxy that normalizes requests across hundreds of LLM backends (OpenAI, Anthropic, Bedrock, Ollama, and others) using a single OpenAI-compatible API. This page covers:

  • What you need from the LiteLLM side.
  • How to fill in each step of the Cribl configuration wizard.

For the generic wizard flow, prerequisites, and how to edit or delete a configured provider, see Configure Custom AI Providers.

Prerequisites ​

In addition to the general prerequisites, you need:

  • A running LiteLLM proxy instance reachable from your Cribl deployment.
  • A Deployment URL for the proxy (for example, https://litellm.internal.example.com).
  • An API key configured on your LiteLLM proxy for authentication (LITELLM_MASTER_KEY or a virtual key).
  • Model names configured on the proxy that you will assign to the small, frontier, and reasoning tiers.
  • On-prem only: Network connectivity from the Leader to your LiteLLM proxy endpoint.

Model IDs ​

Unlike Bedrock and Foundry, Cribl does not maintain a supported model list for LiteLLM. You supply the model IDs yourself in the wizard, and they must match what your LiteLLM proxy exposes.

LiteLLM uses a model field in each request. The model IDs you enter in the wizard are passed directly to the proxy. Common patterns include:

  • A native provider path: bedrock/anthropic.claude-3-5-sonnet-20241022-v2:0
  • An alias defined in config.yaml: my-sonnet, my-haiku
  • An OpenAI-style name: gpt-4o

Make sure that each model ID you assign is listed in your LiteLLM config.yaml or is otherwise reachable through the proxy. Requests sent to an undefined model will fail.

For details on what each tier is used for, see Model IDs and Feature Routing.

Step 1: Select the Provider Type ​

Navigate to your AI Settings and open the configuration wizard:

  1. Select Use Custom AI Provider (or Try it!).
  2. Select the liteLLM tile.
  3. Select Continue.

Step 2: Enter Connection Details ​

Provide the credentials Cribl needs to reach your LiteLLM proxy.

  • ID: Enter a short, unique identifier (for example, litellm-prod). Avoid putting secrets or keys in this field.
  • Description: Enter a human-readable label for the provider card (for example, LiteLLM - production proxy). This field is optional.
  • URL: Enter the base URL of your LiteLLM proxy.

Include the https:// prefix. Do not include trailing slashes or path suffixes such as /chat/completions – Cribl appends these automatically.

  • Virtual key: Enter the LITELLM_MASTER_KEY or a virtual key with access to the models you plan to assign.

Select Continue.

Step 3: Assign Models to Tiers and Test ​

Assign one model ID to each tier. For LiteLLM, the wizard does not restrict or suggest models – you enter any model ID your proxy accepts.

  1. For the Small tier, enter the model ID to use for lightweight tasks (for example, gpt-4o-mini or an alias defined in your proxy config).
  2. For the Frontier tier, enter the model ID to use for multi-step agent tasks.
  3. For the Reasoning tier, enter one or more model IDs for extended-context tasks. The first model in the list serves as the system default. While most features exclusively use this first model, Cribl Search investigations allow you to switch between any listed models via a dropdown menu.
  4. Select Test and Save.

Cribl tests each assigned model by sending a request through your configured proxy:

  • A results panel shows how many models passed and failed.
  • If any test fails, verify that:
    • The Deployment URL points to your running LiteLLM proxy and is reachable from the Cribl Leader.
    • The API Key is valid and authorized for the models you assigned.
    • Each model ID is defined and reachable in your LiteLLM config.yaml.
  • Select Retry to test again after making corrections.
  • When all models pass, the configuration is saved automatically.

After you save:

  • A Custom AI provider card appears at the top of AI Settings, showing your ID, description, LiteLLM as the provider type, and the number of configured models.
  • Supported Cribl AI capabilities in that Workspace begin routing through your LiteLLM proxy.

Verify Cribl AI Behavior ​

To confirm that your LiteLLM provider is correctly configured:

  1. In AI Settings, confirm that the Custom AI provider card lists LiteLLM.
  2. Use a supported Cribl AI capability (for example, the KQL assistant or Git commit message suggestions).
  3. Verify that:
    • Requests succeed without provider-related errors.
    • Latency and behavior align with your expectations for the models your proxy serves.

If you see failures or unexpected behavior:

  • Check the LiteLLM proxy logs for errors related to the model IDs or authentication.
  • Select the gear (⚙️) on the provider card and select Edit to reopen the wizard, then use Test and Save to review error details.
  • Confirm that the Cribl Leader has uninterrupted network access to the proxy endpoint.

Change or Stop Using LiteLLM ​