On This Page

Home / Edge/ Secure Your Deployment/ Centralize and Reuse Global Secrets/ Connect External Secret Stores/Connect HashiCorp Vault Secret Stores

Connect HashiCorp Vault Secret Stores ​

Configure a secret store that is backed by HashiCorp Vault to retrieve global secrets from the KV v2 secrets engine. The KV v1 secret engine is not supported.

  1. In Global Settings > Security > Secrets > Secret Stores, select Add Secret Store.

  2. In the New Secret Store modal, configure the store using the following fields:

    • Secret store: Select HashiCorp Vault.

    • Store name: Unique identifier for the secret store.

    • Description: Brief description of the secret store (optional).

    • Access via: How Cribl will connect to the secret store. Select Leader to connect from the Leader. Select Outpost to connect from an Outpost if the Leader cannot reach HashiCorp Vault.

    • Vault URL: URL where your HashiCorp Vault instance is running (for example, https://vault.example.com:8200). The host part of the URL must be an IPv4 address, an IPv6 address, or a fully qualified domain name. Cribl refuses short names like vault and link-local addresses.

    • Namespace: Full path to the HashiCorp Vault Enterprise namespace where Cribl should retrieve global secrets. Enter the full path from the root namespace, not only the leaf segment (such as admin/engineering/sre for HashiCorp Cloud Platform (HCP) Vault or root/engineering/sre for self-hosted Vault). Omit if you do not use Vault namespaces.

      • HashiCorp Vault Enterprise namespaces are isolated and do not inherit authentication configurations from parent namespaces. If you store secrets in more than one Vault namespace, add a separate Vault-backed secret store in Cribl for each namespace.

      • In each namespace, your Vault administrator must configure an authentication method and policies that allow Cribl to authenticate and read the paths that you provide when you add a global secret.

      • When you add a global secret, Cribl validates each path against the namespace you provide. If HashiCorp Vault does not have a secret at the specified path (for example, because the secret is in a different namespace), Cribl will not add the global secret.

    • Mount: Mount point of the HashiCorp Vault KV v2 secrets engine to retrieve global secrets from. Default is secret. The KV v1 secrets engine is not supported.

    • Authentication > Authentication provider: Provider to use for authenticating to HashiCorp Vault.

      • Token: In the Token field, enter your HashiCorp Vault authentication token.

      • AWS IAM:

        • Authentication method: Select Auto or Manual.
          • Auto: Use the AWS credentials available on the host that connects to HashiCorp Vault. For a direct connection, the host is the Leader. For an Outpost connection, the host is the Outpost.
          • Manual: Use static credentials. Enter your Access key and Secret key credentials.
        • Vault AWS IAM Server ID: Value for the X-Vault-AWS-IAM-Server-ID header. Must match the value configured for IAM authentication on your HashiCorp Vault server.
        • Vault role: Name of the role configured in the HashiCorp Vault AWS auth method that grants Cribl access to secrets stored in Vault.
          • If you provided the path to a HashiCorp Vault Enterprise namespace in the Namespace field, this role must be configured in the same namespace. Namespaces do not inherit authentication configurations from parent namespaces.
        • Custom authentication path: Auth method mount path in HashiCorp Vault, if different from the default aws (optional).
        • AssumeRole > Enable for Vault Auth: Toggle on to assume a different IAM role via AWS Security Token Service (STS) before authenticating to HashiCorp Vault. Use when the Cribl instance’s own IAM role does not have direct Vault access, such as for cross-account access.
        • AssumeRole > AssumeRole ARN: ARN of the IAM role to assume. Required when AssumeRole is enabled.
        • AssumeRole > External ID: Identifier required to authorize third-party access. Required when the role’s trust policy enforces an external ID condition.

        Your Vault administrator must allow the IAM principal that matches the Authentication method and AssumeRole settings. In Cribl.Cloud, when multiple Workspaces run in the same AWS account, each Workspace authenticates with a different IAM principal, so the Vault role must include every principal that needs access.

        The assumed role session lasts 15 minutes. Cribl does not show a duration setting. A duration saved from an older configuration is ignored.

      • AWS EC2: Use the identity of the EC2 instance that connects to Vault. For a direct connection, the instance is the Leader. For an Outpost connection, the instance is the Outpost.

        • Vault role: Name of the HashiCorp Vault role to authenticate against when using AWS EC2 authentication.
        • Custom authentication path: Auth method mount path in HashiCorp Vault, if different from the default aws (optional).
      • AppRole:

        • Role ID: Identifier for the AppRole defined in HashiCorp Vault.
        • Secret ID: Secret credential associated with the AppRole defined in HashiCorp Vault.
        • Custom authentication path: Auth method mount path in HashiCorp Vault, if different from the default approle (optional).
    • TLS Settings (Client Side):

      If your HashiCorp Vault deployment uses a self-signed certificate or a certificate signed by a private certificate authority (CA), add the CA certificate to the trusted CA list on the Leader. For details, see CA Certificates and Environment Variables.

      • Server name (SNI): Server Name Indication (SNI) value to send in the TLS handshake. Use when the HashiCorp Vault server certificate CN or SAN does not match the Vault URL hostname.

      • Minimum TLS version: Minimum TLS version that Cribl will accept when connecting to HashiCorp Vault.

      • Maximum TLS version: Maximum TLS version that Cribl will accept when connecting to HashiCorp Vault.

    • Health Check > Health check endpoint: HashiCorp Vault API path to use for health checks to validate Vault connectivity. Default is /v1/sys/health.

  3. Select Test and Save.

Health Check Error Messages ​

After five consecutive failed health checks for a HashiCorp Vault-backed secret store, Cribl displays a system bulletin message. Health checks run every 30 seconds, so a single transient failure can clear on its own. For details about health checks and reload, deploy, and recovery, see Secret Store Health Check Failures.

Error messageFailure
Could not validate the provided Vault configuration: ECONNREFUSEDConnection was refused or Vault is unreachable
Vault is sealed and cannot serve requestsVault is sealed
Vault rejected the configured credentials (token invalid or expired)Token is invalid or expired
Vault access denied (HTTP 403). Verify that Cribl's token or policy has not been revoked.Authentication credentials configured for the secret store do not have the correct permissions to read the Vault health check endpoint.
HTTP status code and response body for diagnosisHealth endpoint returned a non-200 response

Self-Hosted HashiCorp Vault and IP Allowlists ​

When the Leader resolves global secrets from a self-hosted HashiCorp Vault instance, it makes outbound connections to your Vault server to authenticate and resolve secrets. These connections come from the Leader’s egress IP address. If your HashiCorp Vault server uses an IP allowlist to restrict inbound connections, your Vault administrator must add the Leader’s egress IP address to the Vault allowlist.

Cribl.Cloud ​

Cribl.Cloud Workspaces use public egress by default, and the Leader’s egress IP is ephemeral. When Cribl replaces the Leader during upgrades or maintenance, the egress IP changes, and the allowlist entry that uses the existing egress IP stops working. You must use a stable Leader egress IP for a reliable integration if your HashiCorp Vault server uses an IP allowlist.

Enable static egress for the Workspace region so the Leader uses a stable set of outbound IPs. Your Vault administrator must add every address listed in Workspace > Connections > Static Egress IP to the Vault allowlist. If you later disable static egress, those IPs change and the Vault allowlist must be updated.

If the Cribl.Cloud Leader cannot reach HashiCorp Vault, see Connect to HashiCorp Vault from an Outpost.

On-Prem Deployments ​

In on-prem Cribl deployments, the Leader Node typically has a stable egress IP that you control. Add that IP address (or the IP addresses for all Leaders that resolve Vault secrets, such as in a Leader HA pair) to your HashiCorp Vault allowlist.

Connect to HashiCorp Vault from an Outpost ​

Use an Outpost Node to read HashiCorp Vault when Vault stays inside your network or when the Cribl.Cloud Leader cannot connect to Vault. The Outpost reads HashiCorp Vault inside your network and returns the values on its existing outbound connection to the Leader. The Leader never opens a connection to HashiCorp Vault, and you do not need to open an inbound firewall rule.

Global secrets still resolve on the Leader. Cribl distributes the resolved global secret values to Worker Groups and Edge Fleets when you commit and deploy. The Leader still directly reads secret stores with no Outpost binding.

Requirements ​

To use an on-prem Outpost to read HashiCorp Vault:

  • Set Up Cribl Outpost on a host that can reach HashiCorp Vault. In the sidebar, select Outposts, select the Outpost Group, and confirm that the Outpost is Connected.
  • To limit which Vault URLs the Outpost contacts, see Limit Vault URLs for Outpost Connections.
  • Confirm that each Outpost in the Group is running Cribl version 4.21.0 or newer.

Limit Vault URLs for Outpost Connections ​

To read a HashiCorp Vault secret store, an Outpost contacts the Vault URL configured on the store. You can use the optional CRIBL_OUTPOST_SECRET_STORE_ALLOWED_HOSTS environment variable to limit which Vault URLs the Outpost contacts. When the variable is set, the Outpost does not open a connection or send credentials to any URLs that are not listed.

Set CRIBL_OUTPOST_SECRET_STORE_ALLOWED_HOSTS in the process environment on each Outpost in the Group. Each Outpost reads the variable once, when its process starts. Restart each Outpost after you set or change the variable.

Use the same value for CRIBL_OUTPOST_SECRET_STORE_ALLOWED_HOSTS on every Outpost in the Group. Each time the Leader asks an Outpost to contact HashiCorp Vault, it uses one connected Outpost from the secret store’s Outpost Group.

Specify the Vault URLs in a comma-separated list. The following table lists the valid forms for the Vault URLs:

FormExampleDetails
Exact hostnamevault.corp.exampleAdmits the host on any port, over HTTP or HTTPS. Matching ignores case.

Use when you want to allow a change of scheme or port on the host.
Hostname with a wildcard for one subdomain level*.corp.exampleAdmits any hostname that adds one part in front of the domain (for example, vault.corp.example or myvault.corp.example).

Does not admit the domain itself or a deeper name (for example, corp.example or a.b.corp.example).

Use when you want to cover several hosts that differ by one name under the same domain.
IP address or CIDR range10.0.0.5 or 10.0.0.0/24Admits addresses in the specified range on any port, over HTTP or HTTPS.

Use when the Vault URL itself is an address. A hostname does not match an address entry, including the address that the name resolves to.
Scheme, host, and porthttps://vault.corp.example:8200Admits the specified combination only. If the port is omitted, admits only the scheme default: 443 for HTTPS, or 80 for HTTP.

Use when you want to refuse a change of scheme or port on the host.

Troubleshoot Outpost Connections ​

Cribl displays the following error messages when an Outpost connection to HashiCorp Vault fails.

Error messageFailure
the group does not exist or is not configured as an Outpost GroupThe Outpost Group specified in the secret store configuration does not exist or is a Worker Group or Edge Fleet.
the group has no connected membersNo Outpost in the Group is connected to the Leader.
no connected member runs Cribl 4.21.0 or laterNo connected Outpost in the Group runs Cribl version 4.21.0 or newer.
is not admitted by this Outpost's CRIBL_OUTPOST_SECRET_STORE_ALLOWED_HOSTS allowlistThe Vault URL specified in the secret store configuration is missing from CRIBL_OUTPOST_SECRET_STORE_ALLOWED_HOSTS on the Outpost named in the Leader log.
the Outpost or the store behind it did not respond in timeThe Outpost did not respond or could not reach the Vault URL specified in the secret store configuration.