On This Page

Home / Edge/ Secure Your Deployment/Centralize and Reuse Global Secrets

Centralize and Reuse Global Secrets ​

Use global secrets to define credentials once and reuse them across Cribl products by referencing them in resource configurations. Teams that span products can use the same credentials instead of maintaining duplicate credentials that can drift out of sync. When you rotate credentials, you can update the values for global secrets in one place.

Cribl organizes global secrets in secret folders that are backed by secret stores. Access to global secrets is controlled at the folder level. Manage secret stores, secret folders, and global secrets in the UI under Global Settings > Security > Secrets or with the Cribl API.

You can use global secrets in Cribl Stream and Edge (Sources, Destinations, Collectors, and Packs); Cribl Search (Sources); and Cribl Lake (HTTP and Zscaler Sources). Any legacy secrets that you use in existing resource configurations will continue to work as you migrate to global secrets.

Global secrets are not supported when GitOps is enabled in Cribl Stream, and you cannot enable GitOps on a Leader that already has global secrets configured. Cribl excludes global secret values from Git, so those values never reach a Git remote and cannot promote across environments. If GitOps is enabled, use legacy secrets instead.

A typical workflow for global secrets is:

  1. Connect an external provider to add a secret store, or use the built-in cribl secret store.
  2. Add a secret folder and make it accessible to the Worker Groups, Edge Fleets, and products that need it. You can also clone an existing secret folder to create a new secret folder that uses the same secret store.
  3. Add global secrets to the secret folder.
  4. Reference global secrets in Cribl resources that support them.
  5. Commit and deploy your configuration.
  6. Rotate credentials as needed in the built-in cribl secret store or an externally backed secret store.

Deployment Scope ​

In Cribl.Cloud, global secrets are scoped to a single Workspace. Workspaces in the same Organization cannot access each other’s global secrets.

In on-prem deployments, global secrets are scoped to one Leader. Unrelated Leader installations cannot access global secrets configured on a different Leader.

The Leader persists global secrets configuration and resolves secret values. In Cribl.Cloud, this means the Leader for the Workspace. In on-prem deployments, this means the Leader Node. Cribl adds global secrets to deployment bundles for Worker Groups and Edge Fleets through the bundle push path. Cribl adds a global secret when its secret folder is accessible to the Group or Fleet, even if no resource configuration in the Group or Fleet references the secret.

Global secrets are available only in Distributed deployments. If you run Cribl Edge in single-instance mode, use legacy secrets instead.

Secret Stores ​

Secret stores are the backing stores for the secret folders that contain global secrets. You can use the built-in cribl secret store or connect an externally backed secret store.

The built-in cribl secret store cannot be deleted, and it persists encrypted global secret values on the Leader. To delete a secret store that is backed by an external provider, you must first delete all of its secret folders.

Cribl Edge uses cribl.secret as the primary encryption key to protect sensitive values in configuration files. It is created when the Leader starts and is stored by default at $CRIBL_HOME/local/cribl/auth/cribl.secret. Global secrets in the built-in cribl secret store are encrypted with this key.

When you configure an external KMS provider (HashiCorp Vault or AWS KMS) to manage cribl.secret, the key is removed from disk and managed by the KMS provider instead. This protection extends to global secrets in the built-in cribl secret store. The KMS provider configuration is separate from the global secret store. When using an external KMS, ensure that IAM permissions are correctly scoped (for example, kms:Decrypt for AWS KMS). Misconfigured permissions can prevent the Leader from starting.

Back up cribl.secret before major changes such as enabling an external KMS or rotating keys.

Secret Folders ​

Within secret stores, global secrets are organized into secret folders. Each secret folder can only belong to one secret store.

For secret folders backed by the built-in cribl secret store, Cribl persists encrypted secret values on the Leader in per-folder storage. For secret folders that are backed by external secret stores, Cribl stores metadata for folders and global secrets on the Leader and resolves secret values from the external provider at runtime.

Access Model ​

Global secrets are defined once on the Leader within the deployment scope. Cribl controls global secrets at the folder level, not per secret.

When you add a secret folder, the Accessible to setting determines which Cribl Worker Groups, Edge Fleets, and products can use the folder’s global secrets in resource configurations. Adding a Worker Group, Edge Fleet, or product to the folder’s Accessible to list makes every global secret in that folder available to use throughout the Group, Fleet, or product. If you make a secret folder accessible to an Edge Fleet or Subfleet, its Subfleets inherit the ability to use the global secrets in the folder.

Deployment bundles include every global secret in a folder that is accessible to a Worker Group, Edge Fleet, or product. Take care to make secret folders accessible only to the Groups, Fleets, and products that should be able to use the secrets that the folders contain.

After a secret folder is accessible to a Worker Group, Edge Fleet, or product, supported resources in that Group, Fleet, or product expose references to the global secrets in the folder. Within one Workspace (Cribl.Cloud) or Leader (on-prem), any Worker Group, Edge Fleet, or product that can use the folder can reuse the same global secret definitions.

To remove a Worker Group, Edge Fleet, or product from a secret folder’s Accessible to list, first remove all references to the global secrets in the folder from resources in the Group, Fleet, or product. This includes references used in Subfleets that will lose access to the global secrets if you remove access for the parent Fleet or Subfleet.

Making a secret folder accessible to a Worker Group, Edge Fleet, or product does not grant Member Permissions on the folder. Members with the Admin Permission on a higher-level object can view and manage the folder’s secrets. For other Members and Teams, an Admin must share the folder and assign the Shared Folder Secrets Viewer or Shared Folder Secrets Maintainer Permission.

Leader Restart and Externally Backed Secret Folders ​

When the Leader restarts, it reinitializes each secret store that is backed by an external provider and loads global secrets from that provider. The Leader keeps externally backed secret values in memory only (they are not persisted on disk).

For a HashiCorp Vault secret store that connects from an Outpost, the Outpost reads Vault and returns the values. The Leader does not connect to HashiCorp Vault.

If credential values changed in the external store since the last commit and deploy, the Leader might resolve the updated values after it restarts. This does not distribute the new credentials to Worker Groups or Edge Fleets. Commit and deploy still controls when updated global secrets reach nodes. To rotate credentials through the normal workflow, see Rotate Global Secrets in Externally Backed Secret Stores.

If the Leader or an Outpost cannot reach the external provider when the Leader restarts, secret store initialization or secret resolution can fail. That happens because Cribl has not fetched and cached the global secret values yet. For secret stores that connect from an Outpost, resolution also fails when no Outpost in the Outpost Group is connected and when no connected Outpost runs Cribl version 4.21.0 or newer.

Cribl surfaces the failure through secret store health checks as system bulletin messages. Deploy is blocked for Worker Groups and Edge Fleets with access to secret folders that are backed by the unhealthy store until the Leader successfully resolves those secrets. You can still commit configuration changes.