Cribl LogStream – Docs

Cribl LogStream Documentation

Questions? We'd love to help you! Meet us in #Cribl Community Slack (sign up here)
Download entire manual as PDF – v.3.1.2

Securing

You can secure Cribl LogStream access and traffic using various combinations of SSL (Secure Sockets Layer), TLS (Transport Layer Security), custom HTTP headers, and external KMS (Key Management Service) options.

SSL Certificate Configuration

You can secure LogStream's API and UI access by configuring SSL. To do so, you can use your own certs and private keys, or you can generate a pair with OpenSSL, as shown here:

openssl req -nodes -new -x509 -newkey rsa:2048 -keyout myKey.pem -out myCert.pem -days 420

This command will generate both a self-signed cert (certified for 420 days), and an unencrypted, 2048-bit RSA private key.

In the LogStream UI, you can configure the cert via global ⚙️ Settings (lower left) > Security > Certificates. You can configure the key via:

Alternatively, you can edit the local/cribl.yml file's api section to directly set the privKeyPath and certPath attributes. For example:

api:
  host: 0.0.0.0
  port: 9000
  disabled : false
  ssl:
    disabled: false
    privKeyPath: /path/to/myKey.pem
    certPath: /path/to/myCert.pem
...

Custom HTTP Headers

You can encode custom, security-related HTTP headers, as needed. As shown in the examples below, you specify these at global ⚙️ Settings (lower left) > General > API Server Settings > Advanced > HTTP Headers. Click + Add Header to display extra rows for new key-value pairs.

Custom HTTP headersCustom HTTP headers

Custom HTTP headers

TLS Settings and Traffic Types

This table shows TLS client/server pairs, and encryption defaults, per traffic type.

Traffic Type

TLS Client

TLS Server

Encryption

Cert Auth

CN* Check

UI

Browser

Cribl LogStream

Default disabled

Default disabled

Default disabled

API

Worker

Master

Default disabled

Default disabled

Default disabled

Worker-to-Master

Worker

Master

Default disabled

Default disabled

Default disabled

Data

Any data sender

Cribl LogStream (Source)

Default disabled

Default disabled

Default disabled

Data

Cribl LogStream (Destination)

Any data receiver

Default disabled

Default disabled

Default disabled

Authentication

————

————

————

————

————

• Local

Browser

Cribl LogStream

Default Disabled

N/A

N/A

• LDAP

Cribl LogStream

LDAP Provider

Custom

N/A

Default Disabled

• Splunk

Cribl LogStream

Splunk Search Head

Default Enabled

N/A

Default Disabled

• OIDC†/​Okta

Browser and Cribl LogStream

Okta

Default Enabled

N/A

Enabled (Browser)

• OIDC/​Google

Browser and Cribl LogStream

Google

Default Enabled

N/A

Enabled (Browser)

* Common name
† OpenID Connect

You can configure advanced, system-wide TLS settings for versions, cipher lists, and ECDH Curve names via global ⚙️ Settings (lower left) > System > General Settings > Default TLS Settings.

Encryption Keys

You can create and manage keys that LogStream will use for real-time encryption of fields and patterns within events. For details on applying the keys that you define here, see Encryption.

Accessing Keys

  • In a single-instance deployment, select global ⚙️ Settings (lower left) > Security > Encryption Keys.
  • In a distributed deployment with one Worker Group, select Configure > Settings > Security > Encryption Keys.
  • In a distributed deployment with multiple Worker Groups, keys are managed per Worker Group. Select Groups > <group-name> Settings > Security > Encryption Keys.

On the resulting Manage Encryption Keys page, you can configure existing keys, and/or use the following options to add new keys.

Get Key Bundle

To import existing keys, click Get Key Bundle. You'll be prompted to supply a login and password to proceed.

Add New Key

To define a new key. click + Add New The resulting New Key modal provides the following controls:

Key ID: LogStream will automatically generate this unique identifier.

Description: Optionally, enter a description summarizing this key's purpose.

Encryption algorithm: Currently, aes-25-cbc is the only option supported here.

KMS for this key: Defaults to local (LogStream's internal Key Management Service). If you've configured an external KMS, you can select that option instead.

Key Class: Classes are arbitrary collections of keys that you can map to different levels of access control. For details, see Encryption. This value defaults to 0; you can assign more classes, as needed.

Expiration time: Optionally, assign the key an expiration date. Directly enter the date or select it from the date picker.

Secrets

With LogStream's secrets store, you can centrally manage secrets that LogStream instances use to authenticate on integrated services. Use this UI section to create and update authorization tokens, username/password combinations, and API‑key/secret‑key combinations for reuse across the application.

Accessing Secrets

  • In a single-instance deployment, select global ⚙️ Settings (lower left) > Security > Secrets.
  • In a distributed deployment with one Worker Group, select Configure > Settings > Security > Secrets.
  • In a distributed deployment with multiple Worker Groups, secrets are managed on each Worker Group. Select Groups > <group-name> Settings > Security > Secrets.

On the resulting Manage Secrets page, you can configure existing secrets, and/or click + Add New to define new secrets.

Add New Secret

The New Secret modal provides the following controls:

Secret name: Enter an arbitrary, unique name for this secret.

Secret type: See below for this second field's options, some of which expose additional controls.

Description: Optionally, enter a description summarizing this secret's purpose.

Tags: Optionally, enter one or multiple tags related to this secret.

Secret Type

This drop-down offers the following types:

Text: This default type exposes a Value field where you directly enter the secret. Text secrets can currently be used only with LogStream's Google Cloud Pub/Sub Source and Destination, where they are the only supported secret type.

API key and secret key: Exposes API key and Secret key fields, used to retrieve the secret from a secure endpoint. This is the only secret type supported on LogStream's AWS-based Sources, Collectors, and Destinations, and on our Google Cloud Storage Destination.

CA Certificates and Environment Variables

Where LogStream Sources and Destinations support TLS, each Source's or Destination's configuration provides a CA Certificate Path field where you can point to corresponding Certificate Authority (CA) .pem file(s). However, you can also use environment variables to manage CAs globally. Here are some common scenarios:

  1. How do I add a set of trusted root CAs to the list of trusted CAs that LogStream trusts?

    Set this environment variable in each Worker's environment (e.g., in its systemd unit file): NODE_EXTRA_CA_CERTS=/path/to/file_with_certs.pem. For details, see the nodejs docs.

  2. How do I make LogStream trust all TLS certificates presented by any server it connects to?

    Set this environment variable: NODE_TLS_REJECT_UNAUTHORIZED=0 – for details, see the nodejs docs.

KMS Configuration

Configure LogStream's Key Management Service at global ⚙️ Settings (lower left) > Security > KMS. This is a global setting on both single-instance and distributed deployments.

As of version 3.0, administrators with a LogStream Enterprise or Standard license can integrate an external KMS provider to manage the key that LogStream uses for encrypting secrets on Worker Groups and Workers. The external option initially available is HashiCorp Vault.

📘

To integrate an external KMS provider Into a distributed deployment, LogStream's Leader Node must have Internet access.

When you initially install a license in distributed mode, a known bug prevents immediate use of KMS features within Worker Groups. Here is the workaround:

  1. Open global ⚙️ Settings (lower left) > Worker Processes.
  2. In the list of processes, locate any with a Role of CONFIG_HELPER
  3. Click that process' Restart button.

Upon restarting, KMS will be available for use in the corresponding Worker Group.

LogStream Internal KMS

The KMS provider field defaults to LogStream Internal. With this option, no configuration is required here. See Secrets to configure individual secrets.

HashiCorp Vault

Setting the KMS provider drop-down to HashiCorp Vault exposes the following configuration options:

KMS Settings

Vault URL: Enter the Vault server's URL (e.g., http://localhost:8200).

Authentication

Auth provider: The method for authenticating requests to Vault server. Select one of Token, AWS IAM, or AWS EC2. Your selection determines the remaining Authentication options displayed.

Token-based Authentication

Token: Enter the authentication token. This token will be used only to generate child tokens for further authentication actions.

AWS IAM Authentication

Use the Authentication method buttons to select one of the following AWS methods:

  • Auto: Uses the AWS instance's metadata service to automatically obtain short-lived credentials from the IAM role attached to an EC2 instance. The attached IAM role grants LogStream Workers access to authorized AWS resources. Can also use the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Works only when running on AWS.

  • Manual: If not running on AWS, you can select this option to enter a static set of user-associated IAM credentials directly or by reference. This is useful for Workers not in an AWS VPC, e.g., those running a private cloud. It prompts you to provide an Access key and a Secret key.

Vault AWS IAM Server ID: Value to use for the Vault-AWS-IAM-Server-ID header value. This should match the value configured with IAM authentication on Vault.

Vault Role: Authentication role to use in Vault.

Assume Role

This section is displayed for all AWS IAM authentication methods.

Enable for Vault Auth: Toggle to Yes if you want to use your Assume Role credentials to access Vault authentication.

AssumeRole ARN: Enter the Amazon Resource Name (ARN) of the role to assume.

External ID: Enter the External ID to use when assuming the role.

AWS EC2 Authentication

Vault Role: Enter the authentication role to use in Vault.

Secret Engine

Mount: Mount point of the Vault secrets engine to use. (Currently, only the KVv2 engine is supported.) Defaults to secret.

Secret path: Enter the path on which the LogStream secret should be stored, e.g.: <somePath>/cribl‑secret.

🚧

In a distributed deployment, the Leader, and each Worker Group, require a distinct secret. This location cannot be shared between them.

Advanced

Enable health check: Whether to perform a health check before migrating secrets data. Defaults to Yes.

Health check endpoint: Configurable endpoint to use for validating system health. Defaults to /v1/sys/health.

Updated 6 days ago


Securing


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.