These docs are for Cribl Stream 3.0, a product version we no longer actively maintain.
See the latest version (4.5).
Securing
You can secure Cribl LogStream access and traffic using various combinations of SSL (Secure Sockets Layer), TLS (Transport Layer Security), custom HTTP headers, and external KMS (Key Management Service) options.
SSL Certificate Configuration
You can secure LogStream’s API and UI access by configuring SSL. To do so, you can use your own certs and private keys, or you can generate a pair with OpenSSL, as shown here:
openssl req -nodes -new -x509 -newkey rsa:2048 -keyout myKey.pem -out myCert.pem -days 420
This command will generate both a self-signed cert (certified for 420 days), and an unencrypted, 2048-bit RSA private key.
In the LogStream UI, you can configure the cert via global ⚙️ Settings (lower left) > Security > Certificates. You can configure the key via:
- Global ⚙️ Settings (lower left) > Security > Encryption Keys (single-instance deployments), or
- Groups >
<group‑name>
> Settings > Security > Encryption Keys (distributed deployments).
Alternatively, you can edit the local/cribl.yml
file’s api
section to directly set the privKeyPath
and certPath
attributes. For example:
api:
host: 0.0.0.0
port: 9000
disabled : false
ssl:
disabled: false
privKeyPath: /path/to/myKey.pem
certPath: /path/to/myCert.pem
...
Custom HTTP Headers
You can encode custom, security-related HTTP headers, as needed. As shown in the examples below, you specify these at global ⚙️ Settings (lower left) > General > API Server Settings > Advanced > HTTP Headers. Click + Add Header to display extra rows for new key-value pairs.
TLS Settings and Traffic Types
This table shows TLS client/server pairs, and encryption defaults, per traffic type.
Traffic Type | TLS Client | TLS Server | Encryption | Cert Auth | CN* Check |
---|---|---|---|---|---|
UI | Browser | Cribl LogStream | Default disabled | Default disabled | Default disabled |
API | Worker | Master | Default disabled | Default disabled | Default disabled |
Worker-to-Master | Worker | Master | Default disabled | Default disabled | Default disabled |
Data | Any data sender | Cribl LogStream (Source) | Default disabled | Default disabled | Default disabled |
Data | Cribl LogStream (Destination) | Any data receiver | Default disabled | Default disabled | Default disabled |
Authentication | ———— | ———— | ———— | ———— | ———— |
• Local | Browser | Cribl LogStream | Default Disabled | N/A | N/A |
• LDAP | Cribl LogStream | LDAP Provider | Custom | N/A | Default Disabled |
• Splunk | Cribl LogStream | Splunk Search Head | Default Enabled | N/A | Default Disabled |
• OIDC†/Okta | Browser and Cribl LogStream | Okta | Default Enabled | N/A | Enabled (Browser) |
• OIDC/Google | Browser and Cribl LogStream | Default Enabled | N/A | Enabled (Browser) |
* Common name † OpenID Connect
You can configure advanced, system-wide TLS settings for versions, cipher lists, and ECDH Curve names via global ⚙️ Settings (lower left) > System > General Settings > Default TLS Settings.
Encryption Keys
You can create and manage keys that LogStream will use for real-time encryption of fields and patterns within events. For details on applying the keys that you define here, see Encryption.
Accessing Keys
- In a single-instance deployment, select global ⚙️ Settings (lower left) > Security > Encryption Keys.
- In a distributed deployment with one Worker Group, select Configure > Settings > Security > Encryption Keys.
- In a distributed deployment with multiple Worker Groups, keys are managed per Worker Group. Select Groups >
<group-name>
Settings > Security > Encryption Keys.
On the resulting Manage Encryption Keys page, you can configure existing keys, and/or use the following options to add new keys.
Get Key Bundle
To import existing keys, click Get Key Bundle. You’ll be prompted to supply a login and password to proceed.
Add New Key
To define a new key. click + Add New The resulting New Key modal provides the following controls:
Key ID: LogStream will automatically generate this unique identifier.
Description: Optionally, enter a description summarizing this key’s purpose.
Encryption algorithm: Currently, aes-256-cbc
is the only option supported here.
KMS for this key: Defaults to local
(LogStream’s internal Key Management Service). If you’ve configured an external KMS, you can select that option instead.
Key Class: Classes are arbitrary collections of keys that you can map to different levels of access control. For details, see Encryption. This value defaults to 0
; you can assign more classes, as needed.
Expiration time: Optionally, assign the key an expiration date. Directly enter the date or select it from the date picker.
Secrets
With LogStream’s secrets store, you can centrally manage secrets that LogStream instances use to authenticate on integrated services. Use this UI section to create and update authorization tokens, username/password combinations, and API‑key/secret‑key combinations for reuse across the application.
Accessing Secrets
- In a single-instance deployment, select global ⚙️ Settings (lower left) > Security > Secrets.
- In a distributed deployment with one Worker Group, select Configure > Settings > Security > Secrets.
- In a distributed deployment with multiple Worker Groups, secrets are managed on each Worker Group. Select Groups >
<group-name>
Settings > Security > Secrets.
On the resulting Manage Secrets page, you can configure existing secrets, and/or click + Add New to define new secrets.
Add New Secret
The New Secret modal provides the following controls:
Secret name: Enter an arbitrary, unique name for this secret.
Secret type: See below for this second field’s options, some of which expose additional controls.
Description: Optionally, enter a description summarizing this secret’s purpose.
Tags: Optionally, enter one or multiple tags related to this secret.
Secret Type
This drop-down offers the following types:
Text: This default type exposes a Value field where you directly enter the secret. Text secrets can currently be used only with LogStream’s Google Cloud Pub/Sub Source and Destination, where they are the only supported secret type.
API key and secret key: Exposes API key and Secret key fields, used to retrieve the secret from a secure endpoint. This is the only secret type supported on LogStream’s AWS-based Sources, Collectors, and Destinations, and on our Google Cloud Storage Destination.
CA Certificates and Environment Variables
Where LogStream Sources and Destinations support TLS, each Source’s or Destination’s configuration provides a CA Certificate Path field where you can point to corresponding Certificate Authority (CA) .pem
file(s). However, you can also use environment variables to manage CAs globally. Here are some common scenarios:
How do I add a set of trusted root CAs to the list of trusted CAs that LogStream trusts?
Set this environment variable in each Worker’s environment (e.g., in its systemd unit file):NODE_EXTRA_CA_CERTS=/path/to/file_with_certs.pem
. For details, see the nodejs docs.How do I make LogStream trust all TLS certificates presented by any server it connects to?
Set this environment variable:NODE_TLS_REJECT_UNAUTHORIZED=0
– for details, see the nodejs docs.
KMS Configuration
Configure LogStream’s Key Management Service at global ⚙️ Settings (lower left) > Security > KMS. This is a global setting on both single-instance and distributed deployments.
As of version 3.0, administrators with a LogStream Enterprise or Standard license can integrate an external KMS provider to manage the key that LogStream uses for encrypting secrets on Worker Groups and Workers. The external option initially available is HashiCorp Vault
.
To integrate an external KMS provider Into a distributed deployment, LogStream’s Leader Node must have Internet access.
When you initially install a license in distributed mode, a known bug prevents immediate use of KMS features within Worker Groups. Here is the workaround:
- Open global ⚙️ Settings (lower left) > Worker Processes.
- In the list of processes, locate any with a Role of
CONFIG_HELPER
- Click that process’ Restart button.
Upon restarting, KMS will be available for use in the corresponding Worker Group.
LogStream Internal KMS
The KMS provider field defaults to LogStream Internal
. With this option, no configuration is required here. See Secrets to configure individual secrets.
HashiCorp Vault
Setting the KMS provider drop-down to HashiCorp Vault
exposes the following configuration options:
KMS Settings
Vault URL: Enter the Vault server’s URL (e.g., http://localhost:8200).
Authentication
Auth provider: The method for authenticating requests to Vault server. Select one of Token
, AWS IAM
, or AWS EC2
. Your selection determines the remaining Authentication options displayed.
Token-based Authentication
Token: Enter the authentication token. This token will be used only to generate child tokens for further authentication actions.
AWS IAM Authentication
Use the Authentication method buttons to select one of the following AWS methods:
Auto: Uses the AWS instance’s metadata service to automatically obtain short-lived credentials from the IAM role attached to an EC2 instance. The attached IAM role grants LogStream Workers access to authorized AWS resources. Can also use the environment variables
AWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
. Works only when running on AWS.Manual: If not running on AWS, you can select this option to enter a static set of user-associated IAM credentials directly or by reference. This is useful for Workers not in an AWS VPC, e.g., those running a private cloud. It prompts you to provide an Access key and a Secret key.
Vault AWS IAM Server ID: Value to use for the Vault-AWS-IAM-Server-ID
header value. This should match the value configured with IAM authentication on Vault.
Vault Role: Authentication role to use in Vault.
Assume Role
This section is displayed for all AWS IAM authentication methods.
Enable for Vault Auth: Toggle to Yes
if you want to use your Assume Role credentials to access Vault authentication.
AssumeRole ARN: Enter the Amazon Resource Name (ARN) of the role to assume.
External ID: Enter the External ID to use when assuming the role.
AWS EC2 Authentication
Vault Role: Enter the authentication role to use in Vault.
Secret Engine
Mount: Mount point of the Vault secrets engine to use. (Currently, only the KVv2 engine is supported.) Defaults to secret
.
Secret path: Enter the path on which the LogStream secret should be stored, e.g.: <somePath>/cribl‑secret
.
In a distributed deployment, the Leader, and each Worker Group, require a distinct secret. This location cannot be shared between them.
Advanced
Enable health check: Whether to perform a health check before migrating secrets data. Defaults to Yes
.
Health check endpoint: Configurable endpoint to use for validating system health. Defaults to /v1/sys/health
.