Once you have configured TLS on the Leader Node, you can implement client certificate exchange to enable mutual authentication (mTLS). This allows Cribl Stream to permit only explicitly authorized clients, which hold valid certificates, to connect to the Leader.

For encrypting data in transit and simple server verification, standard TLS is sufficient. Opt for mTLS when security policies mandate additional client verification.

When a client certificate is presented to a Leader, two things happen:

1. The Leader validates the client certificate presented to Cribl Stream. The Validate client certificates setting is optional, but highly recommended. When enabled, the Leader checks the certificate against the trust store, to see if it has been signed by a valid certificate authority (CA).

   The Leader checks the list of certificates in the CA certificate path box first (if populated), then against the list of built-in system certificates.

2. The Leader checks whether the Common Name (CN) matches the regular expression in the configuration. Cribl Stream’s default is to accept any value in the Common name field. You can customize this as needed.

   Within the Common Name, Cribl Stream validates against the value after the CN=string. If your Common Name is CN=stream.worker, you would enter stream\.worker in the Common name field - including the backslash, because the value entered is a regular expression.

Prerequisites for mTLS for Leader Communication ​

Enabling mTLS for Leader communication requires:

• A CA certificate chain that can validate the client certificates.
• Client certificates issued to each client that will connect to to the Leader.
• Client systems configured to present their certificates during connection.

You can’t use mTLS for Leader-Node communication in Cribl.Cloud.

Certificate Requirements for mTLS ​

mTLS provides a significantly higher security posture by requiring both the server and the client to prove their identities. For this handshake to succeed, Cribl Stream relies on the underlying TLS stack to validate the integrity and purpose of each certificate.

Before enabling mTLS, verify that your certificate infrastructure meets the standards outlined below. If a certificate is expired, uses a legacy algorithm, or cannot be traced back to a trusted root, the TLS connection will be rejected immediately to prevent unauthorized access.

General Certificate Standards ​

Regardless of their role in the handshake, all certificates used by the Leader, Worker/Edge Nodes, and Sources/Destinations must adhere to these baseline criteria:

• Validity Period: Certificates must be current. TLS handshakes will fail if the system time is outside the NotBefore or NotAfter window.
• Modern Cryptography: Use certificates signed with modern algorithms (such as RSA or ECDSA). Avoid legacy or deprecated hash functions like MD4, MD5, or SHA-1, which are often blocked by TLS.
• Chain of Trust: The peer validating the certificate must be able to build a complete path from the end-entity certificate through any Intermediate CAs to a trusted root. In Cribl Stream, this trust is established via the CA Certificates Path configuration or the host system trust store.

Server-Specific Requirements ​

Server certificates are presented by components that terminate TLS (such as the Leader on its management port). These certificates must prove they are authorized to act as a “server” to the connecting client:

• Key usage: The certificate Extended Key Usage (EKU) must include serverAuth.
• Hostname Verification: To prevent man-in-the-middle attacks, the Common Name (CN) or Subject Alternative Name (SAN) must exactly match the hostname or IP address that clients use to connect (for example, leader.cribl.com).

Client-Specific Requirements ​

Client certificates are presented by the component initiating the connection (such as a Worker/Edge Node connecting to a Leader). These are subject to stricter identity checks:

• Client Authentication: The certificate EKU must explicitly include clientAuth. Public CAs (such as Let’s Encrypt) have increasingly disallowed this attribute in their standard issuance policies to comply with stricter industry root program requirements. For mTLS, you will likely need to use an internal PKI or a specialized certificate product that permits client-side usage.
• Identity Constraints: Client certificates must be leaf certificates (where Basic Constraints are set to CA=false). Root or Intermediate CA certificates should never be used as client identifiers.
• CN Regex Matching: Cribl Stream provides an additional layer of security by checking the certificate CN against a user-defined regular expression to ensure only specific named clients are permitted.

Limitations on mTLS Authentication ​

When configuring mTLS authentication on Worker Nodes, make sure you place your certificates into a separate directory outside of $CRIBL_HOME. If you place the certificates inside $CRIBL_HOME, they’ll be removed when the next config bundle is deployed from the Leader.

Similarly, you can’t bootstrap Worker Nodes with mutual authentication already populated. To bootstrap Worker Nodes, you supply only the shared authentication token. Certificates should be viewed as two-factor authentication; so placing the certificates in the config bundle defeats the purpose of two-factor authentication.

Configure mTLS Communication ​

To configure mTLS for Leader-Worker Node communication, first, enable mTLS on the Leader side.

1. In Settings > Global > System > Distributed Settings > TLS Settings, enable Authenticate client (mutual auth).
2. Optionally, modify the Common Name regex to allow checking for the common name of the certificates of nodes trying to connect to the Leader.

Next, configure the Worker Node certificates. You can do it via the UI, the instance.yml config file, or the CRIBL_DIST_LEADER_URL environment variable.

Configure Worker Node mTLS via the UI ​

To configure mTLS on a Worker Node via the UI:

1. Navigate to a Worker Node’s Worker Settings > System > Distributed Settings > TLS Settings.
2. Toggle Enable Server TLS on.
3. Fill in Private key path (mutual auth), Certificate path (mutual auth), and CA certificate path with your certificate information.
4. Optionally, you can set both the Minimum TLS version and Maximum TLS version.

Configure Worker Node mTLS Using YAML Config File ​

To set up mTLS authentication via the Worker Node’s instance.yml config file, use the following configuration:

distributed:
  mode: worker
  master:
    host: <hostname>
    port: 4200
    authToken: <token>
    tls:
      disabled: false
      rejectUnauthorized: true # false if ignoring untrusted certs
      requestCert: true
      privKeyPath: /path/to/certs/worker.key
      certPath: /path/to/certs/worker.pem
      caPath: /path/to/certs/root.pem
    resiliency: none
  group: default

Configure Worker Node mTLS Using Environment Variables ​

You can set up mTLS authentication by configuring the CRIBL_DIST_LEADER_URL environment variable, using the format shown in this example:

CRIBL_DIST_LEADER_URL="tls://<authToken>@leader.cribl:4200?tls.privKeyPath=/path/to/certs/worker.key&tls.certPath=/path/to/certs/workers.pem&tls.caPath=/path/to/certs/root.pem&tls.requestCert=true&tls.rejectUnauthorized=true"

Once you’ve set this variable, restart the Worker Node. You should see the Worker Node successfully reconnect to the Leader.

If the Worker Node doesn’t connect, check cribl.log on both the Worker Node and Leader for more context about the problem. You should see errors related to dist leader communications.

To build your own Certificate Authority (a self-signed CA), see our blog post on How to Secure Cribl Stream Worker-to-Leader Communications.