Home / Stream/ Securing·Secure Leader/Nodes Communication

Secure Leader/Worker Nodes Communication

You can configure secure communication between your Leader and Worker Nodes. As an additional security measure, we recommend you prevent direct browser access to the UI for all Worker Nodes.

Secure Browser Access to Worker Node

Cribl recommends that in enterprise distributed deployments, you prevent direct browser access to the UI for all Worker Nodes.

On the Leader Node, start by enabling UI access to Worker Nodes (Stream, Edge). This way, admins will still be able to tunnel through from the Leader to the UI on any Worker Node. This is also a prerequisite for Connect to the Leader Securely described below.

  1. Select a Worker Group.
  2. Open Group Settings (top right).
  3. Under General Settings > API Server Settings, select Advanced.
  4. Toggle Local UI access to No.
  5. Select Save.

Connect to the Leader Securely

You can configure secure communication between your Leader and Worker Nodes using the UI, the instance.yml config file, or environment variables.

Secure via the UI

To set up secure communication via the UI, you configure first the Worker Group, then the Worker Node, then the Leader.

Worker Group Setup

For each Worker Group whose Worker Nodes you want to secure:

  1. Open your TLS/SSL certificate file, and copy its contents to your clipboard. (This can be the same certificate you uploaded to the Leader, or a different certificate.)
  2. In the sidebar, select Worker Groups, and then select the Worker Group you want to configure.
  3. Select Group Settings or Fleet Settings (upper right).
  4. Select Security > Certificates > TLS.
  5. Click Add Certificate.
  6. In the New Certificates modal, add the certificate’s contents to the Certificate field.
    (You can paste the file’s contents, or you can drag/drop or upload the .pem file.)
  7. As you did on the Leader, also insert your Private key, and (as needed) your Passphrase and CA certificate.
  8. Click Save.
  9. Commit and Deploy the Worker Group’s new configuration, including the new certificate.
  10. Repeat the preceding steps on each Worker Group.
Worker Group-level certificates are configured like the Leader’s certificate
Worker Group-level certificates are configured like the Leader’s certificate

Worker Node Setup

For each Worker Node that you want to secure:

  1. Enable the Leader’s UI access to each desired Worker Node (Stream, Edge).
  2. Tunnel through from the Leader to the Worker Node’s UI.
  3. Navigate to this Worker Node’s Worker Settings (upper right) > System > Distributed Settings > TLS Settings.
  4. Toggle Enable Server TLS to Yes.
    This will expose the remaining TLS settings.
  5. From the Certificate name drop-down, select the certificate you uploaded to the parent Worker Group.
    This will prefill all the required fields. (See all deployed certificates at the left nav’s Security > Certificates link.)
  6. Click Save.
    The Worker will be unavailable during a short lag, while it restarts with the new configuration.
  7. Repeat the preceding steps on each Worker Node.
Configuring TLS on Worker Node’s UI, from the Leader
Configuring TLS on Worker Node’s UI, from the Leader

Leader Setup

Next, return to the Leader’s UI:

  1. Select Settings > Global > System > Distributed Settings > TLS Settings.
  2. Toggle Enable server TLS to Yes.
  3. In the Certificate name drop-down, select an existing certificate. This will auto-populate the corresponding certificate fields.
  4. Click Save.

After you’ve enabled TLS on the Leader, generating bootstrap scripts to add or update Worker Nodes will automatically prepend https:// to the Leader’s URL.

Secure via the YAML Config File

You can also configure the Leader’s $CRIBL_HOME/local/_system/instance.yml file to ensure that TLS is enabled. Here’s the relevant section:

distributed:
  mode: managed-edge
  master:
    host: <hostname>
    port: 4200
    authToken: <token>
    tls:
      disabled: false
      rejectUnauthorized: false
      requestCert: false
    resiliency: none
  group: default_fleet

After you’ve enabled TLS on the Leader, generating bootstrap scripts to add or update Worker Nodes will automatically prepend https:// to the Leader’s URL.

Secure via Environment Variables

Another way to set up secure communications between Worker Nodes and the Leader is via environment variables (Stream, Edge).

If you deploy your Worker Nodes in a container, you can enable encrypted TLS communications with the Leader by configuring the CRIBL_DIST_LEADER_URL with the tls: protocol. This will override the default setting in instance.yml. Here’s the format:

CRIBL_DIST_LEADER_URL=tls://<authToken>@leader:4200

Secure the Leader Node

This is a best practice that enables the Leader to validate itself to clients. We can secure it using the self-signed certificate we created in Configure SSL Certificates:

  1. Navigate to Settings > Global > General Settings > API Server Settings > TLS.
  2. Slide the toggle to Enabled.
  3. From the Certificate Name drop-down, select a cert you’ve previously imported. This will populate the corresponding fields here.
  4. Click Save.

After this save, you must prepend https:// to all Cribl Stream URLs on the Leader Node. E.g., to get back to the Settings page you just configured, you’ll now need to use https://<hostname>:<port>/settings/system)`.