Configurations and Restart
In a Distributed deployment, Worker Nodes poll the Leader for configuration changes. Many of these changes require a quick reload to read the new configuration, while others require a restart of the Cribl processes on the Worker Node.
You can Restart and Reload via the UI. In the sidebar, select Settings, then Global. Under System > Controls select Reload or Restart.
Upon restarts, be aware of the following:
- Syslog data still being received over UDP might be dropped.
- Worker Nodes will temporarily disappear from the Leader’s Workers or Edge Nodes page.
- Aggregation and suppression operations will start over.
- Worker Nodes’ local copies of Monitoring metrics will be dropped.
- Cribl Stream will drop any events still in RAM that were bound for persistent queues. (However, PQ data already written to disk will persist through the restart.)
Changes that require reloads include configuration changes to:
- Functions
- Pipelines
- Packs
- Routes
- Lookups
- Parquet schemas
- Global variables
- Group Settings > Limits
- Group Settings > Logging > Levels
Changes that require restarts include configuration changes to:
- Distributed mode (Leader versus Managed Worker Node or Single instance)
- Worker Group assignment
- Event Breakers
- QuickConnect configs
- Sources
- Destinations
- Group Settings > General Settings > TLS
- Group Settings > General Settings > Advanced
- Group Settings > Worker Processes > Process count and Memory
How Configuration Changes Apply
In a Distributed deployment, how a change applies across each Worker Node in a Worker Group depends on the type of change:
- Sources and Destinations: Cribl Stream restarts Worker Processes on each Worker Node to apply the change.
- In Cribl Stream, a Worker Node can run multiple Worker Processes, so they restart in a rolling fashion and the Node keeps processing data during the change. For the rolling rollout mechanics and configurable settings, see Worker Process Configuration Rollout in the Cribl Stream docs.
- In Cribl Edge, each Edge Node runs a single Worker Process, so it simply restarts that process.
- Routes, Pipelines, and Lookups or other knowledge objects: Cribl Stream reloads the configuration without restarting Worker Processes, so data flow continues uninterrupted.
- In Cribl Stream, reloads also roll out in batches across Worker Processes. If a Worker Process does not finish reloading within the configured timeout, Cribl Stream restarts it. For details, see Worker Process Configuration Rollout.
- Worker Group settings that involve the API Process (such as API, TLS, Proxy, and Security settings): Cribl Stream restarts all Worker Processes at once rather than in a rolling fashion. All Worker Processes stop before the API Process restarts, and then the API Process starts them again.
Some general guidelines to keep in mind:
- Most UI interactions, such as changing the order of Functions in a Pipeline or changing the order of Routes, do not require restarts.
- Some configuration changes in the Settings UI do require restarts. These prompt you for confirmation before restarting.
- All direct edits of configuration files in
$CRIBL_HOME/local/require restarts. - Worker Nodes might temporarily disappear from the Leader’s Workers or Edge Nodes tab while restarting.
- A
git commitcommand on the Leader Node’s host (using a freestandinggitclient not embedded in Cribl’s CLI or UI) requires either a reload or restart. - When using the Cribl App for Splunk, changes to Splunk configuration files might or might not require restarts. Please check current Splunk docs.