There are at least two key factors that will determine the type of Cribl LogStream deployment in your environment:
Amount of Incoming Data: This is defined as the amount of data planned to be ingested per unit of time. E.g. How many MB/s or GB/day?
Amount of Data Processing: This is defined as the amount of processing that will happen on incoming data. E.g., is most data passing through and just being routed? Or are there a lot of transformations, regex extractions, field encryptions? Is there a need for heavy re-serialization?
When volume is low and/or amount of processing is light, you can get started with a single instance deployment.
To accommodate increased load, we recommend scaling up and perhaps out with multiple instances.
Cribl App for Splunk Deprecation Notice
Please see details here.
You can deploy LogStream Leader Nodes (or single instances) and Worker Nodes via Cribl's Helm charts.
You can deploy LogStream instances using images from Cribl's public Docker Hub.
All LogStream deployments are based on a shared-nothing architecture pattern, where instances/Nodes and their Worker Processes operate separately, serving all inputs, outputs, and event processing independently of each other.
This allows the overall system to continue to operate even if individual Processes or Nodes fail. It also allows individual Nodes to upgrade without system-wide downtime.
Updated 6 days ago