On-Prem Architecture Planning

Cribl on-prem deployment is the best choice when you need to maintain control over your data, leverage existing infrastructure, or meet strict data sovereignty requirements. This guide will walk you through the key considerations and best practices for designing a robust, scalable, and resilient Cribl on-prem architecture. Let’s dive into the core decisions that will shape your deployment, starting with how to choose the right architecture for your needs.

Choosing Your On-Prem Deployment Architecture

​

When planning an on-prem deployment, your choice of architecture will depend on two key factors:

• Incoming Data: The total volume of data you need to ingest daily.
• Processing Complexity: The complexity of your processing, such as heavy transformations, Regex extractions, and serialization.
• Data Type: ​​The type of data you’re handling, such as SQS, also significantly influences processing needs.

For Cribl Edge, you’ll need to consider a few additional points:

• Routing and Cloning: Determine if data is going to a single destination or being routed to multiple places. This is important because destination-specific serialization can be expensive.
• Air-Gapped Deployment: If deploying Edge Nodes on servers without internet access, ensure each node can communicate with an on-prem Leader Node (not a Cribl.Cloud Leader).

For details, see Deployment Planning for Cribl Stream and Cribl Edge.

Deployment Types

​

Once you’ve assessed your architectural needs, the next step is to select a deployment model that aligns with your environment and goals. Here’s a breakdown of the supported deployment types.

General Sizing Considerations

​

Regardless of the deployment type you choose, proper sizing is a foundational step. A stable, efficient deployment starts with a solid foundation – and that means properly sizing your infrastructure.

For specific guidelines, see OS and System Requirements for Cribl Stream and Cribl Edge and General Sizing Considerations.

Port Considerations: Enabling Communication

​

After you’ve determined your sizing, the next critical step is configuring your network. Correct port configuration is essential for inter-node communication and data flow.

Custom Port Overrides: Default ports can be overridden in the cribl.yml and inputs.yml configuration files located in $CRIBL\_HOME/local/.

For details, see Ports in Cribl Stream and Cribl Edge.

High Availability And Scaling Design

​

Once the foundational elements of your architecture and networking are in place, you must plan for long-term stability. Designing for high availability (HA) and proper scaling is key to a resilient deployment that can withstand unexpected events and grow with your needs.

Leader Redundancy Architecture

​

For Leader redundancy, deploy at least one standby Leader with NFS-backed shared storage for the configuration and state. Use a load balancer in front of the Leader’s UI and API ports (e.g., 9000, 4200) to ensure reliable access.

To build a comprehensive failover and recovery plan, you should monitor the health of your system, configure persistent queues for your most critical data sources, and establish regular backup and restore processes for all configurations and secrets. For details, see Leader High Availability/Failover.

Worker Node Scaling

​

Beyond the Leader, a resilient deployment also relies on a robust Worker infrastructure. Proper scaling of your Worker Nodes is important for ensuring uninterrupted service and handling data spikes. For resilient deployments, we recommend a minimum of three Worker Nodes. This ensures the system can handle maintenance or an unplanned outage without a loss of service.

In Cribl, a Worker Process is an independent instance that handles data processing. Think of it as a separate child process. A single Worker Node can run multiple Worker Processes to provide workload distribution and increase throughput.

For details see, More About Worker Groups, Nodes, and Processes – Definitions and How Many Worker Nodes?

As you plan your scaling strategy, consider the unpredictable nature of data flow.

Throughput Variability: Data flow is rarely constant. Plan for variability and spikes, and start with about 10% headroom to handle unexpected demands. Use historical per-second throughput data to set thresholds and prevent system failures. For details, see Throughput.

Risk vs. Cost Balance: Also, plan for contingency capacity to handle resource demands during failure scenarios. The goal is to allocate enough contingency capacity to handle resource demands that arise during failures. Failure scenarios often require additional Worker Nodes to process backlogged data after recovery. If you don’t have enough spare capacity, this can overwhelm your existing nodes, leading to data loss or expanding delays. Your investment in contingency resources should align with your desired level of reliability. For details, see Finding the Balance Between Risk and Cost.

Container Strategy

​

If you’ve chosen a containerized approach, a specific set of best practices will help you get the most out of your deployment. When deploying Cribl products in a containerized environment, following a few best practices ensures optimal performance, scalability, and resilience for both Cribl Edge and Cribl Stream.

Kubernetes Orchestration

​

Use Kubernetes for a scalable, managed deployment. This is ideal for large-scale environments that require automated orchestration and resource management.

For details, see Orchestrated Deployment in Cribl Stream and Deploy Cribl Edge via Kubernetes.

Docker Configuration And Container Lifecycle

​

Beyond orchestration, a robust strategy for container configuration and lifecycle management is essential for ensuring stability, performance, and resilience in Docker environments.

For configuration details, see Docker Deployment on Cribl Stream and Run Cribl Edge in a Docker Container

Container-Specific Architectural Decisions

​

Before you finalize your plan, it’s important to step back and make a few high-level architectural decisions that will guide your deployment. Making these key decisions up front will ensure a solid foundation for your on-prem architecture.

On-Prem Planning

​

Making these key decisions up front will ensure a solid foundation for your on-prem architecture.

• Infrastructure Isolation: Isolate your production, development, and test environments using network segmentation or VLANs.
• Leader Placement and Sizing: Place the Leader in a resilient part of the network with secure, high-performance storage and adequate CPU/RAM for configuration management. For details, see (General Sizing Considerations)[#sizing] and (High Availability And Scaling Design)[#ha].
• Scaling Architecture: Plan for both vertical and horizontal scaling. Isolate critical data flows in dedicated Worker Groups/Fleets for easier, safer scaling.
• Upgrade Coordination: Upgrades are manual and require careful coordination. Always back up your configurations and secrets before making changes.

On-Prem Deployment Trade-offs

​

Finally, it’s crucial to understand the trade-offs that come with an on-prem deployment. While it offers control, it also requires you to take on specific responsibilities. The following table highlights the key areas where your on-prem choices have a direct impact.