You can send data processed through Cribl to other various destinations.
In Cribl, each pipeline can be independently configured with a destination definition. See the "Destinations" block on right below for a list of destination types.
When non-streaming destination definitions are associated with a pipeline, Cribl will use a staging directory in the local filesystem to format and write outputted events. After a set of conditions (below) is met, typically file size and number of files, data is then compressed and then moved or copied to the final destination. An inventory of open, or in-progress files is kept in the root of staging directory in order to avoid having to walk that directory at startup. This can get expensive if staging is the final directory. At startup, Cribl will check for any left over files in progress from prior sessions and ensure they're moved/copied to final destination. The process of moving to final destination is delayed after startup (default 30 sec) and (b) processing of these files is paced at one per service period (default 1 second).
There are a number of conditions that govern when files are closed and rolled out:
- File reaches its configured max size
- File reaches its configured max open time
- File reaches its configured max idle time
If a new file needs to be open, Cribl will enforce the number of max open files, by closing them in the order in which they were opened.
There is a always at least one destination configured in Cribl. This is referred to as the default destination. In this version of Cribl, while each pipeline can be associated with any destination definition, in the event that that destination is unreachable, Cribl will send data to default. In the event that default is unavailable, the data will be dropped.
Destinations that accept events in real-time and support back-pressure are referred to as streaming destinations. Supported destinations:
Destinations that accept events in groups or batches are referred to as non-streaming destinations. Supported destinations:
For each destination type users can create multiple definitions depending on their requirements.
To configure destinations, click on Destinations, select the desired type from the left vertical menu then click Add New.