Prepare data for sending to Loki or Prometheus.

Loki and Prometheus, Grafana’s systems for aggregating logs and for metrics-based monitoring, respectively, require specific setup to collect and store data coming from Cribl Edge.

Loki stores log contents unindexed, relying on labels and timestamps that are indexed to form event streams. The method is similar to how data would be stored in a bucket or index on other systems.

Similarly, Prometheus uses names, labels (dimensions), and numerical values.

For more straightforward setups where less customization is needed and you will ultimately use the data in Grafana Cloud, you can use the Grafana Cloud Destination.

Configure a Loki Pipeline

​

Loki stores logs based on labels. This is a great way to organize your stored data so that retrieval is easier when looking for specific log entries. By default, no labels are defined for the Loki Destination. Because Loki requires at least one label, Cribl will automatically add a default label source with the value cribl_${__inputId}.

See Loki Destination Processing settings for more information.

To ensure data is shaped correctly for Loki, you can configure a Pipeline with an Eval Function to add the labels to the __labels field.

To create a Pipeline called loki-pipeline in the Cribl Edge UI, follow these steps:

1. On the top bar, select Products, and then select Edge. Under Fleets, select a Fleet.

2. Select More, then Pipelines.

3. Select Add Pipeline.

4. In ID, enter loki-pipeline and provide an optional Description.

5. Confirm with Save.

6. Select Add Function and choose Eval.

7. In the Evaluate Fields section, add the configuration for your labels. For example, to prepare Kubernetes logs data, use:

   Name	Value Expression
   __labels	{}
   __labels.namespace	kube_namespace
   __labels.pod	kube_pod
   __labels.container	kube_container

8. Select Save.

Next, in the Fleet’s submenu, go to Collect to attach your Source (for example, a Kubernetes Logs Source) to Loki using the loki-pipeline Pipeline as connection.

Configure a Prometheus Pipeline

​

Cribl Edge offers a prebuilt prometheus_metrics Pipeline. You can modify it to change a few metadata fields and use it to connect to a Prometheus Destination.

1. On the top bar, select Products, and then select Edge. Under Fleets, select a Fleet.

2. Select More, then Pipelines, then prometheus_metrics.

3. Select the Eval Function and modify the fields in the Evaluate Fields section as needed. For example, to prepare Kubernetes metrics data, use:

   Name	Value Expression
   job	job || 'cribl'
   instance	__metadata.kube.node.metadata.name || host

4. Select Save.

Next, in the Fleet’s submenu, go to Collect to attach your Source (for example, a Kubernetes Metrics Source) to the Prometheus Destination, using the prometheus_metrics Pipeline as the connection.