Cribl LogStream – Docs

Cribl LogStream Documentation

Questions? We'd love to help you! Meet us in #Cribl Community Slack (sign up here)
Download entire manual as PDF - v2.4.4

Azure Event Hubs

Cribl LogStream supports receiving data records from Azure Event Hubs.

📘

Type: Pull | TLS Support: YES (secure API) | Event Breaker Support: No

Configuring Cribl LogStream to Receive Data from Azure Event Hubs

Select Data > Sources, then select Azure Event Hubs from the Data Sources page's tiles or left menu. Click Add New to open the Azure Event Hubs > New Source modal, which provides the following fields.

General Settings

Input ID: Enter a unique name to identify this source definition.

Brokers: List of Event Hubs Kafka brokers to connect to, e.g., yourdomain.servicebus.windows.net:9093. Get the hostname from the host portion of the primary or secondary connection string in Shared Access Policies.

Event Hub name: The name of the Event Hub (a.k.a. Kafka Topic) to subscribe to.

Group ID: Specifies the name of the consumer group to which this Cribl LogStream instance belongs. Should always be $Default for Event Hubs.

From beginning: Whether to start reading from the earliest available data. Relevant only during initial subscription. Defaults to Yes.

TLS Settings (Client Side)

Enabled: Defaults to Yes.

Validate server certs: Whether to reject connections to servers without signed certificates. Defaults to No – and for Event Hubs, must always be disabled.

Authentication

Enabled: Defaults to No. When toggled to Yes:

  • SASL mechanism: SASL (Simple Authentication and Security Layer) authentication mechanism to use. Currently, PLAIN is the only mechanism supported for Event Hubs Kafka brokers.

  • Username: The username for authentication. For Event Hubs, this should always be $ConnectionString.

  • Password: Event Hubs primary or secondary connection string. From Microsoft's documentation, the format is:

    Endpoint=sb://<FQDN>/;SharedAccessKeyName=<KeyName>;SharedAccessKey=<KeyValue>

    Example entry:

    Endpoint=sb://dummynamespace.servicebus.windows.net/;SharedAccessKeyName=dummyaccesskeyname;SharedAccessKey=5dOntTRytoC24opYThisAsit3is2B+OGY1US/fuL3ly=

Processing Settings

Fields (Metadata)

In this section, you can add fields/metadata to each event using Eval-like functionality.

Name: Field name.

Value: JavaScript expression to compute field's value (can be a constant).

Pre-Processing

In this section's Pipeline drop-down list, you can select a single existing Pipeline to process data from this input before the data is sent through the Routes.

Updated about a month ago

Azure Event Hubs


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.