Cribl - Docs

Getting started with Cribl LogStream

Questions? We'd love to help you! Meet us in #cribl (sign up)

Changelog    Guides

HTTP(S)

Cribl supports receiving of data over HTTP/S using the Elastic Bulk API or Cribl Bulk API.

Configuring Cribl to receive data over HTTP(S)


While on Sources screen, select HTTP from the vertical menu, then click Add New:

  • Input Id: Enter a unique name to identify this HTTP(S) source definition.
  • Host: Enter hostname/IP to listen for HTTP(S) data. E.g. localhost or 0.0.0.0.
  • Port: Enter port number.
  • Shared secret (authToken): Shared secret to be provided by HTTP client in header (as Authorization: <authToken>). If empty, unauthenticated access will be permitted.
  • Elastic API Endpoint (Bulk API): Absolute path where to listen for the Elastic API requests. At the moment only _bulk is available. Others are faked as success. Use empty string to disable. Default to /elastic.
  • Cribl HTTP Event API: Absolute path where to listen for Cribl HTTP API requests. Use empty string to disable. Defaults to /cribl.

TLS Settings (server side)

  • Disabled defaults to Yes. When toggled to No:
    • Private Key Path: Path on server where to find the private key to use in PEM format. Path can reference $ENV_VARS.
    • Certificate Path : Path on server where to find certificates to use in PEM format. Path can reference $ENV_VARS.
    • CA Certificate Path : Path on server where to find CA certificates to use in PEM format. Path can reference $ENV_VARS.
    • Authenticate Client (mutual auth): Require clients to present their certificates. Used to perform mutual authentication using SSL certs. Defaults to No.
    • Validate Client Certs: Require server to reject any connection which is not authorized with the list of supplied CAs. Defaults to No.

Then, click Save

Format & Endpoint


At the time of this writing, HTTP(S) events are expected to use the following format:

  1. A JSON record per event.
{"_time":1541280341, "_raw":"this is a sample event ", "host":"myHost", "source":"mySource", "fieldA":"valueA", "fieldB":"valueB"}
{"_time":1541280341, "host":"myOtherHost", "source":"myOtherSource", "_raw": "{\"message\":\"Something informative happened\", \"severity\":\"INFO\"}"}

Note 1: Events can be sent as separate POSTs but it is highly recommended that multiple of them are newline delimited, grouped and POSTed together.

Note 2: if a HTTP(S) source is routed to a Splunk destination, fields within the JSON payload are mapped to Splunk fields. Fields that do not have corresponding (native) Splunk fields become index-time fields. For example, let's assume we have a HTTP(S) event as below:

{"_time":1541280341, "host":"myHost", "source":"mySource", "_raw":"this is a sample event ", "fieldA":"valueA"}

_time, host and source become their corresponding fields in Splunk. The value of _raw becomes the actual body of the event and fieldA becomes an index-time field. (fieldA::valueA)

Example


  1. Configure Cribl to listen on port 10080 for HTTP (default). Set authToken to myToken42.
  2. Send a payload to your Cribl host.
Cribl Endpoint:
---------------
curl -k http://<myCriblHost>:10080/cribl/_bulk -H 'Authorization: myToken42' -d '{"_raw":"this is a sample event ", "host":"myHost", "source":"mySource", "fieldA":"valueA", "fieldB":"valueB"}'
Cribl Endpoint:
---------------
curl -k http://<myCriblHost>:10080/cribl/_bulk -H 'Authorization: myToken42' -d $'{"_raw":"this is a sample event ", "host":"myHost", "source":"mySource", "fieldA":"valueA", "fieldB":"valueB"} \n {"_raw":"this is another sample event ", "host":"myOtherHost", "source":"myOtherSource", "fieldA":"valueA", "fieldB":"valueB"}'