On This Page

Home / Cribl as Code/ Cribl API/ API Code Examples/Configure Resources with the Cribl API

Configure Resources with the Cribl API
​

These code examples demonstrate how to use the Cribl API to create the following resources in Cribl Stream:

  • A Syslog Source to receive data on port 9021.
  • An S3 Destination to store processed data.
  • A Pipeline that filters events and keeps only data in the eventSource and eventID fields.
  • A Route that connects the Source, Pipeline, and Destination.

The examples also include deploying the resource configurations to a Worker Group to make them active.

The examples use the Worker Group created in Configure Worker Groups to manage the resource configuration.

About the Code Examples
​

The code examples use Bearer token authentication. Read the API authentication documentation to learn how to configure authentication. The Permissions granted to your Bearer token must include creating and managing resources.

Replace the variables in the examples with the corresponding information for your Cribl deployment.

For on-prem deployments, to use https in the URLs, you must configure Transport Layer Security (TLS).

The resource configurations in the examples do not include all available body parameters. For a complete list of body parameters for each resource, refer to the endpoint documentation in the API Reference.

Create a Source
​

This example creates a Syslog Source to receive data on port 9021.

API (Cribl.Cloud and Hybrid)API (On-Prem)
curl --request POST \
--url "https://${workspaceName}-${organizationId}.cribl.cloud/api/v1/m/my-worker-group/system/inputs" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "id": "in_syslog_9021",
  "type": "syslog",
  "disabled": true,
  "tcpPort": 9021,
  "host": "192.168.1.100"
}'
curl --request POST \
--url "https://${hostname}:${port}/api/v1/m/my-worker-group/system/inputs" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "id": "in_syslog_9021",
  "type": "syslog",
  "disabled": true,
  "tcpPort": 9021,
  "host": "192.168.1.100"
}'

Create a Destination
​

This example creates an S3 Destination to store processed data. Replace placeholder values like your-aws-api-key before you run the example.

API (Cribl.Cloud and Hybrid)API (On-Prem)
curl --request POST \
--url "https://${workspaceName}-${organizationId}.cribl.cloud/api/v1/m/my-worker-group/system/outputs" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "id": "out_s3",
  "type": "s3",
  "awsAuthenticationMethod": "manual",
  "awsApiKey": "your-aws-api-key",
  "awsSecretKey": "your-aws-secret-key",
  "region": "us-east-2",
  "bucket": "your-aws-bucket-name",
  "compress": "gzip",
  "compressionLevel": "best_speed",
  "stagePath": "$CRIBL_HOME/state/outputs/staging",
  "emptyDirCleanupSec": 300
}'
curl --request POST \
--url "https://${hostname}:${port}/api/v1/m/my-worker-group/system/outputs" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "id": "out_s3",
  "type": "s3",
  "awsAuthenticationMethod": "manual",
  "awsApiKey": "your-aws-api-key",
  "awsSecretKey": "your-aws-secret-key",
  "region": "us-east-2",
  "bucket": "your-aws-bucket-name",
  "compress": "gzip",
  "compressionLevel": "best_speed",
  "stagePath": "$CRIBL_HOME/state/outputs/staging",
  "emptyDirCleanupSec": 300
}'

Create a Pipeline
​

This example creates a Pipeline that filters events and keeps only data in the eventSource and eventID fields.

API (Cribl.Cloud and Hybrid)API (On-Prem)
curl --request POST \
--url "https://${workspaceName}-${organizationId}.cribl.cloud/api/v1/m/my-worker-group/pipelines" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "id": "my_pipeline",
  "conf": {
    "asyncFuncTimeout": 1000,
    "functions": [
      {
        "filter": "true",
        "conf": {
          "remove": [
            "*"
          ],
          "keep": [
            "eventSource",
            "eventID"
          ]
        },
        "id": "eval",
        "final": true
      }
    ]
  }
}'
curl --request POST \
--url "https://${hostname}:${port}/api/v1/m/my-worker-group/pipelines" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "id": "my_pipeline",
  "conf": {
    "asyncFuncTimeout": 1000,
    "functions": [
      {
        "filter": "true",
        "conf": {
          "remove": [
            "*"
          ],
          "keep": [
            "eventSource",
            "eventID"
          ]
        },
        "id": "eval",
        "final": true
      }
    ]
  }
}'

Create a Route
​

This example creates a Route that connects the Syslog Source, Pipeline, and S3 Destination and adds it to the end of the default Routing table. The Routing table already includes a default Route, so this request changes the default Route’s setting for final to false so that it won’t block the new Route.

The PATCH /routes/default endpoint requires a complete representation of the Routing table and its existing Routes in the request body. This endpoint does not support partial updates. Cribl removes any omitted fields when updating the Routing table. Use GET /routes/default to retrieve the existing Routing table to use in the body of the PATCH request.

API (Cribl.Cloud and Hybrid)API (On-Prem)
curl --request PATCH \
--url "https://${workspaceName}-${organizationId}.cribl.cloud/api/v1/m/my-worker-group/routes/default" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "id": "default",
  "routes": [
    {
      "id": "0abcd9",
      "name": "default",
      "final": false,
      "disabled": false,
      "pipeline": "main",
      "enableOutputExpression": false,
      "filter": "true",
      "output": "default"
    },
    {
      "name": "my_route",
      "final": true,
      "disabled": false,
      "pipeline": "my_pipeline",
      "enableOutputExpression": false,
      "filter": "__inputId == 'syslog:in_syslog_9021:tcp'",
      "output": "out_s3",
      "description": "This is my new Route"
    }
  ]
}'
curl --request PATCH \
--url "https://${hostname}:${port}/api/v1/m/my-worker-group/routes/default" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "id": "default",
  "routes": [
    {
      "id": "0abcd9",
      "name": "default",
      "final": false,
      "disabled": false,
      "pipeline": "main",
      "enableOutputExpression": false,
      "filter": "true",
      "output": "default"
    },
    {
      "name": "my_route",
      "final": true,
      "disabled": false,
      "pipeline": "my_pipeline",
      "enableOutputExpression": false,
      "filter": "__inputId == 'syslog:in_syslog_9021:tcp'",
      "output": "out_s3",
      "description": "This is my new Route"
    }
  ]
}'

Commit and Deploy the Resource Configurations
​

This example demonstrates how to commit and deploy the resource configurations to your Worker Group, then commit to the Leader to keep it in sync with the Worker Group.

Committing and deploying the Worker Group configuration requires three requests: commit to the Worker Group, deploy to the Worker Group, and commit the the Leader to keep it in sync.

First, commit the pending resource configurations to the Worker Group:

API (Cribl.Cloud and Hybrid)API (On-Prem)
curl --request POST \
--url "https://${workspaceName}-${organizationId}.cribl.cloud/api/v1/version/commit?groupId=my-worker-group" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "message": "Commit resource configurations to my-worker-group"
}'
curl --request POST \
--url "https://${hostname}:${port}/api/v1/version/commit?groupId=my-worker-group" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "message": "Commit resource configurations to my-worker-group"
}'

Next, deploy the committed changes to the Worker Group. This request includes the version body parameter, which uses the value of commit from the response body for the commit request:

API (Cribl.Cloud/Hybrid)API (On-Prem)
curl --request PATCH \
--url "https://${workspaceName}-${organizationId}.cribl.cloud/api/v1/products/stream/groups/my-worker-group/deploy" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "version": 1234abcd5678efgh9012ijkl3456mnop7EXAMPLE
}'
curl --request PATCH \
--url "https://${hostname}:${port}/api/v1/products/stream/groups/my-worker-group/deploy" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "version": 1234abcd5678efgh9012ijkl3456mnop7EXAMPLE
}'

Finally, commit the changes to the Leader to keep the Leader in sync with the Worker Group:

API (Cribl.Cloud/Hybrid)API (On-Prem)
curl --request POST \
--url "https://${workspaceName}-${organizationId}.cribl.cloud/api/v1/version/commit" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "message": "Sync my-worker-group resource configurations with Leader"
}
curl --request POST \
--url "https://${hostname}:${port}/api/v1/version/commit" \
--header "Authorization: Bearer ${token}" \
--header "Content-Type: application/json" \
--data '{
  "message": "Sync my-worker-group resource configurations with Leader"
}