On This Page

Home / Stream/ Integrations/ Integrating with Other Services/ Amazon/AWS Cloud Connections

AWS Cloud Connections ​

AWS Cloud Connections lets you connect a single AWS account or an AWS Organization to Cribl Stream, automatically discover eligible data sources, and onboard those sources as inputs without Cribl modifying your environment.

This feature is available only on Cribl.Cloud deployments with an AWS-hosted Leader and at least one AWS-hosted Worker Group.

Overview ​

Onboarding cloud telemetry has traditionally required knowing exactly which AWS services you have, where logs are stored, and how to configure each one manually. AWS Cloud Connections removes that upfront investigation by doing the inventory for you and walking you through configuration in a single flow.

Once connected, Cribl performs a read-only scan to discover eligible data sources. For an Organization connection, Cribl discovers resources across the member accounts you target. You then select the resources you want to onboard, and Cribl generates the configuration needed to route data into your Worker Group.

The workflow uses CloudFormation templates. A connection template establishes access (a stack for a single account, or a StackSet for an Organization). When you onboard resources, Cribl generates additional templates that route data from your selected resources into Cribl.

Prerequisites ​

Confirm that your environment meets all of these requirements:

  • Cribl.Cloud: Your deployment runs on Cribl.Cloud. AWS Cloud Connections is not available in self-hosted deployments.
  • AWS-hosted Leader and Worker Group: Your Cribl Stream deployment includes an AWS-hosted Leader and at least one AWS-hosted Worker Group.
  • Admin role: You must be assigned the Cribl Stream Admin role to add or manage AWS connections.

For an Organization Root connection, also confirm:

  • You can sign in to the AWS Organizations management account.
  • You have permission to create and deploy CloudFormation StackSets to the Organizational Units or Organization root you will target.
  • You know the 12-digit management account ID and the Organizational Unit IDs or Organization root ID (r-... or ou-...) to include.

Discovered AWS Services ​

Cribl scans for resources from the following AWS services:

Add an AWS Connection ​

In your Worker Group, select Data > Sources. The Cloud Connections tile appears at the top of the Sources page. Select the Cloud Connections AWS tile, then select Add Connection.

Select a Connection Level, then complete the fields for that option.

Organization Root ​

Use Organization Root to connect an AWS Organization and access resources in Organization member accounts. Complete these fields, then select Launch Stack:

Assumed Role Name: IAM role name Cribl assumes in Organization member accounts. This role should not already exist. The connection CloudFormation template creates it. For example, CriblRole.

Organization Management Account ID: The 12-digit ID for the management account of the AWS Organization.

Organizational Unit IDs: One or more Organization root IDs (r-...) or Organizational Unit IDs (ou-...). Up to 50. To target an entire Organization, use a single root ID.

AWS STS External ID: Unique string Cribl includes when assuming the Assumed Role above.

Display Name: Optionally, set a friendly name for this connection as it appears in the Cribl UI.

Scan Compute Mode: Compute used to run data scans. Choose Auto (default), Lambda, or Fargate.

StackSet Account Filter Type: How account inclusion/exclusion combines with the selected Organizational Units when the StackSet deploys. See AWS account-level StackSet targets for details. Choose NONE to deploy to all accounts in the selected Organizational Units (or root), INTERSECTION to deploy only to accounts that appear both in the selected Organizational Units and in Accounts to Include, or DIFFERENCE to deploy to all accounts in the selected Organizational Units except those listed in Accounts to Exclude.

Accounts to Include: Account IDs to include when StackSet Account Filter Type is INTERSECTION. Select Add Include for each account ID.

Accounts to Exclude: Account IDs to exclude when StackSet Account Filter Type is DIFFERENCE. Select Add Exclude for each account ID.

Single Account ​

Use Single Account to connect one AWS account. Complete these fields, then select Continue:

Assumed Role Name: IAM role name Cribl assumes in the connected AWS account. This role should not already exist. The connection CloudFormation template creates it. For example, CriblRole.

Account ID: Your 12-digit AWS account ID. You can find this ID in the account menu on your AWS dashboard.

AWS STS External ID: Unique string Cribl includes when assuming the Assumed Role above.

Display Name: Optionally, set a friendly name for this connection as it appears in the Cribl UI.

Scan Compute Mode: Compute used to run data scans. Choose Auto (default), Lambda, or Fargate.

Grant Permissions ​

Cribl generates a CloudFormation template that creates the required read-only IAM role. For a single account, the template creates a stack in that account. For an Organization, the template creates a StackSet that provisions the role into the member accounts you targeted.

Organization Root ​

Cribl uses a CloudFormation StackSet to automatically deploy to your AWS accounts. Before deploying, confirm the following:

  • Enable trusted access for the CloudFormation StackSets service via the StackSets console.
  • This browser is logged in to your AWS Organization’s root account.
  • Your root account has appropriate permissions to create a CloudFormation stack.

Copy or download the template using the icons to the right of the template, add and apply it in AWS, then select Continue.

Single Account ​

Cribl uses a CloudFormation stack to deploy to your AWS account. Before deploying, confirm the following:

  • You can view the Stacks console to confirm your stack deployments.
  • This browser is logged in to your AWS account.
  • Your account has appropriate permissions to create a CloudFormation stack.

Copy or download the template using the icons to the right of the template, add and apply it to your AWS account, then select Continue.

Establish Connection ​

During this step, Cribl validates that the CloudFormation stack or StackSet deployed successfully and that the IAM role is reachable. Your connected account or Organization appears as a tile showing the display name and account ID. Select Continue to proceed.

This step can take up to 15 minutes for large accounts or Organizations. You can safely navigate away during the scan. To return, go to the Cloud Connections AWS screen and select Account details > Add Sources on your connected account.

Onboard Data ​

After the connection is established, Cribl scans the connected account (or member accounts for an Organization) and presents a filterable table of discovered resources. Each row shows:

  • Resource ID: The unique identifier of the discovered AWS resource.
  • Region: The AWS region where the resource is located.
  • Resource Type: The type of AWS resource, such as S3 Bucket or VPC Flow Logs.
  • Source ARN: The Amazon Resource Name of the source resource Cribl will read from.
  • Destination ARN: The Amazon Resource Name of the destination resource, such as an SQS queue, where data will be delivered before Cribl ingests it.

Use the Filter resources by ID, Filter by resource type, and Filter by region controls to narrow the list. You can also use Group By to group rows by resource type. You can only select resources from a single region at a time. Select the resources you want to onboard and select Configure Selected.

For an Organization connection, member accounts can finish scanning at different times. If some accounts are still scanning, Cribl shows results from the accounts that have completed so far and displays how many member accounts are included.

Configure Integrations ​

Cribl can ingest AWS logs through S3 buckets or through Kinesis Data Firehose delivery streams. This step ensures the selected resources are configured to deliver data into Cribl.

Data Grouping: Choose how Cribl creates Sources for the selected resources:

  • Individual: Each discovered resource gets its own Cribl Source. An S3 bucket Source is created where delivery is to a bucket, and a Kinesis Data Firehose Source is created where delivery is to a stream. This option is more flexible but creates more Sources. When you select this option, the next section is labeled Integration Settings.
  • Grouped by Service and Delivery Type: Resources that share the same AWS origin service and the same delivery type (S3 or Firehose) share one set of integration names. Cribl creates one Source configuration per group. When you select this option, the next section is labeled Service Group Settings.

Under Integration Settings or Service Group Settings, review and optionally edit the following fields for each resource or group:

  • Source ID: The name Cribl will use for the Source. Pre-populated based on the discovered resource.
  • Bucket Name: The S3 bucket where logs are delivered. Pre-populated from discovery and cannot be changed if using an existing bucket.
  • SQS Queue Name: The SQS queue Cribl will poll for S3 event notifications. Pre-populated with a generated name.
  • Assume Role Name: The IAM role Cribl will assume to access the resource. Pre-populated based on the role created during the Grant Permissions step.

Deploy Template ​

Your integrations are configured. Cribl generates CloudFormation scripts that create the AWS-side resources needed for those Sources to receive data (for example SQS queues, S3 event notifications, flow log configurations, and IAM permissions).

Cribl creates Sources for the selected integrations. They appear on this Worker Group’s Sources page. Commit and deploy the Worker Group so the new Sources are included in the group’s saved config. The CloudFormation scripts complete the AWS-side setup.

Under AWS Configuration Script, review the generated YAML. Each AWS region has its own script (shown as tabs). Copy or download a script to run in AWS. If you have more than one region, you can also select Download All.

When you are ready, select Finish.

Manage Connected Accounts ​

Connected accounts and Organizations appear on the main Cloud Connections AWS screen. Each listing is labeled Account or Organization, shows the display name and account ID, and lists the Sources configured from that connection. Those Sources also appear on the standard Sources pages for their respective input types (S3 or Kinesis Data Firehose, depending on the originating AWS service).

On a listing, select Account Details or Organization Details to view last-scan information and refresh the scan in place. Open the manage menu (⋯) for these actions:

  • Add Resources: Re-enter the onboarding flow to discover and onboard additional resources from this connection.
  • Refresh: Re-scan to update the inventory of discovered resources.
  • Account Settings or Organization Settings: View or modify the connection configuration.
  • Disconnect Account or Disconnect Organization: Remove the connection and its associated credentials from Cribl. Disconnecting removes only the Cribl connection. For a single account, delete the CloudFormation stack in AWS to clean up AWS resources. For an Organization, delete the DSD StackSet and its stack instances from the Organization management account. Any Sources created from this connection remain in your Worker Group and continue to function after you disconnect.

Security Considerations ​

AWS Cloud Connections is read-only for discovery. Cribl does not write to, modify, or delete customer data resources during the scan.

The IAM role provisioned by the initial CloudFormation template or StackSet grants only the permissions needed to identify resources and read their configurations. Cribl intentionally avoids AWS managed policies, which would grant broader access than necessary.

The role’s trust policy restricts assumption to a role that is local to your Cribl tenant and requires the configured STS External ID. Your Cribl Leader stores the associated secret in encrypted form. The resource scanning service runs within your tenant and assumes the tenant-local role before assuming the role in your AWS account or Organization member accounts.

Scan results are stored securely within your Cribl tenant and are automatically deleted after a short retention period.