As an alternative to downloading and deploying the LogStream binary to a physical or virtual machine, you can register a hosted LogStream instance on our Cribl.Cloud portal. This launches a SaaS version of LogStream.
If you're new to LogStream, please see our Basic Concepts page and Getting Started Guide for orientation. The current page focuses on a Cloud deployment's differences from other deployment options – referred to below as "LogStream binaries" or "customer-managed deployments."
LogStream Cloud is designed to simplify deployment, and to provide certain advantages over using your own infrastructure, in exchange for some current restrictions.
- Tap LogStream’s power, with no responsibility to install or manage software. LogStream Cloud is fully hosted and managed by Cribl. so you can launch a configured instance within minutes.
- Continuous delivery of upgrades and new features.
- Free, up to 1 TB/day of data throughput (data ingress + egress) for all new accounts through Jan. 1, 2022.
- Quickly expand your LogStream Cloud deployment by purchasing metered billing. Pay only for what you use.
LogStream Cloud provides (by design) a simplified deployment and a simplified user interface.
These are the current differences – detailed below – from the LogStream binary deployments that are described in the remainder of this documentation:
- No Worker Groups, and no Worker Mappings controls.
- No Filesystem Source, Collector, or Destination. A Cloud deployment has no local filesystem to read from or write to.
- No Scripts or Script Collector.
- Simplified administration. Cribl manages backup and restore of all configurations, high availability, and upgrade and maintenance, all automatically on your behalf.
- No KMS secrets stores (available in LogStream binaries, v.3.0 and above).
- TLS is provided on certain Sources for encryption only, with predefined certificates. LogStream Cloud does not currently support importing your own certificates for TLS mutual authentication.
- Hosted initially on AWS' US West Region, with more options to follow.
Ready to take the red pill? This section explains how to register and manage a LogStream Cloud instance.
To get started:
- Start at: https://cribl.cloud/signup/
- Select the New User? Free signup option, and register.
- Follow Cribl's email link to confirm your registration and sign in.
- Bookmark your Cribl.Cloud portal page, for all that follows.
When you own or are a member of multiple Cribl.Cloud Organizations, the Select Organization splash page – displayed after you sign in – enables you to select which Organization you want to work with.
Click any tile's
\/ accordion to reveal a detailed description, if provided. Click the appropriate tile (or its open accordion's Dashboard button) to configure that Organization.
You can click Leave if you want to remove yourself as a member of another owner's Organization. This option requires confirmation – proceed only if you're sure! (You won't see this button on Organizations that you own.)
Now that you're here – explore the furniture. The Cribl.Cloud portal's left sidebar allows you to navigate among the following pages/links:
When you log into the Cribl.Cloud portal, you'll land on this page's Overview tab. This is where you'll launch your LogStream instance, and where you'll connect to it on subsequent logins.
The big Connect to your LogStream button is the main event here – click it to launch your LogStream instance. However, the surrounding pane displays the following useful information:
Location: Fully-qualified URL at which you access the associated LogStream instance.
Egress Address: The instance's current public IP address. This address is dynamic; Cribl will occasionally update it when we need to rescale core infrastructure.
Last Updated: Date on which Cribl last pushed an infrastructure change (notably including changes to the above Egress Address).
Version: Your deployed LogStream version.
Latest indicates that you're in sync with the most-recent downloadable LogStream binary.
Region: The AWS Region where your LogStream instance is running.
The same page's Data Sources tab lists ports, and data ingestion inputs, that are open and available to use. Return to this tab to copy Ingest Addresses (endpoints) as needed.
This left tab is displayed only to an Organization's owner. It offers the following tabs along its top.
The Organization Details tab offers these controls to make your Cribl.Cloud deployment more recognizable than its randomly generated Organization ID:
Alias: Optionally, enter a "friendly" name for your Organization. Upon signing in, members will see this alias above the Organization ID on the Select Organization page.
Description: Optionally, use this field to add further details about your Organization. On the Select Organization page, members can view these details by expanding the Organization's tile.
Click Save to immediately apply your changes.
There is also a Delete Organization button at the lower left. Don't click that. (Not unless you mean it.)
The Organization > Members tab provides access to inviting and managing other users.
The Learning page links to everything you need to learn about LogStream, to goat forth and do great things:
- Sandboxes (free, interactive tutorials on fully hosted integrations).
- LogStream versions comparison.
- Concept/demo videos.
If you prefer to take the blue pill, this page offers download links for Cribl's LogStream and AppScope software. You can download either binaries or Docker containers (hosting Ubuntu 20.04) to install and manage on your own hardware or virtual machines.
This tab offers a self-explanatory Sign Out link, and a link to the Select Organization page, where you can traverse to other Organizations.
Once you've registered on the portal, here's how to access LogStream Cloud:
- Sign in to your Cribl.Cloud portal page.
- Select the Organization to work with.
- From the protal's Overview tab, select Connect to your LogStream.
- The LogStream Cloud UI will open in a new tab or window – ready to goat!
Note the Tenant ID link at the LogStream Cloud home page's upper left, under the Welcome! message. You can click this link to reopen the Cribl.Cloud portal page, to access Data Sources configurations. To return to this home page from anywhere else in LogStream Cloud's UI, click the LogStream logo in the upper-left corner.
From the Organization > Members tab, an Organization's owner can invite new users to join the Organization, assign access roles to new and existing members, and remove pending invites and/or existing members.
Click + Invite Member to open the modal shown below. Enter the Email address of the new user you want to invite, assign them a Role (explained just below), and then click Invite to send the invitation.
Each Role that you can assign to members confers a default Role within the Organization's LogStream instance. Here are the Roles, their corresponding permissions, and who can assign each:
|Member Role||LogStream Role||Options/Restrictions|
|Admin||Any Organization owner can assign|
|Editor||Assignable only with Enterprise plan|
|Read-Only||Assignable only with Enterprise plan|
|Owner||Can't be assigned, but can manage Organization details|
Full role-based access control is available only with an Enterprise plan. (For all available Enterprise features, see LogStream Pricing.)
Owners of non-Enterprise Cribl.Cloud Organizations will be able to assign only the
AdminRole in the Invite User modal shown above.
The one Member Role that you cannot assign or transfer is your own Owner Role. A user can acquire this superuser Role only by signing up as the owner of their own Cribl.Cloud Organization.
Only an Organization's Owner can manage the Organization's details.
When you assign a Cribl.Cloud Member Role, it is mapped to a LogStream Role as described above. However, these users will not be visible as local users within the LogStream UI.
At the address you entered, the new member receives an email with an Accept Invitation link to either sign into their existing Cribl.Cloud account, or else sign up to create an account and its credentials.
After signing in, they'll have access to your Organization and LogStream instance at the Role level you've specified.
While an invite is pending, the Organization > Members tab offers you these options to deal with commonly encountered issues:
Reinvite: If your invited member didn't receive your invitation email, you can click this button to resend it.
Copy Link: If emails aren't getting through at all, click this button to copy and share a URL that will take the invitee directly to the signup page. This target page encapsulates the same identity, Organization, and Role you specified in the original email invite.
Remove: This is for scenarios where you need to revoke a pending invite. (You sent someone a duplicate invite, your invitee is spending too much time in space to be a productive collaborator, etc.) After clicking this button, you'll see a confirmation dialog.
After 7 days, if an invite has been neither accepted nor revoked, it expires. In this case, it is removed from the Members tab.
Once a user has accepted an invite, the Organization > Members tab offers you these options to modify their membership in your Organization:
Edit: Switch this member to a different Role. (The Edit option is displayed only if you have an Enterprise plan.)
Remove: Remove this member from your Organization. After clicking this button, you'll see a confirmation dialog. (Proceeding will not affect this user's access to any other Cribl.Cloud Organizations they might own or be members of.)
Beyond the free tier, an optional paid LogStream Cloud account offers support from 8am–5pm, plus the ability to expand to 5 TB/day of throughput. In the Cribl.Cloud portal, select Request Pricing to talk with Cribl about upgrading your free account.
You'll pay only for what you use – the data you send to LogStream, and the data sent to external destinations. However, data sent to your AWS S3 storage is always free.
|Data Direction||Monthly Charge||Annual Charge|
|Price per GB sent in to LogStream||$0.15/GB||$0.125/GB|
|Price per GB sent out to external destinations||$0.15/GB||$0.125/GB|
Assume that you want to send 1,000 GB/day to LogStream. You reduce that data by 40% (a standard reduction that we see every day for customers). And you send the remaining 600 GB to an external destination:
(1,000 GB/day in + 600 GB/day out) x $0.15 (monthly rate per GB) = $240/day
Note that you can send a copy of all of your data to an S3 location of your choice, for no added data-egress charge.
LogStream Cloud differs from a deployed LogStream binary in the following ways. Keep all these differences in mind as you navigate LogStream's current UI, in-app help (including tooltips), and documentation.
Compared to a LogStream binary that you deploy on your own infrastructure, LogStream Cloud's left nav, and particularly its Settings page, are much simpler.
Here are the key options streamlined out of the Cloud version.
The Settings > Worker Processes and Settings > Distributed Settings links are omitted, and the left nav contains no Worker Groups or Mappings links. LogStream Cloud is configured like a distributed deployment with a single Worker Group. All Workers will share the same configuration.
The Settings > Distributed Settings > Git Settings section is omitted. A local
git client is preconfigured in your Cribl.Cloud portal. On LogStream Cloud's left nav, use the Changes link to commit/push changes to
git. Select Deploy at the UI's top right to deploy your committed changes. LogStream Cloud does not support Git remote repos.
The Settings > Controls and Settings > Upgrade links are omitted. Cribl handles restarts and version upgrades automatically on your behalf.
The Settings > Access Management section is omitted. All users of a given LogStream Cloud instance share a single
The Settings > Security section is omitted. Certificates are predefined for you on the Cribl.Cloud portal's Data Sources tab (see Available Ports and TLS Configurations below). LogStream Cloud does not support KMS secrets stores.
The Settings > Licensing link is omitted. Your license is managed by your parent Cribl.Cloud portal.
The Settings > Scripts link is omitted. LogStream Cloud does not support configuring or running shell scripts.
The Settings > Diagnostics link is omitted. For help with any troubleshooting needs:
- Click the Intercom link at LogStream's lower right.
- Join Cribl's Community Slack
- If you have a paid account, contact Cribl Support.
To get data into LogStream Cloud, your Cribl.Cloud portal provides several data sources and ports already enabled for you, plus 10 additional ports (
20010) that you can use to add and configure more LogStream Sources.
The Cribl.Cloud portal's Data Sources tab displays the pre‑enabled Sources, their endpoints, reserved and available ports, and protocol details. For the existing Sources listed here, Cribl recommends using the preconfigured endpoint and port to send data into LogStream.
TLS encryption is enabled for you on several Sources, also indicated on the Cribl.Cloud portal's Data Sources tab. All TLS is terminated by the Network Load Balancer (NLB) sitting in front of the Workers.
Currently, LogStream Cloud does not enable you to import your own certificates for mutual TLS authentication. LogStream Cloud uses TLS to provide encryption in the wire, but leaves authentication at the protocol layer – e.g., Splunk HEC or S2S tokens, Kafka authorization, etc.
LogStream Cloud provides no Filesystem Source, Filesystem Collector, or Filesystem Destination. (A Cloud deployment has no local filesystem to read from or write to.)
Several commonly used Sources are preconfigured for you, within LogStream Cloud's UI, and ready to use.
In a preconfigured Source's configuration, never change the Address field, even though the UI shows an editable field. If you change these fields' value, the Source will not work as expected.
After you create a Source and deploy the changes, it can take a few minutes for the Source to become available in LogStream Cloud's load balancer. However, LogStream will open the port and be able to receive data immediately.
Several LogStream Cloud Sources' configuration modals include a TLS Settings (Server Side) tab, inherited from the LogStream binary's UI. Do not set this tab's Enabled slider to
Yes, nor configure any of the resulting fields. Any settings that you configure will conflict with LogStream Cloud Sources' predefined TLS configurations.
Here are some common questions and answers about LogStream Cloud's current state and anticipated evolution.
As of August/September 2021, there is a charge, because the data must pass through a public internet interface. Cribl is exploring ways to mitigate this using virtual networks.
As of August/September 2021, it is important to size for anticipated usage. When you anticipate higher throughput, you must communicate this to Cribl, so we can provision additional Workers. Cribl is exploring automated provisioning.
Later in 2021, Cribl anticipates separating the control plane from the data plane. You will then be able to manage all your LogStream Worker Groups (cloud and/or on-prem) centrally through the Cloud interface.
If you use an AWS load balancer, use only a Classic Load Balancer. LogStream Cloud currently defaults to using AWS Network Load Balancers (NLBs) in front of each Cloud stack, and KDF is not compatible with NLBs or Application Load Balancers. (For details, see Amazon's Troubleshooting Amazon Kinesis Data Firehose documentation.)
Also, enable duration-based sticky sessions with cookie expiration disabled. (For details, see Amazon's Duration-Based Session Stickiness documentation.)
As an alternative KDF, you can write to an S3 bucket and use LogStream Cloud's native SQS-based S3 Source. You can also use a Lambda function to define the source data, connecting it to a LogStream Cloud REST Collector with HTTP discovery.
Updated 6 days ago