Add a lakehouse engine so you can ingest data directly into Cribl Search.

Highlights

​

• Lakehouse engines store and accelerate your data within Cribl Search.
• Choose an engine size that covers your raw, uncompressed daily ingest. Resize or add more engines as needed.
• Storage auto-scales with your data volume and retention settings.

Lakehouse Engines Store and Accelerate Data

​

When you send data into Cribl Search with Sources, it’s parsed by Datatypes and stored in Search Datasets on a lakehouse engine.

A Lakehouse engine is a storage-plus-compute unit that ingests, stores, and accelerates your data inside Cribl Search. It can keep your data hot for up to 10 years, with no storage tiering to manage.

Each lakehouse engine:

• Accepts data from one or more Sources.
• Uses Datatypes to break events into fields.
• Stores data in Search Datasets until their retention expires.
• Powers fast, schema-aware searches and AI workflows over that stored data.

Lakehouse Engine Sizes

​

To choose the right lakehouse engine size, think about the amount of raw, uncompressed data you expect per day, then include headroom for spikes and growth.

If your ingest rate changes, or you experience ingest or search latency, you can resize your lakehouse engine. If the available sizes are not enough, you can add more lakehouse engines to distribute the workload.

Lakehouse Engine Sizes Available

​

The ingest rate limit applies only to raw incoming data, not to fields you add or transform during processing.

Lakehouse Engine Compression Ratio

​

Cribl Search compresses ingested data at rest. The exact compression ratio depends on many factors, including the shape and content of your events, but it typically falls between 10:1 and 12:1.

Estimate Lakehouse Engine Costs

​

Because engine size acts as a hard limit on ingest, your costs are bounded, with no surprises from traffic spikes. You can scale your lakehouse engine up or down at any time to match your actual data needs.

With each lakehouse engine, you’re charged for two things:

To estimate and optimize storage, set individual retention periods of your Search Datasets. See Plan Your Search Datasets for details.

To see how engine size and storage translate to costs, see Cribl Search Pricing.

Lakehouse Engine Retention

​

You don’t set retention for a lakehouse engine as a whole, but for each of its Search Datasets individually. Each Search Dataset can keep data for 1 day to 10 years, and its storage scales accordingly. After the retention period ends, Cribl Search deletes the data.

For details, see Create Search Datasets and Organize Data with Dataset Rules.

Add a New Lakehouse Engine

​

Search Admins and above can add lakehouse engines from the Cribl Search Engines tab.

1. On the Cribl.Cloud top bar, select Products > Search > Data.
2. Select the Engines tab, then Add Engine.
3. Give your engine an ID (for example, palo_alto_logs) unique across your Workspace. You won’t be able to change it later.

   The main ID is reserved.

4. Set the Lakehouse engine Size. You can resize it later if needed.

   See Lakehouse Engine Sizes.

5. Confirm with Save.

When the lakehouse engine status is Ready, you can create Search Datasets and connect your Sources.

Check Lakehouse Engine Status

​

Select Refresh page to check for status updates.

Resize a Lakehouse Engine

​

Search Admins and above can resize lakehouse engines from the Cribl Search Engines tab.

1. On the Cribl.Cloud top bar, select Products > Search > Data > Engines.
2. Select the lakehouse engine you want to resize.
3. Set the new lakehouse engine Size. See Lakehouse Engine Sizes.
4. Confirm with Save.

Wait until the lakehouse engine status changes from Resizing to Ready again.

Next Steps

​

Now that your lakehouse engine is ready, create Search Datasets to organize your data and set retention.