Connect Cribl Search to Cribl Lake

Configure Cribl Search to query your Cribl Lake or Lakehouse data.

Cribl Lake Datasets work as Cribl Search Datasets out-of-the-box, so you can instantly start searching them.

You can also assign Lake Datasets to Lakehouses, for dramatically faster search responses and more-predictable billing.

For more information on Lake and Lakehouses, see the Cribl Lake docs.

Add a Cribl Lake Dataset Provider

​

Cribl Search comes with a preconfigured Dataset Provider for Cribl Lake (called cribl_lake), so you don’t need to add it explicitly.

You can see it by going to Products (on the top bar) > Search > Data > Dataset Providers.

Add a Cribl Lake Dataset

​

You add Lake Datasets in the Cribl Lake UI. To find out how, see the Cribl Lake docs: Create a New Dataset.

Send Cribl Search Results to Cribl Lake

​

After you create a Dataset in Cribl lake, you can send Cribl Search results to it, using the export operator. Here is a simple example:

// export Cribl Search results to an existing Lake Dataset
dataset="cribl_search_sample"
| export to myLakeDataset

(You can find more export examples here.)

Typically, you’re exporting data in which Cribl Search has already parsed field names and values. This makes the data compatible with Cribl Lakehouse. However, if you happen to send unparsed data, you won’t be able to search it at Lakehouse speed.

Search a Cribl Lake Dataset

​

You can query your Lake Datasets just like any other Search Datasets. See Searching Cribl Lake for some query examples.

Search a Cribl Lakehouse

​

Searching a Lake Dataset that’s assigned to a Lakehouse is significantly faster, as long as the query’s time range falls within the Lakehouse retention period.

If the query goes beyond the retention period, Cribl Search performs a regular search with corresponding latency.

To create a new Lakehouse, see the Cribl Lake docs: Add a New Lakehouse.

Find Out if a Dataset is Assigned to a Lakehouse

​

To see if a Dataset you’re about to search is assigned to a Lakehouse or not, look for the icon. For example, look at the list of available datasets in your Search Home:

You can also go to Products (on the top bar) > Search > Data > Datasets and look at the Lakehouse column.

Verify Lakehouse Use

​

To find out whether a search successfully used Cribl Lakehouse, take a look at the tracking bar.

If the search failed to use Lakehouse, the bar presents information about potential reasons.

Search Multiple Lakehouse Datasets

​

You can run a single query against multiple Lakehouse-assigned Datasets. For the query to execute at Lakehouse speed, all Datasets in the query must be Lakehouse-assigned, and your query must also meet one of these conditions:

• Include only operators from the following group: cribl, centralize, extend, extract, foldkeys, limit, mv-expand, pivot, project, project-away, project-rename, search, and where.

• Or, if you use any other operators, insert the centralize or limit operator before them. (Result counts will be further constrained by usage group limits.)

If neither of the above conditions is met, or if your query includes non-Lakehouse Datasets, the query will run at standard speed.

Disable Lakehouse Use

​

You can use a set statement with the lakehouse option to control whether a query uses Lakehouse or not.

For example, to test how a Dataset assigned to a Lakehouse performs without it:

set lakehouse="off" dataset="lakehouse_dataset"

Cribl Search Differences with Lakehouse

​

Executing Cribl Search queries against a Lakehouse-assigned Dataset changes some behavior and results, compared to executing the same queries without Lakehouse caching. For details, see Lakehouse Search Differences.