Parse and structure your data at ingest time, so you can search at speed from the start.

About Datatyping

​

When your data flows from Sources into a lakehouse engine, it’s parsed into structured events through a process called Datatyping. Here’s how it works:

1. Use AI Datatyping: First, let Cribl Search assign Datatypes automatically.
2. Check for uncategorized data: See if any events were missed.
3. Define your own Datatype rules: To handle the uncategorized data, map specific patterns to the existing stock Datatypes.
4. Add custom Datatypes: To parse data not covered by the stock Datatypes, edit or add entirely new Datatypes.

Want to know more about Datatyping? See Datatypes in Cribl Search.

Parse Using AI

​

By default, Cribl Search uses AI to analyze incoming events and automatically assign a matching Datatype. This requires no configuration on your part, and covers many common log types and data formats.

For most types of data, Datatyping just works.

Check for Uncategorized Data

​

Data that doesn’t match any Datatypes displays as Uncategorized.

1. On the Cribl.Cloud top bar, select Products > Search > Data > Get Data In > Datatyping (auto).
2. Under Uncategorized Data, select:
   • View Live Data to see a sample of uncategorized data as it arrives.
   • View Last 24h to run a search for uncategorized data from the past 24 hours.

Manually Match Existing Datatypes

​

To handle the uncategorized data, you can define your own rules that map specific data patterns to existing stock Datatypes.

For that, add a custom Datatype rule:

1. On the Cribl.Cloud top bar, select Products > Search > Data > Get Data In > Datatyping (auto).
2. Select Add Datatype Rule. Name and describe your rule.
3. In Kusto expression to match, enter a KQL expression that matches a subset of your data.

   For example, _raw contains "GET /login".

4. From Datatype, select an existing stock Datatype.

   For example, apache_httpd_accesslog_common.

5. Make sure that Enabled in the top right corner is checked, and confirm with Add.

Now, any log whose raw text contains GET /login will get the apache_httpd_accesslog_common Datatype assigned.

Add Custom Datatypes

​

To handle uncategorized data that’s not covered by the stock Datatypes, you can add custom Datatypes.