Regex Filtering

To filter events in real time (data in motion), we use the out-of-the-box Regex Filter Function. This is similar to nullqueueing with TRANSFORMS in Splunk, but the matching condition is way more flexible.

Regex Filtering Example​

Let’s see how we can filter out any sourcetype=='access_combined' events whose _raw field contains the pattern Opera:

First, make sure you have a Route and Pipeline configured to match desired events.

Next, let’s add a Regex Filter Function to it:

Defining the Regex Filter Function
Defining the Regex Filter Function

Next, verify that this search does not return any results: sourcetype="access_combined" Opera

You can add more conditions to the Filter input field. For example, to further limit the filtering to only events from hosts with domain dnto.ca, change the filter input to sourcetype=='access_combined' && host.endsWith('dnto.ca').

This is a very flexible method for filtering incoming events in real time, on virtually any arbitrary conditions.