These docs are for Cribl Stream 4.10 and are no longer actively maintained.
See the latest version (4.13).
Updating Lookups in Multiple Packs
Lookups provide a flexible way to enrich events en route to their Destinations. You can use Lookups in multiple Packs to share complex configurations and workflows across multiple Worker Groups, or across organizations.
Here, we will be walking you through a particularly challenging setup: What happens if you have a very large Lookup that requires frequent updates? To make it more interesting, what happens if the same Lookup is being used in multiple Packs? This topic explains how to update a Lookup file in multiple Packs. To learn how to set up a large Lookup file that needs frequent updates, see Deploy an Updated Lookup File to Worker Nodes.
Updating Lookups in Multiple Packs
Manually updating multiple Packs with an ever-evolving Lookup table is a non-starter, so let’s examine a better alternative.
Create the Lookup File in the Worker Group
The first step is to create a Lookup file in the Worker Group. For ease of sharing and use, make sure the Lookup file is not located in the Pack.
Our example Lookup file is host_destination_test.csv
.
Validate that the Lookup File Is on the Worker Node
After you Commit and Deploy to propagate the Lookup table to the Workers, validate that you can view the Lookup.
In our example, the path is: $CRIBL_HOME/data/lookups/host_destination_test.csv
.
Point the Pack(s) to the Lookup File
In your Pack(s), add a Lookup Function and point it to the Worker Node Lookup file location. In our example, the path is: $CRIBL_HOME/data/lookups/host_destination_test.csv
.
Do not point the Pack to the Leader Lookup File location, such as:
$CRIBL_HOME/groups/<your group>/data/lookups/host_destination_test.csv
.
Also, do not point it directly to the Pack Lookup file, such ashost_destination_test.csv
.
