Collect logs from a variety of files on your systems using the File Monitor.

To collect logs from files (including text files, compressed, archived, and some binary log files), you can use the File Monitor.

For a full description and reference of the Source’s settings, see File Monitor Source.

The File Monitor Source collects those logs and converts them to events based on the extracted lines and records.

Cribl Edge comes with two preconfigured, but disabled File Monitor Sources: in_file_auto and in_file_varlog.

in_file_auto uses the Auto discovery mode, which automatically discovers files that running processes have open for writing. in_file_varlog is configured with the Manual discovery mode, which enables you to specify a single path from which Edge will collect log files. The predefined Source is aimed at the /var/log/ path.

Use Preconfigured File Monitor Sources

​

To use one of the preconfigured File Monitor Sources:

1. On the top bar, select Products, and then select Edge. Under Fleets, select a Fleet.
2. Select Collect.
3. Select the File Monitor stack to view both preconfigured Sources.
4. Select and drag the plus icon next to one of the Sources to a Destination (for starters, you can choose DevNull).
5. In the Connection Configuration modal, select how you want the data to flow through to the Destination. Select Passthru to send all the data without processing, then select Save.
6. To start the Source, select Commit, enter a commit name, and select Commit & Deploy.

Now, ensure that data is being collected: hover over the File Monitor Source and select Capture. After a short time you should see sample data captured by the Source. Once you’re satisfied, you can switch from DevNull to your desired target Destination.

Monitor Files in Specific Folders

​

To manually select a specific path to discover files in, configure a new File Monitor Source.

1. On the top bar, select Products, and then select Edge. Under Fleets, select a Fleet.
2. Select More > Sources. Select File Manager.
3. Next, select Add Source.
4. Enter a unique identifier for the Source in Input ID.
5. Make sure Discovery Mode is set to Manual.
6. In Search path, provide the path to the directory you want to discover. The path can’t contain wildcards.
7. Optionally, set Max depth to define whether collection should be limited to a specified depth of subdirectories. To limit collection to the specified Search path only, set Max depth to 0. A value of 1 will include only the next level of subdirectories, and higher values will dig deeper.

Further settings allow you to fine-tune the log collection.

You can control which files are monitored by filling in the Filename allowlist. For guidance on how to use allowlists without creating duplicates, see Using Allowlists Effectively.

To make sure that all desired files are read, you can fine-tune the length and composition of hashes that the File Monitor uses to track files. See File Tracking with Hashes for more information.