What are Pipelines
After the data has been matched by a route it gets delivered to a pipeline. A pipeline is set of functions that work on the data and that are composed in a very specific list. Similar to routes, the order in which the functions are listed matters.
Function evaluation order matters!
Functions in a pipeline are evaluated in order, top down.
How do Pipelines Work
Events are always delivered at the beginning of a pipeline via a route . They are processed by each function, in order. A pipeline of chained functions will always move events forward. When they exit the pipeline, they cannot be processed again by another one. Their next step is out of the system. This is on purpose so as to keep the design simple and avoid potential loops.
Other Considerations
Functions in a pipeline are equipped with their own filters. Even though they're not required, it advised that they're used as often as possible. Similar to routes, the general goal is to minimize extra work that a function will do; the fewer events a function has to operate on the better the overall performance. For example, if a pipeline has two functions, f1--f2 and if f1 operates on source 'foo' and f2 that operates on source 'bar' it may make sense to apply source=='foo' and source=='bar' filters on each one respectively.