The Code Function provides a way to execute custom JavaScript logic directly within your data processing pipeline. This Function helps you solve complex data transformation challenges that require procedural logic, loops, or conditional handling beyond what the standard Functions provide.

For Advanced Users Only

The Code Function should only be configured by experienced JavaScript developers. Because this Function executes custom scripts, poorly validated code can cause issues such as infinite loops or failed returns, which can significantly increase your throughput volume or degrade system performance.

Use this Function with extreme care. Always test and validate your logic in a development environment before deploying to production.

Cribl Stream’s standard Functions, such as Eval, cover the majority of scenarios that users typically need to implement. You should use Code Functions only as a last resort, such as when you need to construct a complex block of code.

Use this Function to:

• Execute custom or complex procedural logic: Perform multi-step transformations that require variables, loops, and nested conditional statements.
• Handle non-standard data structures: Parse or rebuild deeply nested objects or custom proprietary formats that don’t fit into standard schemas or patterns.

How the Code Function Works

​

The Code Function processes each incoming event in the following order:

1. Input validation: The Function identifies the incoming event and ensures it is available for processing within the JavaScript environment.

2. Script execution: Cribl Stream executes your custom code against the event. Your logic can create new fields or it can read, modify, or delete existing ones.

3. Error handling and logging: If the script encounters an error (such as a syntax issue or a runtime exception), the Function captures the error. It then logs the diagnostic information based on your configured error log sample rate. See Advanced Settings for more information about this setting.

   The Code Function executes your script line-by-line. If an error occurs midway through your code, the Function stops immediately for that event. However, any changes made to the event prior to the error are retained. This can result in partially processed events entering your Pipeline. See the example in the Partially Processed Events section of the troubleshooting guide for more information.

4. Event return: The Function returns the processed event back to the Pipeline for additional processing or routing.

Prohibited Elements

​

Generally, any syntax forbidden in JavaScript Strict Mode is also forbidden within the Code Function. To ensure system stability and security, Cribl Stream disallows using the following JavaScript features and objects.

The following JavaScript features and objects are not allowed:

• Global objects: window, global, globalThis.
• Timers and async: Promises, setTimeout, setInterval.
• Dynamic execution: eval, uneval, Function (constructor).
• Logging: console (use the C.log utility instead).
• Data Structures: set.

Supported Logic

​

Cribl Stream supports the ECMAScript 2015 Language Specification. This means you can use standard procedural logic and array methods including loops, array methods, and standard math.

With some exceptions, the Code Function supports the options described in the following MDN JavaScript Guide topics:

• Expressions and Operators
• Global variables
• Control flow and error handling
• Loops and iteration
• Functions
• Numbers and dates
• Text formatting
• Regular Expressions
• Indexed collections
• Keyed collections
• Working with Objects

Access to C.* Global Object

​

From the Code’s Function body, you can access global C.* object in Cribl Stream and its associated Cribl Expressions.

Custom Functions

​

You can define your own functions to better organize your code. Both traditional and arrow functions are allowed.

Configure the Code Function

​

To configure this Function:

1. Add a Source and Destination to begin generating data. See Integrations for more information.

2. Add a new Pipeline or open an existing Pipeline. Add or capture sample data test your Pipeline as you build it. See Pipelines for more information.

3. At the top of the Pipeline, select Add Function and search for Code, then select it.

4. In the Code modal, configure the following general settings:

   • Filter: A JavaScript filter expression that selects which data to process through the Function. Defaults to true, meaning it evaluates all data events. If you don’t want to use the default, you can use a JavaScript expression to apply this Function to specific data events. See Build Custom Logic to Route and Process Your Data for examples.

   • Description: A brief description of how this Function modifies your data to help other users understand its purpose later. Defaults to empty.

   • Final: When enabled, stops data from continuing to downstream Functions for additional processing. Default: Off. Toggle Final on if the Code Function is the last Function in your Pipeline.

   • Code: The JavaScript editor where you define your transformation logic. To interact with data in a Code Function, use the __e reserved variable. This variable represents the current event context. For example, to access a field named field-name, use the following bracket notation:

     __e['field-name']

     Copy

     You can conceptualize your code as executing within a wrapper such as:

     function(__e: Event) {
     // your code here
     }

     Copy

     See Usage Examples for specific code examples.

5. Configure additional settings as needed. For more information, see Advanced Settings.

6. Test that you configured the Function correctly by comparing sample incoming data with outgoing data in the Data Preview pane and Pipeline Diagnostics tool. See Troubleshoot the Code Function for additional information about troubleshooting your code.

Advanced Settings

​

Iteration limit: The maximum number of loops (iterations) allowed per execution of this Function. This safety boundary prevents infinite loops from hanging the system. Default: 5000; Maximum: 10000.

Error log sample rate: The frequency at which JavaScript runtime errors are written to the system log. For example, 1 logs every error, while 10 logs every tenth occurrence. Use a higher value to prevent log flooding during high-volume processing. Default: 1000; Maximum: 1,000,000.

Use unique log channel: When enabled, Cribl Stream creates a specific log channel for this instance: func:code:${pipelineName}:${functionIndex}. This is recommended for troubleshooting specific Pipelines. When toggled off, logs are sent to the generic func:code channel.

Usage Examples

​

The following examples use a common sample event representing CPU metrics. You can copy this JSON object into the Sample Data pane to test the code snippets and compare the outputs of different code operations.

{
  "cpus": [
    {"number": 1, "name": "CPU1", "value": 2.3},
    {"number": 2, "name": "CPU2", "value": 3.1},
    {"number": 3, "name": "CPU3", "value": 5.1},
    {"number": 4, "name": "CPU4", "value": 1.3}
  ],
  "arch": "Intel x64"
}

Filter Arrays

​

Use the filter() method to remove noise from an array by keeping only the elements that meet a specific condition. In this case, the goal is to only retain CPU objects where the value is greater than or equal to 3. This allows for more fine-grained filtering by nested elements in an array.

// Create a new field containing only high-value CPU metrics
__e['cpus_filtered'] = __e['cpus'].filter(entry => entry.value >= 3);

The output now only contains events 2 and 3, which were greater than 3:

{
  "cpus_filtered": [
    {"number": 2, "name": "CPU2", "value": 3.1},
    {"number": 3, "name": "CPU3", "value": 5.1}
  ]
}

Logic-Based Conditional Mapping

​

Use the Code Function to apply complex, range-based logic that would be difficult to manage in a standard Lookup table. This allows you to dynamically inject fields based on numeric thresholds or multiple combined conditions. To accomplish this, use a map() function with conditional logic to categorize data based on its value.

This code example adds severity or status levels to metrics dynamically:

// Add a status field to each CPU based on its load value
__e['cpus'] = __e['cpus'].map(entry => {
    let status = (entry.value > 4) ? 'CRITICAL' : 'NORMAL';
    return Object.assign(entry, { status });
});

The resulting output now has a status indicator based on its value:

{
  "cpus": [
    {"number": 1, "name": "CPU1", "value": 2.3, "status": "NORMAL"},
    {"number": 3, "name": "CPU3", "value": 5.1, "status": "CRITICAL"}
  ]
}

Modify Array Elements

​

Use the map() method to transform every element within an array. This example uses Object.assign to safely update the name field to lowercase for every CPU without losing other data points like number or value.

Code:

// Transform all CPU names to lowercase for naming consistency
__e['cpus'] = __e['cpus'].map(entry => Object.assign(entry, {'name': entry.name.toLowerCase()}));

The resulting output now uses lowercase name values instead of uppercase:

{
  "cpus": [
    {"number": 1, "name": "cpu1", "value": 2.3},
    {"number": 2, "name": "cpu2", "value": 3.1}
  ]
}

Troubleshoot the Code Function

​

Because the Code Function executes custom JavaScript, issues usually stem from script syntax, resource limits, or unintended logic loops. Use the following guide to identify and resolve common errors.

Identify Error Types

​

When a Code Function fails, Cribl Stream logs the error to the internal logs. You can view these by clicking the Logs tab in the Preview pane.

Debug

​

The Code Function provides a built-in debug() method to help you trace logic without flooding your production logs.

Messages logged by this debug helper function are shown in the Preview Log by default.

You can also route these messages to regular logs, although this requires setting the Function’s log level (func:code) to debug.

// Example: Debugging a conditional branch
if (__e.status === 'CRITICAL') {
    debug('Processing a critical event', { eventId: __e._id });
    // your logic here
}

Preview

​

By using the expanded editor, you can run the expression against a sample event, and can preview the transformation:

Partially Processed Events

​

If you notice that some events have new fields while others are missing expected transformations, you are likely encountering a runtime interruption. Because Cribl Stream does not roll back changes when a script fails, an error on line 5 will preserve any modifications made on lines 1 through 4.

To find where your script is failing, compare your code against an event that looks “half-processed.” The point of failure is almost always the first missing field in your logic.

For example, take the following Code Function definition, which includes an if statement error in lines 6-8:

1
2
3
4
5
6
7
8
9

if (__e.name === 'Mario') {
 __e.maybePlumber = true;
}
const MAX_LENGTH = 100;
// The if statement does not check for undefined/null values of address
if (__e.address.length > MAX_LENGTH) {
 __e.addressTruncated = __e.address.slice(0, MAX_LENGTH);
}
__e.lastName = __e.lastName.toUpperCase();

When you run this example against the code:

const event1 = {
 name: 'Mario',
 lastName: 'Mario',
 address: '123 Plumber St, NYC'
};

It will produce the expected output:

const event1Processed = {
  name: 'Mario',
  lastName: 'MARIO',
  address: '123 Plumber St, NYC',
  maybePlumber: true
}

However, the code will fail when it receives this event, which has an undefined value for the address field:

const event2 = {
  name: 'Mario'
}

This event is only partially processed because the error occurred upstream of the address field logic. Since the Code Function executes line-by-line, the script terminated before reaching the instructions for the address field. The resulting output is incomplete:

const event2Processed = {
  name: 'Mario',
  lastName: 'Pablo',
  maybePlumber: true
};