On This Page

Home / Edge/ Collecting Data/AppScope Source

AppScope Source

AppScope is an open-source instrumentation utility from Cribl. It offers visibility into any Linux command or application, regardless of runtime, with no code modification. For details about configuring the AppScope CLI, loader, and library, see: https://appscope.dev/docs. Note that AppScope is no longer being actively developed by Cribl.

Type: Push | TLS Support: YES | Event Breaker Support: YES

Configuring Cribl Edge to Receive AppScope Data
​

From the top nav, click Manage, then select a Fleet to configure. Next, you have two options:

To configure via the graphical QuickConnect UI, click Routing > QuickConnect (Stream) or Collect (Edge). Next, click Add Source at left. From the resulting drawer’s tiles, select [System and Internal >] AppScope. Next, click either Add Destination or (if displayed) Select Existing. The resulting drawer will provide the options below.

Or, to configure via the Routing UI, click Data > Sources (Stream) or More > Sources (Edge). From the resulting page’s tiles or left nav, select [System and Internal >] AppScope. Next, click New Source to open a New Source modal that provides the options below.

Downloading AppScope
​

Cribl Edge must download the AppScope package from the Cribl CDN the first time it performs an operation on this Source. This will also update the AppScope package’s version, if a newer one is available in the CDN.

If Cribl Edge cannot access the CDN (for example, if you are working on an airgapped instance), you should manually download the binary and place it in $CRIBL_HOME/state/download/scope.

General Settings
​

Input ID: Enter a unique name to identify this AppScope Source definition. If you clone this Source, Cribl Edge will add -CLONE to the original Input ID.s

UNIX domain socket: When toggled to Yes, exposes the following two fields to specify a file-backed UNIX domain socket connection to listen on.

  • UNIX socket path: Path to the UNIX domain socket. Defaults to $CRIBL_HOME/state/appscope.sock.
  • UNIX socket permissions: Permissions to set for this socket, for example, 777. If empty, Cribl Edge will use the runtime user’s default permissions.

When UNIX domain socket is set to No, you instead see the following two fields to specify a network host and port.

  • Address: Enter the hostname/IP on which to listen for AppScope data. (For example, localhost.) Defaults to 0.0.0.0, meaning all addresses.
  • Port: Enter the port number to listen on.

By default:

  • In Cribl Stream, UNIX domain socket is set to No, with default network connections (address and port) of 0.0.0.0:10090 for TCP, and 0.0.0.0:10091 for TLS, respectively.
  • In Cribl Edge, UNIX domain socket is set to Yes, with a UNIX socket path of $CRIBL_HOME/state/appscope.sock.

Authentication Settings
​

Use the Authentication method drop-down to select one of these options:

  • Manual: Use this default option to enter the shared secret clients must provide in the authToken header field. Click Generate if you need a new auth token. If empty, unauthorized access will be permitted.

  • Secret: This option exposes an Auth token (text secret) drop-down, in which you can select a stored secret that references the auth token described above. The secret can reside in Cribl Edge’s internal secrets manager or (if enabled) in an external KMS. Click Create if you need to configure a new secret.

Optional Settings
​

UNIX socket permissions: Permissions to set for socket. For the preconfigured in_appscope source, defaults to 777. When creating a new AppScope Source, you should set this to 777. If empty, falls back to the runtime user’s default permissions.

Tags: Optionally, add tags that you can use to filter and group Sources in Cribl Edge’s Manage Sources page. These tags aren’t added to processed events. Use a tab or hard return between (arbitrary) tag names.

TLS Settings (Server Side)
​

This left tab is displayed only when the Optional Settings > UNIX domain socket toggle is set to No. It provides the following options.

Enabled defaults to No. When toggled to Yes:

Certificate name: Name of the predefined certificate.

Private key path: Server path containing the private key (in PEM format) to use. Path can reference $ENV_VARS.

Passphrase: Passphrase to use to decrypt private key.

Certificate path: Server path containing certificates (in PEM format) to use. Path can reference $ENV_VARS.

CA certificate path: Server path containing CA certificates (in PEM format) to use. Path can reference $ENV_VARS.

Authenticate client (mutual auth): Require clients to present their certificates. Used to perform mutual authentication using SSL certs. Defaults to No. When toggled to Yes:

  • Validate client certs: Reject certificates that are not authorized by a CA in the CA certificate path, or by another trusted CA (for example, the system’s CA). Defaults to Yes.

  • Common Name: Regex that a peer certificate’s subject attribute must match in order to connect. Defaults to .*. Matches on the substring after CN=. As needed, escape regex tokens to match literal characters. (For example, to match the subject CN=worker.cribl.local, you would enter: worker\.cribl\.local.) If the subject attribute contains Subject Alternative Name (SAN) entries, the Source will check the regex against all of those but ignore the Common Name (CN) entry (if any). If the certificate has no SAN extension, the Source will check the regex against the single name in the CN.

Minimum TLS version: Optionally, select the minimum TLS version to accept from connections.

Maximum TLS version: Optionally, select the maximum TLS version to accept from connections.

AppScope Rules
​

The AppScope Rules settings are available:

  • In Cribl Stream single-instance – but not Cribl.Cloud – deployments.
  • In Cribl Edge, both single-instance and Cloud.

Rules: Click Add Rule to include processes to scope, and to link to an AppScope config. Once you have saved the configuration, and committed and deployed your changes, AppScope will instrument any process that matches a Rule, on any Edge Node in the Fleet.

(When no Rules are defined, you can still scope by PID in the Edge Processes page. Scope by PID only instruments a single process running in one Edge Node.)

  • Process name: Matches if the string value you enter corresponds to the basename of the scoped process.

  • Process argument: Matches if the string value you enter appears as a substring anywhere in the scoped process’ full command line (including options and arguments).

  • AppScope config: Select an AppScope config.

Transport override: Enter a URL to override aspects of the transport configuration, such as the hostname, port, or TLS settings. For details, see Transport Override Details.

Transport Override Details
​

In scenarios like the following, use the Transport override option to extend the defaults in AppScope’s transport configuration:

  • When this Source is set to TCP mode, it typically listens on the default address 0.0.0.0. Scoped clients will need a specific IP or hostname to connect. In these cases, set Transport override URL to a specific IP/hostname (example format: tcp://my.host.name). This Source will parse the URL and look for the hostname and port, then use those values to override what would otherwise be sent to the scope start command.

  • When this Source is set to UNIX domain socket, it listens by default on $CRIBL_HOME/state/appscope.sock. The socket is often created on a mounted volume in a container. The path to that socket might be different outside the container, or when mounted into another container. In these cases, set the Transport override URL to specify an alternative path (example format: unix:///some/other/volume/appscope.sock).

Persistent Queue Settings
​

In this section, you can optionally specify persistent queue storage, using the following controls. This will buffer and preserve incoming events when a downstream Destination is down, or exhibiting backpressure.

On Cribl-managed Cribl.Cloud Workers (with an Enterprise plan), this tab exposes only the Enable Persistent Queue toggle. If enabled, PQ is automatically configured in Always On mode, with a maximum queue size of 1 GB disk space allocated per PQ‑enabled Source, per Worker Process.

The 1 GB limit is on uncompressed inbound data, and no compression is applied to the queue. This limit is not configurable. For configurable queue size, compression, mode, and other options below, use a hybrid Group.

Enable Persistent Queue: Defaults to No. When toggled to Yes:

Mode: Select a condition for engaging persistent queues.

  • Always On: This default option will always write events to the persistent queue, before forwarding them to Cribl Edge’s data processing engine.
  • Smart: This option will engage PQ only when the Source detects backpressure from Cribl Edge’s data processing engine.

Max buffer size: The maximum number of events to hold in memory before reporting backpressure to the sender and writing the queue to disk. Defaults to 1000. (This buffer is per connection, not just per Worker Process – and this can dramatically expand memory usage.)

Commit frequency: The number of events to send downstream before committing that Stream has read them. Defaults to 42.

Max file size: The maximum data volume to store in each queue file before closing it and (optionally) applying the configured Compression. Enter a numeral with units of KB, MB, etc. If not specified, Cribl Edge applies the default 1 MB.

Max queue size: The maximum amount of disk space that the queue is allowed to consume on each Worker Process. Once this limit is reached, this Source will stop queueing data and block incoming data. Required, and defaults to 5 GB. Accepts positive numbers with units of KB, MB, GB, etc. Can be set as high as 1 TB, unless you’ve configured a different Max PQ size per Worker Process in Fleet Settings.

Queue file path: The location for the persistent queue files. Defaults to $CRIBL_HOME/state/queues. To this field’s specified path, Cribl Edge will append /<worker-id>/inputs/<input-id>.

Compression: Optional codec to compress the persisted data after a file is closed. Defaults to None; Gzip is also available.

In Cribl Edge 4.1 and later, Source-side PQ’s default Mode is Always on, to best ensure events’ delivery. For details on optimizing this selection, see Always On versus Smart Mode.

You can optimize Workers’ startup connections and CPU load at Fleet Settings > Worker Processes.

Processing Settings
​

Event Breakers
​

Event Breaker rulesets: A list of event breaking rulesets that will be applied to the input data stream before the data is sent through the Routes. Defaults to System Default Rule.

Event Breaker buffer timeout: How long (in milliseconds) the Event Breaker will wait for new data to be sent to a specific channel, before flushing out the data stream, as-is, to the Routes. Minimum 10 ms, default 10000 (10 sec), maximum 43200000 (12 hours).

Fields
​

In this section, you can add Fields to each event using Eval-like functionality.

Name: Field name.

Value: JavaScript expression to compute field’s value, enclosed in quotes or backticks. (Can evaluate to a constant.)

Pre-Processing
​

In this section’s Pipeline drop-down list, you can select a single existing Pipeline to process data from this input before the data is sent through the Routes.

Disk Spooling
​

For Cribl Search to access the data that arrives at an AppScope Source, Disk Spooling must be enabled.

Enable disk persistence: Whether to save metrics to disk. When set to Yes, exposes this section’s remaining fields.

Bucket time span: The amount of time that data is held in each bucket before it’s written to disk. The default is 10 minutes (10m).

Max data size: Maximum disk space the persistent metrics can consume. Once reached, Cribl Edge will delete older data. Example values: 420 MB, 4 GB. Default value: 100 MB.

Max data age: How long to retain data. Once reached, Cribl Edge will delete older data. Example values: 2h, 4d. Default value: 24h (24 hours).

Compression: Optionally compress the data before sending. Defaults to gzip compression. Select none to send uncompressed data.

Path location: Path to write metrics to. Default value is $CRIBL_HOME/state/appscope.sock.

Advanced Settings
​

Enable proxy protocol: Toggle to Yes if the connection is proxied by a device that supports Proxy Protocol v1 or v2.

IP allowlist regex: Regex matching IP addresses that are allowed to establish a connection.

Max active connections: Maximum number of active connections allowed per Worker Process. Defaults to 1000; enter 0 to allow unlimited connections.

Environment: If you’re using GitOps, optionally use this field to specify a single Git branch on which to enable this configuration. If empty, the config will be enabled everywhere.

Connected Destinations
​

Select Send to Routes to enable conditional routing, filtering, and cloning of this Source’s data via the Routing table.

Select QuickConnect to send this Source’s data to one or more Destinations via independent, direct connections.

AppScope with Edge on Kubernetes
​

When Cribl Edge detects that a scope’d process is running inside a Kubernetes container, it reports the Kubernetes metadata as kube_** properties. It adds these to the incoming events and metrics, and you can view the combined events and metrics on the AppScope Source’s Live Data tab.

For Cribl Edge to detect that it’s running in a Kubernetes Pod, you must first set the CRIBL_K8S_POD environment variable.

Internal Fields
​

Cribl Edge uses a set of internal fields to assist in handling of data. These “meta” fields are not part of an event, but they are accessible, and Functions can use them to make processing decisions.

Fields for this Source:

  • __inputId
  • __srcIpPort

Examples
​

The following examples work only in Cribl Stream. You can vary the scoped commands (ps -ef and curl) as desired.

Cribl.Cloud – TLS
​

An in_appscope_tls TLS Source is preconfigured for you on Cribl.Cloud, using port 10090. You can send it AppScope data using this command:

./scope run -c <Your-Ingress-Address>:10090 -- ps -ef

Cribl Cloud – TCP
​

An in_appscope_tcp TCP Source is preconfigured for you on Cribl.Cloud, using port 10091. You can send it AppScope data using this command:

./scope run -c tcp://<Your-Ingress-Address>:10091 -- curl -so /dev/null \
https://wttr.in/94105

Periodic Logging
​

Cribl Edge logs metrics about incoming requests and ingested events once per minute.

These logs are stored in the metrics.log file. To view them in the UI, open the Source’s Logs tab and choose Worker Process X Metrics from the drop-down, where X is the desired Worker process.

This kind of periodic logging helps you determine whether a Source is in fact still healthy even when no data is coming in.