The Microsoft Copilot Source collects Microsoft 365 Copilot interaction data (user prompts and Copilot responses) from the Interaction Export API. On each scheduled run, the Source enumerates users in the tenant, queries each user’s Copilot interaction history since the last successful collection, and emits each interaction as an event.

This Source can backfill historic interactions, but it queries each user separately, so it is a poor fit for large tenants (hundreds or thousands of users).

Type: Pull | TLS Support: YES | Event Breaker Support: YES

TLS is enabled via HTTPS on the Microsoft Graph REST API that this Source uses. An active Leader is required for job management.

Prerequisites ​

To collect Microsoft 365 Copilot interaction data, you need:

• A registered application with the Microsoft identity platform
• The Directory (tenant) ID and Application (client) ID from that app
• Credentials (a client secret or certificate)

For a complete walkthrough, see Microsoft 365 App Registration for Cribl Sources.

Microsoft Permissions ​

Grant the registered application these application permissions. In Microsoft Entra ID, set Type to Application for both API permissions (Delegated is not sufficient), then grant admin consent in the tenant.

• AiEnterpriseInteraction.Read.All - Required to read Copilot enterprise interactions.
• User.Read.All - Required to enumerate tenant users during discovery.

Users whose interactions you collect must have a valid Microsoft 365 Copilot license, including the Microsoft Copilot with Graph-grounded chat service plan. See Get all enterprise interactions in the Microsoft documentation.

By default, the Source discovers only users who hold one of a pre-defined list of Copilot licenses, so you do not have to identify licensed users yourself. See Discovery Settings for that list and for how to edit it.

How the Source Collects Data ​

On each scheduled job, the Source:

1. Discovers users with GET /v1.0/users (paginated). When license filtering is on (the default), discovery includes only users who have an assigned Copilot SKU.
2. For each discovered user, collects interactions with GET /v1.0/copilot/users/{id}/interactionHistory/getAllEnterpriseInteractions.
3. Emits each aiInteraction as an event. _time comes from createdDateTime. The payload includes appClass, which identifies the Copilot app (Teams, Chat, Word, and so on).
4. Advances a global watermark on createdDateTime so the next run collects only newer interactions.

The Interaction Export API paginates by app folder. Items are not globally sorted by createdDateTime across pages, so the Source tracks a running maximum timestamp rather than the last item on the page.

On the Interaction Export API, a 403 for an unlicensed user is an expected per-user skip. The Source logs a warning and continues collecting from other users. The Source retries 429, 500, and 503 with exponential backoff, and rate-limited (429) requests honor the Retry-After header. If those retries are exhausted, the failure is fatal and the watermark does not advance for that run.

Configure a Microsoft Copilot Source ​

1. On the top bar, select Products, and then select Cribl Stream. Under Worker Groups, select a Worker Group. Next, you have two options:
   • To configure via QuickConnect, navigate to Routing > QuickConnect. Select Add Source and select the Source you want from the list, choosing either Select Existing or Add New.
   • To configure via the Routes, select Data > Sources. Select Microsoft Copilot. Next, select Add Source.
2. In the New Source modal, configure the following under General Settings:
   • Input ID: Enter a unique name to identify this Source definition. If you clone this Source, Cribl Stream will add -CLONE to the original Input ID.
   • Description: Optionally, enter a description.
   • Tenant ID: Enter the Directory (tenant) ID for your Microsoft 365 tenant.
   • Client ID: Enter the Application (client) ID for your Microsoft 365 app registration.
3. Under Authentication, select the Authentication method:
   • OAuth (client secret): Default. In Client secret, select or create a stored text secret that references the client secret from your app registration.
   • OAuth (certificate): Authenticate with a certificate from the app registration. Treat all fields here as required, except for the optional Passphrase:
     • Certificate: Use the drop-down to select an existing certificate, or select Create to store a new, reusable cert.
     • Private key path: Path to the private key in PEM format. You can reference $ENV_VARS.
     • Passphrase: If your private key requires a passphrase to decrypt it, enter that passphrase here.
     • Certificate path: Path to the certificate in PEM format. You can reference $ENV_VARS.
4. Optionally, configure the following under Optional Settings:
   • Subscription plan: Select the Microsoft 365 subscription plan that matches your Organization. This determines the authentication and Microsoft Graph endpoints that the Source calls. Defaults to Microsoft 365 Enterprise. Options:
     • Microsoft 365 Enterprise
     • Microsoft 365 GCC
     • Microsoft 365 GCC High
     • Microsoft 365 DoD
   • Schedule: Cron schedule for collection runs. Defaults to */15 * * * * (every 15 minutes).
   • Earliest: Start of the collection window, relative to now. Used as the initial lower bound on the first run. Defaults to -7d.
   • Latest: End of the collection window, relative to now. Defaults to now.
   • Page size: Number of interactions to request per page ($top). Defaults to 1000. Minimum 1, maximum 1000.
   • App class filter: Limit collection to specific Copilot app classes. Leave empty to collect all. Options:
     • Teams (incl. Meeting Copilot)
     • M365 Copilot Chat
     • Copilot web
     • Word
     • Excel
   • Tags: Optionally, add tags that you can use to filter and group Sources in Cribl Stream’s UI. These tags aren’t added to processed events. Use a tab or hard return between (arbitrary) tag names.
5. Optionally, you can adjust the Processing, Discovery, Retries, and Advanced settings, or Connected Destinations outlined in the sections below.
6. Select Save, then Commit & Deploy.

Processing Settings ​

Event Breakers ​

Event Breaker rulesets: A list of event breaking rulesets applied to the input data stream before data is sent through the Routes. Defaults to OData JSON. You can select additional Event Breaker rulesets if needed.

Event Breaker buffer timeout (ms): How long (in milliseconds) the Event Breaker waits for new data before flushing the stream. Defaults to 10000.

Fields ​

In this section, you can define new fields or modify existing ones using JavaScript expressions, similar to the Eval function.

• The Field Name can either be a new field (unique within the event) or an existing field name to modify its value.
• The Value is a JavaScript expression (enclosed in quotes or backticks) to compute the field’s value (can be a constant). Select this field’s advanced mode icon (far right) if you’d like to open a modal where you can work with sample data and iterate on results.

This flexibility means you can:

• Add new fields to enrich the event.
• Modify existing fields by overwriting their values.
• Compute logic or transformations using JavaScript expressions.

Pre-Processing ​

Select a single existing Pipeline or Pack from the Pipeline drop-down list to process data from this input before the data is sent through the Routes.

Discovery Settings ​

User discovery filtering by product license: Toggle on to filter the /users discovery call to users who have an assigned Copilot SKU. This excludes unlicensed users during discovery instead of skipping them later with 403 responses, which reduces the number of Microsoft Graph calls that each job makes. Defaults to toggled on.

Copilot SKU IDs: The Microsoft 365 SKU GUIDs that grant access to the Interaction Export API. A user is included in discovery when at least one of these SKUs appears in their assignedLicenses. This field is available only when User discovery filtering by product license is toggled on, and it comes pre-populated with the following known Copilot plans:

The list is editable. Remove any plan that your tenant does not use to narrow discovery further, and add the SKU GUID for any tenant-specific or newly released plan that is not listed. To look up a GUID, see Product names and service plan identifiers for licensing in the Microsoft documentation.

Filtering relies on eventually consistent Microsoft Graph advanced queries, so an unlicensed user can still appear in discovery. Those users return 403 during collection. The Source logs a warning and continues.

Toggle User discovery filtering by product license off only when you need to attempt collection for every user in the tenant, such as when your tenant uses a plan whose SKU GUID you cannot determine. With filtering off, the Source calls the Interaction Export API for every discovered user and skips unlicensed users on the resulting 403 responses.

Retries ​

Retry type: The algorithm to use when performing HTTP retries. Options include Backoff (the default), Static, and Disabled.

Initial retry interval (ms): Time interval between failed request and first retry (kickoff). Defaults to 1000. Maximum allowed value is 20,000 ms (1/3 minute). A value of 0 means retry immediately until reaching the limit specified in Retry limit.

Retry limit: Maximum number of times to retry a failed HTTP request. Defaults to 5. Maximum: 20. A value of 0 means don’t retry at all.

Backoff multiplier: Base for exponential backoff. A value of 2 (default) means that Cribl Stream will retry after 2 seconds, then 4 seconds, then 8 seconds, and so on.

Retry HTTP codes: List of HTTP codes that trigger a retry. Defaults to 429, 500, and 503. Cribl Stream does not retry codes in the 200 series.

Honor Retry-After header: When toggled on (the default) and the retry-after header is present, Cribl Stream honors any retry-after header that specifies a delay, up to a maximum of 20 seconds. Cribl Stream always ignores retry-after headers that specify a delay longer than 20 seconds.

Retry connection timeout: Toggle on to automatically retry a single connection attempt after a timeout (ETIMEDOUT) to ensure data continuity. Defaults to toggled off.

Retry connection reset: Toggle on to automatically retry a connection after a peer reset (ECONNRESET) to maintain data flow. Defaults to toggled off.

Advanced Settings ​

Request timeout (seconds): HTTP request inactivity timeout, in seconds. Defaults to 30. Enter 0 to wait indefinitely. Maximum 2400.

Job timeout: Maximum time the job is allowed to run (for example, 30, 45s, or 15m). Units are seconds if not specified. Enter 0 for unlimited time. Defaults to 60m.

Time to live: How long to keep the job’s artifacts on disk after job completion. This also affects how long a job is listed in Job Inspector. Defaults to 4h.

Manage State: View, modify, or delete the collection watermark. For more information, see Manage State. The Source always tracks state on createdDateTime.

Environment: If you’re using GitOps, optionally use this field to specify a single Git branch on which to enable this configuration. If empty, the config is enabled everywhere.

Connected Destinations ​

Select Send to Routes to enable conditional routing, filtering, and cloning of data from this Source via the Routing table.

Select QuickConnect to send data from this Source to one or more Destinations via independent, direct connections.

Internal Fields ​

Cribl Stream uses a set of internal fields to assist in handling of data. These “meta” fields are not part of an event, but they are accessible, and Functions can use them to make processing decisions.

• __inputId: Identifies the Source in the format microsoft_copilot:<input-id>.
• __collectible
• __final
• __isBroken
• __source

Troubleshooting ​

The Source’s configuration modal has helpful tabs for troubleshooting:

Live Data: Try capturing live data to see real-time events as they are ingested. On the Live Data tab, click Start Capture to begin viewing real-time data.

Logs: Review and search the logs that provide detailed information about the ingestion process, including any errors or warnings that may have occurred.

You can also view the Monitoring page that provides a comprehensive overview of data volume and rate, helping you identify ingestion issues. Analyze the graphs showing events and bytes in/out over time.

Common Issues ​

Collection Is Slow or Hits Graph Rate Limits ​

This Source creates one collection task per discovered user. Large tenants generate many Graph calls (user list plus per-user interaction pages). Enable license filtering, raise the Schedule interval, and narrow App class filter and Earliest/Latest.

403 for Some Users ​

The Interaction Export API returns 403 for users who don’t have a Copilot license. With license filtering on, those users should be excluded at discovery. If filtering is off, or if an unlicensed user still appears in discovery, the Source logs a warning and continues.

404 for Some Users ​

A 404 on a user’s Interaction Export request fails that collection task. The Source does not skip 404 responses. Check whether the user was deleted, and use Job Inspector to see the failed task.

Watermark Does Not Advance ​

A fatal error after retries of 429, 500, or 503 blocks the watermark for that run. Check Job Inspector, confirm Job timeout is high enough for the user count, and use Manage State only if you need to reset the latestCreatedAt watermark.

Newly Licensed Users Missing for One Cycle ​

License filtering uses eventually consistent Graph advanced queries. A license assigned just before a run might not appear until the next scheduled collection.