Skip to main content
Version: 3.2

Office 365 Message Trace

Cribl LogStream supports receiving Office 365 Message Trace data. This mail-flow metadata can be used to detect and report on malicious activity including bulk emails, spoofed-domain emails, and data exfiltration.

Type: Pull | TLS Support: YES | Event Breaker Support: YES

TLS is enabled via the HTTPS protocol on this Source's underlying REST API.

Office 365 Setup

Your Office 365 service account should include a role with Message Tracking and View‑Only Recipients permissions, assigned to the Office 365 user that will integrate with LogStream.

Configuring LogStream to Receive Office 365 Message Trace Data

From the top nav of a LogStream instance or Group, select Data > Sources. From the resulting page's tiles or the Sources left nav, select [Pull >] Office 365 > Message Trace. Next, click + Add New to open a New Source modal that provides the following options and fields.

This Source cannot currently be selected or enabled in the QuickConnect UI.

General Settings

Input ID: Enter a unique name to identify this Office 365 Message Trace definition.

Report URL: Enter the URL to use when retrieving report data. Defaults to: https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MessageTrace.

Poll interval: How often (in minutes) to run the report. Must divide evenly into 60 minutes to create a predictable schedule, or Save will fail. See About Polling Intervals below.

Username: Username with which to run the Message Trace API call.

Password: Password with which to run the Message Trace API call.

Date range start: The relative time in the past that begins the search date range. (E.g., -3h@h.) Message Trace data is delayed; this parameter (with Date range end) compensates for delay and gaps.

Date range end: The relative time in the past that ends the search date range. (E.g., -2h@h.) Message Trace data is delayed; this parameter (with Date range start) compensates for delay and gaps.

Log level: For data collection's runtime log, set the verbosity level to one of debug, info, warn, or error. (If not selected, defaults to info.)

About Polling Intervals

To poll the Office 365 Message Trace API, LogStream uses the Poll interval field's value to establish the cron schedule. (e.g.: */${interval} * * * *).

Because the interval is set in minutes, it must divide evenly into 60 minutes to create a predictable schedule. Dividing 60 by intervals like 1, 2, 3, 4, 5, 6, 10, 12, 15, 20, or 60 itself yields an integer, so you can enter any of these values.

LogStream will reject intervals like 23, 42, or 45, or 75 – which would yield non-integer results, meaning unpredictable schedules.

Processing Settings

Fields (Metadata)

In this section, you can add fields/metadata to each event, using Eval-like functionality.

Name: Field name.

Value: JavaScript expression to compute field's value (can be a constant).

Pre-Processing

In this section's Pipeline drop-down list, you can select a single existing Pipeline to process data from this input before the data is sent through the Routes.

Advanced Settings

Keep Alive time (seconds): How often Workers should check in with the scheduler to keep their job subscription alive. Defaults to 60.

Worker timeout (periods): The number of Keep Alive Time periods before an inactive Worker will have its job subscription revoked. Defaults to 3.

Timeout (secs): Maximum time to wait for an individual Message Trace API request to complete. Defaults to 600 seconds (10 minutes). Enter 0 to disable metering, allowing unlimited response time. Because there is a single request to the Message Trace API per page of data, this timeout is applied at the page (request) level.

Disable time filter: Disables Collector event time filtering when a date range is specified in General Settings. Toggle to No to allow filtering.

Environment: If you're using GitOps, optionally use this field to specify a single Git branch on which to enable this configuration. If empty, the config will be enabled everywhere.

Internal Fields

Cribl LogStream uses a set of internal fields to assist in handling of data. These "meta" fields are not part of an event, but they are accessible, and Functions can use them to make processing decisions.

Fields for this Source:

  • __final
  • __inputId
  • __isBroken
  • __source

How LogStream Pulls Data

The Office 365 Message Trace Source uses a scheduled REST Collector. It runs one collection task every Poll interval, and a single Worker will process the collection. The data is paginated, so the Worker might make multiple calls to fetch the data.

Viewing Scheduled Jobs

This Source executes LogStream's scheduled collection jobs. Once you've configured and saved the Source, you can view those jobs' results by reopening the Source's config modal and clicking its Job Inspector tab.

Each content type that you enabled gets its own separate scheduled job.

You can also view these jobs (among scheduled jobs for other Collectors and Sources) in the Monitoring > System > Job Inspector > Currently Scheduled tab.