Skip to main content
Version: 3.2

Office 365 Activity

Cribl LogStream supports receiving data from the Office 365 Management Activity API. This facilitates analyzing actions and events on Azure Active Directory, Exchange, and SharePoint, along with global auditing and Data Loss Prevention data.

Type: Pull | TLS Support: YES | Event Breaker Support: YES

TLS is enabled via the HTTPS protocol on this Source's underlying REST API.

Azure AD Permissions

In Azure Active Directory, the application representing your LogStream instance must be granted the following permissions to pull data. Each permission's Type must be Application – Delegated is not sufficient:

  • ActivityFeed Read – Required for all Content Types except DLP.All.
  • ActivityFeed.ReadDlp – Required for the DLP.All Content Type.
Registered application permissions

Office 365 Subscriptions

LogStream does not support starting/stopping Office 365 subscriptions. You can start subscriptions either via another Office 365 API client, or simply via curl commands. We document the curl command method below in Starting Content Subscriptions.

Configuring LogStream to Receive Data from the Activity API

From the top nav of a LogStream instance or Group, select Data > Sources. From the resulting page's tiles or the Sources left nav, select [Pull >] Office 365 > Activity*. Next, click + Add New to open a New Source modal that provides the following options and fields.

This Source cannot currently be selected or enabled in the QuickConnect UI.

General Settings

Input ID: Enter a unique name to identify this Office 365 Activity definition.

Tenant ID: Enter the Office 365 Azure tenant ID.

App ID: Enter the Office 365 Azure application ID.

Client secret: Enter the Office 365 Azure client secret.

Subscription Plan: Select the Office 365 subscription plan for your organization. This is typically Enterprise and GCC Government Plan.

Content Types

Here, you can configure polling independently for the following types of audit data from the Office 365 Management Activity API:

  • Active Directory
  • Exchange
  • SharePoint
  • General: All workloads not included in the above content types
  • DLP.All: Data Loss Prevention events only, for all workloads

For each of these content types, this section provides the following controls:

Enabled: Toggle this to Yes for each service that you want to poll.

Interval: Optionally, override the default polling interval. See About Polling Intervals below.

Log level: Set the verbosity level to one of debug, info (the default), warn, or error.

About Polling Intervals

To poll the Office 365 Management Activity API, LogStream uses the Interval field's value to establish the search date range and the cron schedule (e.g.: */${interval} * * * *).

Therefore, intervals set in minutes must divide evenly into 60 minutes to create a predictable schedule. Dividing 60 by intervals like 1, 2, 3, 4, 5, 6, 10, 12, 15, 20, or 60 itself yields an integer, so you can enter any of these values.

LogStream will reject intervals like 23, 42, or 45, or 75 – which would yield non-integer results, meaning unpredictable schedules.

Processing Settings

Fields (Metadata)

In this section, you can add fields/metadata to each event, using Eval-like functionality.

Name: Field name.

Value: JavaScript expression to compute field's value (can be a constant).

Pre-Processing

In this section's Pipeline drop-down list, you can select a single existing Pipeline to process data from this input before the data is sent through the Routes.

Advanced Settings

Keep Alive Time (seconds): How often Workers should check in with the scheduler to keep their job subscription alive. Defaults to 60.

Worker timeout (periods): The number of Keep Alive Time periods before an inactive Worker will have its job subscription revoked. Defaults to 3.

Timeout (secs): The maximum time period for an HTTP request to complete before LogStream treats it as timed out. Defaults to 300 (i.e., 5 minutes). Enter 0 to disable timeout metering.

Environment: If you're using GitOps, optionally use this field to specify a single Git branch on which to enable this configuration. If empty, the config will be enabled everywhere.

Internal Fields

Cribl LogStream uses a set of internal fields to assist in handling of data. These "meta" fields are not part of an event, but they are accessible, and Functions can use them to make processing decisions.

Fields for this Source:

  • __final
  • __inputId
  • __isBroken
  • __source

Starting Content Subscriptions

Content subscriptions (a different concept from the O365 subscription plans) are required in order for LogStream to be able to begin retrieving O365 data. There is a separate subscription required for each Content Type. If you are using an existing Azure-registered application ID that already has subscriptions started, then you can ignore this section. But if you are:

  • Using a newly registered application ID, and therefore never had any subscriptions started, or
  • Reusing an application ID that had subscriptions started, but are currently stopped

...then you will need to use this procedure to manually start the necessary subscriptions.

This is a two-step process. The first command obtains an auth token, which is used in the second command to actually start the subscription. To execute these commands, you'll need the same information (i.e. client secret, application ID, and tenant ID) that you already require to configure this Source in LogStream's GUI. Replace those three variables as appropriate in the commands below.

  1. curl -d "client_secret=<client secret>&resource=https://manage.office.com&client_id=<app id>&grant_type=client_credentials" -X POST https://login.windows.net/<tenant id>/oauth2/token

  2. curl -d "" -H "Authorization: Bearer <access token>" -X POST https://manage.office.com/api/v1.0/<tenant id>/activity/feed/subscriptions/start?contentType=<content_type_name>

Here is an example of each command executed and expected output:

Example Command #1

$ curl -d "client_secret=abcdefghijklmnopqrstuvwxyz12345678&resource=https://manage.office.com&client_id=00000000-ffff-ffff-ffff-aaaaaaaaaaaa&grant_type=client_credentials" -X POST https://login.windows.net/12345678-aaaa-4233-cccc-160c6c30154a/oauth2/token

Output:

{"token_type":"Bearer","expires_in":"3599","ext_expires_in":"3599","expires_on":"1622089429","not_before":"1622085529","resource":"https://manage.office.com","access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6Im5PbzNaRHJPRFhFSzFqS1doWHNsSFJfS1hFZyIsImtpZCI6Im5PbzNaRHJPRFhFSzFqS1doWHNsSFJfS1hFZyJ9.eyJhdWQiOiJodHRwczovL21hbmFnZS5vZmZpY2UuY29tIiwiaXNzIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvZmFiZDI0NGYtZGZhOS00MjMzLTg2ZGEtMTYwYzZjMzAxNTRhLyIsImlhdCI6MTYyMjA4NTUyOSwibmJmIjoxNjIyMDg1NTI5LCJleHAiOjE2MjIwODk0MjksImFpbyI6IkUyWmdZSmdTSGlWcjR0VW45bDhnU0ZwNzMvTlpBQT09IiwiYXBwaWQiOiIwMGZhYzVmYS1jNDRiLTRmYmItYWIxOC1lNDdiZjJkNjYzNTkiLCJhcHBpZGFjciI6IjEiLCJpZHAiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9mYWJkMjQ0Zi1kZmE5LTQyMzMtODZkYS0xNjBjNmMzMDE1NGEvIiwib2lkIjoiZTZlMTkzOGMtMmIzMi00MDY2LTgyMmUtNDhlNTQ5NGEzMTljIiwicmgiOiIwLkFWb0FUeVM5LXFuZk0wS0cyaFlNYkRBVlN2ckYtZ0JMeEx0UHF4amtlX0xXWTFsYUFBQS4iLCJyb2xlcyI6WyJBY3Rpdml0eUZlZWQuUmVhZERscCIsIlNlcnZpY2VIZWFsdGguUmVhZCIsIkFjdGl2aXR5RmVlZC5SZWFkIl0sInN1YiI6ImU2ZTE5MzhjLTJiMzItNDA2Ni04MjJlLTQ4ZTU0OTRhMzE5YyIsInRpZCI6ImZhYmQyNDRmLWRmYTktNDIzMy04NmRhLTE2MGM2YzMwMTU0YSIsInV0aSI6IkdIUUNweFFDZ2tLNWlZMnV3OHdZQUEiLCJ2ZXIiOiIxLjAifQ.RyXNthPtvVBsd6UJdF1e4F6qhYw1fGC0GAcQjK54zzZOM5C6n57QviK-w8ea-gbQQv_e8mGuPWd7_-NTPcjKQwwt1hElpVjnudhyHL9HPRMD__scKAxmorvKpURk_42FqxWEJCuD_NEzQSoCJibyg8RmbNCrbe4Qq3-6Pd_3LEqXUrSX30YO0yg82-yjbJhipa_aP0-SRYskDbYwQN1hciGddnISHvINc-ay5rxlczPgylPsSqMiTqLeSf438i3g9riZltK7g2WonZFStF7gewTlPWLqlLGi2FY7-cEwjWGeDjGH_UQ3j_gkHNOVR9t7JtjqEwS4ObA-ky32GMRDvw"}

Example Command #2

$ curl -d "" -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6Im5PbzNaRHJPRFhFSzFqS1doWHNsSFJfS1hFZyIsImtpZCI6Im5PbzNaRHJPRFhFSzFqS1doWHNsSFJfS1hFZyJ9.eyJhdWQiOiJodHRwczovL21hbmFnZS5vZmZpY2UuY29tIiwiaXNzIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvZmFiZDI0NGYtZGZhOS00MjMzLTg2ZGEtMTYwYzZjMzAxNTRhLyIsImlhdCI6MTYyMjA4NTUyOSwibmJmIjoxNjIyMDg1NTI5LCJleHAiOjE2MjIwODk0MjksImFpbyI6IkUyWmdZSmdTSGlWcjR0VW45bDhnU0ZwNzMvTlpBQT09IiwiYXBwaWQiOiIwMGZhYzVmYS1jNDRiLTRmYmItYWIxOC1lNDdiZjJkNjYzNTkiLCJhcHBpZGFjciI6IjEiLCJpZHAiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9mYWJkMjQ0Zi1kZmE5LTQyMzMtODZkYS0xNjBjNmMzMDE1NGEvIiwib2lkIjoiZTZlMTkzOGMtMmIzMi00MDY2LTgyMmUtNDhlNTQ5NGEzMTljIiwicmgiOiIwLkFWb0FUeVM5LXFuZk0wS0cyaFlNYkRBVlN2ckYtZ0JMeEx0UHF4amtlX0xXWTFsYUFBQS4iLCJyb2xlcyI6WyJBY3Rpdml0eUZlZWQuUmVhZERscCIsIlNlcnZpY2VIZWFsdGguUmVhZCIsIkFjdGl2aXR5RmVlZC5SZWFkIl0sInN1YiI6ImU2ZTE5MzhjLTJiMzItNDA2Ni04MjJlLTQ4ZTU0OTRhMzE5YyIsInRpZCI6ImZhYmQyNDRmLWRmYTktNDIzMy04NmRhLTE2MGM2YzMwMTU0YSIsInV0aSI6IkdIUUNweFFDZ2tLNWlZMnV3OHdZQUEiLCJ2ZXIiOiIxLjAifQ.RyXNthPtvVBsd6UJdF1e4F6qhYw1fGC0GAcQjK54zzZOM5C6n57QviK-w8ea-gbQQv_e8mGuPWd7_-NTPcjKQwwt1hElpVjnudhyHL9HPRMD__scKAxmorvKpURk_42FqxWEJCuD_NEzQSoCJibyg8RmbNCrbe4Qq3-6Pd_3LEqXUrSX30YO0yg82-yjbJhipa_aP0-SRYskDbYwQN1hciGddnISHvINc-ay5rxlczPgylPsSqMiTqLeSf438i3g9riZltK7g2WonZFStF7gewTlPWLqlLGi2FY7-cEwjWGeDjGH_UQ3j_gkHNOVR9t7JtjqEwS4ObA-ky32GMRDvw" -X POST https://manage.office.com/api/v1.0/12345678-aaaa-4233-cccc-160c6c30154a/activity/feed/subscriptions/start?contentType=Audit.AzureActiveDirectory

Output:

{"contentType":"Audit.AzureActiveDirectory","status":"enabled","webhook":null}

Note there is no output when executing this second command with a stop operation.

You'll need to execute the second command for each Content Type whose logs you wish to collect. Use the exact strings below to specify Content Types in that command:

  • Audit.AzureActiveDirectory
  • Audit.Exchange
  • Audit.SharePoint
  • Audit.General
  • DLP.All

How LogStream Pulls Data

The Office 365 Activity Source retrieves data using LogStream scheduled Collection jobs, which include Discover and Collection phases. The Discover phase task returns the URL of the content to collect.

In the Source's General Settings > Content Types > Interval column, you configure the polling schedule for each Content Type independently.

The job scheduler spreads the Collection tasks across all available Workers. The collected content is paginated, so the collection phase might include multiple calls to fetch data.

Viewing Scheduled Jobs

This Source executes LogStream's scheduled collection jobs. Once you've configured and saved the Source, you can view those jobs' results by reopening the Source's config modal and clicking its Job Inspector tab.

Each content type that you enabled gets its own separate scheduled job.

You can also view these jobs (among scheduled jobs for other Collectors and Sources) in the Monitoring > System > Job Inspector > Currently Scheduled tab.