OpenAI

The Sumo Logic app for OpenAI provides visibility into your OpenAI organization's cost, usage, security, and audit activity. It monitors API spend by model and project, tracks authentication events and failed login patterns, analyzes identity and access management changes, and provides geographic and threat intelligence insights. Use this app to optimize costs, detect anomalous behavior, and maintain governance across your OpenAI environment.

Log types

This app uses the OpenAI Source to collect data from the OpenAI Administration API. The source collects:

• Audit Logs. A chronological record of user actions and configuration changes within the organization, including authentication events, API key operations, project changes, and role modifications.
• Organization Usage Costs. Aggregated API spend broken down by model, project, and API key.

Sample log messages

Audit Log

{
  "id": "audit_log-yyy__20240101",
  "type": "api_key.updated",
  "effective_at": 1720804190,
  "actor": {
    "type": "session",
    "session": {
      "user": {
        "id": "user-xxx",
        "email": "user@example.com"
      },
      "ip_address": "127.0.0.1",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36",
      "ja3": "a497151ce4338a12c4418c44d375173e",
      "ja4": "q13d0313h3_55b375c5d22e_c7319ce65786",
      "ip_address_details": {
        "country": "US",
        "city": "San Francisco",
        "region": "California",
        "region_code": "CA",
        "asn": "1234",
        "latitude": "37.77490",
        "longitude": "-122.41940"
      }
    }
  },
  "api_key.updated": {
    "id": "key_xxxx",
    "data": {
      "scopes": ["resource_2.operation_2"]
    }
  }
}

Cost Log

{
  "object": "organization.costs.result",
  "amount": { "value": 100, "currency": "usd" },
  "line_item": "images",
  "project_id": "proj_001",
  "api_key_id": "key_001",
  "quantity": 1,
  "start_time": 1791284283,
  "end_time": 1791284283
}

Sample queries

Successful Logins Over Time

_sourceCategory={{Logsdatasource}} organization.audit_log login.succeeded
| json "id", "object", "type", "actor.type", "actor.session.user.email", "actor.api_key.user.email" as id, object_type, event_type, actor_type, session_user_email, api_key_user_email nodrop

| where object_type = "organization.audit_log" and event_type = "login.succeeded"
| if(isBlank(session_user_email),api_key_user_email,session_user_email) as user_email

// global filters
| where if ("{{user_email}}" = "*", true, user_email matches "{{user_email}}")
| where if("{{actor_type}}" = "*", true, actor_type matches "{{actor_type}}")

// Panel specific
| count by id, _messagetime
| timeslice 1d
| count by _timeslice
| fillmissing timeslice(1d)

Total Spend

_sourceCategory={{Logsdatasource}} organization.costs.result amount line_item
| json "object", "amount.value", "line_item", "project_id", "api_key_id" as object_type, cost_value, line_item, project_id, api_key_id nodrop

| where object_type = "organization.costs.result"
| where !isBlank(cost_value)

// global filters
| where if ("{{project_id}}" = "*", true, project_id matches "{{project_id}}")
| where if ("{{line_item}}" = "*", true, line_item matches "{{line_item}}")
| where if ("{{api_key_id}}" = "*", true, api_key_id matches "{{api_key_id}}")
| todouble(cost_value) as cost_value

| count by cost_value, _messagetime
| sum(cost_value) as total_spend

Collection configuration and app installation

Depending on the set up collection method, you can configure and install the app in three ways:

• Create a new collector and install the app. Create a new Sumo Logic Cloud-to-Cloud (C2C) source under a new Sumo Logic Collector and later install the app, or
• Use an existing collector and install the app. Create a new Sumo Logic Cloud-to-Cloud (C2C) source under an existing Sumo Logic Collector and later install the app, or
• Use existing source and install the app. Use your existing configured Sumo Logic Cloud-to-Cloud (C2C) source and install the app.

tip

Use the OpenAI Source to create the source and use the same source category while installing the app.

Create a new collector and install the app

To set up collection and install the app, do the following:

note

Next-Gen App: To install or update the app, you must be an account administrator or a user with Manage Apps, Manage Monitors, Manage Fields, Manage Metric Rules, and Manage Collectors capabilities depending upon the different content types part of the app.

1. Select App Catalog.
2. In the 🔎 Search Apps field, run a search for your desired app, then select it.
3. Click Install App.
   note

   Sometimes this button says Add Integration.

4. In the Set Up Collection section of your respective app, select Create a new Collector.
   1. Collector Name. Enter a Name to display the Source in the Sumo Logic web application. The description is optional.
   2. Timezone. Set the default time zone when it is not extracted from the log timestamp. Time zone settings on Sources override a Collector time zone setting.
   3. (Optional) Metadata. Click the +Add Metadata link to add a custom log Metadata Fields. Define the fields you want to associate, each metadata field needs a name (key) and value.
      • A green circle with a checkmark is shown when the field exists and is enabled in the Fields table schema.
      • An orange triangle with an exclamation point is shown when the field doesn't exist, or is disabled in the Fields table schema. In this case, you'll see an option to automatically add or enable the nonexistent fields to the Fields table schema. If a field is sent to Sumo Logic but isn’t present or enabled in the schema, it’s ignored and marked as Dropped.
   4. Click Next.
5. Configure the source as specified in the Info box above, ensuring all required fields are included.
6. In the Configure section of your respective app, complete the following fields.
   1. Field Name. If you already have collectors and sources set up, select the configured metadata field name (eg _sourcecategory) or specify other custom metadata (eg: _collector) along with its metadata Field Value.
7. Click Next. You will be redirected to the Preview & Done section.

Post-installation

Once your app is installed, it will appear in your Installed Apps folder, and dashboard panels will start to fill automatically.

Each panel slowly fills with data matching the time range query received since the panel was created. Results will not immediately be available but will be updated with full graphs and charts over time.

Use an existing collector and install the app

To set up the source in the existing collector and install the app, do the following:

note

Next-Gen App: To install or update the app, you must be an account administrator or a user with Manage Apps, Manage Monitors, Manage Fields, Manage Metric Rules, and Manage Collectors capabilities depending upon the different content types part of the app.

1. Select App Catalog.
2. In the 🔎 Search Apps field, run a search for your desired app, then select it.
3. Click Install App.
   note

   Sometimes this button says Add Integration.

4. In the Set Up Collection section of your respective app, select Use an existing Collector.
5. From the Select Collector dropdown, select the collector that you want to set up your source with and click Next.
6. Configure the source as specified in the Info box above, ensuring all required fields are included.
7. In the Configure section of your respective app, complete the following fields.
   1. Field Name. If you already have collectors and sources set up, select the configured metadata field name (eg _sourcecategory) or specify other custom metadata (eg: _collector) along with its metadata Field Value.
8. Click Next. You will be redirected to the Preview & Done section.

Post-installation

Once your app is installed, it will appear in your Installed Apps folder, and dashboard panels will start to fill automatically.

Each panel slowly fills with data matching the time range query received since the panel was created. Results will not immediately be available but will be updated with full graphs and charts over time.

Use an existing source and install the app

To skip collection and only install the app, do the following:

note

Next-Gen App: To install or update the app, you must be an account administrator or a user with Manage Apps, Manage Monitors, Manage Fields, Manage Metric Rules, and Manage Collectors capabilities depending upon the different content types part of the app.

1. Select App Catalog.
2. In the 🔎 Search Apps field, run a search for your desired app, then select it.
3. Click Install App.
   note

   Sometimes this button says Add Integration.

4. In the Set Up Collection section of your respective app, select Skip this step and use existing source and click Next.
5. In the Configure section of your respective app, complete the following fields.
   1. Field Name. If you already have collectors and sources set up, select the configured metadata field name (eg _sourcecategory) or specify other custom metadata (eg: _collector) along with its metadata Field Value.
6. Click Next. You will be redirected to the Preview & Done section.

Post-installation

Once your app is installed, it will appear in your Installed Apps folder, and dashboard panels will start to fill automatically.

Each panel slowly fills with data matching the time range query received since the panel was created. Results will not immediately be available but will be updated with full graphs and charts over time.

Viewing OpenAI dashboards

All dashboards have a set of filters that you can apply to the entire dashboard. Use these filters to drill down and examine the data to a granular level.

• You can change the time range for a dashboard or panel by selecting a predefined interval from a drop-down list, choosing a recently used time range, or specifying custom dates and times. Learn more.
• You can use template variables to drill down and examine the data on a granular level. For more information, see Filtering Dashboards with Template Variables.
• Most Next-Gen apps allow you to provide the scope at the installation time and are comprised of a key (_sourceCategory by default) and a default value for this key. Based on your input, the app dashboards will be parameterized with a dashboard variable, allowing you to change the dataset queried by all panels. This eliminates the need to create multiple copies of the same dashboard with different queries.

Audit Overview

The OpenAI - Audit Overview dashboard provides a unified view of audit activity across the OpenAI organization. It visualizes event volume, top actors, event categories, and project activity while highlighting geographic threats and access from embargoed locations. Use this dashboard to identify anomalous activity and monitor overall audit health.

Cost Monitoring

The OpenAI - Cost Monitoring dashboard provides an overview of total spend, cost trends, and budget health across the organization. It tracks cost by model, project, and API key, highlights unusual spending patterns, and shows cumulative spend over time. Use this dashboard to identify cost anomalies and major cost drivers.

Failed Login Monitoring

The OpenAI - Failed Login Monitoring dashboard focuses on failed authentication events to help identify brute-force attempts, suspicious IP addresses, and credential-related issues. It provides insights into failure reasons, error codes, geographic activity, multi-device login patterns, and repeated authentication failures.

Identity and Access Management

The OpenAI - Identity and Access Management dashboard monitors user lifecycle changes, invitations, group management, role assignments, and service account activity across the organization. It tracks IAM activity over time, highlights top actors, and provides detailed audit information for user, role, and group operations.

Security and Infrastructure Configuration

The OpenAI - Security and Infrastructure Configuration dashboard monitors security and infrastructure configuration changes, including IP allowlists, rate limits, SCIM configuration, certificates, tunnels, workload identity providers, checkpoint permissions, and organization-level settings. It categorizes security events, highlights the top actors, and provides detailed audit information for each configuration area.

Successful Login Monitoring

The OpenAI - Successful Login Monitoring dashboard provides visibility into successful authentication activity across the OpenAI organization. It shows login trends, geographic distribution, multi-IP login patterns, and access from embargoed locations to help identify unusual activity.

User Agent Analysis

The OpenAI - User Agent Analysis dashboard analyzes the clients and platforms accessing the OpenAI environment. It provides visibility into browser, operating system, platform, and automated versus human activity, while highlighting user agents associated with failed or potentially suspicious activity.

Create monitors for the OpenAI app

From your App Catalog:

1. From the Sumo Logic navigation, select App Catalog.
2. In the Search Apps field, search for and then select your app.
3. Make sure the app is installed.
4. Navigate to What's Included tab and scroll down to the Monitors section.
5. Click Create next to the pre-configured monitors. In the create monitors window, adjust the trigger conditions and notifications settings based on your requirements.
6. Scroll down to Monitor Details.
7. Under Location click on New Folder.
   note

   By default, monitor will be saved in the root folder. So to make the maintenance easier, create a new folder in the location of your choice.

8. Enter Folder Name. Folder Description is optional.
   tip

   Using app version in the folder name will be helpful to determine the versioning for future updates.

9. Click Create. Once the folder is created, click on Save.

Upgrading/Downgrading the OpenAI app (Optional)

To update the app, do the following:

note

Next-Gen App: To install or update the app, you must be an account administrator or a user with Manage Apps, Manage Monitors, Manage Fields, Manage Metric Rules, and Manage Collectors capabilities depending upon the different content types part of the app.

1. Select App Catalog.
2. In the Search Apps field, search for and then select your app.
   Optionally, you can identify apps that can be upgraded in the Upgrade available section.
3. To upgrade the app, select Upgrade from the Manage dropdown.
   1. If the upgrade does not have any configuration or property changes, you will be redirected to the Preview & Done section.
   2. If the upgrade has any configuration or property changes, you will be redirected to the Setup Data page.
   3. In the Configure section of your respective app, complete the following fields.
      1. Field Name. If you already have collectors and sources set up, select the configured metadata field name (eg _sourcecategory) or specify other custom metadata (eg: _collector) along with its metadata Field Value.
   4. Click Next. You will be redirected to the Preview & Done section.

Post-update

Your upgraded app will be installed in the Installed Apps folder and dashboard panels will start to fill automatically.

note

See our Release Notes changelog for new updates in the app.

To revert the app to a previous version, do the following:

1. Select App Catalog.
2. In the Search Apps field, search for and then select your app.
3. To version down the app, select Revert to < previous version of your app > from the Manage dropdown.

Uninstalling the OpenAI app (Optional)

To uninstall the app, do the following:

1. Select App Catalog.
2. In the 🔎 Search Apps field, run a search for your desired app, then select it.
3. Click Uninstall.