Additional Configurations Reference

Sumo Logic Distribution for OpenTelemetry stores the collector configuration in the following directory, based on your platform:

Linux	/etc/otelcol-sumo/conf.d
macOS	/etc/otelcol-sumo/conf.d
Windows	C:\ProgramData\Sumo Logic\OpenTelemetry Collector\config\conf.d

Configuration location and structure

The Sumo Logic OpenTelemetry Collector configuration is comprised of two parts. Based on the platform, the configuration by default is stored at the following location:

• Linux and macOS: /etc/otelcol-sumo/
• Windows: C:\ProgramData\Sumo Logic\OpenTelemetry Collector\config

note

If you manually installed the collector, your configuration may be in a different location.

Sumo Logic-defined configuration

This is required by the collector to communicate with the Sumo Logic service and transmit data. All Sumo Logic preconfigured components are stored in the sumologic.yaml file. This file is managed by the installation script and should never be changed manually.

Data source configuration

Here, you can define all the configuration that tells the collector what data to collect, how to process it (including adding metadata), and send it to Sumo Logic. All the user-defined configuration resides under the conf.d directory.

Any configuration for a source (for example, MySQL, Nginx, or application logs) should be stored in a separate file with a descriptive name under the conf.d directory. For example, a file named conf.d/mysql.yaml can contain the configuration to collect MySQL data (logs and metrics), including the receivers, processors, and pipeline that together tell the collector how to collect and send the MySQL data.

tip

It is recommended to maintain the configuration of all reusable components in conf.d/common.yaml. For example, a file named conf.d/mysql.yaml can contain the MySQL receiver along with any processors that are intended to modify the collected data before sending it to Sumo Logic.

Our App Catalog provides a simple UI form to create these configuration files for you.

Default collector name

Each collector name must be unique. By default, the collector uses the hostname of the machine where it's installed as its name. If you install a collector with the same hostname as an existing collector, the system automatically appends a 13-digit Unix timestamp to the new collector's name.

Forcing a collector's name with clobber

Set the clobber flag to true if you want to delete the existing collector when a new collector is installed on a machine with the same hostname. This is useful when a VM is deleted and recreated with the same hostname. To enable this property, create a file named sumo-ot-clobber.yaml in otelcol-sumo/conf.d, add the following configuration, and restart your collector.

extensions:
  sumologic:
    clobber: true

info

Setting the clobber flag to true deletes ("clobbers") any existing collector with the same collector name or hostname, so make sure that's what you want to do. This only works before the new collector has been registered (activated) with Sumo Logic.

Clobber rate limiting

Use clobber: true only when the original collector with that name is no longer running, for example after you delete and recreate a VM. If two running collectors share a name and both set clobber: true, each one deletes the other when it registers. The two collectors then keep deregistering and reinstalling each other in a loop.

To prevent this loop, Sumo Logic rate limits OpenTelemetry (OT) Collector registrations that use clobber: true. The limit applies to each collector name in your organization. After about 50 registration attempts with the same collector name within an hour, Sumo Logic rejects further attempts with an HTTP 429 response (ot-collector-registration:clobber_rate_limit_exceeded). Registrations that do not set clobber: true are not affected.

note

This rate limit applies to OT Collector only. It does not apply to Installed Collector registrations.

The registration fails with the following message:

Too many collector registrations with clobber=true for collector name '<name>'. This usually indicates multiple machines are attempting to register with the same collector name simultaneously. Use unique collector names per machine or disable clobber.

To avoid or resolve a rejected registration:

1. Use a unique collector_name for each machine. Set the collector_name property under extensions.sumologic.
2. Use clobber: true only to replace a collector that is no longer running. If the same name is in use by a running collector, remove clobber: true.
3. If the rate limit was reached, stop all registration attempts with that name and wait about an hour before you try again. Each attempt, including a rejected one, resets the one-hour timer.
4. Restart the collector.

For more information, see Registration is rejected with an HTTP 429 error when the clobber flag is enabled.

Custom configuration

Use custom configuration to collect your logs, metrics, and traces in Sumo Logic. Learn more about configuration.

There are some processors provided in sumologic.yaml that are intended to be used in every pipeline.

• Memory limiter processor. It is used to prevent out-of-memory situations on the collector. It should be always first on the processor's list. For more information, refer to the OpenTelemetry documentation.
• ResourceDetection/System processor. It can be used to detect resource information from the host in a format that conforms to the OpenTelemetry resource semantic conventions, and append or override the resource value in telemetry data with this information. You can also tag labels like host.name, host.id, os.type.

We also expect the Sumo Logic exporter to be included in the exporters section.

See the following example:

service:
  pipelines:
    metrics/default:
      receivers:
        - otlp
      processors:
        - memory_limiter
      exporters:
        - sumologic

Batch exported data

Exporter batching converts processed data into batches larger than a specified size or time interval. This compresses the data more effectively and reduces the number of requests the exporters send.

We recommend using this configuration in every pipeline. Define it under individual exporters.

In addition to setting a lower batch size, you can also set a maximum batch size.

We recommend setting that limit to avoid sudden increases in request sizes if more data is temporarily received. Set it to 2 * min_size.

Overall, we recommend the following default configuration for this processor.

exporters:
  sumologic:
    metric_format: prometheus
    sending_queue:
      queue_size: 1024000
      batch:
        flush_timeout: 1s
        min_size: 1024
        max_size: 2048 ## = 2 * 1024

note

The Batch processor from upstream is planned for deprecation in favor of exporter-level batching. Ref

note

If you are utilizing the Sumo Logic exporter to send data in a format other than OTLP, it is possible to explicitly restrict the size of the requests in bytes by utilizing the configuration option max_request_body_size.

Learn more about these processors:

• memorylimiterprocessor
• filterprocessor

Learn more about these exporters

• exporterbatching
• sumologicexporter

Data tagging recommendations

We recommend reading the Metadata Naming Conventions document before continuing to become more familiar with the terms used below. The following terms are important for data tagging:

• Source Category (_sourceCategory)
• Source Host (_sourceHost)
• Source Name (_sourceName)

The above attributes are taken from the Resource attributes. If they are not set, they are taken from the collector configuration.

See the following example which shows how to set them using the Resource attributes processor:

processors:
  resource/static fields:
    attributes:
    - key: _sourceName
      value: my source name
      ## upsert will override existing _sourceName field
      action: upsert

If you would like to add your own custom metadata fields to your logs, you can use the same attributes processor and use action: insert:

processors:
  resource/custom_fields:
    attributes:
      - key: collection_method
        value: otel
        action: insert

note

If you are adding custom attributes to your log metadata, then you need to ensure that these fields are also added in Sumo Logic under the Logs > Fields section.

For more advanced processing capabilities in your data pipeline, you can use the Transform processor. This processor enables you to perform more complex transformations on your data. For example, you can concatenate multiple Resource attributes into the _sourceCategory field, with a / character separating each attribute.

processors:
  transform/custom fields:
    log_statements:
    - context: resource
      statements:
      - set(attributes["_sourceCategory"], Concat([attributes["service.name"], attributes["http.method"]], "/"))

You can create custom tags when processing data, not just the special tags listed above. All resource attributes are treated as tags, giving you flexibility when categorizing and processing data.

You can set default tags in collector configuration using the collector_fields option:

extensions:
  sumologic:
    collector_fields:
      cluster: staging

Tags are useful for building queries and help you correlate different signals (metrics, logs, and traces) with each other.

note

Refer to the Fields documentation for more information on fields and how to use them.