Source and Collector Wise Cardinality

This document explains how to identify and analyze cardinality ingested per source and collector, helping you pinpoint high-cardinality sources and manage ingestion limits more effectively.

Previously, there was no direct visibility into the contribution of cardinality from individual sources. When limits were exceeded, sources were paused based on total cardinality, which could affect stable high-volume sources that were not responsible for the breach. This lack of source-level visibility made it difficult to accurately identify and address the root cause of high cardinality.

To address this, you can now run a query to identify total cardinality and cardinality at the source level and analyze which sources contribute the most within a given collector. This enables more targeted actions and helps avoid unnecessary impact on other sources.

Use the query below to determine the cardinality associated with sources for a specific collector.

_view=sumologic_volume _sourceCategory=cardinalityPerSourceCollector
| parse regex "bucket:\s+(?<bucket>[^,\s]+), totalCardinality: (?<totalCardinality>\d+)"
| parse regex "(?<collectorName>[^;\s]+);(?<sourceName>[^:]+?):(?<sourceCardinality>\d+)" multi nodrop
| num(sourceCardinality)
| num(totalcardinality )
| fields bucket, collectorName, sourceName, sourceCardinality, totalCardinality

Limitations

• If an organization has more than 100 sources, only the top 100 sources (by cardinality) are displayed for an audit log.
• Source and collector data is refreshed every hour, and a new audit log is generated to reflect the latest cardinality data.
• The sum of individual sources may not match the total cardinality count. This is because tracing-to-metrics conversions and internal metrics are excluded from individual source counts.