Unstructured logs

What are unstructured logs?

Unstructured logs are machine-generated records of events, activities, or messages that don’t follow a predefined format or structure. Unlike structured logs, which follow a consistent format like JSON or CSV, an unstructured log is unpredictable and inconsistent, making it harder to parse and analyze without specialized tools. 

Think of an unstructured log as notes scribbled on a napkin: messy but often filled with useful, contextual information.

Why use unstructured logs?

While harder to analyze, unstructured log data contains valuable insights for observability and root cause analysis. They provide critical insights into system issues that structured data may overlook, helping you know about critical issues before it’s too late. If you need help troubleshooting, conducting security investigations, and performance monitoring, unstructured logs are your go-to pal. 

One drawback is that since they lack structure, extracting insights from your unstructured logs without the help of a log analysis tool or observability platform can be extremely difficult, as it requires sophisticated log parsing, normalization, and indexing. 

But when you properly ingest and aggregate your unstructured logs, you can begin to detect important patterns from your unstructured logs.

Examples of unstructured logs

Here are some data sources that you’ll find in unstructured log files: 

• AI/ML output logs, such as LLM prompt/response logs (AI)
• Docker stdout log (containers)
• Email server logs
• Web server logs
• IOT logs
• System application logs with free-text error messages
• Network traffic logs with detailed descriptions
• Event logs from custom-built software

Structured log vs. unstructured log example

Unstructured container log:

Structured JSON log:

How to use unstructured logs

Unstructured logs can feel overwhelming, but with the right approach, they offer powerful insights. Here’s how to make the most of them:

• Ingest at scale: Use Sumo Logic’s log management platform to collect logs from across your systems and applications.
• Normalize the data: Apply parsing rules or machine learning techniques to add structure to the log content.
• Correlate events: Link unstructured logs with metrics and traces to gain deeper context into system behavior and performance.
• Set alerts and dashboards: Create queries to surface anomalies and visualize trends for faster decision-making.
• Automate investigations: Integrate with security tools to accelerate threat detection and root cause analysis.

Unstructured logs become especially valuable when combined with other telemetry data, turning noisy, plain text messages into actionable intelligence.

Gain actionable insights from unstructured logs using Sumo Logic

Sumo Logic’s cloud-native log analytics platform automatically indexes, ingests, and analyzes both structured and unstructured logs, helping teams: 

• Extract content from messy or free-form logs
• Create dashboards and alerts using unified data
• Speed up incident investigations and threat detection
• Reduce MTTR across applications and infrastructure

Understand the “why” behind system data by analyzing your unstructured logs. Leverage your unstructured and structured data for deeper observability.