Pricing Login
Pricing
Support
Demo
Interactive demos

Click through interactive platform demos now.

Live demo, real expert

Schedule a platform demo with a Sumo Logic expert.

Start free trial
Back to blog results

August 22, 2024 By Manny Lopez

What you should know about Datadog Flex Logs

What you should know about Datadog Flex Logs

Late last year, Datadog announced something called Flex Logs, a “more affordable” warm storage tier for log data. Designed for high-volume datasets that are infrequently queried and don't require real-time analysis, the Flex Tier offers Datadog Log Management customers a third option for data storage.

It was basically Datadog’s way of saying, “Hey, we know storing all your log data with us costs an arm and a leg, so here’s a cheaper option.” It’s a warm storage tier designed for all those logs you collect but never really look at—kind of like that stack of New Yorker magazines you swear you’ll read someday. The Flex Tier offers Datadog Log Management customers a third option for data storage, Standard, Flex and Archive tiers.

Pricing

Description

Standard Tier

$0.10 per GB ingested + $2.55 per million log events per month

Hot storage is designed to support real-time use cases such as dashboarding, alerting, and troubleshooting.

Flex Tier

$0.10 per GB ingested + $0.05 per million log events per month + compute tier cost (see below)

Warm storage is designed for supporting historical data and less urgent investigations.

Archive Tier*

$0.10 per GB ingested

Warm storage is designed for supporting historical data and less urgent investigations.

*To analyze data stored in the Archive Tier, you need to "rehydrate" it into the Standard Tier and pay the applicable fees.

The trade-off: cost vs. performance

Datadog can apply a $0.05-per-million-event charge to Flex Logs because it separates compute and storage retention costs. Compute refers to the querying capacity needed to run queries for Flex Logs. This is used when querying logs in the Flex Logs tier. The available compute tiers, based on log volume and price, are listed below:

Size

Volume (events stored)

Month to month

Annual

On-demand

Extra Small (XS)

20 - 50 billion

$12,000

$10,000

$14,000

Small (S)

50 - 200 billion

$42,000

$35,000

$50,400

Medium (M)

200 - 500 billion

$90,000

$75,000

$10,800

Large (L)

500 billion - 1 trillion

$180,000

$150,000

$216,00

Flex Logs Storage

Prices per 1M stored

$0.06

$0.05

$0.08

Choosing Datadog’s Flex Tier for storage comes with a trade-off: you save on storage costs at the expense of search performance. While it’s cheaper to store data in the Flex Tier, running queries and retrieving results will take longer compared to the standard index. Essentially, if you want faster performance, you’ll need to sacrifice some of those cost savings.

Making this decision isn’t as straightforward as it seems. It requires a deep understanding of how your organization will use the data once it’s ingested—a question that’s often easier asked than answered.

Which storage tier should you use?

To truly appreciate the value of Datadog’s Flex Logs, it's crucial to categorize your logs into the appropriate storage tiers: Standard (hot), Flex (warm), and Archive (cold). But doing this effectively means you need to have a thorough grasp of the data you're feeding into Datadog's log management solution. Consider these key questions:

  • What’s your daily data ingestion volume, and what types of log files are you ingesting?

  • How valuable is the data? Is it redundant and noisy, or is it critical to your observability strategy?

  • How does your team use this data? Is it for troubleshooting or compliance purposes?

  • How often are these logs queried, and how urgently do users need results – yesterday, or whenever?

Unfortunately, Datadog doesn’t provide clear guidance on answering these questions. The responsibility of determining which logs to route to which tier falls squarely on the customer. And getting this wrong can be costly.

Picture this: It’s the middle of an outage, and the logs you desperately need are stored in the Flex Tier. Now your team has to play a game of “Wait for the Data” while they all hit the system at once and max out compute resources, potentially even dropping queries, all before they can even start fixing the problem. Fun, right?

The impracticality of storage tiers

In theory, storage tiers sound great as a way to save money and keep focused on the logs you need. But in practice, they’re not always as useful, with outsized drawbacks.

This is one reason why Sumo Logic has moved away from storage tiers in favor of a usage-based pricing model called Flex Pricing. This model lets you ingest as much data as you need without worrying about performance trade-offs or deciding which tier to use. It provides a scalable and efficient log analytics architecture that adapts to the demands of enterprise-wide and cloud-scale log ingestion, without forcing you to compromise on cost or performance. It’s like having your cake, eating it, and then getting another cake because you really can ingest everything.

Learn more about Sumo Logic pricing or sign up for your free trial.

Complete visibility for DevSecOps

Reduce downtime and move from reactive to proactive monitoring.

Sumo Logic cloud-native SaaS analytics

Build, run, and secure modern applications and cloud infrastructures.

Start free trial
Manny Lopez

Manny Lopez

Director, Competitive Intelligence

Manny Lopez has more than 20 years of experience in market research, focused primarily on competitive analysis. His diverse background includes, most recently, a 5+ year stint at Druva. He also has a broad background in market research that spans many years, including being a research analyst with IDC for over eight years, based in their Hong Kong and Beijing offices.

More posts by Manny Lopez.

People who read this also enjoyed