When you’re planning an application performance monitoring (APM) strategy, collecting metrics from storage services like Amazon S3 may not seem like a priority. After all, part of the point of object storage is that applications can read and write from storage buckets seamlessly, with minimal configuration and overhead. Unlike databases or file systems, storage buckets don’t require complex configurations that could lead to performance issues.
Nonetheless, there are a variety of things that could go wrong with Amazon S3 or other object storage services. Monitoring S3 metrics is the only way to ensure that problems with S3 don’t create problems for your larger software stack.
[Learn More: AWS Monitoring]
Keep reading for an overview of the top six S3 metrics to monitor.
1. AllRequests
The AllRequests S3 metric tracks the total number of requests made to each S3 bucket. Monitoring this metric allows you to establish a baseline for normal S3 activity.
Sudden increases or decreases in requests that don’t align with increases or decreases in application load could indicate a problem like communication issues between your application and storage buckets. A rapid spike in requests could also cause performance problems if it eats up more resources than your infrastructure can handle.
Amazon S3 also exposes more granular request metrics, like GetRequests and PutRequests, which can be helpful if you want to be able to drill down further into specific types of requests. But AllRequests provides a bird’s eye view of overall request activity.
2. TotalRequestLatency
This metric tracks how long S3 takes to serve data, measured in terms of the time between the first byte and the last byte of each request. This is one of the most important S3 metrics to track in order to ensure that slow response times by S3 buckets don’t lead to slower performance for your applications.
3. 4xxErrors
The 4xxErrors S3 metric reports the total number of 4xx error status codes that are made in response to client requests. Track this metric to establish a baseline for normal 4xx error rates, and then investigate anomalies, which could be a sign of a configuration problem, networking issue, or other performance disruption.
4. NumberOfObjects
The NumberOfObjects metric tracks the total number of objects stored in each S3 bucket.
While fluctuations in object totals aren’t likely to correspond with performance issues, you’ll probably want to know if there is a sudden increase or decrease in total objects. Such an event could be a sign of accidental data deletion, the failure of data rotation policies, or other issues within your applications.
5. BucketSizeBytes
Alongside NumberOfObjects, the BucketSizeBytes metric, which tracks the total size in bytes of the data within each storage bucket, is a useful indicator of potential issues with the way applications are interacting with S3.
In addition, because you can filter this metric based on different S3 storage classes, it provides a means of monitoring whether data lifecycle policies are being enforced (if you configured these policies). Data lifecycle policies can be used to cycle data into lower-cost storage classes as the data becomes older, which can save money.
6. ReplicationLatency
If you replicate S3 data by storing redundant copies of your buckets, you’ll want to track the ReplicationLatency metric. This metric measures delays in syncing replicated data between multiple buckets.
Delays in replication could lead to data availability issues in the event that one of your S3 locations fails. They could also be a sign of configuration problems that could affect the performance of your broader software stack. For these reasons, it’s worth tracking ReplicationLatency and investigating deviations from the norm.
How Do You Track S3 Metrics?
CloudWatch, Amazon’s native monitoring tool, can be used to collect S3 metrics and perform basic analytics on them.
In most cases, however, CloudWatch on its own is not sufficient for efficient S3 monitoring. CloudWatch typically works on an account-by-account basis, which makes it inefficient if you have multiple S3 accounts. It also lacks advanced analytics and visualization features. For these reasons, consider using a third-party tool like Sumo Logic to collect S3 logs via CloudWatch.
Conclusion: Getting the Most from S3 Metrics
This isn’t an exhaustive list of S3 metrics. For a full overview of available S3 metrics, check out Amazon’s documentation. To learn more about how Sumo Logic validates critical data for your S3, check out our integration. But if you’re just getting started and want to identify the most important metrics to track, start with the six core metrics identified above.
