In the last post, we talked about how Fastly, a content delivery network, provides a global infrastructure footprint to enterprises, and enables them to move apps and websites closer to their end users. Using Fastly CDN, they can serve content and deploy updates quickly, optimize web performance, and improve overall user experience.
In this post, we will discuss how to collect, analyze, and monitor Fastly logs.
Fastly serves enormous amounts of traffic on behalf of their customers. Because of this, collecting, analyzing, and monitoring log data are crucial in making sure that web properties are optimally configured, secure, and performing at their best.
Fastly supports real-time log streaming of all data that’s coursed through Fastly. This helps their customers get better visibility into the performance of the CDN, as well as allows them to use syslog-compatible providers to monitor log data.
How to stream Fastly CDN logs
For storage and analysis, Fastly makes real-time log streaming available, mainly through syslog-compatible logging providers like Sumo Logic.
By using Sumo Logic’s Hosted Collector, setting up log collection for the Fastly app is straightforward and easy to do.
(A more detailed breakdown of the steps needed to collect CDN log data in Sumo Logic will be discussed in the next post of this blog series.)
Fastly also supports other methods of sending logs:
- Through a syslog endpoint available for users who choose to send log data to other syslog-based software
- Pushing log files to S3s
- FTP uploading
Fastly CDN log streaming features
- Transport Layer Security (TLS) support – Potentially sensitive information can be sent in a secure way by sending them over TLS. This feature allows Fastly users to protect their data as logging configuration information is encrypted in transit when sent over TLS.
- Encryption – Fastly encrypts files using OpenPGP for certain endpoints. For Amazon S3, they also support server-side encryption.
- Custom log formats – Users can change log formats as long as they provide variables that are compatible with the Apache Common log format.
- Log file locations – There are two ways users can change where log files are written–by changing the timestamp format and by configuring the specific path where the files will be written.
- Allowlisting – Users can toggle Fastly-only access to logging serves through their list of IP ranges.
How to Analyze Fastly CDN Logs
Analyzing Fastly CDN logs can be done out of the box on the Fastly web interface or, for enhanced actionable insights, through services like Sumo Logic.
When logs are treated as data that can and should be analyzed, users are able to get actionable insights that can be used to improve performance, troubleshoot connectivity issues, identify configurations that need to be tuned, pinpoint the causes of disruptions, and overall maintain a high level of customer experience.
Analyzing log data on Fastly’s web interface
Fastly allows its users to access log data in formats that are easy to analyze, although data available out of the bo can be limited.
Fastly provides the following stats per log event: Client IP, Time Stamp, Request Type, Request URL, and HTTP Status Code. Using their VCL extensions, users can include other information like:
- Time and Date Variables
Data that provide flexible ways to view and handle date and time stats like hit times and miss times.
- Size Related Variables
Breakdown of header and body size per log event, and other high-level request and response size variables - GeoIP Related Variables
Points of presence (POP) location where requests come through as well as the continent, country, and city from client IPs
Fastly users with appropriate access can view Real-time Stats on Fastly’s web interface in the All Services page to view the following stats:
- Hits – number of times requested data is found in the cache
- Hit Time – processing time spent on hits
- Misses – number of times requested data is not found in the cache
- Hit Ratio – how much content currently in the cache is being accessed, as expressed as a percentage against all cacheable content
- Miss Time – processing time spent on misses
- Requests – number of requests received by Fastly on behalf of your site
- Errors – total number of error requests
From the All Services page, users can also access and view graphs that summarize cache activity as well as Historical Stats which show historical numbers from your site’s statistical data.
Analyzing Fastly logs using Sumo Logic
Fastly users also have the option of using a provider like Sumo Logic to get access to enhanced log data and analytics by having log streaming or log collection in place.
Streaming Fastly CDN logs to Sumo Logic and using the Sumo Logic App for Fastly give users access to pre-configured search queries, panels, and dashboards that provide a deeper understanding of edge data.
Using the Sumo Logic App for Fastly, users are able to:
- Correlate Fastly data with origin data and other internal data sets to gauge CDN performance and service level
- Use additional data fields to provide context to log events
- Monitor bot traffic and stay alert on suspicious activities
- Pinpoint anomalies in request and user patterns
- Access quick analysis for advanced troubleshooting and diagnostics.
(A more in-depth discussion of Sumo Logic dashboards will be included in the next post of this series.)
How Monitoring Fastly CDN Logs Improves Performance
Performance should be top of mind for any enterprise that has a web presence. When users interact with your site, their perception and subsequent reaction are greatly impacted by their experience on your site, which then directly influences retention and conversion.
To ensure that your web properties are running optimally, monitoring performance is key. Just monitoring availability is simply not enough. Ensuring that your infrastructure is live doesn’t provide insight into user experience.
Monitoring performance through the analysis of Fastly CDN logs is crucial in getting a deeper understanding of traffic running through Fastly. Limiting your analysis to availability data gives you a narrow understanding of how users are interacting with your site, failing to give you a more intimate understanding of user behavior.
How to Monitor Fastly CDN Logs
The importance of monitoring Fastly CDN Logs and getting access to advanced insights through Sumo Logic can’t be understated.
In the next post, we will talk about how to start monitoring Fastly logs with Sumo Logic including how to start collecting log data, how to set up the Sumo Logic for Fastly App, as well as how to use Sumo Logic’s features to monitor Fastly CDN performance.
