Log analyzers are software that helps contextualize massive amounts of log data. Log analyzers reduce the amount of time that it takes to perform root cause analysis, and they provide development organizations with valuable insights that help them improve their products for their end users.
NGINX is a web server that produces error and access logs that capture key information about each request made to the server. Log data is crucial for understanding both user behavior and any issues that may be encountered as a result of a request. However, the sheer amount of data produced by a single NGINX web server (or, more commonly, many NGINX servers) makes it nearly impossible to analyze this data with any efficiency. Log analyzers solve this problem, enabling DevOps personnel (who need to understand how their applications and infrastructure are faring) to use this data effectively. The NGINX log analyzer app from Sumo Logic was designed to help development organizations overcome the challenges of performing efficient and effective NGINX log analysis.
NGINX Log Format
The NGINX access logs record events in the combined log format by default. Events in this log file will look similar to the following:
XXX.XXX.X.XX - username [29/Jul/2020:08:56:22 -0400] "GET / HTTP/1.1" 200 396 "www.refererurl.com" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36"
These fields represent the following information:
IP address of the client making the request:
XXX.XXX.X.XX
Client identity (this field is usually blank):
-
User making the request (if authenticated):
username
Date and time of the request:
[29/Jul/2020:08:56:22 -0400]
Request type:
GET / HTTP/1.1
HTTP status code returned to the client:
200
Size of the request (bytes):
396
Referer:
www.refererurl.com
User Agent:
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36
Why Is NGINX Log Analysis So Challenging?
Log analysis can be challenging for several key reasons, and NGINX log analysis is no exception. Consider the following:
Log Management Is A Complex Task
Modern software environments are highly distributed. Therefore, it’s highly unlikely that all of your log files exist in one place. Rather, it’s much more probable that multiple NGINX instances are running instances of your applications at the same time and producing log events that are being written to different locations. It’s easy to see how the task of managing these logs would become highly complex very quickly, and it’s even easier to see how it would be nearly impossible to derive any usable insights from the voluminous data being produced on your own.
In order to solve this problem, you need to centralize these log files so that a full dataset can be accessed, managed, and analyzed in a single location.
Getting to the Root Cause Is Not Simple
Root cause analysis is the process of determining the root cause of an issue with an application or its supporting infrastructure. Proper root cause analysis ensures that the most complete and permanent fix can be applied, which leads to a high level of quality moving forward. Developers and IT personnel do this on a daily basis, and log data is often considered a critical component of this process.
In order to effectively leverage log data in the root cause analysis process, the data must be in a format in which trends are easy to identify and traces can be easily followed across multiple server instances. Looking through thousands of lines of data in a text editor just to see a piece of the overall picture simply doesn’t make sense for those who want to get the job done in an efficient manner. Finding the root cause of a problem is difficult enough without the added headache of trying to isolate occurrences of the problem in raw, unsearchable log files.
Teams Have to Toggle between Multiple Tools
Using multiple log analysis tools may sound good in theory (more functionality!), but it doesn’t work so well in practice. When teams leverage multiple tools to manage their logs, they find themselves incurring the cost of configuring and maintaining these tools in addition to having to decide which tool to use in a given situation. It makes much more sense to utilize one platform when at all possible. This will make the lives of your developers and IT folks much easier when it comes time to troubleshoot the next problem.
Centralized Logging with the Sumo Logic NGINX Log Analyzer
We’ve already addressed the importance of centralizing NGINX logs. The NGINX Log Analyzer from SumoLogic enables server logs from each instance to be consumed, thus providing development teams with a one stop shop for all log analysis. This enables them to see the full picture of what is occurring with their applications across their entire infrastructure.
Manage All NGINX Logs on Cloud
Centralized log analysis is much easier when leveraging a cloud-based log management platform. This is what SumoLogic and the NGINX Log Analyzer bring to the table, and it’s especially valuable when running applications at scale. More application and server instances plus high volumes of traffic to these instances means a high volume of log data being stored in multiple locations, thus complicating the processes for log management and analysis. SumoLogic simplifies things. In just a few steps, all NGINX server logs can be streamed to SumoLogic, where they are centralized for easy and complete analysis. These steps are documented here in detail. By following these instructions, you can begin working with server logs from all of your NGINX instances in one convenient location.
Reduce the Time It Takes to Resolve Issues
Automating analysis helps reduce the time it takes to resolve application or infrastructure problems. The SumoLogic NGINX Log Analyzer provides several key features that help with this process. Let’s take a look at a few.
Get Out-Of-The-Box Dashboards for NGINX Log Analysis
Some of the most valuable features of the NGINX Log Analyzer are available to development organizations out-of-the-box. This includes prebuilt dashboards that allow developers and IT personnel to view key NGINX metrics including traffic volume and traffic distribution (and much more). These out-of-the-box visualizations provide immediate insight into commonly utilized metrics from this log data, meaning that the log analyzer provides value within minutes of setting up the NGINX collector for your web server instance.
Monitor Your NGINX Logs with Alerts
You can’t fix a problem that you don’t know about. With the NGINX Log Analyzer, organizations can ensure that their development staff will be made aware of critical issues in a timely manner through alert configuration. By configuring alerts within the log analyzer, the correct personnel can be notified of an issue that appears in NGINX logs as soon as it occurs. This shortens the amount of time between the occurrence of the issue and the beginning of the root cause analysis process, which means that the problems can be solved faster.
Analyze Logs and Uncover Insights in Real Time
Real-time log analysis is another important component of a log management strategy that strives to reduce MTTR. By allowing log events to be analyzed as they are consumed, the NGINX Log Analyzer ensures that development teams have the information necessary to derive actionable insights from their log data immediately.
This can help maximize the efficiency of a development team’s responses to various issues, such as a new bug that routes visitors to a non-existent page (resulting in a spike in 404 errors) or a repeated failed login attempt by a malicious user.
Search through NGINX Log Data with a Simple Query Language
A massive benefit of working with SumoLogic’s NGINX Log Analyzer is its ability to search your NGINX logs with a simple query language. By providing development teams with the ability to easily search log entries, the NGINX Log Analyzer helps filter out the noise and irrelevant log events as well as pinpoint entries of interest when debugging specific application or infrastructure problems.
Use Additional Trace Context to Reduce Troubleshooting Time
Centralizing your logs means having all information from across your entire infrastructure in one location. This simplifies the process of isolating and tracing requests that may span multiple application or server instances. In a situation that would otherwise be a nightmare to debug, the NGINX Log Analyzer enables log events to be correlated in a way that allows development teams to thoroughly understand the issue at hand. This, in turn, leads to the development of a proper resolution.
Building an NGINX Access Log Monitoring Dashboard
If the prebuilt dashboards don’t provide enough information to meet your log analysis needs, the NGINX Log Analyzer has the extensibility to allow for a level of customization that will help you get there. Let’s take a deeper look at the process of creating customized dashboards.
Creating a Monitoring Dashboard
One of the major benefits of working with the NGINX Log Analyzer app is the ability to create customized analytics dashboards. Once you’re set up to consume your NGINX logs with a SumoLogic collector, you can create a customized dashboard in just a few easy steps.
When you view your logs and related analysis dashboards in SumoLogic, you will see a “New” button in the top right corner of the screen. Clicking this button will drop down a list of options, including one for the creation of a customized dashboard.
Selecting “Dashboard (New)” will give you options for various panel types. From here, you can select a panel type and begin to design a customized dashboard to meet your analytics needs.
These panels can help organizations derive critical insights from the aggregation and analysis of various types of data. Let’s take a look at some of the data that can be gleaned from NGINX server logs and how these visualizations can help analyze application issues, potentially leading to a more secure application and a better understanding of your user base.
Geo Access
Knowing the geographic locations of users who are accessing an application can be helpful in several different ways. It can allow development teams to identify locations where their application is becoming more popular, and it can enable IT personnel to identify potentially alarming trends (such as when visitors from known areas of concern attempt to access your application with malicious intent).
NGINX URLs
Visualizations that show which pages are visited most frequently help organizations to see which ones are most popular, and thus, which content and features their users deem most valuable. This provides feedback that helps an organization determine where to focus their efforts moving forward, and it’s critical for driving business growth.
NGINX HTTP Code
Every application development team should be interested in HTTP code analysis. For example, if users are encountering an unusually high number of 404 codes, it’s possible that there’s a link to a missing resource somewhere in the application. Or maybe users are seeing high instances of 408 codes, indicating possible server availability issues that lead to request timeouts.
Regardless of the scenario, it’s critical that development teams keep an eye on HTTP code analysis in order to ensure that users are able to successfully access the content they are requesting with a high level of reliability. Organizations that leverage NGINX servers can accomplish this with the NGINX Log Analyzer.
Analysis of NGINX User Agent (OS, Browser, Etc.)
The User Agent allows the server to identify critical visitor information including the operating system and the browser being used to make the request. This is important to know because it helps development teams determine if their testing strategy is adequate for ensuring that their application is functioning properly for all common browser and OS combinations. In addition, as new operating systems are released and browsers are updated, keeping an eye on this information will help to ensure application quality and viability moving forward.
NGINX Access Log Monitoring Dashboards
The above (and more) is available within the NGINX Log Analyzer application as part of the prebuilt dashboards that analyze NGINX access log data. Analyzing data from all NGINX server instances makes it easier for development teams to identify problematic trends and improve root cause analysis – and thus to enhance their applications. Access logs contain a lot of data for each request, but this data is not easy to contextualize when scrolling through thousands of lines in a log file.
It’s easier to let a platform like Sumo Logic’s NGINX Log Analyzer do the hard work. This will allow teams to focus less on managing their logs and more on utilizing log data to bring value to their end users.
