Sumo Logic ahead of the packRead article
Complete visibility for DevSecOps
Reduce downtime and move from reactive to proactive monitoring.
Log analyzers are software that helps contextualize massive amounts of log data. Log analyzers reduce the amount of time that it takes to perform root cause analysis, and they provide development organizations with valuable insights that help them improve their products for their end users.
NGINX is a web server that produces error and access logs that capture key information about each request made to the server. Log data is crucial for understanding both user behavior and any issues that may be encountered as a result of a request. However, the sheer amount of data produced by a single NGINX web server (or, more commonly, many NGINX servers) makes it nearly impossible to analyze this data with any efficiency. Log analyzers solve this problem, enabling DevOps personnel (who need to understand how their applications and infrastructure are faring) to use this data effectively. The NGINX log analyzer app from Sumo Logic was designed to help development organizations overcome the challenges of performing efficient and effective NGINX log analysis.
The NGINX access logs record events in the combined log format by default. Events in this log file will look similar to the following:
XXX.XXX.X.XX - username [29/Jul/2020:08:56:22 -0400] "GET / HTTP/1.1" 200 396 "www.refererurl.com" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36" These fields represent the following information: IP address of the client making the request: XXX.XXX.X.XX Client identity (this field is usually blank): - User making the request (if authenticated): username Date and time of the request: [29/Jul/2020:08:56:22 -0400] Request type: GET / HTTP/1.1 HTTP status code returned to the client: 200 Size of the request (bytes): 396 Referer: www.refererurl.com User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36
Log analysis can be challenging for several key reasons, and NGINX log analysis is no exception. Consider the following:
Modern software environments are highly distributed. Therefore, it’s highly unlikely that all of your log files exist in one place. Rather, it’s much more probable that multiple NGINX instances are running instances of your applications at the same time and producing log events that are being written to different locations. It’s easy to see how the task of managing these logs would become highly complex very quickly, and it’s even easier to see how it would be nearly impossible to derive any usable insights from the voluminous data being produced on your own.
In order to solve this problem, you need to centralize these log files so that a full dataset can be accessed, managed, and analyzed in a single location.
Root cause analysis is the process of determining the root cause of an issue with an application or its supporting infrastructure. Proper root cause analysis ensures that the most complete and permanent fix can be applied, which leads to a high level of quality moving forward. Developers and IT personnel do this on a daily basis, and log data is often considered a critical component of this process.
In order to effectively leverage log data in the root cause analysis process, the data must be in a format in which trends are easy to identify and traces can be easily followed across multiple server instances. Looking through thousands of lines of data in a text editor just to see a piece of the overall picture simply doesn’t make sense for those who want to get the job done in an efficient manner. Finding the root cause of a problem is difficult enough without the added headache of trying to isolate occurrences of the problem in raw, unsearchable log files.
Using multiple log analysis tools may sound good in theory (more functionality!), but it doesn’t work so well in practice. When teams leverage multiple tools to manage their logs, they find themselves incurring the cost of configuring and maintaining these tools in addition to having to decide which tool to use in a given situation. It makes much more sense to utilize one platform when at all possible. This will make the lives of your developers and IT folks much easier when it comes time to troubleshoot the next problem.
We’ve already addressed the importance of centralizing NGINX logs. The NGINX Log Analyzer from SumoLogic enables server logs from each instance to be consumed, thus providing development teams with a one stop shop for all log analysis. This enables them to see the full picture of what is occurring with their applications across their entire infrastructure.
Centralized log analysis is much easier when leveraging a cloud-based log management platform. This is what SumoLogic and the NGINX Log Analyzer bring to the table, and it’s especially valuable when running applications at scale. More application and server instances plus high volumes of traffic to these instances means a high volume of log data being stored in multiple locations, thus complicating the processes for log management and analysis. SumoLogic simplifies things. In just a few steps, all NGINX server logs can be streamed to SumoLogic, where they are centralized for easy and complete analysis. These steps are documented here in detail. By following these instructions, you can begin working with server logs from all of your NGINX instances in one convenient location.
Automating analysis helps reduce the time it takes to resolve application or infrastructure problems. The SumoLogic NGINX Log Analyzer provides several key features that help with this process. Let’s take a look at a few.
Some of the most valuable features of the NGINX Log Analyzer are available to development organizations out-of-the-box. This includes prebuilt dashboards that allow developers and IT personnel to view key NGINX metrics including traffic volume and traffic distribution (and much more). These out-of-the-box visualizations provide immediate insight into commonly utilized metrics from this log data, meaning that the log analyzer provides value within minutes of setting up the NGINX collector for your web server instance.
You can’t fix a problem that you don’t know about. With the NGINX Log Analyzer, organizations can ensure that their development staff will be made aware of critical issues in a timely manner through alert configuration. By configuring alerts within the log analyzer, the correct personnel can be notified of an issue that appears in NGINX logs as soon as it occurs. This shortens the amount of time between the occurrence of the issue and the beginning of the root cause analysis process, which means that the problems can be solved faster.
Real-time log analysis is another important component of a log management strategy that strives to reduce MTTR. By allowing log events to be analyzed as they are consumed, the NGINX Log Analyzer ensures that development teams have the information necessary to derive actionable insights from their log data immediately.
This can help maximize the efficiency of a development team’s responses to various issues, such as a new bug that routes visitors to a non-existent page (resulting in a spike in 404 errors) or a repeated failed login attempt by a malicious user.
A massive benefit of working with SumoLogic’s NGINX Log Analyzer is its ability to search your NGINX logs with a simple query language. By providing development teams with the ability to easily search log entries, the NGINX Log Analyzer helps filter out the noise and irrelevant log events as well as pinpoint entries of interest when debugging specific application or infrastructure problems.
Centralizing your logs means having all information from across your entire infrastructure in one location. This simplifies the process of isolating and tracing requests that may span multiple application or server instances. In a situation that would otherwise be a nightmare to debug, the NGINX Log Analyzer enables log events to be correlated in a way that allows development teams to thoroughly understand the issue at hand. This, in turn, leads to the development of a proper resolution.
If the prebuilt dashboards don’t provide enough information to meet your log analysis needs, the NGINX Log Analyzer has the extensibility to allow for a level of customization that will help you get there. Let’s take a deeper look at the process of creating customized dashboards.
One of the major benefits of working with the NGINX Log Analyzer app is the ability to create customized analytics dashboards. Once you’re set up to consume your NGINX logs with a SumoLogic collector, you can create a customized dashboard in just a few easy steps.
When you view your logs and related analysis dashboards in SumoLogic, you will see a “New” button in the top right corner of the screen. Clicking this button will drop down a list of options, including one for the creation of a customized dashboard.
Selecting “Dashboard (New)” will give you options for various panel types. From here, you can select a panel type and begin to design a customized dashboard to meet your analytics needs.
These panels can help organizations derive critical insights from the aggregation and analysis of various types of data. Let’s take a look at some of the data that can be gleaned from NGINX server logs and how these visualizations can help analyze application issues, potentially leading to a more secure application and a better understanding of your user base.
Knowing the geographic locations of users who are accessing an application can be helpful in several different ways. It can allow development teams to identify locations where their application is becoming more popular, and it can enable IT personnel to identify potentially alarming trends (such as when visitors from known areas of concern attempt to access your application with malicious intent).
Visualizations that show which pages are visited most frequently help organizations to see which ones are most popular, and thus, which content and features their users deem most valuable. This provides feedback that helps an organization determine where to focus their efforts moving forward, and it’s critical for driving business growth.
Every application development team should be interested in HTTP code analysis. For example, if users are encountering an unusually high number of 404 codes, it’s possible that there’s a link to a missing resource somewhere in the application. Or maybe users are seeing high instances of 408 codes, indicating possible server availability issues that lead to request timeouts.
Regardless of the scenario, it’s critical that development teams keep an eye on HTTP code analysis in order to ensure that users are able to successfully access the content they are requesting with a high level of reliability. Organizations that leverage NGINX servers can accomplish this with the NGINX Log Analyzer.
The User Agent allows the server to identify critical visitor information including the operating system and the browser being used to make the request. This is important to know because it helps development teams determine if their testing strategy is adequate for ensuring that their application is functioning properly for all common browser and OS combinations. In addition, as new operating systems are released and browsers are updated, keeping an eye on this information will help to ensure application quality and viability moving forward.
The above (and more) is available within the NGINX Log Analyzer application as part of the prebuilt dashboards that analyze NGINX access log data. Analyzing data from all NGINX server instances makes it easier for development teams to identify problematic trends and improve root cause analysis – and thus to enhance their applications. Access logs contain a lot of data for each request, but this data is not easy to contextualize when scrolling through thousands of lines in a log file.
It’s easier to let a platform like Sumo Logic’s NGINX Log Analyzer do the hard work. This will allow teams to focus less on managing their logs and more on utilizing log data to bring value to their end users.
Reduce downtime and move from reactive to proactive monitoring.
Build, run, and secure modern applications and cloud infrastructures.Start free trial
Moving to the cloud offers more than economics; it comes with unique security challenges that on-premises solutions cannot address. In minutes, Cloud Infrastructure Security for AWS from Sumo Logic brings cloud-native security analytics to AWS cloud environments. Curated workflows, out-of-the-box dashboards and AI-driven anomaly detection help security personnel easily monitor cloud security posture and cloud configurations and manage cloud risk from a centralized platform.
In a perfect world, computers would function properly on the network at all times. There would be no issues with the operating system and no problems with the applications. Unfortunately, this isn’t a perfect world. System failures can and will occur, and when they do, it is the responsibility of system administrators to diagnose and resolve the issues. But where can system administrators begin the search for solutions when problems arise? The answer is Windows event logs.