Using Logs to Speed Your DevOps Workflow

Why is this the case? How can organizations take advantage of logging to help drive their DevOps workflow? Keep reading for a discussion of how logging and DevOps fit together.

Defining DevOps

If you are familiar with DevOps, you know that it’s a new (well, new as of the past five years or so) philosophy of software development and delivery. It prioritizes the following goals:

• Constant communication. DevOps advocates the removal of “silos” between different teams. That way, everyone involved in development and delivery, from programmers to QA folks to production-environment admins, can collaborate readily and remain constantly aware of the state of development.
• Continuous integration and delivery. Rather than making updates to code incrementally, DevOps prioritizes a continuous workflow. Code is updated continuously, the updates are tested continuously, and validated code is delivered to users continuously.
• Flexibility. DevOps encourages organizations to use whichever tool is the best for solving a particular challenge, and to update tool sets regularly as new solutions emerge. Organizations should not wed themselves to a particular framework.
• Efficient testing. Rather than waiting until code is near production to test it, DevOps prioritizes testing early in the development cycle. (This strategy is sometimes called “shift-left” testing.) This practice assures that problems are discovered when the time required to fix them is minimal, rather than waiting until they require a great deal of back peddling.

DevOps and Logging

None of the goals outlined above deal with logging specifically. And indeed, because DevOps is all about real-time workflows and continuous delivery, it can be easy to ignore logging entirely when you’re trying to migrate to a DevOps-oriented development strategy.

Yet logs are, in fact, a crucial part of an efficient DevOps workflow. Log collection and analysis help organizations to optimize their DevOps practices in a number of ways.

Consider how log aggregation impacts the following areas.

Communication and Workflows in DevOps

Implementing effective communication across your team is about more than simply breaking down silos and ensuring that everyone has real-time communication tools available. Logs also facilitate efficient communication. They do this in two ways.

First, they maximize visibility into the development process for all members of the team. By aggregating logs from across the delivery pipeline—from code commits to testing logs to production server logs—log analytics platforms assure that everyone can quickly locate and analyze information related to any part of the development cycle. That’s crucial if you want your team members to be able to remain up to speed with the state of development.

Second, log analytics help different members of the organization understand one another. Your QA team is likely to have only a basic ability to interpret raw code commits. Your programmers are not specialists in reading test results. Your admins, who deploy software into production, are experts in only that part of the delivery pipeline.

Log analytics, however, can be used to help any member of the team interpret data associated with any part of the development process. Rather than having to understand raw data from a part of the workflow with which they are not familiar, team members can rely on analytics results to learn what they need quickly.

Continuous visibility

To achieve continuous delivery, you need to have continuous visibility into your development pipeline. In other words, you have to know, on a constant, ongoing basis, what is happening with your code. Otherwise, you’re integrating in the dark.

Log aggregation and analytics help deliver continuous visibility. They provide a rich source of information about your pipeline at all stages of development. If you want to know the current quality and stability of your code, for example, you can quickly pull analytics from testing logs to find that information. If you want to know how your app is performing in production, you can do the same with server logs.

Flexibility from log analytics

In order to switch between development frameworks and programming languages at will, you have to ensure that moving between platforms requires as little change as possible to your overall workflow. Without log aggregation, however, you’ll have to overhaul the way you store and analyze logs every time you add or subtract a tool from your pipeline.

A better approach is to use a log aggregation and analytics platform like Sumo Logic. By supporting a wide variety of logging configurations, Sumo Logic assures that you can modify your development environment as needed, while keeping your logging solution constant.

Faster testing through log analytics

Performing tests earlier in the development cycle leads to faster, more efficient delivery only if you are able to fix the problems discovered by tests quickly. Otherwise, the bugs that your tests reveal will hold up your workflow, no matter how early the bugs are found.

Log analytics are a helpful tool for getting to the bottom of bugs. By aggregating and analyzing logs from across your pipeline, you can quickly get to the source of a problem with your code. Logs help keep the continuous delivery pipeline flowing smoothly, and maximize the value of shift-left testing.

Moving Towards a DevOps Workflow

Log aggregation and analytics may not be the first things that come to mind when you think of DevOps. But effective collection and interpretation of log data is a crucial part of achieving a DevOps-inspired workflow.

Logging on its own won’t get you to continuous delivery, of course. That requires many other ingredients, too. But it will get you closer if you’re not already there. And if you are currently delivering continuously, effective logging can help to make your pipeline even faster and more efficient.

About the Author

Chris Tozzi has worked as a journalist and Linux systems administrator. He has particular interests in open source, agile infrastructure and networking. He is Senior Editor of content and a DevOps Analyst at Fixate IO.