OverviewWhen logging information about your operating system, services, network, or anything else, usually there’s predefined log structures in place by the vendor. There are times when there aren’t predefined logs created by some software or you have custom application logs from your own software. Without properly planning your log syntax you’ll be using, things can get messy and your data may lose its integrity to properly tell the story. These best practices for creating custom logs can be applied to most logging solutions.
The 5 W'sThere are 5 critical components of a good log structure*:
- When did it happen (timestamp)
- What happened (e.g., error codes, impact level, etc)
- Where did it happen (e.g., hostnames, gateways, etc)
- Who was involved (e.g., usernames)
- Where he, she, or it came from (e.g., source IP)
2017-04-10 09:50:32 -0700 - dan12345 - 10.0.24.123 - GET - /checkout/flight/ - credit.payments.io - Success - 2 - 241.98This log message shows when it was performed, what was performed, where it happened in your system, who performed it, and where that user came from. It’s also structured cleanly with a space-dash-space as the delimiters of each field. Optionally, you can also have key-value pairs to assist with parsing:
timestamp: 2017-04-10 09:50:32 -0700 - username: dan12345 - source_ip: 10.0.24.123 - method: GET - resource: /checkout/flight/ - gateway: credit.payments.io - audit: Success - flights_purchased: 2 - value: 241.98Once you have your log syntax and what will be going into the logs, be sure to document this somewhere. You can document it by adding a comment at the top of each log file. Without documentation, you may forget or someone may not know what something like “2” or “241.98” represents (for this example, it means 2 flights in the checkout at a value of $241.98). You can document our log syntax as such:
Timestamp - username - user_ip - method - resource - gateway - audit - flights_purchased - valueIn the second part of this three part series, we'll go into deeper details around timestamps and log content. In the final part, we'll go even deeper into log syntax and documentation.
*Source: Logging and Log Management: The Authoritative Guide to Understanding the Concepts Surrounding Logging and Log Management. Chuvakin, A., Phillips, C., & Schmidt, K. J. (2013).
Complete visibility for DevSecOps
Reduce downtime and move from reactive to proactive monitoring.