Back to blog results

October 10, 2019 By Katie Lane

Kubernetes DevSecOps with Sumo Logic

With Sumo Logic, we can put all of these pieces together to build end-to-end Observability in Kubernetes.

  1. Setup and Collection - The entire collection process can be set up with a single Helm chart. Fluentbit, Fluentd, Prometheus, and Falco are deployed throughout the cluster in order to collect log, metric, event and security data.
  2. Enrichment - Once collected, the data flows into a centralized Fluntd pipeline for metadata enrichment. Data is enriched— tagged— with the details about where in the cluster it originated; the service, deployment, namespace, node, pod, container, and their labels.
  3. Sumo Logic - Finally, the data is sent to Sumo Logic via HTTP for storage, access, and most importantly analytics.

Note: Labels — When you create objects in Kubernetes, you can assign custom key-value pairs to each of those objects, called labels. These labels can help you organize and track additional information about each object. For example, you might have a label that represents the application name, the environment the pod is running in or perhaps what team owns this resource. These labels are entirely flexible and can be defined as you need. Our FluentD plugin ensures those labels get captured along with the logs giving you continuity between the resources you have created and the log files they are producing.

Metadata Enrichment

Unified metadata enrichment is critical to building context about the data in your cluster, and the hierarchy of the components present. Standalone prometheus or fluentd deployments give some context about the data — node, container, and pod level information — but not valuable insight to the service, deployment or namespace. Sumo Logic uses Fluentd as a centralized metadata pipeline to ping the API server and gain rich context about the data getting pass into Sumo Logic.

By centralizing metadata enrichment, the Sumo Logic solution reduces the load on the Kubernetes API server and ensures consistent metadata tagging across logs, metrics and events without which it would be impossible to correlate data when troubleshooting. You can use this metadata when searching through your logs and your metrics and use them together to have a unified experience when navigating your machine data.

Namespace overview gives quick visibility into pods experiencing issues or in this case, in a CrashLoopBackOff state.

Ingestion into Sumo Logic

There is tremendous value in having this data come to a single place. With metrics serving as the smoke detector, and logs enabling us to drill down to the root cause, unifying these data sources around a common metadata language enables us to easily correlate these signals. We can pivot from the metrics data about a cluster to the events data about a cluster to the logs data about an application.

Metadata enables us to build a hierarchical view of a cluster. By connecting pods to their services or group nodes by cluster, it becomes easier to explore the Kubernetes stack. By tapping into the Auto-discovery capabilities inherent in Prometheus, we can ensure that the hierarchy visualized in Sumo Logic is accurate and up to date.

Rich metadata enables Sumo Logic to automatically build out the explorer hierarchy of the components present in your cluster, and keep the explorer up to date as pods are added and removed.

Tying together DevOps and SecOps

We can take this further by providing data about security relevant events in the context of the Kubernetes mental model. Below we can see top security rules triggered in the cluster overview. Zoom in and we see this same data for the service or namespace and so on.

Displaying security information within the natural hierarchies of Kubernetes, we can enable a consistent view across DevOps and SecOp to build closer and more efficient DevSecOps cooperation.

Security visibility is available at the cluster level alongside log, metric, and event data.

Kubernetes security, application security, and network security

Zooming out, we can also take out Kubernetes security data and insert it in our high-level security dashboards. Combining infrastructure security, network security, full-stack security, and Kubernetes security gives us comprehensive visibility into the entire security story.

Kubernetes Observability - Free ebook

Monitoring, troubleshooting and securing Kubernetes with Sumo Logic

Katie Lane

Katie Lane

Product Marketing Manager - Operational Analytics

More posts by Katie Lane.

People who read this also enjoyed