})
The Sumo Logic app for Varnish provides dashboards that help you analyze log and metric events generated by Varnish servers. This app allows you to identify traffic sources, monitor and improve application and website workflows, and understand how customers use your product.
This app is tested with the following versions:
* For Kubernetes environments: Varnish Version 6.4.
* Non-Kubernetes environments: Varnish Version 6.0.7.
## Sample log messages
})
The first service in the pipeline is Telegraf. Telegraf collects metrics from Varnish. Note that we’re running Telegraf in each pod we want to collect metrics from as a sidecar deployment, for example, Telegraf runs in the same pod as the containers it monitors. Telegraf uses the Varnish input plugin to obtain metrics. For simplicity, the diagram doesn’t show the input plugins. The injection of the Telegraf sidecar container is done by the Telegraf Operator.
Prometheus pulls metrics from Telegraf and sends them to [Sumo Logic Distribution for OpenTelemetry Collector](https://github.com/SumoLogic/sumologic-otel-collector), which enriches metadata and sends metrics to Sumo Logic.
In the logs pipeline, Sumo Logic Distribution for OpenTelemetry Collector collects logs written to standard out and forwards them to another instance of Sumo Logic Distribution for OpenTelemetry Collector, which enriches metadata and sends logs to Sumo Logic.
:::note Prerequisites
It’s assumed that you are using the latest Helm chart version. If not, upgrade using the instructions [here](/docs/send-data/kubernetes).
:::
### Configure metrics collection
This section explains the steps to collect Varnish metrics from a Kubernetes environment.
1. [Set up Kubernetes Collection with the Telegraf Operator](/docs/send-data/collect-from-other-data-sources/collect-metrics-telegraf/install-telegraf).
2. On your Varnish Pods, add the following annotations:
```sql
annotations:
telegraf.influxdata.com/class: sumologic-prometheus
prometheus.io/scrape: "true"
prometheus.io/port: "9273"
telegraf.influxdata.com/inputs: |+
[[inputs.varnish]]
use_sudo = true
binary = "/usr/bin/varnishstat"
stats = ["*"]
[inputs.varnish.tags]
component="cache"
environment="dev_CHANGME"
cache_system="varnish"
cache_cluster="varnish_on_k8s_CHANGEME"
```
Enter in values for the following parameters (marked `CHANGEME` in the snippet above):
* `telegraf.influxdata.com/inputs`. This contains the required configuration for the Telegraf Varnish input plugin. Please refer to this [doc](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/redis) for more information on configuring the Varnish input plugin for Telegraf. Note: As Telegraf will be run as a sidecar, the host should always be localhost.
* In the input plugins section, which is `[[inputs.varnish]]`
* `binary`. The default location of the Varnish stat binary. Please override as per your configuration.
* `use_sudo`. If running as a restricted user, prepend sudo for additional access.
* `stats`. Stats may also be set to `["*"]`, which will collect all stats. See [this doc](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/varnish) for more information on additional parameters for configuring the Varnish input plugin for Telegraf.
* In the tags section, which is `[inputs.varnish.tags]`
* `environment`. This is the deployment environment where the Varnish cluster identified by the value of servers resides. For example: dev, prod, or qa. While this value is optional, we highly recommend setting it.
* `cache_cluster`. Enter a name to identify this Varnish cluster. This cluster name will be shown in the Sumo Logic dashboards.
**Do not modify** the following values set by this configuration, as it will cause the Sumo Logic app to not function correctly.
* `telegraf.influxdata.com/class: sumologic-prometheus`. This instructs the Telegraf operator what output to use. This should not be changed.
* `prometheus.io/scrape: "true"`. This ensures our Prometheus will scrape the metrics.
* `prometheus.io/port: "9273"`. This tells Prometheus what ports to scrape on. This should not be changed.
* `telegraf.influxdata.com/inputs`
* In the tags section, `[inputs.varnish.tags]`
* `component: “cache”`. This value is used by Sumo Logic apps to identify application components.
* `cache_system: “varnish”`. This value identifies the web server system.
For all other parameters, see [this doc](/docs/send-data/collect-from-other-data-sources/collect-metrics-telegraf/install-telegraf#configuring-telegraf) for more parameters that can be configured in the Telegraf agent globally.
3. Sumo Logic Kubernetes collection will automatically start collecting metrics from the pods having the labels and annotations defined in the previous step.
4. Verify metrics in Sumo Logic.
### Configure logs collection
This section explains the steps to collect Varnish logs from a Kubernetes environment.
1. **(Recommended Method) Add labels on your Varnish pods to capture logs from standard output**. Follow the instructions below to capture Varnish logs from stdout on Kubernetes.
1. Apply the following labels to the Varnish pods:
```bash
environment: "prod_CHANGEME"
component: "cache"
cache_system: "varnish"
cache_cluster: "varnish_on_k8s_CHANGEME"
```
2. Enter the values for the following parameters (marked `CHANGEME` in the snippet above):
* `environment`. This is the deployment environment where the Varnish cluster identified by the value of `servers` resides. For example: dev, prod, or qa. While this value is optional,l we highly recommend setting it.
* `cache_cluster`. Enter a name to identify this Varnish cluster. This cluster name will be shown in the Sumo Logic dashboards.
**Do not modify** the following values set by this configuration, as it will cause the Sumo Logic app to not function correctly.
* `component: “cache”`. This value is used by Sumo Logic apps to identify application components.
* `cache_system: “varnish”`. This value identifies the cache system.
For all other parameters, see [this doc](/docs/send-data/collect-from-other-data-sources/collect-metrics-telegraf/install-telegraf#configuring-telegraf) for more parameters that can be configured in the Telegraf agent globally.
2. (Optional) Collecting Varnish Logs from a Log File. Follow the steps below to capture Varnish logs from a log file on Kubernetes.
1. Install the Sumo Logic [tailing sidecar operator](https://github.com/SumoLogic/tailing-sidecar/tree/main/operator#deploy-tailing-sidecar-operator).
2. Add the following annotation in addition to the existing annotations.
```xml
annotations:
tailing-sidecar: sidecarconfig;**FER to normalize the fields in Kubernetes environments.** Labels created in Kubernetes environments automatically are prefixed with pod_labels. To normalize these for our app to work, a Field Extraction Rule named **AppObservabilityVarnishFER** is automatically created for Varnish Application Components.
})
Telegraf runs on the same system as Varnish and uses the [Varnish input plugin](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/varnish) to obtain Varnish metrics, and the Sumo Logic output plugin to send the metrics to Sumo Logic. Logs from Varnish, on the other hand, are sent to a Sumo Logic Local File source.
### Configure metrics collection
This section provides instructions for configuring metrics collection for the Sumo Logic app for Varnish.
1. **Configure a Hosted Collector**. To create a new Sumo Logic hosted collector, perform the steps in the[Create a Hosted Collector](/docs/send-data/hosted-collectors/configure-hosted-collector) section of the Sumo Logic documentation.
2. **Configure an HTTP Logs and Metrics Source**. Create a new HTTP Logs and Metrics Source in the hosted collector created above by following the instructions given in this [document](/docs/send-data/hosted-collectors/http-source/logs-metrics). Make a note of the **HTTP Source URL**.
3. **Install Telegraf**. Use the[ following steps](/docs/send-data/collect-from-other-data-sources/collect-metrics-telegraf/install-telegraf.md) to install Telegraf.
4. **Configure and start Telegraf**. As part of collecting metrics data from Telegraf, we will use the [Varnish input plugin](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/varnish) to get data from Telegraf and the [Sumo Logic output plugin](https://github.com/SumoLogic/fluentd-output-sumologic) to send data to Sumo Logic.
Create or modify telegraf.conf and copy and paste the text below:
```sql
use_sudo = true
binary = "/usr/bin/varnishstat"
stats = ["*"]
[inputs.varnish.tags]
component="cache"
environment="dev_CHANGME"
cache_system="varnish"
cache_cluster="varnish_on_premise_CHANGEME"
url = "})
### Visitor Traffic Insight
The **Varnish (Classic) - Visitor Traffic Insight** dashboard provides detailed information on the top documents accessed, top referrers, top search terms from popular search engines, and the media types served.
Use this dashboard to:
* Gain insights into visitor traffic.
* Identify top URLs visited.
* Determine visitor locations.
* Platforms, browsers, PC, Mac versions are used by the visitors to access.
})
### Web Server Operations
The **Varnish (Classic) - Web Server Operations** dashboard provides a high-level view combined with detailed information on the top ten bots, geographic locations, and data for clients with high error rates, server errors over time, and non-200 response code status codes. Dashboard panels also show server error logs, error log levels, error responses by the server, and the top URLs responsible for 404 responses.
Use this dashboard to:
* Determine failures in responding.
* Identify visitor locations with 4xx errors.
* Gain insights into Clients causing a lot of 4xx errors.
})
### Traffic Timeline Analysis
The **Varnish (Classic) - Traffic Timeline Analysis** dashboard provides a high-level view of the activity and health of Varnish servers on your network. Dashboard panels display visual graphs and detailed information on traffic volume and distribution, responses over time, as well as time comparisons for visitor locations and server hits.
Use this dashboard to:
* To understand the traffic distribution across servers, provide insights for resource planning by analyzing data volume and bytes served.
* Gain insights into the origin of traffic location by region. This can help you allocate compute resources to different regions according to their needs.
})
### Outlier Analysis
The **Varnish (Classic) - Outlier Analysis** dashboard provides a high-level view of Varnish server outlier metrics for bytes served, the number of visitors, and server errors. You can select the time interval over which outliers are aggregated, then hover the cursor over the graph to display detailed information for that point in time.
Use this dashboard to:
* Detect outliers in your infrastructure with Sumo Logic’s machine learning algorithm.
* To identify outliers in incoming traffic and the number of errors encountered by your servers.
})
### Threat Intel
The **Varnish (Classic) - Threat Intel** dashboard provides an at-a-glance view of threats to Varnish servers on your network. Dashboard panels display threat count over a selected time period, geographic locations where threats occurred, source breakdown, actors responsible for threats, severity, and a correlation of IP addresses, method, and status code of threats.
Use this dashboard to:
* To gain insights and understand threats in incoming traffic and discover potential IOCs.
* Incoming traffic requests are analyzed using Sumo Logic [threat intelligence](/docs/security/threat-intelligence/).
})
### Backend Servers
The **Varnish (Classic) - Backend Servers** dashboard provides several metrics that describe the communication between Varnish and its backend servers.
Use this dashboard to:
* Review and manage the health of backend and frontend communication.
})
### Bans and Bans Lurker
The **Varnish (Classic) - Bans and Bans Lurker** dashboard provides you with the list of Bans filters applied to keep Varnish from serving stale content.
Use this dashboard to:
* Gain insights into bans and make sure that Varnish is serving the latest content.
})
### Cache Performance
The **Varnish (Classic) - Cache Performance** dashboard provides worker thread-related metrics to tell you if your thread pools are healthy and functioning well.
Use this dashboard to:
* Gain insights into the performance and health of Varnish Cache.
* Determine if any corrective actions are required to provide high performance and availability.
})
### Clients
The **Varnish (Classic) - Clients** dashboard check collects Varnish metrics regarding connections and requests.
Use this dashboard to:
* Review the current sessions and load on Varnish.
* Determine if there are failures because of overloading and if additional resources are required.
})
### Threads
The **Varnish (Classic) - Threads** Dashboard helps you to keep track of thread metrics to watch your Varnish Cache.
Use this dashboard to:
* Manage and understand threads in the Varnish system.
})
## Create monitors for Varnish app
import CreateMonitors from '../../reuse/apps/create-monitors.md';