RabbitMQ - OpenTelemetry Collector

RabbitMQ logs are sent to Sumo Logic through the OpenTelemetry filelog receiver. RabbitMQ metrics are sent through the RabbitMQ metrics receiver.

info

This app includes built-in monitors. For details on creating custom monitors, refer to the Create monitors for RabbitMQ app.

Fields creation in Sumo Logic for RabbitMQ

Following are the Fields which will be created as part of RabbitMQ App install if not already present.

• sumo.datasource. Fixed value of rabbitmq.
• messaging.system. Fixed value of rabbitmq.
• deployment.environment. This is a user-configured field set at the time of collector installation. It identifies the environment where the rabbitmq env resides, such as dev, prod, or qa.
• messaging.cluster.name. User configured. Enter a name to uniquely identify your RabbitMQ cluster. This cluster name will be shown in the Sumo Logic dashboards.
• messaging.node.name. Includes the value of the hostname of the machine which is being monitored.

Prerequisites

For metrics collection

note

The RabbitMQ receiver supports RabbitMQ versions 3.8 and 3.9.

Enable the RabbitMQ management plugin by following the instructions in the RabbitMQ documentation. Also, make sure you at least have the monitoring level permissions to monitor the metrics.

For logs collection

This section provides instructions for configuring log collection for RabbitMQ running on a non-Kubernetes environment for the Sumo Logic App for RabbitMQ. By default, RabbitMQ logs are stored in a log file.

Follow the instructions to set up log collection:

1. RabbitMQ logs have six levels of verbosity: debug, info, warning, error, critical, and none. For more information, refer to the RabbitMQ documentation. To make sure the dashboard work properly, the log level needs to be set to debug. Default log level is info. All logging settings are located in RabbitMQ.conf.
2. Configure RabbitMQ to write log lines to a local file. By default, RabbitMQ logs are stored in /var/log/rabbitmq/rabbit@<hostname>.log. The default directory for log files is listed in the RabbitMQ.conf file. To configure the log output destination to a log file, use one of the following settings, either in the configuration file. Edit or create /etc/rabbitmq/rabbitmq.conf file config:

log.dir = /var/log/rabbitmq
log.file = rabbitmq.log
log.file.level = debug

Once the logs are configured to be written to a local file, follow the below steps to configure collection in Sumo Logic.

For Linux systems with ACL Support, the otelcol install process should have created the ACL grants necessary for the otelcol system user to access default log locations. You can verify the active ACL grants using the getfacl command. Install the ACL in your Linux environment, if not installed.

The required ACL may not be supported for some rare cases, for example, Linux OS Distro, which is officially not supported by Sumo Logic. In this case, you can run the following command to explicitly grant the permissions.

sudo setfacl -R -m d:u:otelcol-sumo:r-x,d:g:otelcol-sumo:r-x,u:otelcol-sumo:r-x,g:otelcol-sumo:r-x <PATH_TO_LOG_FILE>

Run the above command for all the log files in the directory that need to be ingested, which are not residing in the default location.

If Linux ACL Support is not available, traditional Unix-styled user and group permission must be modified. It should be sufficient to add the otelcol system user to the specific group that has access to the log files.

For Windows systems, log files which are collected should be accessible by the SYSTEM group. Use the following set of PowerShell commands if the SYSTEM group does not have access.

$NewAcl = Get-Acl -Path "<PATH_TO_LOG_FILE>"
# Set properties
$identity = "NT AUTHORITY\SYSTEM"
$fileSystemRights = "ReadAndExecute"
$type = "Allow"
# Create new rule
$fileSystemAccessRuleArgumentList = $identity, $fileSystemRights, $type
$fileSystemAccessRule = New-Object -TypeName System.Security.AccessControl.FileSystemAccessRule -ArgumentList $fileSystemAccessRuleArgumentList
# Apply new rule
$NewAcl.SetAccessRule($fileSystemAccessRule)
Set-Acl -Path "<PATH_TO_LOG_FILE>" -AclObject $NewAcl

Collection configuration and app installation

Follow these steps to set up and deploy the source template to collect data in Sumo Logic from a remotely managed OpenTelemetry collector.

Step 1: Set up remotely managed OpenTelemetry collector

note

If you want to configure your source locally, you can do so by downloading the YAML file. For details, see Configure OpenTelemetry collectors locally.

In this step, we'll install remotely managed OpenTelemetry collector and add a uniquely identifiable tag to these collectors.

1. Classic UI. In the main Sumo Logic menu, Select App Catalog.
   New UI. In the Sumo Logic main menu select App Catalog.
2. In the 🔎 Search Apps field, run a search for your desired OpenTelemetry app, then select it. For example, Apache - OpenTelemetry.
3. Click the Manage dropdown and then select Add Collection.
4. On the Set Up Collector page, select Add a New Collector and then click Next.

note

If you want to use an existing OpenTelemetry Collector, you can skip this step by selecting the Use an existing Collector option.

5. Select the platform where you want to install the Sumo Logic OpenTelemetry Collector. For example, Linux.
   This will generate a command that you can execute in the machine environment you need to monitor. Once executed, it will install the Sumo Logic OpenTelemetry Collector.
   The installation commands now include a --remotely-managed flag. When this flag is used, a remotely managed collector is installed automatically instead of a locally managed one. If you prefer to use a locally managed collector, simply remove the flag from the installation command.

Step 2: Configure the source template

In this step, you will configure the YAML required for Local File Collection. Below are the inputs required for configuration:

• Name. Name of the source template.
• Description. Description for the source template.
• Fields/Metadata. You can provide any customer fields to be tagged with the data collected. By default, sumo tags _sourceCategory with the value otel/rabbitmq.
• File Path. Provide the file which needs to be read by OpenTelemetry agent. You can provide path to multiple files by adding new entry to it.
• DenyList. Provide path expression describing the files to be excluded.
• Endpoint. (Default: http://localhost:15672.) The URL of the node to be monitored.

note

The Endpoint value should have http at the beginning. For example, http://localhost:port.

• Username. Required. Enter the RabbitMQ username.
• Password Environment Variable Name. Required. Enter the RabbitMQ password environment variable name.

Timestamp Parsing. You can configure timestamp parsing for logs ingested using this source template. For more information, see Timestamps, Time Zones, Time Ranges, and Date Formats for the OpenTelemetry Collector.

Processing Rules. You can add processing rules for logs/metrics collected. To learn more, refer to Processing Rules.

Step 3: Push the source template to the desired remotely managed collectors

info

A new source template will always be created with the latest version of the source template.

Follow the below steps to create a data collection configuration to gather the required logs and link them to all the collectors with the help of collector tags.

1. Complete the source template form with the name and file path for your logs (for example, error logs or access logs), then click Next.
2. Under Link Collectors, you will have the option to link the collectors using the collector name or by adding tags to find the group of collectors (for example, application = Apache).
   
3. Preview and confirm the collectors that will be linked (fetched automatically) to the newly created source template.
   
   
4. Click Next to complete the source template creation. In the background, the system will apply the configuration to all the linked collectors and will start collecting the respective telemetry data from the remote host (in the example, it would start collecting Apache error logs).
5. Click the Log Search or Metrics Search icons to search for and analyze your data collected for this source template.

Sample log messages

Here's a sample log message you'd find in Non-Kubernetes environments.

2023-01-16 05:53:44.858 [info] <0.44.0> Application cowboy exited with reason: stopped

Sample queries

Logs

This sample Query is from the RabbitMQ - Logs dashboard > Events by Severity panel.

Query String

 %"sumo.datasource"="rabbitmq" %"messaging.cluster.name"=* host.name=*
| json "log" as _rawlog nodrop
| if(isEmpty(_rawlog),_raw,_rawlog) as _raw
| parse "* * [*]" as date,time,severity | count by severity

Metrics

This sample query is from the Average Number of Consumers panel in RabbitMQ - Metrics dashboard.

| avg by messaging.cluster.name 
| sum

Viewing RabbitMQ Dashboards

Logs

The RabbitMQ - Logs dashboard gives you an at-a-glance view of error messages, error by severity, top and last 10 errors, broker, and Event Start/Add log messages.

Metrics

The RabbitMQ - Metrics dashboard gives you an at-a-glance view of your RabbitMQ deployment across brokers, queue, exchange, consumer, and messages.

Create monitors for RabbitMQ app

From your App Catalog:

1. From the Sumo Logic navigation, select App Catalog.
2. In the Search Apps field, search for and then select your app.
3. Make sure the app is installed.
4. Navigate to What's Included tab and scroll down to the Monitors section.
5. Click Create next to the pre-configured monitors. In the create monitors window, adjust the trigger conditions and notifications settings based on your requirements.
6. Scroll down to Monitor Details.
7. Under Location click on New Folder.
   note

   By default, monitor will be saved in the root folder. So to make the maintenance easier, create a new folder in the location of your choice.

8. Enter Folder Name. Folder Description is optional.
   tip

   Using app version in the folder name will be helpful to determine the versioning for future updates.

9. Click Create. Once the folder is created, click on Save.

RabbitMQ alerts

Name	Description	Alert Condition	Recover Condition
RabbitMQ - High Consumer Count	This alert is triggered when consumers are higher than given value (Default 10000) in a queue.	Count >= 10000	Count < 10000
RabbitMQ - High Message Queue Size	This alert is triggered when the number of messages in a queue exceeds a given threshold (Default 10000), indicating potential consumer issues or message processing bottlenecks.	Count >= 10000	Count < 10000
RabbitMQ - High Messages Count	This alert is triggered when messages are higher than given value (Default 10000) in a queue.	Count >= 10000	Count < 10000
RabbitMQ - High Unacknowledged Messages	This alert is triggered when there are too many unacknowledged messages (Default 5000), suggesting consumer processing issues.	Count >= 5000	Count < 5000
RabbitMQ - Node Down	This alert is triggered when a node in the RabbitMQ cluster is down.	Count >= 1	Count < 1
RabbitMQ - Zero Consumers Alert	This alert is triggered when a queue has no consumers, indicating potential service issues.	Count <= 0	Count > 0