Amazon ElastiCache

The Sumo Logic app for Amazon ElastiCache allows you to set up, run, and scale popular open-source compatible in-memory data stores in the cloud.

The Amazon ElastiCache dashboards provide visibility into key event and performance analytics that enable proactive diagnosis and response to system and environment issues. Use the preconfigured dashboards for at-a-glance analysis of event status trends, locations, successes and failures, as well as system health and performance metrics. The dashboards also have additional performance insights for Redis clusters.

Log and metric types

The Sumo Logic app for Amazon ElastiCache uses the following logs and metrics:

Sample log messages

Sample CloudTrail Log Message

{
	"eventVersion":"1.05",
	"userIdentity":{
		"type":"IAMUser",
		"principalId":"A12345678904QEWUABG5Q",
		"arn":"arn:aws:iam::123456789038:user/myuser",
		"accountId":"123456789038",
		"accessKeyId":"A1234567890FHCUQYQRM",
		"userName":"myuser",
		"sessionContext":{
			"attributes":{
				"mfaAuthenticated":"true",
				"creationDate":"2018-10-29T07:08:50Z"
			}
		},
		"invokedBy":"signin.amazonaws.com"
	},
	"eventTime":"2018-10-29T08:38:13Z",
	"eventSource":"elasticache.amazonaws.com",
	"eventName":"CreateCacheSubnetGroup",
	"awsRegion":"us-west-1",
	"sourceIPAddress":"29.28.30.17",
	"userAgent":"signin.amazonaws.com",
	"requestParameters":{
		"cacheSubnetGroupName":"myuser-redis-subnet-grp1",
		"subnetIds":[
			"subnet-b33fc55e"
		]
	},
	"responseElements":{
		"cacheSubnetGroupDescription":" ",
		"vpcId":"vpc-b12fc345",
		"subnets":[
			{
				"subnetAvailabilityZone":{
					"name":"us-west-1a"
				},
				"subnetIdentifier":"subnet-b33fc55e"
			}
		],
		"cacheSubnetGroupName":"myuser-redis-subnet-grp1"
	},
	"requestID":"c6a79737-1234-5678-bb74-9f27f56e6306",
	"eventID":"70c2c865-1234-4567-893c-9800b91e2502",
	"eventType":"AwsApiCall",
	"recipientAccountId":"123456789038"
}

Sample queries

Average Engine CPU Utilization by cacheclusterid and cachenodeid and Metric-based
account=* region=* namespace=aws/elasticache metric=EngineCPUUtilization statistic=Average CacheClusterId=* CacheNodeId=* | avg by CacheClusterId, CacheNodeId, account, region, namespace

ElastiCache Node Reboot Events and CloudTrail-Log-base:
account={{account}} region={{region}} namespace={{namespace}} "\"eventSource\":\"elasticache.amazonaws.com\"" (Reboot* or CacheNodesRebooted)
| json "userIdentity", "eventSource", "eventName", "awsRegion", "sourceIPAddress", "userAgent", "eventType", "recipientAccountId", "requestParameters", "responseElements", "requestID", "errorCode", "errorMessage" as userIdentity, event_source, event_name, region, src_ip, user_agent, event_type, recipient_account_id, requestParameters, responseElements, request_id, error_code, error_message nodrop
| where event_source = "elasticache.amazonaws.com" and (event_name matches "Reboot*" or event_name="CacheNodesRebooted")
| json field=userIdentity "type", "principalId", "arn", "userName", "accountId" nodrop
| json field=userIdentity "sessionContext.attributes.mfaAuthenticated" as mfaAuthenticated nodrop
| json field=requestParameters "replicationGroupId", "engine", "engineVersion", "cacheClusterId" as req_replicationGroupId, req_engine, req_engineVersion, req_cacheClusterId nodrop
| json field=responseElements "replicationGroupId", "engine", "engineVersion", "status", "cacheClusterId", "res_cacheClusterStatus", "snapshotRetentionLimit" as res_replicationGroupId, res_engine, res_engineVersion, res_status, res_cacheClusterId, res_cacheClusterStatus, snapshotretentionlimit  nodrop
| json field=responseElements "autoMinorVersionUpgrade", "cacheNodeType", "numCacheNodes" as auto_minorversion_upgrade, cachenodetype, numcachenodes nodrop
| parse field=arn ":assumed-role/*" as user nodrop  
| parse field=arn "arn:aws:iam::*:*" as accountId, user nodrop
| if (isEmpty(error_code), "Success", "Failure") as eventStatus
| if (isEmpty(userName), user, userName) as user
| if (isEmpty(req_replicationGroupId), res_replicationGroupId, req_replicationGroupId) as replicationgroupid
| if (isEmpty(req_engine), res_engine, req_engine) as engine
| if (isEmpty(req_engineVersion), res_engineVersion, req_engineVersion) as engine_version
| if (isEmpty(req_cacheClusterId), res_cacheClusterId, req_cacheClusterId) as cacheclusterid
| eventStatus as status
| where (tolowercase(cacheclusterid) matches tolowercase("{{CacheClusterId}}")) or IsBlank(cacheclusterid)
| timeslice 1s
| count as Count by _timeslice, event_name, src_ip, user, type, request_id, user_agent, cacheclusterid, replicationgroupid // , engine, engine_version, snapshotretentionlimit, status, auto_minorversion_upgrade, cachenodetype, numcachenodes
| fields _timeslice, event_name, src_ip, user, type, request_id, user_agent, cacheclusterid, replicationgroupid // , engine, engine_version, snapshotretentionlimit, status, auto_minorversion_upgrade, cachenodetype, numcachenodes, Count
| sort by _timeslice

Collecting logs and metrics for Amazon ElastiCache

Configure Hosted Collector

When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see Configure a Hosted Collector and Source.

Collect Amazon ElastiCache CloudWatch metrics

Sumo Logic supports collecting metrics using one of the following source types:

Configure an AWS Kinesis Firehose for Metrics Source (recommended)
Configure an Amazon CloudWatch Source for Metrics

note
Namespace for Amazon ElastiCache service is AWS/ElastiCache.

Follow the steps below to add custom metadata fields with your metrics:

Click +Add Field under Metadata. Each field consists of a name (key) and a corresponding value.
Create a field named account and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the AWS Observability view, and metrics can be queried using the account field.
After adding fields, check their status indicators:
- A green check mark indicates the field exists and is enabled in the Fields table schema.
- An orange exclamation icon indicates the field does not exist or is disabled in the schema.
  - You will have the option to automatically add or enable the field.
  - If a field is sent but not present or enabled in the schema, it is ignored and marked as Dropped.

Collect Amazon ElastiCache CloudTrail logs

Prerequisites

Grant Sumo Logic access to an Amazon S3 bucket.
Create a trail for your AWS account. For more information on what events are logged, refer to the ElastiCache API calls documentation.
Confirm that logs are being delivered to the Amazon S3 bucket.

note

Namespace for Amazon ElastiCache service is AWS/ElastiCache.

Follow the steps below to collect logs for Amazon ElastiCache:

Configure a CloudTrail Logs Source.
Add custom metadata fields with your logs:
1. Click +Add Field under Metadata. Each field consists of a name (key) and a corresponding value.
2. Create a field named account and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the AWS Observability view, and logs can be queried using the account field.
3. After adding fields, check their status indicators:
  - A green check mark indicates the field exists and is enabled in the Fields table schema.
  - An orange exclamation icon indicates the field does not exist or is disabled in the schema.
    - You will have the option to automatically add or enable the field.
    - If a field is sent but not present or enabled in the schema, it is ignored and marked as Dropped.

Centralized AWS CloudTrail log collection

In case you have a centralized collection of CloudTrail logs and are ingesting them from all accounts into a single Sumo Logic CloudTrail log source, create the following Field Extraction Rule to map a proper AWS account(s) friendly name / alias. Create it if not already present / update it as required.

Rule Name: AWS Accounts
Applied at: Ingest Time
Scope (Specific Data): _sourceCategory=aws/observability/cloudtrail/logs

Parse Expression

Enter a parse expression to create an account field that maps to the alias you set for each sub account. For example, if you used the dev alias for an AWS account with ID 528560886094 and the prod alias for an AWS account with ID 567680881046, your parse expression would look like:

| json "recipientAccountId"
// Manually map your aws account id with the AWS account alias you setup earlier for individual child account
| "" as account
| if (recipientAccountId = "528560886094",  "dev", account) as account
| if (recipientAccountId = "567680881046",  "prod", account) as account
| fields account

Installing the Amazon ElastiCache app

This section has instructions for installing the Sumo Logic app for Amazon ElastiCache and descriptions of each of the app dashboards along with associated use cases.

Now that you have set up a collection for Amazon ElastiCache, install the Sumo Logic app to use the pre-configured dashboards that provide visibility into your environment for real-time analysis of overall usage.

To install the app, do the following:

note

Next-Gen App: To install or update the app, you must be an account administrator or a user with Manage Apps, Manage Monitors, Manage Fields, Manage Metric Rules, and Manage Collectors capabilities depending upon the different content types part of the app.

Select App Catalog.
In the 🔎 Search Apps field, run a search for your desired app, then select it.
Click Install App.
note
Sometimes this button says Add Integration.
Click Next in the Setup Data section.
In the Configure section of your respective app, complete the following fields.
1. Field Name. If you already have collectors and sources set up, select the configured metadata field name (eg _sourcecategory) or specify other custom metadata (eg: _collector) along with its metadata Field Value.
Click Next. You will be redirected to the Preview & Done section.

Post-installation

Once your app is installed, it will appear in your Installed Apps folder, and dashboard panels will start to fill automatically.

Each panel slowly fills with data matching the time range query received since the panel was created. Results will not immediately be available but will be updated with full graphs and charts over time.

As part of the app installation process, the following content will be created by default along with dashboards and monitor template:

Fields

account Name / alias to the AWS account.
accountid AWS account id.
region The region to which the resource name belongs to.
namespace Namespace for Amazon ElastiCache service is AWS/ElastiCache.
cacheclusterid A cache cluster ID is a user-supplied, unique name used to identify and manage an Amazon ElastiCache cluster.

Field Extraction Rule(s)

The FER AwsObservabilityElastiCacheCloudTrailLogsFER to extract fields accountid, namespace, region, and cacheclusterid will be created as a part of app installation.

Viewing Amazon ElastiCache dashboards

Host Performance Overview

The Amazon ElastiCache - Host Performance Overview dashboard provides detailed insights into CPU, memory and network performance metrics of hosts running your ElastiCache clusters.

Use this dashboard to:

Get an at-a-glance view of CPU, memory and swap resource utilization of hosts running your ElastiCache clusters.
CPU, memory or swap space on host and swap usage.
Monitor network traffic utilization and compare today’s trends of incoming and outgoing bytes and packets vs. yesterday

Amazon ElastiCache - Host Performance Overview

Audit Event Overview

The Amazon ElastiCache - Audit Event Overview dashboard provides detailed insights into all events associated with ElastiCache clusters and specifically helps identify changes, errors, users and replication groups.

Use this dashboard to:

Monitor all changes being made to ElastiCache clusters
Monitor the location of incoming user activity, top users, and top replication groupsto ensure they match with expectations
Quickly identify top error codes to diagnose any outages
Monitor trends around failed events to identify potential service disruptions that could warrant deeper investigation

Amazon ElastiCache - Audit Event Overview

Redis Performance Overview

The Amazon ElastiCache - Redis Performance Overview dashboard provides an overview into performance, evictions and authentication and authorization failures of ElastiCache Redis clusters.

Use this dashboard to:

Quickly determine if your Redis database is performing as expected

Amazon ElastiCache - Redis Performance Overview

Audit Event Details

The Amazon ElastiCache - Audit Event Details dashboard provides detailed insights into key operations made on your ElastiCache clusters.

Use this dashboard to:

Quickly determine changes made to your ElastiCache clusters while troubleshooting production outages
Determine if any nodes hosting your ElastiCache clusters were rebooted

Amazon ElastiCache - Audit Event Details

Host Performance Details

The Amazon ElastiCache - Host Performance Details dashboard shows an overview of the resource utilization for a given ElastiCache cluster across its nodes and also showcases trends around CPU, memory, swap usage and network traffic.

Use this dashboard to:

Get an at-a-glance view of the performance of all nodes within a given ElastiCache cluster
Determine if CPU, memory, swap memory or network resources need to be scaled up or down for a given cluster or service based on utilization trends

Amazon ElastiCache - Host Performance Details

Redis Performance Details

The Amazon ElastiCache - Redis Performance Details dashboard provides detailed insights into cache hits, keys, replication, connections and failures of Redis ElastiCache clusters.

Use this dashboard to:

Monitor trends around cache hits and misses to determine if Redis clusters need to be tuned
Review trends around defragmentation, replication lag and bytes replicated to determine optimizations
Quickly determine any authentication and authorization failures and grant or revoke privileges accordingly

Amazon ElastiCache - Redis Performance Details

Redis Command Latency

The Amazon ElastiCache - Redis Command Latency dashboard provides detailed insights into latency of various Redis commands.

Use this dashboard to:

To optimize performance of your Redis clusters by monitoring latency observed across get/set operations. Latency can be high due to high CPU usage, swapping or removing cached items. Performance optimizations can therefore be made either via resource allocation or by optimizing on caching.

Amazon ElastiCache - Redis Command Latency

Redis Command Stats

The Amazon ElastiCache - Redis Command Stats dashboard provides detailed insights into the number of commands being performed.

Use this dashboard to:

Monitor various Get and Set commands received by your ElastiCache clusters and nodes
Correlate with the Redis Command Latency dashboard to identify most/least frequently used and least performant command types

If high latency commands are not being processed frequently, you will want to look into monitoring and potentially allocating more CPU resources.

Amazon ElastiCache - Redis Command Stats

Create monitors for Amazon ElastiCache app

From your App Catalog:

From the Sumo Logic navigation, select App Catalog.
In the Search Apps field, search for and then select your app.
Make sure the app is installed.
Navigate to What's Included tab and scroll down to the Monitors section.
Click Create next to the pre-configured monitors. In the create monitors window, adjust the trigger conditions and notifications settings based on your requirements.
Scroll down to Monitor Details.
Under Location click on New Folder.
note
By default, monitor will be saved in the root folder. So to make the maintenance easier, create a new folder in the location of your choice.
Enter Folder Name. Folder Description is optional.
tip
Using app version in the folder name will be helpful to determine the versioning for future updates.
Click Create. Once the folder is created, click on Save.