Amazon ECS without Container Insights and Traces

Amazon Elastic Container Service (Amazon ECS) is a container management service that allows you to manage Docker containers on a cluster of Amazon EC2 instances. The Sumo Logic app for Amazon ECS provides preconfigured searches and Dashboards that allow you to monitor various metrics (CPU and Memory Utilization, CPU and Memory Reservation) across ECS clusters and services. The app also monitors API calls made by or on behalf of Amazon ECS in your AWS account.

We offer two different ECS versions, which have separate data collection steps:

Collect Logs and Metrics for ECS. This version collects ECS CloudWatch Metrics and ECS Events using AWS CloudTrail.
Collect Logs, Metrics (Container Insights+CloudWatch) and Traces for ECS. This version collects ECS CloudWatch Metrics, Container Insights Metrics, ECS Events using AWS CloudTrail, Application Logs and Traces. Metrics collected by Container Insights are charged as custom metrics. For more information about CloudWatch pricing, see Amazon CloudWatch Pricing. This solution enables you to monitor both EC2 and Fargate based ECS deployments. For instructions on collecting this data, refer to the Amazon Elastic Container Service (ECS) using Container Insights and CloudWatch.

This documentation has instructions for collecting logs and metrics for the Amazon ECS app without Container Insights and Traces.

Log and metric types

The Sumo Logic app for Amazon ECS without Container Insights and Traces uses the following logs and metrics:

Amazon CloudWatch Metrics
Amazon CloudTrail Logs

Sample log messages

Click to expand

{
   "eventVersion":"1.04",
   "userIdentity":{
      "type":"AssumedRole",
      "principalId":"ADFDDDFF7FDF7GFFF2DF0:i-76vfa923",
      "arn":"arn:aws:sts::435456556566:assumed-role/ecsInstanceRole/i-76vfa923",
      "accountId":"435456556566",
      "accessKeyId":"AOFGPJFIJFFOIJFIOJHF",
      "sessionContext":{
         "attributes":{
            "mfaAuthenticated":"false",
            "creationDate":"2017-10-02T20:08:54.107Z"
         },
         "sessionIssuer":{
            "type":"Role",
            "principalId":"ADFDDDFF7FDF7GFFF2DF0",
            "arn":"arn:aws:iam::435456556566:role/ecsInstanceRole",
            "accountId":"435456556566",
            "userName":"kevin"
         }
      }
   },
   "eventTime":"2017-10-02T20:08:54.107Z",
   "eventSource":"ecs.amazonaws.com",
   "eventName":"RegisterTaskDefinition",
   "awsRegion":"us-west-1",
   "sourceIPAddress":"73.168.34.72",
   "userAgent":"Amazon ECS Agent - v1.12.2 (ecda8a6) (+http://aws.amazon.com/ecs/)",
   "requestParameters":{
      "attributes":[
         {
            "name":"com.amazonaws.ecs.capability.privileged-container"
         },
         {
            "name":"com.amazonaws.ecs.capability.docker-remote-api.1.17"
         },
         {
            "name":"com.amazonaws.ecs.capability.docker-remote-api.1.18"
         },
         {
            "name":"com.amazonaws.ecs.capability.docker-remote-api.1.19"
         },
         {
            "name":"com.amazonaws.ecs.capability.docker-remote-api.1.20"
         },
         {
            "name":"com.amazonaws.ecs.capability.docker-remote-api.1.21"
         },
         {
            "name":"com.amazonaws.ecs.capability.docker-remote-api.1.22"
         },
         {
            "name":"com.amazonaws.ecs.capability.logging-driver.json-file"
         },
         {
            "name":"com.amazonaws.ecs.capability.logging-driver.syslog"
         },
         {
            "name":"com.amazonaws.ecs.capability.logging-driver.awslogs"
         },
         {
            "name":"com.amazonaws.ecs.capability.ecr-auth"
         },
         {
            "name":"com.amazonaws.ecs.capability.task-iam-role"
         },
         {
            "name":"com.amazonaws.ecs.capability.task-iam-role-network-host"
         }
      ],
      "totalResources":[
         {
            "type":"INTEGER",
            "doubleValue":0.0,
            "integerValue":1024,
            "longValue":0,
            "name":"CPU"
         },
         {
            "type":"INTEGER",
            "doubleValue":0.0,
            "integerValue":995,
            "longValue":0,
            "name":"MEMORY"
         },
         {
            "type":"STRINGSET",
            "stringSetValue":[
               "22",
               "2375",
               "2376",
               "51678",
               "51679"
            ],
            "doubleValue":0.0,
            "integerValue":0,
            "longValue":0,
            "name":"PORTS"
         },
         {
            "type":"STRINGSET",
            "stringSetValue":[ ],
            "doubleValue":0.0,
            "integerValue":0,
            "longValue":0,
            "name":"PORTS_UDP"
         }
      ],
      "instanceIdentityDocumentSignature":"pqWe1trtreertermhC6vz\nZ0e/ZyOVVKXOb0fiiouyuyturtyreuFaoghqQ0wWurXzcHb6CrtreyteV6hPM=",
      "cluster":"graphite",
      "instanceIdentityDocument":"{\n  \"privateIp\" : \"10.0.1.83\",\n  \"devpayProductCodes\" : null,\n  \"availabilityZone\" : \"us-west-1c\",\n  \"accountId\" : \"435456556566\",\n  \"version\" : \"2010-08-31\",\n  \"instanceId\" : \"i-76vfa923\",\n  \"billingProducts\" : null,\n  \"instanceType\" : \"t2.micro\",\n  \"imageId\" : \"ami-444d0224\",\n  \"pendingTime\" : \"2016-11-15T21:07:08Z\",\n  \"architecture\" : \"x86_64\",\n  \"kernelId\" : null,\n  \"ramdiskId\" : null,\n  \"region\" : \"us-west-1\"\n}"
   },
   "responseElements":{
      "containerInstance":{
         "versionInfo":{ },
         "runningTasksCount":0,
         "ec2InstanceId":"i-13dcar4566",
         "remainingResources":[
            {
               "type":"INTEGER",
               "doubleValue":0.0,
               "integerValue":1024,
               "longValue":0,
               "name":"CPU"
            },
            {
               "type":"INTEGER",
               "doubleValue":0.0,
               "integerValue":995,
               "longValue":0,
               "name":"MEMORY"
            },
            {
               "type":"STRINGSET",
               "stringSetValue":[
                  "22",
                  "2376",
                  "2375",
                  "51678",
                  "51679"
               ],
               "doubleValue":0.0,
               "integerValue":0,
               "longValue":0,
               "name":"PORTS"
            },
            {
               "type":"STRINGSET",
               "stringSetValue":[ ],
               "doubleValue":0.0,
               "integerValue":0,
               "longValue":0,
               "name":"PORTS_UDP"
            }
         ],
         "agentConnected":true,
         "pendingTasksCount":0,
         "registeredResources":[
            {
               "type":"INTEGER",
               "doubleValue":0.0,
               "integerValue":1024,
               "longValue":0,
               "name":"CPU"
            },
            {
               "type":"INTEGER",
               "doubleValue":0.0,
               "integerValue":995,
               "longValue":0,
               "name":"MEMORY"
            },
            {
               "type":"STRINGSET",
               "stringSetValue":[
                  "22",
                  "2376",
                  "2375",
                  "51678",
                  "51679"
               ],
               "doubleValue":0.0,
               "integerValue":0,
               "longValue":0,
               "name":"PORTS"
            },
            {
               "type":"STRINGSET",
               "stringSetValue":[ ],
               "doubleValue":0.0,
               "integerValue":0,
               "longValue":0,
               "name":"PORTS_UDP"
            }
         ],
         "containerInstanceArn":"arn:aws:ecs:us-west-1:435456556566:container-instance/3f28c319-u9n2-1476-3d2n-b7c254fv411",
         "attributes":[
            {
               "name":"com.amazonaws.ecs.capability.privileged-container"
            },
            {
               "name":"com.amazonaws.ecs.capability.docker-remote-api.1.17"
            },
            {
               "name":"com.amazonaws.ecs.capability.docker-remote-api.1.18"
            },
            {
               "name":"com.amazonaws.ecs.capability.docker-remote-api.1.19"
            },
            {
               "name":"com.amazonaws.ecs.capability.docker-remote-api.1.20"
            },
            {
               "name":"com.amazonaws.ecs.capability.docker-remote-api.1.21"
            },
            {
               "name":"com.amazonaws.ecs.capability.docker-remote-api.1.22"
            },
            {
               "name":"com.amazonaws.ecs.capability.logging-driver.json-file"
            },
            {
               "name":"com.amazonaws.ecs.capability.logging-driver.syslog"
            },
            {
               "name":"com.amazonaws.ecs.capability.logging-driver.awslogs"
            },
            {
               "name":"com.amazonaws.ecs.capability.ecr-auth"
            },
            {
               "name":"com.amazonaws.ecs.capability.task-iam-role"
            },
            {
               "name":"com.amazonaws.ecs.capability.task-iam-role-network-host"
            }
         ],
         "status":"ACTIVE",
         "version":1
      }
   },
   "requestID":"ae86b372-ab77-11e6-824c-c7c4220f0423",
   "eventID":"ff9fc985-1fbe-4717-965b-607dda32f620",
   "eventType":"AwsApiCall",
   "recipientAccountId":"435456556566"
}

Sample queries

Deleted Resources Over Time
_sourceCategory=ecs* (DeleteCluster or DeleteService or DeregisterContainerInstance or DeregisterTaskDefinition or StopTask) and !(InternalFailure)
| json "eventName" as event_name
| parse "\"userName\":\"*\"" as user
| parse "\"awsRegion\":\"*\"" as region
| parse "\"cluster\":\"*\"" as cluster
| timeslice 1h
| parse regex field=event_name "^(?:Delete|Deregister|Stop)(?<resource_type>[A-Z][A-Za-z]+)"
| count by resource_type, _timeslice
| transpose row _timeslice column resource_type

Collecting logs and metrics for Amazon ECS

Configure Hosted Collector

When you create an AWS Source, you'll need to identify the Hosted Collector you want to use or create a new Hosted Collector. Once you create an AWS Source, associate it with a Hosted Collector. For instructions, see Configure a Hosted Collector and Source.

Collect Amazon ECS CloudWatch metrics

Sumo Logic supports collecting metrics using one of the following source types:

Configure an AWS Kinesis Firehose for Metrics Source (recommended)
Configure an Amazon CloudWatch Source for Metrics

note
Namespace for Amazon ECS service is AWS/ECS.

Follow the steps below to add custom metadata fields with your metrics:

Click +Add Field under Metadata. Each field consists of a name (key) and a corresponding value.
Create a field named account and assign it a value that represents a friendly name or alias to your AWS account from which metrics are collected. This value will appear in the AWS Observability view, and metrics can be queried using the account field.
After adding fields, check their status indicators:
- A green check mark indicates the field exists and is enabled in the Fields table schema.
- An orange exclamation icon indicates the field does not exist or is disabled in the schema.
  - You will have the option to automatically add or enable the field.
  - If a field is sent but not present or enabled in the schema, it is ignored and marked as Dropped.

Collect Amazon ECS CloudTrail logs

Grant Sumo Logic access to an Amazon S3 bucket.
Create a trail for your AWS account.
Confirm that logs are being delivered to the Amazon S3 bucket.

note
Namespace for Amazon ECS service is AWS/ECS.

Follow the steps below to collect logs for Amazon ECS:

Configure a CloudTrail Logs Source.
Add custom metadata fields with your logs:
1. Click +Add Field under Metadata. Each field consists of a name (key) and a corresponding value.
2. Create a field named account and assign it a value that represents a friendly name or alias to your AWS account from which logs are collected. This value will appear in the AWS Observability view, and logs can be queried using the account field.
3. After adding fields, check their status indicators:
  - A green check mark indicates the field exists and is enabled in the Fields table schema.
  - An orange exclamation icon indicates the field does not exist or is disabled in the schema.
    - You will have the option to automatically add or enable the field.
    - If a field is sent but not present or enabled in the schema, it is ignored and marked as Dropped.

Centralized AWS CloudTrail log collection

In case you have a centralized collection of CloudTraillogs and are ingesting them from all accounts into a single Sumo Logic CloudTraillog source, create following Field Extraction Rule to map proper AWS account(s) friendly name/alias. Create it if not already present / update it as required.

Rule Name: AWS Accounts
Applied at: Ingest Time
Scope (Specific Data):
_sourceCategory=aws/observability/cloudtrail/logs

Parse Expression

Enter a parse expression to create an “account” field that maps to the alias you set for each sub-account. For example, if you used the “dev” alias for an AWS account with ID "528560886094" and the “prod” alias for an AWS account with ID "567680881046", your parse expression would look like this:

| json "recipientAccountId"
// Manually map your aws account id with the AWS account alias you setup earlier for individual child account
| "" as account
| if (recipientAccountId = "528560886094",  "dev", account) as account
| if (recipientAccountId = "567680881046",  "prod", account) as account
| fields account

Installing the Amazon ECS app

Now that you have set up collection for Amazon ECS, install the Sumo Logic app for Amazon ECS to use the pre-configured searches and dashboards that provide visibility into your environment for real-time analysis of overall usage.

To install the app, do the following:

note

Next-Gen App: To install or update the app, you must be an account administrator or a user with Manage Apps, Manage Monitors, Manage Fields, Manage Metric Rules, and Manage Collectors capabilities depending upon the different content types part of the app.

Select App Catalog.
In the 🔎 Search Apps field, run a search for your desired app, then select it.
Click Install App.
note
Sometimes this button says Add Integration.
Click Next in the Setup Data section.
In the Configure section of your respective app, complete the following fields.
1. Field Name. If you already have collectors and sources set up, select the configured metadata field name (eg _sourcecategory) or specify other custom metadata (eg: _collector) along with its metadata Field Value.
Click Next. You will be redirected to the Preview & Done section.

Post-installation

Once your app is installed, it will appear in your Installed Apps folder, and dashboard panels will start to fill automatically.

Each panel slowly fills with data matching the time range query received since the panel was created. Results will not immediately be available but will be updated with full graphs and charts over time.

As part of the app installation process, the following content will be created by default along with dashboards and monitor template:

Fields

account Name / alias to the AWS account.
accountid AWS account id.
region The region to which the resource name belongs to.
namespace Namespace for Amazon ECS Service is AWS/ECS.
clustername The name of the ECS cluster.

Field Extraction Rule(s)

The FER AwsObservabilityECSCloudTrailLogsFER to extract fields region, namespace, clustername, and accountid will be created as a part of app installation.

Viewing the Amazon ECS app dashboards

All dashboards have a set of filters that you can apply to the entire dashboard. Use these filters to drill down and examine the data to a granular level.

You can change the time range for a dashboard or panel by selecting a predefined interval from a drop-down list, choosing a recently used time range, or specifying custom dates and times. Learn more.
You can use template variables to drill down and examine the data on a granular level. For more information, see Filtering Dashboards with Template Variables.
Most Next-Gen apps allow you to provide the scope at the installation time and are comprised of a key (_sourceCategory by default) and a default value for this key. Based on your input, the app dashboards will be parameterized with a dashboard variable, allowing you to change the dataset queried by all panels. This eliminates the need to create multiple copies of the same dashboard with different queries.

Overview

The Amazon ECS - Overview dashboard provides an overview of CPU and memory utilization across all your ECS clusters and services. You can determine which services are high in utilization and accordingly make decisions for the ECS deployment.

Use this dashboard to:

Quickly determine resource utilization across all your ECS clusters and services so as to provision more capacity or optimize on cost.
Get a quick snapshot of overall ECS cluster health, including CPU and memory utilization and reservation levels.
Identify clusters or services with unusually high or low resource usage or reservation percentages.
View details of individual clusters and services, including their regions and associated accounts.

Audit Events

The Amazon ECS - Audit Events dashboard provides insights into changes to your ECS environment including top IAM users, locations of events. The dashboard also shows the created, updated, and deleted events with respect to time, along with the details for the top 10 AWS Identity and Access Management users, and the last 20 Container Registration and Deregistration Events.

Use this dashboard to:

Quickly identify all changes to your ECS environment.
Monitor locations from which changes are being made locations.
Examine details and trends for created, updated and deleted ECS resources.
Investigate specific container registration and deregistration events in different regions and clusters.

Resource Utilization

The Amazon ECS - Resource Utilization dashboard provides trends around CPU and Memory utilization for clusters and services.

Cluster CPU or Cluster memory utilization metrics are only used for tasks using the EC2 launch type.
Service CPU or service memory utilization metrics are used for tasks using both the Fargate and the EC2 launch type.

Use this dashboard to:

Monitor real-time CPU and memory usage across your ECS clusters and services.
Identify performance bottlenecks or underutilized resources in your ECS environment.
Compare utilization patterns between clusters and individual services to optimize resource allocation.

Resource Reservation

The Amazon ECS - Resource Reservation dashboard provides detailed insights into the average reservation (units utilized) by CPU, Memory, and GPU for a given cluster.

These metrics are available for clusters only.
This metric is used only on clusters with tasks or services using the EC2 launch type. It's not supported on clusters with tasks using the Fargate launch type.

Use this dashboard to:

Track average resource reservation levels across different ECS clusters and services.
Identify potential resource constraints or overprovisioning in your ECS environment.
Compare reservation patterns between different types of resources (CPU, memory, GPU) over time.

Create monitors for Amazon ECS app

From your App Catalog:

From the Sumo Logic navigation, select App Catalog.
In the Search Apps field, search for and then select your app.
Make sure the app is installed.
Navigate to What's Included tab and scroll down to the Monitors section.
Click Create next to the pre-configured monitors. In the create monitors window, adjust the trigger conditions and notifications settings based on your requirements.
Scroll down to Monitor Details.
Under Location click on New Folder.
note
By default, monitor will be saved in the root folder. So to make the maintenance easier, create a new folder in the location of your choice.
Enter Folder Name. Folder Description is optional.
tip
Using app version in the folder name will be helpful to determine the versioning for future updates.
Click Create. Once the folder is created, click on Save.

Amazon ECS alerts

Name	Description	Alert Condition	Recover Condition
`Amazon ECS - High CPU Utilization`	This alert fires when the average CPU utilization within a 5 minute interval for a service within a cluster is high (>=85%).	Count > = 85	Count < 85
`Amazon ECS - High Memory Utilization`	This alert fires when the average memory utilization within a 5 minute interval for a service within a cluster is high (>=85%).	Count > = 85	Count < 85

Upgrade/Downgrade the Amazon ECS (Without Container Insights and Traces) app (Optional)

To update the app, do the following:

note

Select App Catalog.
In the Search Apps field, search for and then select your app.
Optionally, you can identify apps that can be upgraded in the Upgrade available section.
To upgrade the app, select Upgrade from the Manage dropdown.
1. If the upgrade does not have any configuration or property changes, you will be redirected to the Preview & Done section.
2. If the upgrade has any configuration or property changes, you will be redirected to the Setup Data page.
3. In the Configure section of your respective app, complete the following fields.
  1. Field Name. If you already have collectors and sources set up, select the configured metadata field name (eg _sourcecategory) or specify other custom metadata (eg: _collector) along with its metadata Field Value.
4. Click Next. You will be redirected to the Preview & Done section.

Post-update

Your upgraded app will be installed in the Installed Apps folder and dashboard panels will start to fill automatically.

note

See our Release Notes changelog for new updates in the app.

To revert the app to a previous version, do the following:

Select App Catalog.
In the Search Apps field, search for and then select your app.
To version down the app, select Revert to < previous version of your app > from the Manage dropdown.

Uninstalling the Amazon ECS (Without Container Insights and Traces) app (Optional)

To uninstall the app, do the following:

Select App Catalog.
In the 🔎 Search Apps field, run a search for your desired app, then select it.
Click Uninstall.

Log and metric types​

Sample log messages​

Sample queries​

Collecting logs and metrics for Amazon ECS​

Configure Hosted Collector​

Collect Amazon ECS CloudWatch metrics​

Collect Amazon ECS CloudTrail logs​

Centralized AWS CloudTrail log collection​

Parse Expression​

Installing the Amazon ECS app​

Post-installation​

Fields​

Field Extraction Rule(s)​

Viewing the Amazon ECS app dashboards​

Overview​

Audit Events​

Resource Utilization​

Resource Reservation​

Create monitors for Amazon ECS app​

Amazon ECS alerts​

Upgrade/Downgrade the Amazon ECS (Without Container Insights and Traces) app (Optional)​

Uninstalling the Amazon ECS (Without Container Insights and Traces) app (Optional)​

Log and metric types

Sample log messages

Sample queries

Collecting logs and metrics for Amazon ECS

Configure Hosted Collector

Collect Amazon ECS CloudWatch metrics

Collect Amazon ECS CloudTrail logs

Centralized AWS CloudTrail log collection

Parse Expression

Installing the Amazon ECS app

Post-installation

Fields

Field Extraction Rule(s)

Viewing the Amazon ECS app dashboards

Overview

Audit Events

Resource Utilization

Resource Reservation

Create monitors for Amazon ECS app

Amazon ECS alerts

Upgrade/Downgrade the Amazon ECS (Without Container Insights and Traces) app (Optional)

Uninstalling the Amazon ECS (Without Container Insights and Traces) app (Optional)