---
id: collect-with-lambda-function
title: Collect Amazon CloudWatch Logs using a Lambda Function
sidebar_label: Collect Logs using a Lambda Function
description: Learn how to collect Amazon CloudWatch Logs with a Lambda function.
slug: /help/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/collect-with-lambda-function/
canonical: https://www.sumologic.com/help/docs/send-data/collect-from-other-data-sources/amazon-cloudwatch-logs/collect-with-lambda-function/
---

import useBaseUrl from '@docusaurus/useBaseUrl';

:::note
We strongly recommend the alternative collection process described on [AWS Kinesis Firehose for Logs Source](/docs/send-data/hosted-collectors/amazon-aws/aws-kinesis-firehose-logs-source/), which is more robust and reliable, where you do not need to manage resources. Lambda based collection methods are limited by time out, concurrency, and memory limits.
:::

:::warning
This method is highly not recommended as it does not handle failures and may lead to data loss. Additionally, the data format generated by this method might be incompatible with some applications in our **App Catalog** and thus requires customization of those apps.
:::

The following instructions describe how to download and configure an AWS Lambda function for Amazon CloudWatch Logs and send them to Sumo Logic.

## Step 1: Add a Hosted Collector and HTTP Source

1. Configure a [Hosted Collector](/docs/send-data/hosted-collectors/configure-hosted-collector).
1. Configure an [HTTP Source](/docs/send-data/hosted-collectors/http-source/logs-metrics).

:::note
When you configure the HTTP Source, make sure to save the HTTP Source Address URL. You will need this to configure the Lambda Function.  
:::

## Step 2: Create Lambda function

Sumo provides a Lambda function for use with Amazon Web Services (AWS). It collects AWS Lambda logs using CloudWatch Logs and it extracts and adds a `RequestId` field to each log line to make correlation easier. 

To add an Amazon Lambda function:
1. Sign in to the [AWS Lambda Console](https://console.aws.amazon.com/lambda/home#/functions).
1. On the **Functions** page, click **Create a Function**.
1. On the **Create Functions** page, select **Use a blueprint** option.
1. In the **Basic information** section, enter the following.<br/><img src={useBaseUrl('img/send-data/basic-information.png')} style={{border: '1px solid gray'}} alt="Basic information" width="700"/>
    * **Blueprint name**. Enter `sumologic` and select **Send Cloudwatch logs to Sumologic**.
    * **Function name**. Enter a name for the function. Default is `sumologic-cwl-lambda`.
    * **Role**. Choose one of the following options.
      * **Use an existing role.** If you have any appropriate roles, you can select one.
      * **Create a new role with basic Lambda permissions**. If you select this option, you can continue without choosing any policy templates—it will create a role with basic Lambda execution privileges by default.
      * **Create a new role from AWS policy templates**.
        * **Role Name**. Enter a name for the role.
        * **Policy templates**. Choose the template and make sure that the policy has following permissions `logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents` so that Lambda can push the logs to Cloudwatch. 
1. In the **CloudWatch Logs trigger** section, you can create a trigger, or click **Remove** if you prefer to create it later in [Step 3](#step-3-assign-cloudwatch-log-groups-to-your-lambda-function). To create the trigger:<br/><img src={useBaseUrl('img/send-data/cloudWatch-logs-trigger.png')} style={{border: '1px solid gray'}} alt="CloudWatch logs trigger" width="700"/>
    * **Log Group**. Select the log group that serves as the event source. Events sent to the log source will trigger your Lambda function. 
    * **Filter Name**. Enter a filter name.
    * **Filter Pattern (Optional)**. May be left blank. For information about AWS filter patterns, see [Filter and Pattern Syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html) in AWS Help.
1. In the **Environment Variables** section, set the value of the variable to the URL of the HTTP source from [Step 1](#step-1-add-a-hosted-collector-and-http-source) to which your logs will be sent. In addition, you can set any of the following optional variables:<br/><img src={useBaseUrl('img/send-data/environment-variables.png')} style={{border: '1px solid gray'}} alt="Environment variables" width="700"/>
    * **ENCODING (Optional)**. Use when decoding CloudWatch log events, default is **utf-1**.
    * **SOURCE_CATEGORY_OVERRIDE (Optional)**. Override `_sourceCategory` value configured for the HTTP source.  
    * **SOURCE_HOST_OVERRIDE (Optional)**. Override `_sourceHost` value configured for the HTTP source.
    * **SOURCE_NAME_OVERRIDE (Optional)**. Override `_sourceName` value configured for the HTTP source.
1. Click **Create Function.** A success message will be displayed.<br/><img src={useBaseUrl('img/send-data/success.png')} style={{border: '1px solid gray'}} alt="success" width="700"/>

:::note
This solution only ingests log messages to Sumo Logic. If you want to include log levels (such as INFO, DEBUG, etc.) to Sumo Logic, find and update the regex in [Lambda code](https://github.com/SumoLogic/sumologic-aws-lambda/blob/main/cloudwatchlogs/cloudwatchlogs_lambda.js#L32) deployed in your AWS account with 
<br/>`var consoleFormatRegex = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}.\d{3}Z\s(\w+?-\w+?-\w+?-\w+?-\w+)\s/;`
:::

## Step 3: Assign CloudWatch Log Groups to Your Lambda Function

:::info
You will need at least one CloudWatch Log Group to assign to your Lambda function. For details on how to create a CloudWatch Log Group, see [Create a CloudWatch Log Group](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Create-Log-Group.html).
:::

:::note
If you have more than one CloudWatch Log Group to assign to your Lambda function, repeat the below steps for each Log Group. Otherwise, skip this step.
:::

1. Sign in to the [AWS Lambda Console](https://console.aws.amazon.com/lambda/home#/functions).
1. On the **Functions** page, select the Lambda function created in [Step 2](#step-2-create-lambda-function).
1. In the **Function overview** section, click the **+ Add trigger** button.<br/><img src={useBaseUrl('img/send-data/add-trigger.png')} style={{border: '1px solid gray'}} alt="Add trigger" width="700"/>
1. In the **Add Trigger** prompt, select **CloudWatch Logs** from the **Select a source** dropdown.<br/><img src={useBaseUrl('img/send-data/select-a-source.png')} style={{border: '1px solid gray'}} alt="Select a source" width="700"/>
1. Select a **CloudWatch Log Group** to add to your function. You need at least one CloudWatch Log Group to see this option.
1. Add a **Filter Name** to your trigger.
1. (Optional) Add a **Filter Pattern** to your trigger. For information about AWS filter patterns, see [Filter and Pattern Syntax](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html) in the AWS documentation.
1. Click **Add** to add the trigger to your Lambda function.

:::note
Sample CloudWatch log ingested for Amazon RDS PostgreSQL using Lambda log forwarder. 
```json 
{
    "timestamp":1705908605000,
    "message":"2024-01-22 07:30:05 UTC::@:[561]:LOG:  checkpoint complete: wrote 0 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.001 s, sync=0.001 s, total=0.002 s; sync files=0, longest=0.000 s, average=0.000 s; distance=0 kB, estimate=0 kB"
}
```
:::