CloudFormation and Sumo Logic - Build Monitoring into your Stack

Curious about Infrastructure as Code (IaC)? Whether you're new to AWS CloudFormation, or you control all of your cloud infrastructure through CloudFormation templates, this post demonstrates how to integrate Sumo Logic's monitoring platform into an AWS CloudFormation stack.

Collect Logs and Metrics from your Stack

Sumo Logic's ability to Unify your Logs and Metrics can be built into your CloudFormation Templates. Collect operating system logs, web server logs, application logs, and other logs from an EC2 instance. Additionally, Host Metrics, AWS CloudWatch Metrics, and Graphite formatted metrics can be collected and analyzed.With CloudFormation and Sumo Logic, you can achieve version control of your AWS infrastructure and your monitoring platform the same way you version and improve your software.

CloudFormation Wordpress Stack with Sumo Logic Built-In

Building off of the resources Adrian Cantrill provided in his Advanced CloudFormation course via A Cloud Guru, we will launch a test Wordpress stack with the following components:

• Linux EC2 instance - you choose the size!
• RDS instance - again, with a configurable size
• S3 bucket

The Linux EC2 instance is bootstrapped with the following to create a LAMP stack:

• Apache
• MySQL
• PHP
• MySQL-PHP Libraries

We also install Wordpress, and the latest version of the Sumo Logic Linux collector agent. Using the cfn-init script in our template, we rely on the file key of AWS::CloudFormation::Init metadata to install a sources.json file on the instance. This file instructs Sumo Logic to collect various types of logs and metrics from the EC2 instance:

• Linux OS Logs (Audit logs, Messages logs, Secure logs)
• Host Metrics (CPU, Memory, TCP, Network, Disk)
• Apache Access logs
• cfn-init logs

Tutorial - Launch a CloudFormation Stack and Monitor Logs and Metrics Instantly

First, you'll need a few things:

• A Sumo Logic account - Get a free one Here
• Access to an AWS account - If you don't have access you can sign up for the free tier here
• A local EC2 Key Pair - if you don't have one you can create one like this

After you have access to your Sumo Logic account and an AWS account, navigate to an unused Region if you have one. This will give you a more isolated sandbox to test in so that we can more clearly see what our CloudFormation template creates. Make sure you have an EC2 key pair in that Region, you'll need to add this to the template.*Leveraging pseudo parameters, the template is portable, meaning it can be launched in any Region.

1. First, log into AWS and navigate to CloudFormation. Choose 'Create New Stack'
2. Then, download the example CloudFormation template from GitHub here
3. Next, on line 87, in the EC2 Resources section, make sure to edit the value of the "KeyName" field to whatever your EC2 key is named for your current Region*Make sure the Region you choose to launch the stack in has an EC2 Key Pair, and that you update line 87 with your key's name. If you forget to do this your stack will fail to launch!
4. Select 'Choose File' and upload the template you just downloaded and edited, then click Next
5. Title your stack
6. Log into Sumo Logic. and in the top-right click on your email username, then preferences, then '+' to create a Sumo Logic Access key pair
7. Enter the Sumo Logic key pair into the stack details page. You can also select an EC2 and RDS instance size, and enter a test string that we can navigate to later when checking that we can communicate with the instance.
8. Click 'Next', name/tag your stack if you'd like, then click 'Next' again, the select 'Create' to launch your stack!

Now What? View Streaming Logs and Metrics!

You've now launched your stack. In about 10-15 minutes, we can visit our Wordpress server to verify everything is working. We can also search our Apache logs and see any visitors (probably just us) that are interacting with the instance. Follow these steps to explore your new stack, and your Sumo Logic analytics:

1. View the CloudFormation Events log. You should see four CREATE_COMPLETE statuses like so:
2. Check your Sumo Logic account to see the collector and sources that have been automatically provisioned for you:

What's Next?

Sumo Logic collects AWS CloudWatch metrics, S3 Audit logs, and much more. Below is more information on the integrations for AWS RDS Metrics and also S3 Audit Logs:

• Amazon RDS Metrics
• Amazon S3 Audit

Explore your logs! Try visiting your web server by navigating to your EC2 instance's public IP address

• This template uses the default security group of your Region's VPC, so you'll need to temporarily allow inbound HTTP traffic from either your IP, or anywhere (your IP is recommended)
• To do this, navigate to the EC2 console and select the Linux machine launched via the CloudFormation Template
• Then, scroll down to the Security Group and click 'default' as shown below
• Edit the inbound rules to allow HTTP traffic in, either from your IP or anywhere
• After you've allowed inbound HTTP traffic, navigate in your browser to <your-public-ip>/wordpress (something like 54.149.214.198/wordpress) and you'll see you're new Wordpress front end:
• You can also test the string we entered during setup by navigating to <your-public-ip>/index2.html
• Search you Sumo Logic account with _sourceCategory=test/apache and view your visits to your new Wordpress web server in the logs
• Finally, check out the metrics on your instance by installing the Host Metrics App:

Cleanup

Make sure to delete you stack as shown below, and to remove inbound HTTP rules on your default Security Group.