Sign up for a live Kubernetes or DevSecOps demo

Click here
Ankit Goel

Ankit Goel

Ankit Goel is Solutions Architect at Sumo Logic with 10+ years of experience in designing and architecting applications. He is passionate about Machine Learning and Big Data projects. Ankit graduated from Carnegie Mellon University with a masters degree in Information Systems.

Posts by Ankit Goel

Blog

Improve Alert Visibility and Monitoring with Sumo Logic and Opsgenie

Blog

How to Monitor Azure Services with Sumo Logic

Blog

11 New Google Cloud Platform (GCP) Apps for Continued Multi-Cloud Support

Blog

Monitor AWS Lambda Functions with Sumo Logic

Blog

Monitor DynamoDB with Sumo Logic

What is DynamoDB ? DynamoDB is a fast and flexible NoSQL database service provided by AWS. This cloud based database supports both document and key-value store models. It was internally developed at AWS to address the need for an incrementally scalable, highly-available key-value storage system. Due to its auto scaling, high performance, and flexible data schema – it’s been widely adopted by various industries such as gaming, mobile, ad tech, IoT, and many other applications. Sumo Logic App for DynamoDB Sumo logic recently released an App for Amazon DynamoDB. It’s a unified Log and Metrics App, and collects data from two sources : CloudTrail API calls DynamoDB CloudWatch Metrics App covers following high level use cases: DynamoDB API calls (Create/Update/Delete Table), Geolocation, and User Info How to plan ‘Capacity’ of DynamoDB Capacity : Number of Read/Write per second per table Read/Write Throttle Events Successful and Throttle Requests by Table and Operation Name Latency and Errors User and System Error Count Latency by Table Name and Operation Name Conditional Check Failed Request by Table Name DynamoDB Overview Dashboard Key DynamoDB Performance Metrics Metrics Description Percent of Provisioned Write Consumed This metric tells you percentage of Provisioned Write Capacity consumed by Table. It should stay below 100%, if it exceeds, then DynamoDB can throttle requests. It’s calculated by : (ConsumedWriteCapacityUnits/ProvisionedWriteCapacityUnits) x 100 Percent of Provisioned Read Consumed This metric tells you percentage of Provisioned Read Capacity consumed by Table (s). It should stay below 100% – if it exceeds, then DynamoDB can throttle your requests. It’s calculated by : (ConsumedReadCapacityUnits/ProvisionedReadCapacityUnits) x 100 Read Throttle Events by Table and GSI Requests to DynamoDB that exceed the provisioned read capacity units for a table or a global secondary index. Write Throttle Events by Table and GSI Requests to DynamoDB that exceed the provisioned write capacity units for a table or a global secondary index. These Read/Write Throttle Events should be zero all the time, if it is not then your requests are being throttled by DynamoDB, and you should re-adjust your capacity. As for how much to provision for your table, it depends a lot on your workload. You could start with provisioning to something like 80% of your peaks and then adjust your table capacity depending on how many throttles you receive. Hence monitoring throttle helps you plan your capacity against your workload. Before you decide on how much to adjust capacity, consider the best practices at Consider Workload Uniformity When Adjusting Provisioned Throughput. Throttle Requests by Table and Operation Name Requests to DynamoDB that exceed the provisioned throughput limits on a resource. Number of Successful Requests by Table and Operation Name The number of successful requests (SampleCount) during specified time period User Error Count Number of requests to DynamoDB that generate HTTP 400 status code during the specified time period. An HTTP 400 usually indicates a client-side error such as an invalid combination of parameters, attempting to update a nonexistent table, or an incorrect request signature.To ensure your services are interacting smoothly with DynamoDB, this count should always be Zero. Latency by Table Name Time taken by DynamoDB in Milliseconds to complete processing the request. You should monitor this, and if this crosses normal threshold level or keep increasing, then you should get involved as it can impact the performance of your services. Sumo Logic’s Machine Learning based Outlier detector automatically lets you detect any outlier in latency, and also let you set up Metrics Monitor for alerting. System Error Count by Table and Operation Name Number of requests to DynamoDB that generate HTTP 500 status code during the specified time period. An HTTP 500 usually indicates an internal service error. This count should always be zero if your services are working fine, and if not then you should be immediately involved in debugging services generating 500 error code. Conditional Check Failed Request Count The number of failed attempts to perform conditional writes. The PutItem, UpdateItem, and DeleteItem operations let you provide a logical condition that must evaluate to true before the operation can proceed. If this condition evaluates to false, ConditionalCheckFailedRequests is incremented by one. Unify DynamoDB API calls and Metrics You can use Log overlay feature to identify if there is any correlation between number of API calls being made to a specific table and increased Latency on that table. Metrics Outlier and Alert Metrics outlier feature automatically identifies the data points which are not in normal range. Furthermore, you can configure different knobs available to filter out the noise. For example, here Sumo Logic’s machine learning algorithm automatically detects outlier in ReadThrottleEvents Count. For more info, see here Also, you can set up alerts, and then send an email or web-hook notifications if your metrics crosses a certain threshold. For more info, see here Secure your DynamoDB API calls. You can correlate DynamoDB CloudTrail events with Sumo Logic – CrowdStrike Threat Intel feed to secure your infrastructure from any malicious activity and user. Conclusion With Sumo Logic App for DynamoDB: You can monitor and alert on key dynamoDB metrics. You can detect any outlier in metrics. You can overlay DynamoDB API Calls with metrics to start debugging issues easily. You can find any malicious activities in DynamoDB environment by correlating CloudTrail events with Sumo Logic Threat Intel Offering. What’s next ? If you already have Sumo Logic account then DynamoDB App is available for free to use. If you are new to Sumo Logic, then start by signing up for free account here. Questions Thanks for reading! If you have any questions or comments feel free to reach out via email (ankit@sumologic.com) or LinkedIn

AWS

November 9, 2017

Blog

Analyze Azure Network Watcher Flow Logs with Sumo Logic

Azure Network Watcher Azure Network Watcher is a network performance and diagnostic service which enables you to monitor your Azure Network. This service lets you collect “Network Security Group (NSG) Flow Logs”. NSG flows logs have 5-tuple information (source, destination, Traffic Flow, Traffic : Allowed/Denied) about ingress and egress IP traffic that are either blocked or allowed by the NSG, allowing you to troubleshoot traffic and security issues. NSG flow logs can enabled via Portal, PowerShell and CLI, more info here. Why Integrate and Analyze Azure Network Watcher Flow Logs with Sumo Logic ? Using Sumo Logic’s machine learning algorithm and search capabilities, you can monitor your Azure Network and alert on key metrics to rapidly identify problems and security issues. Sumo Logic App for Azure Network Watcher leverages NSG flow logs to provide real-time visibility and analysis of your Azure Network. It provides preconfigured Dashboards that allow you to monitor inbound traffic, outlier in traffic flow, and denied flows. Furthermore, this data can be co-related with other Sumo Logic App for Azure Web Apps and Audit for more contextual information. Also, Sumo Logic Threat Intelligence feed can give you extra layer of security on the top of your flow logs. Sumo Logic App for Azure Network Watcher comes with following preconfigured dashboards: Network Watcher – Overview This Dashboard provides general information of the NSG flow logs, including Panels that drill-down into queries with NIC, tuple and traffic flow information. The Overview Dashboard gives a good starting point for detecting outlier in denied traffic and geographic hotspots for inbound traffic. Dashboard also allows panels to be filtered by rule name, source/destination IP and port, and other metadata fields. Network Watcher – Overview Source Address Location of Inbound Traffic. Displays geolocation of Inbound Traffic Flow Traffic by Rule Name. Shows the breakdown of all traffic by security rule name set up at NSG level. Denied Traffic per Minute. Shows trend in denied inbound traffic flow per minute. Breakdown of Traffic (Allowed or Denied). Displays traffic breakdown by Allowed or Denied flow. Top 10 Destination Ports. Shows top 10 destination ports in last 24 hours. Flow Traffic by Protocol. Displays trend of traffic by its protocol ( TCP/UDP). Denied Traffic per Hour – Outlier. This panel, using Sumo Logic machine learning Outlier operator, shows any unexpected sequence in denied traffic. Denied Traffic Comparison (Today Vs Yesterday) – Outlier. Compares denied traffic of last 24 hours with previous 24 hours and shows any unexpected difference between two time periods. Get Started with Sumo Logic App for Azure Network Watcher For more info on the App – please visit Sumo Logic for Azure Network Watcher. To set up the App, follow Collect Logs for Azure Network Watcher and Install the Azure Network Watcher App section at Azure App page

Blog

Integrate Azure Functions with Sumo Logic Schedule Search