Sign up for a live Kubernetes or DevSecOps demo

Click here
Ben Newton

Ben Newton

Ben is a veteran of the IT Operations market, with a two decade career across large and small companies like Loudcloud, BladeLogic, Northrop Grumman, EDS, and BMC. Ben got to do DevOps before DevOps was cool, working with government agencies and major commercial brands to be more agile and move faster. More recently, Ben spent 5 years in product management at Sumo Logic, and is now running product marketing for Operations Analytics at Sumo Logic. His latest project, Masters of Data, has let him combine his love of podcasts and music with his love of good conversations.

Posts by Ben Newton

Blog

Context is Everything - How SPS Commerce uses context to embrace complexity

Blog

Reimagining Observability: Announcing Sumo Logic’s new Kubernetes Solution

Blog

Recycling is for Cardboard, not Analytics Tools

Blog

Announcing the Sumo Logic Global Intelligence Service at Illuminate 2018

In today’s hyper-connected world, a company’s differentiation is completely dependent upon delivering a better customer experience, at scale, and at a lower cost than the competition. This is no easy feat, and involves a combination of many things, particularly adopting new technologies and architectures, as well as making better use of data and analytics. Sumo Logic is committed to helping our customers excel in this challenging environment by making it easier to adopt the latest application architectures while also making the most of their precious data. The Power of the Platform As a multi-tenant cloud-native platform, Sumo Logic has a unique opportunity to provide context and data to our customers that is not available anywhere else. Why is this? First of all, when an enterprise wants to explore new architectures and evaluate options, it is very difficult to find broad overviews of industry trends based on real-time data rather than surveys or guesswork. Second, it is difficult to find reliable information about how exactly companies are using technologies at the implementation level, all the way down to the configurations and performance characteristics. Finally, once implemented, companies struggle to make the best use of the massive amount of machine data exhaust from their applications, particularly for non-traditional audiences like data scientists. It is with this backdrop in mind that Sumo Logic is announcing the Global Intelligence Service today during the keynote presentation at our second annual user conference, Illuminate, in Burlingame, Calif. This unprecedented initiative of data democratization is composed of three primary areas of innovation. Industry Insights — What Trends Should I be Watching? Sumo Logic is continuing to building on the success of its recently released third annual ‘State of Modern Applications and DevSecOps in the Cloud’ report to provide more real-time and actionable insights about industry trends. In order to stay on top of a constantly changing technology landscape, this report is expanding to include more frequent updates and instant-access options to help customers develop the right modern application or cloud migration strategy for their business, operational and security needs. Chart depicting clusters of AWS Services used frequently together Community Insights — What Are Companies like us, and Teams like ours, Doing? Sumo Logic is applying the power of machine learning to derive actionable insights for getting the most out of your technology investments. We have found that many engineering teams lack the right resources and education needed to make the best technology choices early on in their prototyping phases. And then, when the system is in production, it is often too late to make changes. That’s why Sumo Logic has an opportunity to save our customers pain and frustration by giving them benchmarking and comparison information when they most need it. We all like to think that our use cases are each a beautiful, unique snowflake. The reality is that, while each of us is unique, our uses of technology fall into some predictable clusters. So, looking over a customer base of thousands, Sumo Logic can infer patterns and best practices about how similar organizations are using technologies. Uses those patterns, we will be building recommendations and content for our customers that can be used to compare performance against a baseline of usage across their peers. Chart depicting how the performance behavior across customers tend to cluster Data Science Insights — Data Scientists Need Love, Too Data scientists are under more pressure than ever to deliver stunning results, while also getting pushback from society about the quality of their models and the biases that may or may not be there. At the end of the day, while data scientists have control over their models, they may have less control over the data. If the data is incomplete or biased in any way that can directly influence the results. To alleviate this issue, Sumo Logic is providing an open source integration with the industry standard Jupyter and Apache Zeppelin notebooks in order to make it easier for data scientists to leverage the treasure trove of knowledge currently buried in their application machine data. Empower the People who Power Modern Business You may still be wondering, why does all of this matter? At the end of the day, it is all about making our customers successful by making their people successful. A business is only as effective as the people who do the work, and it is our mission at Sumo Logic to empower those users to excel in their roles, which in return contributes to overall company growth and performance. And we also want to set users outside of the traditional IT, DevOps, and security teams up for success as well by making machine data analytics more accessible for them. So, don’t forget that you heard it here first: Democratizing machine data is all about empowering the people with love (and with unique machine data analytics and insights)! Additional Resources Download the 2018 ‘State of Modern Applications and DevSecOps in the Cloud’ report and/or read the press release for more detailed insights. Read the Sumo Logic platform enhancement release to learn more about our latest platform enhancements and innovations Sign up for Sumo Logic for free

Blog

Why Cloud-Native is the Way to Go for Managing Modern Application Data

Are your on-premises analytics and security solutions failing you in today’s digital world? Don’t have the visibility you need across your full application stack? Unable to effectively monitor, troubleshoot and secure your microservices and multi-cloud architectures? If this sounds like your organization, then be sure to watch this short video explaining why a cloud-native, scalable and elastic machine data analytics platform approach is the right answer for building, running and securing your modern applications and cloud infrastructures. To learn more about how Sumo Logic is uniquely positioned to offer development, security and operations (DevSecOps) teams the right tools for their cloud environments, watch our Miles Ahead in the Cloud and DevOps Redemption videos, visit our website or sign up for Sumo Logic for free here. Video Transcription You’ve decided to run your business in the cloud. You chose this to leverage all the benefits the cloud enables – speed to rapidly scale your business; elasticity to handle the buying cycles of your customers; and the ability to offload data center management headaches to someone else so you can focus your time, energy and innovation on building a great customer experience. So, when you need insights into your app to monitor, troubleshoot or learn more about your customers, why would you choose a solution that doesn’t work the same way? Why would you manage your app with a tool that locks you into a peak support contract, one that’s not designed to handle the unpredictability of your data? Sumo Logic is a cloud-native, multi-tenant service that lets you monitor, troubleshoot, and secure your application with the same standards of scalability, elasticity and security you hold yourself to. Sumo Logic is built on a modern app stack for modern app stacks. Its scalable………….elastic…………resilient cloud architecture has the agility to move as fast as your app moves, quickly scaling up for data volume. Its advanced analytics based on machine learning are designed to cope with change. So, when that data volume spikes, Sumo Logic is there with the capacity and the answers you need. Sumo Logic is built with security as a 1st principle. That means security is baked in at the code level, and that the platform has the credentials and attestations you need to manage compliance for your industry. Sumo Logic’s security analytics and integrated threat intelligence also help you detect threats and breaches faster, with no additional costs. Sumo Logic delivers all this value in a single platform solution. No more swivel chair analytics to slow you down or impede your decision-making. You have one place to see and correlate the continuum of operations, security and customer experience analytics – this is what we call continuous intelligence for modern apps. So, don’t try to support your cloud app with a tool that was designed for the old, on-premise world, or a pretend cloud-tool. Leverage the intelligence solution that fully replicates what you’re doing with your own cloud-business — Sumo Logic, the industry leading, cloud-native, machine data analytics platform delivered to you as a service. Sumo Logic. Continuous Intelligence for Modern Applications.

Blog

DevOps Redemption: Don't Let Outdated Data Analytics Tools Slow You Down

Blog

Log Management and Analytics for the AWS ELB Classic Service

Quick Refresher Earlier this year, we showed you how to monitor Amazon Web Services Elastic Load Balancer (AWS ELB) with Cloudwatch. This piece is a follow up to that, and will focus on Classic Load Balancers. Classic Load Balancers provide basic load balancing across multiple Amazon EC2 instances and operate at both the request level and connection level. Classic Load Balancers are intended for applications that were built within the EC2-Classic network. AWS provides the ability to monitor your ELB configuration with detailed logs of all the requests made to your load balancers. There is a wealth of data in the logs generated by ELB, and it is extremely simple to set up. How to Get Started: Setting up AWS ELB Logs Logging is not enabled in AWS ELB by default. It is important to set up logging when you start using the service so you don’t miss any important details! Step 1: Create an S3 Bucket and Enable ELB Logging Note: If you have more than one AWS account (such as ops, dev, and so on) or multiple regions that generate Elastic Load Balancing data, you’ll probably need to configure each of these separately. Here are the key steps you need to follow Create an S3 Bucket to store the logs Note: Want to learn more about S3? Look no further (link) Allow AWS ELB access to the S3 Bucket Enable AWS ELB Logging in the AWS Console Verify that it is working Step 2: Allow Access to external Log Management Tools To add AWS ELB logs to your log management strategy, you need to give access to your log management tool! The easiest way to do that is by creating a special user and policy. Create a user in AWS Identity and Access Management (IAM) with Programmatic Access. For more information about this, refer to the appropriate section of the AWS User Guide. Note: Make sure to store the Access Key ID and Secret Access Key credentials in a secure location. You will need to provide these later to provide access to your tools! Create a Custom Policy for the new IAM user. We recommend you use the following JSON policy: { “Version”:”2012-10-17″, “Statement”:[ { “Action”:[ “s3:GetObject”, “s3:GetObjectVersion”, “s3:ListBucketVersions”, “s3:ListBucket” ], “Effect”:”Allow”, “Resource”:[ “arn:aws:s3:::your_bucketname/*”, “arn:aws:s3:::your_bucketname” ] } ] } Note: All of the Action parameters shown above are required. Replace the “your_bucketname” placeholders in the Resource section of the JSON policy with your actual S3 bucket name. Refer to the Access Policies section of the AWS User Guide for more info. What do the Logs look like? ELB logs are stored as .log files in the S3 buckets you specify when you enable logging. The file names of the access logs use the following format: bucket[/prefix]/AWSLogs/aws-account-id/elasticloadbalancing/region/yyyy/mm/dd/aws-account-id_elasticloadbalancing_region_load-balancer-name_end-time_ip-address_random-string.log bucket The name of the S3 bucket. prefix The prefix (logical hierarchy) in the bucket. If you don’t specify a prefix, the logs are placed at the root level of the bucket. aws-account-id The AWS account ID of the owner. region The region for your load balancer and S3 bucket. yyyy/mm/dd The date that the log was delivered. load-balancer-name The name of the load balancer. end-time The date and time that the logging interval ended. For example, an end time of 20140215T2340Z contains entries for requests made between 23:35 and 23:40 if the publishing interval is 5 minutes. ip-address The IP address of the load balancer node that handled the request. For an internal load balancer, this is a private IP address. random-string A system-generated random string. The following is an example log file name: s3://my-loadbalancer-logs/my-app/AWSLogs/123456789012/elasticloadbalancing/us-west-2/2014/02/15/123456789012_elasticloadbalancing_us-west-2_my-loadbalancer_20140215T2340Z_172.160.001.192_20sg8hgm.log Syntax Each log entry contains the details of a single request made to the load balancer. All fields in the log entry are delimited by spaces. Each entry in the log file has the following format: timestamp elb client:port backend:port request_processing_time backend_processing_time response_processing_time elb_status_code backend_status_code received_bytes sent_bytes “request” “user_agent” ssl_cipher ssl_protocol The following table explains the different fields in the log file. Note: ELB can process HTTP requests and TCP requests, and the differences are noted below: Field Description timestamp The time when the load balancer received the request from the client, in ISO 8601 format. elb The name of the load balancer client:port The IP address and port of the requesting client. backend:port The IP address and port of the registered instance that processed this request. request_processing_time [HTTP listener] The total time elapsed, in seconds, from the time the load balancer received the request until the time it sent it to a registered instance.

AWS

June 19, 2018

Blog

Join the Data Revolution - Listen to our New Masters of Data Podcast

In today’s world, we are surrounded by data. It’s flying through the air all around us over radio signals, wireless internet, mobile networks and more. We’re so immersed in data every day that we rarely stop to think about what the data means and why it is there. I, for one, rarely have a quiet moment where I am not directly, or indirectly, absorbing a flow of information, regardless of whether I want to or not. So, I wonder, are we going to master this all-encompassing data, or be mastered by it? Data is the new currency of our world Data has become the lifeblood of the modern business, and those who succeed in today’s competitive landscape are those who leverage data the best. Amazing applications exist that take the raw data flowing out of the exhaust pipe of the modern economy (and our lives) and enable companies to develop products we couldn’t have even conceived of a few years ago. Artificial intelligence-driven innovations are anticipating our next move. Social media products are connecting us and targeting us. Self-driving cars are swimming in data to navigate a world full of faulty humans. But, how often do we stop to talk about the good, the bad and the ugly of data? [epq-quote align="align-right"]"A single conversation across the table with a wise man is better than ten years mere study of books." - Henry Wadsworth Longfellow [/epq-quote] The discussions now about data privacy and the nonstop stream of hackers stealing our personal information, are actually elevating data from a sub-theme of our culture into a primary topic, even for non-experts. The value of data is also rising into the awareness of boardrooms, and this is a good thing. The only way to keep ourselves honest about how we use data is to talk about how we use data — the wonderful and the despicable, the innovative and the regressive. A new podcast to explore the data revolution [epq-quote align="align-left"]The only way to keep ourselves honest about how we use data is to talk about how we use data — the wonderful and the despicable, the innovative and the regressive" - Ben Newton, director of product marketing, Sumo Logic[/epq-quote] As a long-time podcast listener and fan of the spoken word, I am excited to announce a new podcast about this data revolution — Masters of Data. Each episode we interview innovators, big thinkers and provocateurs, to learn theirs views about data. We also want to meet the people behind the data, who can humanize the data by helping us understand the cultural content and intention of the data on human experience. That way we turn this from stories about widgets and gimmicks into stories about humans and the value of data, as well as the dangers of misusing it. Our first podcast is now live and features Bill Burns, the chief trust officer at Informatica. Bill and I take a journey through time and security and discuss the evolving tech landscape over the last 20 years. We talk about how he got started in a computer lab, cut his teeth at Netscape, helped change the world at Netflix, and is on his next journey at Informatica. How to listen and subscribe To listen visit www.mastersofdata.com or subscribe via the iTunes or Google Play app stores. Once you’ve subscribed on your favorite platform, be sure to check back for new episodes and leave us a review to let us know what you think! We will be releasing several discussions over the next few months, and we look forward your feedback! Until then, listen and enjoy. And don’t forget to join the conversation on Twitter #mastersofdata

May 15, 2018

Blog

Monitoring AWS Elastic Load Balancing with Cloudwatch

Quick Refresher – What is AWS Elastic Load Balancing? A key part of any modern application is the ability to spread the load of user requests to your application across multiple resources, which makes it much easier to scale as traffic naturally goes up and down during the day and the week. Amazon Web Services’ answer to load balancing in the cloud is the Elastic Load Balancer (AWS ELB) service – Classic ELB and Application ELB. AWS ELB integrates seamlessly with Amazon’s other cloud services, automatically spinning up new ELB instances without manual intervention to meet high demand periods and scaling them back, in off peak hours to get the most out of your IT budget, while also providing a great experience to your users. AWS provides the ability to monitor your ELB configuration through AWS Cloudwatch with detailed metrics about the requests made to your load balancers. There is a wealth of data in these metrics generated by ELB, and it is extremely simple to set up. And best of all, these metrics are included with the service! Understanding AWS Cloudwatch metrics for AWS ELB First, you need to understand the concept “Namespace”. For every service monitored by AWS Cloudwatch, there is a Namespace dimension that tells you where the data is coming. For each of the three ELB services, there is a corresponding namespace as well. Namespace Namespace Classic Load Balancers AWS/ELB Application Load Balancers AWS/ApplicationELB Network Load Balancers AWS/NetworkELB One of the most important aspects to understand with Cloudwatch metrics are the “dimensions”. Dimensions tell you the identity of what is monitoring – what it is and where it is from. For this type of metric, there are two key dimensions: Dimension Description Availability Zone What Availability Zone the ELB Instance is in LoadBalancerName The name of the ELB instance Note: AWS automatically provides rollup metrics over dimensions as well. So, for example, if you see a measurement with no Load Balancer dimension, but still has an Availability Zone (AZ), that is a rollup over all of the Load Balancers in that AZ. Another part of the metrics are the “Statistic”. Cloudwatch metrics are not raw measurements, but are actually aggregated up to more digestible data volumes. So, in order to not lose the behavior of the underlying data, Cloudwatch provides several statistics which can use depending on what you need: Statistic Description Minimum The minimum value over the reporting period (typically 1 min) Maximum The minimum value over the reporting period (typically 1 min) Sum The sum of all values over the reporting period (typically 1 min) Average The average value over the reporting period (typically 1 min) SampleCount The number of samples over the reporting period (typically 1 min) What are the key metrics to watch? There are a lot of metrics gathered by Cloudwatch, but we can divide those into two main categories: Metrics about the Load Balancer, Metrics about the Backend Instances. We will show you the key ones to watch, and what statistics are appropriate when analyzing the metric. Key performance indicators for the load balancer The key performance indicators (KPIs) will help you understand how the actual ELB instances are performing and how they are interacting with the incoming requests, as opposed to how your backend instances may be responding to the traffic. Metric What it Means and How to Use it Statistics to Use RequestCount This metric tracks the number of requests that the load balancer, or group of load balancers, has received. This is the baseline metric for any kind of traffic analysis, particularly if you don’t have auto-scaling enabled. Sum (other statistics aren’t useful) SurgeQueueLength This tells you the number of inbound requests waiting to be accepted and processed by a backend instance. This can tell you if you need to scale out your backend resources. Maximum is the most useful, but Average and Minimum can be helpful in addition to Maximum. SpilloverCount This is the number of rejected requests because the surge queue is full.

AWS

April 9, 2018

Blog

Don't Fly Blind - Use Machine Data Analytics to Provide the Best Customer Experience

Blog

A Toddler’s Guide to Data Analytics

Any parent of a two-year old appreciates the power of speaking a common language. There is nothing more frustrating to my two-year old son than his inability to communicate what he wants. Learning to say things like “milk” and “applesauce” has transformed the breakfast experience. On the other hand, learning to say “trash” means that any object in reach is in danger of being tossed in the trash bin for the glory of saying “twash” over and over again. I see this in my world world as well. Millions of dollars are spent in the software world translating from one “language” to another. In fact, a whole industry has popped up around shared Application Program Interfaces (APIs) to standardize how systems communicate. Despite this trend, there seems to be more emphasis on the “communication” rather than the “data” itself. Data Analytics products in particular seem happy to shove all types of data into one mold, since the output is the same. That is where we at Sumo Logic decided to take the road less travelled. We believe in the idea that data needs to be treated “natively” – in other words, we don’t want to shove a square data peg in a round systems hole. Just like speaking a language natively changes the experience of travel – speaking the native language of data transforms the analytics experience. When we decided to build our Unified Logs and Metrics (ULM) product, we also decided that it was essential that we become bi-lingual – speaking the native language of both raw logs and time-series metrics. And here is why it matters – according to my Toddler. Answer my Question – Quickly Toddlers are well acquainted with the frustration of not being understood. Everything takes too long, and they need it now. And you know what. I get it. I have had the occasion of speaking to people that don’t speak my language before, and it is hard. I once spent 15 minutes in a pharmacy in Geneva trying to order anti-bacterial cream. There was a lot of waving of hands and short, obvious words (Arm. Cut. Ow!). We would have faced the same obstacles if we used a log system to store metrics. Every query needs to be translated, from one language to another, and it takes forever. At end of the day, you can try optimize a log system – built to search for needles in haystacks – to perform the equivalent of speed reading, but eventually the laws of physics intervene. You can only make it so fast. It takes too long – and like my toddler, you will just stop asking the question. What’s the use in that? Cleaning up is Hard I am always amazed with how my two-year old can turn a nice stack of puzzles or a bucket of toys into a room-sized disaster zone – it is the same components, but vastly different results. Storage optimization is essential in the world of operational data. There is a natural assumption underneath a true log-analytics system. We assume on some level that each log is a special snowflake. There is, of course, a lot of repetition, but the key is to be flexible and optimize for finding key terms very quickly. Metrics, on the hand, are repetitive by design. Every record of a measurement is the same – except for the measurement itself. Once you know you are collecting something – say system CPU performance on some server – you don’t need to capture that reference every time. You can optimize heavily for storing and retrieving long lists of numbers. Storing time series metrics as logs, or events, is extremely wasteful. You can incur any where from 3x to 10x more storage costs – and that is without the same performance. To achieve the same performance as most metrics system can reach, you are looking at 10-20x in storage costs. This, of course, is the reason why no log-analytics companies are really used for performance metrics at scale – the immense costs involved just don’t justify the benefit of tool reduction. I Want to Play with my Cars Anywhere One of the funniest things my son does is how he plays with his toy cars. He has race tracks, roads, and other appropriate surfaces. He rarely uses them. He prefers to race his cars on tables, up walls, and on daddy’s leg. The flexibility of having wheels is essential. He has other “cars” that don’t roll – he doesn’t really play with them. It is the same core truth with data analytics. Once you have high performance with cost effective storage – uses just present themselves. Now you can perform complex analytics without fear of slowing down the system to a crawl. You can compare performance over months and years, rather than minutes and hours – because storage is so much cheaper. Innovative use cases will always fill up the new space created by platform enhancements – just as restricted platforms will always restrict the use cases as well. Choose Wisely So, it’s 2 AM. Your application is down. Your DevOps/Ops/Engineering team is trying to solve the problem. They can either be frustrated that they can’t get their questions answered, or they can breeze through their operational data to get the answers they need. I know what two-year old would tell you to do. Time to put your old approach to data analytics in the twash.

January 10, 2017

Blog

Together Forever - Logs and Metrics

Together forever and never to part Together forever we two And don't you know I would move heaven and earth To be together forever with you Rick Astley There are lots of things in this world that when put together, prove to be more than the sum of their parts. For example, take a favorite dessert of nearby Napa Valley - vanilla ice cream, olive oil and sea salt. The first time you hear it, you might think -- “You are going to pour oil on my ice cream, and then top it off with some salt? Yuck! Thanks for ruining dinner.” Take the simple, but delicious peanut butter and jelly. Many europeans think it's disgusting (this is the same culture that finds snails or marmite delectable, so consider the source.). Yet it is now so American that I have heard on good authority it’s being considered as a requirement for U.S. citizenship (along with eating apple pie and listening to Taylor Swift, of course). So, before you start thinking this is a food blog, let’s get to the point. In the world of IT we have something similar - logs and metrics. Simply put, logs are usually records of something that happened at some point in time (e.g. application errors, warnings, application events, etc.) and metrics are typically measurements of something over time (e.g. the response time of the home page, server CPU, server memory, etc.). In the past, those types of data were often treated very differently. We at Sumo Logic believe that’s a problem, and have worked to solve it. In this blog, I want to dig a little deeper into why logs and metrics go together like peanut butter and jelly. Building the Tesla for Logs & Metrics Once a software company builds a product and starts selling it, it’s very hard to change the skeleton of the thing - the architecture. Switching analogies for a second - consider for a moment that software is similar to cars. When automotive designers are faced with a new problem or scenario, the easiest approach is to tinker with the existing design. This works if you’re adding new tires to an SUV for off-roading. It does not work if you are designing a fully electric car. Tesla completely reworked everything - the transmission, the engine - even the sales model. In the case of systems dealing with machine data, the tinkering approach has definitely not been successful. For example, some log analytics companies, having built engines to make sense of messy, unstructured logs, have also tweaked their mechanics to address highly structured time series data (e.g. metrics). The end result is a lot of hoop jumping to get their engines to perform the way users expect. On the other hand, companies that have chosen to build metrics analysis engines have tried their hand at logs - or at least they say they do. However, in reality, the “logs” passing through the metrics engines are so massaged and the messiness wrung mercilessly out of them, that you end up with something more akin to metrics data. So, this approach may improve engine performance, but breaks down when it comes to accuracy -- all for the sake of expediency. We dealt with this same conundrum at Sumo Logic. We decided to take a big risk and actually evolve our engine design, resulting in a next generation architecture. We built a purpose-built, high-performance engine for time series data, alongside, and integrated with, our high performance log analytics engine. We didn’t forget all of the lessons of the past in building a highly scalable, multi-tenant application -- we just didn’t take shortcuts. The end result is a platform that treats both types of data natively, or, what I consider to be the respect they deserve. It’s all About the Analytics The secret of a great peanut butter and jelly sandwich is not just bringing unique ingredients together, but creating it with the entire sandwich experience in mind. That applied to us as well when we unified logs and metrics -- what’s the point of it all? For example, we aren’t a data warehouse in the sky. People come to Sumo Logic to make sense of their data -- to do analytics. And, the core of analytics is the type of questions asked of the data. Because of data’s nature, some types of data answer certain types of questions better than others. In the case of logs, users often find themselves looking for the proverbial needle in the haystack, or, to be more accurate, hundreds of needles sprinkled across a field of haystacks. So, log analytics has to excel at “x-raying” those haystacks to get to the offending needles very quickly, and even better, detecting patterns in the needles - basically troubleshooting and root cause analysis. Counting the needles, haystacks, or even the hay straw itself is of secondary concern. With metrics, the goal is often to to look for behavior or trends over time. Users rely on time-series metrics to understand how the things they measure change over time, and whether it indicates something deeper beneath the surface. Fitness devices (like the Apple Watch) are a perfect example here. I can track my heart-rate, running speed, distance, etc., over time. This helps me determine if getting out of bed to run outside was remotely worth it. If my stats improve over time, then yes. If no, then I’m sleeping in! At Sumo Logic, we knew we couldn’t afford to ignore the different ways that people use their data. And, simply put, we couldn’t treat metrics like a half-brother of logs. So, we focused on tools that help uncover system and application behavior, not just draw nice looking graphs. Users can overlay related data to find anomalies and patterns in the data. And we strove to make the system as real-time as possible, shaving off microseconds wherever we could so the data is as relevant and timely as possible. Listen to the Users A platform that optimizes for the data and the analytics, but ignores the user, will end up being stiff and hard to use. In building this new functionality at Sumo Logic, we’ve strived to understand our users. That meant investing a lot of time and energy in talking to our users, and listening to their feedback -- and then being willing to change. At the end of the day, that is why we embarked on this journey in the first place. Our customers intuitively understand that having multiple tools for logs and metrics was slowing them down. They have transitioned, or are transitioning, to a world of small, collaborative teams, with cross-functional skill sets, who own segments of their applications. The implication is that the majority of our users aren’t log experts, server experts, or network specialists, but software developers, and/or DevOps teams that support modern applications. They know what they need to measure and to analyze because they write the code and support it in the wild. What they are not interested in is learning the deep intricacies of machine data or advanced analytics. So, after listening to our customers, we embarked on a journey to empower them to explore their data without learning complex search languages or getting a Ph.D. in machine learning. We want our users to be able to lose themselves in the analysis of their data without context-switching from one platform to another, and without diverting time away from their tasks. When it is 2 a.m., and the application is down, learning the theory of statistical analysis is definitely not fun. So, wrapping up, I’ll conclude with this: we are committed to providing the best possible experience for our users -- and that has meant questioning many of our own assumptions about how machine data analytics works. While it might have been easy to recognize that peanut butter and jelly belong together, it takes dedicated hard work and perseverance to get it exactly right. We didn’t do it to prove a point. We did it because our customers, in one way or another, asked us to. They have taken immense risks on their own to ride the cutting edge of innovation in their fields, and they deserve a machine data analytics tool that is right there in the thick of it with them. But we are strong, each in our purpose, and we are all more strong together. ― Van Helsing in Bram Stoker’s Dracula

April 12, 2016

Blog

Why are you treating your data like corn? Silos are for corn, not data

“Toto, I’ve a feeling we’re not in Kansas any more.” Dorothy, The Wizard of Oz We, the software vendors of the world, love to talk about ridding the IT world of silos. So what, may you ask, is a silo anyway? According to the source of all knowledge (Wikipedia, of course), silo comes from the greek word σιρός (siros), which translates to “pit for holding grain.” For most of us, Wikipedia’s definition brings to mind large, metal containers that dot the golden landscapes of states like Idaho or Nebraska. But the term “silos” has also become a great visual analogy for many of the problems with IT and software development, so it’s worth delving beneath the surface to see what we can learn. Why do we need silos? My brother lived in Kansas for a few years and we’ve talked about the ways in which local farmers use grain elevators and silos. It’s amazing to think that farmers from hundreds of miles around bring their grain to these giant grain elevators so the fruit of their labor can be deposited into giant concrete silos. Why does this make sense for farmers? Because their corn is a commodity, i.e.,. all of the corn in that silo is basically the same and sold in bulk just like oil, copper ore, beans, etc. For a long time corporations have done and continue to do something very similar with their data. They create large scale processes to dump related types of data into purpose-built databases. These “data silos” often mirror the organizational structure of the IT organization–network monitoring data goes here, systems monitoring here, etc. Do Data Silos work? Silos serve an important, rather obvious function for the commodities industry. They enable the storage of commodities. No one would put rice in oil or even rice with corn. Silos optimize for costs and efficiency since the value of the commodity is in its volume, or bulk–not an individual kernel of corn. From a corporate perspective, data silos have the opposite effect of siloed corn–they increase costs, decrease efficiency and limit innovation. In the modern corporation, IT culture is rapidly evolving to a shared ownership model removing the organizational inertia to sharing data and ideas. When data is siloed, it actually creates unnecessary friction–hindering collaboration and problem solving across functional boundaries. Conversely, mixing related data can dramatically increase the value of the data by uncovering important patterns and correlating issues across different components. One only needs to look back at the recent past of data warehousing to see a mostly failed attempt to uncover this untapped value. Who benefits from silos anyway? So, who benefits from silos? In the world of commodities, silos empower industrial scale commodities trading and efficiencies. For the agriculture industry, in particular, siloed commodities have led to lower prices for consumers though, one might argue, some reduction in quality. Regardless, silo-based storage has mostly been a net positive. What about data silos? In the world of modern applications, data silos are definitely not a help to application developers and operators. And, they rarely help business decision makers. Most importantly, they don’t help end users or customers. So, who do they help? One clear answer–software vendors. You depend on their products to derive value from the data they house for you. These software products and services either empower you to gain the fullest possible value from your data or hinder and hobble your efforts. In the past, software vendors could sidestep this thorny issue because their interests were aligned with the interests of the traditional organizational structure. No more. The architecture of the modern application (microservices, containers, etc.) demands the free flow of data to connect the dots and form a complete picture. Cross-functional, empowered teams of the Agile/DevOps revolution don’t defer to functional boundaries when solving problems. So, does it make sense to use machine data analytics tools that operate in this same siloed way, holding your team back from delivering value to your customers? Better Together The last 10 years have brought amazing changes to the power of software to deliver business value. The cause of this shift is the symbiotic relationship between powerful cultural changes in how companies organize themselves and get work done along with the tectonic shifts in how business value is delivered over the Internet to customers at massive scale and near real-time speed. Unfortunately, many corporations are still stuck in the world of data silos and losing out on the rich value their data has to offer by treating it like corn. So, it’s time to see how valuable your organization’s data truly is. Don’t let anyone, or any product, stand in your way.

April 5, 2016

Blog

Korean Short Rib Taco or the Blue Plate Special - Three Reasons to Order Data Analytics as a Service with Your Microservices Architecture

One of the current market shifts getting a lot of attention these days is the transition from traditional monolithic application architectures to microservices, the impact of which can be felt from technological innovation to business/team culture. We, at Sumo Logic, believe in the importance of this trend because the same forces driving companies to microservices are also driving them to cloud-based services for supporting capabilities like machine data analytics. Below are three reasons why the benefits of consuming machine data analytics as a service greatly outweighs the old monolithic application delivery model. Focus on the Product, not the Skillsets In recent years, the gourmet food truck has become one of my favorite food trends. The expectation of lines of food trucks all peddling some highly refined comfort food has been a welcome motivation on many family excursions. My wife can get korean-style fried chicken, my daughter a quesadilla, and I can indulge in a completely ridiculous barbeque sandwich with an egg on top. In my mind, microservices are like food trucks while monolithic applications remind me of those cafeterias my grandma dragged us to as kids. Bin after bin of unremarkable, over salted food made by general purpose cooks counting down the hours to clock-out. Sound familiar? Companies today face the same dilemma as the food industry - optimize for predictable mediocrity or allow for creativity, with the higher risk of failure. Customers have made that decision for them - the old way of doing things is not working. What this means is the the modern, digital company has had to move to an organization philosophy focused on business capabilities, not silo-ed, generalized capabilities like database architects and web operations teams. This allows those companies to rapidly develop and experiment with new functionality without toppling the entire edifice, and, most important, make a better quality product in the process. Essentially, create specialization around your product - “korean-style fried chicken”, not platform components - “warmer of canned green beans”. Shared ownership with shared standards Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure. -- Melvyn Conway, 1967 What I have found with food trucks is that their product tends to be highly unique and, at least for the successful ones, highly attuned to the tastes of their users. On the other extreme, the cafeteria is attuned to nobody, and responsive to nobody. In today’s world of opinionated consumers, restaurants have to balance these extremes in order to be successful - the quality of the food truck balanced against the efficient operations of the tightly managed cafeteria. Companies, large and small, are facing this same conundrum - moving from hierarchical decision making that produces middling applications to more dynamic shared-ownership models that can produce unique engaging experiences. For example, traditional, monolithic applications rely on shared components like databases and web services. With microservices, small, cross-functional, multi-skilled teams are expected, and empowered, to maintain stable components with well-known integrations points and APIs, built on common infrastructure components. The quality of the output and the agility of the team is valued above office politics. Consuming your data analytics as a cloud-native service delivers similar benefits to microservices.. The service accepts data from your application in well-understood ways with published methods for processing and analyzing data. And just like with microservices, this allows your team to focus on their application services, rather than having to worry about supporting data services like log or metrics analytics. Focus on Agility Remaining relevant to today’s discerning customers makes speed essential to business. My grandma’s preferred cafeteria menu remained unchanged in the 20 odd years I can remember going there. Food trucks, on the other hand, can change their menu at the drop of a hat in response to the fickle foodies everywhere. The new modern application architecture emphasizes agility and flexibility to respond to similar expectations in the Internet world. Users change loyalties as often as hipsters take food selfies, and businesses can no longer accept nail-biting code deployments every few months. Just like food trucks enable cooks to experiment at the micro-level within a standard kitchen platform, microservices have allowed engineering teams to simplify the act of making changes by pushing complexity down into the smallest box possible. Now, instead of standardizing on lumbering platform components, these companies have standardized on APIs and infrastructure automation. The appeal of data analytics as a service is driven by the same imperatives. Engineers can bake machine data spigots into their microservices components and then depend on the data analytics services’ to provide the appropriate services, as published. I have good memories of eating bland mash potatoes with my grandma, just like I have fond memories of supporting monolithic applications in my early career. But you won’t catch me sneaking into a cafeteria or a chain restaurant for plate full of highly-salted vegetables these days. I’ve moved on. The world has moved on. You can say the same with the modern application. It’s time to focus on making the best possible korean taco or falafel sandwich your business can possibly make, and let us take care of machine data analytics for you. Now, go get some lunch.

Blog

It’s so hard to say goodbye - An ode to whoopie cushions and the modern application

“I thought we'd get to see forever But forever's gone away It's so hard to say goodbye to yesterday. “ Boyz II Men “Everything changes and nothing stands still.” Heraclitus of Ephesus Change comes at us fast and furious. It seems like only yesterday that my four-year-old daughter thought peek-a-boo was the height of comedy, and now she is begging me to use her whoopie cushion on colleagues at work. Changes in technology are only slightly less of a blur. Machinery powering the Internet has evolved far beyond what most of us thought possible even a few years ago. As I reflect back on this evolution, below are the changes in devops methodology and toolsets that rise to the top for me. Change 1: Today’s App ain’t your Daddy’s App Back when I started out in the world of IT in the early 2000s, the 3-tier app dominated - web, app, and database. Over that same decade, that pedestrian view of the world shattered, albeit slowly, with the introduction and adoption of virtualization, the “Cloud”, distributed data centers, etc., etc. Then, as the behemoths of the Internet age really got going, we’ve seen micro-service architectures drive us to new heights in the form of distributed processing, platform as a service, and virtualization driven to its logical conclusion in containerization - just to name a few. And here’s the core idea that has hit me like a ton of bricks - these new massively scalable apps don’t lend themselves to nicely laid out architecture diagrams. What used to be a nice hierarchal tree is now spaghetti. The old approaches to architecture just can’t handle this kind of complexity and constant change. Change 2: The speed of Business is the speed of the Internet user Accusing the modern software team of doing “waterfall” is about the same as asking your mother-in-law’s if she’s put on few pounds. Bad, bad idea. Why? The digital corporations of 2016 depend on software quality and innovation for their survival. Their users have no patience for spinning icons and slowly loading pages. The mistakes these companies make will immediately be taken advantage of by their competitors. The grassroots answer to this need for agility has been a focus on small team empowerment and collaboration, while also distributing the responsibility for quality and uptime from just operations teams to cross-functional engineering teams. Consequently, these teams are now demanding technologies that supports their collaborative approach, while also being flexible enough to adapt to their particular approach to systems management. Change 3: The Modern Application demands a better toolset We talk a lot these days about how the big 4 (HP, BMC, CA, IBM) are imploding. But that isn’t the end of it. The IT management children born of the failures of the big 4 are now themselves quickly becoming outmoded. We are seeing once-innovative companies trying to convince the world that slapping “On Demand” or “Cloud” labels on on-premise software means they are up-to-snuff. Savvy customers aren’t buying it. And then on the other hand, we have monitoring and troubleshooting tools that weren’t built for the micro-services, code-to-container world. The deeper implication here is that not only do tools need to evolve to support this new microservices architecture world, but the answer to “what do I need to measure” is rapidly changing. The modern application development team doesn’t need to be spoon-fed the “proper” metrics, when they know how to measure their app’s performance - they wrote it. The twin revolutions of DevOps and Agile have pulled the wary software developer out of the safety of waterfall into the controlled chaos of continuous integration and deployment. She is on the hook for the quality of her software, and she can write her own metrics - thank you very much. “It's the end of the world as we know it, and I feel fine” R.E.M. So, what’s the takeaway of the story here? To start off, I am recruiting my accomplices at work for the “great whoopie cushion” caper. I am going to embrace my daughter’s humor where it is at, and stop making “daddy” excuses. So, don’t resist the change - embrace it. And when you do - remember that you aren’t alone. You can ask more from the tools and services you use to support your application. You and your team deserve better.

Blog

Explore your Data with Interactive Dashboards

We are excited about one of our newest features, Interactive Dashboards. Interactive Dashboards is the second major release of a longer term project for changing the way Sumo Logic dashboards work. Based on the feedback from you, our users, we are working to make our dashboarding technology more flexible and easier to use. In this blog article we’ll discuss how the new dashboards work, and how they can help you get more out of your data. What are Interactive Dashboards? Interactive Dashboards are built on the power of our search engine. That means powerful analytics with lots of flexibility. It also means that you can look at historical data immediately, explore the data more easily, and make changes to your dashboards without worrying about delays. So, let’s talk details. Interactive Dashboards help you Explore your data There are really three major challenges that we have tried to solve with this technology – making it easier to look at the data, opening up historical exploration of data, and navigating data with simple to use filters. Free Users from the Query Language Many of Sumo Logic users cringe when they see the search window, and that’s ok. Their use of Sumo Logic shouldn’t require knowing the intricacies of parsing or any familiarity with regex. Interactive Dashboards helps those users. Now subject matter experts can use their experience to build great dashboards, and point novice users at those dashboards. No need to know the queries. No need to understand the underlying logs. Consider your history One major request from our customers has been to use dashboards to look at things that happened last night, last week, or a month ago. Now users can apply any date range to an Interactive dashboard (within your data retention period, of course), and see the data. For example, let’s say you want to compare last weeks user visits to this weeks. No problem. Look at a web server issue last Tuesday. No problem. Any user viewing an Interactive Dashboard can set the time range on individual panels or reset the whole dashboard. Reduce the Noise One of the most requested features for dashboards has been filters. Now you can filter on any field in your data, whether you pulled that field out in the query itself, or it was pulled out on ingest. So, why does it matter? Now you can create dashboards and give your colleagues a way to easily navigate the data. For example, maybe let’s say you are creating a dashboard to look at web statistics. The problem is that you have 10 web sites with at a few apps on each of them. With Interactive Dashboards you can create a more generic dashboard with a filter for web site and another for app. Now you get a global view, plus any user can dig into the data without understanding anything about the structure of the data or complexity of your queries! Worried about Unauthorized Access to Data – Worry no more One of the problems with streaming dashboards is that by their nature there is one stream – and that stream has to be based on some user’s view. So, with our traditional dashboards, if you shared a dashboard, you shared the data. Now, with Interactive Dashboards, that is no longer the case. The new dashboards take any access controls you have setup into account. For example, if you have shared a security dashboard that has sensitive data, users in Roles that cannot see that sensitive data will not see it in the dashboards either. In conclusion, as I mentioned above, Interactive Dashboards is only one of many steps we will be taking to making our dashboards more powerful and useful for you. As you try out Interactive Dashboards, we look forward to your comments and feedback.

August 12, 2015

Blog

3 Ways to make your Data work for you - on AWS and everywhere else

This is an exciting time to be in the Enterprise software business. Many of the predictions we heard in the late 90s and early 2000s are finally coming true. First, we are finally moving from the world of the server artist (pronounced “Arteeste”) lovingly caring for his server, to a world where any self-respecting operations engineer is capable of runnings hundreds of systems on a bad day. That transformation has come on the back of organizations finally embracing things like configuration automation and standardization on top of powerful cloud platforms like Amazon Web Services (AWS). Container technologies like Docker have only accelerated that transition. Second, the evolution of the DevOps movement is changing the ways operations teams approach their jobs. Now they tend towards small, collaborative teams swarming problems with no respect for technology silos. Developers are also being held accountable now for the uptime of their software, so they are putting more of the operational machinery in code and changing the data from their apps to suit their needs. “So, what now?” Thanks for asking. Now that we are mechanizing our software and operations, we need to spend more thinking about what our applications and platforms spit out - the machine data. Here are three ways to up your machine data game: Embrace the Format Why are we still producing logs that look like they were written by a drunk monkey? If you are planning on getting any value out of those logs at scale you have to put them in a format that makes them easier to analyze. JSON is really the best choice, but even embracing simple key-value pairs will make a huge difference. Embrace the Content Imagination is great for jam sessions with your college buddies, but horrible for operational logging. The more your data looks the same, the easier it is to analyze, and to understand at 2 AM when your application is down. And how do you do that at scale? One way is to embrace standardized metrics and logging libraries (Log4J, NLog, etc.). Another way is to use your platform - well, that’s my third point. Embrace the Platform The primary reason why AWS is so successful is not the nebulous “cloud” concept. AWS is powerful because it has standardized and commoditized operational details away. Need a load-balancer? No need to order a $50k box with blinking lights. Press a button in your AWS console. Need another app server, or 10 or 100.? No problem. Now AWS is doing the same thing with data. It is simple to generate streams of useful data with CloudTrail, Cloudwatch, Kinesis, etc. If you use the standardized services like ELB, RDS, or S3, you get even more metrics and logs out of the box. So - embrace the data, embrace the platform. So, what’s the wrap? Just remember to go all in! Half-measures don’t help you scale to that next level. Flexibility is great for that yoga class you just enrolled in, but standardization will make life easier when the proverbial stuff hits the fan at oh-dark:30 in the morning. Use the tools at hand to squeeze unnecessary mess out of your data and focus on the fun part - making pretty pictures for your boss with that newly massaged data. Box Chart, anyone?

AWS

August 11, 2015

Blog

Piercing the Fog in a Devops World

Two things still amaze me about the San Francisco Bay area two years on after moving here from the east coast – the blindingly blue, cloudless skies – and the fog. It is hard to describe how beautiful it is to drive up the spine of the San Francisco Peninsula on northbound I-280 as the fog rolls over the Santa Cruz mountains. You can see the fog pouring slowly over the peaks of the mountains, and see the highway in front of you disappear into the white, fuzzy nothingness of its inexorable progress down the valley. There is always some part of me that wonders what will happen to my car as I pass into the fog. But then I look at my GPS, know that I have driven this road hundreds of times, and assure myself that my house does still exist in there – somewhere. Now, I can contrast that experience with learning to drive in the Blue Ridge Mountains of North Carolina. Here’s the background – It’s only my second time behind the wheel, and my Mom takes me on this crazy stretch of road called the Viaduct. Basically, imagine a road hanging off the side of a mountain, with a sheer mountain side on the one side, and a whole lot of nothing on the other. Now, imagine that road covered in pea-soup fog with 10 ft visibility, and a line of a half dozen cars being led by a terrified teenager with white knuckled hands on the wheel of a minivan hoping he won’t careen off the side of the road to a premature death. Completely different experience. So, what’s the difference between those two experiences. Well, 20 years of driving, and GPS for starters. I don’t worry about driving into the thick fog as I drive home because I have done it before, I know exactly where I am, how fast I am going, and I am confident that I can avoid obstacles. That knowledge, insight, and experience make all the difference between an awe-inspiring journey and a gut-wrenching nail-biter. This is really not that different from running a state of the art application. Just like I need GPS and experience to brave the fog going home, the difference between confidently innovating and delighting your customers, versus living in constant fear of the next disaster, is both driven by technology and culture. Here are some ways I would flesh out the analogy: GPS for DevOps An app team without visibility into their metrics and errors is a team that will never do world-class operations. Machine Data Analytics provides the means to gather the telemetry data and then provide that insight in real-time. This empowers App Ops and DevOps teams to move more quickly and innovate. Fog Lights for Avoiding Obstacles You can’t avoid obstacles if you can’t see them in time. You need the right real-time analytics to quickly detect issues and avoid them before they wreck your operations. Experience Brings Confidence If you have driven the road before, it is always increases confidence and speed. Signature-Based anomaly detection means that the time that senior engineers put in to classify previous events gives the entire team the confidence to classify and debug issues. So, as you drive your Application Operations and DevOps teams to push your application to the cutting edge of performance, remember that driving confidently into the DevOps fog is only possible with the right kind of visibility. Images linked from: http://searchresearch1.blogspot.com/2012/09/wednesday-search-challenge-9512-view-of.html https://www.sumologic.com/blog... class="at-below-post-recommended addthis_tool">

Blog

What is StatsD and How Can DevOps Use It?

Blog

Harder, Better, Faster, Stronger - Machine Data Analytics and DevOps

Work It Harder, Make It Better Do It Faster, Makes Us Stronger More Than Ever Hour After Our Work Is Never Over Daft Punk - "Harder, Better, Faster, Stronger" When trying to explain the essence of DevOps to colleagues last week, I found myself unwittingly quoting the kings of electronica, the French duo Daft Punk (and Kanye West, who sampled the song in "Stronger"). So often, I find the "spirit" of DevOps being reduced to mere automation, the takeover of Ops by Dev (or vice versa), or other over-simplications. This is natural for any new, potentially over-hyped, trend. But how do we capture the DevOps "essence" - programmable architecture, agile development, and lean methodology - in a few words? It seems like the short lyrics really sum up the essence of the flexible, agile, constantly improving ideal of a DevOps "team", and the continuous improvement aspects of lean and agile methodology. So, what does this have to do with machine data analytics and Sumo Logic? Part of the DevOps revolution is a deep and wrenching re-evaluation of the state of IT Operations tools. As the pace of technological change and ferocity of competition keep increasing for any company daring to make money on the Internet (which is almost everybody at this point), the IT departments are facing a difficult problem. Do they try to adapt the process-heavy, tops-down approaches as exemplified by ITIL, or do they embrace a state of constant change that is DevOps? In the DevOps model, the explosion of creativity that comes with unleashing your development and operations teams to innovate quickly overwhelms traditional, static tools. More fundamentally, the continuous improvement model of agile development and DevOps is only as good as the metrics used to measure success. So, the most successful DevOps teams are incredibly data hungry. And this is where machine data analytics, and Sumo Logic in particular, really comes into its own, and is fundamentally in tune with the DevOps approach. 1. Let the data speak for itself Unlike the management tools of the past, Sumo Logic makes only basic assumptions about the data being consumed (time stamped, text-based, etc.). The important patterns are determined by the data itself, and not by pre-judging what patterns are relevant, and which are not. This means that as the application rapidly changes, Sumo Logic can detect new patterns - both good and ill - that would escape the inflexible tools of the past. 2. Continuous reinterpretation Sumo Logic never tries to force the machine data into tired old buckets that are forever out of date. The data is stored raw so that it can continually be reinterpreted and re-parsed to reveal new meaning. Fast moving DevOps teams can't wait for the stodgy software vendor to change their code or send their consultant onsite. They need it now. 3. Any metric you want, any time you want it The power of the new DevOps approach to management is that the people that know the app the best, the developers, are producing the metrics needed to keep the app humming. This seems obvious in retrospect, yet very few performance management vendors support this kind of flexibility. It is much easier for developers to throw more data at Sumo Logic by outputting more data to the logs than to integrate with management tools. The extra insight that this detailed, highly specific data can provide into your customers' experience and the operation of your applications is truly groundbreaking. 4. Set the data free Free-flow of data is the new norm, and mash-ups provide the most useful metrics. Specifically, pulling business data from outside of the machine data context allows you to put it in the proper perspective. We do this extensively at Sumo Logic with our own APIs, and it allows us to view our customers as more than nameless organization ID numbers. DevOps is driven by the need to keep customers happy. 5. Develop DevOps applications, not DevOps tools The IT Software industry has fundamentally failed its customers. In general, IT software is badly written, buggy, hard to use, costly to maintain, and inflexible. Is it any wonder that the top DevOps shops overwhelmingly use open source tools and write much of the logic themselves?! Sumo Logic allows DevOps teams the flexibility and access to get the data they need when they need it, without forcing them into a paradigm that has no relevance for them. And why should DevOps teams even be managing the tools they use? It is no longer acceptable to spend months with vendor consultants, and then maintain extra staff and hardware to run a tool. DevOps teams should be able to do what they are good at - developing, releasing, and operating their apps, while the vendors should take the burden of tool management off their shoulders. The IT industry is changing fast, and DevOps teams need tools that can keep up with the pace - and make their job easier, not more difficult. Sumo Logic is excited to be in the forefront of that trend. Sign up for Sumo Logic Free and prove it out for yourself.

Blog

Finding Needles in the the Machine Data Haystack - LogReduce in the Wild

As with any new, innovative feature in a product, it is one thing to say it is helpful for customers - it is quite another to see it in action in the wild. Case in point, I had a great discussion with a customer about using LogReduce™ in their environment. LogReduce is a groundbreaking tool for uncovering the unknown in machine data, and sifting through the inevitable noise in the sea of log data our customers put in Sumo Logic. The customer in question had some great use cases for LogReduce that I would like to share. Daily Summaries With massive amounts of log data flowing through modern data centers, it is very difficult to get a bird's eye view of what is happening. More importantly, the kind of summary that provides actionable data about the day's events is elusive at best. In our customer example, they have been using LogReduce to provide exactly that type of daily, high-level overview of the previous day's log data. How does it work? Instead of using obvious characteristics to group log data like the source (e.g. Window's Events) or host (e.g. server01 in data center A), LogReduce uses "fuzzy logic" to look for patterns across all of your machine data at once - letting the data itself dictate the summary. Log data with the same patterns, or signatures, are grouped together - meaning that new patterns in the data will immediately stand out, and the noise will be condensed to a manageable level. Our customer is also able to supply context to the LogReduce results - adjusting and extending signatures, and adjusting relevance as necessary. In particular, by adjusting the signatures that LogReduce finds, the customer is to "teach" LogReduce to provide the best results in the most relevant way. This allows them to separate the critical errors out, while still acknowledging the background noise of known messages. The end-result is a daily summary that is both more relevant because of the user-supplied, business context as well as being flexible enough to find important, new patterns. Discovering the Unknown And finding those new patterns is the essential essence of Big Data analytics. A machine-data analytics tool should be able to find unknown patterns, not simply reinforce the well-known ones. In this use case, our customer already has alerting established for known, critical errors. The LogReduce summary provides a way to identify, and proactively address, new, unknown errors. In particular, by using LogReduce's baseline and compare functionality, Sumo Logic customers can establish a known state for log data and then easily identify anomalies by comparing the current state to the known, baselined state. In summary, LogReduce provides the essence of Big Machine Data analytics to our customers - reducing the the constant noise of today's datacenter, while finding those needles in the proverbial haystack. This is good news for customers who want to leverage the true value of their machine data without the huge investments in the time and expertise required in the past.

March 19, 2013

Blog

Why I joined Sumo Logic and moved to Silicon Valley

We make hundreds of decisions every day, mostly small ones, that are just part of life’s ebb and flow. And then there are the big decisions that don’t merely create ripples in the flow of your life – they redirect it entirely. The massive, life-defining decisions like marriage and children; the career-defining decisions like choosing your first job after college. I’ve had my share of career-defining decisions – leaving a physics graduate program to chase after the dot com craze, leaving consulting for sales engineering, etc. The thing about this latest decision is that it combines both. I am joining Sumo Logic, leaving behind a safe job in marketing, and moving to Silicon Valley – away from my friends, family, and community. So, why did I do it? Now is the time for Start-Ups in Enterprise Software. Consumer start-ups get all the press, but the enterprise startups are where the real action is. The rash of consolidations in the last five years or so has created an innovation gap that companies like Sumo Logic are primed to exploit. The perfect storm of cloud computing, SaaS, Big Data, and DevOps/Agile is forcing customers to start looking outside of their comfort zones to find the solutions they need. Sumo Logic brings together all of that innovation in a way that is too good to not be a part of it. The Enterprise SaaS Revolution is Inevitable. The SaaS business model, combined with Agile development practices, is completely changing the ways companies buy enterprise software. Gartner sees companies replacing legacy software with SaaS more than ever. The antiquated term-licenses of on-premise software with its massive up-front costs, double digit maintenance charges, and “true-ups” seem positively barbaric by comparison to the flexibility of SaaS. And crucially for me, Sumo Logic is also one of the few true SaaS companies that is delving into the final frontier of the previously untouchable data center. Big Data is the “Killer App” for the Cloud. “Big Data” analytics, using highly parallel-ized architectures like Hadoop or Cassandra, is one of the first innovations in enterprise IT to truly be “born in the cloud”. These new approaches were built to solve problems that just didn’t exist ten, or even five, years ago. The Big Data aspect of Sumo Logic is exciting to me. I am convinced that we are only scratching the surface of what is possible with Sumo Logic’s technology, and I want to be there on the bleeding edge with them. Management Teams Matter. When it really comes down to it, I joined Sumo Logic because I have first-hand knowledge of the skills that Sumo Logic’s management team brings to the table. I have complete confidence in Vance Loiselle’s leadership as CEO, and Sumo Logic has an unbeatable combination of know-how and get-it-done people . And clearly some of the top venture capital firms in the world agree with me. This is a winning team, and I like to win! Silicon Valley is still Nirvana for Geeks and the best place for Start-Ups. Other cities are catching up, but Silicon Valley is still the best place to start a tech company. The combination of brainpower, money, and critical mass is just hard to beat. On a personal level I have resisted the siren call of San Francisco Bay Area for too long. I am strangely excited to be in a place where I can wear my glasses as a badge of honor, and discuss my love for gadgets and science fiction without shame. Luckily for me, I am blessed with a wife that has embraced my geek needs, and supports me whole heartedly (and a 21-month-old who doesn’t care either way). So, here’s to a great adventure with the Sumo Logic team, to a new life in Silicon Valley, and to living on the edge of innovation. P.S. If you want to see what I am so excited about, get a Sumo Logic Free account and check it out.