Brandon Mensing

Using the Join Operator

10.29.2013 | Posted by Brandon Mensing

The powerful analytics capabilities of the Sumo Logic platform have always provided the greatest insights into your machine data. Recently we added an operator – bringing the essence of a SQL JOIN to your stream of unstructured data, giving you even more flexibility.

In a standard relational join, the datasets in the tables to be joined are fixed at query time. However, matching up IDs between log messages from different days within your search timeframe likely produces the wrong result because actions performed yesterday should not be associated with a login event that occurred today. For this reason, our Join operator provides for a specified moving timeframe within which to join log messages. In the diagram below, the pink and orange represent two streams of disparate log messages. They both contain a key/value pair that we want to match on and the messages are only joined on that key/value when they both occur within the time window indicated by the black box.

Join_operator_window

 

Now let’s put this to use. Suppose an application has both real and machine-controlled users. I’m interested in knowing which users are which so that I can keep an eye out for any machine-controlled users that are impacting performance. I have to find a way to differentiate between the real vs the machine-controlled users. As it turns out, the human users create requests at a reasonably low rate while the machine-controlled users (accessing via an API) are able to generate several requests per second and always immediately after the login event.

Join_operator_Log_snippets

In these logs, there are several different messages coming in with varying purposes and values. Using Join, I can query for both the logins and requests and then restrict the time window of the matching logic to combine the two messages streams. The two sub queries in my search will look for request/query events and login events respectively. I’ve restricted the match window to just 15 seconds so that I’m finding the volume of requests that are very close to the login event. Then I’m filtering out users who made less than 10 requests in that 15-second time frame following a login. The result is a clear view of the users that are actively issuing a large volume of requests via the API immediately upon logging in. Here is my example query:

(login or (creating query))
| join
(parse "Creating query: '*'" as query, "auth=User:*:" as user) as query,
(parse "Login success for: '*'" as user) as login
on query.user = login.user
timewindow 15s
| count by query_user
| where _count > 10
| sort _count

As you can see from the above syntax, the subqueries are written with the same syntax and even support the use of aggregates (count, sum, average, etc) so that you can join complex results together and achieve the insights you need. And of course, we support joining more than just two streams of logs – combining all your favorite data into one query!

Mark Musselman

A Sort of Homecoming: Back From Akamai Edge

10.15.2013 | Posted by Mark Musselman

This is my first blog post for Sumo Logic.  It took 18 months but I was always a late bloomer and we have some Hemingway-class bloggers on staff anyway.  No doubt I was shy as my music production partner in MOMU, JD Moyer, is now a prolific blogger with an immense following. 

Nonetheless, when I was asked to write about the experience at last week’s Akamai Edge – the worldwide customer and partner conclave – due to my unique position of having worked at Akamai from 2002 to 2005, I jumped at the opportunity. The day I started at Akamai the stock was either at 52 cents or 56 cents – I don’t recall exactly.  The day I left it was at $56 bucks – I do remember that.  In those three years, I was able to bring onboard and expanded Akamai’s presence at companies like eBay, The Gap, RingCentral, Netflix, Walmart.com and E*Trade, all of whom bought into the business value that Akamai delivered.  This culminated in being Named the top Major Account Executive for the Americas in 2004 – definitely a personally “pinnacle” achievement….

Akamai is an amazing company for way too many reasons to list, but the people and the culture top the list.  In fact, when I think about the best places I have worked, from Ritz-Carlton to BladeLogic, the common thread among these favorite employers of mine was and is the people.  Smart, aggressive, coachable, creative, daring, fearless and fun people, with amazing founders.   

I want to key in on the similarity that I see between Akamai and Sumo Logic. Akamai is the first Cloud Company.  REALLY Cloud.  My goal when I arrived at Sumo Logic last year was to help build a culture that weaved in the best of two great worlds – BladeLogic and Akamai, with a maniacal focus on the Customer Experience.  At all of these places a common theme was the “DNA” of the staff.  There is magnificent art in taking a cutting edge, disruptive product and meshing it with the intense sense of urgency and thoughtful execution.  Having the opportunity to help build this from scratch at Sumo Logic was too good to pass up.  I fell in love with Christian and Kumar’s vision and the innovation around the technology.

There are many more similarities than just the clarity of vision and the incredible focus on execution.  The inimitable George Conrades once told a prospect of ours in a meeting how many lines of code that Akamai had written – in 2003 – and it was a massive number.  We are both software companies at the core.  We both rely heavily on algorithms to create customer value and massive differentiation.  We both go to market with a recurring revenue model.  We both allow for instant elasticity and on-demand usage.  We both are totally focused on a great product that helps our customers fight the demands of the digital world with the best tools available.  Last but not least we are both entranced by The Algorithm…

Back to Akamai Edge.   It is incredible to see how much of the online world continues to run and thrive through Akamai.  2.2 billion log lines every 60 seconds.  Yes, you read that right.  Staggering scale.  The session on the Dominant Design principle blew me away.  With the new announcement of Akamai opening up its platform to developers and partners, Akamai is even more Open.  Sumo Logic is thrilled to become a charter Member of the Open Platform Initiative – we already have many joint customers salivating to send the Akamai logs directly to Sumo Logic, where they can “join” them with the rest of their infrastructure logs – all for real-time insights across their entire infrastructure.  The beta customers are all happy that we have come so far so quickly together.  This is an alliance with legs AND brains.

George, Paul, Tom, Bob, Brad, Doug, John, Tim, Mark, Gary, Jennie, Rick, Kevin, Kris, Alyson, Brian, Mike, Andy, Dave, Ed (and so many more)….it was great to see you and it is GREAT to be working with you again.  The new hires I met seem to have the DNA you need to get to the next Scaling Point.

Akamai and Sumo Logic: Faster Forward Together, Moving at the speed of Cloud.

Now stop reading my rant and go sell something, will ya, and check out our new Sumo Logic Application for Akamai.  

Bruno Kurtic, Founding Vice President of Product and Strategy

Akamai and Sumo Logic integrate for real-time application insights!

10.09.2013 | Posted by Bruno Kurtic, Founding Vice President of Product and Strategy

I’m very pleased to announce our strategic alliance with Akamai. Our integrated solution delivers a unified view of application availability, performance, security, and business analytics based on application log data.  Customers who rely on Akamai’s globally distributed infrastructure now can get the real-time feed of all logs generated by Akamai’s infrastructure into their Sumo Logic account in order to integrate and cross-analyze them with their internally generated application data sets!

What problems does the integrated solution solve?

To date, there have been two machine data sets generated by applications that leverage Akamai:

1. Application logs at the origin data centers, which application owners can usually access.

2. Logs generated by Akamai as an application is distributed globally. Application owners typically have zero or limited access to these logs.

Both of these data sets provide important metrics and insights for delivering highly-available, secure applications that also provide detailed view of business results. Until today there was no way to get these data sets into a single tool for real-time analysis, causing the following issues:

  • No single view of performance. While origin performance could be monitored, but that provides little confidence that the app is performant for end users.
  • Difficult to understand user interaction. Without data on how real users interact with an application, it was difficult to gauge how users interacted with the app, what content was served, and ultimately how the app performed for those users (and if performance had any impact on conversions).
  • Issues impacting customer experience remained hidden. The root cause of end-user issues  caused at the origin remained hidden, impacting customer experience for long periods of time.
  • Web App Firewall (WAF) security information not readily available. Security teams were not able to detect and respond to attacks in real-time and take defensive actions to minimize exposure.

The solution!

Quality of Service

Akamai Cloud Monitor and Sumo Logic provide an integrated approach to solving these problems. Sumo Logic has developed an application specifically crafted for customers to extract insights from their Akamai data, which is sent to Sumo Logic in real time.  The solution has been deployed by joint customers (at terabyte scale) to address the following use cases:

  • Real-time analytics about user behavior.  Combine Akamai real-user monitoring data and internal data sets to gain granular insights into user behavior. For example, learn how users behave across different device types, geographies, or even how Akamai quality of service impacts user behavior and business results.

  • AttacksSecurity information management and forensics. Security incidents and attacks on an application can be investigated by deep-diving into sessions, IP addresses, and individual URLs that attackers are attempting to exploit and breach.

  • Application performance management from edge to origin. Quickly determine if an application’s performance issue is caused by your origin or by Akamai’s infrastructure, and which regions, user agents, or devices are impacted.

  • Application release and quality management. Receive an alert as soon as Akamai detects that one or more origins have an elevated number of 4xx or 5xx errors that may be caused by new code push, configuration change, or another issue within your origin application infrastructure.

  • Impact of quality of service and operational excellence. Correlate how quality of service impacts conversions or other business metrics to optimize performance and drive better results

I could go on, but I’m sure you have plenty of ideas of your own.

Join us for a free trial here – as always, there is nothing to install, nothing to manage, nothing to run – we do it all for you.  You can also read our announcement here or read more about the Sumo Logic application for Akamai here.  Take a look at the Akamai press release here.

Christian Beedgen, Co-Founder & CTO

Meatballs And Flying Tacos Don’t Make a Cloud

10.02.2013 | Posted by Christian Beedgen, Co-Founder & CTO

Yes, we are cloud and proud. Puppies, ponies, rainbows, unicorns. We got them all. And this, too. But the cloud is not a personal choice for us at Sumo Logic. It is an imperative. An imperative to build a better product, for happier customers.

We strongly believe that if designed correctly, there is no need to fragment your product into many different pieces, each with different functional and performance characteristics that confuse decision-makers. We have built the Sumo Logic platform from the very beginning with a mindset of scalability. Sumo Logic is a service that is designed to appeal and adapt to many use cases. This explains why in just three short years we have been successful in a variety of enterprise accounts across three continents because – first and foremost – our product scales.

On the surface, scale is all about the big numbers. We got Big Data, thank you. So do our customers, and we scale to the level required by enterprise customers. Yet, scaling doesn’t mean scaling up by sizes of data sets. Scaling also means being able to scale back, to get out of the way, and provide value to everyone, including those customers that might not have terabytes of data to deal with. Our Sumo Free offering has proven that our approach to scaling is holistic – one product for everyone. No hard decisions to be made now, and no hard decisions to be made later. Just do it and get value.

Another compelling advantage of our multi-tenant, one service approach is that we can very finely adjust to the amount of data and processing required by every customer, all the time. Elasticity is key, because it enables agility. Agile is the way of business today. Why would anyone want to get themselves tied into a fixed price license, and on top of that provision large amount of compute and storage resources permanently upfront just to buy insurance for those days of the year where business spikes, or, God forbid, a black swan walks into the lobby? Sumo Logic is the cure for anti-agility in the machine data analytics space. As a customer, you get all the power you need, when you need it, without having to pay for it when you don’t.

Finally, Sumo Logic scales insight. With our recently announced anomaly detection capability, you can now rely on the army of squirrels housed in our infrastructure to generate and vet millions of hypotheses about potential problems on your behalf. Only the most highly correlated anomalies survive this rigorous process, meaning you get actionable insight into potential infrastructure issues for free.  You will notice repetitive events and be able to annotate them precisely and improve your operational processes. Even better – you will be able to share documented anomalous events with and consume them back from the Sumo Logic community. What scales to six billion humans? Sumo Logic does.

One more thing: as a cloud-native company, we have also scaled the product development process, to release more features, more improvements, and yes, more bug fixes than any incumbent vendor. Sumo Logic runs at the time of now, and new stuff rolls out on a weekly basis. Tired of waiting for a year to get issues addressed? Tired of then having to provision an IT project to just update the monitoring infrastructure? Scared of how that same issue will apply even if the vendor “hosts” the software for you? We can help.

Sumo Logic scales, along all dimensions. You like scale? Come on over.

Oh, and thanks for the date, Praveen. I’ll let you take the check.

Twitter