Distributed Processing with Azure Functions

As distributed computing becomes more and more available, it’s good to evaluate the different options accessible to you and your organization. At Build 2016, Microsoft announced the upcoming availability of Azure Functions, an event-driven, on-demand compute system similar to AWS Lambda Functions. Azure Functions are still a new feature in Azure, so there are bound to be growing pains while they mature. With that in mind, let’s dig deeper!

Building a Twitter Aggregator with Azure Functions

I’ve always lived by the mantra that learning is best done by doing. Imagine that we are building a service that generates profile information for users by aggregating their content from different websites/services. We can create a Twitter aggregator as an Azure Function.

This Function (which would be part of a group of Functions for aggregating content on-demand) will accept two arguments from a web request: ID and Handle. The ID will represent a mythical unique identifier created by our system elsewhere. The Handle will be the supplied Twitter handle/screen-name of the user who has requested a profile be generated. Once these arguments are supplied, the Function should connect to Twitter and pull down the five most recent tweets for use in the profile.

Creating a new Function is incredibly simple. Head over to portal.azure.com/, and search for “Function App” in the marketplace to add a new resource. It is listed underneath “Web + Mobile.” Alternatively you can go to functions.azure.com/ and get a more streamlined creation process. Either way, you are presented with the ability to name your Function and choose things like its physical location and subscription. After that, you are taken straight to an editor where you can begin configuring and coding the Function. I’ll skip through the boring parts and get to the code because there’s a bit of effort involved due to working with Twitter. Here’s a link to the completed Function:

I won’t go through every line of code, but there are important elements to discuss.

1. Line #1 is #r “Newtonsoft.Json”, which is a way of referencing assemblies. You can provide new assemblies by taking advantage of the Continuous Integration options and tying your Function to a source control repository with a bin directory that contains pre-compiled DLLs. For more information, see the documentation here.
2. You’ll notice that I’m not creating any new classes within my Function. This is not a requirement, as this is simply a C# “script.” You are free to create classes, but as of the time of this writing, you still cannot create namespaces for organizing your classes. (This can still be achieved by building libraries that your script uses, as referenced in #1).
3. I am making use of the HttpRequestMessage.CreateResponse() method for my Function’s responses, but as I’ll discuss in a moment, it was the “quick and dirty” approach to building a new Function. There are other options that may fit your use cases better. Feel free to explore.
4. I opted to not use any pre-built Twitter libraries for simplicity. I don’t generally recommend this, but it also isn’t terribly complicated for app-only usage, so make a decision based on your use case.

Possibilities Everywhere

Although I chose to build my Function in C#, Microsoft provides numerous language options including:

• C#
• F#
• Powershell
• Batch
• Bash
• JavaScript
• PHP
• Python

There’s also a wide range of options for input and output. I set up my Function as a generic webhook, meaning it provides me with an API endpoint to call for triggering the Function. If that’s not your cup of tea, though, maybe one of these satisfies:

• Storage Blobs (trigger whenever a file is updated/added to an Azure Storage Container)
• Event Hubs (trigger whenever an event is received from an Azure Event Hub)
• GitHub WebHook (trigger whenever a GitHub webhook request is received)
• HttpTrigger (trigger based off of HTTP requests)
• QueueTrigger (trigger whenever a message is added to an Azure Storage Queue)
• ServiceBusQueueTrigger (trigger whenever a message is added to a ServiceBus instance)
• TimerTrigger (trigger on a schedule, similar to scheduled tasks or cron jobs)

There is also a full set of options included to direct request output. For my Function, I simply assumed that returning the JSON as a web response was reasonable, but there are likely much better/scalable ways of directing the output into our profiles.

See the full documentation for more information.

In Closing

While Sumo Logic doesn’t yet provide an integration for Azure Functions, you can get visibility into other Microsoft technologies using tools like the Sumo Logic Application for Office 365 (currently in Preview) that offersPre-built dashboards, searches and reports visually highlight key activity across your O365 environment including OneDrive, Exchange Online, SharePoint, and Azure Active Directory. Check out other apps like the App for Windows Directory at sumologic.com.

Hopefully this has been a simple introduction to the concept of Azure Functions. The sheer number of things that can be accomplished with them is daunting, but they are fantastic for the world of distributed computing. The documentation and community for Functions is already growing, so don’t be afraid to take them for a spin.

Resources

• Complete Source for Example

About the Author

Andrew Male (@AndyM84) is a senior engineer at an enterprise development company in Boston, MA. Andrew has been programming from a young age and is entirely self-taught. He has spent time in many corners of the programming world, including game/VR work, agency work, and teaching development to students and adults alike. He spends most of his time working on architecture design and pursuing his favorite hobby—physics.