Creating event driven integrations using Azure Event Grid

Yesterday Microsoft announced their newest service on Azure, and it is called Azure Event Grid. With this new service, we now have event based serverless routing, from any source, to any destination. Of course, we all love integration, and Azure Event Grid brings a whole new world of possibilities. The service is currently in public preview, and we already have various publishes and event handlers to our disposal, and more will be rolling out over the coming months. In the end, expect every service within Azure to have a connection to Event Grid.

Event Publishers and Handlers

Event Publishers and Handlers

Event Publishers

Publishers are the services which will be sending events to Azure Event Grid. This could be something like a resource being created in a resource group, a user being added to a subscription, or a file being uploaded into blob storage. We also have Custom Topics, which we can use to send event into Event Grid from any service. This allows us to integrate any application or service through Event Grid.

Overview of publisher available

Overview of publisher available

Event Handlers

Event handlers are our subscribers to the events in Azure Event Grid. An event can be sent to any number of event handlers, allowing us to fan these out. We already have support for two of the, in my opinion, best serverless technologies out there, Azure Functions and Logic Apps. This allows us to start some logic by sending the event to an Azure Function, or start a workflow using Logic Apps. We also have the option to add web hooks as event handlers, allowing us to integrate any service as a subscriber as well. Event handlers can also define filters, so they can also subscribe to specific events.

Overview of subscribers available

Overview of subscribers available

Scenarios

When looking at this new service, I can see several scenarios where this would make a great impact. Here are some just a couple of examples where I could see real added value of Event Grid. The services used in these examples are either already available, or announced to be following soon.

IoT

The Internet of Things is defined by events, and I could therefor see a great future for Azure Event Grid in this space. Being able to route events dynamically to various services, starting workflows, calling events, storing information, this is a great addition to the existing tools we already have in Azure.

Example of IoT scenario

Example of IoT scenario

Messaging

Azure Event Grid solves the missing piece in the serverless messaging space, routing messages to various endpoints, calling services as needed, and all this with low latency, high reliability and massive througput, Event Grid is extremely friendly priced at $0.60 per million messages ($0.30 while in preview), and the first 100000 events are for free, so you can easily start testing your own scenarios.

Example of Messaging scenario

Example of Messaging scenario

Operations

As we will also have all our Azure services publishing events, we can easily add more control to operations. For example, we might want to subscribe to events on our subscription, and have a notification by email. Or we might want to automatically add tags, when a new SQL database has been created.

Example of Operation scenario

Example of Operation scenario

Implement Solution

So let’s see how we can implement our own solution with Azure Event Grid. In this sample, we will receive events from Event Hubs Capture, and fan them out based on the contents of the Event Hub from which we received the message. Events from the Notifications Event Hub will be sent to a Logic App, which sends out an email, while all events will be sent to an Azure Functions to be stored in Table Storage.

Solution outline

Solution outline

We will start by creating a Event Hubs namespace . Today Azure Event Grid is available in West US 2 and West Central US, but more regions will be coming soon.

Create Event Hubs namespace

Create Event Hubs namespace

Next we create a Logic App, which will subscribe to the events from the Notifications Event Hub, and send out an email.

Create Logic App

Create Logic App

Create an empty Logic App, and add an Event Grid trigger.

Add Event Grid trigger

Add Event Grid trigger

If you are prompted to sign in, do so with the account of your subscription. If for some reason this does not work, you can also create a service principal with contributor rights, and use that instead.

Now you will have to specify the subscription which will be used to get the messages from your Event Hub. Choose the subscription under which your Event Hubs namespace is created, set the resource type to Microsoft.EventHub.namespaces, and choose the resource name of the Event Hub we just created. Under advanced options, provide a name for the subscription, and set the filter to only subscribe to events from the Notifications Event Hub. Finally add an Outlook action which will be used to send out an email.

Logic App outline

Logic App outline

Now go back to the Event Hub namespace we created earlier, and open the Event Grid blade. We will now see the subscription we just created. Open the subscription, and under Event Types select Capture File Created, and save your changes.

Update subscription on Event Hub namespace

Update subscription on Event Hub namespace

As Event Grid currently only works with the Capture functionality of Event Hubs, we will need to create a storage account.

Create Storage Account

Create Storage Account

After the Storage Account has been created, go back to the Event Hub namespace we created earlier, and create two Event Hubs, Notifications and Other. Make sure to turn on Capture, select the Storage Account we just created, and create a container for the Capture files.

Create Notifications Event Hub with Capture

Create Notifications Event Hub with Capture

Create Other Event Hub with Capture

Create Other Event Hub with Capture

Next we will create the Function. Start by creating a new Function App.

Create Function App

Create Function App

Once created, add a new Function to the Function App. Choose the Webhook + API template.

Create Azure Function

Create Azure Function

Add a Azure Table Storage output to the Azure function.

Add Table Storage output

Add Table Storage output

Add a new Storage account connection to create a new Storage Account for the table. Once added, save the Function.

Add Storage Account

Add Storage Account

Now switch back to the code of the Function, and implement the following:

using System.Net;
 
public static async Task Run(HttpRequestMessage req, TraceWriter log, ICollector outputTable)
{
    log.Info("C# HTTP trigger function processed a request.");
 
    // Get request body
    dynamic data = await req.Content.ReadAsAsync<object width="300" height="150">();
    var temp = data.ToString();
    log.Info(temp);
 
    outputTable.Add(
    new MyTableEntity() 
        { 
            PartitionKey = "Test", 
            RowKey = DateTime.Now.ToString("yyyyMMddHHmm"), 
            Data = temp
        }
    );
 
    return req.CreateResponse(HttpStatusCode.OK, string.Empty);
}
 
public class MyTableEntity
{
    public string RowKey { get; set;}
    public string PartitionKey {get; set;}
    public string Data { get; set;}
}
 
 
Now get the endpoint of your Azure Function, and switch over to the Event Grid Subscriptions blade. Here you will find the subscription we created earlier for Event Hubs. Create a new Event Subscription. Set the Topic Type to <em>EventHubs Namespace</em>, choose the topic created earlier, and set the Subscriber Endpoint to the endpoint of the Function.
[caption id="attachment_1786" align="aligncenter" width="300"]<a href="https://blog.eldert.net/wp-content/uploads/2017/08/event-subscriptions-grid-preview-u-refresh-a-.png"><img class="wp-image-1786 size-medium" src="https://blog.eldert.net/wp-content/uploads/2017/08/event-subscriptions-grid-preview-u-refresh-a--300x107.png" alt="Create subscription for Azure Function" width="300" height="107" /></a> Create subscription for Azure Function[/caption]
 
And that's it! We can now send in a message to the Event Hubs, for example using ServiceBus360. If the message is sent to the <em>Notifications</em> Event Hub it will be delivered to the Logic App sending out an email, and to the Function placing it in the Storage Table.
[caption id="attachment_1787" align="aligncenter" width="300"]<a href="https://blog.eldert.net/wp-content/uploads/2017/08/machine-generated-alternative-text-send-events-.png"><img class="wp-image-1787 size-medium" src="https://blog.eldert.net/wp-content/uploads/2017/08/machine-generated-alternative-text-send-events--300x259.png" alt="Send message to Event Hub" width="300" height="259" /></a> Send message to Event Hub[/caption]
[caption id="attachment_1788" align="aligncenter" width="300"]<a href="https://blog.eldert.net/wp-content/uploads/2017/08/machine-generated-alternative-text-outtable-x-e.png"><img class="wp-image-1788 size-medium" src="https://blog.eldert.net/wp-content/uploads/2017/08/machine-generated-alternative-text-outtable-x-e-300x37.png" alt="Message in Storage Table" width="300" height="37" /></a> Message in Storage Table[/caption]
[caption id="attachment_1789" align="aligncenter" width="283"]<a href="https://blog.eldert.net/wp-content/uploads/2017/08/machine-generated-alternative-text-high-priority.png"><img class="wp-image-1789 size-full" src="https://blog.eldert.net/wp-content/uploads/2017/08/machine-generated-alternative-text-high-priority.png" alt="Email sent from Logic App" width="283" height="241" /></a> Email sent from Logic App[/caption]
 
As you can see, Event Grid makes it easy for us to route our events across services. And with more services being integrated out of the box, it will make handling events a breeze! For even more samples, have a look <a href="https://docs.microsoft.com/en-us/azure/event-grid/monitor-virtual-machine-changes-event-grid-logic-app">at the documentation</a>, or read this <a href="https://blog.steef-jan-wiggers.com/2017/08/building-reactive-event-driven-solutions-new-azure-event-grid-service/">blogpost by Steef-Jan</a>.
</object>