Last week the new Gartner Magic Quadrant for Enterprise Integration Platform as a Service (EiPaaS) was published, listing Microsoft in the coveted leader space. Having worked with Azure’s iPaaS products for a long time now, I wholeheartedly agree with this decision, and congratulate all the teams within Microsoft who have been working so hard to get to where we are today. The complete report, with all requirements and results can be found in this report.
When working with Azure Logic Apps, I like to have each Logic App do a single piece of work, as this allows us to mix and match these Logic Apps in various flows. For this demo, we will be using a very simple representation of this, where we have one Logic App which receives the message and send back a response to the original caller, another Logic App which does transformation of the message, and finally a Logic App which calls a backend system. To decouple these Logic Apps we will be using Azure Service Bus topics, providing us with routing capabilities and allowing us to handle downtime more easily.
Now the challenge we were running into, is that we needed to give the response which we received from the backend, back as a response to the client.
Of course, since we have implemented communication between the Logic Apps asynchronously and decoupled by using Service Bus in between, we don’t have a return channel on which we can send the response. In this post, I will show how we can solve this by using Service Bus sessions.
In the previous post, we have seen how we can implement the Message Router pattern when working with Logic Apps. As discussed, Logic Apps are a great fit if you have a limited set of endpoints to which you want to route the message, and if you have a need for various connectors. In this post we will look into another technology to implement this pattern, Azure Service Bus Topics. Topics are a great solution if we want to implement a publish / subscribe mechanism.
- Capability to send our messages to one or more subscriptions in our topic.
- Each subscription represents a virtual queue, from where subscribers can pull their messages, allowing receiving systems to process messages at their own speed.
- Receiver and sender are completely decoupled, so systems can work independently from each other.
- Topics have dead-lettering capabilities built in, so messages are not lost even in case of issues.
- Easily add new subscriptions, so we can quickly on-board new systems.
When implementing software, it’s always a good idea to follow existing patterns, as these allow us to use proven and reliable techniques. The same applies in integration, where we have been working with integration patterns in technologies like BizTalk, MSMQ etc. These days we are working more and more with new technologies in Azure, giving us new tools like Service Bus, Logic Apps, and since recently Event Grid. But even though we are working with new tools, these integration patterns are still very useful, and should be followed whenever possible. This post is the first in a series where I will be showing how we can implement integration patterns using various services in Azure.
Yesterday Microsoft announced their newest service on Azure, and it is called Azure Event Grid. With this new service, we now have event based serverless routing, from any source, to any destination. Of course, we all love integration, and Azure Event Grid brings a whole new world of possibilities. The service is currently in public preview, and we already have various publishes and event handlers to our disposal, and more will be rolling out over the coming months. In the end, expect every service within Azure to have a connection to Event Grid.
In this post, I will show how we can use Visual Studio to write Azure Functions and use these in our Logic Apps. Azure Functions, which went GA on November 15th are a great way to write small pieces of code, which then can be used from various places, like being triggered by a HTTP request or a message on Service Bus, and can easily integrate with other Azure services like Storage, Service Bus, DocumentDB and more. We can also use our Azure Functions from Logic Apps, which gives us powerful integrations and workflow, using the out of the box Logic Apps connectors and actions, and placing our custom code in re-usable Functions.
Writing Azure Functions from Visual Studio
However, most developers will be used to develop from Visual Studio, with its great debugging abilities and easy integration with our source control. Luckily, earlier this month the preview of Visual Studio Tools for Azure Functions was announced, giving us the ability to write Functions from our beloved IDE. For this post I used a machine with Visual Studio 2015 installed, along with Microsoft Web Developer Tools and Azure 2.9.6 .NET SDK.
Last week I attended Integrate 2016 in London, the biggest Microsoft integration event this year, organized by BizTalk360. The outcome was almost 400 attendees, and there were sessions from the Microsoft Product Group, industry leaders and MVP’s.
The major take-away I have from the event, is that Microsoft now has a great vision on the future of integration, which I felt was missing the last couple of years. Now though, they recognize that even though the cloud is a great asset, on premise is not going away for a long time. Also Microsoft has now officially announced that their on-premise integration solution will be BizTalk, which has not been getting a lot of love lately.
In my my previous post, I showed how we can use a WebJob to process a Service Bus queue and store the data in an Azure SQL database. This was pretty simple to set up, but it did require a good understanding of how to connect with these and process the data. Sometimes however we just want to do a quick integration without needing to set up all this plumbing. Recently Microsoft announced a new feature called Azure Functions, with now makes this possible. Azure functions can be used to create a small function which can run stand-alone, or be called from other applications, for example from a logic app, as has been described here by Sandro Pereira. Azure Functions provide out of the box connections for triggers, input and output to a lot of other Azure features, including Event Hubs, Service Bus, Azure Storage and DocumentDB. In this post I will show how we can process our message from the queue we created in this blogpost, and store it in an Azure Storage table. We will start by creating a new Function App in the portal.
This is the fifth post in my series on Integration of Things. In this post I showed how you can send messages from a Raspberry Pi 2 into a Service Bus Queue, and in our previous blogpost we have set up a library for connecting to an Azure SQL database. Today I will explain how we can use a WebJob to retrieve the messages from the queue and send them to our database. The code for this blogpost can be found here.
A WebJob is a simple way to set up a background job, which can process continuously or on a schedule. WebJobs differ from a cloud service (which we discussed in this blogpost) as it gives you get less fine-grained control over your processing environment, making it a more true PaaS service.
We will need a Web App to host our WebJob, so lets create one in the Azure Portal. You can create a new Web App by going to App Services, and selecting New.
This is the fourth post in my series on Integration of Things. In this post, we will use Entity Framework Code First to set up an Azure Sql database, which will later on be filled with the data we receive from our Service Bus Queue. As we will want to access this database from multiple projects, we will add it to the DataTypes Class Library we created in the previous blogpost. The code for this blogpost can be found here.
First we will create an empty database in Azure.