One of my favorite services in Azure is Functions, which allow you to create serverless micro-services. Triggered by events, after which they run their code, Functions are perfect for the event-driven architectures we strive for these days. These events can come from various sources, like when a message is available in Service Bus, timers, an event sent from Event Grid, etc. However, the one we still use a lot is an HTTP trigger, where we expose the Function as a REST endpoint, available for consumers to call into.
Often we will have an architectural guideline, that every REST endpoint needs to be exposed through Azure API Management. Therefor, we expose these HTTP triggered Functions via API Management as well. We also have guidelines that everything is deployed as Infrastructure as Code, so we do this through ARM templates. In this post we dive into the security side of this, and how to set this up in ARM.
In a previous blog post, we have seen how to retrieve secrets from Azure Key Vault from an API Management policy. This works great, however, we might start to run into throttling due to the limitations which Key Vault imposes.
This might be due to having an API exposed which we need to call frequently, or because we retrieve secrets from Key Vault in multiple implementations, all of which adds to the restrictions. Luckily, API Management has another policy expression which helps us out here, namely the caching policy.
When working with Azure API Management, often we need to include secrets in our policies. For example, we may need to send a password in our authentication header, or to validate a key in a JWT token. There are several options to store these secrets. We could hardcode them into our policy, however this would mean anyone with access to our API Management instance could read them. An not just them, but also everyone who can look into our source control. because we deploy our policies as Infrastructure as Code.
The second option is to place the secret in a named value. This even provides us with the option to set the value as a secret, meaning it will not show the actual value in the overview.
However, anyone with access to API Management can still come into the instance, and untick the secret option, and grab the secret. Consequently, this is still not a good option, as we want the management of our secrets to be separate from our API Management administration. Therefor, we will instead store the secret in Azure Key Vault, and retrieve it in our policy.
We use Azure API Management quite extensively at our clients, where we use this service whenever our services (APIs) go across application boundaries. Basically, API Management implements a facade in front of all our services. Any consumer, whether they are internal or external, uses this gateway to communicate with our services.
When we look at the messaging space on Azure, we have a wide variety of options, each with it’s own strengths and weaknesses. For example, we have Event Hubs, which provides capabilities for processing large amounts of data streams, Service Bus for enterprise messaging, and Event Grid for reactive eventing.
In this blog post, we will look into the the Publish – Subscribe pattern, and how we could implement this using Azure services. For this scenario, we receive messages from one or multiple sources, which we need to route to one or more subscribers, based on properties or content. Of the different messaging offerings, two of these jump out for this scenario, Service Bus and Event Grid. Let’s have a look at when to use which, and the reasons behind these decisions.
When working with Azure, we should always put our secrets into a secure store, such as Key Vault. This ensures that we can limit who can see the values of our secrets, while still being able to work with them. How we work with these secrets is different over the various services, and in this article we will focus on Logic Apps, while other services will be explained in their own posts later on.
In Logic Apps we often will need some sort of secret, for example a subscription key for API Management, or a SAS key for Event Grid. In the scenario for this blog post we are going to send our secret to a RequestBin endpoint, so we can see that we indeed get the correct value.
We use Key Vault extensively in our solutions, to store any secrets we might need. For example in an API through code, in Azure Functions via the application settings, or in a Logic App through a REST call. If you go to your secrets in Key Vault, you will notice that the link to the secret includes a version number, in the format of https://kv-we-retrieve-kv-secret.vault.azure.net/secrets/MySecretValue/80df3e46ffcd4f1cb187f79905e9a1e8.
Of course, this is great if we want to reference a specific version of a secret. However, often we will just want to reference the latest version, so we stay up to date even when the secret has been changed, for example because it is a rotating password.
It turns out, this is very easy, without the need to update the version number in all our applications whenever a new version is created. This is done by just omitting the version number from our link! So the will instead look like https://kv-we-retrieve-kv-secret.vault.azure.net/secrets/MySecretValue/.
Important to notice is the trailing slash ( / ), which needs to be included, otherwise you will just get a 404 error.
Being a big fan of the Infrastructure as Code (IaC) paradigm, I create all my Azure resources as Azure Resource Manager (ARM) templates. Using an IaC approach, our Azure environments are described as code, allowing us to deploy services in an automated and consistent manner. Often different resources reference other resources, which can introduce the need for authentication. Sometimes we can use Azure Active Directory identities for this (explained more detailed in a later post), however at other times this may require some other form of secrets to set up communication.
Of course, we could create a resource, get the secrets out manually, and then pass them into the referencing resource’s template. However, the idea of Infrastructure as Code, especially when combined with a CI/CD strategy, is to have a minimal amount of manual steps. So instead, let’s have a look at how we can retrieve these secrets from our ARM templates. Consequently, this allows us to set these secrets at deployment time, without any manual interference needed. This article is the first in a series of blog posts, each focusing on a different service, starting with Azure Storage access keys in this post.