Welcome to our deep dive into the world of Microsoft Azure's secure information management – the Azure Key Vault. In this digital era, where data breaches and security concerns are part and parcel of every online venture, having an iron-clad method to manage sensitive information is crucial. As developers, it's our responsibility to ensure that our applications are not just robust and efficient, but also safe and trustworthy. This blog aims to guide you on a journey through integrating Azure Key Vault within your .NET Core web API project, ensuring that your secrets remain just that – secret.

But first, what is Azure Key Vault?

Azure Key Vault is a cloud service provided by Microsoft Azure designed specifically for safeguarding cryptographic keys and secrets (like connection strings, API keys, or any sensitive data). Think of it as a bank vault where you can store your most valuable digital assets. Instead of embedding these secrets directly in your code, which can lead to potential exposure, Azure Key Vault provides a centralised and secure storage system. This way, only applications and users with the appropriate permissions can access them.

Stay with us as we navigate through the setup, intricacies, and best practices of incorporating Azure Key Vault into your .NET Core Web API projects. Whether you're a seasoned Azure developer or just getting your feet wet in the vast ocean of cloud computing, this blog aims to provide insights and steps to bolster the security of your digital solutions.

Getting started & prerequisites

Prerequisites

• Basic understanding of C#.
• Basic experience using Microsoft Visual Studio.
• Basic experience using Microsoft Azure.

Getting started

We will get started by creating a .NET Core project in Microsoft Visual Studio. For demonstration purposes, we will use the good old weather forecast web API template provided by Microsoft.

1. In the project templates, search for "API" and select the "ASP.NET Core Web API" option.
2. Name the project; I've named mine "AzureKeyVaultExample".
3. On the additional information window, leave everything that's pre-selected and click "Create".

With our demo project now in place, our next move is to deploy it to Microsoft Azure. This deployment isn't merely a formality. By publishing the application, we can incorporate it into a resource group on Azure. Furthermore, if you're aiming to use Azure Key Vault, it's a prerequisite to have your application registered within Azure. Follow these steps to deploy your application to Azure:

Deploying to Azure

Right-click on the main project file and click "Publish".

Next, click on "Add a publish profile", select "Azure" and click "Next".

Now click "Azure App Service (Windows)", followed by clicking "Next".

Click "Create new".

Within the new app service window, we are going to create a new resource group.

Firstly, select whichever Azure subscription you wish to use.

Next, in the "Resource group" section, click on "New".

Name the resource group as you wish. I've named mine "RG-Azure-Key-Vault-Demo". For my objects in Azure, I like to abbreviate the object type before the name for easy recognition. In this case, "RG" is "Resource Group".

Now, under the "Hosting Plan" section, click "New".

Within the hosting plan window, make your selections and click "Next" when you're done.

With the new app service created, we can now click on "Create".

In the publish window, select the new app service we just created and click "Next".

After clicking next, you will be taken to the API management screen (the tab highlighted in yellow). We are going to create a new one for this project, so click "Create new".

In the new API management screen, you'll be asked to provide a name. Provide a name where highlighted in yellow below. After this step, under the "Resource group" section, select the new resource group we just created. Upon finishing this step, in the "API Management service" section, click on "New".

In the new API management service screen, give the service an appropriate name and set the options in the other sections best suited to your needs. When finished, click "Ok".

We are now directed back to the previous window. For now, we can ignore the "API URL suffix" section and click on "Create". This will take a few moments, so feel free to go make a tea or coffee whilst you wait! ☕️.

Next, under the new resource group and API management service we have just created, click on the new API and click "Finish".

Lastly, we are now ready to publish to Azure. Click on "Publish" to continue.

Upon successful publication, a browser window will pop up with the weather forecast template via a live URL. If you add "/WeatherForecast" to the end of the URL, you will be able to see the JSON response via the pre-defined "Get" controller endpoint.

Please note that this URL is live, and you may want to secure it using IP restrictions so that you don't receive unwanted traffic and hit data usage quotas.

With the Visual Studio project now live in Azure, we can now register the application so that when configuring Azure Key Vault, the service is aware of our newly deployed application.

Registering the Application

Assuming you are already logged into the Azure portal, navigate to "App registrations". If you are unsure how to find it, just search for "App registrations" in the top search bar.

Within the "App registrations" menu, click on "New registration."

Give the app registration a name and leave the rest of the details as they are. Click "Register". You can leave the rest of the options at their default values.

That's it for this section at this moment. We will come back to the application registration later in this blog to create a client secret value, while also adding the relevant configuration parameters to our Visual Studio project.

Key Vault Configuration

In this section, we will create an Azure Key Vault and set a secret value that can be retrieved from the key vault.

Navigate to "Key Vaults" within the Azure portal or type "Key Vaults" in the main search bar.

Creating the Azure Key Vault

In the "Azure Key Vaults" menu, click on "Create."

In the next screen, select the subscription you'd like to use. Select the newly created resource group that we created when publishing the application from Visual Studio. Give the new key vault a name and select your desired region. Once you are happy with the rest of the options, click "Review and create."

In the "Review & Create" screen, click on "Create."

Account RBAC Role

Before creating the secret key and value, we need to set a new RBAC role that assigns your Azure account as an administrator within the newly created key vault. To do this, from the main menu in the left sidebar, click on "Access control (IAM)."

In the following screen, click on "Add role assignment."

Within the "Role Assignment" screen, in the search bar, search for "Key Vault Administrator," select the "Key Vault Administrator" option, and click on "Next."

In the following screen, click on "Select members" and search for the name or email associated with your Azure account. After selecting your account, click on "Next."

Next, click on "Review + Assign." After this, you'll be redirected back to the main menu of the key vault.

Creating the Secret Key and Value

It's now time to create the secret key and value. Within the key vault's main menu, click on the "Secrets" option in the left sidebar.

In the following window, click on "Generate / Import."

In the "Create a Secret" screen, give the secret a key value. This key value will be used when retrieving the secret value from the key vault. Set the secret value and click on "Create." All other options can remain at their default settings.

After clicking on "Create," you will be directed to the secrets menu of the key vault. A message should display stating that the secret has successfully been created.

Retrieving the Secret Value from Azure Key Vault

Now that we have configured the application's key vault, we can start accessing the secret value via the Visual Studio project we previously created and deployed to Azure.

To use Azure Key Vaults in our application, we need to install some packages. Return to the Visual Studio project that we deployed to Microsoft Azure earlier and open the NuGet Package Manager.

With the "Browse" tab selected, search for "Azure.Identity". When you find the option, select it. In the right-side window, choose the "Latest stable" version and click "Install".

After installing the above package, type "Azure.Security.KeyVault.Secrets" into the search bar. Install the latest stable package version.

We now need to install our final package. In the search bar, type "Azure.Extensions.AspNetCore.Configuration.Secrets", then select and install the latest stable version of the package.

Next, we'll start storing our key vault configuration parameters within our Visual Studio app.

In your project's main directory, navigate to the appsettings.json file and add the following:

"AzurekeyVaultConfig": {
  "KVUrl": "",
  "KVClientId": "",
  "KVTenantId": "",
  "KVClientSecretId": ""
 },

To populate the empty values we just added, the next steps will involve going back to our application registration in Azure that we completed earlier to create a new client secret.

Navigate to "App registrations" within the Azure portal. Find and click on the application you previously registered to go to its overview screen.

In the app registration's main menu, click on "Certificates & secrets" from the left sidebar. Next, click on "New client secret".

In the new window that appears, name the new client secret. Set the expiration to your desired value; I've left it at the default of 180 days. Click "Add".

After the new client secret is created, make a note of the secret value in the "Value" column of the table:

Return to the appsettings.json file and set this value as the "KVClientSecretId" key:

"AzurekeyVaultConfig": {
  "KVUrl": "",
  "KVClientId": "",
  "KVTenantId": "",
  "KVClientSecretId": "Client-Secret-Id-Goes-Here"
 },

Next, navigate to the overview tab of the app registration. We will be need to make a note of the paramaters "Application (client) ID" & "Directory (tenant) ID":

In the appsettings.json file within the Visual Studio project, set the "Application (client) ID" as the value for "KVClientId" and the "Directory (tenant) ID" as the value for "KVTenantId":

"AzurekeyVaultConfig": {
  "KVUrl": "",
  "KVClientId": "Client-ID-Goes-Here",
  "KVTenantId": "Tenant-ID-Goes-Here",
  "KVClientSecretId": "Client-Secret-Id-Goes-Here"
 },

Lastly, to assign the value for the key "KVUrl," we need to obtain the URL of the new Azure Key Vault we created earlier.

To do this, navigate to the Key Vaults console within the Azure portal. In the Key Vaults window, search for and find the key vault you created earlier, then click on it. After completing this step, you'll be directed to the overview screen of the key vault:

Copy the value of the parameter "Vault URI" and paste it as the value for the "KVUrl" key within the Visual Studio appsettings.json file:

"AzurekeyVaultConfig": {
  "KVUrl": "Web-API-URL-Goes-Here",
  "KVClientId": "Client-ID-Goes-Here",
  "KVTenantId": "Tenant-ID-Goes-Here",
  "KVClientSecretId": "Client-Secret-Id-Goes-Here"
},

Modifying program.cs

Now that we have our configuration parameters in place, let's modify the program.cs file within the main directory of the Visual Studio project. This will allow us to load the newly added key vault configurations within the application's main entry point.

Add the following lines to program.cs:

{string keyVaultUrl = builder.Configuration["AzurekeyVaultConfig:KVUrl"];
string tenantId = builder.Configuration["AzurekeyVaultConfig:KVTenantId"];
string clientId = builder.Configuration["AzurekeyVaultConfig:KVClientId"];
string clientSecretId = builder.Configuration["AzurekeyVaultConfig:KVClientSecretId"];

builder.Configuration.SetBasePath(builder.Environment.ContentRootPath)
    .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
    .AddEnvironmentVariables();

var clientSecretCredential = new ClientSecretCredential(tenantId, clientId, clientSecretId);
var secretClient = new SecretClient(new Uri(keyVaultUrl), clientSecretCredential);
}

Next, we will add a dependency injection line to our program.cs file. This enables our controller to access our repository methods without having to create a new instance of our repository class each time a user hits our controller endpoint. To achieve this, add the following lines of code underneath the commented line "//Add services to the container":

builder.Services.AddScoped<IRepository, Repository>();

Please note that adding the lines above will temporarily generate an error because the class and interface have not been created yet. We will add them shortly.

Finally, if Visual Studio does not automatically import the necessary namespaces after you add the above lines of code, add the following import statements to the top of the file:

using Azure.Identity;
using Azure.Security.KeyVault.Secrets;

Your overall program.cs file should now look something like this:

using Azure.Identity;
using Azure.Security.KeyVault.Secrets;

var builder = WebApplication.CreateBuilder(args);

string keyVaultUrl = builder.Configuration["AzurekeyVaultConfig:KVUrl"];
string tenantId = builder.Configuration["AzurekeyVaultConfig:KVTenantId"];
string clientId = builder.Configuration["AzurekeyVaultConfig:KVClientId"];
string clientSecretId = builder.Configuration["AzurekeyVaultConfig:KVClientSecretId"];
builder.Configuration.SetBasePath(builder.Environment.ContentRootPath)
    .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
    .AddEnvironmentVariables();

var clientSecretCredential = new ClientSecretCredential(tenantId, clientId, clientSecretId);

var secretClient = new SecretClient(new Uri(keyVaultUrl), clientSecretCredential);

// Add services to the container.
builder.Services.AddScoped<IRepository, Repository>();

builder.Services.AddControllers();
// Learn more about configuring Swagger/OpenAPI at https://aka.ms/aspnetcore/swashbuckle
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

var app = builder.Build();

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();

app.UseAuthorization();

app.MapControllers();

app.Run();

Implementing the Repository and Interface Methods

In the last step of modifying program.cs, we added a dependency injection line. We will now proceed to create the repository and interface that correspond to this line of code.

Create a folder in the main directory of your Visual Studio project and name it "Repositories." Create another folder and name it "Interfaces."

In the "Repositories" folder, create a new class file named "Repository.cs."

Return to the "Repository.cs" class file and ensure it inherits from the "IRepository" interface that we just created, like so:

public class Repository : IRepository
{

}

After adding the repository and interface, we can now reference them within our program.cs file. At the top of the file, add the following import statements, making sure to use your project's name:

using AzureKeyVaultExample.Interface;
using AzureKeyVaultExample.Repository;

Navigate back to the newly created "Repository.cs" file and declare a field of type IConfiguration. Initialize the constructor as follows:

public class Repository : IRepository
{
    private readonly IConfiguration _configuration;
    public Repository(IConfiguration configuration)
    {
        _configuration = configuration;
    }
}

The purpose of this field is to allow us to read various values from the appsettings.json file that we will need for the key vault configuration.

Next, in our Repository class file, we will add a method that retrieves the secret value we created within Azure Key Vaults. This method will later be used in our controller class file.

Add the following lines of code to "Repository.cs":

public string GetSecretValue(string secretName)
{
    string keyVaultUrl = _configuration["AzurekeyVaultConfig:KVUrl"];

    var client = new SecretClient(new Uri(keyVaultUrl), new DefaultAzureCredential());
            
    try
    {
        KeyVaultSecret secret = client.GetSecret(secretName);
        return secret.Value;
    }
    catch (Exception ex)
    {
        // Handle the exception as appropriate for your application.
        Console.WriteLine($"Error retrieving secret: {ex.Message}");
        return null;
    }
}

In the above code snippet, the URL for the key vault is read from our appsettings.json file. The secret name is passed from our controller endpoint that will call this method, which we will implement shortly.

Next, navigate back to the interface file "IRepository.cs" and ensure you add the new method we implemented in "Repository.cs". This ensures that it adheres to the repository contract:

public interface IRepository
{
    public string GetSecretValue(string secretName);
}

Since this is a demo on how to use Azure Key Vaults, we will not create a wrapper class for Microsoft's Azure Key Vaults class objects. However, in a production application, it would be advantageous to use a wrapper class for controlled interaction, testing purposes, and adaptability. Stay tuned for another blog where we will discuss implementing Azure Key Vaults in a TDD approach.

Modifying the Controller

In this section, we will modify the file "WeatherForecastController.cs" so that the weather forecast endpoint will retrieve the secret value stored in Azure Key Vaults, provided the correct key value is supplied.

Open "WeatherForecastController.cs" and add the following code. This code will allow the constructor of the class to use the repository dependency injection we created earlier, along with field declarations for the repository and configuration interfaces:

private readonly ILogger<WeatherForecastController> _logger;
private readonly IRepository _repository;
private readonly IConfiguration _configuration;

public WeatherForecastController(ILogger<WeatherForecastController> logger, IRepository repository, IConfiguration configuration)
{
    _logger = logger;
    _repository = repository;
    _configuration = configuration;
}

Next, navigate to the "Get" method in the controller and replace the existing method with the following code:

[HttpGet(Name = "GetWeatherForecast")]
public IEnumerable<WeatherForecast> Get(string secretName)
{
    if (_repository.GetSecretValue(secretName) != null)
    {
        return Enumerable.Range(1, 5).Select(index => new WeatherForecast
        {
            Date = DateTime.Now.AddDays(index),
            TemperatureC = Random.Shared.Next(-20, 55),
            Summary = Summaries[Random.Shared.Next(Summaries.Length)]
        })
        .ToArray();
    }
    else
    {
        return null;
    }
}

Running the Application

With the "Get" controller updated, it's time to test whether retrieving the secret is working as expected. Make sure to note down the key of the secret you created in the Azure Key Vault earlier, so that you can pass it as an argument to the controller endpoint:

Run the application. Once the Swagger main page is loaded, click on the "Try it out" button:

After that, make sure to type in the correct name of the key for the secret we created in the key vault and click on "Execute". Within a few seconds, the weather forecast JSON results will be returned to you. If you provide an invalid key name, you should receive an error 204 message:

That concludes this blog post. In a future blog, I will implement this setup in a more TDD-focused manner, along with some coding best practices and principles. Until then, goodbye! :)