Modern Web Application - Azure Blob Storage for Uploaded Files

File upload is a very common and critical feature for many web applications. It allows website users to uploaded images and documents to the web server, which will then be displayed in web pages or apps.  

Classic monolith web application

In a classic monolith web application, web server usually stores all the uploaded files. The high-level process looks like the following:

The File Store/Retrieve Handler in the above diagram usually just stores the files locally and will call native I/O APIs to store or retrieve file from Web Server disk.

This architecture works well with small applications but will struggle to meet big online applications requirement as it has the following limitations:

• Hard to scale. Usually the disk size in the web application is limited 
• I/O heavy on Web Server. I/O read and write operations are required to handle web requests frequently. 
• Potential security issues. Files permissions are coupled with web application executable permissions and sometimes can be leveraged by hackers.
• Coupled and complex application. The file storage is bundled with web application program storage. It will become difficult to migrate and maintain.
• Higher cost. We don't need I/O optimized storage for user uploaded files compared with the web application storage. To save those files on high performance web application disk is a waste of money.

Fortunately, with Cloud platforms like Azure, we can easily resolve these problems. I'm using Azure for this article, the concept can apply to other cloud platforms as GCP, AWS, Alibaba Cloud, etc. 

Modern scalable web application on Cloud

To address these issues in the monolith web application, we can migrate it to Cloud. For image or document file storage, we can use Azure Blob Storage; for web application hosting, we can use Azure App Service (elastic and containerization option is also available).

The following diagram shows the revised file upload and retrieve sequences:

In this architecture, Azure App Service application calls Azure Blob Storage APIs to store file into blob storage container.  Azure Blob Storage API also returns the URLs for the stored object file. The URL file will be then passed back to the user client. Users can then use the absolute Azure Blob Storage file object URL to view or download the application. 

Enhanced security

In this architecture, Azure Blob Storage file URLs are directly exposed. This means all users that know the absolute URL or each blob  file can directly access it and container needs to be configured as publicly accessible. 

What if you want to control the permissions to these resources in Azure Blob Storage?

There are two main approaches:

1. Integrate Azure Active Directory into your web applications and configure IAM at container using Azure Active Directory.
2. Implement another logic layer for file upload and download in your web application and manage permissions in your applications. In this approach, connections to Azure Blob Storage endpoints are only exposed to the web application.

In Kontext (this website), I'm using the second approach. There is a web API exposed to retrieve uploaded files from Azure Blob Storage. 

For example, this URL https://kontext.tech/api/flex/medias/obj-1713 will returns the following image:

The absolute URL for this image on Azure is: https://***.blob.core.windows.net/site-images/u-42/2020041102023-microsoft-azure-sql-database-microsoft-sql-server-database.png (storage account name is masked).

No one will be able to directly access this image as I am not making it public; however everybody can access it through this website as the API layer is handling authentication and authorization for user's requests and it also interacts with Azure Blob Storage via APIs. You can find more details in the Code snippets section.

With this approach, the application can decide who can access the image based on their roles or claims in the web applications; when an resource is deleted, we can softly delete it in the application only without physically deleting it on Azure Blob Storage. This also gives me flexibility to handle the following scenarios:

• Requested resources are deleted by users. 
• User has no access to the requested resource.

For example, https://kontext.tech/api/flex/medias/obj-100000713 shows the following customized SVG image as it doesn't exist on Kontext:

The diagram for sequence activities now looks like the following:

Code snippets

Azure SDK provides APIs to access Azure Blob Storage resources easily. At the moment, the following SDK or tools are available to interact with this service:

Fro this article, I'm going to use .NET (ASP.NET Core) to show you how to write files into blob storage or retrieve data from it. 

Prerequisites 

Create a ASP.NET Core 3 web application with reference to Azure.Storage.Blobs package in the project file:

<ItemGroup>
    <PackageReference Include="Azure.Storage.Blobs" Version="12.2.0" />
</ItemGroup>

We will use this .NET library to access Azure Blob Storage.

Create a Azure Blob Storage container using Azure Portal. Get a connection string for read data from and write data into the container.

The connection string looks like the following:

DefaultEndpointsProtocol=https;AccountName=youraccountname;AccountKey=gkd******==;EndpointSuffix=core.windows.net

You can retrieve this from your Azure storage account Access keys in Settings:

Write file into Azure Blob Storage

The following code snippet shows you how to write a file upload HTTP stream into the storage.

public async Task<(bool IsSuccessful, string Path, string PhysicalPath, string Error)> UploadFileAsync(string fileName, string fileType, Stream stream, object bucket)
        {
            var actualFileName = fileName.Replace('\\', '_').Replace('/', '_').Replace(" ", string.Empty);
            try
            {
                BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
                var blobFilePath = $"{bucket}/{actualFileName}";
                var blobContainer = blobServiceClient.GetBlobContainerClient(containerName);
                var blobClient = blobContainer.GetBlobClient(blobFilePath);
                await blobClient.UploadAsync(stream);
                await blobClient.SetMetadataAsync(new Dictionary<string, string> { { "userid", bucket.ToString() } });
                stream.Close();
                return (true, blobFilePath, blobClient.Uri.ToString(), null);
            }
            catch (Exception ex)
            {
                return (false, null, null, ex.ToString());
            }

        }

In the above code snippet, I used one parameter bucket for each of web users to ensure the blob file path is unique to each user. The stream parameter is the object constructed from the HTTP file upload stream. 

BlobClient instance will return the full URI of the uploaded file which can be then directly accessed. I also called SetMetadataAsync function to tag the file uploaded. Tags can help us to link the stored files with other entities in your website applications. 

Download file from Azure Blob Storage

The following code snippet shows how to download file from the storage using BlobClient object. The returned BlobDownloadInfo object has attribute Value.Content which is an instance of Stream class that can be directly used in your HTTP handler. 

public async Task<(bool IsSuccessful, Stream Stream, string Error)> DownloadFileAsync(string fileName, string fileType, string virtualFilePath)
        {
            try
            {
                BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
                var blobContainer = blobServiceClient.GetBlobContainerClient(containerName);
                var blobClient = blobContainer.GetBlobClient(virtualFilePath);
                var download = await blobClient.DownloadAsync();
                return (true, download.Value.Content, null);
            }
            catch (Exception ex)
            {
                return (false, null, ex.ToString());
            }
        }

Delete file from Azure Blob Storage

Similarly, you can also delete file from the storage using the following code snippet:

 public async Task<(bool IsSuccessful, string Error)> DeleteFileAsync(string fileName, string fileType, string filePath)
        {
            try
            {
                BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
                var blobContainer = blobServiceClient.GetBlobContainerClient(containerName);
                var blobClient = blobContainer.GetBlobClient(filePath);
                await blobClient.DeleteIfExistsAsync();
                return (true, null);
            }
            catch (Exception ex)
            {
                return (false, ex.ToString());
            }
        }

BlobClient.DeleteIfExistsAsync function is used to delete the file from storage.

There are many other operations you can perform using this library. Feel free to explore the documentation on official documentation website. 

Summary

It's very easy to utilize Azure to implement modern web applications that supports massive scalable user upload files. Once storage (files) is segregated form computing (web application program host), we can also deploy the application as package in Azure DevOps piepline. Refer the following documentation for details: Run your app in Azure App Service directly from a ZIP package.

If you have other better approaches to deign user content storage for web applications on Cloud, feel free to post a comment here. 

References

The following article used similar approach as this article but implemented through customized IActionResult and executor:

Efficiently serving Blob Storage in ASP.NET Core | CodeX