Is Azure storage Copy Blob From URL operation atomic - Stack Overflow

I have a C# application that uploads data to an Azure BLOB storage container as a temp file and then renames the temp file to some well-known name. Since the file's data can be used elsewhere immidiately, I rely on the rename operation being atomic (either succeeding or failing completely with no intermediate state).

In ADLS, there is a native rename API which I know to be atomic. Since BLOB storage does not have such an API, the rename can be executed by copying "temp" to a file with the new name and then deleting "temp".

My question:

Is the BLOB storage Copy Blob From URL API atomic (or can it be made atomic with some conditions)? I read the documentation here: but can't seem to find a definitive answer.

I have a C# application that uploads data to an Azure BLOB storage container as a temp file and then renames the temp file to some well-known name. Since the file's data can be used elsewhere immidiately, I rely on the rename operation being atomic (either succeeding or failing completely with no intermediate state).

In ADLS, there is a native rename API which I know to be atomic. Since BLOB storage does not have such an API, the rename can be executed by copying "temp" to a file with the new name and then deleting "temp".

My question:

Is the BLOB storage Copy Blob From URL API atomic (or can it be made atomic with some conditions)? I read the documentation here: https://learn.microsoft.com/en-us/rest/api/storageservices/copy-blob-from-url?tabs=microsoft-entra-id but can't seem to find a definitive answer.

• azure-blob-storage

Share Improve this question Follow asked Jan 15 at 6:55 Uri ShapiraUri Shapira 37111 gold badge55 silver badges1919 bronze badges 4

• 1 Azure Blob Storage Copy Blob From URL API itself does not guarantee atomicity. The operation involves copying data from a source blob (or URL) to a destination blob, but it does not provide built-in atomicity like a native rename operation would. – Venkatesan Commented Jan 15 at 7:23
• If you want to ensure the source is not modified take a lease on it during the copy operation – Peter Bons Commented Jan 15 at 7:25
• @PeterBons why would the source be modified? I'm not really concerned about that happening, my main concern is the copy operation failing in the middle and leaving a corrupted renamed file which is visible externally (not hidden) – Uri Shapira Commented Jan 15 at 7:27
• @UriShapira Check the below answer. – Venkatesan Commented Jan 15 at 8:01

Add a comment  | 

1 Answer 1

Sorted by: Reset to default 0

Is Azure storage Copy Blob From URL operation atomic

Azure Blob Storage Copy Blob From URL API itself does not guarantee atomicity. The operation involves copying data from a source blob (or URL) to a destination blob, but it does not provide built-in atomicity like a native rename operation would.

You can use the below code by checking the CopyStatus of the destination blob until it reaches CopyStatus.Success using C#.

Code:

using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using System;
using System.Threading.Tasks;

public class BlobStorageExample
{
    private readonly BlobServiceClient _blobServiceClient;

    public BlobStorageExample(string connectionString)
    {
        _blobServiceClient = new BlobServiceClient(connectionString);
    }

    public async Task RenameBlobAsync(string containerName, string tempBlobName, string newBlobName)
    {
        var containerClient = _blobServiceClient.GetBlobContainerClient(containerName);
        var tempBlobClient = containerClient.GetBlobClient(tempBlobName);
        var newBlobClient = containerClient.GetBlobClient(newBlobName);

        var copyOperation = await newBlobClient.StartCopyFromUriAsync(tempBlobClient.Uri);

        BlobProperties properties;
        do
        {
            // Retrieve the properties of the destination blob to check the copy status
            properties = await newBlobClient.GetPropertiesAsync();
            Console.WriteLine($"Copy status: {properties.CopyStatus}");
            await Task.Delay(1000);
        } while (properties.CopyStatus == CopyStatus.Pending);  // Continue polling until the copy completes

        if (properties.CopyStatus == CopyStatus.Success)
        {
            await tempBlobClient.DeleteIfExistsAsync();
            Console.WriteLine("Blob renamed successfully.");
        }
        else
        {
            Console.WriteLine($"Copy failed with status: {properties.CopyStatus}");
        }
    }

    public static async Task Main(string[] args)
    {
        string connectionString = "";  // Replace with your Azure Blob Storage connection string
        string containerName = "data";  // Replace with your blob container name
        string tempBlobName = "test.pdf";  // Replace with the temporary blob name
        string newBlobName = "sample.pdf";  // Replace with the new blob name

        // Create an instance of the BlobStorageExample class
        var blobStorageExample = new BlobStorageExample(connectionString);

        // Call the RenameBlobAsync method to rename the blob
        await blobStorageExample.RenameBlobAsync(containerName, tempBlobName, newBlobName);
    }
}

Output:

Copy status: Success
Blob renamed successfully.

If you don't need this above method also, you can use the Azure data lake gen2 storage account to rename the file and folders using REST API.

Reference: Rename File (REST API) - Azure Storage | Microsoft Learn