Skip to content

Azure Upsert, the Good, the Bad, and the Ugly

September 19, 2011

The latest release of Azure SDK version 1.5 introduces table “upserts.” Upsert is shorthand for “update or insert,” meaning if a record exists, then update it, else insert it. This is a good thing, but I have issues.

The Good

I like this feature because now that upsert is atomic, I may clean up some code that manually dealt with it (and not have to worry about a race condition that might occur if something happened between the two operations). Before the code looked like this:

bool TryInsertOrReplaceEntity(string tableName, MyTableEntity entity)
{
    bool success = false;
    bool tryUpdate = false;
    try
    {
        TableServiceContext context = _client.GetDataServiceContext();
        context.AddObject(tableName, entity);
        context.SaveChangesWithRetries();
        success = true;
    }
    catch (Exception e)
    {
        var inner = e.InnerException as DataServiceClientException;
        if (inner != null && inner.StatusCode == (int)HttpStatusCode.Conflict)
        {
            // Probably not an error:  conflict means entity already exists, so update it.
            // Note: maybe suspect to race condition if delete happens now!
            tryUpdate = true;
        }
        else
        { /* do some error handling */ }
    }
    if (tryUpdate)
    {
        success = TryUpdateEntity(tableName, entity);
    }
    return success;
}
 
public bool TryUpdateEntity(string tableName, MyTableEntity entity)
{
    bool success = false;
    try
    {
        TableServiceContext context = _client.GetDataServiceContext();
        context.AttachTo(tableName, entity, "*");
        context.UpdateObject(entity);
        context.SaveChangesWithRetries();
        success = true;
    }
    catch
    { /* do some error handling */ }
    return success;
}

First, it makes an attempt to insert the entity into the table. If it errors because of a conflict, it assumes this means the entity already exists, and then tries an update instead. An alternate way to do the same is attempt updating first, followed by an insert on failure.

Now, this is replaced with the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
public bool TryUpsertEntity(string tableName, MyTableEntity entity)
{
    bool success = false;
    try
    {
        TableServiceContext context = _client.GetDataServiceContext();
        context.AttachTo(tableName, entity);
        context.UpdateObject(entity);
        context.SendingRequest +=
            (sender, args) => (args.Request).Headers["x-ms-version"] = "2011-08-18";
        context.SaveChangesWithRetries(SaveChangesOptions.ReplaceOnUpdate);
        success = true;
    }
    catch (Exception e)
    { /* do some error handling */ }
    return success;
}

You might think this much more straightforward approach is cause for rejoice. Instead, I have two gripes about this new feature, one minor and one major.

The Bad

Notice the instruction on lines 9-10 in the example above. In order for upsert to work, we must override the default version that the Azure SDK client DLL uses when communicating with the storage service. Also notice that the MSDN page describing upsert makes no mention of this (in the section titled “Insert Or Replace Entity Using the .NET Client”). I discovered this when trying to debug why upsert failed with the message “One of the request inputs is not valid.”

Using fiddler (a tool that lets you snoop on HTTP messages on your computer), I examined the headers and noticed the discrepancy between the “x-ms-version: 2011-08-18” of the samples on the MSDN site, and the “x-ms-version: 2009-09-19” my messages were sending. (Also, I was somewhat confused because today is September 19, 2011 and the 2009-09-19 version looked like a bug in the year, when actually it coincides with a previous version).

Finding this article first would’ve saved time because the sample code shows the need to explicitly override the version (and demonstrates a different way to do it).

It’s very annoying that the 1.5 SDK client DLLs were released without handling the versioning issue for us. What’s the point of the new DLL if not to take advantage of the features that shipped with it? Apparently, this was left out because they “didn’t have time” (quoting an Azure developer). And now that version selection must be explicit in the code, when the next iteration of storage service is released, to take advantage of its features I’ll need to remember to fix this code in addition to updating the libraries.

The Ugly

While the version problem is bad, it’s not the worst part of this new feature. Ultimately, I won’t be able to use it due to one little comment on the MSDN documentation stating:

This operation is not currently supported by the local storage service.

Not currently supported? Usually when testing, we use the local development storage account instead of real Azure storage. There are some advantages to working this way, including the fact that you don’t have to pay to use your local storage like you do to use Azure. There are other advantages too, such as not needing a separate account to isolate test runs from production runs, etc.

Inability to use the upsert feature with local storage means I either must maintain two code paths, one for local storage with the old non-atomic technique, and another for Azure storage using the new API. But this also means my production code is not covered by test cases. Which is pretty much unacceptable (not to mention missing the point of testing).

Next Time

My happiness that Azure added upsert was quickly diminished by the way they seemed to rush the feature out with ill-suited support, both from their client library and their developer tools. Hopefully, they will get a patch out soon to address these issues.

One Comment leave one →
  1. October 24, 2011 6:12 PM

    Hi,
    Thanks a lot for throwing some light on the need to specify the headers specifically. Had been struggling on the same “INput not valid” for a while before finding your blog!
    Regards
    Anshulee

Leave a Reply

Your email address will not be published. Required fields are marked *