Skip to content

Querying More than 1000 Azure Table Entries

August 22, 2011

A typical Azure table operation that accesses all entries of a table, using the Azure SDK CloudTableClient API, may look like this:

TableServiceContext context = client.GetDataServiceContext();
var query = (from r in context.CreateQuery<EntityType>(tableName) select r);
foreach (var entry in query.Execute())

But you might be surprised to find that if the table contains more than a thousand entries, the foreach loop only sees the first 1000. I only noticed it when I happened to put a counter in the loop, and then suspected that 1000 seemed too round to be coincidental. I don’t know an easy way to discover the number of entries in a table without querying the entire table and counting them, so couldn’t check the value against a known amount.

This happens because the query to the Azure table store returns data in pages of size 1000 (or, possibly less, as other events may trigger a page break, such as the size of data transferred or a timeout). If you’re using the REST API to access the table, the HTTP response header returns a continuation token. If you resubmit the same query with this continuation token in the header of the request, it will resume the query where it left off, returning the next page.

When using the SDK API, the presence of continuation tokens is not obvious, because it’s not explicitly exposed by the API (although there is a way to access them). The good news is that the SDK API knows how to handle the continuation so you don’t need to, by letting a CloudTableQuery object execute the query for you. This is accomplished with an extra step, inserted at line 4:

TableServiceContext context = _client.GetDataServiceContext();
var query = (from r in context.CreateQuery<EntityType>(tableName) select r);
var cloudTableQuery = query.AsTableServiceQuery<EntityType>();
foreach (var entry in cloudTableQuery.Execute())

AsTableServiceQuery() is an extension function that returns a new CloudTableQuery object that wraps the original query object with the code to handle continuations.

No comments yet

Leave a Reply

Your email address will not be published. Required fields are marked *