Automatic Pagination of Responses in the AWS SDK for .NET (Preview)

As part of our recent preview release of Resource APIs for .NET we have exposed one of the underlying features in the low-level .NET SDK as well.

Many of the AWS APIs that return collections of items have a pagination interface. Rather than return all results at once, the service returns a certain number of results per request, and provides a token in the response to get the next "page" of results. In this way, you can chain requests together using the token to get as many results as you need.

Here’s what that looks like using the SDK for .NET to get all the IAM users for an account, 20 at a time:

ListUsersResponse response;
ListUsersRequest request = new ListUsersRequest { MaxItems = 20 };

do
{
    response = iam.ListUsers(request);
    ProcessUsers(response.Users);
    request.Marker = response.Marker;
}
while (response.IsTruncated);

In order to make the resource APIs feel more natural, we built in a mechanism that does something like the above code behind the scenes through an IEnumerable interface. Using the resource APIs, you can get the users like this:

var users = iam.GetUsers();
foreach (var user in users)
{
    Console.WriteLine("User: {0}", user.Name);
}

The first line does not result in a service call. No service calls will be made until your code starts iterating over IEnumerable, and subsequent calls will be made as needed under the covers.

This seemed useful to expose through the low-level API as well, so we added methods on some of the clients as part of the SDK Preview for the following clients:

• Amazon.GlacierClient
• Amazon.IdentityManagementServiceClient
• Amazon.OpsWorksClient
• Amazon.SimpleNotificationServiceClient

Using the paginators from the low-level request interface looks like this:

var users = client.ListUsersEnumerator(new ListUsersRequest { MaxItems = 20 });
foreach(var user in users)
{
    Console.WriteLine("User: {0}", user.Name);
}

As usual with IEnumerable, you will need to pay special attention when using LINQ and/or the System.Linq.Enumerable extension methods. Calling various extensions like .Count(), .Where(), or .Last() on one of the IEnumerables returned by these methods could result in multiple, unintended calls to the service. In those instances where you do need to use those methods, it can be a good idea to cache the IEnumerable returned for as long as possible.

Let us know if you find this facility useful. We look forward to hearing from you on GitHub and the AWS forums.