Time-to-Live Support in Amazon DynamoDB

Amazon DynamoDB recently added Time-to-Live (TTL) support, a way to automatically delete expired items from your DynamoDB table. This blog post discusses this feature, how it’s exposed in the AWS SDK for .NET, and how you can take advantage of it.

Using Time-to-Live

At a high-level, you configure TTL by choosing a particular attribute on a table that will be treated as a timestamp. Then you simply store an expiration time into this attribute on every item that you need to expire. A periodic process in DynamoDB identifies whether an item’s TTL timestamp attribute is now in the past, and then schedules the removal of that item from the table. The timestamps must be stored as epoch seconds (number of seconds since 12:00:00 AM January 1st, 1970 UTC), which you must calculate or have the SDK calculate for you.

The AWS SDK for .NET has three different DynamoDB APIs, so you have three different ways to use TTL. In the following sections, we discuss these APIs and how you use the TTL feature from each of them.

Low-Level Model – Control Plane

First, the low-level model. This is a thin wrapper around the DynamoDB service operations that you use by instantiating AmazonDynamoDBClient and calling its various operations. This model provides you with the most control, but it also doesn’t have the helpful abstractions of the higher-level APIs. Using the low-level model, you can enable and disable the TTL feature and configure Time-to-Live for your data.

Here’s an example of checking the status of TTL for a table.

using (var client = new AmazonDynamoDBClient())
{
    // Retrieve TTL status
    var ttl = client.DescribeTimeToLive(new DescribeTimeToLiveRequest
    {
        TableName = "SessionData"
    }).TimeToLiveDescription;
    Console.WriteLine($"TTL status = {ttl.TimeToLiveStatus}");
    Console.WriteLine($"TTL attribute {(ttl.AttributeName == null ? "has not been set" : $"= {ttl.AttributeName}")}");

    // Enable TTL
    client.UpdateTimeToLive(new UpdateTimeToLiveRequest
    {
        TableName = "SessionData",
        TimeToLiveSpecification = new TimeToLiveSpecification
        {
            Enabled = true,
            AttributeName = "ExpirationTime"
        }
    });

    // Disable TTL
    client.UpdateTimeToLive(new UpdateTimeToLiveRequest
    {
        TableName = "SessionData",
        TimeToLiveSpecification = new TimeToLiveSpecification
        {
            Enabled = false,
            AttributeName = "ExpirationTime"
        }
    });
}

Note: There is a limit to how often you can enable or disable TTL in a given period of time. Running this sample multiple times will likely result in a ValidationException being thrown.

Low Level – Data Plane

Actually writing and reading TTL data in an item is fairly straightforward, but you are required to write epoch seconds into an AttributeValue. You can calculate the epoch seconds manually or use helper methods in AWSSDKUtils, as shown below.

Here’s an example of using the low-level API to work with TTL data.

using (var client = new AmazonDynamoDBClient())
{
    // Writing TTL attribute
    DateTime expirationTime = DateTime.Now.AddDays(7);
    Console.WriteLine($"Storing expiration time = {expirationTime}");
    int epochSeconds = AWSSDKUtils.ConvertToUnixEpochSeconds(expirationTime);
    client.PutItem("SessionData", new Dictionary<string, AttributeValue>
    {
        { "UserName", new AttributeValue { S = "user1" } },
        { "ExpirationTime", new AttributeValue { N = epochSeconds.ToString() } }
    });

    // Reading TTL attribute
    var item = client.GetItem("SessionData", new Dictionary<string, AttributeValue>
    {
        { "UserName", new AttributeValue { S = "user1" } },
    }).Item;
    string epochSecondsString = item["ExpirationTime"].N;
    epochSeconds = int.Parse(epochSecondsString);
    expirationTime = AWSSDKUtils.ConvertFromUnixEpochSeconds(epochSeconds);
    Console.WriteLine($"Stored expiration time = {expirationTime}");
}

Document Model

The Document Model provides you with Table objects that represent a DynamoDB table, and Document objects that represent a single row of data in a table. You can store primitive .NET types directly in a Document, with the required conversion to DynamoDB types happening in the background. This makes the Document Model API easier to use than the low-level model.

Using the Document Model API, you can easily configure which attributes you’d like to store as epoch seconds by setting the TableConfig.AttributesToStoreAsEpoch collection. Then you can use DateTime objects without needing to convert the data to epoch seconds manually. If you don’t specify which attributes to store as epoch seconds, then instead of writing epoch seconds in that attribute you would end up storing the DateTime as an ISO-8601 string, such as “2017-03-09T05:49:38.631Z”. In that case, DynamoDB Time-to-Live would NOT automatically delete the item. So you need to be sure to specify AttributesToStoreAsEpoch correctly when you’re creating the Table object.

Here’s an example of configuring the Table object, then writing and reading TTL items.

// Set up the Table object
var tableConfig = new TableConfig("SessionData")
{
    AttributesToStoreAsEpoch = new List { "ExpirationTime" }
};
var table = Table.LoadTable(client, tableConfig);

// Write TTL data
var doc = new Document();
doc["UserName"] = "user2";

DateTime expirationTime = DateTime.Now.AddDays(7);
Console.WriteLine($"Storing expiration time = {expirationTime}");
doc["ExpirationTime"] = expirationTime;

table.PutItem(doc);

// Read TTL data
doc = table.GetItem("user2");
expirationTime = doc["ExpirationTime"].AsDateTime();
Console.WriteLine($"Stored expiration time = {expirationTime}");

Object Persistence Model

The Object Persistence Model simplifies interaction with DynamoDB even more, by enabling you to use .NET classes with DynamoDB. This interaction is done by passing objects to the DynamoDBContext, which handles all the conversion logic. Using TTL with the Object Persistence Model is just as straightforward as using it with the Document model: you simply identify the attributes to store as epoch seconds and the SDK performs the required conversions for you.

Consider the following class definition.

[DynamoDBTable("SessionData")]
public class User
{
    [DynamoDBHashKey]
    public string UserName { get; set; }

    [DynamoDBProperty(StoreAsEpoch = true)]
    public DateTime ExpirationTime { get; set; }
}

Once we’ve added the [DynamoDBProperty(StoreAsEpoch = true)] attribute, we can use DateTime objects with the class just like we normally would. However, this time we store epoch seconds, and the items we create are eligible for TTL automatic deletion. And just as with the Document Model, if you omit the StoreAsEpoch attribution, the objects you write will contain ISO-8601 dates and are not eligible for TTL deletion.

Here’s an example of creating the DynamoDBContext object, writing a User object, and reading it out again.

using (var context = new DynamoDBContext(client))
{
    // Writing TTL data
    DateTime expirationTime = DateTime.Now.AddDays(7);
    Console.WriteLine($"Storing expiration time = {expirationTime}");

    var user = new User
    {
        UserName = "user3",
        ExpirationTime = expirationTime
    };
    context.Save(user);

    // Reading TTL data
    user = context.Load("user3");
    expirationTime = user.ExpirationTime;
    Console.WriteLine($"Stored expiration time = {expirationTime}");
}

Conclusion

In this blog post we showed how you can toggle the new Time-to-Live feature for a table. We’ve also showed multiple ways to work with this data. The approach you choose is up to you and, hopefully with these examples, you’ll find it quite easy to schedule your data for automatic deletion. Happy coding!