Archiving and Backing-up Data with the AWS SDK for .NET

Jason Fulghum recently posted a blog entry about using Glacier with the AWS SDK for Java that I thought would be interesting for .NET developers. Here is Jason’s post with the code replaced with the C# equivalent.

Do you or your company have important data that you need to archive? Have you explored Amazon Glacier yet? Amazon Glacier is an extremely low-cost storage service that provides secure and durable storage for data archiving and backup. Just like with other AWS offerings, you pay only for what you use. You don’t have to pay large upfront infrastructure costs or predict capacity requirements like you do with on-premise solutions. Simply use what you need, when you need it, and pay only for what you use.

There are two easy ways to leverage Amazon Glacier for data archives and backups using the AWS SDK for .NET. The first option is to interact with the Amazon Glacier service directly. The AWS SDK for .NET includes a high-level API called ArchiveTransferManager for easily working with transfers into and out of Amazon Glacier.

IAmazonGlacier glacierClient = new AmazonGlacierClient();
ArchiveTransferManager atm = new ArchiveTransferManager(glacierClient);
UploadResult uploadResult = atm.Upload("myVaultName", "old logs", @"C:logsoldLogs.zip");

// later, when you need to retrieve your data
atm.Download("myVaultName", uploadResult.ArchiveId, @"C:downloadlogs.zip");

The second easy way of getting your data into Amazon Glacier using the SDK is to get your data into Amazon S3 first, and use a bucket lifecycle to automatically archive your objects into Amazon Glacier after a certain period.

It’s easy to configure an Amazon S3 bucket’s lifecycle using the AWS Management Console or the AWS SDKs. Here’s how to create a bucket lifecycle that will copy objects under the “logs/” key prefix to Amazon Glacier after 365 days, and will remove them completely from your Amazon S3 bucket a few days later at the 370 day mark.

IAmazonS3 s3 = new AmazonS3Client();

var configuration = new LifecycleConfiguration()
{
    Rules = new List
    {
        new LifecycleRule
        {
            Id = "log-archival-rule",
            Prefix = "logs/",
            Transition = new LifecycleTransition()
            {
                Days = 365,
                StorageClass = S3StorageClass.Glacier
            },
            Status = LifecycleRuleStatus.Enabled,
            Expiration = new LifecycleRuleExpiration()
            {
                Days = 370
            }
        }
    }
};

s3.PutLifecycleConfiguration(new PutLifecycleConfigurationRequest()
{
    BucketName = myBucket,
    Configuration = configuration
});

Are you using Amazon Glacier yet? Let know how you’re using it and how it’s working for you!