Move to managed file storage in minutes with AWS DataSync

You’re probably familiar with the high-level benefits of the cloud: focus on your core business as opposed to maintaining IT infrastructure, increased agility and innovation, and growth to both your top and bottom line. AWS’ fully managed file services portfolio helps you realize those benefits, even in terms of storage.

Let’s start with that point of focus in a little more depth, which we sometimes call “removing undifferentiated heavy lifting.” Chances are, running a file server likely isn’t your core competency. Most of the time, file servers are instead required in support of an application you run, or a service you’re hosting for your customers. If you’re running on premises, managing a file server involves several things you need to address. Among these include hardware procurement, coordinating floor space and facilities contracts, provisioning capacity (perhaps over-provisioning to accommodate peak demand), and having a plan for business continuity and disaster recovery at the storage layer.

In this post, I talk about the business drivers for moving to managed storage in the cloud. I also walk through how to get starting using AWS DataSync, a fully managed data transfer service.

Fully managed storage in the cloud

By running your own file server in the cloud, the ability to provision infrastructure on-demand lessens the burden of procurement and physical data center space. However, you still have the complexity of capacity and infrastructure management, as well as the challenge of best taking advantage of cloud storage building blocks and architecting for scale.

By moving to a fully managed storage service, you further reduce the effort needed for capacity planning, infrastructure and operating system maintenance, and architecting for scale and high availability, just to name a few. With a fully managed file system, you can maximize time-to-value from your applications, since you don’t need to rewrite your applications or refactor your environment. You can simply load your data in the format your applications expect and start using it – it’s that easy. Even better, activities like capacity planning, backup, and high availability are features that are available right out of the box.

Here’s what LoanLogics, one of our financial services customers, had to say about moving to managed storage:

“We needed to immediately scale our storage capacity to support the onboarding of a new, large customer. AWS file storage services allowed us to scale our infrastructure in a matter of days with no code changes to our applications.”

–Terrell Cassada, CIO, LoanLogics

AWS file and data transfer services for your application needs

AWS offers several fully managed file system services for business-critical applications. Two of these are Amazon Elastic File System (Amazon EFS), which provides a simple, scalable, and elastic file system over NFS, and Amazon FSx for Windows File Server (Amazon FSx), which provides a managed file system over SMB. For in-depth information on topics like performance, pricing, and getting the most out of your applications once you’ve moved to managed storage, check out this presentation from re:Invent 2019:

To move your data from on-premises to AWS, we offer a fully managed data transfer service, AWS DataSync, which natively supports both EFS and Amazon FSx. The rest of this post helps you get started using managed cloud storage by introducing you to DataSync.

DataSync makes it easy to move your applications into the cloud, minimizing or even eliminating any changes you must make to your existing environments. Using DataSync, you can migrate your application storage into one of our cloud-native file system services, keeping your files in their existing format and preserving metadata like ownership and permissions. I walk through an example of getting started, and you can always refer to the DataSync user guide for more technical detail.

For the purposes of this post, I assume you’ve already set up and activated your DataSync agent. The agent runs the DataSync software and automates your file transfer. If you’re running on-premises, DataSync makes it easy to securely transfer your data over the internet. Or, if you’re running your own file server in the cloud, DataSync supports AWS PrivateLink, so you can configure your transfer to never leave your VPC.

Getting started with AWS DataSync

With your agent activated and ready to go, the next steps are setting up your source and destination file systems, and setting up a job to execute the transfer. In DataSync lingo, the source and destination are called “locations.” DataSync supports connecting to your on-premises Linux and Windows storage, using either the NFS protocol or the SMB protocol. Let’s say you’re moving an on-premises application that uses NFS into the cloud, so you’re considering EFS as your cloud storage solution. The following is an example of how to set up an NFS location and an EFS location for your DataSync transfer. Let’s say you want to copy data from an NFS export available at 192.168.1.152 to an EFS file system with ID fs-862f29cd. First, create locations for each of these file systems:

With your locations created, you can now set up a job to transfer data between them, known in DataSync as a “task.”

Once your task is created, you can execute it at any time.

It’s as simple as completing these few steps to get started! For Windows workloads, you can just as easily create locations for SMB and Amazon FSx (as opposed to NFS and EFS). DataSync also gives you the flexibility to manage permissions, rate limit your transfer speed, schedule automatic periodic transfers, and so on. For more information on migrating your data to Amazon FSx using DataSync, check out this video prepared by an AWS Senior Solutions Architect:

Give it a test drive!

I hope this post helps you think about how moving to a managed storage service can help your business. If you want to test it out yourself, this post shows how you how quickly you can set up a proof of concept to validate how AWS can meet your file storage needs. By moving to managed file storage in the cloud, not only will you will reduce the time and effort required to maintain infrastructure, but you can quickly and easily scale your applications to meet your business needs. To get started, consider checking out the Amazon EFS product page, the Amazon FSx for Windows File Server product page, and the AWS DataSync product page. Don’t forget you can test out the examples in this post using the DataSync console. Thanks for reading this blog post, please leave any questions or comments you may have in the comments section.