Migrating data access and Microsoft Active Directory with Amazon FSx for NetApp ONTAP

In today’s digital era, enterprises face significant challenges in data center modernization during their digital transformation journey. Traditional on-premises solutions struggle with high costs, complex management, and data growth. Organizations with intricate file-sharing systems and user permissions face difficulties in preserving user experiences and security. The tight integration of enterprise IDCs with complex Microsoft Active Directory structures further complicates migration, which necessitates flexible and efficient solutions.

Amazon FSx for NetApp ONTAP addresses these challenges by enabling seamless migration of NetApp environments to the cloud. It preserves familiar functions while offering cloud benefits such as scalability and availability. This solution supports Windows environments and Server Message Block (SMB) protocols for efficient file access, as well as NFS for Linux- and UNIX-based clients. Crucially, it facilitates IDC AD information migration, which maintains complex authentication and permission systems, allowing users to access new storage using existing methods, thus minimizing disruption.

This post details how to use AWS services to achieve seamless migration from on-premises NetApp storage to FSx for ONTAP as in Figure 1. We explore maintaining user permission management, AD information, and data access consistency while optimizing storage efficiency, reducing Total Cost of Ownership (TCO), and gaining cloud computing flexibility and scalability. This solution empowers enterprises to significantly enhance their data management capabilities, better respond to business fluctuations, and lay a solid foundation for future growth and exploration. It represents not only a technological upgrade, but also a crucial step in an enterprise’s digital transformation journey.

Figure 1: Architecture diagram for FSx for ONTAP migration from IDC to AWS cloud

Designing a smooth migration plan

The overall architecture is shown in Figure 2.

Figure 2: Architecture diagram for the hands-on experiment in this post

This experiment creates two Amazon Virtual Private Clouds (Amazon VPCs), one representing the on-premises data center (IDC) environment and the other representing the AWS cloud environment. The two networks are connected through VPC Peering. Furthermore, a one-way trust is established for managing permissions between IDC and AWS AD, because IDC has stricter permission control and resource access policies. IDC AD users can access resources in the Amazon cloud, but Amazon cloud AD users can’t access resources in IDC. Building on this foundation, NetApp’s SnapMirror protocol is used to synchronize data from IDC NetApp to FSx for ONTAP, and automatically synchronize IDC AD user and permission information to FSx for ONTAP.

• Microsoft AD one-way trust: Microsoft AD one-way trust enables user authentication between tenants without reciprocal trust. It provides centralized identity management solutions for multinational enterprises, streamlining processes, enhancing security, and reducing costs while maintaining regional independence and compliance.
• NetApp SnapMirror: NetApp SnapMirror is a data replication technology for NetApp systems. It offers real-time synchronization, reduced downtime, and optimized bandwidth usage, which make it ideal for efficient and reliable cloud migration. SnapMirror can be used between on-premises systems, from on-premises to cloud, or between cloud systems.
• Smooth migration of user permissions: For user permissions and data synchronization migration, before and after migration, use a consistent Microsoft AD user account system for user permission management, without the need to separately migrate AD information from IDC or recreate user account systems in the AWS cloud.
• Smooth migration of user access experience: Before and after migration, users can continue to use the same URLs to access their data and applications, without the need to modify code for applications using IDC NetApp as the data source.

Hands-on lab: migration solution design

The following sections outline the migration solution design.

User requirements

• Access via Old URL: Both aws_user and idc_user should be able to access the new FSx for ONTAP share on AWS cloud using the old URL IDC-FSX-0903.IDC09032.COM.
• Short domain access: Both users should also have access through the short domain name IDC-FSX-0903.

Environment parameters

• IDC Domain Controller: idc-ad-demo-09-03-2
• IDC DNS IP Address: 172.31.45.51
• IDC AD Domain Name: idc09032.com
• AWS Cloud Domain Controller: aws-ad-demo-09-03-2
• AWS DNS IP Address: 10.0.11.171
• AWS AD Domain Name: aws09032.com

Domain controller configuration

Refer to the official AWS documentation for configuring domain controllers, focusing on the following sections:

1. Create a Windows Server 2019 EC2 instance: Use Amazon Machine Image (AMI) ID ami-02bb8fbdb93603b5a for this experiment.
2. Promote your server to a domain controller: Perform these steps in both the IDC and AWS cloud environments.
3. Configure DNS conditional forwarders: Make sure of proper DNS forwarding between the domains to facilitate seamless access.

The DNS server of the domain controller in the IDC environment points to the IP address of the domain controller in the AWS environment, as shown in the Figure 3:

Figure 3: Configuration of conditional forwarders in the IDC environment

The DNS server of the domain controller in the AWS environment points to the IP address of the domain controller in the IDC environment, as shown in figure 4:

Figure 4: Configuration of conditional forwarders in the AWS environment

One-way trust configuration steps

The following sets of steps each outline the incoming trust configuration steps and the outgoing trust configuration steps.

Incoming trust configuration steps

To configure the incoming trust from the IDC environment AD to the AWS environment AD, follow these steps:

1. Log in to com and open Server Manager. Go to Tools > Active Directory Domains and Trusts.
2. In the left pane, right-click com and choose Properties. On the Trusts tab, choose New Trust to start the wizard.
3. Enter “aws09032.com” as the trust name. Choose Forest trust as the trust type and set it as One-way: incoming.
4. Choose This domain only for the sides of trust. Enter the trust password when prompted.
5. Review the trust selections and creation details in the following screens, choosing Next to proceed.
6. When asked to confirm the incoming trust, choose No, do not confirm the incoming trust.
7. Complete the wizard by choosing Finish, then choose OK on the final configuration page, as shown in Figure 5.

Figure 5: Results of incoming trust configuration in the IDC environment

Outgoing trust configuration steps

To configure the outgoing trust from the AWS cloud environment AD to the IDC environment AD, follow these steps:

1. Log in to com and open Server Manager. Go to Tools > Active Directory Domains and Trusts.
2. In the left pane, right-click com and choose Properties. On the Trusts tab, choose New Trust to start the wizard.
3. Enter “idc09032.com” as the trust name. Choose Forest trust as the trust type and set it as One-way: outgoing.
4. Choose This domain only for the sides of trust and Forest-wide authentication for the outgoing trust authentication level.
5. Enter the trust password when prompted. Review the trust selections and creation details in the following screens.
6. When asked to confirm the outgoing trust, choose No, do not confirm the outgoing trust.
7. Complete the wizard by choosing Finish, then choose OK on the final configuration page, as shown in Figure 6.

Figure 6: Results of outgoing trust configuration in the AWS environment

Verify one-way trust configuration result

The IDC environment can successfully access the folder in the AWS cloud environment, as shown in Figure 7.

Figure 7: Screenshot of accessing folders in the AWS environment from the IDC environment

The AWS cloud environment can’t access the share folder in IDC, and it prompts for username and password information, as shown in Figure 8.

Figure 8: Screenshot of accessing folders in the IDC environment from the AWS environment

Add FSx for ONTAP to the AD in the AWS cloud environment, as shown in Figure 9.

Figure 9: Screenshot of FSx for ONTAP integrated with AD in the AWS environment

Data and permission synchronization from IDC NetApp to FSx for ONTAP

1. Alias preparation

To facilitate understanding of how to input your values in the AWS Command Line Interface (AWS CLI), we have set the following aliases for the cluster, Storage Virtual Machine (SVM), and volumes:

• OnPrem-Source: Identifier for the source cluster
• FSx-Dest: Identifier for the target cluster (FsxIdxxxxxxxxxxxxx)
• SourceSVM: Name of the source SVM
• DestSVM: Name of the target SVM
• The source volume is named vol1, and the target volume is named vol1_dst

2. Log in to the target cluster and create your target volume

Start by accessing your FSx for ONTAP file management system (target cluster) through Secure SSH. You can find the <ManagementIP> on the AWS Management Console.

$ ssh fsxadmin@<ManagementIP>

Before creating a SnapMirror relationship, you need to have a destination volume on the target cluster with a capacity at least equal to the source volume. It must also be created with the -type DP option so that it can serve as a SnapMirror destination. Name this destination “volume vol1_dst”.

3. Collect intercluster LIFs for source and target clusters

SnapMirror uses intercluster LIFs for data transfer between source and target clusters. These intercluster LIFs are automatically provisioned when your file system is created, and are used for cluster peering relationships. For FSx for ONTAP systems, you can obtain the IP addresses through the Console. Go to your file system, then proceed to the Network & Security section. Under Endpoints find the Intercluster Endpoint – IP address.

Alternatively, you can retrieve the intercluster LIF IP from the ONTAP CLI using the following command:

FSx-Dest::> network interface show -role intercluster -fields address

vserver                lif     address

---------------------- ------- ----------

FsxId01704c7f9d0b63887 inter_1 10.0.6.176

FsxId01704c7f9d0b63887 inter_2 10.0.15.156

2 entries were displayed.

Store the IP addresses for LIFs inter_1 and inter_2. We refer to the FSx-Dest addresses as dest_inter_1 and dest_inter_2, and the OnPrem-Source addresses as source_inter_1 and source_inter_2.

4. Establish peering between source and target clusters

On the target cluster, create a cluster peer relationship by providing source_inter_1 and source_inter_2 from the source cluster. This command prompts you to create a passphrase, which you must enter when establishing the cluster peer on the source cluster. By default, intercluster traffic is encrypted by ONTAP.

You need to specify all intercluster LIFs from the source file system based on your actual environment. In our lab demonstration, we use two LIFs as an example.

FSx-Dest::> cluster peer create -peer-addrs <source_inter_1>,<source_inter_2>

Establish the cluster peer relationship on the source cluster by providing dest_inter_1 and dest_inter_2 of the target cluster. You must enter the previously created passphrase for authentication.

OnPrem-Source::> cluster peer create -address-family ipv4 -peer-addrs <dest_inter_1>,<dest_inter_2>

Verify the peer connection on the source cluster using the following command. The status should display as Available under the Availability column.

OnPrem-Source::> cluster peer show
Peer Cluster Name Availability Authentication
----------------- -------------- --------------
FSx-Dest Available ok

5.Create SVM peering relationship

After establishing cluster peering, the next step is to peer the SVMs. On the target cluster (FSx-Dest), use the following command to create an SVM peer relationship. In addition to the aliases we previously listed, we use:

• <DestLocalName> as the destination SVM, referred to as destsvm01 on the source cluster
• <SourceLocalName> as the source SVM, referred to as sourcesvm01 on the target cluster

When source and destination SVMs share the same name (for example both named “fsx”), you must specify a local name in ONTAP peering configuration to distinguish them.

Initiate the peering request from the source cluster:

OnPrem-Source::> vserver peer create -vserver <DestSVM> -peer-vserver <DestLocalName> -peer-cluster <OnPrem-Source> -applications snapmirror -local-name <DestLocalName> Info: [Job 207] 'vserver peer create' job queued

Accept the peering request on the target cluster:

FSx-Dest::> vserver peer accept -vserver <DestSVM> -peer-vserver <SourceSVM> -local-name <SourceLocalName>
Info: [Job 211] 'vserver peer accept' job queued

Verify the SVM peering status using the following command. The Peer State should display as peered:

OnPrem-Source::> vserver peer show
Peer Peer Peer Peering Remote
vserver Vserver State Cluster Applications Vserver
------- -------- ------ -------- ------------- ---------
svm01 destsvm1 peered FSx-Dest snapmirror svm01

6. Create and initialize SnapMirror relationship

After establishing SVM peering, the next step is to create and initialize the SnapMirror relationship on the target cluster. You can use the -throttle flag, which sets the maximum bandwidth (in KB/second) that the SnapMirror relationship can use.

FSx-Dest::> snapmirror create -source-path <SourceLocalName>:vol1 -destination-path <DestSVM>:vol1_dst -vserver <DestSVM>
Operation succeeded: snapmirror create for the relationship with destination "DestSVM:vol1_dst".

After creating the SnapMirror relationship, use the following command on the target cluster to initialize the SnapMirror relationship. This initiates the transfer of snapshot data from the source volume to the destination volume.

FSx-Dest::> snapmirror initialize -destination-path <DestSVM>:vol1_dst -source-path <SourceLocalName>:vol1

Enter the following command on the destination end, and you can observe vol1_dst on the destination server:

Note: The data is read-only until the volume is flipped from type DP to type RW.

FSx-Dest::> volume mount -volume vol1_dst -junction-path /vol1_dst -vserver fsx

7. Verify data synchronization results:

1) Data information in the IDC environment is as shown Figure 10:

Figure 10: Data information in the IDC environment before migration

2) Data information in the AWS cloud environment is as shown in Figure 11:

Figure 11: Data information in the AWS environment after migration

Verify AD user and permission synchronization results:

1) User and permission information in the IDC environment is as shown in Figure 12:

Figure 12: User and AD permission information in the IDC environment before migration

2) User and permission information in the AWS cloud environment is as shown in Figure 13:

Figure 13: User and AD permission information in the AWS environment after migration

Maintaining IDC NetApp access URL configuration

As of this writing, users access the NetApp system in their IDC environment through two primary methods: directly using the IDC NetApp URL to access the file system, or through applications that have integrated the NetApp URL. To make sure of consistency in user experience before and after migration and minimize migration impact, this experiment provides a method to retain the IDC NetApp URL configuration.

To detach the existing IDC NetApp from AD, enter the following commands in sequence:

OnPrem-Source::> vserver cifs modify -vserver fsx -status-admin down
OnPrem-Source::> vserver cifs delete -vserver fsx
Error: command failed: Cannot delete CIFS/AD account "IDC-FSX-0903" because secure DNS dynamic updates are enabled. To
disable secure DNS updates, run "vserver services name-service dns dynamic-update modify -use-secure false", and
then try the operation again.
OnPrem-Source::> vserver services name-service dns dynamic-update modify -use-secure false
OnPrem-Source::> vserver cifs delete -vserver fsx

Enter the username and password with sufficient privileges to detach the IDC NetApp from the current AD. When prompted, enter y to confirm.

Add an A record in the existing IDC DNS environment pointing to the AWS environment. Follow these steps:

1. Log in to com, open Server Manager, and from the Tools menu, choose DNS.
2. In the DNS Manager, expand Forward Lookup zones, choose com, right-click and choose New Host(A or AAAA).
3. In the Host field, enter the NetBIOS name of the IDC NetApp’s SVM. For example: IDC-FSX-0903
4. In the FQDN field, enter the SMB DNS name of the IDC NetApp, for example IDC-FSX-0903.IDC09032.COM
5. In the IP address field, enter the management IP address of the SVM in the AWS cloud environment., for example 10.0.10.69
6. Log in to com and verify by entering the IDC environment’s NetApp URL. This successfully reuses the IDC environment’s NetApp URL in the AWS environment, as shown in Figure 14:

Figure 14: Accessing AWS FSx for ONTAP using the original IDC NetApp access URL

If you want to use the short name IDC-FSX-0903 to access the NetApp in the AWS cloud environment, then you can create a CNAME record in the DNS server for IDC-FSX-0903, which points to IDC-FSX-0903.IDC09032.COM.

Optimizing costs with ONTAP deduplication and capacity pool

• The FSx for ONTAP deduplication feature reduces storage space by eliminating duplicate data blocks, lowering costs, and enhancing efficiency. It provides up to 65% storage savings and around 46% cost savings, making cloud operations more efficient.
• The FSx for ONTAP Capacity Pool offers pay-as-you-go storage with dynamic scaling. It significantly reduces costs by charging only for used capacity, which optimizes resources for fluctuating workloads. User validation demonstrates potential cost reduction to 40% of original expenses.

User benefits summary

In this post, we showed how Amazon FSx for NetApp ONTAP enhances architecture through deduplication and seamless AD permission migration, thus maintaining user familiarity and operational continuity. It improves efficiency, reduces costs, and offers scalability for future growth. This comprehensive solution balances technical optimization, user comfort, operational efficiency, and cost-effectiveness, thus supporting long-term business development with flexibility for evolving enterprise needs.