Boosting Unreal Engine Performance with Amazon FSx for OpenZFS as a Shared Derived Data Cache

Creating video games is a complex process, requiring diverse teams to collaborate across various aspects of a project. With recent world events, development environments are increasingly hybrid. Team members are now contributing from both home and office locations, as well as adopting virtual workstations. Caching is essential in the development of large, immersive games to ensure efficient asset retrieval and performance optimization. For instance, Unreal Engine uses a mechanism known as the Derived Data Cache (DDC) for this purpose. Configuring shared DDC in the cloud complements virtual workstations with faster response times and a more efficient development process.

The Derived Data Cache in Unreal Engine serves as a storage mechanism for data that is computationally expensive to generate. It preserves this data, eliminating the need for regeneration unless any changes occur. DDC stores processed derived asset data such as texture compressions, shader compilations, and geometry conversions that can be quickly accessed by the engine during load times. This is especially important for larger teams and projects, as it reduces the need to recompile assets constantly, thus saving time and computational resource. DDC can be local to a single user or shared across a team via a network to improve the efficiency of the development pipeline. Setting up a cloud-based shared Derived Data Cache offers scalability for your Unreal Engine projects. By leveraging AWS storage services, you can easily share and access cached data from anywhere. In this post, we are going to focus on shred DDC with Amazon FSx for OpenZFS.

Amazon FSx for OpenZFS is a fully managed shared storage built on the popular OpenZFS filesystem. It is a simple and powerful shared NFS file storage that delivers leading price/performance on AWS. Each Amazon FSx for OpenZFS has a file server, which is equipped with a high-speed, in-memory cache. Beyond the in-memory cache, Single-AZ 2 filesystems offer an additional NVMe cache which can be used to store a large derived assets and have better response times.  Amazon FSx for OpenZFS leverages both the ARC and L2ARC in the OpenZFS filesystem, which enhances data access from in-memory and NVMe caches, for faster data access and improved performance. Although Amazon FSx for OpenZFS is capable of multi-AZ deployments, we could alternatively use Single-AZ 2 to help optimize the cost of Unreal Engine DDC. Given that the derived data is not critical and can be easily reproduced, a Single-AZ is a good fit.

With DDC pulls, there are times of high request rates with times of long idle durations. Amazon FSx for OpenZFS has the capability to burst beyond baseline speeds. It accommodates both network I/O and disk I/O operations with the help of I/O credit mechanism. You can learn more about Amazon FSx for OpenZFS performance in the user documentation.

Setting up Amazon FSx for OpenZFS as Unreal Engine DDC:

In this blog, we are going use Amazon WorkSpaces as the client running Windows Server 2019 OS, and the Lyra Starter game for the derived cache fill. (Lyra is a modular sample game project by Epic Games)

Step 1: Create and Configure the Amazon FSx for OpenZFS File System
In the AWS console, navigate to Amazon FSx for OpenZFS and click “Create filesystem”. Select ‘’ filesystem type with ‘standard create’ method.

Choose your Amazon VPC and subnets and create the filesystem with deployment type ‘Single-AZ 2’. This deployment type supports NVMe. Here is the configuration to create your Amazon FSx for OpenZFS filesystem.

Step 2: Configuring Network access

Proper network configuration is essential for allowing communication between the Amazon FSx for OpenZFS filesystem and your workstation.
Find the Elastic Network Interface (ENI) associated with your Amazon FSx for OpenZFS filesystem in the AWS console in the Network & Security Tab. This is the network interface that handles the filesystem traffic.

We need to configure the required ports for the Amazon FSx for OpenZFS over the NFS protocol for proper communication. You can find additional information regarding filesystem access control here.

Add inbound security group rules to the Amazon FSx for OpenZFS ENI security group to allow access from your WorkSpaces security group.

Step 3: Mount Amazon FSx for OpenZFS to your Workstation

Mount the configured Amazon FSx for OpenZFS filesystem to your workspaces to facilitate communication between the Amazon FSx for OpenZFS filesystem and your workspaces. Click on your Amazon FSx for OpenZFS filesystem and click ‘Attach’ to find instructions to mount the filesystem to your workstation.

In this post, we have used Windows based WorkSpaces and mounted the filesystem as the Z drive. Mounting OpenZFS volumes to Windows clients leverages the NFS v3 protocol. So, first you need to install the NFS Client. In your workstation, Open PowerShell as an administrator, and install the NFS client.

Install-WindowsFeature -Name NFS-Client

Open command prompt and execute the following command with your filesystem ID (you can replace “Z:” with any other available drive letter)

mount \\<filesystemID>.fsx.<region>.amazonaws.com\fsx\ Z:

Step 4: Configure your Unreal Engine Editor to use Amazon FSx for OpenZFS as DDC

Now that your filesystem is ready and mounted to your workstation, now you can configure the Unreal Engine Editor to recognize Amazon FSx for OpenZFS as the DDC. Navigate to Edit and then to Editor Preferences in your Unreal Engine Editor.

In the Unreal Engine Editor Preferences, look for Global Shared DDC Path Parameter. Here, you will input the Amazon FSx for OpenZFS mount path that you have configured. Essentially, you are directing the Unreal Engine to use the assigned Amazon FSx for OpenZFS mount path as the location for the Global Shared Derived Data Cache.

Step 5: Validate Your Configuration

To ensure that everything is set up correctly, you’ll need to fill the DDC with recomputed game data. Start cooking your builds, and you will see the Derived Data Cache populate in the mounted Amazon FSx for OpenZFS path. Here is the command you need to run, making sure you replace the placeholders with the actual paths to your Editor and project.

"<path to Editor>\UnrealEditor.exe" "<Path to project>\LyraStarterGame" -run=DerivedDataCache -fill

You can see your derive data populated in your Amazon FSx for OpenZFS shared cache mount.

It is important to address the clean-up process to avoid any additional costs after the completion of your project. Begin by removing the Amazon FSx for OpenZFS path from the Unreal Engine Editor’s preferences to ensure the engine no longer attempts to access the DDC. Then, unmount the FSx filesystem from your workstation. Execute the command net use Z: /delete to unmount the Amazon FSx for OpenZFS filesystem. (Replace “Z:” with the actual drive letter used when mounting). Once unmounted, it’s time to delete the filesystem from the AWS Management Console. Navigate to the FSx service in the AWS Management Console and Select your Amazon FSx for OpenZFS filesystem. Choose ‘Delete file system’ and follow the prompts to ensure that all data is deleted. As you are cleaning up, it’s important to also reverse any changes made to your network configuration. Remove the inbound security group rules from the Amazon FSx for OpenZFS ENI security group to re-secure your network configuration.

For those who have used Terraform template to set up the DDC, run the ‘terraform destroy’ command to de-provision the related infrastructure.

In Conclusion, we provided a step-by-step guide to configure Amazon FSx for OpenZFS as a Derived Data Cache (DDC) for Unreal Engine. Amazon FSx for OpenZFS provides a fully managed, secure, and scalable file storage which makes it an good fit for configuring as a DDC in Unreal Engine, especially for shared development environments. For those who prefer an Infrastructure as Code (IaC) approach, we have published a Terraform template to simplify the setup process.

You can find the Terraform template in the AWS GitHub repository at aws-samples/unreal-engine-ddc. This repository contains all the necessary code and instructions to set up your Derived Data Cache (DDC) for Unreal Engine using Amazon FSx for OpenZFS.