Supporting climate model simulations to accelerate climate science

This post was authored by Zac Flamig, Tech Lead Amazon Sustainability Data Initiative (AWS/ASDI) and Dale Morris, Senior Research Associate (University of Oklahoma/CIMMS and NOAA)

The Amazon Sustainability Data Initiative (ASDI), AWS is donating cloud resources, technical support, and access to scalable infrastructure and fast networking providing high performance computing (HPC) solutions to support simulations of near-term climate using the National Center for Atmospheric Research (NCAR) Community Earth System Model Version 2 (CESM2) and its Whole Atmosphere Community Climate Model (WACCM). These are among the most sophisticated climate and atmospheric models in the world.

ASDI seeks to accelerate sustainability research and innovation by minimizing the cost and time required to acquire and analyze large sustainability datasets. The program supports innovators and researchers with the data, tools, and technical expertise they need to advance the field of sustainability. In collaboration with ASDI, AWS, and SilverLining, a nonprofit dedicated to ensuring a safe climate, the National Center for Atmospheric Research (NCAR) will run an ensemble of 30 climate-model simulations on AWS. The climate runs will simulate the Earth system over the period of years 2035-2070 under a median scenario for warming. Related simulations will be performed by the UK Met Office using their Earth System Model, UKESM1, and by academic researchers using NASA’s GISS model with all of the resulting datasets hosted and openly accessible through the AWS Open Data Program. The simulation work will demonstrate the ability to use cloud infrastructure to advance climate models in support of robust scientific studies by researchers around the world and aims to accelerate and democratize climate science.

AWS worked with the NCAR team to develop an architecture that supports this project and is capable of supporting other weather and climate modeling workloads. The recommend architecture uses AWS ParallelCluster to create the compute environment depicted in the following architecture diagram.

The simulations utilize Amazon EC2 with C5n.18xlarge instances supporting Elastic Fabric Adapter (EFA). EFA enables the low latency Message Passing Interface (MPI) necessary for performant scaling of the climate simulations. The simulation data will first be stored on a shared storage system using Amazon FSx for Lustre. FSx enables cost effective, high performance, scalable storage to be shared amongst all of the nodes running the climate simulation. After the simulations are complete, the data will be post-processed before being transferred to Amazon S3, where it will be shared for the community to use.

Those interested in learning how to set up this compute environment can explore several AWS workshops that provide guidance through the entire process. The AWS ParallelCluster workshop takes you step-by-step through the installation of AWS ParallelCluster, cluster configuration, the process for submitting your first HPC job, and finally terminating the cluster when you have completed your work. Similarly for working with FSx for Lustre, the high performance file system workshop can provide guidance on how to set up FSx for Lustre for the high performance shared storage system for all compute notes. The workshop will also demonstrate how to benchmark storage system performance and monitor related metrics in Amazon CloudWatch.

“As the world prepares to gather for the United Nations Climate Change Conference (COP26), this initiative provides technology leadership and funding to run these important models at scale on AWS and help advance climate research,” said Kara Hurst, vice president of worldwide sustainability at Amazon. “It is critical that we understand how the climate is changing and how that will impact communities and business. Amazon is proud to work with NCAR, one of the world’s leading climate modeling centers, and an innovative organization like SilverLining, to accelerate progress and to expand access to powerful climate research tools and data.”

Collaborating research teams will use the data generated through these simulations to study impacts on Earth and human systems—including agriculture, drought, flooding, and human health—in various parts of the world. These studies will advance understanding of near-term climate and climate-intervention responses, and accelerate progress on a time-sensitive problem for humanity.

“Climate research is critical in places like Cape Town, where severe drought and other impacts are happening now and projected to worsen in the next few decades. Our ability to participate in climate research is limited by local computing resources and networks,” said Dr. Chris Lennard, senior researcher, Climate System Analysis Group (CSAG) at the University of Cape Town. “Open access to models and datasets, and cloud-based high performance computing environments to work with them, allows us to engage equally as researchers and to design and execute studies to help people in our parts of the world that were not conceivable in the past.”

AWS promotional credit is being provided through the ASDI grant program that supports experimentation and development for sustainability-related projects. The promotional credit program is open to anyone seeking to prototype software, tools for sustainability-related work, perform proofs of concept for moving research workloads or open datasets to the cloud, or train a broader community on the use of cloud for sustainability workloads. Applications are encouraged and available at https://amazonsdi.com/credit.

Find out more about Amazon Sustainability Data Initiative and the AWS Open Data Program.