50 years of innovation: How open data is supporting NOAA’s “science, service, and stewardship” mission

This month, the National Oceanic and Atmospheric Administration (NOAA) celebrates 50 years of “science, service, and stewardship.”

Over the past five decades, NOAA has demonstrated its ability to push the boundaries of technological innovation to collect and understand data, as well as share that knowledge and information with others.

Amazon Web Services (AWS) is proud to support NOAA’s mission, in particular by providing public access to the agency’s environmental datasets since 2015 through the Registry of Open Data on AWS. The agency has an open data policy that makes available its data from satellites, radars, ships, weather models, and other sources to help organizations accelerate research and innovation.

“The massive amount of environmental data that NOAA collects must be used to its full potential – and the cloud makes that possible,” said Neil Jacobs, Ph.D., acting administrator of NOAA. “By bringing this data into the cloud, the public, private industry, and academia can better harness the data we are collecting literally down to the minute.”

By providing access to NOAA data on AWS without needing to download and store their own copies, researchers and entrepreneurs can deploy compute resources on-demand in the cloud and perform analysis quickly and efficiently. This makes it less expensive for researchers to ask more questions and experiment more easily.

In December 2019, NOAA announced new collaborations with multiple cloud providers, including AWS, as part of NOAA’s Big Data Program. According to AWS Vice President for U.S. Government, Nonprofit and Healthcare Businesses Dave Levy, “As customers gather more data about Earth, cloud offers new opportunities to understand our world and contribute to a sustainable future. With initiatives like the Big Data Program, researchers are leveraging AWS services to lower the barriers to affordable, on demand, and scalable data analysis. As a result, researchers are gaining greater access to NOAA data, and speeding up the time to discovery, at a fraction of the price and time previously required.”

NOAA is constantly updating existing datasets or making available new ones through the Registry of Open Data on AWS. In the past few months, NOAA moved more data to the cloud through the NOAA Big Data Program. This includes updates to its global ensemble forecast system (GEFS)—now in version 12, a significant improvement over the old system. NOAA also compiled a reforecast archive, which is being made available only on AWS. In April, NOAA made available the Himawari-8 dataset, the Japan Meteorological Agency’s (JMA’s) satellite dataset. NESDIS is now the single source of the Himawari-8 dataset in the US. Himawari-8’s Advanced Himawari Imager (AHI) has many new channels (total of 16) compared with previous geostationary satellites.

The data available through the NOAA Big Data Program is also helpful to Amazon as it works towards its sustainability goals, including The Climate Pledge. NOAA data is made available to Amazon teams through the Amazon Sustainability Data Initiative (ASDI), a global and authoritative source for open sourced weather, climate, and sustainability data, which is a collaboration between the AWS Open Data team and Amazon Sustainability.

Looking to the cloud for the future

In July, NOAA announced the release of its cloud strategy and data strategy. The cloud strategy, which aligns with the U.S. Federal Cloud Strategy, shows how the cloud is part of the growth plan and the future of NOAA. The strategy aims to evolve how NOAA manages its data and maximize its value.

According to NOAA, the Cloud Strategy, “provides a common vision and guide for future cloud initiatives, building on NOAA’s robust past experience with cloud services. Recent NOAA cloud initiatives are already demonstrating significant potential improvements in data storage, access and analytics of environmental data in areas such as numerical weather prediction, ocean models, assessments of living marine resources, and big data analysis, storage and dissemination. Looking forward, the strategy seeks to accelerate innovation in areas such as AI [artificial intelligence] and ‘Omics through rapid adoption of cloud services, ensure a smart transition to the cloud, promote broad and secure access to NOAA’s data, develop effective governance mechanisms for shared enterprise cloud services and enable a cloud-ready workforce.”

“The cloud is our present and the future,” added Jacobs. “Looking ahead, I envision improved computer forecast models, safely sharing model code publicly, and allowing easier access to a wider range of environmental data that entrepreneurs can leverage to build new enterprises.”

“NOAA’s Cloud Strategy is a testament to its commitment to innovation and the role the cloud plays in supporting science, service, and stewardship,” said Levy. “Continuing to increase the output of NOAA’s open data will drive critical research efforts, improve our understanding of Earth, and contribute to a more sustainable future for the next fifty years and beyond.”

Learn more about the Registry of Open Data on AWS, NOAA, and AWS for aerospace and satellite. And join the conversation on NOAA’s anniversary with the hashtags #NOAAat50 and #50YearsOfNOAA.

Happy 50th anniversary, NOAA!