University of Adelaide Provides Seamless Bioinformatics Training Using AWS

The University of Adelaide, established in South Australia in 1874, maintains a rich history of scientific innovation. For more than 140 years, the institution and its researchers have had an impact all over the world—making vital contributions to the invention of X-ray crystallography, insulin, penicillin, and the Olympic torch.

Bioinformaticians from the university’s School of Biological Sciences are carrying on this tradition of innovation. The researchers foster collaborations across the university with the aim of shaping the future of bioscience research at the institution. To support their goals, they develop and deliver bioinformatics training on campus. Recently, in collaboration with Dr. Radosław Suchecki, research scientist at the Commonwealth Scientific and Industrial Research Organization, they offered hands-on virtual training in the Snakemake workflow engine and Nextflow workflow manager to bioinformaticians across Australia, with the support of Australian BioCommons.

To deliver this training, they needed to quickly spin up automated virtual training environments, which presented unique challenges. “In this field, the compute requirements can be very large and usually beyond what one can expect in a computing suite at any one institution,” says Dr. Nathan Watson-Haigh, research fellow in bioinformatics at the university’s School of Biological Sciences. “In addition, we needed to set up a training environment where everyone could access the same resources, regardless of where they were or what their local infrastructure was. These were capabilities we didn’t have ourselves.”

The bioinformaticians engaged RONIN, a Select Consulting Partner in the Amazon Web Services (AWS) Partner Network (APN), to help design and deploy an AWS-based compute cluster that runs on hundreds of Amazon Elastic Compute Cloud (Amazon EC2) instances. The cluster uses AWS Auto Scaling for automatic on-demand scalability, and it takes advantage of the Slurm open-source cluster management system.

“We wanted to develop training for tools that would work in a cluster environment that bioinformatics researchers would typically work on,” says Watson-Haigh. “We knew the AWS Cloud was the right technology, and RONIN was a trusted technology partner we had worked with previously.”

The university collaborated with RONIN to conduct a trial of the cluster before the training event. “The barrier to entry seemed significant at the time, so when we conducted the trial, we clearly saw how good the technology was, as well as the support from RONIN. Just the fact that we could spin something up with no more than a few clicks and have infrastructure at our fingertips was very powerful.”

Using the RONIN solution, the university’s bioinformaticians created a workshop that ran simultaneously across eight locations spanning four time zones in Australia. The two-day event included 152 bioinformaticians and 15 facilitators across the country.

With the RONIN compute cluster, the trainers were able to quickly create virtual training for event participants. “Using RONIN and AWS, we spun up training environments in minutes outside of the institutional firewall, so more than 150 bioinformaticians got access to the same virtual resources, regardless of their location,” says Watson-Haigh.

Because of the simplicity of setting up the environments, the university created and configured an image that was used when spinning up resources again, without having to be reconfigured each time. “Once we configured the image, we could just use that for the duration of the event,” Watson-Haigh says. “We were able to simply bring resources online for several hours and take them down when the workshop was completed. That helped us keep costs down.”

By automatically spinning up a training cluster, the instructors were able to seamlessly train bioinformaticians across Australia. Each participant completed two 4-hour sessions. “We trained researchers in Snakemake and Nextflow, which are two important workflow management systems in our industry,” says Watson-Haigh. “Accessing our AWS Auto Scaling cluster meant we could provide researchers with an environment that would be similar to what they likely have access to locally, without the limitations of local firewalls and security challenges. Running on AWS, we made it easier for people to access this critical training, wherever they were located.”

Based on the easy scalability of the workshops, the trainers chose to use the exact same solution at additional events. For example, it used the cluster to deliver an onsite Snakemake workshop at the Australian Bioinformatics and Computational Biology Society annual conference. “This solution is so simple to set up and configure that we expect to keep using it for our workshops, whether onsite or virtual,” says Watson-Haigh. “RONIN and AWS give us the reliability, scalability, and ease of use we need to support our training mission throughout Australia.”

To learn more, visit thinkwithwp.com/hpc.