AWS HPC Blog
Tag: AWS Batch
Application deep-dive into the AWS Graviton3E-based Amazon EC2 Hpc7g instance
In this post we’ll show you application performance and scaling results from Hpc7g, a new instance powered by AWS Graviton3E across a wide range of HPC workloads and disciplines.
Accelerating the shift towards a sustainable economy using HPC on AWS
The transition to a sustainable economy is a major goal of many organizations today. HPC plays a pivotal role and in this post we’ll explore how HPC on AWS is enabling the shift towards a sustainable future.
How SeatGeek simulates massive load with AWS Batch to prepare for big events
In this post we explore SeatGeek’s load testing system that simulates 50k simultaneous users. Originally built to prep SeatGeek for large-event traffic spikes, it now runs weekly to help them harden their code.
Streamlining distributed ML workflow orchestration using Covalent with AWS Batch
Complicated multi-step workflows can be challenging to deploy, especially when using a variety of high-compute resources. Covalent is an open-source orchestration tool that streamlines the deployment of distributed workloads on AWS resources. In this post, we outline key concepts in Covalent and develop a machine learning workflow for AWS Batch in just a handful of steps.
Run Celery workers for compute-intensive tasks with AWS Batch
Many applications leverage distributed task systems like Celery to handle asynchronous work. In this post, we describe how to handle compute-intensive Celery tasks using AWS Batch to scale the compute resources and run worker agents.
How Evolvere Biosciences performs macromolecule design on AWS
In this blog post, we catch a glimpse into drug discovery to see how Evolvere Biosciences has deployed a customized architecture w/ AWS Batch and Nextflow to quickly and easily run its macromolecule design pipeline.
Explore costs of AWS Batch jobs run on Amazon EKS using pod labels and Kubecost
Today we show you how to get insights into the costs of running AWS Batch workloads on Amazon EKS using Kubernetes pod labels with Kubecost.
Second generation EFA: improving HPC and ML application performance in the cloud
Since launch, EFA has seen continuous improvements in performance. In this post, we talk about our 2nd generation of EFA, which takes another step in improving Machine Learning and High Performance Computing in the Cloud.
Avoid overspending with AWS Batch using a serverless cost guardian monitoring architecture
Pay-as-you-go resources are a compelling but budget-limited researchers performing HPC workloads need help working within the bounds of their grants. In this post, we show how to build a real-time cost guardian for AWS Batch to help enforce those limits.
How AWS Batch developed support for Amazon Elastic Kubernetes Service
Today, we discuss AWS batch on Amazon EKS, and the initial motivation and design choices the team made when we developed the service, and some of the challenges to overcome.