Amazon EC2 Trn2 instances are generally available
Today, AWS announces the general availability of Amazon Elastic Compute Cloud (Amazon EC2) Trn2 instances and preview of Trn2 UltraServers, powered by AWS Trainium2 chips. Available via EC2 Capacity Blocks, Trn2 instances and UltraServers are the most powerful EC2 compute solutions for deep learning and generative AI training and inference.
You can use Trn2 instances to train and deploy the most demanding foundation models including large language models (LLMs), multi-modal models, diffusion transformers and more to build a broad set of AI applications. To reduce training times and deliver breakthrough response times (per-token-latency) for the most capable, state-of-the-art models you might need more compute and memory than a single instance can deliver. Trn2 UltraServers is a completely new EC2 offering that uses NeuronLink, a high-bandwidth, low-latency fabric, to connect 64 Trainium2 chips across 4 Trn2 instances into one node unlocking unparalleled performance. For inference, UltraServers help deliver industry-leading response times to create the best real-time experiences. For training, UltraServers boost model training speed and efficiency with faster collective communication for model parallelism as compared to standalone instances.
Trn2 instances feature 16 Trainium2 chips to deliver up to 20.8 petaflops of FP8 compute, 1.5 TB high bandwidth memory with 46 TB/s of memory bandwidth, and 3.2 Tbps of EFA networking. Trn2 UltraServers feature 64 Trainium2 chips to deliver up to 83.2 petaflops of FP8 compute, 6 TB of total high bandwidth memory with 185 TB/s of total memory bandwidth, and 12.8 Tbps of EFA networking. They both are deployed in EC2 UltraClusters to provide non-blocking, petabit scale-out capabilities for distributed training. Trn2 instances are generally available in the trn2.48xlarge size in the US East (Ohio) AWS Region through EC2 Capacity Blocks for ML.
To learn more about Trn2 instances and request access to Trn2 UltraServers please visit the Trn2 instances page.