Listing Thumbnail

    ParaTools Pro for E4S™: AI/ML & HPC Tools on ParallelCluster (x86)

     Info
    Free Trial
    AWS Free Tier
    This product has charges associated with it for optional seller support and pre-configured stack (ParaTools Pro for E4S™). ParaTools Pro for E4S™ is the hardened Extreme-scale Scientific Software Stack using AWS Parallel Cluster, with over 100 HPC and AI/ML tools, a remote desktop environment (DCV), and cluster configuration built using the Spack package manager and the proprietary MVAPICH MPI tuned for AWS Elastic Fabric adapter (EFA). It is a platform for developing AI/ML applications using tools such as NVIDIA NeMo™, TensorFlow, PyTorch, JAX, Keras, and Horovod tuned for AWS ParallelCluster, with a complete HPC software stack including numerical libraries (such as PETSc, Trilinos, SuperLU-dist), visualization tools (such as ParaView and VisIt), performance evaluation tools (such as TAU and HPCToolkit), HPC applications (such as OpenFOAM, LAMMPS, Xyce, CP2K, deal.II, GROMACS and Quantum Espresso).
    Listing Thumbnail

    ParaTools Pro for E4S™: AI/ML & HPC Tools on ParallelCluster (x86)

     Info

    Overview

    ParaTools Pro for E4S™1  - the Extreme-scale Scientific Software Stack E4S2  hardened for commercial clouds and supported by ParaTools, Inc. provides a platform for developing and deploying HPC and AI/ML applications. It features a performant remote desktop environment (based on DCV) on the login node and compute nodes interconnected by a low-latency, high bandwidth network adapter based on AWS Elastic Fabric Adapter (EFA). ParaTools Pro for E4S™ features a suite of over 100 HPC tools built using the Spack3  package manager and the proprietary MVAPICH MPI tuned for EFA. It features ready to use HPC applications (such as OpenFOAM, LAMMPS, Xyce, CP2K, deal.II, GROMACS, Quantum Espresso) as well as AI/ML tools based on Python (such as NVIDIA NeMo™, TensorFlow, PyTorch, JAX, Horovod, Keras, OpenCV, matplotlib and supports Jupyter notebooks) and the Codium IDE. New packages can be easily installed using Spack and pip and are accessible on the cluster compute and login nodes. It may be used for developing the next generation of generative AI applications using a suite of Python tools and interfaces.

    E4S has built a unified computing environment for deployment of open-source projects. E4S was originally developed to provide a common software environment for the exascale leadership computing systems currently being deployed at DOE National Laboratories across the U.S. Support for ParaTools Pro for E4S™ is available through ParaTools, Inc. This product has additional charges associated with it for optional product support and updates.

    This material is based upon work supported by the U.S. Department of Energy, Office of Science, Office of Office of Advanced Scientific Computing and Research (ASCR), under SBIR Award Number DE-SC0022502 ("E4S: Extreme-Scale Scientific Software Stack for Commercial Clouds").

    Note: This product contains repackaged and tuned open source software (e.g., E4S, Spack and AI/ML tools like NVIDIA NeMo™, Horovod, JAX, Keras etc.) which is configured and linked against a proprietary MVAPICH MPI implementation specifically developed and tuned for EFA.

    Highlights

    • Full E4S™ and Machine Learning stacks built and optimized for AWS EFA and ParallelCluster
    • A proprietary MVAPICH2 implementation that offers lower latency and higher throughput than default OpenMPI implementations for pre-installed applications and user installed applications
    • HPC software environment managed using the Spack package manager

    Details

    Delivery method

    Delivery option
    64-bit (x86) Amazon Machine Image (AMI)

    Latest version

    Operating system
    Ubuntu 22.04

    Typical total price

    This estimate is based on use of the seller's recommended configuration (c5n.9xlarge) in the US East (N. Virginia) Region. View pricing details

    $2.934/hour

    Features and programs

    Financing for AWS Marketplace purchases

    AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
    Financing for AWS Marketplace purchases

    Pricing

    Free trial

    Try this product at no cost for 31 days according to the free trial terms set by the vendor. Usage-based pricing is in effect for usage beyond the free trial terms. Your free trial gets automatically converted to a paid subscription when the trial ends, but may be canceled any time before that.

    ParaTools Pro for E4S™: AI/ML & HPC Tools on ParallelCluster (x86)

     Info
    Pricing is based on actual usage, with charges varying according to how much you consume. Subscriptions have no end date and may be canceled any time.
    Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator  to estimate your infrastructure costs.

    Usage costs (568)

     Info
    • ...
    Instance type
    Product cost/hour
    EC2 cost/hour
    Total/hour
    t2.large
    $0.99
    $0.093
    $1.083
    t2.xlarge
    $0.99
    $0.186
    $1.176
    t2.2xlarge
    $0.99
    $0.371
    $1.361
    t3.nano
    $0.99
    $0.005
    $0.995
    t3.micro
    AWS Free Tier
    $0.99
    $0.01
    $1.00
    t3.small
    $0.99
    $0.021
    $1.011
    t3.medium
    $0.99
    $0.042
    $1.032
    t3.large
    $0.99
    $0.083
    $1.073
    t3.xlarge
    $0.99
    $0.166
    $1.156
    t3.2xlarge
    $0.99
    $0.333
    $1.323

    Additional AWS infrastructure costs

    Type
    Cost
    EBS General Purpose SSD (gp3) volumes
    $0.08/per GB/month of provisioned storage

    Vendor refund policy

    Refund Policy: The standard refund policy outlined here will be followed: https://docs.thinkwithwp.com/marketplace/latest/userguide/refunds.html 

    Any additional refund enquiries can be sent to support@paratools.com  and will be considered individually on a case by case basis.

    Legal

    Vendor terms and conditions

    Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .

    Content disclaimer

    Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.

    Usage information

     Info

    Delivery details

    64-bit (x86) Amazon Machine Image (AMI)

    Amazon Machine Image (AMI)

    An AMI is a virtual image that provides the information required to launch an instance. Amazon EC2 (Elastic Compute Cloud) instances are virtual servers on which you can run your applications and workloads, offering varying combinations of CPU, memory, storage, and networking resources. You can launch as many instances from as many different AMIs as you need.

    Version release notes

    AWS ParallelCluster compatibility: pcluster v3.12.0 E4S release: 24.05 Inter-node communication requirement: EFA compatible instances Default MPI: mvapich2-x-aws OS: Ubuntu 22.04

    CRITICAL:

    • Some versions prior to v2024.05.16.1904-pcluster-3.9.1-e4s-24.05-amd64 are exposed to the Log4Shell (CVE-2021-44228) vulnerability. (Vulnerable log4js versions were getting pulled in as a dependency of hive.) Please upgrade as soon as possible to this version or a version equal to or newer than v2024.05.16.1904-pcluster-3.9.1-e4s-24.05-amd64 to mitigate this vulnerability.
    • Recent changes to AWS EFA instances have broken the MVAPICH2-X-AWS installation on older versions of this product. Please upgrade to this version or a version equal to or newer than version v2024.01.25.1225-pcluster-3.8.0-e4s-23.11-amd64 to ensure proper operation of MVAPICH2-X-AWS and the installed software stack.

    IMPORTANT:

    1. To use this product with AWS ParallelCluster, you must use the same version of parlallel cluster, v3.12.0
    2. The version of MVAPICH2-X-AWS installed requires compute node instances to have EFA support for inter-node communication. If you do not plan to run multi-node MPI jobs you can safely disregard this. The head node may be a non-EFA instance type, but the compute nodes MUST be EFA capable. To get a list of instance types that are EFA compatible you can use the AWS CLI:
    aws ec2 describe-instance-types --filters Name=network-info.efa-supported,Values=true \ --query InstanceTypes[*].[InstanceType] \ --output text | sort

    Updates in this version:

    1. Upgrade parallel cluster version to v3.12.0
    2. Improvements to NeMo installation/environment
    3. Kubernetes 1.30 installed

    ParaTools Pro for E4S (TM) provides a turn-key cloud HPC solution with the E4S Software stack and machine learning tools and SDKs installed and optimized to run on AWS. The key software features and versions installed are:

    • Ubuntu 22.04, X86_64
    • DCV remote access installed
    • Codium IDE
    • Kubernetes
    • E4S 24.05 CPU applications installed
    • NVIDIA NeMo bare-metal install v2.0.0rc0
    • MVAPICH2-X-AWS (EFA is required for inter-node communication, optimized for CPU, not CUDA aware, but GDR support will be availablie in future versions)
    • Python/Machine learning tools, SDKs, Frameworks etc.
      • TensorFlow, PyTorch, JAX, Horovod, Keras, OpenCV, OpenAI, Scikit-Learn
      • numpy, scipy, matplotlib, pandas, opencv-python
      • NCCL 2.19.3
      • jupyter notebook
      • pywaggle

    The full list of E4S applications installed via Spack is as follows:

    -- linux-ubuntu22.04-x86_64 / gcc@11.4.0 -------------- adios2@2.10.0 adios@1.13.1 alquimia@1.1.0 aml@0.2.1 amrex@24.04 arborx@1.6 argobots@1.2 ascent@0.9.2 axom@0.9.0 boost@1.79.0 bricks@2023.08.25 butterflypack@2.4.0 cabana@0.6.0 caliper@2.10.0 chai@2024.02.0 charliecloud@0.37 conduit@0.9.1 cp2k@2024.1 cusz@0.3.1 darshan-runtime@3.4.4 darshan-util@3.4.4 datatransferkit@3.1.1 dealii@9.5.1 dyninst@13.0.0 e4s-alc@1.0.2 e4s-cl@1.0.4 ecp-data-vis-sdk@1.0 exago@1.6.0 faodel@1.2108.1 flecsi@2.2.1 flit@2.1.0 flux-core@0.61.2 fortrilinos@2.3.0 fpm@0.10.0 gasnet@2023.9.0 ginkgo@1.7.0 globalarrays@5.8.2 glvis@4.2 gmp@6.2.1 gotcha@1.0.6 gptune@4.0.0 gromacs@2024.1 h5bench@1.4 hdf5-vol-async@1.7 hdf5-vol-cache@v1.1 hdf5-vol-log@1.4.0 hdf5@1.12.2 hdf5@1.12.3 hdf5@1.14.3 heffte@2.4.0 hpctoolkit@2024.01.1 hpx@1.9.1 hypre@2.31.0 kokkos-kernels@4.3.00 kokkos@4.3.00 laghos@3.1 lammps@20230802.3 lbann@0.104 legion@24.03.0 libcatalyst@2.0.0-rc4 libnrm@0.1.0 libpressio@0.95.1 libquo@1.4 libunwind@1.6.2 loki@0.1.7 magma@2.8.0 mercury@2.3.1 metall@0.25 mfem@4.6.0 mgard@2023-12-09 mpark-variant@1.4.0 mpich@4.2.1 mpifileutils@0.11.1 nccmp@1.9.1.0 nco@5.1.9 nekbone@17.0 netcdf-fortran@4.6.1 netlib-scalapack@2.2.0 nrm@0.1.0 nvhpc@24.3 omega-h@9.34.13 openfoam@2312 openmpi@4.1.6 openpmd-api@0.15.2 papi@7.1.0 papyrus@1.0.2 parallel-netcdf@1.12.3 paraview@5.12.0 parsec@3.0.2209 pdt@3.25.2 petsc@3.21.0 phist@1.12.0 plasma@23.8.2 plumed@2.9.0 precice@3.1.1 pruners-ninja@1.0.1 pumi@2.2.8 py-cinemasci@1.7.0 py-deephyper@0.6.0 py-h5py@3.11.0 py-jupyterhub@1.4.1 py-libensemble@1.2.2 py-petsc4py@3.21.0 py-warpx@23.08 qthreads@1.18 quantum-espresso@7.3.1 raja@2024.02.0 rempi@1.1.0 scr@3.0.1 slate@2023.11.05 slepc@3.21.0 strumpack@7.2.0 sundials@7.0.0 superlu-dist@8.2.1 superlu@5.3.0 swig@4.0.2-fortran sz3@3.1.7 sz@2.1.12.5 tasmanian@8.0 tau@2.33.2 trilinos@13.0.1 trilinos@15.1.1 umap@2.1.0 umpire@2024.02.0 unifyfs@2.0 upcxx@2023.9.0 variorum@0.7.0 veloc@1.7 vtk-m@2.0.0 vtk-m@2.1.0 wannier90@3.1.0 xyce@7.8.0 zfp@0.5.5

    Additional details

    Usage instructions

    The 1-Click Security Group opens port 22 only so that you can access your instance via SSH using login 'ubuntu', you may change this later.

    For software development and basic usage:

    1. Launch the ParaTools Pro for E4S (TM) AMI via 1-Click
    2. On the 'EC2 Launch an Instance' page pick the key pair you will use to login
    3. On the 'EC2 Launch an Instance' page optionally edit the network settings by pressing the edit button. Adjust the firewall rules if needed to ensure ssh access and enable Auto-assign public IP if you plan to access the instance remotely from a non-AWS IP address.
    4. Click 'Launch Instance'
    5. Find your running instance in the EC2 Instances section of the EC2 dashboard, and connect to the instance via SSH using the key pair you previously selected by picking the instance and pressing the connect button.

    For more advanced usage, including launching an ParaTools Pro for E4S (TM) cluster with AWS ParallelCluster, please see:

    https://docs.paratoolspro.com/AWS/getting-started-AWS/ 

    Support

    Vendor support

    For general support questions, please email support@paratools.com 

    Paid support contracts and custom AMIs and computing environments are available. Please see https://paratoolspro.com/  for additional details.

    AWS infrastructure support

    AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.

    Similar products

    Customer reviews

    Ratings and reviews

     Info
    0 ratings
    5 star
    4 star
    3 star
    2 star
    1 star
    0%
    0%
    0%
    0%
    0%
    0 AWS reviews
    No customer reviews yet
    Be the first to write a review for this product.