AWS Machine Learning Blog

Secure prediction calls in Amazon SageMaker with AWS PrivateLink

Amazon SageMaker now supports Amazon Virtual Private Cloud (VPC) Endpoints via AWS PrivateLink so you can initiate prediction calls to your machine learning models hosted on Amazon SageMaker inside your VPC, without going over the internet.

Amazon SageMaker is a fully-managed platform that enables developers and data scientists to quickly and easily build, train, and deploy machine learning models at any scale. After a machine learning model is deployed into production using Amazon SageMaker, it sets up a secure HTTPS endpoint for your application. Your application simply needs to use the SageMaker Runtime API to achieve low latency / high throughput predictions. With AWS PrivateLink support, the SageMaker Runtime API can be called through an interface endpoint within the VPC instead of connecting over the internet. Since the communication between the client application and the SageMaker Runtime API is inside the VPC, there is no need for an Internet Gateway, a NAT device, a VPN connection, or AWS Direct Connect.

You can create a VPC interface endpoint to connect to SageMaker Runtime using AWS Command Line Interface (AWS CLI) commands or using the AWS Management Console. To create a VPC endpoint from the console, navigate to the Amazon VPC console, open the Endpoints page, and create a new Endpoint.

Three attributes are required:

  1. The SageMaker Runtime service name. Select AWS services as the Service category and select amazonaws.us-east-2.sagemaker.runtime.
  2. The VPC and Availability Zones that you want to use.
  3. The security group to be associated with the endpoint network interface. If you don’t specify a security group, the default security group for your VPC will be associated.

After you have created a VPC endpoint, you can use the following example CLI commands that use the endpoint-url parameter to specify interface endpoints to Amazon SageMaker Runtime:

aws sagemaker-runtime invoke-endpoint –-endpoint-url VPC_Endpoint_ID.runtime.sagemaker.Region.vpce.amazonaws.com  \
    --endpoint-name Endpoint_Name \
    --body "Endpoint_Body" \
    --content-type "Content_Type" \
Output_File

Optionally, if you enable private DNS hostnames for your VPC endpoint, you don’t need to specify the endpoint URL.

A private hosted zone enables you to access the resources in your VPC using custom DNS domain names, such as example.com, instead of using private IPv4 addresses or AWS-provided private DNS hostnames. The Amazon SageMaker Runtime DNS hostname that the CLI and Amazon SageMaker Runtime SDKs use by default (https://runtime.sagemaker.Region.amazonaws.com) resolves to your VPC endpoint.

Summary

Amazon SageMaker prediction calls are now supported on AWS PrivateLink, keeping them secure from the internet. You can initiate prediction calls to your machine learning models hosted on Amazon SageMaker inside your VPC without going over the internet.

VPC Endpoints for SageMaker Runtime is now available in all AWS Regions where both Amazon VPC and Amazon SageMaker are available.

About the Author

Urvashi Chowdhary is a Senior Product Manager for Amazon SageMaker. She is passionate about working with customers and making machine learning more accessible. In her spare time, she loves sailing, paddle boarding, and kayaking.