AWS Compute Blog

Python 3.11 runtime now available in AWS Lambda

This post is written by Ramesh Mathikumar, Senior DevOps Consultant and Francesco Vergona, Solutions Architect.

AWS Lambda now supports Python 3.11 as both a managed runtime and container base image. Python 3.11 contains significant performance enhancements over Python 3.10. Features like reduced startup time, streamlined stack frames and CPython specialization adaptive interpreter help many workloads using Python 3.11 run faster and cheaper, thanks to Lambda’s per-millisecond billing model. With this release, Python developers can now take advantage of new features and improvements introduced in Python 3.11 when creating serverless applications on Lambda.

You can use Python 3.11 with Lambda Powertools for Python, a developer toolkit to implement Serverless best practices and increase developer velocity. Lambda Powertools includes proven libraries to support common patterns such as observability, parameter store integration, idempotency, batch processing, feature flags, and more. Learn more about PowerTools for AWS Lambda for Python in the documentation.

You can also use Python 3.11 with Lambda@Edge, allowing you to customize low-latency content delivered through Amazon CloudFront.

Python is a popular language for building serverless applications. The Python 3.11 release includes both performance improvements and new language features. For customers who deploy their Lambda functions using container image, the base image for Python 3.11 also includes changes to make managing installed packages easier.

This blog post reviews these changes in turn, followed by an overview of how you can get started with Python 3.11 in Lambda.

Performance improvements

Optimizations to CPython introduced by Python 3.11 brings significant performance enhancements, making it an average of 25% faster than Python 3.10, based on Python community benchmark tests using the Python Performance Benchmark Suite.

This release focuses on two key areas:

  • Faster startup: core modules essential for Python are now “frozen,” with statically allocated code objects, resulting in a 10–15% faster interpreter start up relative to Python 3.10.
  • Faster function execution: improvements include streamlined frame creation, in-lined Python function calls for reduced C stack usage, and the implementation of a Specializing Adaptive Interpreter, which specializes the interpreter for “hot code” (code that’s executed multiple times) and reducing the overhead during execution.

These optimizations can improve performance by 10-60% depending on the workload. In the context of a Lambda function execution, this results in performance improvements for both ”cold start“ and ”warm start“ invocations

In addition to faster CPython performance improvements, Python 3.11 also provides performance improvements across other areas. For example:

  • String formatting with printf-style% codes is now as fast as f-string expressions.
  • Integer division is around 20% faster on x86-64 for certain scenarios.
  • Operations like sum() and list resizing have seen notable speed enhancements.
  • Dictionaries save memory by not storing hash values when keys are Unicode objects.
  • Improvements to asyncio. DatagramProtocol introduce significantly faster large file transfers over UDP.
  • Math functions, statistics functions, and unicodedata.normalize() also benefit from substantial speed improvements.

Language features

Thanks to its simplicity, readability, and extensive community support, Python is a popular language for building serverless applications. The Python 3.11 release includes several new language features, including:

  • Variadic generics (PEP 646): Python 3.11 introduces TypeVarTuple, enabling parameterization with an arbitrary number of types.
  • Marking individual TypedDict items as required or not-required (PEP 655): The introduction of Required and NotRequired in TypedDict allows for explicit marking of individual item requirements, eliminating the need for inheritance.
  • Self type (PEP 673): The Self annotation simplifies the annotation of methods returning an instance of their class, similarly to TypeVar in PEP 484
  • Arbitrary literal string type (PEP 675): The LiteralString annotation allows a function parameter to accept any literal string type, including strings created from literals.
  • Data class transforms (PEP 681): The @dataclass_transform() decorator enables objects to utilize runtime transformations for dataclass-like functionalities.

For the full list of Python 3.11 changes, see the Python 3.11 release notes.

Change in pre-installed modules location and search path

Previously, Lambda base container images for Python included the /var/runtime directory before the /var/lang/lib/python3.x directory in the search path. This meant that packages in /var/runtime are loaded in preference to packages installed via pip into /var/lang/lib/python3.x. Since the AWS SDK for Python (boto3/botocore) was pre-installed into /var/runtime, this made it harder for base container images customers to upgrade the SDK version.

With the Python 3.11 runtime, the AWS SDK and its dependencies are now pre-installed into the /var/lang/lib/python3.11 directory, and the search path has been modified so this directory has precedence over /var/runtime. This change means customers who build and deploy Lambda functions using the Python 3.11 base container image can now override the SDK simply by running pip install on a newer version. This change also enables pip to verify and track that the pre-installed SDK and its dependencies are compatible with any customer-installed packages.

This is the default sys.path before Python 3.11 (where X.Y is the Python major.minor version):

  • /var/task/: User Function
  • /opt/python/lib/pythonX.Y/site-packages/: User Layer
  • /opt/python/: User Layer
  • /var/runtime/: Pre-installed modules
  • /var/lang/lib/pythonX.Y/site-packages/: Default pip install location

Here is the default sys.path starting from Python 3.11:

  • /var/task/: User Function
  • /opt/python/lib/pythonX.Y/site-packages/: User Layer
  • /opt/python/: User Layer
  • /var/lang/lib/pythonX.Y/site-packages/: Pre-installed modules and default pip install location
  • /var/runtime/: No pre-installed modules

Using Python 3.11 in Lambda

AWS Management Console

To use the Python 3.11 runtime to develop your Lambda functions, specify a runtime parameter value Python 3.11 when creating or updating a function. Python 3.11 version is now available in the Runtime dropdown in the Create function page.

Create function

To update an existing Lambda function to Python 3.11, navigate to the function in the Lambda console, then choose Edit in the Runtime settings panel. The new version of Python is available in the Runtime dropdown:

Edit function

AWS Lambda – Container Image

Change the Python base image version by modifying FROM statement in the Dockerfile:

FROM public.ecr.aws/lambda/python:3.11
# Copy function code
COPY lambda_handler.py ${LAMBDA_TASK_ROOT}

To learn more, refer to the usage tab on building functions as container images.

AWS Serverless Application Model (AWS SAM)

In AWS SAM, set the Runtime attribute to python3.11 to use this version.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Simple Lambda Function
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      Description: My Python Lambda Function
      CodeUri: my_function/
      Handler: lambda_function.lambda_handler
      Runtime: python3.11

AWS SAM supports generating this template with Python 3.11 out of the box for new serverless applications using the sam init command. Refer to the AWS SAM documentation here.

AWS Cloud Development Kit (AWS CDK)

In the AWS CDK, set the runtime attribute to Runtime.PYTHON_3_11 to use this version. In Python:

from constructs import Construct 
from aws_cdk import ( App, Stack, aws_lambda as _lambda )

class SampleLambdaStack(Stack):
    def __init__(self, scope: Construct, id: str, **kwargs) -> None:
        super().__init__(scope, id, **kwargs)
        
        base_lambda = _lambda.Function(self, 'SampleLambda', 
                                       handler='lambda_handler.handler', 
                                    runtime=_lambda.Runtime.PYTHON_3_11, 
                                 code=_lambda.Code.from_asset('lambda'))

In TypeScript:

import * as cdk from 'aws-cdk-lib';
import * as lambda from 'aws-cdk-lib/aws-lambda'
import * as path from 'path';
import { Construct } from 'constructs';

export class CdkStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // The code that defines your stack goes here

    // The python3.11 enabled Lambda Function
    const lambdaFunction = new lambda.Function(this, 'python311LambdaFunction', {
      runtime: lambda.Runtime.PYTHON_3_11,
      memorySize: 512,
      code: lambda.Code.fromAsset(path.join(__dirname, '/../lambda')),
      handler: 'lambda_handler.handler'
    })
  }
}

Conclusion

You can build and deploy functions using Python 3.11 using the AWS Management Console, AWS CLI, AWS SDK, AWS SAM, AWS CDK, or your choice of Infrastructure as Code (IaC). You can also use the Python 3.11 container base image if you prefer to build and deploy your functions using container images.

We are excited to bring Python 3.11 runtime support to Lambda and empower developers to build more efficient, powerful, and scalable serverless applications. Try Python 3.11 runtime in Lambda today and experience the benefits of this updated language version and take advantage of improved performance and new language features.
For more serverless learning resources, visit Serverless Land