HTTP keep-alive is on by default in modular AWS SDK for JavaScript

On December 15th, 2020, we announced the general availability of the AWS SDK for JavaScript, version 3 (v3). In v3, we keep the Node.js HTTP connections alive by default. This blog post explains how it’s done. It also describes the benefits of keeping the sockets around, so they can be used for future requests without having to reestablish a TCP connection.

Motivation

The HTTP/1.1 relies on Transmission Control Protocol (TCP) at its transport layer. TCP is connection-oriented, and a connection between client and server is established before data can be sent. To establish a connection, TCP uses a three-way handshake using SYN, SYN+ACK and ACK packets as shown below.

The delay imposed by the three-way handshake makes the new TCP connection expensive to create compared to reusing already open connections. Reusing a connection also avoids the overhead of making a DNS lookup and performing an SSL handshake, thus improving application performance.

Background

As per the HTTP/1.1 spec explained in RFC 2616, the persistent connections, also called as HTTP keep-alive, or HTTP connection reuse, are the default behavior. Unless otherwise indicated, the client should assume that the server will maintain a persistent connection, even after error responses from the server. However, the HTTP client of Node.js doesn’t enable persistent connections by default.

In Node.js, the http.Agent maintains a queue of pending requests for a given host and port, reusing a single socket connection for each until the queue is empty, at which time the socket is destroyed by default. This is because the keepAlive option is set to false by default.

If keepAlive option is explicitly set true during creation of Node.js http.Agent, the socket is put into a pool after the queue of pending requests gets empty. This socket is reused for the future requests to the same host and port instead of creating new one.

For short-lived operations, such as DynamoDB queries, the latency overhead of setting up a TCP connection might be greater than the operation itself. Additionally, since DynamoDB encryption at Rest is integrated with AWS KMS, you may experience latencies from the database having to re-establish new AWS KMS cache entries for each operation. Using persistent connections is also recommended by Lambda as an optimization tip.

In AWS SDK for JavaScript v2, the keepAlive option was not set by default. Our customer often requested turning on keepAlive by default in GitHub Issues to help improve application performance.

Implementation

The persistent connections were enabled in modular AWS SDK for JavaScript by setting up keepAlive by default while creating Node.js http.Agent. This is done in NodeHttpHandler internal code as follows:

this.httpAgent = httpAgent || new http.Agent({ keepAlive: true });
this.httpsAgent = httpsAgent || new https.Agent({ keepAlive: true });

Although not recommended, you can explicitly disable persistent connections by passing a your own httpsAgent with keepAlive not defined or set to false.

const { Agent } = require("https");
const { DynamoDBClient } = require("@aws-sdk/client-dynamodb");
const { NodeHttpHandler } = require("@aws-sdk/node-http-handler");

const dynamodbClient = new DynamoDBClient({
  requestHandler: new NodeHttpHandler({
    httpsAgent: new Agent({ keepAlive: false }),
  }),
});

Benchmarks

The following benchmark code calls DynamoDB.listTables operation in a for loop 100 times, and uses a timer to track how long all the operations take to complete.

const { Agent } = require("https");
const { DynamoDB } = require("@aws-sdk/client-dynamodb");
const { NodeHttpHandler } = require("@aws-sdk/node-http-handler");

const makeListTablesCalls = async (client, count) => {
  const { keepAlive } = client.config.requestHandler.httpsAgent;
  console.time(`keep-alive ${keepAlive}`);
  for (let i = 0; i < count; i++) {
    await client.listTables({});
  }
  console.timeEnd(`keep-alive ${keepAlive}`);
};

(async () => {
  const region = "us-east-1";
  const COUNT = 100;

  await makeListTablesCalls(new DynamoDB({ region }), COUNT);

  await makeListTablesCalls(
    new DynamoDB({
      region,
      requestHandler: new NodeHttpHandler({
        httpsAgent: new Agent({ keepAlive: false }),
      }),
    }),
    COUNT
  );
})();

The benchmark code with default keep-alive true takes ~60% less time than the code with keep-alive false.

keep-alive true: 9.864s
keep-alive false: 26.594s

Feedback

We value your feedback, so please tell us what you like and don’t like by opening an issue on GitHub.