Retry Throttling

In this blog post, we discuss the existing request retry feature, and the new retry throttling feature that we have rolled out in the AWS SDK for .NET V3 from version 3.3.4.0 of the AWSSDK.Core package.

In request retry, client side requests are retried, and often succeed, in cases involving transient network or service issues. The advantage to you as a client is that you don’t have to experience noise resulting from these exceptions, and are saved the trouble of writing code that would retry these requests. The downside to this retry feature is that situations such as network connectivity or unavailability, in which all retried requests fail, leads to tying up the client application thread and fail-slow behavior. The client eventually ends up getting a service unavailable exception that could have been relayed earlier, without the retry loop. This delay in surfacing the exception hurts the client’s recovery times and prolongs the client side impact. We want to walk a middle ground where we provide the retry request feature but with some limiting constraints.

Retry throttling, like its name suggests, throttles retry attempts when a large number of retry requests are failing. Each time a retry request is made, an internal retry capacity pool is drained. Retry requests are no longer made if the capacity pool is depleted. Retry requests are attempted only when the client starts getting successful responses, which refills the client’s capacity pool. Retry throttling takes care of “retry storm” situations by entering a fail-fast mode, in which the exceptions are surfaced and the needless retry loop is skipped. Also, because retry throttling kicks in only when a large number of requests and their retry attempts fail, transient retries are unaffected by this feature.

The AWS SDK for Java has already introduced this feature to great effect. Their blog post contains the metrics that compare situations when throttling is enabled versus when it is not.

AmazonS3Config config = new AmazonS3Config();
config.ThrottleRetries = false; // Default value is true