SDK does not consider "read: connection reset" as retryable

[swami-m]

Please fill out the sections below to help us address your issue.

Version of AWS SDK for Go?

v1.23.22

Version of Go (go version)?

v1.6.x

What issue did you see?

Recently we migrated from SDK v1.16.15 to v1.23.22 and started seeing a lot of errors causing job failures. Here's an example (sanitized) error:

RequestError: send request failed
caused by: Get <path_to_s3_object>: read tcp <snip/>: read: connection reset by peer.

These jobs complete successfully if we rollback the SDK. Should we expect these errors to happen more frequently in the newer version ? If so, why isn't this considered retryable by AWS ?

Steps to reproduce

Since these are transient errors, its hard to reproduce them.

If you have an runnable example, please include it.

[diehlaws]

Hi @swami-m, thanks for reaching out to us. Connection resets can be caused by a number of things, the most common of which is an unstable connection to the service(s) with which you are interacting from the host running your code. As @skmcgrail mentioned in this comment for PR #2926 this behavior was introduced as part of commit c3d2710 because the SDK cannot determine the state of the API call on which the connection was reset, and we do not want to have the SDK blindly re-issue non-idempotent requests that were received and processed by the service before the connection was reset.

If you are confident that the requests you are issuing with the SDK are idempotent (or don't mind if non-idempotent requests upon which the connection is reset are duplicated from the service end) you can implement a custom retryer that does retry connection reset errors.

[swami-m]

@diehlaws thanks for the explanation and the pointer to the other thread.
Does the AWS Go SDK team have any plans to implement a helper utility mentioned in this comment ?

If not, then we'll go ahead and make the necessary changes on our end to retry these types of errors wherever appropriate.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.