aws-s3-multipart: fix queueing behaviours #1855
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes an issue where the plugin was queueing too many things, causing deadlock. The entire
MultipartUploadobject was added as a queue entry, but then that object would queue more things, and wait for those things to complete; but theMultipartUploadobjects were clogging up the queue, so those things would never start. Since the MultipartUpload object manages its requests internally, and it participates in the rate limiting queue, its requests don't need to be queued again.Fixes an issue where cancelling many queued uploads would cause them to quickly start and then immediately be cancelled, because cancelling the ongoing uploads would synchronously push the queued uploads to the front of the upload queue, synchronously start them, and then synchronously cancel them. Now, queue items are not moved immediately when an earlier upload is cancelled, but wait until the end of the JavaScript tick before moving. The queue can be cleared out synchronously without causing queued items to start and then cancel.
Fixes #1146, where aws-s3-multipart would send abort requests for uploads that had not yet started (either because they were queued or because the
createMultipartUpload()request was ongoing). Now, abort requests are not sent at all if an upload was not started, and abort requests wait for the createMultipartUpload() request to succeed before aborting.