Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(Core): fixes limit stream errors #7117

Merged

Conversation

vishwarajanand
Copy link
Contributor

@vishwarajanand vishwarajanand commented Mar 3, 2024

Fixes #7088

Fix as per this google doc

Test before fix
➜  Storage git:(fix_resumable_uploader_crash) ✗ php issue_7088.php
Network error while uploading file to bucket, trying to reconnect and continue uploading
Resuming
PHP Fatal error:  Uncaught TypeError: GuzzleHttp\Psr7\LimitStream::__construct(): Argument #3 ($offset) must be of type int, null given, called in /usr/local/google/home/vishwarajanand/github/google-cloud-php/Storage/vendor/google/cloud-core/src/Upload/ResumableUploader.php on line 158 and defined in /usr/local/google/home/vishwarajanand/github/google-cloud-php/Storage/vendor/guzzlehttp/psr7/src/LimitStream.php:32
Stack trace:
#0 /usr/local/google/home/vishwarajanand/github/google-cloud-php/Storage/vendor/google/cloud-core/src/Upload/ResumableUploader.php(158): GuzzleHttp\Psr7\LimitStream->__construct()
#1 /usr/local/google/home/vishwarajanand/github/google-cloud-php/Storage/vendor/google/cloud-core/src/Upload/ResumableUploader.php(138): Google\Cloud\Core\Upload\ResumableUploader->upload()
#2 /usr/local/google/home/vishwarajanand/github/google-cloud-php/Storage/issue_7088.php(39): Google\Cloud\Core\Upload\ResumableUploader->resume()
#3 {main}
  thrown in /usr/local/google/home/vishwarajanand/github/google-cloud-php/Storage/vendor/guzzlehttp/psr7/src/LimitStream.php on line 32
➜  Storage git:(fix_resumable_uploader_crash) ✗ 
Test after fix:
➜  Storage git:(fix_resumable_uploader_crash) ✗ php issue_7088.php
Network error while uploading file to bucket, trying to reconnect and continue uploading
Resuming
Resuming
Uploaded {
    "kind": "storage#object",
    "id": "gcloXXXXXXa8\/test\/issue_7088\/1708986505212283",
    "selfLink": "https:\/\/www.googleapis.com\/storage\/v1\/b\/gcloXXXXXXa8\/o\/test%2Fissue_7088",
    "mediaLink": "https:\/\/storage.googleapis.com\/download\/storage\/v1\/b\/gcloXXXXa8\/o\/test%2Fissue_7088?generation=1708986505212283&alt=media",
    "name": "test\/issue_7088",
    "bucket": "gcloXXXXea8",
    "generation": "1709471285379177",
    "metageneration": "1",
    "contentType": "application\/octet-stream",
    "storageClass": "STANDARD",
    "size": "601457281",
    "md5Hash": "Szwlox2XSBCDj1PSp2r0jQ==",
    "crc32c": "o5mfPw==",
    "etag": "COmA4OWU2IQDEAE=",
    "timeCreated": "2024-03-03T13:08:05.408Z",
    "updated": "2024-03-03T13:08:05.408Z",
    "timeStorageClassUpdated": "2024-03-03T13:08:05.408Z"
}
➜  Storage git:(fix_resumable_uploader_crash) ✗ 

test scripts

@vishwarajanand vishwarajanand marked this pull request as ready for review March 4, 2024 03:57
@vishwarajanand vishwarajanand requested review from a team as code owners March 4, 2024 03:57
Copy link
Contributor

@yash30201 yash30201 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anand has confirmed with the Storage team that in the event when we get 308 response without any headers from the backend, we can just restart the object uploading from the starting.

Moreover, it seems #7088 issue is very rare as the user encountered it just two times on a project of 1000+ uploads per day. So resuming the upload from the starting seems like a good choice.

LGTM

@vishwarajanand vishwarajanand merged commit fd396c8 into googleapis:main Mar 6, 2024
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

storage: sometimes ResumableUploader crashes on resume()
3 participants