-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Blob][Fix]fix downloader.chunks() return chunks in different size #17559
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work and test LGTM :) please add the comment I mentioned in my review for that case as it will be helpful to understand the code for future devs.
|
||
return self._current_content | ||
return self._get_chunk_data() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the reason for calling self._get_chunk_data()
in this case? as far as I understand if we hit this case (we're done iterating the first single get call and now iterating through our download chunks like we usually do) then current content should always be the same as the chunk size so we can just return self.current_content
. It seems from your try-except
that there is an edge case where there will be excess content - could you just put a small comment on that in the code - its not obvious to me haha
EDIT: I just realized the case is when the current content from the first get is still there but smaller than chunk size therefore we want to make sure its also included. I think adding a comment in the code would be very helpful here!
Good work thinking of all the cases
when iterate through downloader.chunks() the chunk size is different. The first chunk size is always equals to max_single_get_size while it should equal to max_chunk_get_size
This will fix #9419 and #15648