feat: add support for range requests #15
Conversation
| .err_into() | ||
| .boxed(); | ||
| Ok(GetResult::Stream( | ||
| self.get_object(location, None).await?.boxed(), |
There was a problem hiding this comment.
Moved to get_object
| format!("bytes={}-{}", range.start, range.end.saturating_sub(1)) | ||
| } | ||
|
|
||
| /// Collect a stream into [`Bytes`] avoiding copying in the event of a single chunk |
There was a problem hiding this comment.
This is a trick copied from hyper
| /// Return the bytes that are stored at the specified location | ||
| /// in the given byte range | ||
| async fn get_range(&self, location: &Path, range: Range<usize>) -> Result<Bytes>; |
There was a problem hiding this comment.
What's the expected behavior on out-of-range requests? Do they error or soft-cap at the end of the object? Would probably be nice to add a test for that behavior.
src/lib.rs
Outdated
| // Azurite doesn't support range requests | ||
| let err = range_result.unwrap_err().to_string(); | ||
| assert!(err.contains("x-ms-range-get-content-crc64 header or parameter is not supported in Azurite strict mode"), "{}", err); |
There was a problem hiding this comment.
Is there an upstream ticket for that?
src/util.rs
Outdated
| /// Returns the range to be passed to an object store | ||
| #[cfg(any(feature = "aws"))] | ||
| pub fn format_range(range: std::ops::Range<usize>) -> String { | ||
| format!("bytes={}-{}", range.start, range.end.saturating_sub(1)) | ||
| } |
There was a problem hiding this comment.
This is AWS-specific, no? If it's specific, it should probably live within the AWS module. If it's not AWS-specific but follows some common HTTP standard, it would be nice to maybe rename the function to format_http_range and cite a standard.
I think for the limited subset of functionality I would support that, not only for GCP. All the official upstream crates just bring a massive amount of dependencies. |
This adds support for range requests, unfortunately:
I personally wonder about just rolling our own GCS crate, it is a really straightforward API, with simple token-based auth, and #14 also runs into limitations of this crate