Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify get_object by waiting for response headers #1171

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

passaro
Copy link
Contributor

@passaro passaro commented Nov 27, 2024

S3CrtClient::get_object was originally implemented so that it would complete immediately and return a GetObjectRequest implementation (extending Stream) to retrieve body parts. Any error from the S3 request would be returned through the stream.
We recently added additional methods (get_object_metadata in #1065 and get_object_checksum in #1123) to the response that rely on the headers returned by the (first) GetObject request. The new methods required an async signature and a complicated implementation in order to account for failures and they still do not correctly report accurate error information in some cases.
With this change, we modify get_object to await for response headers before returning either an error or a GetObjectResponse (note the name change) implementation. The ergonomics of get_object are improved:

  • awaiting the initial call can already return some errors (e.g. bucket/key not found),
  • get_object_checksum and get_object_metadata are now sync functions.

Does this change impact existing behavior?

Yes, get_object behavior is different, get_object_checksum and get_object_metadata signatures have changed, and GetObjectRequest was renamed to GetObjectResponse.

Does this change need a changelog entry?

Yes, it requires a breaking change entry for mountpoint-s3-client.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

@passaro passaro temporarily deployed to PR integration tests November 27, 2024 10:55 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 10:55 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 10:55 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 10:55 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 10:56 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 10:56 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 10:56 — with GitHub Actions Inactive
passaro added a commit to passaro/mountpoint-s3 that referenced this pull request Nov 27, 2024
Signed-off-by: Alessandro Passaro <[email protected]>
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 11:04 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 11:04 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 11:04 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 11:04 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 11:04 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 11:04 — with GitHub Actions Inactive
@passaro passaro marked this pull request as ready for review November 27, 2024 11:05
@passaro passaro temporarily deployed to PR integration tests November 27, 2024 11:28 — with GitHub Actions Inactive
headers = headers_receiver => headers.unwrap(),
result = request => {
// If we did not received the headers first, the request must have failed.
result?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This syntax is a little confusing on first read. Do you think something like result.err().or(...) would be more readable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, I've gone back and forth with it. An alternative would be this:

return Err(result.err().unwrap_or_else(|| ObjectClientError::ClientError(S3RequestError::InternalError(Box::new(ObjectHeadersError::MissingHeaders)))));

Not sure which one is less readable...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM to me now, but the crucial bit here, which requires some thinking to figure out, is that request will never be returned with S3GetObjectResponse if awaiting it already returned Poll::Ready.

Otherwise, awaiting on S3GetObjectResponse may block forever in some edge cases (given that S3HttpRequest::receiver is Fuse).

I'd add an assert!(!request.is_terminated()); before the Ok(..) return statement or protect from this possible bug in S3GetObjectResponse::poll_next or at least a comment.

passaro added a commit to passaro/mountpoint-s3 that referenced this pull request Nov 29, 2024
Signed-off-by: Alessandro Passaro <[email protected]>
@passaro passaro temporarily deployed to PR integration tests November 29, 2024 08:40 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 29, 2024 08:40 — with GitHub Actions Inactive
@passaro passaro temporarily deployed to PR integration tests November 29, 2024 08:40 — with GitHub Actions Inactive
github-merge-queue bot pushed a commit that referenced this pull request Dec 2, 2024
Update the CRT libraries to the latest releases. In particular, include:
* Amazon S3 introduces support for AWS Dedicated Local Zones.
([awslabs/aws-c-s3#465](awslabs/aws-c-s3#465))
 

<details>
  <summary>Full CRT changelog:</summary>
  
```
Submodule mountpoint-s3-crt-sys/crt/aws-c-auth 48d647bf..3982bd75:
  > Update CMake to 3.9 (#255)
Submodule mountpoint-s3-crt-sys/crt/aws-c-cal 2cb1d2ea..fbbe2612:
  > RSA PKCS1.5 SHA1 signing (#201)
  > chore: Modified bug issue template to add checkbox to report potential regression. (#199)
  > Update CMake to 3.9 (#200)
Submodule mountpoint-s3-crt-sys/crt/aws-c-common f58e807d..be8ed873:
  > Fix test that made no sense (#1172)
  > Update CBMC proof tooling to latest releases (#1164)
  > Forward CMake variables to prebuilding dependencies (#1161)
  > Remove reliance on hardcoded user in ci (#1170)
  > Doc fix for cbor (#1171)
  > switch c compiler check to different cmake variable (#1169)
  > disable visibility hidden on old gcc (#1167)
  > fix empty xml node handling (#1168)
  > Unlink shutdown callback from ref count (#1166)
  > check if numa available or not before loading numa functions (#1163)
  > chore: Modified bug issue template to add checkbox to report potential regression. (#1151)
  > Update CMake to 3.9 (#1159)
  > Support Swift CXX Interop  (#1160)
Submodule mountpoint-s3-crt-sys/crt/aws-c-compression f36d0167..c6c1191e:
  > Update CMake to 3.9 (#70)
Submodule mountpoint-s3-crt-sys/crt/aws-c-http 6068653e..fc3eded2:
  > Update for event loop API changes (#491)
  > Add cxx support (#490)
  > chore: Modified bug issue template to add checkbox to report potential regression. (#486)
  > Update CMake to 3.9 (#489)
  > Tweak error message for AWS_ERROR_HTTP_RESPONSE_FIRST_BYTE_TIMEOUT (#488)
Submodule mountpoint-s3-crt-sys/crt/aws-c-io e3637404..fcb38c80:
  > Add an Option to disable retries (#694)
  > Update checksum based on previous PR changes (#695)
  > Add ML-KEM Support (#693)
  > Event loop public api (#691)
  > Add cxx support (#689)
  > Fix s2n cleanup (#687)
  > chore: Modified bug issue template to add checkbox to report potential regression. (#671)
  > Update CMake to 3.9 (#686)
Submodule mountpoint-s3-crt-sys/crt/aws-c-s3 16701501..45894ed3:
  > Amazon S3 introduces support for AWS Dedicated Local Zones (#465)
  > Support trailing checksum with no signing (#459)
  > support if-none-match for upload (#462)
  > Use proper public event loop group API (#460)
  > chore: Modified bug issue template to add checkbox to report potential regression. (#452)
  > Update CMake to 3.9 (#458)
  > Support header checksum (#454)
Submodule mountpoint-s3-crt-sys/crt/aws-c-sdkutils 4658412a..ce09f797:
  > Add cxx support (#48)
  > chore: Modified bug issue template to add checkbox to report potential regression. (#45)
  > Update CMake to 3.9 (#47)
Submodule mountpoint-s3-crt-sys/crt/aws-checksums ce04ab00..3e4101b9:
  > fix predefines for bswap for old compilers (#99)
  > CRC big endian support (#97)
  > chore: Modified bug issue template to add checkbox to report potential regression. (#95)
  > Update CMake to 3.9 (#98)
Submodule mountpoint-s3-crt-sys/crt/aws-lc 8b2ebfcf..59828538:
  > Prepare for v1.40.0 release (#2019)
  > [EC] Use s2n-bignum point doubling for P-384 and P-521 (#2011)
  > Document TLS Server Renegotiation Behavior (#2018)
  > Fail FIPS rsa_keygen_pubexp on change (#2016)
  > Adding -verify and expanding -x509 options for our OpenSSL tool (#1951)
  > Upstream merge 2024-11-11 (#1985)
  > Implement PKCS7_encrypt and PKC7_decrypt (#1996)
  > [EC] Unify scalar_mul_public for ec_nistp curves (#2004)
  > Adding the OpenSSL s_client tool (#1959)
  > Add Clang 19 to CI (#1998)
  > [EC] Unify scalar_mul_base point for ec_nistp curves (#2003)
  > Add internal APIs for ML-DSA (#1999)
  > Test cleanup (#2000)
  > Minor improvement to DSA (ASN1) + DSA Tests (#1990)
  > Implement PKCS7_dataInit and PKCS7_dataFinal (#1816)
  > Addition of generic NIST-DSA PKEY and ASN1 to support ML-DSA (#1963)
  > Expose a bit of lhash/conf for Ruby (#1987)
  > Allow ASN1_get_object to parse indefinite and universal (#1994)
  > Added CRL tool to CLI (#1976)
  > Prepare release AWS-LC v1.39.0 (#1995)
  > Revert "Replace CONF's internal representation with something more typesafe" (#1986)
  > Add Cyrus-SASL to our CI (#1988)
  > Cleanup test File utilities (#1989)
  > Account for cipher auth with multiple cert slots (#1956)
  > Allocate 16k scratch on heap (#1991)
  > Add CRYPTO_sysrand benchmarks to speed.cc (#1978)
  > Update PQREADME to add link to the KEM readme file (#1973)
  > Avoid compiler warning (#1981)
  > Ruby Support - More EVP_PKEY_DSA (#1954)
  > Upstream merge 2024-10-23 (#1955)
  > CI gcc-4.8 - use 4.8.5 tag (#1980)
  > Fix sess_hits counter on the server (#1974)
  > Support Finished-based APIs for TLS 1.3 (#1952)
  > Fix i2d behavior for i2d_SSL_SESSION (#1966)
  > fix `-Wcast-function-type` build issues (#1972)
  > Prepare v1.38.0 release (#1975)
  > Expose AES_cfb1_encrypt and AES_cfb8_encrypt (#1967)
  > EDDSA PCT (#1968)
  > ML-KEM keygen Pairwise Consistency Test (#1964)
  > Coverity Fix Null Check (#1965)
  > Actually add support for SSL_get_server/peer_tmp_key (#1945)
  > Also test w/ gcc 4.8 (#1962)
  > Fixes for Coverity Alerts (#1960)
  > Add support for POINT_CONVERSION_HYBRID (#1936)
  > Ruby Support - DSA custom md (#1953)
  > Add PKCS7-internal BIO_f_md (#1886)
  > Add PKCS7-internal BIO_f_cipher (#1836)
  > Expand support for EVP_PKEY_HMAC (#1933)
  > Support encode or decode ∞ like OpenSSL (#1930)
  > Fix FIPS.md typo (#1950)
  > Missing functionality + Adding Nmap to our CI (#1915)
  > HKDF, HKDF_expand, and PBKDF Truncated SHA2-512 (#1946)
  > bump mysql CI to 9.1.0 (#1939)
  > PQ README (#1932)
  > Add p4p, bump up time (#1943)
  > Remove retries on PCT failure in EC and RSA key generation. (#1938)
  > Remove old Intel CPU types (#1942)
  > Upstream merge 2024 10 17 (#1934)
  > DH paramgen callback (#1928)
  > Add null check in dh testing (#1937)
  > Use illegal_parameter instead of decode_error for invalid key shares (#1923)
  > Also prune SSM documents from ec2-test-framework (#1925)
  > Marshalling/Unmarshalling DH public keys (#1916)
  > 800-131Ar1: length of the key-derivation key shall be at least 112 bits. (#1924)
  > Prepare 1.37.0 release (#1927)
  > Add 2024 FIPS and fix build issues on older arm FIPS (#1920)
  > Align X509 PARTIAL_CHAIN behavior with 1.1.1 (#1917)
  > P161732527 coverity cleanup (#1918)
  > build: fix pkgconfig files (#1913)
  > Avoid allocating EVP_PKEY on size checks (#1911)
  > Add EC_GROUP mutablility to custom curves (#1881)
  > Implement more EVP_PKEY_DH functionality (#1880)
  > ML-DSA parameter refactor (#1910)
  > Update FIPS docs w/ certs (#1900)
  > Handle Windows not supporting static array dimension (#1912)
  > Remove duplicate s2n-bignum prefix include option (#1909)
  > Add support for EVP_PKEY_CTX callback functions (#1905)
  > P159598331 coverity cleanup (#1908)
  > Add Alpine-Linux-x86 to GitHub Actions CI (#1753)
  > Upstream merge 2024 09 16 (#1862)
  > Update Dilithium from crystals upstream (#1894)
  > Create mutable EC_GROUP API for OpenSSL compatibility (#1860)
  > ML-KEM FIPS 203 destruction of intermediate values (#1883)
  > Remove special s2n-bignum symbol handling sauce from build (#1903)
Submodule mountpoint-s3-crt-sys/crt/s2n-tls ffe0bf42..493b7716:
  >  feat: Reworking cleanup behavior (#4871)
  > chore: broaden use of flaky mark (#4865)
  > chore: configure dependabot (#4861)
  > fix: fix open AF_INET sockets in s2n_self_talk_ktls_test.c (#4852)
  > chore: update github PR template (#4885)
  > feat: add new security policy `20241106` (#4874)
  > chore: remove unused benchmarks (#4869)
  > ci: Clean dup source tree for CRT (#4882)
  > ci: remove www.mozilla.com from well-known to unblock CI (#4880)
  > fix: move prelude inclusion as PRIVATE (#4876)
  > build: add s2n_prelude.h to consolidate defines (#4465)
  > chore: bindings release 0.3.6 (#4867)
  > doc: fix incorrect README references (#4863)
  > fix: typo in comment of s2n_self_talk_tls13_test (#4864)
  > fix: close all /dev/urandom open fds (#4835)
  > docs: update fips documentation to specify supported libcrypto (#4857)
  > fix(bindings): correct poll_flush implementation (#4859)
  > feat: Adds cleanup_final (#4853)
  > test(bindings): Consolidate test pems (#4858)
  > chore: bindings release 0.3.5 (#4860)
  > chore: grant duvet action more permissions (#4854)
  > (feat): Adds certificate match metrics API (#4844)
  > chore: Fix failing OIDC workflows; cleanup unused actions (#4848)
  > chore(GHA): Update duvet arguments (#4850)
  > chore: remove unused compile definition (#4815)
  > Add new MLKEM TLS Policies (#4830)
  > fix: fix opened AF_UNIX sockets that didn't call s2n_io_pair_close (#4833)
  > bindings: pin openssl crate to 0.10.66 (#4849)
  > chore: flip 2 GHAs to use short lived creds. (#4839)
  > fix: fix s2n_io_pair_close_one_end (#4841)
  > ci: Re-enable asan and ubsan for fuzz tests (#4840)
  > fix: some open AF_UNIX sockets in forked child processes (#4834)
  > Update FIPS rules for ML-KEM (#4829)
  > ci: update ubuntu versions (#4828)
  > Add initial support for MLKEM768 (without any new Security Policies) (#4816)
  > chore: Adds print statements to help debug s2n_dynamic_load_test (#4836)
  > ci: add more libcryptos for fuzz batch & follow cmake idioms (#4795)
  > feature: bump cert authorities max size to 20kb (#4832)
  > ci: Add ubuntu24 with a new cmake buildspec (#4824)
  > Add ML-KEM Feature Probe and Test (#4823)
  > docs: update stateful resumption doc (#4818)
  > chore: remove make fuzz and AFL fuzz (#4808)
```
</details>

---

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license and I agree to the terms of
the [Developer Certificate of Origin
(DCO)](https://developercertificate.org/).

Signed-off-by: Alessandro Passaro <[email protected]>
Signed-off-by: Alessandro Passaro <[email protected]>
Signed-off-by: Alessandro Passaro <[email protected]>
Signed-off-by: Alessandro Passaro <[email protected]>
Copy link
Contributor

@vladem vladem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of comments + could you pls run benchmarks on this PR?

@@ -951,7 +953,7 @@ impl<'a> S3Message<'a> {
#[pin_project(PinnedDrop)]
struct S3HttpRequest<T, E> {
#[pin]
receiver: oneshot::Receiver<ObjectClientResult<T, E, S3RequestError>>,
receiver: Fuse<oneshot::Receiver<ObjectClientResult<T, E, S3RequestError>>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: document that this is the receiver for the value provided in the on_finish callback of CRT meta request

headers = headers_receiver => headers.unwrap(),
result = request => {
// If we did not received the headers first, the request must have failed.
result?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM to me now, but the crucial bit here, which requires some thinking to figure out, is that request will never be returned with S3GetObjectResponse if awaiting it already returned Poll::Ready.

Otherwise, awaiting on S3GetObjectResponse may block forever in some edge cases (given that S3HttpRequest::receiver is Fuse).

I'd add an assert!(!request.is_terminated()); before the Ok(..) return statement or protect from this possible bug in S3GetObjectResponse::poll_next or at least a comment.

Signed-off-by: Alessandro Passaro <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance PRs to run benchmarks on
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants