You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Server issues new credentials via STS, scoped to just the /temp:<region>/<random uuid> bucket prefix
Client generates presigned URLs to PUT files to
Uploads fail with SignatureDoesNotMatch
But I noticed it wasn't failing all the time! One out of every ten or twenty tries would succeed, indicating it wasn't some complete misconfiguration. There are many, many threads about the SignatureDoesNotMatch issue (here's a big one), some are user error, but many seemed to be resolved by regenerating credentials with no /, +, or = in the secret, but I tried that, and the same issue happened.
So to continue debugging, I did a few things:
I wrote a quick tool that uses the credentials I was generating (via /get_temp_credentials), and generates signed PUT URLs, then uploads files to them. That worked fine, indicating that the credentials aren't the problem
And the swap worked, my local hacked up Logseq client can now reliably upload files with the presigned S3 URLs it generates with the short-lived STS credentials:
15:26:32.398 › update remote files[txid=1]: ["journals/2023_11_25.md", "pages/This is a test.md"]
15:26:32.626 › upload progress: 100% 360/360 journals/2023_11_25.md
15:26:32.627 › upload progress: 100% 304/304 pages/This is a test.md
15:26:33.758 › copy page file to version-files: "journals/2023_11_25.md"
15:26:33.759 › copy page file to version-files: "pages/This is a test.md"
15:26:33.759 › update remote files success, txid=2
So I'm pretty confident the issue is with the s3-presign package. I don't know what the official Logseq Sync server implementation does (likely during STS credential generation?) such that this issue doesn't occur, but it seems there's some edge case that causes it to generate invalid signatures.
The text was updated successfully, but these errors were encountered:
I've been working on a self-hostable Logseq Sync backend, and I was having trouble with the issued STS credentials. The flow looked like:
/get_temp_credentials
/temp:<region>/<random uuid>
bucket prefixPUT
files toSignatureDoesNotMatch
But I noticed it wasn't failing all the time! One out of every ten or twenty tries would succeed, indicating it wasn't some complete misconfiguration. There are many, many threads about the
SignatureDoesNotMatch
issue (here's a big one), some are user error, but many seemed to be resolved by regenerating credentials with no/
,+
, or=
in the secret, but I tried that, and the same issue happened.So to continue debugging, I did a few things:
/get_temp_credentials
), and generates signed PUT URLs, then uploads files to them. That worked fine, indicating that the credentials aren't the problems3-presign
crate with durch/rust-s3, to see if that was the issue, you can see the relevant changes here.And the swap worked, my local hacked up Logseq client can now reliably upload files with the presigned S3 URLs it generates with the short-lived STS credentials:
So I'm pretty confident the issue is with the
s3-presign
package. I don't know what the official Logseq Sync server implementation does (likely during STS credential generation?) such that this issue doesn't occur, but it seems there's some edge case that causes it to generate invalid signatures.The text was updated successfully, but these errors were encountered: