S3StorageURL Authorization header is malformed; the region 'auto' is wrong.

[Sam-tesouro]

Component(s)

controlplane

Component version

[email protected]

wgc version

[email protected]

controlplane version

[email protected]

router version

N/A

What happened?

Description

Targeting s3 for CDN and ControlPlane with URL format https://ID:SECRET@s3.[REGION].amazonaws.com/[BUCKET-NAME] leads to

"stack":"AuthorizationHeaderMalformed: The authorization header is malformed; the region 'auto' is wrong; expecting '[REGION]'"

Steps to Reproduce

https://github.com/Sam-tesouro/cosmo-controlplane-s3-url-issue/tree/main

Expected Result

Successfully list buckets

Actual Result

node_modules/@smithy/smithy-client/dist-cjs/index.js:839
  const response = new exceptionCtor({
                   ^
AuthorizationHeaderMalformed: The authorization header is malformed; the region 'auto' is wrong; expecting 'us-east-1'
    at throwDefaultError

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Package Manager: pnpm, npm, yarn, etc
Compiler(if manually compiled): (e.g., "go 14.2")

Router configuration

No response

Router execution config

No response

Log output

No response

Additional context

The s3 client is forcing path style urls which were originally slated for deprecation 4 years ago. https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/

Is there any reason why you aren't supporting Virtual-hosted–style requests?

Regardless I suspect I am just missing the correct incantation for an s3 storage url for Cosmo. After trying many different permutations I am hoping it's simple and I can save others the headache in the future with this issue!

Thank you!

[github-actions]

WunderGraph commits fully to Open Source and we want to make sure that we can help you as fast as possible.
The roadmap is driven by our customers and we have to prioritize issues that are important to them.
You can influence the priority by becoming a customer. Please contact us here.

[jensneuse]

We're taking a look, thank you!

[AndreasZeissner]

Hi @Sam-tesouro,

I wanted to let you know that we’ve recently added support for AWS S3 URLs.

You might want to have a look here:

[email protected]:

• GitHub Release
• ArtifactHub Package
  Here’s an example for CDN configuration:

cdn:
  configuration:
    s3StorageUrl: 'https://[username]:[password]@your-bucket.s3.amazonaws.com'
    s3Region: 'us-east-1'

You can find more options here:

• Values.yaml - Line 120
• Values.yaml - Line 196

In these sections, you'll find details on how to pass secret keys, access key IDs, and more. The CDN configuration needs to be applied similarly for the controlplane.

Give it a try and see how it works for your setup.

Additional release links:

• [email protected]: GitHub Release
• [email protected]: GitHub Release

[AndreasZeissner]

Hi @Sam-tesouro,

we had to ship a small bug fix: https://github.com/wundergraph/cosmo/releases/tag/controlplane%400.105.1

You would need to set you configuration like so:

cdn:
  configuration:
    s3StorageUrl: 'https://[username]:[password]@your-bucket.s3.amazonaws.com'
    s3Region: 'us-east-1'
    s3ForcePathStyle: 'false'

when updating to this version, let me know how things work for you.

Thanks!