aws javascript sdk v3 - signature mismatch error

[beez-dev]

I can generate the presigned url following the steps as described in this section, so I wanted to test uploading a specific image marble.jpg and I tried to use postman to test the upload. So, I copied the presigned url and hit the endpoint with a PUT request, and I got this error:

<?xml version="1.0" encoding="UTF-8"?>
<Error>
    <Code>SignatureDoesNotMatch</Code>
    <Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
    <Key>records/marble_cave.jpg</Key>
    <BucketName>bucket</BucketName>
    <Resource>/bucket/records/marble.jpg</Resource>
    <RequestId>17E3999B521ABB65</RequestId>
    <HostId>50abb07a-2ad0-4948-96e0-23403f661cba</HostId>
</Error>

The following resources are setup:

I'm using the min.io server to test this locally.
I'm using aws-sdk version 3 of the nodejs sdk for aws
I've triple checked my credentials, simple minio creds with no special characters also, I'm definitely making a PUT request.

The only thing I haven't tried is to add the signatureVersion like one would in sdk version prior to 3:

var s3  = new AWS.S3({
          accessKeyId: 'YOUR-ACCESSKEYID' ,
          secretAccessKey: 'YOUR-SECRETACCESSKEY' ,
          endpoint: 'http://127.0.0.1:9000' ,
          s3ForcePathStyle: true, // needed with minio?
          signatureVersion: 'v4'
});  

but, using javascript sdk for s3 version 3 this is not available while constructing the s3 client.

So, The question is:

Is this due to the signature version not being specified? If it is, how to do so?. ( The getSignedUrl is used to generate presigned url in v3 of the sdk, `import { getSignedUrl } from '@aws-sdk/s3-request-presigner';`)

what other causes might be there such that this error is occuring?(Already checked credentials are not an issue)

[arthyn]

I'm encountering the same issue. According to https://github.com/aws/aws-sdk-js-v3/blob/main/UPGRADING.md v3 only supports signature v4, but apparently there's something different it does from the v2 SDK, because that one actually works with Minio. Is there a special setting in Minio that needs to be changed for this to be compatible?

[beez-dev]

yeah, I've been stuck in this for 2 days now, no apparent solution. The v2 works with minio but v3 doesn't and not specific logs or error messages are shown

[harshavardhana]

@beez-dev please capture mc admin trace -v output while using the SDK - also provide us a full reproducer.

[beez-dev]

The postman request ss are:

[harshavardhana]

And what is your host header here you seen to be using 172.x IP is that what you are using with curl?

[beez-dev]

I have used both the api end points given by mino s3, they result the same:
The above is just using one of those

[beez-dev]

and I just used postman to test, although, it probably wouldn't be different with curl

[beez-dev]

@harshavardhana are you able to replicate the problem with the code snippet above?

[beez-dev]

Comparing the two generated urls side by side, I can see something extra in V3 which probably is (in my guess) in turn causing this:

aws-sdk v3 generated presigned url :

http://127.0.0.1:9000/bucket/records/blurry_marbles.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=minioadmin%2F20220411%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220410T164150Z&X-Amz-Expires=10000&X-Amz-Signature=4f029d9355a1f46a9ecc998c02f19f775554e21cf4eb2d25ae3a62d6e2fbb416&X-Amz-SignedHeaders=host&x-id=PutObject  

http://127.0.0.1:9000/bucket/records/blurry_marbles.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minioadmin%2F20220411%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220410T164850Z&X-Amz-Expires=900&X-Amz-Signature=a0823a83b56d23f0130fe9fe5870040f3451164ec2ecf72b2fbcd37ce1b3f8e9&X-Amz-SignedHeaders=host

Is this something that can be adjusted or something will have to be fixed from minio?

[harshavardhana]

This is a stupid regression in aws-sdk-js-v3 @beez-dev they ignore the port passed to the endpoint as part of signature calculation

PUT
/myBucket/marbles.jpg
X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=minioadmin%2F20220411%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220411T024038Z&X-Amz-Expires=10000&X-Amz-SignedHeaders=host&x-id=PutObject
host:localhost

host
UNSIGNED-PAYLOAD
http://localhost:9000/myBucket/marbles.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=minioadmin%2F20220411%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220411T024038Z&X-Amz-Expires=10000&X-Amz-Signature=512f93d4d6d3f85098a47d9b543d162f6b2561945a33b77358dd4728d677f72e&X-Amz-SignedHeaders=host&x-id=PutObject

Here is the canonical request generated by the client

host:localhost

As a workaround start the server at port "80" for now - ask upstream to fix their bugs @beez-dev

import { getSignedUrl } from '@aws-sdk/s3-request-presigner';
import { PutObjectCommand, S3Client } from '@aws-sdk/client-s3';
const s3Client = new S3Client({
    region: 'us-east-1',
    credentials: {
        accessKeyId: 'minioadmin',
        secretAccessKey: 'minioadmin',
    },
    endpoint: "http://localhost",
    forcePathStyle: true,
});

const bucketParams = {
    Bucket: 'myBucket',
    Key: `marbles.jpg`,
};  

const command = new PutObjectCommand(bucketParams);

(async function () {
   try {
     const signedUrl = await getSignedUrl(s3Client, command, {
       expiresIn: 10000,
    })
    console.log(signedUrl)
} catch (err) {
   console.error(err)
}   
})();

This works perfectly fine.

[phatj]

I encountered this error while building my application.

Unfortunately, I have another service running on port 80, so I sought out another solution.

From what I can tell, the issue exists here: https://github.com/aws/aws-sdk-js-v3/blob/main/packages/s3-request-presigner/src/presigner.ts#L41. The logic for setting the host/port seems a little odd.

Fortunately, looking at https://github.com/aws/aws-sdk-js-v3/blob/main/packages/s3-request-presigner/src/getSignedUrl.ts#L8, you can spoof the client by using creating an object that contains the client config and a cloned, custom middleware stack.

import { S3Client, ServiceInputTypes, ServiceOutputTypes } from "@aws-sdk/client-s3";
import { BuildMiddleware, HttpRequest } from "@aws-sdk/types";
import { getSignedUrl } from '@aws-sdk/s3-request-presigner';

function getPresignerClient(client: S3Client, port?: number | string) {
  if (!port) {
    // early return if port isn't set
    return client;
  }

  const middlewareName = "presignHeaderMiddleware";

  const presignHeaderMiddleware: BuildMiddleware<
    ServiceInputTypes,
    ServiceOutputTypes
  > = (next) => async (args) => {
    const request = args.request as HttpRequest;
    request.headers["host"] = [request.hostname, port].join(":");

    return await next(args);
  };

  const middlewareStack = client.middlewareStack.clone();

  middlewareStack.addRelativeTo(presignHeaderMiddleware, {
    name: middlewareName,
    relation: "before",
    toMiddleware: "presignInterceptMiddleware",
    override: true,
  });

  return { middlewareStack, config: client.config } as S3Client;
}

const client: S3Client; // instantiate or use your existing client
const port: string | number;  // set or use your current config

return await getSignedUrl(getPresignerClient(client, port), command);

I'm not sure if override is necessary (I duplicated the structure from getSignedUrl.
It's important to note that you can't blanket add this middleware if you want it to turn before presignInterceptMiddleware, which I why we clone it instead and do it on a per request basis.

It may be possible to setup the middleware once for all operations, but I haven't tested that, yet. I'd have to remove relation, toMiddlware optiong, however.

Hope that helps.

[oscarhermoso]

For others that find this thread, you can define the signer that is passed into the S3Client constructor, works great

aws/aws-sdk-js-v3#3858 (comment)