AWS.MetadataService.request does not support IMDSv2 out of the box

[michaelwittig]

Describe the bug
When an EC2 instance is configured to only allow IMDSv2, the AWS.MetadataService.request() doesn't work. The returned error null is not helpful. The workaround is to manually get a IMDS token and pass it to AWS.MetadataService.request(). The Java SDK handles this transparent to the developer which seems the better approach.

Is the issue in the browser/Node.js?
Node.js

If on Node.js, are you running this on AWS Lambda?
no

Details of the browser/Node.js version
v12.20.0

SDK version number
2.792.0 (but master branch has the issues as well)

To Reproduce (observed behavior)

const AWS = require("aws-sdk");
const meta = new AWS.MetadataService();
meta.request("/latest/meta-data/local-ipv4", function (err, data) {
  if (err) {
    throw err;
  } else {
    console.log(data);
  }
});

throws

Error: null
    at IncomingMessage.<anonymous> (/etc/home/ec2-user/app/src/node_modules/aws-sdk/lib/util.js:899:34)
    at IncomingMessage.emit (events.js:326:22)
    at IncomingMessage.EventEmitter.emit (domain.js:483:12)
    at endReadableNT (_stream_readable.js:1241:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {
  statusCode: 401,
  retryable: false,
  time: 2020-12-17T13:59:03.709Z
}

Expected behavior
return 10.0.32.49

Work around

const AWS = require("aws-sdk");
const meta = new AWS.MetadataService();
meta.fetchMetadataToken(function (err, token) {
  if (err) {
    throw err;
  } else {
    meta.request(
      "/latest/meta-data/local-ipv4",
      {
        headers: { "x-aws-ec2-metadata-token": token },
      },
      function (err, data) {
        if (err) {
          throw err;
        } else {
          console.log(data);
        }
      }
    );
  }
});

[ajredniwja]

Hey @michaelwittig thanks for opening this, I agree with you, the SDK is configured to try IMDSv2 for first and then fall back to IMDSv1 if it fails, so if EC2 instance is configured to only allow IMDSv2, it will fail, the error here is not helpful at all but I think it should fail.

The version 3 of the SDK, provides with better error and is modular, can you give it a try?

The version 3.x of the AWS SDK for JavaScript is generally available. For more information see the Developer Guide or API Reference.

[michaelwittig]

Hi @ajredniwja I don't think that we will invest into migrating from 2x to 3x only because of a bug? I would appreciate a fix.

[ajredniwja]

I can bring this up with the team to discuss the priority of this fix.

[trivikr]

Here is the Developer Guide on How Instance Metadata Service Version 2 works

The following example uses a Linux shell script and IMDSv2 to retrieve the top-level instance metadata items. The example command:

• Creates a session token lasting six hours (21,600 seconds) using the PUT request
• Stores the session token header in a variable named TOKEN
• Requests the top-level metadata items using the token

$ TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/

After you've created a token, you can reuse it until it expires. In the following example command, which gets the ID of the AMI used to launch the instance, the token that is stored in $TOKEN in the previous example is reused.

$ curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/ami-id

When you use IMDSv2 to request instance metadata, the request must include the following:

1. Use a PUT request to initiate a session to the instance metadata service. The PUT request returns a token that must be included in subsequent GET requests to the instance metadata service. The token is required to access metadata using IMDSv2.

2. Include the token in all GET requests to the instance metadata service. When token usage is set to required, requests without a valid token or with an expired token receive a 401 - Unauthorized HTTP error code. For information about changing the token usage requirement, see modify-instance-metadata-options in the AWS CLI Command Reference.

The PR in JS SDK v2 which implemented IMDSv2 is #2964, and it works as designed. The implementation in Java SDK is optional as per internal spec provided to the SDKs given below:

SDK may store the token value for use for any subsequent IMDS calls, and use this stored token until the SDK's refresh strategy dictates fetching a new token. SDK may also fetch a token for every credentials request instead.

Marking this issue as a feature-request.

[trivikr]

The EC2 Developer Guide to view the IPv4 addresses also mentions explicitly making PUT request to fetch metadata token:

IMDSv2

TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-?seconds: 21600"` \
&& curl -H "X-aws-ec2-metadata-token: $TOKEN" -v http://169.254.169.254/latest/meta-data/local-ipv4

[nishant-dani]

@ajredniwja Per the suggestion to try v3.

Per my search the MetadataService that was part of the AWS SDK does not appear to be part of the v3 SDK. I am able to successfully get the token using the following code;

  const { fromContainerMetadata } = require('@aws-sdk/credential-provider-imds')

  let response = await fromContainerMetadata()()

  // FromContainerMetadata returns
  // "accessKeyId": "**",
  // "secretAccessKey": "**",
  // "sessionToken": "**",
  // "expiration": "2021-02-17T04:54:28.000Z"
  
  console.log(`getToken: token is ${JSON.stringify(response.sessionToken)}`)
  return response.sessionToken

But there appears to be no way in the aws sdk v3 to get to the actual meta data profile categories including the hostname or the ip address.

[thomasl8]

@trivikr The fetchMetadataToken function in the v2 SDK appears to be marked as private so it is not appearing in the typescript typings.

[jpike88]

@trivikr it's like 10 months later, and typing for fetchMetadataToken haven't been sorted. Will this even be fixed?

[github-actions]

Greetings! We’re closing this issue because it has been open a long time and hasn’t been updated in a while and may not be getting the attention it deserves. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to comment or open a new issue.

[jpike88]

I like how the bot just closed a legitimate issue, without any feedback or conclusion from the AWS team. It's great.

This new issue echoes the problem, and it's causing people to abort upgrade to AWS SDK v3. Meanwhile, AWS is telling me in my server logs to upgrade. What is going on???
aws/aws-sdk-js-v3#4004

[rynop]

Yikes this is not documented well. IMHO this should be built into the SDK with a simple useV2 flag when calling AWS.MetadataService().request().

Here is what I came up with to get IMDSv2 working with AWS js SDK v2 working. Hope this helps the next person

// AWS SDK for JS v2 does not support `.promise()` on the `AWS.MetadataService()` (at least AFAICT)
// this.metaService = new AWS.MetadataService();

return new Promise((resolve, reject) => {
      this.metaService.fetchMetadataToken((tokenError, token) => {
        if (tokenError) {
          console.log('IMDSv2 token error', tokenError, {category});
          return resolve(null);
        }

        this.metaService.request(
          `/latest/meta-data/${category}`,
          {
            headers: {
              'x-aws-ec2-metadata-token': token,
            },
          },
          (err, data) => {
            if (err) {
              console.log(err, {category});
              return resolve(null);
            }

            resolve(data);
          }
        );
      });
    });