Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli): automatically determine region on EC2 instances #9313

Merged
merged 16 commits into from
Aug 25, 2020
Merged

feat(cli): automatically determine region on EC2 instances #9313

merged 16 commits into from
Aug 25, 2020

Conversation

joel-aws
Copy link
Contributor

Enables EC2 instances to automatically determine their current region by querying the Instance Metadata Service (IMDS). Both IMDSv1 and v2 are supported.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@mergify
Copy link
Contributor

mergify bot commented Jul 28, 2020

Title does not follow the guidelines of Conventional Commits. Please adjust title before merge.

@SomayaB SomayaB assigned shivlaks and ericzbeard and unassigned shivlaks Jul 30, 2020
@ericzbeard ericzbeard assigned rix0rrr and unassigned ericzbeard Aug 3, 2020
packages/aws-cdk/lib/api/aws-auth/awscli-compatible.ts Outdated Show resolved Hide resolved
packages/aws-cdk/lib/api/aws-auth/awscli-compatible.ts Outdated Show resolved Hide resolved
headers: { 'x-aws-ec2-metadata-token-ttl-seconds': '60' },
},
(err: AWS.AWSError, token: string | undefined) => {
if (err || !token) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be something like:

if (err) { 
  fail(err);
} else if (!token) {
  fail(new Error('IMDS returned an empty token'));
} else {
  resolve(token);
}

Copy link
Contributor Author

@joel-aws joel-aws Aug 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to:

      (err: AWS.AWSError, token: string | undefined) => {
        if (err) {
          reject(err);
        } else if (!token) {
          reject(new Error('IMDS did not return a token.'));
        } else {
          resolve(token);
        }
      });

packages/aws-cdk/lib/api/aws-auth/awscli-compatible.ts Outdated Show resolved Hide resolved
httpOptions: { timeout: 1000, connectTimeout: 1000 }, maxRetries: 2,
};
const metadataService = new AWS.MetadataService(imdsOptions);
const token = await getImdsV2Token(metadataService);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rewrite the helpers below to properly throw an error, and then try/catch and log any errors here.

Copy link
Contributor Author

@joel-aws joel-aws Aug 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I rewrote it as such:

    const token = await getImdsV2Token(metadataService)
       .catch((error) => {
         debug(`No token returned from IMDS; Error: ${error}`)
         return undefined;
       });
     region = await getRegionFromImds(metadataService, token)
       .catch((error) => {
         debug(`No Instance Identity Document returned from IMDS; Error: ${error}`)
         return undefined;
       });

@rix0rrr rix0rrr changed the title fix(aws-auth): enable EC2 instances to query IMDS to determine their region feat(cli): automatically determine region on EC2 instances Aug 10, 2020
@mergify
Copy link
Contributor

mergify bot commented Aug 10, 2020

Title does not follow the guidelines of Conventional Commits. Please adjust title before merge.

@mergify mergify bot dismissed rix0rrr’s stale review August 13, 2020 00:38

Pull request has been modified.

@joel-aws joel-aws requested a review from rix0rrr August 15, 2020 16:19
packages/aws-cdk/lib/api/aws-auth/awscli-compatible.ts Outdated Show resolved Hide resolved
httpOptions: { timeout: 1000, connectTimeout: 1000 }, maxRetries: 2,
};
const metadataService = new AWS.MetadataService(imdsOptions);
const token = await getImdsV2Token(metadataService)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kinda don't like the style this is written. It confuses me to combine async/await and thenables in this way.

Why not:

try {
  const token = await getImdsV2Token(metadataService);
  region = await getRegionFromImds(metadataService, token);
  debug(`Retrieved AWS region "${region}" from the IMDS.`);
} catch (e) {
  debug(`Unable to retrieve region from IMDS: ${e.message}`);
} 

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about that, but was concerned about a case where IMDSv2 was not supported -- potentially in partition where it's not implemented. In that situation, getImdsV2Token would throw an error but IMDSv1 should still be queried to get the region, just without a token.

packages/aws-cdk/lib/api/aws-auth/awscli-compatible.ts Outdated Show resolved Hide resolved
packages/aws-cdk/lib/api/aws-auth/awscli-compatible.ts Outdated Show resolved Hide resolved
packages/aws-cdk/lib/api/aws-auth/awscli-compatible.ts Outdated Show resolved Hide resolved
@rix0rrr rix0rrr added the pr-linter/exempt-readme The PR linter will not require README changes label Aug 17, 2020
Copy link
Contributor

@rix0rrr rix0rrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By the way, this is also going to need a test.

@mergify mergify bot dismissed rix0rrr’s stale review August 17, 2020 19:33

Pull request has been modified.

@joel-aws
Copy link
Contributor Author

joel-aws commented Aug 17, 2020

By the way, this is also going to need a test.

@rix0rrr -- In addition to your thoughts on continuing to query IMDSv1 if we don't get an IMDSv2 token, would you also mind pointing me in the right direction for writing a test? Maybe just a file that I should write the test in and where I could copy syntax from? Thanks!

@rix0rrr rix0rrr added pr-linter/exempt-test The PR linter will not require test changes pr/requires-two-approvers This PR is critical (e.g., security, broadly-impacting) and requires 2 approvers to be merged. labels Aug 18, 2020
rix0rrr
rix0rrr previously approved these changes Aug 18, 2020
Copy link
Contributor

@rix0rrr rix0rrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is way too hard to write without an actual EC2 instance to run an integ test on, which we're not set up for.

The delta is small enough that I trust the code upon review, but I'm going to request a second review.

Copy link
Contributor

@nija-at nija-at left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is way too hard to write without an actual EC2 instance to run an integ test on, which we're not set up for.

Can we at least write unit tests with mocking so as to detect regression on future refactor? It should be easy to mock the MetadataService

njlynch
njlynch previously approved these changes Aug 18, 2020
@nija-at nija-at removed the pr-linter/exempt-test The PR linter will not require test changes label Aug 18, 2020
@mergify mergify bot dismissed stale reviews from rix0rrr, nija-at, and njlynch August 18, 2020 13:12

Pull request has been modified.

@rix0rrr rix0rrr removed the pr/requires-two-approvers This PR is critical (e.g., security, broadly-impacting) and requires 2 approvers to be merged. label Aug 18, 2020
rix0rrr
rix0rrr previously approved these changes Aug 18, 2020
nija-at
nija-at previously approved these changes Aug 18, 2020
Comment on lines +151 to +165
async function isEc2Instance() {
if (isEc2InstanceCache === undefined) {
debug("Determining if we're on an EC2 instance.");
let instance = false;
if (process.platform === 'win32') {
// https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/identify_ec2_instances.html
const result = await util.promisify(child_process.exec)('wmic path win32_computersystemproduct get uuid', { encoding: 'utf-8' });
// output looks like
// UUID
// EC2AE145-D1DC-13B2-94ED-01234ABCDEF
const lines = result.stdout.toString().split('\n');
instance = lines.some(x => matchesRegex(/^ec2/i, x));
} else {
// https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/identify_ec2_instances.html
const files: Array<[string, RegExp]> = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider adding similar test cases for this method and its conditionals. Since this method already existed, I'll let you make the final call.

.mockImplementationOnce((_1, _2, cb) => { cb(undefined as any, JSON.stringify({ region: 'some-region' })); });

const region = await AwsCliCompatible.region({ ec2instance: true });
expect(region).toEqual('some-region');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: add an expectation that the mocks were actually called

@nija-at nija-at added the pr/do-not-merge This PR should not be merged at this time. label Aug 18, 2020
@rix0rrr rix0rrr removed the pr/do-not-merge This PR should not be merged at this time. label Aug 24, 2020
@mergify
Copy link
Contributor

mergify bot commented Aug 24, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot dismissed stale reviews from rix0rrr and nija-at August 24, 2020 12:30

Pull request has been modified.

@mergify
Copy link
Contributor

mergify bot commented Aug 25, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: fb46a7a
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Contributor

mergify bot commented Aug 25, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 1cf986d into aws:master Aug 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-linter/exempt-readme The PR linter will not require README changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants