(@aws-cdk/aws-apigateway): Wrong Logical ID for UsagePlanKeyResource

[ghost]

❓ General Issue

The Question

Hi there! We migrated a lot of APIs from our on-premise-solution into the AWS API Gateway. To control access to our APIs we configured Cognito (with OAuth Scopes) and API Keys. These API Keys are generated with an array holding a number of subscribers (other stakeholders or applications):

subscriber: ['user1', 'user2', 'user3']

Our code iterates over this array, generates API Keys for each element and adds them to the APIs UsagePlan:

export const addApiKeysToUsagePlan = (api: RestApi, name: string, apiSubscriber: string[], usagePlan: UsagePlan): void => {
  apiSubscriber.forEach(function (subscriber: string) {
    const apiKey = new ApiKey(api, name + '-apiKey-' + subscriber, {
      apiKeyName: name + '-' + subscriber,
    });
    usagePlan.addApiKey(apiKey);
  });
};

This is the output of a "cdk diff" when adding three freshly new subscriber to a already deployed API:

[-] AWS::ApiGateway::Deployment testapiDeployment356D2C350f0d9c33af21b1b5acbe01dd96f30695 destroy
[+] AWS::ApiGateway::Deployment test-api/Deployment testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231
[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource testapiUsagePlanKeyResource6D2F267F
[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource:testapistacktestapitestapiapiKeyuser2A184AC36 testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser2A184AC36802022E8
[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource:testapistacktestapitestapiapiKeyuser315D7663C testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser315D7663CF36A0768
[+] AWS::ApiGateway::ApiKey test-api/test-api-apiKey-user1 testapitestapiapiKeyuser100D43FD3
[+] AWS::ApiGateway::ApiKey test-api/test-api-apiKey-user2 testapitestapiapiKeyuser26097BDA4
[+] AWS::ApiGateway::ApiKey test-api/test-api-apiKey-user3 testapitestapiapiKeyuser372A97416
[~] AWS::ApiGateway::Stage test-api/DeploymentStage.apigw testapiDeploymentStageapigwEA4DAFFA
 └─ [~] DeploymentId
     └─ [~] .Ref:
         ├─ [-] testapiDeployment356D2C350f0d9c33af21b1b5acbe01dd96f30695
         └─ [+] testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231
[~] AWS::ApiGateway::Method test-api/Default/GET testapiGETD8DE4ED1
 └─ [~] AuthorizationScopes
     └─ @@ -1,1 +1,5 @@
        [-] []
        [+] [
        [+]   "subscriber/user1",
        [+]   "subscriber/user2",
        [+]   "subscriber/user3"
        [+] ]

You can see, that three new API Keys will be added to the UsagePlan. Please note the differences here:

[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource testapiUsagePlanKeyResource6D2F267F
[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource:testapistacktestapitestapiapiKeyuser2A184AC36 testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser2A184AC36802022E8
[+] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource:testapistacktestapitestapiapiKeyuser315D7663C testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser315D7663CF36A0768

The Logical ID of the first UsagePlanKeyResource differs heavily from the other two. This is a screenshot from the AWS console to make things more clear (the screenshot was taken earlier, thats why the IDs of the cdk diff and the aws console don't match - please understand the screenshot as a visual support):

There are three API Keys and they are all related to the UsagePlan with a UsagePlanKey, with the exception of the first UsagePlanKeys Logical ID being different.

With this issue in mind I could remove user2 and/or user3 from the array with no problems (cdk diff):

[-] AWS::ApiGateway::Deployment testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231 destroy
[-] AWS::ApiGateway::UsagePlanKey testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser2A184AC36802022E8 destroy
[-] AWS::ApiGateway::UsagePlanKey testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser315D7663CF36A0768 destroy
[-] AWS::ApiGateway::ApiKey testapitestapiapiKeyuser26097BDA4 destroy
[-] AWS::ApiGateway::ApiKey testapitestapiapiKeyuser372A97416 destroy
[+] AWS::ApiGateway::Deployment test-api/Deployment testapiDeployment356D2C35f7dc1c595d8e9063183b9e1c925537c0
[~] AWS::ApiGateway::Stage test-api/DeploymentStage.apigw testapiDeploymentStageapigwEA4DAFFA
 └─ [~] DeploymentId
     └─ [~] .Ref:
         ├─ [-] testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231
         └─ [+] testapiDeployment356D2C35f7dc1c595d8e9063183b9e1c925537c0
[~] AWS::ApiGateway::Method test-api/Default/GET testapiGETD8DE4ED1
 └─ [~] AuthorizationScopes
     └─ @@ -1,5 +1,3 @@
        [ ] [
        [-]   "subscriber/user1",
        [-]   "subscriber/user2",
        [-]   "subscriber/user3"
        [+]   "subscriber/user1"
        [ ] ]

But when I try to remove user1 instead, the stack fails because of a replacement:

[-] AWS::ApiGateway::Deployment testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231 destroy
[-] AWS::ApiGateway::UsagePlanKey testapiUsagePlanKeyResourcetestapistacktestapitestapiapiKeyuser2A184AC36802022E8 destroy
[-] AWS::ApiGateway::ApiKey testapitestapiapiKeyuser100D43FD3 destroy
[+] AWS::ApiGateway::Deployment test-api/Deployment testapiDeployment356D2C35d4202d75979c652d5556c72542102bf6
[~] AWS::ApiGateway::Stage test-api/DeploymentStage.apigw testapiDeploymentStageapigwEA4DAFFA
 └─ [~] DeploymentId
     └─ [~] .Ref:
         ├─ [-] testapiDeployment356D2C35b1f6494fcf179c88cb95f6e66e796231
         └─ [+] testapiDeployment356D2C35d4202d75979c652d5556c72542102bf6
[~] AWS::ApiGateway::Method test-api/Default/GET testapiGETD8DE4ED1
 └─ [~] AuthorizationScopes
     └─ @@ -1,5 +1,4 @@
        [ ] [
        [-]   "subscriber/user1",
        [ ]   "subscriber/user2",
        [ ]   "subscriber/user3"
        [ ] ]
[~] AWS::ApiGateway::UsagePlanKey test-api/test-api/UsagePlanKeyResource testapiUsagePlanKeyResource6D2F267F replace
 └─ [~] KeyId (requires replacement)
     └─ [~] .Ref:
         ├─ [-] testapitestapiapiKeyuser100D43FD3
         └─ [+] testapitestapiapiKeyuser26097BDA4

3/8 | 15:21:45 | UPDATE_IN_PROGRESS | AWS::ApiGateway::UsagePlanKey | test-api/test-api/UsagePlanKeyResource (testapiUsagePlanKeyResource6D2F267F) Requested update requires the creation of a new physical resource; hence creating one. 3/8 | 15:21:45 | UPDATE_FAILED | AWS::ApiGateway::UsagePlanKey | test-api/test-api/UsagePlanKeyResource (testapiUsagePlanKeyResource6D2F267F) ezplnhe139:vxxai7 already exists in stack arn:aws:cloudformation:eu-central-1:318255054997:stack/test-api-stack/63771480-2e4d-11eb-83d4-02886036c898

I can't figure out why this happens. There is no difference in the code regarding the array elements 0,1,...,n. Please note: When adding just one API Key (or just one array element) I can remove it without problems, but the Logical IDs strange look remains.

The replacement of the KeyId happens only if the number of API Keys is greater than one and the to be removed element is the first one. Is this some kind of bug in the cdk?

Any help is greatly appreciated - thanks!

Kind Regards,
Marcus

Environment

• CDK CLI Version: 1.76.0 (build c207717)
• Module Version: 1.76.0
• Node.js Version: v12.19.0
• OS: Windows 10
• Language (Version): TypeScript (4.0.5) | Python (3.9.0)

[nija-at]

Hi @makruege -

I'm looking to reproduce this issue.

I started with a simple CDK app -

import { App, Stack } from '@aws-cdk/core';
import { RestApi } from '@aws-cdk/aws-apigateway';

const app = new App();
const stack = new Stack(app, 'MyStack');

const api = new RestApi(stack, 'MyRestApi');
api.root.addMethod('ANY');

const users = [ 'user1', 'user2', 'user3'];
const usagePlan = api.addUsagePlan('MyUsagePlan');

users.forEach(u => {
  api.addApiKey(`${u}-apikey`, {
    apiKeyName: `${u}`
  });
});

I'm unable to see any of the symptoms you're pointing at.

The logical ids look normal -

MyRestApiuser1apikey4A9A7FD0
MyRestApiuser2apikey0AB405A5
MyRestApiuser3apikey4AFC6252

I was able to deploy the stack and follow it up with updates to remove these users. They all seem to work fine.
Let me know if I missed something.

I suspect there may be something else in your app that's causing this?

[ghost]

Hi @nija-at!

I've created a blank new cdk project and used your code, so we get the same results.

Yes, the API Key IDs are fine. You have to add the API Keys to the Usage Plan to get the "error":

const usagePlan = api.addUsagePlan("MyUsagePlan");

    users.forEach((u) => {
      const x = api.addApiKey(`${u}-apikey`, {
        apiKeyName: `${u}`,
      });
      usagePlan.addApiKey(x);
    });

And this is the result of a cdk diff:

[+] AWS::ApiGateway::RestApi MyRestApi MyRestApi2D1F47A9
[+] AWS::IAM::Role MyRestApi/CloudWatchRole MyRestApiCloudWatchRoleD4042E8E
[+] AWS::ApiGateway::Account MyRestApi/Account MyRestApiAccount2FB6DB7A
[+] AWS::ApiGateway::Deployment MyRestApi/Deployment MyRestApiDeploymentB555B582f54628948638987a5a88b0c483d20ae5
[+] AWS::ApiGateway::Stage MyRestApi/DeploymentStage.prod MyRestApiDeploymentStageprodC33B8E5F
[+] AWS::ApiGateway::Method MyRestApi/Default/ANY MyRestApiANY05143F93
[+] AWS::ApiGateway::UsagePlan MyRestApi/MyUsagePlan MyRestApiMyUsagePlan9F57E96D
[+] AWS::ApiGateway::UsagePlanKey MyRestApi/MyUsagePlan/UsagePlanKeyResource MyRestApiMyUsagePlanUsagePlanKeyResource19FC21A7
[+] AWS::ApiGateway::UsagePlanKey MyRestApi/MyUsagePlan/UsagePlanKeyResource:MyStackMyRestApiuser2apikey1CE210CD MyRestApiMyUsagePlanUsagePlanKeyResourceMyStackMyRestApiuser2apikey1CE210CDFD917191
[+] AWS::ApiGateway::UsagePlanKey MyRestApi/MyUsagePlan/UsagePlanKeyResource:MyStackMyRestApiuser3apikey9D79E7F7 MyRestApiMyUsagePlanUsagePlanKeyResourceMyStackMyRestApiuser3apikey9D79E7F710E9C5B8
[+] AWS::ApiGateway::ApiKey MyRestApi/user1-apikey MyRestApiuser1apikey4A9A7FD0
[+] AWS::ApiGateway::ApiKey MyRestApi/user2-apikey MyRestApiuser2apikey0AB405A5
[+] AWS::ApiGateway::ApiKey MyRestApi/user3-apikey MyRestApiuser3apikey4AFC6252

You can see three UsagePlanKeys and the first one (MyRestApiMyUsagePlanUsagePlanKeyResource19FC21A7) differs.

After a initial deployment with all three users I remove user2 and user3 (const users = ["user1"];):

λ cdk diff MyStack --profile xxx
Stack MyStack
Resources
[-] AWS::ApiGateway::UsagePlanKey MyRestApiMyUsagePlanUsagePlanKeyResourceMyStackMyRestApiuser2apikey1CE210CDFD917191 destroy
[-] AWS::ApiGateway::UsagePlanKey MyRestApiMyUsagePlanUsagePlanKeyResourceMyStackMyRestApiuser3apikey9D79E7F710E9C5B8 destroy
[-] AWS::ApiGateway::ApiKey MyRestApiuser2apikey0AB405A5 destroy
[-] AWS::ApiGateway::ApiKey MyRestApiuser3apikey4AFC6252 destroy

Seems fine!

Now I remove user1 (const users = ["user2", "user3"];):

λ cdk diff MyStack --profile xxx
Stack MyStack
Resources
[-] AWS::ApiGateway::UsagePlanKey MyRestApiMyUsagePlanUsagePlanKeyResourceMyStackMyRestApiuser2apikey1CE210CDFD917191 destroy
[-] AWS::ApiGateway::ApiKey MyRestApiuser1apikey4A9A7FD0 destroy
[~] AWS::ApiGateway::UsagePlanKey MyRestApi/MyUsagePlan/UsagePlanKeyResource MyRestApiMyUsagePlanUsagePlanKeyResource19FC21A7 replace
 └─ [~] KeyId (requires replacement)
     └─ [~] .Ref:
         ├─ [-] MyRestApiuser1apikey4A9A7FD0
         └─ [+] MyRestApiuser2apikey0AB405A5

It is the same result as in my original comment (replacement) and it is not dependent on the project.

[nija-at]

Thanks for responding. This is indeed a bug in the CDK, that was introduced more than a year ago - 142bd0e#diff-8ec617398ff0629bdc92654be9d4a063d7e8d64fedaf33263604579c2f4e9ea4

[ghost]

Thank you a lot @nija-at! Hopefully it will be fixed soon :-)

[nija-at]

I've prepared a fix here - #12505

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[nija-at]

Unfortunately, the fix has caused unintended regression during stack updates. This change is going to be reverted - #12745

Re-opening this issue.

[ghost]

Hi @nija-at!

I had to make the following changes to apply the fix to our already deployed stacks:

• Renamed the ID of the UsagePlan, otherwise the stack fails because of a duplicate UsagePlanKey (create before destroy)
• Removed the DeploymentStage, otherwise the stack fails because it tries to apply a single API Key to multiple Stages ('Usage Plan xxx cannot be added because API Key yyy cannot reference multiple Usage Plans with the same API Stage')

After the deployment with these changes and the upgrade to 1.87.0 I have reverted these changes and deployed the stacks again, so the UsagePlan IDs and Stages were as before.

This is our original code without the changes:

export const addUsagePlan = (api: RestApi, name: string, rateLimit: number, burstLimit: number): UsagePlan => {
  const usagePlan = api.addUsagePlan(name, {
    name: name,
    throttle: {
      rateLimit,
      burstLimit,
    },
  });

  usagePlan.addApiStage({
    stage: api.deploymentStage,
  });

  return usagePlan;
};

And now with the mentioned changes:

export const addUsagePlan = (api: RestApi, name: string, rateLimit: number, burstLimit: number): UsagePlan => {
  const usagePlan = api.addUsagePlan(name + '-new', {
    name: name,
    throttle: {
      rateLimit,
      burstLimit,
    },
  });

  /*usagePlan.addApiStage({
    stage: api.deploymentStage,
  });*/

  return usagePlan;
};

Maybe these information are useful to you.

[nija-at]

Thanks @makruege. I'll use these to see if there's a better fix for this. Versions 1.87.1 and later will not carry this fix above.
So you may have to add your changes back in. Sorry about that.

[devangpatel01]

@nija-at I have a stack managed by java cdk. I see that logical id for apikeyusageplan is changing sometimes when cdk synth command runs. This causes the stack to complain that resource already exists. Do we have a fix for this?

[ghost]

Hi @nija-at! Because this is a really old bug: What are the chances that there will be a fix in the near future? When we want to be able to remove certain entries in our project we have to stay on version 1.87.0 and cannot upgrade our project to the latest versions.

[nija-at]

@makruege - They key to this issue is that the logical id of the AWS::ApiGateway::UsagePlanKey resources (that you don't intend to remove) needs to stay unchanged. If you want to upgrade to later versions, you could get the CfnUsagePlan constructs from the construct tree, something like this -

const usagePlanKeys = usagePlan.node.children.filter(c => c instanceof CfnUsagePlanKey) as CfnUsagePlanKey[];

and then setting the logical id to the value that it already has by using the overrideLogicalId() method

---

I've taken another stab at preparing a fix for this - #13817. I hope for better luck with this change.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[b-c-lucas]

@makruege - They key to this issue is that the logical id of the AWS::ApiGateway::UsagePlanKey resources (that you don't intend to remove) needs to stay unchanged. If you want to upgrade to later versions, you could get the CfnUsagePlan constructs from the construct tree, something like this -

const usagePlanKeys = usagePlan.node.children.filter(c => c instanceof CfnUsagePlanKey) as CfnUsagePlanKey[];

and then setting the logical id to the value that it already has by using the overrideLogicalId() method

@nija-at @makru21 I'm running into this problem now when trying to upgrade to CDK v2 (albeit in Python) and as someone newer to CDK I'm having trouble figuring out how to patch this code.

• after which point should the usage plan's children be filtered and the override applied?

api_key = api.add_api_key(f"{id}APIKey", value=value)
usage_plan = api.add_usage_plan(f"{id}APIUsagePlan")
usage_plan.add_api_key(api_key)
usage_plan.add_api_stage(stage=api.deployment_stage)

• how do you programmatically get the previous logical ID from within the Stack?

[yosefbs]

@brad-lucas try to add :

"context": {
    "@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": false
  }

in your cdk.json file.
in cdk1 default value(for existing project) was false but when updated to cdk2 the default value is true

[nbagnallx]

@brad-lucas / @yosefbs I was only able to fix this issue by implementing a fix similar to ghost's (user earlier in this thread). Basically I just made a small change to the logical IDs for the usage plan and for the gateway API stage, in effect forcing CloudFormation to re-create these constructs (as well as a usage plan key).