integ-tests: integ-runner fails updating snapshot

[jumic]

Describe the bug

I'm trying to update an integration test snapshot. I execute command yarn integ-runner --force sns/integ.sns-event-rule-target.js. Unfortunately the output contains an error:

Test Results: 

Tests:    1 passed, 1 total
Error: Some snapshot tests failed!
To re-run failed tests run: yarn integ-runner --update-on-failed
    at main (/workspace/aws-cdk/packages/@aws-cdk/integ-runner/lib/cli.js:118:15)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Expected Behavior

Integration snapshot update is executed without error message.

Current Behavior

Output of command yarn integ-runner --force sns/integ.sns-event-rule-target.js in folder packages/@aws-cdk/aws-events-targets:

gitpod@aws-awscdk-4jwed33pden:/workspace/aws-cdk/packages/@aws-cdk/aws-events-targets$ yarn integ-runner --force sns/integ.sns-event-rule-target.js 
yarn run v1.22.18
$ /workspace/aws-cdk/node_modules/.bin/integ-runner --force sns/integ.sns-event-rule-target.js

Verifying integration test snapshots...

  sns/integ.sns-event-rule-target No Change! 0.522s

Snapshot Results: 

Tests:    1 passed, 1 total

Running integration tests for failed tests...

Running in parallel across regions: us-east-1, us-east-2, us-west-2
Running test /workspace/aws-cdk/packages/@aws-cdk/aws-events-targets/test/sns/integ.sns-event-rule-target.js in us-east-1
  sns/integ.sns-event-rule-target Test Succeeded! 198.403s

Test Results: 

Tests:    1 passed, 1 total
Error: Some snapshot tests failed!
To re-run failed tests run: yarn integ-runner --update-on-failed
    at main (/workspace/aws-cdk/packages/@aws-cdk/integ-runner/lib/cli.js:118:15)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Output of command yarn integ-runner --update-on-failed in folder packages/@aws-cdk/aws-events-targets:

gitpod@aws-awscdk-4jwed33pden:/workspace/aws-cdk/packages/@aws-cdk/aws-events-targets$ yarn integ-runner --update-on-failed
yarn run v1.22.18
$ /workspace/aws-cdk/node_modules/.bin/integ-runner --update-on-failed

Verifying integration test snapshots...

  aws-api/integ.aws-api No Change! 2.018s
  batch/integ.job-definition-events No Change! 1.933s
  lambda/integ.events No Change! 1.983s
  event-bus/integ.event-bus No Change! 1.85s
  kinesis-firehose/integ.kinesis-firehose-stream No Change! 2.006s
  codebuild/integ.project-events No Change! 2.189s
  kinesis/integ.kinesis-stream No Change! 1.924s
  codepipeline/integ.pipeline-event-target No Change! 2.124s
  sns/integ.sns-event-rule-target - Snapshot changed! 1.734s
Parameters
[-] Parameter BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"}

Other Changes
[-] Unknown Rules: {"CheckBootstrapVersion":{"Assertions":[{"Assert":{"Fn::Not":[{"Fn::Contains":[["1","2","3","4","5"],{"Ref":"BootstrapVersion"}]}]},"AssertDescription":"CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."}]}}


  sqs/integ.sqs-event-rule-target No Change! 1.999s
  ecs/integ.event-fargate-task No Change! 1.895s
  logs/integ.log-group No Change! 1.998s
  ecs/integ.event-ec2-task.lit No Change! 2.059s

Snapshot Results: 

Tests:    1 failed, 13 total

Running integration tests for failed tests...

Running in parallel across regions: us-east-1, us-east-2, us-west-2
Running test /workspace/aws-cdk/packages/@aws-cdk/aws-events-targets/test/sns/integ.sns-event-rule-target.js in us-east-1
  sns/integ.sns-event-rule-target Test Succeeded! 199.564s

Test Results: 

Tests:    1 passed, 1 total
Error: Some snapshot tests failed!

    at main (/workspace/aws-cdk/packages/@aws-cdk/integ-runner/lib/cli.js:118:15)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Reproduction Steps

1. Open Gitpot (link from CONTRIBUTING.md): https://gitpod.io/#https://github.com/aws/aws-cdk - A new workspace from the current master branch will be initialized.
2. Setup credentials for integration tests (run aws configure)
3. Build all packages: yarn build --skip-test --no-bail --skip-prereqs --skip-compat
4. change to package events-targets: cd packages/@aws-cdk/aws-events-targets
5. verify tests again:yarn build+test --> successful
6. Run integration tests: yarn integ-runner --force sns/integ.sns-event-rule-target.js --> Output contains error (see Current Behavior)

Possible Solution

Unknown, maybe something related to argument --no-version-reporting? The output contains message CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI.

Additional Information/Context

• The problem also occurs on my laptop. I've reproduced it on Gitpod to describe the behavior for this issue.
• I'm trying to run the integration tests for PR feat(events-targets): Add DLQ support for SNS target #20062. After they failed, I recognized that the issue also happens on branch master.

CDK CLI Version

bash: cdk: command not found --> not installed on Gitpod

Framework Version

No response

Node.js Version

v12.22.12

OS

Linux (Gitpod)

Language

Typescript

Language Version

~3.9.10

Other information

No response

[jumic]

In my opinion, the behaviour of the integration logic is correct, again. I could update the snapshots in my PR. The following errors will not displayed anymore:

Parameters
[-] Parameter BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"}

Other Changes
[-] Unknown Rules: {"CheckBootstrapVersion":{"Assertions":[{"Assert":{"Fn::Not":[{"Fn::Contains":[["1","2","3","4","5"],{"Ref":"BootstrapVersion"}]}]},"AssertDescription":"CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."}]}}

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.