Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(integ-runner): fails with "no such file or directory, open 'node_modules/@aws-cdk/integ-runner/lib/workers/db.json.gz'" #27983

Merged
merged 5 commits into from
Nov 14, 2023

Conversation

mrgrain
Copy link
Contributor

@mrgrain mrgrain commented Nov 14, 2023

Mark the @aws-cdk/aws-service-spec as external, because it cannot be bundled.

Changes an import from cdk-assets to be more specific, so we don't bundle the whole of aws-sdk anymore.
This reduced the worker entrypoint size from 70MB -> 1.5MB


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team November 14, 2023 13:41
@github-actions github-actions bot added the p2 label Nov 14, 2023
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Nov 14, 2023
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@mrgrain mrgrain added pr-linter/exempt-test The PR linter will not require test changes pr-linter/exempt-integ-test The PR linter will not require integ test changes labels Nov 14, 2023
@aws-cdk-automation aws-cdk-automation dismissed their stale review November 14, 2023 13:54

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Nov 14, 2023
@@ -1,6 +1,6 @@
import * as path from 'path';
import { AssemblyManifest, Manifest, ArtifactType, AwsCloudFormationStackProperties, ArtifactManifest, MetadataEntry, AssetManifestProperties, ArtifactMetadataEntryType, ContainerImageAssetMetadataEntry, FileAssetMetadataEntry } from '@aws-cdk/cloud-assembly-schema';
import { AssetManifest, FileManifestEntry, DockerImageManifestEntry } from 'cdk-assets';
import { AssetManifest, FileManifestEntry, DockerImageManifestEntry } from 'cdk-assets/lib/asset-manifest';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's ugly. But since it's a monorepo we can do it!

Copy link
Contributor

mergify bot commented Nov 14, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Nov 14, 2023
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 8029b50
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 56daf0b into main Nov 14, 2023
13 checks passed
@mergify mergify bot deleted the mrgrain/fix/integ-runner-fails-to-load-database branch November 14, 2023 16:17
Copy link
Contributor

mergify bot commented Nov 14, 2023

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@AntonioAngelino
Copy link
Contributor

@mrgrain The cli-lib-alpha is still broken for the same reason.

Deployment failed: Error: ENOENT: no such file or directory, open '/home/xxx/yyyy/node_modules/@aws-cdk/cli-lib-alpha/db.json.gz'

@mrgrain
Copy link
Contributor Author

mrgrain commented Nov 23, 2023

Thanks! I know why. Looking into it when I'm back at work next week.

@AntonioAngelino
Copy link
Contributor

@mrgrain let me know if I can help. This issue is a blocker for us. Thank you!

@mrgrain
Copy link
Contributor Author

mrgrain commented Nov 27, 2023

I have the fix ready. Currently making sure this won't happen again. Unfortunately due to re:Invent it won't be released until next week.

@AntonioAngelino
Copy link
Contributor

Ouch. Which old version of aws cdk is not affected by the bug?

We can think of using an older version or a nightly build (if it's possible).

@mrgrain
Copy link
Contributor Author

mrgrain commented Nov 27, 2023

v2.106.0 should be the one that introduced the issue, so anything before that should still work. There isn't anything that's strictly forcing you to keep aws-cdk-lib at the same version, but if you can I would.

We don't have nightly builds at the moment. If you want to hack it, this package contains a db.json.gz file that needs to be copied to ./node_modules/@aws-cdk/cli-lib-alpha/db.json.gz.

@AntonioAngelino
Copy link
Contributor

Ok, thank you. Hacking is fine for now since we're just developing the solution, ty. 👍

mergify bot pushed a commit that referenced this pull request Nov 30, 2023
…e_modules/@aws-cdk/integ-runner/lib/workers/db.json.gz'" (#28199)

After #27813 the `deploy` action was broken with the above error. This is effectively the same as #27983 .

To ensure these kind of issues are not slipping through again, the PR is adding a basic testing harness for `cli-lib` to `@aws-cdk-testing/cli-integtests`.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@mrgrain
Copy link
Contributor Author

mrgrain commented Nov 30, 2023

@AntonioAngelino Patched and put some integration tests in place that actually use the bundled code. Should be shipping tomorrow.

@AntonioAngelino
Copy link
Contributor

Thank you, @mrgrain !

chenjane-dev pushed a commit to chenjane-dev/aws-cdk that referenced this pull request Dec 5, 2023
…e_modules/@aws-cdk/integ-runner/lib/workers/db.json.gz'" (aws#28199)

After aws#27813 the `deploy` action was broken with the above error. This is effectively the same as aws#27983 .

To ensure these kind of issues are not slipping through again, the PR is adding a basic testing harness for `cli-lib` to `@aws-cdk-testing/cli-integtests`.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS. p2 pr-linter/exempt-integ-test The PR linter will not require integ test changes pr-linter/exempt-test The PR linter will not require test changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants