Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(assets): publish Assets in parallel #19193

Closed
1 of 2 tasks
skinny85 opened this issue Feb 28, 2022 · 4 comments · Fixed by #19367
Closed
1 of 2 tasks

(assets): publish Assets in parallel #19193

skinny85 opened this issue Feb 28, 2022 · 4 comments · Fixed by #19367
Labels
@aws-cdk/assets Related to the @aws-cdk/assets package effort/medium Medium work item – several days of effort feature/enhancement A new API to make things easier or more intuitive. A catch-all for general feature requests. feature-request A feature should be added or improved. p2

Comments

@skinny85
Copy link
Contributor

Description

Right now, we publish the Assets serially. This is slow if the customer's App contains many Assets (it's visible with as little Assets as 10).

Use Case

CDK Apps with many (10+) Assets.

Proposed Solution

We should switch to publishing the Assets in parallel, instead of serially. The tricky part of this will be making sure the Asset publishing progress messages are handled correctly.

Other information

No response

Acknowledge

  • I may be able to implement this feature request
  • This feature might incur a breaking change
@skinny85 skinny85 added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Feb 28, 2022
@github-actions github-actions bot added the @aws-cdk/assets Related to the @aws-cdk/assets package label Feb 28, 2022
@skinny85 skinny85 added effort/medium Medium work item – several days of effort feature/enhancement A new API to make things easier or more intuitive. A catch-all for general feature requests. p2 and removed needs-triage This issue or PR still needs to be triaged. labels Feb 28, 2022
@misterjoshua
Copy link
Contributor

In my project, I observed these timings:

11 assets (10 functions):
Synth time: 11.66s
Deployment time: 18.49s
Total time: 30.15 seconds

3 assets (10 functions):
Synth time: 11.47s
Deployment time: 6.61s
Total time: 18.08s

In aggregate, all assets are under 28MB.

@misterjoshua
Copy link
Contributor

misterjoshua commented Mar 12, 2022 •
edited
Loading

I was fiddling with a toy implementation. Here's a hot-swap comparison for 60 assets and 60 lambda functions:

Sequential (57 seconds)
https://user-images.githubusercontent.com/644092/158005926-5892c83e-15f6-4ebe-9158-643314cd6e68.mp4

Parallel Asset Publishing (18 seconds)
https://user-images.githubusercontent.com/46012524/158006769-a27a63bd-98f9-417a-9b03-c9908543533a.mp4

@neilkuan
Copy link
Contributor

Sequential (57 seconds)

publish-sequential.mp4



Parallel Asset Publishing (18 seconds)

publish-parallel.mp4

cc @misterjoshua :)

@misterjoshua misterjoshua mentioned this issue Mar 12, 2022
Merged
@Chriscbr Chriscbr mentioned this issue Mar 12, 2022
Closed
2 tasks
@mergify mergify bot closed this as completed in #19367 Mar 14, 2022
mergify bot pushed a commit that referenced this issue Mar 14, 2022
@misterjoshua
feat(cli): parallel asset publishing (#19367)
c8cafef 
Adds parallel asset publishing. In a 60-asset/60-Lambda test project, this change decreases the total hot-swap time from 57 seconds to 18 seconds.

Fixes #19193

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@bilalq bilalq mentioned this issue Nov 21, 2022
Open
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
@aws-cdk/assets Related to the @aws-cdk/assets package effort/medium Medium work item – several days of effort feature/enhancement A new API to make things easier or more intuitive. A catch-all for general feature requests. feature-request A feature should be added or improved. p2
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(cli): parallel asset publishing misterjoshua/aws-cdk
4 participants
@otaviomacedo @skinny85 @misterjoshua @neilkuan