Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce VCR recording runtime after package split #9722

Merged

Conversation

shuyama1
Copy link
Member

@shuyama1 shuyama1 commented Dec 27, 2023

I've observed some significant recording runtime increases, especially for large size of affected tests, after VCR split.

Example PR: #9666
Before VCR split: ~20 mins for recording #9666 (comment)
After VCR split: ~2 hours for recording #9666 (comment)

What I suspect is, even though VCR split only changed how we run tests in the REPLAYING mode, it skips package compiling for all non-affected services and therefore it needs to compile them (probably multiple times) in the RECORDING mode (previously using compilation cache from the REPLAYING)
Given how we enable the parallelism in RECORDING and the lower size of it due to the limitation of Terraform log setup, it needs much more total compilation time and therefore leads to the long RECORDING runtime.

To resolve this issue, since we've known what packages are affected and needed to run tests against, we can only loop through these packages and run failed tests against them.

Testing: #9717 (comment): recording mode took ~2.5h for 12 TestAccComputeFirewall* and after apply the change in this PR, it takes only ~10 minutes #9721 (comment).

Release Note Template for Downstream PRs (will be copied)


@modular-magician
Copy link
Collaborator

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR hasn't generated any diffs, but I'll let you know if a future commit does.

@shuyama1 shuyama1 requested a review from slevenick December 27, 2023 23:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants