Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SmartFlank default test time is too big #977

Closed
CristianGM opened this issue Aug 11, 2020 · 0 comments · Fixed by #1042
Closed

SmartFlank default test time is too big #977

CristianGM opened this issue Aug 11, 2020 · 0 comments · Fixed by #1042
Assignees

Comments

@CristianGM
Copy link
Contributor

Author the user story for this feature

As a user, I want to reduce the default test time so I can waste less resources.

Is your feature request related to a problem? Please describe.
When I run flank with SmartFlank enabled I see it's using too many shards because it counts 120s for each test without previous times.
Flank suggest (and Fladle sets by default) a maximum shard time of 120s, so, each new test is using a shard for itself, and in our case it can run in 2 seconds, so, we are paying for 58 seconds that we don't use, multiplied per the number of new tests.

Describe the solution you'd like
There are 2 scenarios:

  • first build: then we need to use the default test time, so probably having a parameter for it is the right solution.
  • new tests: probably here we could use the average duration from the other tests

With the average we would fix the worst issue because it happens all the time, but in any case DEFAULT_TEST_TIME_SEC == suggested SHARD_TIME doesn't make any sense.

Describe alternatives you've considered
None that fixes the issue without adding others.

Additional context
The docs say that the default is 10s but it was changed to 120s #635 making it a big issue that otherwise won't be too noticeable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants