Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Load test cases durations from previous run and use for sharding #1998

Merged
merged 4 commits into from
Jun 2, 2021

Conversation

jan-goral
Copy link
Contributor

@jan-goral jan-goral commented Jun 2, 2021

Fixes #1988

Test Plan

  1. Run flank corellium test android run -c="./test_configs/flank-corellium.yml" several times.
  2. Observe the generated android-shards.json have durations different than the default (120)

Drawbacks

The first test case in a suite always has a duration increased by initialization overhead. This is a am instrument command behavior, that not depends on flank implementation. As result, the sharding calculations can be more imprecise for cases with a small number of quick test cases with duration less than initialization overhead.

A possible solution could be evaluating overhead time based on previous JUnit results, and taking them into account in sharding calculations.

Checklist

  • Documented
  • Unit tested

@jan-goral jan-goral self-assigned this Jun 2, 2021
@jan-goral jan-goral changed the title Load test cases durations from previous run and use for sharding feat: Load test cases durations from previous run and use for sharding Jun 2, 2021
@jan-goral jan-goral force-pushed the 1988_Reuse_tests_duration_for_sharding branch from 40aa407 to b498afb Compare June 2, 2021 09:58
@github-actions
Copy link
Contributor

github-actions bot commented Jun 2, 2021

Timestamp: 2021-06-02 10:06:57
Buildscan url for ubuntu-workflow run 898930726
https://gradle.com/s/e7r46v76fimf6

@jan-goral jan-goral marked this pull request as ready for review June 2, 2021 10:19
@mergify mergify bot merged commit 1539afd into master Jun 2, 2021
@mergify mergify bot deleted the 1988_Reuse_tests_duration_for_sharding branch June 2, 2021 14:36
@github-actions github-actions bot locked and limited conversation to collaborators Jun 2, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reuse tests duration for sharding
3 participants