Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: 1665 implement custom sharding -- android #1763

Merged
merged 11 commits into from
Apr 6, 2021

Conversation

pawelpasterz
Copy link
Contributor

@pawelpasterz pawelpasterz commented Mar 30, 2021

Android part for #1665

Detailed documentation

This is the first implementation of the custom sharding feature, below assumptions were made:

  1. flank does not validate provided json -- we trust the user that he/she knows what is doing
  2. sharding-json is the source of truth -- no smart sharding applied
  3. custom sharding can be launched together with robo/game loop test

Test Plan

How do we know the code works?

  1. build flank ./gradlew flankFullRun
  2. create config yml
    gcloud:
      app: [PATH]/app-debug.apk
      robo-script: [PATH]/MainActivity_robo_script.json
      use-orchestrator: false
    flank:
      max-test-shards: 2
      additional-app-test-apks:
        - test: [PATH]/app-single-success-debug-androidTest.apk
        - test: [PATH]/app-multiple-error-debug-androidTest.apk
        - test: [PATH]/app-multiple-success-debug-androidTest.apk
      disable-results-upload: true
      disable-usage-statistics: true
    
  3. run flank firebase test android run -c=[YOUR CONFIG] --dump-shards
  4. copy android_shards.json, rename it and play around with shards, example: add another shard, move tests etc
  5. add sharding-json: [PATH TO YOUR CUSTOM SHARDING JSON] to your config yaml
  6. run flank firebase test android run -c=[YOUR CONFIG] --dump-shards
  7. observe new sharding in android_shards.json (notice that shards are properly assigned to app-test apk pairs)
  8. run flank firebase test android run -c=[YOUR CONFIG]
  9. test run should proceed and end normally with sharding that you created

Note:

  • matrices ids don't matter when preparing custom sharding
  • shards id as well

Checklist

  • Documented
  • Unit tested
  • Integration tests updated

@pawelpasterz pawelpasterz self-assigned this Mar 30, 2021
@github-actions
Copy link
Contributor

github-actions bot commented Mar 30, 2021

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@github-actions
Copy link
Contributor

github-actions bot commented Mar 30, 2021

Timestamp: 2021-04-06 07:21:05
Buildscan url for ubuntu-workflow run 721666364
https://gradle.com/s/rt7yp4pljxvge

@pawelpasterz pawelpasterz force-pushed the 1665-implement-custom-sharding-android branch from 442f599 to e6e8ec0 Compare March 30, 2021 13:25
@pawelpasterz pawelpasterz force-pushed the 1665-implement-custom-sharding-android branch from 3e9220e to 5a3e1be Compare March 31, 2021 11:01
@pawelpasterz pawelpasterz marked this pull request as ready for review March 31, 2021 11:03
@pawelpasterz pawelpasterz enabled auto-merge (squash) March 31, 2021 11:13
Copy link
Contributor

@Sloox Sloox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comments and fixes.

@pawelpasterz pawelpasterz requested a review from Sloox March 31, 2021 12:03
@piotradamczyk5
Copy link
Contributor

@flank-it

@github-actions
Copy link
Contributor

github-actions bot commented Mar 31, 2021

Integration tests failed ❌
Windows status failure - Build scan:
MacOS status failure - Build scan: https://gradle.com/s/yqmy6ee6blu74
Linux status failure - Build scan: https://gradle.com/s/fqwpbzskzn7nc
Workflow run https://github.com/Flank/flank/actions/runs/705044229

@pawelpasterz
Copy link
Contributor Author

Integration tests failed ❌
Windows status failure - Build scan:
MacOS status failure - Build scan: https://gradle.com/s/yqmy6ee6blu74
Linux status failure - Build scan: https://gradle.com/s/fqwpbzskzn7nc
Workflow run https://github.com/Flank/flank/actions/runs/705044229

Quota limit exceeded...

@jan-goral
Copy link
Contributor

jan-goral commented Mar 31, 2021

Great job!!! 🚀 Let's wait with approval until quota refresh and run IT tests once again tomorrow.

@bootstraponline bootstraponline force-pushed the 1665-implement-custom-sharding-android branch from e967854 to f96f25b Compare April 1, 2021 07:50
@pawelpasterz pawelpasterz force-pushed the 1665-implement-custom-sharding-android branch from d82ef10 to 7e83bd7 Compare April 2, 2021 04:59
@bootstraponline bootstraponline force-pushed the 1665-implement-custom-sharding-android branch from 7e83bd7 to 43d1a4d Compare April 2, 2021 07:43
@pawelpasterz
Copy link
Contributor Author

@flank-it

@github-actions
Copy link
Contributor

github-actions bot commented Apr 2, 2021

Integration tests succeed for all OSes ✅
Windows Build scan:
MacOS Build scan: https://gradle.com/s/xfyhelbqzqe3s
Linux Build scan: https://gradle.com/s/pvxklawz3wrxu
Workflow run https://github.com/Flank/flank/actions/runs/711062354

@piotradamczyk5 piotradamczyk5 self-requested a review April 2, 2021 09:34
@pawelpasterz pawelpasterz requested a review from jan-goral April 2, 2021 09:39
@pawelpasterz
Copy link
Contributor Author

@jan-gogo IT pass, let me know if I should put some more effort into this PR or it's good to merge

@bootstraponline bootstraponline force-pushed the 1665-implement-custom-sharding-android branch from 43d1a4d to 1861f45 Compare April 6, 2021 07:15
@pawelpasterz pawelpasterz merged commit 3820f25 into master Apr 6, 2021
@pawelpasterz pawelpasterz deleted the 1665-implement-custom-sharding-android branch April 6, 2021 11:23
@github-actions github-actions bot locked and limited conversation to collaborators Apr 6, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants