Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Additional config options for test pairs #2004

Merged
merged 9 commits into from
Jun 9, 2021

Conversation

pawelpasterz
Copy link
Contributor

@pawelpasterz pawelpasterz commented Jun 3, 2021

Fixes #1815

This PR introduces a couple of changes:

  1. from now on each android test context will have its own copy of AndroidConfig
  2. each copy will be overwritten with values defined by the user
  3. each copy will be validated

With the above enhancements, we can extend additional-test-app-pairs easier in the future. For now, only community-requested options are implemented.

Test Plan

How do we know the code works?

  1. Run ./gradlew flankFullRun
  2. Example config:
    gcloud:
      app: ...
      test: ...
    flank:
      max-test-shards: 2
      additional-app-test-apks:
        - test: ...
          max-test-shards: 5
        - test: ...
          device:
            - model: Nexus6P
              version: 27
  3. Yous should receive 3 matrices with different set up:
    1. 2 shards, NexusLowRes device (default one)
    2. 5 shards, NexusLowRes device
    3. 2 shards, Nexus6P device

Checklist

  • Documented
  • Unit tested
  • Integration tests updated

@pawelpasterz pawelpasterz self-assigned this Jun 3, 2021
@pawelpasterz pawelpasterz force-pushed the 1815-additional-config-options branch from 01647aa to 527fb5d Compare June 4, 2021 05:52
@github-actions
Copy link
Contributor

github-actions bot commented Jun 4, 2021

Timestamp: 2021-06-09 07:26:43
Buildscan url for ubuntu-workflow run 920860017
https://gradle.com/s/jv7unx233jt7e

@pawelpasterz pawelpasterz force-pushed the 1815-additional-config-options branch from 40e05f0 to 2225f1a Compare June 4, 2021 08:51
Copy link
Contributor

@Sloox Sloox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks alright for now. Lets get another pair of eyes on it aswell.
@asadsalman Also have a look at the effect it might have for your changes a while back and if it fixes the issues.

@@ -73,18 +70,11 @@ private fun createAndroidTestMatrix(
}.getOrElse { e -> throw FlankGeneralError(e) }
}

fun TestMatrixAndroid.Config.clientInfo(matrix: TestMatrixAndroid.Type): ClientInfo {
return if (matrix is TestMatrixAndroid.Type.Instrumentation && matrix.clientDetails.isNotEmpty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pawelpasterz I am sure that there were changes done here for a fix by @asadsalman. Can we confirm that it wont be effected by this change as it seems like a revert to the previous version of the code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right - this will revert those changes. But this is intentional, previous implementation merges client details (from main and additional test pair). All the other, 'additional' configs overwrite 'main' configs. I think we should be consistent here. Saying that merging is not the best option and clientDetails should be overwritten as well.

@asadsalman
Copy link
Contributor

@pawelpasterz this PR looks great! Thanks for adding more fields to additional test APKs!

@Sloox, I had added unit tests that that covered the changes I had made previously. They're still passing so I don't think there's issue or regression here. I'll manually verify with an end-to-end run today too but I don't expect any issues.

@pawelpasterz pawelpasterz force-pushed the 1815-additional-config-options branch from 4404cfd to d4fea83 Compare June 8, 2021 04:03
@bootstraponline
Copy link
Contributor

I had added unit tests that that covered the changes I had made previously. They're still passing so I don't think there's issue or regression here.

I love this! That's one reason why tests are valuable. We're less likely to break business logic when refactoring.

@pawelpasterz pawelpasterz requested a review from Sloox June 8, 2021 04:44
@pawelpasterz pawelpasterz marked this pull request as ready for review June 8, 2021 04:44
@pawelpasterz pawelpasterz force-pushed the 1815-additional-config-options branch from 2f625ee to 78a4d3d Compare June 8, 2021 12:28
@@ -9,8 +9,8 @@ import com.google.testing.model.TestMatrix as GoogleTestMatrix

object GoogleTestMatrixAndroid :
TestMatrixAndroid.Execute,
(TestMatrixAndroid.Config, List<TestMatrixAndroid.Type>) -> List<TestMatrix.Data> by { config, types ->
(List<Pair<TestMatrixAndroid.Config, TestMatrixAndroid.Type>>) -> List<TestMatrix.Data> by { configTypePairs ->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it will be better to have a separate data class instead of pair, WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally agree, thanks!

@pawelpasterz pawelpasterz enabled auto-merge (squash) June 9, 2021 06:26
@pawelpasterz pawelpasterz force-pushed the 1815-additional-config-options branch from 35ee69c to 5f10825 Compare June 9, 2021 07:20
@pawelpasterz pawelpasterz merged commit cda65b5 into master Jun 9, 2021
@pawelpasterz pawelpasterz deleted the 1815-additional-config-options branch June 9, 2021 08:52
@github-actions github-actions bot locked and limited conversation to collaborators Jun 9, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow fine turning configurations per APK pair whose tests are ran
5 participants