Fix parametrized tests #840
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #840 +/- ##
============================================
+ Coverage 81.23% 81.33% +0.09%
Complexity 637 637
============================================
Files 167 167
Lines 3240 3257 +17
Branches 463 468 +5
============================================
+ Hits 2632 2649 +17
Misses 343 343
Partials 265 265 |
| @@ -0,0 +1,4 @@ | |||
| gcloud: | |||
| app: ./src/test/kotlin/ftl/fixtures/tmp/apk/app-single-success-parametrized-debug.apk | |||
There was a problem hiding this comment.
Apks shouldn't be added directly to the flank repo, instead we keep them in https://github.com/Flank/test_artifacts to keep the flank repo size small
For the sample app, let's update the source code in this repo so we can easily rebuild the apks. As Android evolves, we'll want to rebuild apks with different options and target platforms.
jan-goral
reviewed
Jun 8, 2020
| .flatMap { annotations -> annotations.values.values } | ||
| .filterIsInstance<DecodedValue.DecodedType>() | ||
| .map { it.value } | ||
| .any { it.toLowerCase().contains("Parameterized".toLowerCase()) }) { // sprawdzamy czy klasa zadana jako parametr adnotacji to Parameterized |
There was a problem hiding this comment.
I wonder if there are any other ways to define tests that can be identified by AndroidJUnitRunner as parameterized
jan-goral
reviewed
Jun 9, 2020
| fun shouldHopefullyPass() { | ||
| assertTrue(paramOne == paramTwo.toInt()) | ||
| } | ||
| } |
…into #809-Fix-parameterized-testing
…into #809-Fix-parameterized-testing # Conflicts: # test_runner/src/test/kotlin/ftl/run/platform/android/CreateAndroidTestContextKtTest.kt
piotradamczyk5
requested review from
jan-goral,
piotradamczyk5,
bootstraponline and
pawelpasterz
jan-goral
force-pushed
the
#809-Fix-parameterized-testing
branch
from
f915f2d to
7796d4c
Compare
jan-goral
reviewed
Jun 9, 2020
…er::class) and @RunWith(Parameterized::class)
…into #809-Fix-parameterized-testing
| .any { it.isParameterizedAnnotation() } | ||
| } | ||
|
|
||
| private fun DecodedValue.DecodedType.isParameterizedAnnotation(): Boolean = |
There was a problem hiding this comment.
This is brilliant! ⭐
jan-goral
approved these changes
Jun 12, 2020
bootstraponline
approved these changes
Jun 12, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #809
Test Plan
When test will executed from Flank should have results like same test executed directly on testlab.
Additional information
Test directly executed from test lab
https://console.firebase.google.com/u/0/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/8109159837961280691
Failed test executed from Flank
https://console.firebase.google.com/u/0/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/8167982987416424556/executions/bs.38eda8775a39b098
and other with video
https://console.firebase.google.com/u/0/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/7496090094871625048/executions/bs.160a4ed4fdc86abe
Checklist
Objectives
Parametrized tests classes should be run without splitting into single methods to avoid run errors.
Tests status
https://github.com/adamfilipow92/flank_parametrized_tests/tree/master/test_apk