You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To Reproduce
A slightly simplified version with relevant blocks is provided. I have a custom test runner which looks for an argument called theme which then sets the theme to either light mode or dark mode during the test run. I would like to run both themes in a single flank run across the same tests so I am setting the same test apk on both.
# See the docs for full gcloud details https://cloud.google.com/sdk/gcloud/reference/firebase/test/android/run
gcloud:
# -- GcloudYml --
### Number of Flaky Test Attempts
## The number of times a TestExecution should be re-attempted if one or more\nof its test cases fail for any reason.
## The maximum number of reruns allowed is 10. Default is 0, which implies no reruns.
num-flaky-test-attempts: 1
# -- AndroidGcloudYml --
## Android Application Path
## The path to the application binary file.
## The path may be in the local filesystem or in Google Cloud Storage using gs:// notation.
## Android App Bundles are specified as .aab, all other files are assumed to be APKs.
app: ./StyleGuide/build/outputs/apk/debug/StyleGuide-debug.apk
### Android Binary File Path
## The path to the binary file containing instrumentation tests.
## The given path may be in the local filesystem or in Google Cloud Storage using a URL beginning with gs://.
test: ./StyleGuide/build/outputs/apk/androidTest/debug/StyleGuide-debug-androidTest.apk
### Use Orchestrator Flag
## Whether each test runs in its own Instrumentation instance with the Android Test Orchestrator
## (default: Orchestrator is used). Disable with --no-use-orchestrator.
## See https://developer.android.com/training/testing/junit-runner.html#using-android-test-orchestrator
use-orchestrator: true
### Environment Variables
## A comma-separated, key=value map of environment variables and their desired values. This flag is repeatable.
## The environment variables are mirrored as extra options to the am instrument -e KEY1 VALUE1 … command and
## passed to your test runner (typically AndroidJUnitRunner)
environment-variables:
clearPackageData: true
theme: darkMode
### Directories to Pull
## A list of paths that will be copied from the device's storage to the designated results bucket after the test
## is complete. These must be absolute paths under /sdcard or /data/local/tmp
directories-to-pull:
- /sdcard/Pictures/SnapshotsTestResults/
### Test Type
## The type of test to run. TYPE must be one of: instrumentation, robo, game-loop.
type: instrumentation
flank:
# -- FlankYml --
### Max Test Shards
## test shards - the amount of groups to split the test suite into
## set to -1 to use one shard per test. default: 1
max-test-shards: 50
### Local Results Directory
## Local folder to store the test result. Folder is DELETED before each run to ensure only artifacts from the new run are saved.
local-result-dir: flank
### Additional App/Test APKS
## Include additional app/test apk pairs in the run. Apks are unique by just filename and not by path!
## If app is omitted, then the top level app is used for that pair.
## You can overwrite global config per each test pair.
## Currently supported options are: max-test-shards, test-targets, client-details, environment-variables, device
additional-app-test-apks:
- test: ./StyleGuide/build/outputs/apk/androidTest/debug/StyleGuide-debug-androidTest.apk
environment-variables:
clearPackageData: true
theme: lightMode
### Enable output report with set type
## Saves output results as parsable file and optionally upload it to Gcloud. Possible values are [none, json].
## Default: none
output-report: json
Expected behavior
Flank would execute two runs on the test matrix, once with the theme passed in being lightMode and once with the theme passed in being darkMode
Details (please complete the following information):
Using the Bitrise Flank step, which uses latest flank version
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered:
Describe the bug
When following the docs for setting different environment variables in different matrices (https://github.com/Flank/flank/blob/master/docs/using_different_environment_variables_in_different_matrices.md) I am not seeing my environment variables changing between runs on Firebase.
To Reproduce
A slightly simplified version with relevant blocks is provided. I have a custom test runner which looks for an argument called
theme
which then sets the theme to either light mode or dark mode during the test run. I would like to run both themes in a single flank run across the same tests so I am setting the same test apk on both.Expected behavior
Flank would execute two runs on the test matrix, once with the theme passed in being
lightMode
and once with the theme passed in beingdarkMode
Details (please complete the following information):
Using the Bitrise Flank step, which uses latest flank version
Additional context
The text was updated successfully, but these errors were encountered: