From 66cff410ff585beffba5f150799b3f77b7065dfb Mon Sep 17 00:00:00 2001 From: "flank-release[bot]" <73127551+flank-release[bot]@users.noreply.github.com> Date: Fri, 1 Jan 2021 12:34:44 +0100 Subject: [PATCH] chore: release notes for v21.01.0 (#1442) --- docs/ascii/flank.jar_-android-run.adoc | 5 ++- .../flank.jar_-firebase-test-android-run.adoc | 6 ++- .../flank.jar_-firebase-test-ios-run.adoc | 22 ++++++---- docs/ascii/flank.jar_-ios-run.adoc | 17 ++++---- flank-scripts/build.gradle.kts | 2 +- .../ci/nexttag/NextReleaseTagGenerator.kt | 3 +- release_notes.md | 41 +++++++++++++++++++ 7 files changed, 75 insertions(+), 21 deletions(-) diff --git a/docs/ascii/flank.jar_-android-run.adoc b/docs/ascii/flank.jar_-android-run.adoc index 63fc2336a0..84489ce6d3 100644 --- a/docs/ascii/flank.jar_-android-run.adoc +++ b/docs/ascii/flank.jar_-android-run.adoc @@ -14,7 +14,7 @@ flank.jar *flank.jar android run* [*-h*] [*--async*] [*--auto-google-login*] [*--disable-results-upload*] [*--disable-sharding*] [*--dry*] - [*--dump-shards*] [*--full-junit-result*] + [*--dump-shards*] [*--fail-fast*] [*--full-junit-result*] [*--ignore-failed-tests*] [*--keep-file-path*] [*--legacy-junit-result*] [*--no-auto-google-login*] [*--no-performance-metrics*] [*--no-record-video*] @@ -135,6 +135,9 @@ Configuration is read from flank.yml *--scenario-numbers*=__[,__...]:: A list of game-loop scenario numbers which will be run as part of the test (default: all scenarios). A maximum of 1024 scenarios may be specified in one test matrix, but the maximum number may also be limited by the overall test --timeout setting. +*--fail-fast*:: + If true, only a single attempt at most will be made to run each execution/shard in the matrix. Flaky test attempts are not affected. Normally, 2 or more attempts are made if a potential infrastructure issue is detected. This feature is for latency sensitive workloads. + *--type*=__:: The type of test to run. TYPE must be one of: instrumentation, robo, xctest, game-loop. diff --git a/docs/ascii/flank.jar_-firebase-test-android-run.adoc b/docs/ascii/flank.jar_-firebase-test-android-run.adoc index eecf2cd19e..2e28b16ec5 100644 --- a/docs/ascii/flank.jar_-firebase-test-android-run.adoc +++ b/docs/ascii/flank.jar_-firebase-test-android-run.adoc @@ -15,7 +15,8 @@ flank.jar firebase test android run* [*-h*] [*--async*] [*--auto-google-login*] [*--disable-results-upload*] [*--disable-sharding*] [*--dry*] - [*--dump-shards*] [*--full-junit-result*] + [*--dump-shards*] [*--fail-fast*] + [*--full-junit-result*] [*--ignore-failed-tests*] [*--keep-file-path*] [*--legacy-junit-result*] [*--no-auto-google-login*] @@ -148,6 +149,9 @@ Configuration is read from flank.yml *--scenario-numbers*=__[,__...]:: A list of game-loop scenario numbers which will be run as part of the test (default: all scenarios). A maximum of 1024 scenarios may be specified in one test matrix, but the maximum number may also be limited by the overall test --timeout setting. +*--fail-fast*:: + If true, only a single attempt at most will be made to run each execution/shard in the matrix. Flaky test attempts are not affected. Normally, 2 or more attempts are made if a potential infrastructure issue is detected. This feature is for latency sensitive workloads. + *--type*=__:: The type of test to run. TYPE must be one of: instrumentation, robo, xctest, game-loop. diff --git a/docs/ascii/flank.jar_-firebase-test-ios-run.adoc b/docs/ascii/flank.jar_-firebase-test-ios-run.adoc index 8c156b1b1f..f7319d05c2 100644 --- a/docs/ascii/flank.jar_-firebase-test-ios-run.adoc +++ b/docs/ascii/flank.jar_-firebase-test-ios-run.adoc @@ -14,9 +14,10 @@ flank.jar *flank.jar firebase test ios run* [*-h*] [*--async*] [*--disable-results-upload*] [*--disable-sharding*] [*--dry*] [*--dump-shards*] - [*--full-junit-result*] [*--ignore-failed-tests*] - [*--keep-file-path*] [*--no-record-video*] - [*--obfuscate*] [*--record-video*] + [*--fail-fast*] [*--full-junit-result*] + [*--ignore-failed-tests*] [*--keep-file-path*] + [*--no-record-video*] [*--obfuscate*] + [*--record-video*] [*--smart-flank-disable-upload*] [*--test-special-entitlements*] [*--use-average-test-time-for-new-tests*] @@ -127,6 +128,9 @@ Configuration is read from flank.yml *--scenario-numbers*=__[,__...]:: A list of game-loop scenario numbers which will be run as part of the test (default: all scenarios). A maximum of 1024 scenarios may be specified in one test matrix, but the maximum number may also be limited by the overall test --timeout setting. +*--fail-fast*:: + If true, only a single attempt at most will be made to run each execution/shard in the matrix. Flaky test attempts are not affected. Normally, 2 or more attempts are made if a potential infrastructure issue is detected. This feature is for latency sensitive workloads. + *--type*=__:: The type of test to run. TYPE must be one of: instrumentation, robo, xctest, game-loop. @@ -184,6 +188,12 @@ Configuration is read from flank.yml *--use-average-test-time-for-new-tests*:: Enable using average time from previous tests duration when using SmartShard and tests did not run before. +*--app*=__:: + The path to the application archive (.ipa file) for game-loop testing. The path may be in the local filesystem or in Google Cloud Storage using gs:// notation. This flag is only valid when --type=game-loop is also set + +*--test*=__:: + The path to the test package (a zip file containing the iOS app and XCTest files). The given path may be in the local filesystem or in Google Cloud Storage using a URL beginning with gs://. Note: any .xctestrun file in this zip file will be ignored if --xctestrun-file is specified. + *--xctestrun-file*=__:: The path to an .xctestrun file that will override any .xctestrun file contained in the --test package. Because the .xctestrun file contains environment variables along with test methods to run and/or ignore, this can be useful for customizing or sharding test suites. The given path may be in the local filesystem or in Google Cloud Storage using a URL beginning with gs://. @@ -198,12 +208,6 @@ Configuration is read from flank.yml + Note: Because this changes the app's identifier, make sure none of the resources in your zip file contain direct references to the test app's bundle id. -*--app*=__:: - The path to the application archive (.ipa file) for game-loop testing. The path may be in the local filesystem or in Google Cloud Storage using gs:// notation. This flag is only valid when --type=game-loop is also set - -*--test*=__:: - The path to the test package (a zip file containing the iOS app and XCTest files). The given path may be in the local filesystem or in Google Cloud Storage using a URL beginning with gs://. Note: any .xctestrun file in this zip file will be ignored if --xctestrun-file is specified. - *--test-targets*=__[,__...]:: A list of one or more test method names to run (default: run all test targets). diff --git a/docs/ascii/flank.jar_-ios-run.adoc b/docs/ascii/flank.jar_-ios-run.adoc index 0905c8d736..18cedeb27d 100644 --- a/docs/ascii/flank.jar_-ios-run.adoc +++ b/docs/ascii/flank.jar_-ios-run.adoc @@ -13,7 +13,7 @@ flank.jar *flank.jar ios run* [*-h*] [*--async*] [*--disable-results-upload*] - [*--disable-sharding*] [*--dry*] [*--dump-shards*] + [*--disable-sharding*] [*--dry*] [*--dump-shards*] [*--fail-fast*] [*--full-junit-result*] [*--ignore-failed-tests*] [*--keep-file-path*] [*--no-record-video*] [*--obfuscate*] [*--record-video*] [*--smart-flank-disable-upload*] @@ -117,6 +117,9 @@ Configuration is read from flank.yml *--scenario-numbers*=__[,__...]:: A list of game-loop scenario numbers which will be run as part of the test (default: all scenarios). A maximum of 1024 scenarios may be specified in one test matrix, but the maximum number may also be limited by the overall test --timeout setting. +*--fail-fast*:: + If true, only a single attempt at most will be made to run each execution/shard in the matrix. Flaky test attempts are not affected. Normally, 2 or more attempts are made if a potential infrastructure issue is detected. This feature is for latency sensitive workloads. + *--type*=__:: The type of test to run. TYPE must be one of: instrumentation, robo, xctest, game-loop. @@ -174,6 +177,12 @@ Configuration is read from flank.yml *--use-average-test-time-for-new-tests*:: Enable using average time from previous tests duration when using SmartShard and tests did not run before. +*--app*=__:: + The path to the application archive (.ipa file) for game-loop testing. The path may be in the local filesystem or in Google Cloud Storage using gs:// notation. This flag is only valid when --type=game-loop is also set + +*--test*=__:: + The path to the test package (a zip file containing the iOS app and XCTest files). The given path may be in the local filesystem or in Google Cloud Storage using a URL beginning with gs://. Note: any .xctestrun file in this zip file will be ignored if --xctestrun-file is specified. + *--xctestrun-file*=__:: The path to an .xctestrun file that will override any .xctestrun file contained in the --test package. Because the .xctestrun file contains environment variables along with test methods to run and/or ignore, this can be useful for customizing or sharding test suites. The given path may be in the local filesystem or in Google Cloud Storage using a URL beginning with gs://. @@ -188,12 +197,6 @@ Configuration is read from flank.yml + Note: Because this changes the app's identifier, make sure none of the resources in your zip file contain direct references to the test app's bundle id. -*--app*=__:: - The path to the application archive (.ipa file) for game-loop testing. The path may be in the local filesystem or in Google Cloud Storage using gs:// notation. This flag is only valid when --type=game-loop is also set - -*--test*=__:: - The path to the test package (a zip file containing the iOS app and XCTest files). The given path may be in the local filesystem or in Google Cloud Storage using a URL beginning with gs://. Note: any .xctestrun file in this zip file will be ignored if --xctestrun-file is specified. - *--test-targets*=__[,__...]:: A list of one or more test method names to run (default: run all test targets). diff --git a/flank-scripts/build.gradle.kts b/flank-scripts/build.gradle.kts index 4f1849115b..65f20877d2 100644 --- a/flank-scripts/build.gradle.kts +++ b/flank-scripts/build.gradle.kts @@ -28,7 +28,7 @@ shadowJar.apply { } } // .. -version = "1.3.0" +version = "1.3.1" group = "com.github.flank" application { diff --git a/flank-scripts/src/main/kotlin/flank/scripts/ci/nexttag/NextReleaseTagGenerator.kt b/flank-scripts/src/main/kotlin/flank/scripts/ci/nexttag/NextReleaseTagGenerator.kt index 93c04ebb01..230f03ca21 100644 --- a/flank-scripts/src/main/kotlin/flank/scripts/ci/nexttag/NextReleaseTagGenerator.kt +++ b/flank-scripts/src/main/kotlin/flank/scripts/ci/nexttag/NextReleaseTagGenerator.kt @@ -1,7 +1,6 @@ package flank.scripts.ci.nexttag import java.time.LocalDate -import java.time.Year import java.time.format.DateTimeFormatter fun generateNextReleaseTag(previousReleaseTag: String): String { @@ -14,7 +13,7 @@ fun generateNextReleaseTag(previousReleaseTag: String): String { } private fun isNextReleaseInCurrentMonth(year: String, month: String) = - DateTimeFormatter.ofPattern("yy").format(Year.now()) == year && + DateTimeFormatter.ofPattern("yy").format(LocalDate.now()) == year && DateTimeFormatter.ofPattern("MM").format(LocalDate.now()) == month private fun currentMonthFirstTag() = "v${DateTimeFormatter.ofPattern("yy.MM").format(LocalDate.now())}.0" diff --git a/release_notes.md b/release_notes.md index fd77620b31..253f9e0464 100644 --- a/release_notes.md +++ b/release_notes.md @@ -1,3 +1,44 @@ +## v21.01.0 +### CI Changes +- [#1439](https://github.com/Flank/flank/pull/1439) Remove sanity checks and change full suite IT OS to macOS ([piotradamczyk5](https://github.com/piotradamczyk5)) +- [#1433](https://github.com/Flank/flank/pull/1433) Change IT workflow comment ([pawelpasterz](https://github.com/pawelpasterz)) +- [#1424](https://github.com/Flank/flank/pull/1424) Auto update firebase api client ([piotradamczyk5](https://github.com/piotradamczyk5)) +- [#1412](https://github.com/Flank/flank/pull/1412) 1411 implement version ([pawelpasterz](https://github.com/pawelpasterz)) +- [#1409](https://github.com/Flank/flank/pull/1409) Trigger integration tests from pr comment ([adamfilipow92](https://github.com/adamfilipow92), [piotradamczyk5](https://github.com/piotradamczyk5)) +- [#1385](https://github.com/Flank/flank/pull/1385) Get rid of binaries submodule ([jan-gogo](https://github.com/jan-gogo), [piotradamczyk5](https://github.com/piotradamczyk5)) +- [#1400](https://github.com/Flank/flank/pull/1400) Change gradle command ([pawelpasterz](https://github.com/pawelpasterz)) +- [#1398](https://github.com/Flank/flank/pull/1398) Add create google account config step ([pawelpasterz](https://github.com/pawelpasterz)) +- [#1353](https://github.com/Flank/flank/pull/1353) Implement workflow for full suite IT (cron + manual) ([pawelpasterz](https://github.com/pawelpasterz), [piotradamczyk5](https://github.com/piotradamczyk5)) +- [#1378](https://github.com/Flank/flank/pull/1378) Update GH API ([pawelpasterz](https://github.com/pawelpasterz)) +### Features +- [#1436](https://github.com/Flank/flank/pull/1436) 1422 autodetect new gcloud features ([pawelpasterz](https://github.com/pawelpasterz)) +- [#1429](https://github.com/Flank/flank/pull/1429) Change uploading file output ([adamfilipow92](https://github.com/adamfilipow92), [piotradamczyk5](https://github.com/piotradamczyk5)) +- [#1426](https://github.com/Flank/flank/pull/1426) Support for iOS gameloop tests ([Sloox](https://github.com/Sloox), [jan-gogo](https://github.com/jan-gogo), [adamfilipow92](https://github.com/adamfilipow92)) +- [#1421](https://github.com/Flank/flank/pull/1421) Default project Id - use GOOGLE_APPLICATION_CREDENTIALS first, then GOOGLE_CLOUD_PROJECT ([pawelpasterz](https://github.com/pawelpasterz)) +- [#1404](https://github.com/Flank/flank/pull/1404) Add basic gameloop IT for Android ([Sloox](https://github.com/Sloox), [adamfilipow92](https://github.com/adamfilipow92)) +- [#1393](https://github.com/Flank/flank/pull/1393) Upload matrix ids ([Sloox](https://github.com/Sloox), [jan-gogo](https://github.com/jan-gogo)) +- [#1362](https://github.com/Flank/flank/pull/1362) Add logic to verify xml results ([adamfilipow92](https://github.com/adamfilipow92), [piotradamczyk5](https://github.com/piotradamczyk5)) +- [#1321](https://github.com/Flank/flank/pull/1321) IOS support for testplans ([axelzuziak-gogo](https://github.com/axelzuziak-gogo), [jan-gogo](https://github.com/jan-gogo)) +- [#1375](https://github.com/Flank/flank/pull/1375) Add new output style to print smaller output ([adamfilipow92](https://github.com/adamfilipow92), [piotradamczyk5](https://github.com/piotradamczyk5)) +- [#1370](https://github.com/Flank/flank/pull/1370) Add support for "fail fast" feature on FTL ([RainNapper](https://github.com/RainNapper)) +### Refactor +- [#1432](https://github.com/Flank/flank/pull/1432) Small changes in flank-scripts module ([pawelpasterz](https://github.com/pawelpasterz)) +- [#1418](https://github.com/Flank/flank/pull/1418) Create module with shared utilities ([piotradamczyk5](https://github.com/piotradamczyk5)) +- [#1417](https://github.com/Flank/flank/pull/1417) Update firebase_apis module ([pawelpasterz](https://github.com/pawelpasterz)) +### Bug Fixes +- [#1414](https://github.com/Flank/flank/pull/1414) Add attempts to performance test ([jan-gogo](https://github.com/jan-gogo)) +- [#1410](https://github.com/Flank/flank/pull/1410) Fix integration tests on CI. ([adamfilipow92](https://github.com/adamfilipow92), [piotradamczyk5](https://github.com/piotradamczyk5)) +- [#1413](https://github.com/Flank/flank/pull/1413) Fix calculate shards test ([pawelpasterz](https://github.com/pawelpasterz), [jan-gogo](https://github.com/jan-gogo)) +- [#1405](https://github.com/Flank/flank/pull/1405) CancelCommandRuns should works independent ([Sloox](https://github.com/Sloox), [adamfilipow92](https://github.com/adamfilipow92)) +- [#1395](https://github.com/Flank/flank/pull/1395) Print args when iOS tests are not found ([jan-gogo](https://github.com/jan-gogo), [adamfilipow92](https://github.com/adamfilipow92)) +- [#1387](https://github.com/Flank/flank/pull/1387) Fix NPE ([pawelpasterz](https://github.com/pawelpasterz), [jan-gogo](https://github.com/jan-gogo)) +- [#1372](https://github.com/Flank/flank/pull/1372) Fix Flank Scripts version verification tasks ([piotradamczyk5](https://github.com/piotradamczyk5)) +### Documentation +- [#1401](https://github.com/Flank/flank/pull/1401) Update orchestrator version in docs ([adamfilipow92](https://github.com/adamfilipow92)) +- [#1380](https://github.com/Flank/flank/pull/1380) Added documentation about handling #1374 issue ([piotradamczyk5](https://github.com/piotradamczyk5)) +### Tests update +- [#1316](https://github.com/Flank/flank/pull/1316) [IT] Add test cases ([pawelpasterz](https://github.com/pawelpasterz), [adamfilipow92](https://github.com/adamfilipow92)) + ## v20.12.0 ### Bug Fixes - [#1366](https://github.com/Flank/flank/pull/1366) Generation of release notes ([piotradamczyk5](https://github.com/piotradamczyk5))