- #1366 Generation of release notes (piotradamczyk5)
- #1357 Fix release Action (Sloox, piotradamczyk5)
- #1333 Publishing Flank snapshot (Sloox, piotradamczyk5)
- #1338 Fix printing supported versions id (adamfilipow92)
- #1329 Uploading performance metrics for multiple matrices (adamfilipow92, piotradamczyk5)
- #1326 Downloading performance Metrics (adamfilipow92, piotradamczyk5)
- #1323 Nested subdirectories in results-dir (piotradamczyk5)
- #1307 Integration tests on windows (pawelpasterz, adamfilipow92)
- #1301 Handling broken token issues (piotradamczyk5)
- #1298 Fix incorrect printed shards number (pawelpasterz)
- #1295 Print formatted message instead of throwing NPE (pawelpasterz)
- #1283 Integration tests on windows (adamfilipow92, piotradamczyk5)
- #1275 Fix getting PR number on pull_request_review (piotradamczyk5)
- #1272 Fix integration test counter and trigger (piotradamczyk5)
- #1219 #842 support for test targets flag in multiple testbundles xctest (axelzuziak-gogo, jan-gogo)
- #1261 Add additional index to matrix when multiple test runs (pawelpasterz, adamfilipow92)
- #1250 Fix running flank workflows on Windows (piotradamczyk5)
- #1247 Flank web documentation link (piotradamczyk5)
- #1243 Disable cache for :test_runner:shadowJar task (pawelpasterz)
- #1233 Fix withClassName filter (pawelpasterz)
- #1221 Always dump shards (pawelpasterz, adamfilipow92)
- #1236 Empty branch name in workflows (jan-gogo)
- #1232 Test_runner:resolveArtifacts task (jan-gogo)
- #1225 Release job missing env ()
- #1188 Revert "feat: Dump shards and upload on every run (#1171)" (pawelpasterz)
- #1177 Fix flank-scripts test (pawelpasterz)
- #1167 Flank action (Sloox)
- #1161 Release notes slack tweaks (Sloox)
- #1164 Dependabot security issues (Sloox)
- #1350 Ios test artifacts cleanup #1345 (axelzuziak-gogo, jan-gogo)
- #1339 Simplify beforeRunTests return singature (jan-gogo)
- #1281 Prepare flank codebase for supporting iOS testplans (axelzuziak-gogo, jan-gogo)
- #1246 Rewrite scripts to Kotlin (adamfilipow92, piotradamczyk5)
- #1361 Publishing and downloading flank scripts (piotradamczyk5)
- #1352 Make flank release more automatically (piotradamczyk5)
- #1344 Add directory filter for IT (pawelpasterz)
- #1310 Copy properties from issue to pull request (adamfilipow92, piotradamczyk5)
- #1320 Added pre-commit-hook (Sloox)
- #1286 Update existing bot comment (pawelpasterz)
- #1269 Add build scan for ubuntu workflow (pawelpasterz)
- #1265 Refresh integration tests (piotradamczyk5)
- #1263 Github Action optimizations (piotradamczyk5)
- #1253 Use Github app token for creating release notes (piotradamczyk5)
- #1248 Update deprecated options ::set-env and ::add-path (piotradamczyk5)
- #1240 Add current date to dependecy PR (pawelpasterz)
- #1148 Add scripts for test artifacts managment (jan-gogo, piotradamczyk5)
- #1210 Enable RC versions for gradle (dependency update task) (pawelpasterz)
- #1158 Fix test artifacts url (jan-gogo)
- #1152 Added WSL workflow (Sloox, piotradamczyk5)
- #1355 Add documentation about code review by step (jan-gogo)
- #1311 Update documentations & Add Headings (Sloox)
- #1222 Add flank website code (jan-gogo)
- #1229 Update documentation (pawelpasterz)
- #1215 Added documentation about missing options (piotradamczyk5)
- #1190 Added documentation about Cucumber support (piotradamczyk5)
- #1181 Investigation incorrect outcome for flaky tests (pawelpasterz, adamfilipow92)
- #1173 Add FAQ section about sync problems (piotradamczyk5)
- #1131 Investigate flank options (jan-gogo, adamfilipow92)
- #1325 IOS GameLoop addition (Sloox, axelzuziak-gogo)
- #1299 Add test targets for shard (Sloox, pawelpasterz)
- #1292 Added Performance Metrics for Android (piotradamczyk5)
- #1289 Add ios app for gameloop (Sloox)
- #1291 Add --test-special-entitlements for iOS (piotradamczyk5)
- #1266 IOS add directories-to-pull (adamfilipow92, piotradamczyk5)
- #1287 Ios scenario numbers (Sloox)
- #1284 Add Type to iOS (piotradamczyk5)
- #1262 Gameloop addition (Sloox)
- #1273 Add Additional Ipas option to iOS (piotradamczyk5)
- #1267 Add Other Files for iOS (adamfilipow92, piotradamczyk5)
- #1255 Obb file addition (Sloox)
- #1256 Update test shard limits (adamfilipow92)
- #1244 Added scenario-numbers (Sloox)
- #1238 Scenario Labels (Sloox)
- #1230 Add type (Sloox)
- #1186 Grant permissions addition (Sloox)
- #1214 Added printing missing options (piotradamczyk5)
- #1175 Android integration tests run on non wsl windows (Sloox, adamfilipow92)
- #1171 Dump shards and upload on every run (adamfilipow92)
- #1163 Integration tests run when pr is approved and not draft (adamfilipow92, piotradamczyk5)
- #1159 Improve error message on matrix failures (pawelpasterz, adamfilipow92)
- #1133 Auto update dependencies (piotradamczyk5)
- #1300 Add flank multi test targets example (jan-gogo)
- #1274 Disable iOS test for Windows (piotradamczyk5)
- #1187 Remove verification test for edge cases of incorrect outcome in flaky tests (adamfilipow92)
- #1174 Added cucumber sample app for testing #1118 (piotradamczyk5)
- #1151 Update flank-scripts serialization (piotradamczyk5)
- #1149 Flank-scripts next release tag generation (piotradamczyk5)
- #1138 Fix cost report (pawelpasterz)
- #1137 Running Code coverage job (piotradamczyk5)
- #1130 Bump action versioning (Sloox)
- #1124 Reflect gclouds outcome for robo tests (pawelpasterz)
- #1121 Dockerfile fix for slack message send (Sloox)
- #1116 Fixes release workflows (Sloox)
- #1141 Add Java 15 compile support (piotradamczyk5)
- #1126 Allow Flank using different environment variables between test-apks (adamfilipow92)
- #1078 Add proguard initial step into Flank configuration (Sloox)
- #1125 Readded shadowjar minimization (Sloox)
- #1109 Integration tests for Flank (adamfilipow92)
- #1139 Optimization docs addition (Sloox)
- #1136 Update documentation related with test-targets-always-run feature (pawelpasterz)
- #1114 Add Windows workflow (Sloox)
- #1095 Add new bash scripts for test artifacts generation (axelzuziak-gogo, jan-gogo)
- #1113 Use single settings.gradle for flank projects (jan-gogo, piotradamczyk5)
- #1111 Check if gcs path exist before run (piotradamczyk5)
- #1110 Finialize the slack sending integration with actions (Sloox)
- #1108 Support for robo tests without robo script (pawelpasterz)
- #1104 Send slack message for releases (Sloox)
- #1097 Print the matrices web link at the end of a run (adamfilipow92)
- #1107 Running iOS test on Linux (piotradamczyk5)
- #1059 Shards and Tests count do not match (pawelpasterz, adamfilipow92)
- #1067 Matrix path not found in json (adamfilipow92, piotradamczyk5)
- #1100 Fix gcs files validation (pawelpasterz)
- #1093 Flank release bot formatting (Sloox)
- #1091 Add missing authors to release notes (adamfilipow92)
- #1087 Fix release action (adamfilipow92)
- #1080 Flaky Flank-scripts test (Sloox)
- #1079 Fix slack notification in slack (Sloox)
- #1054 Firebase refresh fails when test zip file doesn't exist (#1052) (axelzuziak-gogo, jan-gogo)
- #1070 Avoid fast fail while creating a bucket (adamfilipow92)
- #1066 Bad validation of
local-result-dir
by Doctor (piotradamczyk5)
- #1056 Added validation of smart-flank-gcs-path to not override different junit results (adamfilipow92)
- #1040 Add test axis value column to summary table (jan-gogo)
- #1043 Add printing messages for MatrixCanceledError and InfrastructureError (pawelpasterz)
- #1042 Added option for default test time and average time for smart shard (piotradamczyk5)
- #1055 Added missing test t cover duplicated tests issue (piotradamczyk5)
- #1041 Invoking flank yml on gcloud cli (adamfilipow92)
- #1053 Avoid releasing on document changes only (Sloox)
- #1048 Flank finding and running duplicate tests (MatthewTPage)
- #1051 PR check from fork (piotradamczyk5)
- #1044 Master not compiling (Sloox)
- #994 Size annotations support (pgreze)
- #1009 Avoid multiple identical lines printing (adamfilipow92)
- #1031 Disable publishing snapshot to Github Packages (piotradamczyk5)
- #1022 Publish to GitHub packages (piotradamczyk5)
- #1027 OtherNativeCrash NPE (jan-gogo)
- #1023 Release notes message at GitHub release (piotradamczyk5)
- #1025 Update kotlin version to 1.4 (pawelpasterz)
- #1020 Make release notes more organized (piotradamczyk5)
- #1013 Generating docs before release (piotradamczyk5)
- #1012 Add ip-blocks command to test-environment command (pawelpasterz)
- #999 Implement ip-blocks list command (pawelpasterz)
- #996 Auto generate release notes for next release (piotradamczyk5)
- #995 Implement command
models describe
for ios/android (adamfilipow92) - #991 Validate orientation and fail fast (Sloox)
- #969 Add locales description command for ios and android (adamfilipow92)
- #992 Update google api (jan-gogo)
- #988 Add versions description command for ios and android (adamfilipow92)
- #1018 Fetch artifacts async (pawelpasterz)
- #919 Rate limit exceeded (pawelpasterz, jan-gogo)
- #1005 Generation of release notes (piotradamczyk5)
- #1007 Failing tests (piotradamczyk5)
- #990 Exclusion of @Suppress test (piotradamczyk5)
- #1015 Update mergify configuration (piotradamczyk5)
- #1011 Generate release notes for Github release description (piotradamczyk5)
- #989 Check if valid title is used in PR (piotradamczyk5)
- #978 Firebaseopensource.com addition (sloox)
- #937 Improve error message on iOS when test or xctestrun-file not found (sloox)
- #952 Fix version printing on Flank release (sloox)
- #950 Fix crash when --legacy-junit-result set. (adamfilipow92)
- #948 Increment retry tries and change sync tag for jfrogSync. (piotradamczyk5)
- #946 Added tests for flank scripts. (piotradamczyk5)
- #935 Process junit xml even when full junit is not enabled. (kozaxinan)
- #962 Make table text left aligned. (pawelpasterz)
- #965 Fast fail when full-junit-result and legacy-junit-result. (adamfilipow92)
- #970 Fixing Flood of snapshot releases. (piotradamczyk5)
- #972 Fix printing release version. (piotradamczyk5)
- #964 Implement network-profiles describe feature. (pawelpasterz)
- #890 Convert bitrise ubuntu workflow into GitHub actions. (piotradamczyk5)
- #876 Added option to print Android available devices to test against. (piotradamczyk5)
- #895 Added option to print iOS available devices to test against. (piotradamczyk5)
- #894 Added option to print Android available versions to test against. (piotradamczyk5)
- #897 Added option to print iOS available versions to test against. (piotradamczyk5)
- #901 Added option to print Android and iOS available test-environment. (piotradamczyk5)
- #904 Added option to print provided software. (piotradamczyk5)
- #906 Added option to print network profiles. (adamfilipow92)
- #908 Added option to print iOS available locales to test against. (piotradamczyk5)
- #906 Added option to print iOS and Android screen orientations. (adamfilipow92)
- #907 Added option to print Android available locales to test against. (piotradamczyk5)
- #913 Add Gradle Enterprise API example. (pawelpasterz)
- #916 Test artifacts monorepo. (jan-gogo)
- #910 Migrate Bitrise release workflow into GitHub actions. (piotradamczyk5)
- #915 Update virtual devices sharding limit. (adamfilipow92)
- #920 Improve .yml validation on
doctor
command. (piotradamczyk5) - #934 Delete incorrect flank snapshot labels. (piotradamczyk5)
- #926 Flank should reflect gcloud exit codes. (adamfilipow92)
- #917 Fix an incorrect outcome. (pawelpasterz)
- #939 Run *list commands when flank.yml not found should display right output. (adamfilipow92)
- #857 Added multimodule setup for test app. (piotradamczyk5)
- #837 Added obfuscate option to dump shards. (piotradamczyk5)
- #868 Restored weblinks to all test results, not just failures. (rainnapper)
- #828 Store test results in gcloud bucket. (adamfilipow92)
- #865 Flank needs to respect the timeout value as that's a cap for billing purposes. (adamfilipow92, pawelpasterz)
- #866 Fix printing all matrix links. (piotradamczyk5)
- #862 Added printing outcome details. (piotradamczyk5, jan-gogo)
- #876 Added --directories-to-pull validation and avoid making request with empty toolStepResult. (piotradamczyk5)
- #875 Enhance permission denied exception logs. (adamfilipow92, pawelpasterz)
- #853 Store @Ignore tests in the JUnit XML without sending ignored tests to FTL. (piotradamczyk5, adamfilipow92)
- #853 Handle duplicated apk names. (jan-gogo)
- #840 Fix parametrized tests. (jan-gogo, adamfilipow92, pawelpasterz, piotradamczyk5)
- #831 Refactor config entities and arguments. (jan-gogo)
- #817 Add AndroidTestContext as base data for dump shards & test execution. (jan-gogo)
- #801 Omit missing app apk if additional-app-test-apks specified. (jan-gogo)
- #784 Add output-style option. (jan-gogo)
- #779 Print retries & display additional info. (jan-gogo)
- #793 Better error message on file not found. (adamfilipow92)
- #808 Fixed dry run crashes. (piotradamczyk5)
- #807 Fix Bugsnag being initialized during tests. (piotradamczyk5)
- #805 Fix overlapping results. (pawelpasterz)
- #812 Convert bitrise macOS workflow to github action. (piotradamczyk5)
- #799 Refactor Shared object by splitting it into smaller functions. (piotradamczyk5)
- #798 Remove failure nodes from tests that passed on retry so that Jenkins JUnit plugin marks them as successful. (adamfilipow92)
- #822 Allow runtime test discovery when sharding is disabled by not setting test-targets. This unblocks cucumber testing. (adamfilipow92)
- #819 Display matrix results in a table format. (piotradamczyk5)
- #825 Automatically convert -1 in maximum-test-shards to the maximum shard amount. (adamfilipow92)
- #833 More error messages improvements. (piotradamczyk5)
- #826 Omit
environment-variables
for robo test executions instead of throwing exception. (adamfilipow92)
- #781 Remove local exists check on cloud results-dir. Fixes crash when results-dir is set by the user. (adamfilipow92)
- #656 Improve error message reporting. (adamfilipow92)
- #783 Use legacy results for iOS by default. (pawelpasterz)
- #794 Enhance
--dump-shards
to dump shards from all test apks (bootstraponline, jan-gogo)
- #775 Fix exception thrown for null start time. (pawelpasterz)
- #764 Fix crash on parse some control chars. (adamfilipow92)
- #772 Fail fast when results-dir is incorrect. (jan-gogo)
- #757 Reduce memory usage by using Reader and Writer instead of ByteArrays. (jan-gogo)
- #763 Use "localhost" as default for hostname to fix backward compatibility. (jan-gogo)
- #757 Print version and revision before each command. (jan-gogo)
- #759 Add shard name for uploaded xctestrun files. (pawelpasterz)
- #755 Remove ascii doc generated section header. (jan-gogo)
- #731 Refactor jUnit HTML report. (Writhe)
- #754 Sync README.md flank.yml flank.ios.yml. (jan-gogo)
- #746 Ignore apk with filtered out tests instead of failing. (pawelpasterz)
- #741 Allow APKs with zero tests. (fondesa)
- #737 Generate ascii doc. (jan-gogo)
- #720 Update group id from
flank
tocom.github.flank
(bootstraponline) - #714 Add support for num-uniform-shards option. (jan-gogo)
- #712 Add keep file path for ios. (pawelpasterz)
- #711 Remove hardcoded height. (pawelpasterz)
- #708 Add ignore failed tests option to Flank. (pawelpasterz)
- #704 Add robo for robo-directives & robo-script options. (jan-gogo)
- #704 Fix shards calculation when there are ignored tests and shardTime is -1. (jan-gogo)
- #692 Add support for other-files option. (jan-gogo)
- #695 Add support for additional-apks option. (jan-gogo)
- #683 Print web link. (pawelpasterz)
- #692 Add support for network-profiles list command & --network-profile option. (jan-gogo)
- #689 Add support for client-details option. (jan-gogo)
- #687 Debug message printed after every command. (pawelpasterz)
- #684 Add overhead time to junit test case report. (jan-gogo)
- #666 Use API instead of XML for result parsing for android. (jan-gogo)
- #678 Skip Bugsnag initialization if user disabled gcloud analytics. (pawelpasterz)
- #672 Flank timeout feature. (pawelpasterz)
- #657 Fix execution hangs. (pawelpasterz)
- #654 Fix test filters when using both notPackage and notClass. (jan-gogo)
- #648 Include @Ignore JUnit tests in JUnit XML. (pawelpasterz)
- #646 Adopt kotlin-logging as a logging framework. (jan-gogo)
- #644 Use high performance options by default. Video, login, and perf metrics are now disabled by default. (pawelpasterz)
- #643 Add --dry option to android run & ios run. (jan-gogo)
- #642 Flank doctor should exit 1 on validation issues. (pawelpasterz)
- #641 Move all deps to Deps.kt (doodla)
- #640 Update gradle to 6.2.1 and JaCoCo to 0.8.5. (doodla)
- #639 Cache all uploads and downloads to GCS. (Kurt-Bonatz)
- #635 Default to 2m time estimate for unknown tests when sharding. (RainNapper)
- #621 Adopt server side sharding. Max shard count is 50. (bootstraponline)
- #612 Print HtmlErrorReport location. (bootstraponline)
- #615 Add
--keep-file-path
for Android when downloading assets from Google Cloud Storage. (tahirhajizada)
- #608 Use MatrixRollupOutcome to set exit code value. (bootstraponline)
- #595 Rename
flaky-test-attempts
tonum-flaky-test-attempts
. Renamerepeat-tests
tonum-test-runs
. (bootstraponline) - #605 Improve exit code logging. Use matrix outcome to set exit code. (bootstraponline)
- #597 Support parsing testLabExecutionId. (yogurtearl)
- #599 Disable FAIL_ON_UNKNOWN_PROPERTIES for forward compatibility. (narenkmanoharan)
- #589 Fix java.lang.NumberFormatException: empty String. (vfadc)
- #587 Optimize polling. (bootstraponline)
- #586 Poll devices in parallel. Fixes performance regression in Flank v7. (bootstraponline)
- #574 Improve test shard error reporting. Update device catalog to use projectId. (bootstraponline)
- #582 Fix iOS exit code when using flaky-test-attempts. Don't print environment-variables to stdout for security. (bootstraponline)
- #584 Poll all test executions instead of only the first per matrix. (bootstraponline)
- #585 Fix bug in smart flank when sharding tests that run in 0 seconds. (bootstraponline)
- #567 Fix
--app
&--test
on Android CLI. (bootstraponline) - #571 Add
flank ios run --dump-shards
andflank android run --dump-shards
for debugging (bootstraponline) - #572 Fix exit code reporting when using
flaky-test-attempts
(bootstraponline)
- #566 Fix
--test
&--xctestrun-file
on iOS CLI. (bootstraponline)
- #563 Fix CLI support for iOS. (bootstraponline)
- #542 Add
additional-app-test-apks
to include multiple app/test apk pairs in a single run. (bootstraponline)
- #550 Update dex-test-parser to fix Inherited crash. (bootstraponline)
- #541 Rename
--test-shards
CLI flag to--max-test-shards
. Add--smart-flank-gcs-path
CLI flag. (bootstraponline) - #544 Fix empty test targets crash. (bootstraponline)
- #548 Fix flank cancel. Print os name when sharding iOS tests. (bootstraponline)
- #549 Update to llvm 8.0.0 and Swift 5.0.1. (bootstraponline)
- #537 Add
smart-flank-disable-upload
yml option to prevent new results from overriding previous results. (elihart)
- #538 Update
dextestparser
. Fixes APK parsing crash. (bootstraponline) - #536 Always calculate exit code from matrix status instead of JUnit XML. (bootstraponline)
- #534 Fix updateMatrixFile not saving
matrix_ids.json
. (bootstraponline)
- #506 Add bugsnag reporting to detect Flank crashes. (bootstraponline)
- #507 Improve error message when credentials fail to load, folder doesn't exist, and on bucket creation failure. Properly pass through user credential when checking the storage bucket. (bootstraponline)
- #514 Rename
testShards
tomaxTestShards
(miguelslemos) - #518 Add deprecation warnings when old key names are used.
flank android doctor --fix
will auto fix the YAML file. (bootstraponline) - #519 Rename
maxTestShards
tomax-test-shards
,shardTime
toshard-time
,repeatTests
torepeat-tests
,smartFlankGcsPath
tosmart-flank-gcs-path
,disableSharding
todisable-sharding
. Movedproject
fromgcloud
toflank
(bootstraponline) - #523 Add
--local-result-dir
to make it easy to find the test result at a fixed path. (bootstraponline) - #524 Fix iOS test sharding when there's a space in the path (bootstraponline)
- #522 Correctly report test results and exit code when using
flaky-test-attempts
(bootstraponline) - #530 Save files downloaded with
files-to-download
to device root folder. Fix web links in HTML report. (bootstraponline) - #533 Update matrix_ids.json on every matrix change. Useful for debugging runs that crashed. (jschear)
- #505 Fix
flank auth login
by usinggoogle-auth-library-java
. (bootstraponline) - #501 Fix nullability check in SavedMatrix. (bootstraponline)
- #493 Fix getDefaultBucket timeout. (bootstraponline)
- #436 Print how accurate test times are when using smart flank (Macarse)
- #462 Always find all iOS tests when sharding. (bootstraponline)
- #471 Add dynamic bucket counts for Smart Flank via
shardTime
(Macarse) - #473 Add
disableSharding
flag (Macarse) - #476 Retry matrix creation when FTL API errors. (bootstraponline)
- #477 Fix large number merging on iOS (bootstraponline)
- #478 Add iOS regular expression filtering for
test-targets
(Macarse) - #479 Improve invalid iOS regex error message (bootstraponline)
- #481 Add iOS support for locale and orientation (Macarse)
- #482 Document Android code coverage with orchestrator on FTL (bootstraponline)
- #485 Replace : in folder name to improve Windows support. (Sunil-plsr)
- #487 Document all Flank YAML properties in README (bootstraponline)
- Fix create Gcs bucket #444
- Add
files-to-download
to Android and iOS. Specify a list of regular expressions to download files from the Google Cloud Storage bucket. #441 - Add
flank auth login
to authorize with a user account instead of a service account. #446 - Add
flaky-test-attempts
support on Android and iOS which automatically retries failed tests. #454
- Add support for parameterized iOS tests when shard count is 1. #435
- Add support for
SkipTestIdentifiers
when parsing iOS.xctestrun
files. #435
app
,test
, andxctestrun-file
now support~
, environment variables, and globs (*
,**
) when resolving paths. #386- Add CLI support for
flank android run
andflank ios run
- Add experimental
smartFlankGcsPath
to shard iOS and Android tests by time using historical run data. The amount of shards used is set bytestShards
. #385 - Fix parsing empty testcase #402
- Add progress bar when uploading files. #403
- iOS iPhone 8 default device has been updated from 11.2 to 12.0.
- Add
flank cancel
command. Seeflank cancel --help
- Add
flank refresh
command. Seeflank refresh --help
- Automatically detect projectId from service account credential
- Always generate aggregated JUnit XML reports
- Update HTML report to use create-react-app v2.1.0
- Add support for Xcode 10.1
- Improve test parsing to detect Swift tests that throw errors
- Update Flank exit codes, see readme for details.
- Link to matrix root when testing on multiple devices