feat: Add app name to the test result

[pawelpasterz]

Fixes #1674

PR adds a new column (APP NAME) to the result table and updates the outputReport.json structure with an apk file name.

Test Plan

How do we know the code works?

1. Launch test run with flank with output-report: json
2. When the run is finished you should see the additional column in the result table with the header APP NAME
3. There should be apk's file name used in the test
4. Open outputReport.json
5. Check if test_results structure is similar to

   "test_results": {
     "matrix-afvw8zv215owa": {
       "app": "app-debug.apk",
       "test-axises": [
   

Checklist

• Unit tested

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[github-actions]

Timestamp: 2021-03-22 08:14:09
Buildscan url for ubuntu-workflow run 675313277
https://gradle.com/s/xhu2pgpd4j7m4

[pawelpasterz]

@flank-it

[github-actions]

Integration tests failed ❌
Windows status failure - Build scan:
MacOS status failure - Build scan: https://gradle.com/s/m2ursjv6ylriy
Linux status failure - Build scan: https://gradle.com/s/rtnpwfpycc4iu
Workflow run https://github.com/Flank/flank/actions/runs/662287206

[Sloox]

Strange formatting to have a random endline character?
Personal preference or auto corrected?

[pawelpasterz]

formatKotlin did that, I'm not happy with it neither but who am I to challenge it :D
Being serious, I'll think of another formatting that is inline with kotlin format task

[Sloox]

interesting resolution, nice and neat but make sure we dont have any order of preference issues here :)

[pawelpasterz]

The order doesn't matter in that case since only one will have a value that is non-null. It comes from TestSpecification object, and that one is a composition of specs:

• androidInstrumentationTest
• androidTestLoop
• androidRoboTest
• iosXcTest
• iosTestLoop

Since we are allowed to run only one type of test per matrix only one from above is not null. All android specs have appApk, iOS xc -- testsZip and iOS loop -- appIpa. Let me know if that answers your question :)

[pawelpasterz]

@flank-it

[github-actions]

Integration tests succeed for all OSes ✅
Windows Build scan:
MacOS Build scan: https://gradle.com/s/72a6abrq2uux6
Linux Build scan: https://gradle.com/s/or7e6uqptxr7y
Workflow run https://github.com/Flank/flank/actions/runs/664616860

[pawelpasterz]

@Mergifyio rebase

[mergify]

Command rebase: success

Branch already up to date

[piotradamczyk5]

Results table:

┌─────────┬──────────────────────┬───────────────┬────────────────────────────┬────────────────────────────────┐
│ OUTCOME │      MATRIX ID       │   APP NAME    │      TEST AXIS VALUE       │          TEST DETAILS          │
├─────────┼──────────────────────┼───────────────┼────────────────────────────┼────────────────────────────────┤
│ success │ matrix-3qnwchlja84yr │ app-debug.apk │ NexusLowRes-28-en-portrait │ 1 test cases passed, 1 skipped │
└─────────┴──────────────────────┴───────────────┴────────────────────────────┴────────────────────────────────┘

Output report:

"test_results": {
    "matrix-3qnwchlja84yr": {
      "app": "app-debug.apk",
      "test-axises": [
        {
          "device": "NexusLowRes-28-en-portrait",
          "outcome": "success",
          "details": "1 test cases passed, 1 skipped",
          "testSuiteOverview": {
            "total": 2,
            "errors": 0,
            "failures": 0,
            "flakes": 0,
            "skipped": 1,
            "elapsedTime": 3.36,
            "overheadTime": 0.0
          }
        }
      ]
    }
  },

Code review ✅