feat: Add new output style to print smaller output #1375

adamfilipow92 · 2020-12-04T16:48:02Z

Test Plan

How do we know the code works?

When using single, verbose, multi output style, the output should not change.
Set output style to compact (I'm not sure it's the best name for that, feel free to suggest other)
In this output style we should print: yaml, count of test and shards, link to firebase and matrix web links, matrix and cost reports, table with results

Output example


WARNING: disable-sharding enabled with max-test-shards = 1, Flank will ignore max-test-shard and disable sharding.
AndroidArgs
    gcloud:
      results-bucket: test-lab-v9cn46bb990nx-kz69ymd4nm9aq
      results-dir: 2020-12-04_16-34-33.963626_WvMf
      record-video: false
      timeout: 15m
      async: false
      client-details:
      network-profile: null
      results-history-name: null
      # Android gcloud
      app: /Users/adamfilipowicz/Repos/flank/test_runner/src/test/kotlin/ftl/fixtures/tmp/apk/app-debug.apk
      test: /Users/adamfilipowicz/Repos/flank/test_runner/src/test/kotlin/ftl/fixtures/tmp/apk/app-single-success-debug-androidTest.apk
      additional-apks:
      auto-google-login: false
      use-orchestrator: true
      directories-to-pull:
      grant-permissions: all
      type: null
      other-files:
      scenario-numbers:
      scenario-labels:
      obb-files:
      obb-names:
      performance-metrics: false
      num-uniform-shards: null
      test-runner-class: null
      test-targets:
      robo-directives:
      robo-script: null
      device:
        - model: NexusLowRes
          version: 28
          locale: en
          orientation: portrait
      num-flaky-test-attempts: 0
      test-targets-for-shard:
      fail-fast: false

    flank:
      max-test-shards: 1
      shard-time: -1
      num-test-runs: 1
      smart-flank-gcs-path: 
      smart-flank-disable-upload: false
      default-test-time: 120.0
      use-average-test-time-for-new-tests: false
      files-to-download:
      test-targets-always-run:
      disable-sharding: true
      project: flank-open-source
      local-result-dir: results
      full-junit-result: false
      # Android Flank Yml
      keep-file-path: false
      additional-app-test-apks:
      run-timeout: -1
      legacy-junit-result: false
      ignore-failed-tests: false
      output-style: compact
      disable-results-upload: false
      default-class-test-time: 240.0

RunTests
  1 test / 1 shard

  1 matrix ids created in 0m 18s
  https://console.developers.google.com/storage/browser/test-lab-v9cn46bb990nx-kz69ymd4nm9aq/2020-12-04_16-34-33.963626_WvMf

Matrices webLink
  matrix-1wl88r1wuwgov https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/4778909069129894447/executions/bs.b22378a05c11ef79

  3m 46s matrix-1wl88r1wuwgov FINISHED

CostReport
  Virtual devices
    $0.02 for 1m

MatrixResultsReport
  1 / 1 (100.00%)
┌─────────┬──────────────────────┬────────────────────────────┬────────────────────────────────┐
│ OUTCOME │      MATRIX ID       │      TEST AXIS VALUE       │          TEST DETAILS          │
├─────────┼──────────────────────┼────────────────────────────┼────────────────────────────────┤
│ success │ matrix-1wl88r1wuwgov │ NexusLowRes-28-en-portrait │ 1 test cases passed, 1 skipped │
└─────────┴──────────────────────┴────────────────────────────┴────────────────────────────────┘
  
Matrices webLink
  matrix-1wl88r1wuwgov https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/4778909069129894447/executions/bs.b22378a05c11ef79


Process finished with exit code 0

Checklist

Documented
Unit tested
Integration tests passes

github-actions · 2020-12-04T16:53:06Z

Timestamp: 2020-12-07 16:08:47
Buildscan url for ubuntu-workflow run 406226559

Sloox

Looks good 👍

bootstraponline · 2020-12-07T14:52:08Z

docs/logging.md

+1. ```Simple, multi``` and ```verbose``` output style prints logs from ```SIMPLE``` and ```DETAILED``` levels.
+1. ```Compact``` style prints log only from ```SIMPLE``` level.
+1. If you want a print message for all output styles uses ```log``` or ```logLn``` with only ```message``` parameter.
+1. If you want print message more detailed message use ```log``` or ```logLn``` and set ```level``` to ```OutputLogLevel.DETAILED```


do we need to update the # output-style: single section in the user facing docs & example yaml files?

Thanks! Docs and yaml examples updated.

piotradamczyk5 · 2020-12-07T15:03:49Z

flank-scripts/build.gradle.kts

@@ -28,7 +28,7 @@ shadowJar.apply {
    }
 }
 // <breaking change>.<feature added>.<fix/minor change>
-version = "1.1.4"
+version = "1.1.5"


do we need to update this?

I check it one more time and it's fine with 1.1.4. I revert it and we check on CI.

piotradamczyk5 · 2020-12-07T15:07:21Z

test_runner/src/main/kotlin/ftl/args/IArgs.kt

+
+fun IArgs.setLogLevel() = also {
+    ftl.log.setLogLevel(logLevel)
+}


IMHO setSomething needs some parameter, so I would suggest to rename it to setupLogLevel

piotradamczyk5 · 2020-12-07T15:10:12Z

test_runner/src/main/kotlin/ftl/log/OutputLogger.kt

+
+fun logLn(message: Any = "", level: OutputLogLevel = OutputLogLevel.SIMPLE) =
+    if (minimumLogLevel >= level) println(message)
+    else Unit


We do not return need output from this method so I suggest to

fun log(message: Any, level: OutputLogLevel = OutputLogLevel.SIMPLE) { if (minimumLogLevel >= level) print(message) } fun logLn(message: Any = "", level: OutputLogLevel = OutputLogLevel.SIMPLE) { if (minimumLogLevel >= level) println(message) }

codecov-io · 2020-12-07T16:08:42Z

Codecov Report

Merging #1375 (8928dcf) into master (5cc0f1b) will increase coverage by 77.84%.
The diff coverage is 58.26%.

@@              Coverage Diff              @@
##             master    #1375       +/-   ##
=============================================
+ Coverage          0   77.84%   +77.84%     
- Complexity        0      706      +706     
=============================================
  Files             0      245      +245     
  Lines             0     4713     +4713     
  Branches          0      904      +904     
=============================================
+ Hits              0     3669     +3669     
- Misses            0      551      +551     
- Partials          0      493      +493

adamfilipow92 assigned adamfilipow92 and piotradamczyk5 Dec 4, 2020

github-actions bot added Feature Tiger 🐯 labels Dec 4, 2020

adamfilipow92 marked this pull request as ready for review December 7, 2020 11:59

adamfilipow92 requested review from jan-goral, piotradamczyk5, pawelpasterz, bootstraponline and Sloox and removed request for jan-goral and piotradamczyk5 December 7, 2020 11:59

Sloox approved these changes Dec 7, 2020

View reviewed changes

adamfilipow92 added 5 commits December 7, 2020 13:32

Update OutputStyle.kt

03b366f

Replaced direct printing to console by logger

f1f774c

Reduce log functions to logLn and log

a2f2f24

Update Debug.kt

f298a86

Added docs, tests, fix existing tests

0f5faf9

bootstraponline force-pushed the 1306-make-the-flank-output-smaller branch from 65dd556 to 0f5faf9 Compare December 7, 2020 13:32

bootstraponline reviewed Dec 7, 2020

View reviewed changes

piotradamczyk5 reviewed Dec 7, 2020

View reviewed changes

Cr fixes, update docs

8928dcf

piotradamczyk5 self-requested a review December 7, 2020 16:17

piotradamczyk5 approved these changes Dec 7, 2020

View reviewed changes

mergify bot merged commit 628a018 into master Dec 7, 2020

mergify bot deleted the 1306-make-the-flank-output-smaller branch December 7, 2020 16:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Add new output style to print smaller output #1375

feat: Add new output style to print smaller output #1375

adamfilipow92 commented Dec 4, 2020 •

edited

Loading

github-actions bot commented Dec 4, 2020 •

edited

Loading

Sloox left a comment

bootstraponline Dec 7, 2020

adamfilipow92 Dec 7, 2020

piotradamczyk5 Dec 7, 2020

adamfilipow92 Dec 7, 2020

piotradamczyk5 Dec 7, 2020

adamfilipow92 Dec 7, 2020

piotradamczyk5 Dec 7, 2020

adamfilipow92 Dec 7, 2020

codecov-io commented Dec 7, 2020

feat: Add new output style to print smaller output #1375

feat: Add new output style to print smaller output #1375

Conversation

adamfilipow92 commented Dec 4, 2020 • edited Loading

Test Plan

Output example

Checklist

github-actions bot commented Dec 4, 2020 • edited Loading

Sloox left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Dec 7, 2020

Codecov Report

adamfilipow92 commented Dec 4, 2020 •

edited

Loading

github-actions bot commented Dec 4, 2020 •

edited

Loading