feat: gameloop addition #1262

Sloox · 2020-10-22T11:06:50Z

Fixes #1245

adds the gameloop into android (androidtestloop)

Checklist

Documented
Unit tested

…1245-game-loop

test_runner/src/main/kotlin/ftl/run/model/AndroidTestContext.kt

test_runner/src/main/kotlin/ftl/gc/GcAndroidTestMatrix.kt

test_runner/src/main/kotlin/ftl/args/AndroidArgs.kt

test_runner/src/main/kotlin/ftl/gc/android/Utils.kt

test_runner/src/main/kotlin/ftl/run/platform/android/AndroidTestConfig.kt

pawelpasterz

I've noticed some differences in logic compared with gcloud

   app: [path]/app-debug.apk
   test: [path]/app-multiple-success-debug-androidTest.apk
   type: game-loop

runs normal instrumentation test where gcloud throw error

ERROR: (gcloud.alpha.firebase.test.android.run) Invalid value for [test]: may not be used with test type [game-loop]

 app: [path]/app-debug.apk
 type: game-loop

runs sanity robo where gcloud runs game-loop

 app: [path]/app-debug.apk
 obb-files:
   - [path]/test.obb <== the same as in PR
 type: game-loop

results in a nasty exception

RunTests
  Uploading app-debug.apk .
  java.io.IOException: Request failed
    at ftl.http.ExecuteWithRetryKt$executeWithRetry$$inlined$withRetry$1.invokeSuspend(ExecuteWithRetry.kt:40)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:176)
    at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:111)
    at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:308)
    at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:318)
    at kotlinx.coroutines.CancellableContinuationImpl.resumeUndispatched(CancellableContinuationImpl.kt:400)
    at kotlinx.coroutines.EventLoopImplBase$DelayedResumeTask.run(EventLoop.common.kt:489)
    at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
    at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:84)
    at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
    at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
    at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
    at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
    at ftl.http.ExecuteWithRetryKt.executeWithRetry(ExecuteWithRetry.kt:45)
    at ftl.run.platform.RunAndroidTestsKt$executeAndroidTestMatrix$2$invokeSuspend$$inlined$map$lambda$1.invokeSuspend(RunAndroidTests.kt:103)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
POST https://testing.googleapis.com/v1/projects/pawel-p/testMatrices
{
  "code" : 400,
  "errors" : [ {
    "domain" : "global",
    "message" : "Required field should not be empty: google.devtools.testing.v1.ObbFile.obb_file_name",
    "reason" : "badRequest"
  } ],
  "message" : "Required field should not be empty: google.devtools.testing.v1.ObbFile.obb_file_name",
  "status" : "INVALID_ARGUMENT"
}
    at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:149)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:112)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:39)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:443)
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1108)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:541)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
    at ftl.http.ExecuteWithRetryKt$executeWithRetry$$inlined$withRetry$1.invokeSuspend(ExecuteWithRetry.kt:45)
    ... 21 more

and gcloud's

ERROR: (gcloud.alpha.firebase.test.android.run) Invalid value for [obb-files]: 
[path/test.obb] is not a valid OBB file name, which must have the 
format: (main|patch).<versionCode>.<package.name>.obb

 app: [path]/app-debug.apk
 obb-files:
   -  [path]/main.0300110.com.example.android.obb
 type: game-loop

here is interesting because flank throws an exception the same as above where gcloud results:

 ERROR: (gcloud.alpha.firebase.test.android.run)
 Matrix [matrix-2p3op1ukymruk] failed during validation: The app does not have a correctly formatted game-loop intent 
 filter.

I might be wrong and some of those errors are due to my incorrect configuration. Please verify

test_runner/src/main/kotlin/ftl/args/AndroidArgs.kt

test_runner/src/main/kotlin/ftl/args/ValidateAndroidArgs.kt

test_runner/src/main/kotlin/ftl/gc/android/SetupAndroidTest.kt

test_runner/src/main/kotlin/ftl/gc/android/Utils.kt

piotradamczyk5

please address comments and update docs/index.md and flank.yml

Sloox · 2020-10-26T10:14:13Z

I've noticed some differences in logic compared with gcloud

   app: [path]/app-debug.apk
   test: [path]/app-multiple-success-debug-androidTest.apk
   type: game-loop

runs normal instrumentation test where gcloud throw error

ERROR: (gcloud.alpha.firebase.test.android.run) Invalid value for [test]: may not be used with test type [game-loop]

 app: [path]/app-debug.apk
 type: game-loop

runs sanity robo where gcloud runs game-loop

 app: [path]/app-debug.apk
 obb-files:
   - [path]/test.obb <== the same as in PR
 type: game-loop

results in a nasty exception

RunTests
  Uploading app-debug.apk .
  java.io.IOException: Request failed
    at ftl.http.ExecuteWithRetryKt$executeWithRetry$$inlined$withRetry$1.invokeSuspend(ExecuteWithRetry.kt:40)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTaskKt.resume(DispatchedTask.kt:176)
    at kotlinx.coroutines.DispatchedTaskKt.dispatch(DispatchedTask.kt:111)
    at kotlinx.coroutines.CancellableContinuationImpl.dispatchResume(CancellableContinuationImpl.kt:308)
    at kotlinx.coroutines.CancellableContinuationImpl.resumeImpl(CancellableContinuationImpl.kt:318)
    at kotlinx.coroutines.CancellableContinuationImpl.resumeUndispatched(CancellableContinuationImpl.kt:400)
    at kotlinx.coroutines.EventLoopImplBase$DelayedResumeTask.run(EventLoop.common.kt:489)
    at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:274)
    at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:84)
    at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
    at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
    at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
    at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
    at ftl.http.ExecuteWithRetryKt.executeWithRetry(ExecuteWithRetry.kt:45)
    at ftl.run.platform.RunAndroidTestsKt$executeAndroidTestMatrix$2$invokeSuspend$$inlined$map$lambda$1.invokeSuspend(RunAndroidTests.kt:103)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:56)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:738)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
POST https://testing.googleapis.com/v1/projects/pawel-p/testMatrices
{
  "code" : 400,
  "errors" : [ {
    "domain" : "global",
    "message" : "Required field should not be empty: google.devtools.testing.v1.ObbFile.obb_file_name",
    "reason" : "badRequest"
  } ],
  "message" : "Required field should not be empty: google.devtools.testing.v1.ObbFile.obb_file_name",
  "status" : "INVALID_ARGUMENT"
}
    at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:149)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:112)
    at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:39)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:443)
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1108)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:541)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:474)
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:591)
    at ftl.http.ExecuteWithRetryKt$executeWithRetry$$inlined$withRetry$1.invokeSuspend(ExecuteWithRetry.kt:45)
    ... 21 more

and gcloud's

ERROR: (gcloud.alpha.firebase.test.android.run) Invalid value for [obb-files]: 
[path/test.obb] is not a valid OBB file name, which must have the 
format: (main|patch).<versionCode>.<package.name>.obb

 app: [path]/app-debug.apk
 obb-files:
   -  [path]/main.0300110.com.example.android.obb
 type: game-loop

here is interesting because flank throws an exception the same as above where gcloud results:

 ERROR: (gcloud.alpha.firebase.test.android.run)
 Matrix [matrix-2p3op1ukymruk] failed during validation: The app does not have a correctly formatted game-loop intent 
 filter.

I might be wrong and some of those errors are due to my incorrect configuration. Please verify
To address each question

test is removed as per @jan-gogo change request.
unless I have tested incorrectly i am not getting the same behavior as you mention it is running a AndroidLoop. Can you verify
Fixed and added a required addition where the name of the obbfile is required. This is due to a mismatch of documention between ftl and the actual error message. Documentation states "can be null" but it is required.
No longer applicable.
Take note that the obb-names addition needs to now be added and verified and of the correct format.
If not it will fail at the ftl side.
more information about the obb files and their names: https://firebase.google.com/docs/test-lab/reference/testing/rest/v1/projects.testMatrices#obbfile
&& https://developer.android.com/google/play/expansion-files

Sloox · 2020-10-26T10:14:22Z

@pawelpasterz

pawelpasterz · 2020-10-26T10:50:49Z

unless I have tested incorrectly i am not getting the same behavior as you mention it is running a AndroidLoop. Can you verify

This is what I get when launched with flank
https://console.firebase.google.com/project/flank-open-source/testlab/histories/bh.da0c237aaa33732/matrices/7368025719024499574

and gcloud (it actually fails but you can see it's trying to run game-loop)

copy-pasted config

gcloud:
  app: [path]/app-debug.apk
  type: game-loop

pawelpasterz · 2020-10-26T10:52:54Z

I can reproduce it 10/10. Flank built from commit

version: local_snapshot
revision: f91d325cbb5fe547a040d79c9925b479831cb07e

Sloox · 2020-10-27T09:13:08Z

@pawelpasterz Added manual tests have a look at the Debug.kt. Also added a semi-compatible android application that works without crashes. You can see the latest commit to see whats added.

codecov-io · 2020-10-27T09:21:06Z

Codecov Report

Merging #1262 into master will increase coverage by 0.10%.
The diff coverage is 81.29%.

@@             Coverage Diff              @@
##             master    #1262      +/-   ##
============================================
+ Coverage     79.65%   79.75%   +0.10%     
- Complexity      715      724       +9     
============================================
  Files           233      235       +2     
  Lines          4399     4485      +86     
  Branches        756      768      +12     
============================================
+ Hits           3504     3577      +73     
- Misses          496      510      +14     
+ Partials        399      398       -1

…1245-game-loop

test_runner/src/main/kotlin/ftl/gc/android/Utils.kt

test_runner/src/main/kotlin/ftl/args/AndroidArgs.kt

Gameloop addition 1

c3fea79

Sloox self-assigned this Oct 22, 2020

Flank deleted a comment from codecov-io Oct 22, 2020

Sloox and others added 6 commits October 22, 2020 17:30

Updates for gameloop

1fd91ca

Merge branch 'master' into #1245-game-loop

3519b6e

added some validation

bcb0dc4

Merge branch '#1245-game-loop' of https://github.com/Flank/flank into #…

069cfda

…1245-game-loop

Added more tests

6c94bf7

fix tests

a49ce38

Sloox requested review from pawelpasterz and jan-goral October 23, 2020 08:29

Sloox added New Option Used to track PR with new configuration option (useful when updating fladle) Tiger 🐯 labels Oct 23, 2020

Sloox marked this pull request as ready for review October 23, 2020 09:13

Sloox requested review from adamfilipow92, zuziaka, bootstraponline and piotradamczyk5 October 23, 2020 09:16

jan-goral reviewed Oct 23, 2020

View reviewed changes

pawelpasterz suggested changes Oct 23, 2020

View reviewed changes

fix pr comments 1

0151f55

Flank deleted a comment from codecov-io Oct 23, 2020