Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flank should reflect gcloud exit codes #926

Merged
merged 14 commits into from
Aug 5, 2020

Conversation

adamfilipow92
Copy link
Contributor

@adamfilipow92 adamfilipow92 commented Aug 3, 2020

Fixes #802

Test Plan

How do we know the code works?

Flank exit codes reflect gcloud exit codes

Comparision

Documentation about the current state can be found here, please keep in mind this is only my interpretation and feel free to request changes.

Exit code Gcloud Flank
0 All test executions passed. All tests passed
1 A general failure occurred. Possible causes include: a filename that does not exist or an HTTP/network error. All matrices finished but at least one test failed or inconclusive.
2 Testing exited because unknown commands or arguments were provided. Usually indicates missing or wrong usage of flags, incorrect parameters, errors in config files.
3 At least one matrix not finished (usually a FTL internal error) or unexpected error occurred.
10 One or more test cases (tested classes or class methods) within a test execution did not pass.
15 Firebase Test Lab could not determine if the test matrix passed or failed, because of an unexpected error.
18 The test environment for this test execution is not supported because of incompatible test dimensions. This error might occur if the selected Android API level is not supported by the selected device type.
19 The test matrix was canceled by the user.
20 A test infrastructure error occurred.

Mapping plan Flank -> Gcloud and test plan

Status Gcloud Code Old Flank Code Description
0 0 All test executions passed.
10 1 One or more test cases (tested classes or class methods) within a test execution did not pass.
1 2 Apk, yml not found or an HTTP/network error
2 2 Unknown commands
18 2 The test environment for this test execution is not supported because of incompatible test dimensions. Eg. device not support selected API level
15 3 Unexpected errors from gcloud
19 3 The test matrix was canceled by the user.
20 3 A test infrastructure error occurred.

Checklist

  • Exit codes changed
  • Check exceptions what Flank throws and refractor to types reflected gcloud exit codes
  • Documented
  • Unit tested
  • release_notes.md updated
  • readme.md updated

@adamfilipow92 adamfilipow92 self-assigned this Aug 3, 2020
@adamfilipow92 adamfilipow92 force-pushed the 802-flank-should-reflect-gcloud-exit-codes branch from 3aec7ea to 0babda2 Compare August 4, 2020 15:05
@adamfilipow92 adamfilipow92 marked this pull request as ready for review August 4, 2020 15:13
@adamfilipow92 adamfilipow92 force-pushed the 802-flank-should-reflect-gcloud-exit-codes branch from e1a9721 to eabd45e Compare August 4, 2020 15:19
@piotradamczyk5 piotradamczyk5 self-requested a review August 4, 2020 15:36
@adamfilipow92 adamfilipow92 merged commit c9e349a into master Aug 5, 2020
@adamfilipow92 adamfilipow92 deleted the 802-flank-should-reflect-gcloud-exit-codes branch August 5, 2020 09:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flank should reflect gcloud exit codes
2 participants