Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Output inconsistency (ios vs android) #1388

Closed
pawelpasterz opened this issue Dec 9, 2020 · 1 comment · Fixed by #1395
Closed

Output inconsistency (ios vs android) #1388

pawelpasterz opened this issue Dec 9, 2020 · 1 comment · Fixed by #1395
Assignees

Comments

@pawelpasterz
Copy link
Contributor

Describe the bug

A clear and concise description of what the bug is.

There are differences in flank output for iOS and Android when all test are filtered
Below is the output for https://github.com/Flank/flank/blob/master/integration_tests/src/test/kotlin/integration/AllTestFilteredIT.kt

To Reproduce

Steps to reproduce the behavior:

  1. run ios test with config
    gcloud:
      test: [path to zip]
      xctestrun-file: [path to xctestrun]
    flank:
      test-targets:
        - nonExisting/Class
    
  2. run android test with config
    gcloud:
      app: [path to app apk]
      test: [path to test apk]
      test-targets:
        - class non.existing.Class
    flank:
      disable-sharding: true
    

Expected behavior

A clear and concise description of what you expected to happen.

There is a decision to be made if flank should (or not) print args.
Based on the answer:

  • fix should be implemented
  • IT must be updated

Details (please complete the following information):

Have you tested on the latest Flank snapshot?

Post the output of flank --version.

version: local_snapshot
revision: 2d2982474e9d38ee39639ed7282b84fb5e33ce01

Additional context

Add any other context about the problem here.

Introduced in #1219

@adamfilipow92
Copy link
Contributor

Plan

  1. Print args like on android
  2. When no tests, throw the same exception type in android and ios

mergify bot pushed a commit that referenced this issue Dec 11, 2020
Fixes #1315 

## Test Plan
> How do we know the code works?

1. run ```./gradlew integrationTests```
1. integration tests should pass

## Description

Flank should also verify XML results for android and ios integration tests.

This ticket should be merged after #1316 when we update all tests. 

```filter all tests - ios``` is disabled and should be updated in #1388

## Checklist
- [X] Add verification for basic android test
- [X] Add verification for tests from #1316
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants