Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: bringing integration tests back to life #2366

Merged
merged 1 commit into from
Apr 27, 2023
Merged

fix: bringing integration tests back to life #2366

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,8 @@ gcloud:
## This flag may be repeated to specify multiple devices.
## The four device dimensions are: model, version, locale, and orientation.
# device:
# - model: iphone8
# version: 12.0
# - model: iphone13pro
# version: 15.7
# locale: en
# orientation: portrait
# - model: iphonex
Expand Down
4 changes: 2 additions & 2 deletions integration_tests/src/test/kotlin/integration/DumpShardsIT.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,8 @@ class DumpShardsIT {
assertThat(shards.count()).isEqualTo(2)

shards.first().values.flatten().let { firstShard ->
assertThat(firstShard.count()).isEqualTo(8)
assertThat(firstShard).contains("EarlGreyExampleSwiftTests/testWithCustomFailureHandler")
assertThat(firstShard.count()).isEqualTo(2)
assertThat(firstShard).contains("FlankExampleTests/test2")
}
}
}
2 changes: 1 addition & 1 deletion integration_tests/src/test/kotlin/integration/LegacyResultIT.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ class LegacyResultIT {
val testSuiteOverview = outputReport.firstTestSuiteOverview

testSuiteOverview.assertTestCountMatches(
total = 17,
total = 4,
skipped = 0
)
}
Expand Down
4 changes: 2 additions & 2 deletions integration_tests/src/test/resources/cases/all_test_filtered_ios.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
gcloud:
test: ../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/EarlGreyExample/EarlGreyExample.zip
xctestrun-file: ../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/EarlGreyExample/EarlGreyExampleSwiftTests.xctestrun
test: ../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/FlankExample/FlankExample.zip
xctestrun-file: ../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/FlankExample/FlankExample.xctestrun
flank:
test-targets:
- nonExisting/Class
Expand Down
4 changes: 2 additions & 2 deletions integration_tests/src/test/resources/cases/dump_shards_ios.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
gcloud:
test: ../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/EarlGreyExample/EarlGreyExample.zip
xctestrun-file: ../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/EarlGreyExample/EarlGreyExampleSwiftTests.xctestrun
test: ../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/FlankExample/FlankExample.zip
xctestrun-file: ../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/FlankExample/FlankExample.xctestrun

flank:
disable-sharding: false
Expand Down
4 changes: 2 additions & 2 deletions integration_tests/src/test/resources/cases/flank_ios_single_legacy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
gcloud:
test: "../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/EarlGreyExample/EarlGreyExample.zip"
xctestrun-file: "../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/EarlGreyExample/EarlGreyExampleSwiftTests.xctestrun"
test: "../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/FlankExample/FlankExample.zip"
xctestrun-file: "../test_runner/src/test/kotlin/ftl/fixtures/tmp/ios/FlankExample/FlankExample.xctestrun"

flank:
output-report: json
Expand Down
8 changes: 4 additions & 4 deletions integration_tests/src/test/resources/compare/AllTestFilteredIT-ios-compare
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ IosArgs
network-profile: null
results-history-name: null
# iOS gcloud
test: [0-9a-zA-Z\/_.-]*\/test_runner\/src\/test\/kotlin\/ftl\/fixtures\/tmp\/ios\/EarlGreyExample\/EarlGreyExample.zip
xctestrun-file: [0-9a-zA-Z\/_.-]*\/test_runner\/src\/test\/kotlin\/ftl\/fixtures\/tmp\/ios\/EarlGreyExample\/EarlGreyExampleSwiftTests.xctestrun
test: [0-9a-zA-Z\/_.-]*\/test_runner\/src\/test\/kotlin\/ftl\/fixtures\/tmp\/ios\/FlankExample\/FlankExample.zip
xctestrun-file: [0-9a-zA-Z\/_.-]*\/test_runner\/src\/test\/kotlin\/ftl\/fixtures\/tmp\/ios\/FlankExample\/FlankExample.xctestrun
xcode-version: null
device:
- model: iphone8
version: 13.6
- model: iphone13pro
version: 15.7
locale: en
orientation: portrait
num-flaky-test-attempts: 0
Expand Down
4 changes: 2 additions & 2 deletions test_projects/ios/EarlGreyExample/flank.ios.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ gcloud:
test: ./earlgrey_example.zip
xctestrun-file: ./dd_tmp/Build/Products/EarlGreyExampleSwiftTests_iphoneos13.4-arm64e.xctestrun
device:
- model: iphone8
version: 12.0
- model: iphone13pro
version: 15.7
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this work for this EarlGreyExample?
If not, is this example needed for anything else or should it be deleted entirely?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EarlGreyExample (as it is now) won't run on iphone13pro/15.7. I think this particular test is not actively used, but this change might be useful in case someone wants to update to EarlGreyExample v2 in the future.

orientation: portrait
locale: en_US
4 changes: 2 additions & 2 deletions test_runner/flank.ios.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,8 @@ gcloud:
## This flag may be repeated to specify multiple devices.
## The four device dimensions are: model, version, locale, and orientation.
# device:
# - model: iphone8
# version: 12.0
# - model: iphone13pro
# version: 15.7
# locale: en
# orientation: portrait
# - model: iphonex
Expand Down
4 changes: 2 additions & 2 deletions test_runner/src/main/kotlin/ftl/config/FtlConstants.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ object FtlConstants {
const val localhost = "http://localhost:8080"
const val defaultLocale = "en"
const val defaultOrientation = "portrait"
const val defaultIosModel = "iphone8"
const val defaultIosVersion = "13.6"
const val defaultIosModel = "iphone13pro"
const val defaultIosVersion = "15.7"
const val defaultAndroidModel = "NexusLowRes"
const val defaultAndroidVersion = "28"
const val defaultIosConfig = "./flank.ios.yml"
Expand Down
Loading