Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Remove macos-10.15 usage #2312

Merged
merged 9 commits into from
Oct 24, 2022
Merged

Conversation

philipphofmann
Copy link
Member

GH actions will remove the macOS-10.15 image, which contains an iOS 12 simulator on 12/1/22; see actions/runner-images#5583. Neither the macOS-11 nor the macOS-12 image contains an iOS 12 simulator. GH concluded to not add more pre-installed simulators. SauceLabs doesn't support running unit tests and adding another cloud solution as Firebase TestLab would increase the complexity of CI. Not running the unit tests on iOS 12 opens a risk of introducing bugs, which has already happened in the past, especially with swizzling. Therefore, we give manually installing the iOS 12 simulator a try.

Fixes GH-2218

@github-actions
Copy link

github-actions bot commented Oct 21, 2022

Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1230.71 ms 1253.02 ms 22.31 ms
Size 20.50 KiB 338.99 KiB 318.49 KiB

Baseline results on branch: master

Startup times

Revision Plain With Sentry Diff
94b4f2b 1243.31 ms 1262.06 ms 18.75 ms
791123d 1217.52 ms 1253.08 ms 35.56 ms
be47c6c 1230.39 ms 1261.71 ms 31.33 ms
075911d 1256.73 ms 1271.22 ms 14.49 ms
0fdf0b2 1194.37 ms 1227.90 ms 33.53 ms
9231f60 1256.78 ms 1267.74 ms 10.96 ms
7977992 1219.80 ms 1241.92 ms 22.12 ms
fbeb49a 1218.13 ms 1243.70 ms 25.57 ms
c929040 1254.84 ms 1278.42 ms 23.58 ms
fa64784 1203.98 ms 1226.16 ms 22.18 ms

App size

Revision Plain With Sentry Diff
94b4f2b 20.50 KiB 337.70 KiB 317.20 KiB
791123d 20.51 KiB 331.81 KiB 311.30 KiB
be47c6c 20.50 KiB 333.54 KiB 313.04 KiB
075911d 20.51 KiB 332.90 KiB 312.39 KiB
0fdf0b2 20.51 KiB 332.90 KiB 312.39 KiB
9231f60 20.51 KiB 333.58 KiB 313.07 KiB
7977992 20.50 KiB 333.58 KiB 313.07 KiB
fbeb49a 20.51 KiB 333.51 KiB 313.00 KiB
c929040 20.51 KiB 333.10 KiB 312.59 KiB
fa64784 20.50 KiB 335.95 KiB 315.45 KiB

Previous results on branch: ci/install-xcode-simulator

Startup times

Revision Plain With Sentry Diff
75f161c 1234.15 ms 1246.26 ms 12.11 ms

App size

Revision Plain With Sentry Diff
75f161c 20.50 KiB 338.99 KiB 318.49 KiB

@@ -67,8 +67,8 @@ class LaunchUITests: XCTestCase {
let app = XCUIApplication()
app.navigationBars["iOS_Swift.SecondarySplitView"].buttons["Root ViewController"].waitForExistence("SplitView not loaded.")

// This validation is currently not working on iOS 10.
if #available(iOS 11.0, *) {
// This validation is currently not working on iOS 12 and iOS 10.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created an issue for enabling the test again #2313

Copy link
Contributor

@brustolin brustolin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Im just a little worried about the test duration now that it needs to download emulators.
But since its the only way to test iOS 12, and I agree that we can't avoid testing it, so be it.

@philipphofmann philipphofmann merged commit 8168e86 into master Oct 24, 2022
@philipphofmann philipphofmann deleted the ci/install-xcode-simulator branch October 24, 2022 13:19
@philipphofmann
Copy link
Member Author

I'm also a bit worried, but we can try, and if it takes too long, we can reevaluate.

kevinrenskers added a commit that referenced this pull request Oct 24, 2022
* master:
  fix: Fix moving app state to previous app state (#2321)
  ci: Remove macos-10.15 usage (#2312)
  ref: Remove not existent SentryContext (#2320)
@armcknight
Copy link
Member

We could consider moving more time-intensive tests for older versions to a nightly build and just fix any issues they turn up async.

@philipphofmann
Copy link
Member Author

We could consider moving more time-intensive tests for older versions to a nightly build and just fix any issues they turn up async.

Yes, I would go with that option, if the tests get slow. I would keep them running for every PR for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unit test or try building on earliest supported platform versions in CI
3 participants