macOS 11.0 running Xcode UI tests is slow, encounters internal errors

[kyle-n]

Description
I am seeing inconsistent bugs and system failures with a GitHub Action that triggers Xcode UI tests on macOS-11.0.

Area for Triage:
Apple

Question, Bug, or Feature?:
Bug

Virtual environments affected

• Ubuntu 16.04
• Ubuntu 18.04
• Ubuntu 20.04
• macOS 10.15
• macOS 11.0
• Windows Server 2016 R2
• Windows Server 2019

Expected behavior

• My GitHub Action runs the Xcode UI tests and they pass or fail based on the code in the branch that triggered the action
• The tests should also take a comparable amount of time to run if no new tests are added (approx 15min tops in previous runs). Obviously this does not count time queued.
• Each job should run once

Actual behavior

• Jobs can take 1.5-3 hours and fail with the error GitHub Actions has encountered an internal error when running your job..
• Jobs appear to run multiple times?
• If a job does run, random tests fail with the error: (full logs)

2021-01-14T06:09:53.2597130Z /Users/runner/work/HighlightedTextEditor/HighlightedTextEditor/Tests/Essayist/iOS-EssayistUITests/iOS_EssayistUITests.swift:130: error: -[iOS_EssayistUITests.iOS_EssayistUITests testAutocapitalizationModifier] : Failed to get launch progress for <XCUIApplicationImpl: 0x600003ce0150 com.kylenazario.Essayist at /Users/runner/Library/Developer/Xcode/DerivedData/Essayist-amfzprbnvltnuodtpdnynupooynw/Build/Products/Debug-iphonesimulator/Essayist.app>: Failed to create app asset promise. Failed to create promise.

Repro steps
To reproduce, I select the option to re-run all jobs on this action. Please let me know if you need me to re-run all jobs, happy to do so.

More links
yml for this Action

[miketimofeev]

Hi @kyle-n!
Unfortunately, we are currently experiencing queuing issues with macOS 11.0 and investigating possible ways to handle them. That's probably the root cause of your issues as well. More details can be found in the Announcement #1814

[vin-fandemand]

@miketimofeev I am facing the same issue with build. It takes upto 6 hours at time to finish compiling, making github actions unsuitable for ios builds. It is unnecessary waste of time and money. Do you have a work around for this ?

[miketimofeev]

@vin-fandemand do you mean it takes too much time to compile and run tests in comparison to local build?

[vin-fandemand]

Yes correct. I am using the following configuration, and fastlane to build test and sign the ios application. In local, it takes only upto 15-20mins, while in actions, it takes a long time. I have one running now for almost 2.5 hours.

on:
  workflow_dispatch:
jobs:
  build_ios:
    runs-on: "macos-latest"

[miketimofeev]

@vin-fandemand the runner VMs are only 3-cores at the moment, which can explain this poor performance. If you believe it's a different problem then please create a separate issue with repro steps so we can take look deeply.

[vin-fandemand]

@miketimofeev is there a way to make the run faster ? Even if there is a premium charge.

[miketimofeev]

@vin-fandemand, unfortunately, no, at least at the moment. We're continuously working on several improvements, but no ETA for now.