Runtimes download Broken

[MattKiazyk]

Since Xcode 16 updates things are broken.

new ContentType: CryptexDiskImage that isn't handled.

<dict>
			<key>category</key>
			<string>simulator</string>
			<key>contentType</key>
			<string>cryptexDiskImage</string>
			<key>dictionaryVersion</key>
			<integer>2</integer>
			<key>downloadMethod</key>
			<string>mobileAsset</string>
			<key>fileSize</key>
			<integer>8455760175</integer>
			<key>identifier</key>
			<string>com.apple.dmg.iPhoneSimulatorSDK18_0_b1</string>
			<key>name</key>
			<string>iOS 18.0 beta Simulator Runtime</string>
			<key>platform</key>
			<string>com.apple.platform.iphoneos</string>
			<key>simulatorVersion</key>
			<dict>
				<key>buildUpdate</key>
				<string>22A5282m</string>
				<key>version</key>
				<string>18.0</string>
			</dict>
			<key>version</key>
			<string>18.0.0.1</string>
		</dict>
		```

[saxonww]

Just running xcodes runtimes fails here:

$ xcodes runtimes
Error: keyNotFound(CodingKeys(stringValue: "source", intValue: nil), Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "downloadables", intValue: nil), _PlistKey(stringValue: "Index 114", intValue: 114)], debugDescription: "No value associated with key CodingKeys(stringValue: \"source\", intValue: nil) (\"source\").", underlyingError: nil))

This is on a Ventura machine with Xcode 15.0.1. Was working last month.

[MattKiazyk]

Just running xcodes runtimes fails here:

$ xcodes runtimes
Error: keyNotFound(CodingKeys(stringValue: "source", intValue: nil), Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "downloadables", intValue: nil), _PlistKey(stringValue: "Index 114", intValue: 114)], debugDescription: "No value associated with key CodingKeys(stringValue: \"source\", intValue: nil) (\"source\").", underlyingError: nil))

This is on a Ventura machine with Xcode 15.0.1. Was working last month.

Looks like for the iOS 18 runtimes - there's no more source property that gave the URL to download it 😬

[phantomato]

Looks like it's still available at https://download.developer.apple.com/Developer_Tools/iOS_18_beta_Simulator_Runtime/iOS_18_beta_Simulator_Runtime.dmg which has a similar URL structure as the older versions.

Maybe we can infer the URL in case it's not present in the response?

[MattKiazyk]

Maybe we can infer the URL in case it's not present in the response?

On: https://developer.apple.com/documentation/xcode/installing-additional-simulator-runtimes it indicates that at some point the runtimes will be removed from there.

In investigating, I've found a few things.

Apple is using an apple archive now for simulators. People who are smarter then me hopefully can figure it out how we can download and extract the new form

[setoelkahfi]

I just got this when trying out Xcode 16 Beta.

[Ernest0-Production]

@MattKiazyk I think it makes sense to add safer decoding each DownloadableRuntime object.

Currently, only the new SDKs do not have the source field, but the old ones do.

struct DownloadableRuntimesResponse: Decodable {
    let sdkToSimulatorMappings: [SDKToSimulatorMapping]
    let sdkToSeedMappings: [SDKToSeedMapping]
    let refreshInterval: Int
    @SaveArray var downloadables: [DownloadableRuntime] // filters invalid array elements
    let version: String
}

---

I can help contribute this functionality.

[ObjectiveCesar]

@MattKiazyk I think it makes sense to add safer decoding each DownloadableRuntime object.

Currently, only the new SDKs do not have the source field, but the old ones do.

struct DownloadableRuntimesResponse: Decodable {
    let sdkToSimulatorMappings: [SDKToSimulatorMapping]
    let sdkToSeedMappings: [SDKToSeedMapping]
    let refreshInterval: Int
    @SaveArray var downloadables: [DownloadableRuntime] // filters invalid array elements
    let version: String
}

I can help contribute this functionality.

That would be very much appreciated!

[MattKiazyk]

@Ernest0-Production yes that's the plan. At least allow old runtimes download and give proper errors for new ones. There's a branch started already which has this change. Was just investigating if I could do anything with it easily

[chramos]

same issue here

[Duduzera1997]

+1

[rhsalas3]

+1

Specifically, trying to install the 16.4 runtime

[dourgulf]

when contentType is cryptexDiskImage, source field will missed
So, enum ContentType should add cryptexDiskImage case. and source should change type from String to String?

[phantomato]

Hi folks, I've made a simple workaround in my fork that supports installations of older + newer runtimes https://github.com/phantomato/xcodes/releases/tag/1.4.2

At the moment I'm able to install all the runtimes including iOS 18, VisionOS 2, watchOS 11. It works by assuming that the matching non-cryptex disk image still exists at https://developer.apple.com/download/all/ and consistently follows the naming convention. Hopefully, this will last long enough until we figure out the proper way to handle cryptex disk images.

Can be tried out using brew install phantomato/patched/xcodes

[chramos]

Hi folks, I've made a simple workaround in my fork that supports installations of older + newer runtimes https://github.com/phantomato/xcodes/releases/tag/1.4.2

At the moment I'm able to install all the runtimes including iOS 18, VisionOS 2, watchOS 11. It works by assuming that the matching non-cryptex disk image still exists at https://developer.apple.com/download/all/ and consistently follows the naming convention. Hopefully, this will last long enough until we figure out the proper way to handle cryptex disk images.

Can be tried out using brew install phantomato/patched/xcodes

@phantomato I cannot install your package.

this is the error I'm receiving when I try to install it:

Last 15 lines from /Users/ec2-user/Library/Logs/Homebrew/xcodes/01.make:

unsupported argument 'westmere' to option '-march='
clang: error: unsupported argument 'westmere' to option '-march='

unsupported argument 'westmere' to option '-march='
clang: error: unsupported argument 'westmere' to option '-march='

unsupported argument 'westmere' to option '-march='
clang: error: unsupported argument 'westmere' to option '-march='

unsupported argument 'westmere' to option '-march='
clang: error: unsupported argument 'westmere' to option '-march='

Build cancelled
make: *** [xcodes] Error 1

not sure what is the issue, but I have NO issues when I install the xcodes package with brew install xcodes command

EDIT:
it worked now!
I did clone your repo and build it using the make command.
Then I moved the xcodes binary file to /usr/local/bin folder:
sudo cp ~/xcodes/.build/apple/Products/Release/xcodes /usr/local/bin

after that I manage to run xcodes runtimes install "iOS 17.2" and it worked fine!

thx a lot

[LeFatesmith]

Apple has changed the api in breaking ways a few times now, can we get an optional flag to supply the index as a file so we can unblock ourselves?

[testableapple]

Hi team, this issue is open for two weeks and, unfortunately, is a blocker for many of us. Do you have any ETA when it could be fixed for older runtimes? Thanks a million.

[LeFatesmith]

Hi team, this issue is open for two weeks and, unfortunately, is a blocker for many of us. Do you have any ETA when it could be fixed for older runtimes? Thanks a million.

Agreed, using xcodes in our CI/CD environment is blocked every time Apple changes their API, even with all binaries cached.

[nichtverstehen]

We are maintaining public macOS GitHub runners and over time found that we just can't rely on downloading the runtime installers directly from Apple.

Instead of using the xcodes tool we maintain a few scripts to install Xcode and runtimes from existing packages. I published them here in case they would be useful to anyone here:

https://github.com/namespacelabs/xcode-install

[jayshah123]

Using official step from official docs xcodebuild -downloadPlatform iOS https://developer.apple.com/documentation/xcode/installing-additional-simulator-runtimes#Install-and-manage-Simulator-runtimes-from-the-command-line seems to work I guess? Is it recommended?

[PurplePros]

@jayshah123 We use xcodebuild -downloadPlatform iOS for our CI/CD for runtimes installation and it works well. This installs the latest runtime version only though. If you need the ability to specify a specific version or to install multiple versions, then you need something like, i.e., xcodes, to download and mount to your machine.

[testableapple]

It's unfortunate that this issue has been open for quite so long. I decided to switch to ipsw.

Think this might help someone:

echo 'iOS 16.4 Simulator' | ipsw download xcode --sim

[MattKiazyk]

Thanks for your patience. 1.5.0 has been released which fixes the old runtime downloads.

[jervi]

@MattKiazyk Is it by design that installed runtimes are left mounted after the last update? This is a change of behavior from before. Example df -h:

$ df -h
Filesystem        Size    Used   Avail Capacity iused ifree %iused  Mounted on
/dev/disk2s1s1   140Gi   9.6Gi   8.6Gi    53%    404k   90M    0%   /
devfs            253Ki   253Ki     0Bi   100%     874     0  100%   /dev
/dev/disk2s6     140Gi    20Ki   8.6Gi     1%       0   90M    0%   /System/Volumes/VM
/dev/disk2s2     140Gi   5.7Gi   8.6Gi    40%     868   90M    0%   /System/Volumes/Preboot
/dev/disk2s4     140Gi    45Mi   8.6Gi     1%      52   90M    0%   /System/Volumes/Update
/dev/disk1s2     500Mi    20Ki   495Mi     1%       0  5.1M    0%   /System/Volumes/xarts
/dev/disk1s1     500Mi   124Ki   495Mi     1%      29  5.1M    0%   /System/Volumes/iSCPreboot
/dev/disk1s3     500Mi    56Ki   495Mi     1%      16  5.1M    0%   /System/Volumes/Hardware
/dev/disk2s5     140Gi   115Gi   8.6Gi    94%    684k   90M    1%   /System/Volumes/Data
map auto_home      0Bi     0Bi     0Bi   100%       0     0     -   /System/Volumes/Data/home
virtio-fs        1.0Ti   907Gi   117Gi    89%    1.9G  245M   89%   /Volumes/orka
/dev/disk4s1      16Gi    16Gi   467Mi    98%    507k  4.8M   10%   /Library/Developer/CoreSimulator/Volumes/iOS_21C62
/dev/disk6s1      16Gi    15Gi   471Mi    98%    498k  4.8M    9%   /Library/Developer/CoreSimulator/Volumes/iOS_21A342
/dev/disk8s1     9.5Gi   9.2Gi   279Mi    98%    332k  2.9M   10%   /Library/Developer/CoreSimulator/Volumes/watchOS_21S364
/dev/disk10s1     16Gi    16Gi   469Mi    98%    524k  4.8M   10%   /Library/Developer/CoreSimulator/Volumes/iOS_21F79
/dev/disk12s1     16Gi    16Gi   460Mi    98%    521k  4.7M   10%   /Library/Developer/CoreSimulator/Volumes/iOS_21E213
/dev/disk14s1    8.5Gi   8.2Gi   257Mi    98%    216k  2.6M    8%   /Library/Developer/CoreSimulator/Volumes/tvOS_21K364
/dev/disk16s1    4.3Gi   4.0Gi   279Mi    94%    328k  2.9M   10%   /Library/Developer/CoreSimulator/Volumes/watchOS_21R355
/dev/disk18s1    8.7Gi   8.4Gi   283Mi    97%    222k  2.9M    7%   /Library/Developer/CoreSimulator/Volumes/tvOS_21L224
/dev/disk20s1    4.0Gi   3.6Gi   375Mi    91%    212k  3.8M    5%   /Library/Developer/CoreSimulator/Volumes/tvOS_21J353
/dev/disk22s1    9.7Gi   9.3Gi   305Mi    97%    340k  3.1M   10%   /Library/Developer/CoreSimulator/Volumes/watchOS_21T214
/dev/disk24s1    8.7Gi   8.4Gi   284Mi    97%    223k  2.9M    7%   /Library/Developer/CoreSimulator/Volumes/tvOS_21L569
/dev/disk26s1    8.3Gi   7.5Gi   785Mi    91%    498k  8.0M    6%   /Library/Developer/CoreSimulator/Volumes/iOS_21A328
/dev/disk28s1    9.7Gi   9.4Gi   300Mi    97%    341k  3.1M   10%   /Library/Developer/CoreSimulator/Volumes/watchOS_21T575

[uknowmeright]

This seemed to have fixed itself for us for about a month but now the same issue is happening consistently. We are running the following step in Github Actions:

 - name: Download 16.4 sim
        run: "xcodes runtimes install --keep-archive 'iOS 16.4'"

and are consistently getting the following error:

Run xcodes runtimes install --keep-archive 'iOS 16.4'
  xcodes runtimes install --keep-archive 'iOS 16.4'
  shell: /bin/bash -e {0}
  env:
    GITHUB_RUN_URL: xxxx
    MD_APPLE_SDK_ROOT: /Applications/Xcode_15.4.app
Error: Failed executing: `/usr/bin/xcrun simctl runtime add /Users/runner/Downloads/iOS_16.4_Simulator_Runtime.dmg` (1[4]
Installing Runtime
Error: Process completed with exit code 1.

Is this still failing for anyone else?

[testableapple]

@uknowmeright, I faced the same problem on GitHub Actions. It seems like they recently (literally last week) reduced the size of available disk space, so you can't download/mount an additional iOS Runtime.

I solved this by removing the existing simulators before installing the required runtime, e.g.:

- name: Download 16.4 sim
  run: |
    sudo rm -rfv ~/Library/Developer/CoreSimulator/* || true
    sudo xcodes runtimes install --keep-archive 'iOS 16.4'

In other words, now you just need to free some space before downloading the runtimes on GitHub Actions. Hope this helps.

[alvar-bolt]

For people who visit this thread in search of ways to download iOS 18 simulators.

Xcode 16.1 Beta 3 added support for downloading specific simulators:

Xcodebuild’s download platforms command now supports an optional -buildVersion argument to request a specific released simulator runtime. The can be a build update number or the release number. Simulator runtime version that are supported versions are iOS 16.0+, watchOS 9.0+, tvOS 16+, visionOS 1.0+.

Example usage, to download, install and export the iOS 16.0 simulator runtime:

xcodebuild -downloadPlatform iOS -buildVersion 16.0 -exportPath ./SimRuntimeImages/ xcodebuild -downloadPlatform iOS -buildVersion 20A360 -exportPath ./SimRuntimeImages/ (135989441) (FB15126412)

https://developer.apple.com/documentation/xcode-release-notes/xcode-16_1-release-notes