From 28ecd8fa136d1faa93ffcd5c03b83fcd60893a20 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Mon, 6 Feb 2023 18:30:02 +0000 Subject: [PATCH 01/22] Expand multiple sections, update naming of JSON props --- .../0387-cross-compilation-destinations.md | 270 ++++++++++++------ 1 file changed, 190 insertions(+), 80 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 88be2da0f9..783be38019 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -6,8 +6,10 @@ - Review Manager: [Mishal Shah](https://github.com/shahmishal) - Status: **Active Review (January 31st...Feburary 14th, 2023**) - Implementation: [apple/swift-package-manager#5911](https://github.com/apple/swift-package-manager/pull/5911), - [apple/swift-package-manager#5922](https://github.com/apple/swift-package-manager/pull/5922), [apple/swift-package-manager#6023](https://github.com/apple/swift-package-manager/pull/6023) -- Review: ([pitch](https://forums.swift.org/t/pitch-cross-compilation-destination-bundles/61777)) ([review](https://forums.swift.org/t/se-0387-cross-compilation-destination-bundles/62875)) + [apple/swift-package-manager#5922](https://github.com/apple/swift-package-manager/pull/5922), + [apple/swift-package-manager#6023](https://github.com/apple/swift-package-manager/pull/6023) +- Review: ([pitch](https://forums.swift.org/t/pitch-cross-compilation-destination-bundles/61777)) + ([review](https://forums.swift.org/t/se-0387-cross-compilation-destination-bundles/62875)) ## Introduction @@ -30,8 +32,9 @@ triple" terms in this proposal. ## Motivation -Swift cross-compilation (CC) destinations are currently produced on an ad-hoc basis for different combinations of build-time -and run-time triples. For example, scripts that produce macOS → Linux CC destinations were created by both [the Swift +Swift cross-compilation (CC) destinations are currently produced on an ad-hoc basis for different combinations of +build-time and run-time triples. For example, scripts that produce macOS → Linux CC destinations were created by both +[the Swift team](https://github.com/apple/swift-package-manager/blob/swift-5.7-RELEASE/Utilities/build_ubuntu_cross_compilation_toolchain) and [the Swift community](https://github.com/SPMDestinations/homebrew-tap). At the same time, the distribution process of CC destinations is cumbersome. After building a destination tree on the file system, required metadata files rely on @@ -65,6 +68,9 @@ with tools supplied in `.artifactbundle` s installed by `swift destination` invo When the user runs `swift build` with the selected CC destination, the overriding tools from the corresponding bundle are invoked by `swift build` instead of tools from the top-level toolchain. +The proposal is intentionally limited in scope to build-time experience and specifies only configuration metadata, basic +directory layout for proposed artifact bundles, and some CLI helpers to operate on those. + ## Detailed design ### CC Destination Artifact Bundles @@ -80,85 +86,93 @@ The proposed structure of artifact bundles containing CC destinations looks like .artifactbundle ├ info.json ├ -│ ├ -│ │ ├ destination.json -│ │ └ -│ └ -│ ├ destination.json -│ └ +│ ├ destination.json +│ ├ toolset.json +│ └ -│ └ -│ ├ destination.json -│ └ +│ ├ destination.json +│ ├ toolset.json +│ └ ┆ └┄ ``` -For example, a destination bundle allowing to cross-compile Swift 5.7 source code to recent versions of Ubuntu from +For example, a destination bundle allowing to cross-compile Swift 5.8 source code to recent versions of Ubuntu from macOS would look like this: ``` -swift-5.7_ubuntu.artifactbundle +swift-5.8_ubuntu.artifactbundle ├ info.json ├ ubuntu_jammy -│ ├ arm64-apple-darwin -│ │ ├ destination.json +│ ├ destination.json +│ ├ toolset.json +│ ├ +│ ├ aarch64-unknown-linux-gnu │ │ ├ toolset.json -│ │ └ -│ └ x86_64-apple-darwin -│ ├ destination.json +│ │ └ +│ └ x86_64-unknown-linux-gnu │ ├ toolset.json -│ └ +│ └ ├ ubuntu_focal -│ └ x86_64-apple-darwin -│ ├ destination.json +│ ├ destination.json +│ └ x86_64-unknown-linux-gnu │ ├ toolset.json -│ └ +│ └ ├ ubuntu_bionic ┆ └┄ ``` -Here each artifact directory is dedicated to a specific CC destination, while binaries for a specific host platform are -placed in `arm64-apple-darwin` and `x86_64-apple-darwin` subdirectories. - -`info.json` bundle manifests at the root of artifact bundles should specify `"type": "crossCompilationDestination"` -for corresponding artifacts. Artifact identifiers in this manifest uniquely identify a CC destination. The rest of the -properties of bundle manifests introduced in SE-0305 are preserved. +Here each artifact directory is dedicated to a specific CC destination, while files specific to each triple are placed +in `arm64-apple-darwin` and `x86_64-apple-darwin` subdirectories. -### `destination.json` files +`info.json` bundle manifests at the root of artifact bundles should specify `"type": "crossCompilationDestination"` for +corresponding artifacts. Artifact identifiers in this manifest file uniquely identify a CC destination. +`supportedTriples` property in `info.json` should contain build-time triples that a given destination supports. An +optional `toolset` property may be supplied that specifies a toolset for the whole bundle, see [the corresponding +section below](#toolsetjson-files) for more details. The rest of the properties of bundle manifests introduced in +SE-0305 are preserved. -Note the presence of `destination.json` files in each `` subdirectory. These files should contain a JSON -dictionary with an evolved version of the schema of [existing `destination.json` files that SwiftPM already -supports](https://github.com/apple/swift-package-manager/pull/1098) and `destination.json` files presented in the pitch -version of this proposal, hence `"schemaVersion": "3.0"`: +Here's how `info.json` file could look like for `swift-5.8_ubuntu.artifactbundle` introduced in the example +above: ```json { - "schemaVersion": "3.0", - "buildTimeTriples": [""], - "runTimeTriples": [""], - "swiftResourcesPaths": [""], - "includeSearchPaths": [""], - "librarySearchPaths": [""], + "artifacts" : { + "swift-5.8_ubuntu22.04" : { + "type" : "crossCompilationDestination", + "version" : "0.0.1", + "variants" : [ + { + "path" : "ubuntu_jammy", + "supportedTriples" : [ + "arm64-apple-darwin", + "x86_64-apple-darwin" + ] + } + ] + }, + "swift-5.8_ubuntu20.04" : { + "type" : "crossCompilationDestination", + "version" : "0.0.1", + "variants" : [ + { + "path" : "ubuntu_focal", + "supportedTriples" : [ + "arm64-apple-darwin", + "x86_64-apple-darwin" + ] + } + ] + } + }, + "schemaVersion" : "1.0" } ``` -Thanks to the availability of Universal Binaries on macOS and multiarch layouts on Linux, `buildTimeTriples` and -`runTimeTriples` properties use plural in their naming and their values are arrays. - -We propose that all relative paths in `destination.json` files should be validated not to "escape" the destination -bundle for security reasons. That is, `../` components, if present in paths, will not be allowed to reference files and -directories outside of a corresponding destination bundle. Symlinks will also be validated to prevent them from escaping -out of the bundle. - -Since not all platforms can support self-contained destination bundles, users will be able to provide their own -additional paths on the filesystem outside of bundles after a destination is installed. The exact options for specifying -paths are proposed in a subsequent section for a newly introduced `swift destination configure` command. - ### `toolset.json` files We find that properties dedicated to tools configuration are useful outside of the cross-compilation context. Due to -that, a separate `toolset.json` file is introduced: +that, separate toolset configuration files are introduced: ```json5 { @@ -167,44 +181,60 @@ that, a separate `toolset.json` file is introduced: // If `toolsetRootPath` is specified, all relative paths below will be resolved relative to `toolsetRootPath`. "swiftCompiler": { "path": "", - "extraFlags": [""] + "extraCLIOptions": [""] }, "cCompiler": { "path": "", - "extraFlags": [""] + "extraCLIOptions": [""] }, "cxxCompiler": { "path": "", - "extraFlags": [""] + "extraCLIOptions": [""] }, "linker": { "path": "", - "extraFlags": [""] - } + "extraCLIOptions": [""] + }, + "librarian": { + "path": "", + "extraCLIOptions": [""] + }, + "debugger": { + "path": "", + "extraCLIOptions": [""] + }, + "testRunner": { + "path": "", + "extraCLIOptions": [""] + }, } ``` +More types of tools may be enabled in toolset files in the future in addition to those listed above. + Users familiar with CMake can draw an analogy between toolset files and CMake toolchain files. Toolset files are -designed to supplant previous ad-hoc ways to specify paths and flags in SwiftPM, such as `SWIFT_EXEC` and `CC` +designed to supplant previous ad-hoc ways of specifying paths and flags in SwiftPM, such as `SWIFT_EXEC` and `CC` environment variables, which were applied in use cases unrelated to cross-compilation. We propose that -users should be able to pass `--toolset ` option to both `swift build` and `swift test`. +users also should be able to pass `--toolset ` option to `swift build`, `swift test`, and +`swift run`. We'd like to allow using multiple toolset files at once. With this users can "assemble" toolchains on the fly out of -tools that in certain scenarios may even come from different vendors. +tools that in certain scenarios may even come from different vendors. A toolset file can have an arbitrary name, and +each file should be passed with a separate `--toolset` option, i.e. `swift build --toolset t1.json --toolset t2.json`. -All of the properties related to names of the tools are optional, which also allows stacking multiple toolset files. -For example, consider `toolset1.json`: +All of the properties related to names of the tools are optional, which allows merging configuration from multiple +toolset files. For example, consider `toolset1.json`: ```json { "schemaVersion": "1.0", "swiftCompiler": { "path": "/usr/bin/swiftc", - "extraFlags": ["-Xfrontend", "-enable-cxx-interop"] + "extraCLIOptions": ["-Xfrontend", "-enable-cxx-interop"] }, "cCompiler": { "path": "/usr/bin/clang", - "extraFlags": ["-pedantic"] + "extraCLIOptions": ["-pedantic"] } } ``` @@ -215,27 +245,101 @@ and `toolset2.json`: { "schemaVersion": "1.0", "swiftCompiler": { - "path": "/custom/swiftc", - "extraFlags": [] + "path": "/custom/swiftc" } } ``` - With multiple `--toolset` options, passing both of those files will merge them into a single configuration. Tools passed in subsequent `--toolset` options will shadow tools from previous options with the same names. That is, `swift build --toolset toolset1.json --toolset toolset2.json` will build with `/custom/swiftc` and no extra flags, as specified in `toolset2.json`, but `/usr/bin/clang -pedantic` from `toolset1.json` will still be used. Tools not specified in any of the supplied toolset files will be looked up in existing implied search paths that are -used without any presence of toolset files, even when `toolsetRootPath` is present in any toolset file. We'd like -toolsets to be explicit enough in this regard: if any tool would like to participate in toolsets path lookups, it must -have its path provided, either relative or absolute, in any of the provided toolset files. +used without toolsets, even when `toolsetRootPath` is present. We'd like toolsets to be explicit in this regard: if a +tool would like to participate in toolset path lookups, it must provide either a relative or an absolute path in a +toolset. -When cross-compiling, paths in `toolset.json` files supplied in destination artifact bundles should be self-contained -with same rules applied as in `destination.json`: no absolute paths and no escaping symlinks are allowed. Users are -still able to provide their own `toolset.json` files outside of artifact bundles to specify additional developer tools -for which no relative path can be provided within the bundle. +Tools that don't have `path` property but have `extraCLIOptions` present will append options from that property to a +tool with the same name specified in a preceding toolset file. If no other toolset files were provided, these options +will be appended to the default tool invocation. For example `pedanticCCompiler.json` that looks like this + +```json +{ + "schemaVersion": "1.0", + "cCompiler": { + "extraCLIOptions": ["-pedantic"] + } +} +``` + +in `swift build --toolset pedanticCCompiler` will pass `-pedantic` to the C compiler located at a default path. + +When cross-compiling, paths in `toolset.json` files supplied in destination artifact bundles should be self-contained: +no absolute paths and no escaping symlinks are allowed. Users are still able to provide their own `toolset.json` files +outside of artifact bundles to specify additional developer tools for which no relative "non-escaping" path can be +provided within the bundle. + +### `destination.json` files + +Note the presence of `destination.json` files in each `` subdirectory. These files should contain +a JSON dictionary with an evolved version of the schema of [existing `destination.json` files that SwiftPM already +supports](https://github.com/apple/swift-package-manager/pull/1098) and `destination.json` files presented in the pitch +version of this proposal, hence `"schemaVersion": "3.0"`: + +```json +{ + "schemaVersion": "3.0", + "runTimeTriples": [ + "": { + "swiftResourcesPaths": [""], + "includeSearchPaths": [""], + "librarySearchPaths": [""], + "toolsetPaths": [""] + }, + // a destination can support more than one run-time triple: + "": { + "swiftResourcesPaths": [""], + "includeSearchPaths": [""], + "librarySearchPaths": [""], + "toolsetPath": [""] + } + // more triples can be supported by a single destination if needed, primarily for sharing files between them. + ] +} +``` + +We propose that all relative paths in `destination.json` files should be validated not to "escape" the destination +bundle for security reasons, in the same way that `toolset.json` files are validated when contained in destination +bundles. That is, `../` components, if present in paths, will not be allowed to reference files and +directories outside of a corresponding destination bundle. Symlinks will also be validated to prevent them from escaping +out of the bundle. + +Here's `destination.json` file for the `ubuntu_jammy` artifact previously introduced as an example: + +```json +{ + "schemaVersion": "3.0", + "runTimeTriples": [ + "aarch64-unknown-linux-gnu": { + "swiftResourcesPaths": ["aarch64-unknown-linux-gnu/usr/lib/swift"], + "includeSearchPaths": ["aarch64-unknown-linux-gnu/usr/include"], + "librarySearchPaths": ["aarch64-unknown-linux-gnu/usr/lib"], + "toolsetPath": "aarch64-unknown-linux-gnu/toolset.json" + }, + "x86_64-apple-darwin": { + "swiftResourcesPaths": ["x86_64-unknown-linux-gnu/usr/lib/swift"], + "includeSearchPaths": ["x86_64-unknown-linux-gnu/usr/include"], + "librarySearchPaths": ["x86_64-unknown-linux-gnu/usr/lib"], + "toolsetPath": "x86_64-unknown-linux-gnu/toolset.json" + } + ], +} +``` + +Since not all platforms can support self-contained destination bundles, users will be able to provide their own +additional paths on the filesystem outside of bundles after a destination is installed. The exact options for specifying +paths are proposed in a subsequent section for a newly introduced `swift destination configure` command. ### Destination Bundle Installation and Configuration @@ -245,7 +349,13 @@ To manage CC destinations, we'd like to introduce a new `swift destination` comm installs it in a location discoverable by SwiftPM. For destinations installed from remote URLs an additional `--checksum` option is required, through which users of destinations can specify a checksum provided by publishers of destinations. The latter can produce a checksum by running `swift package compute-checksum` command (introduced in - SE-0272) with the destination artifact bundle archive as an argument. + [SE-0272](https://github.com/apple/swift-evolution/blob/main/proposals/0272-swiftpm-binary-dependencies.md)) with the + destination artifact bundle archive as an argument. + + If a destination with a given artifact ID has already been installed and its version is equal or higher to a version + of a new destination, an error message will be printed. If the new version is higher, users should invoke the + `install` subcommand with `--update` flag to allow updating an already installed destination artifact to a new + version. - `swift destination list`, which prints a list of already installed CC destinations with their identifiers. - `swift destination configure `, which allows users to provide additional search paths and toolsets to be used subsequently when building with a given destination. Specifically, multiple `--swift-resources-path`, @@ -260,7 +370,7 @@ which then will be stored as configuration for this destination. After a destination is installed, users can refer to it via its identifier passed to the `--destination` option, e.g. ``` -swift build --destination ubuntu-jammy +swift build --destination ubuntu_jammy ``` We'd also like to make `--destination` flexible enough to recognize run-time triples when there's only a single CC @@ -288,7 +398,7 @@ As an example, destination publishers looking to add a library to an Ubuntu 22.0 `Dockerfile` similar to this one in CC destination generator source code: ```dockerfile -FROM swift:5.7-jammy +FROM swift:5.8-jammy apt-get install -y \ # PostgreSQL library provided as an example. @@ -375,7 +485,7 @@ publish their own artifact bundles with executables, as defined in SE-0305. Platform triples are not specific enough in certain cases. For example, `aarch64-unknown-linux` host triple can’t prevent a user from installing a CC destination bundle on an unsupported Linux distribution. In the future we could -deprecate `hostTriple` and `destinationTriple` JSON properties in favor of dictionaries with keys and values that +deprecate `buildTimeTriples` and `runTimeTriples` JSON properties in favor of dictionaries with keys and values that describe aspects of platforms that are important for destinations. Such dictionaries could look like this: ```json5 From e96b917666b4b17050a7db55d07c9da0981bc31f Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Mon, 6 Feb 2023 19:23:05 +0000 Subject: [PATCH 02/22] Mention package registries in the Future Directions section --- .../0387-cross-compilation-destinations.md | 30 +++++++++++-------- 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 783be38019..b84e3db4ee 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -51,7 +51,7 @@ project is a daunting step that shouldn’t be required. The solution described below is general enough to scale for any build-time/run-time triple combination. -## Proposed solution +## Proposed Solution Since CC destination is a collection of binaries arranged in a certain directory hierarchy, it makes sense to distribute it as an archive. We'd like to build on top of @@ -71,7 +71,7 @@ are invoked by `swift build` instead of tools from the top-level toolchain. The proposal is intentionally limited in scope to build-time experience and specifies only configuration metadata, basic directory layout for proposed artifact bundles, and some CLI helpers to operate on those. -## Detailed design +## Detailed Design ### CC Destination Artifact Bundles @@ -103,6 +103,7 @@ macOS would look like this: ``` swift-5.8_ubuntu.artifactbundle ├ info.json +├ toolset.json ├ ubuntu_jammy │ ├ destination.json │ ├ toolset.json @@ -126,11 +127,9 @@ Here each artifact directory is dedicated to a specific CC destination, while fi in `arm64-apple-darwin` and `x86_64-apple-darwin` subdirectories. `info.json` bundle manifests at the root of artifact bundles should specify `"type": "crossCompilationDestination"` for -corresponding artifacts. Artifact identifiers in this manifest file uniquely identify a CC destination. -`supportedTriples` property in `info.json` should contain build-time triples that a given destination supports. An -optional `toolset` property may be supplied that specifies a toolset for the whole bundle, see [the corresponding -section below](#toolsetjson-files) for more details. The rest of the properties of bundle manifests introduced in -SE-0305 are preserved. +corresponding artifacts. Artifact identifiers in this manifest file uniquely identify a CC destination, and +`supportedTriples` property in `info.json` should contain build-time triples that a given destination supports. The rest +of the properties of bundle manifests introduced in SE-0305 are preserved. Here's how `info.json` file could look like for `swift-5.8_ubuntu.artifactbundle` introduced in the example above: @@ -141,6 +140,7 @@ above: "swift-5.8_ubuntu22.04" : { "type" : "crossCompilationDestination", "version" : "0.0.1", + "toolset": "toolset.json", "variants" : [ { "path" : "ubuntu_jammy", @@ -169,7 +169,7 @@ above: } ``` -### `toolset.json` files +### `toolset.json` Files We find that properties dedicated to tools configuration are useful outside of the cross-compilation context. Due to that, separate toolset configuration files are introduced: @@ -280,7 +280,7 @@ no absolute paths and no escaping symlinks are allowed. Users are still able to outside of artifact bundles to specify additional developer tools for which no relative "non-escaping" path can be provided within the bundle. -### `destination.json` files +### `destination.json` Files Note the presence of `destination.json` files in each `` subdirectory. These files should contain a JSON dictionary with an evolved version of the schema of [existing `destination.json` files that SwiftPM already @@ -415,7 +415,7 @@ ready, the generator copies files from the image to a corresponding `.artifactbu The proposed `--checksum` flag provides basic means of verifying destination bundle's validity. As a future direction, we'd like to consider sandboxing and codesigning toolchains running on macOS. -## Impact on existing packages +## Impact on Existing Packages This is an additive change with no impact on existing packages. @@ -507,7 +507,7 @@ After an application is built with a CC destination, there are other development introduce new types of plugins invoked by `swift run` and `swift test` for purposes of remote running, debugging, and testing. For Linux run-time triples, these plugins could delegate to Docker for running produced executables. -### `swift destination select` subcommand +### `swift destination select` Subcommand While `swift destination select` subcommand or a similar one make sense for selecting a CC destination instead of passing `--destination` to `swift build` every time, users will expect `swift run` and `swift test` to also work for any @@ -515,7 +515,7 @@ destination previously passed to `swift destination select`. That’s out of sco depends on making plugins (from the previous subsection) or some other remote running and testing implementation to fully work. -### SwiftPM and SourceKit-LSP improvements +### SwiftPM and SourceKit-LSP Improvements It is a known issue that SwiftPM can’t run multiple concurrent builds for different run-time triples. This may cause issues when SourceKit-LSP is building a project for indexing purposes (for a host platform by default), while a user may @@ -535,3 +535,9 @@ language](https://andrewkelley.me/post/zig-cc-powerful-drop-in-replacement-gcc-c cross-compilation destination binaries are produced on the fly when needed. We don't consider this option to be mutually exclusive with solutions proposed in this document, and so it could be explored in the future for Swift as well. However, this requires reducing the number of dependencies that Swift runtime and core libraries have. + +### Destination Bundles and Package Registries + +Since `info.json` manifest files contained within bundles contain versions, it would make sense to host destination +bundles at package registries. Althouhg, it remains to be seen whether it makes sense for an arbitrary SwiftPM package +to specify a destination bundle within its list of dependencies. From 9b560794f2de0816ea69c08321e4bd62928fe750 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Mon, 6 Feb 2023 20:12:48 +0000 Subject: [PATCH 03/22] Specify `json5` for syntax highlighting everywhere --- proposals/0387-cross-compilation-destinations.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index b84e3db4ee..6415e72c45 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -134,7 +134,7 @@ of the properties of bundle manifests introduced in SE-0305 are preserved. Here's how `info.json` file could look like for `swift-5.8_ubuntu.artifactbundle` introduced in the example above: -```json +```json5 { "artifacts" : { "swift-5.8_ubuntu22.04" : { @@ -225,7 +225,7 @@ each file should be passed with a separate `--toolset` option, i.e. `swift build All of the properties related to names of the tools are optional, which allows merging configuration from multiple toolset files. For example, consider `toolset1.json`: -```json +```json5 { "schemaVersion": "1.0", "swiftCompiler": { @@ -241,7 +241,7 @@ toolset files. For example, consider `toolset1.json`: and `toolset2.json`: -```json +```json5 { "schemaVersion": "1.0", "swiftCompiler": { @@ -264,7 +264,7 @@ Tools that don't have `path` property but have `extraCLIOptions` present will ap tool with the same name specified in a preceding toolset file. If no other toolset files were provided, these options will be appended to the default tool invocation. For example `pedanticCCompiler.json` that looks like this -```json +```json5 { "schemaVersion": "1.0", "cCompiler": { @@ -287,7 +287,7 @@ a JSON dictionary with an evolved version of the schema of [existing `destinatio supports](https://github.com/apple/swift-package-manager/pull/1098) and `destination.json` files presented in the pitch version of this proposal, hence `"schemaVersion": "3.0"`: -```json +```json5 { "schemaVersion": "3.0", "runTimeTriples": [ @@ -317,7 +317,7 @@ out of the bundle. Here's `destination.json` file for the `ubuntu_jammy` artifact previously introduced as an example: -```json +```json5 { "schemaVersion": "3.0", "runTimeTriples": [ From 1f25bc087775f2662544dcc13a433bc77ae0d160 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Tue, 7 Feb 2023 12:39:17 +0000 Subject: [PATCH 04/22] Address PR feedback --- proposals/0387-cross-compilation-destinations.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 6415e72c45..73ed37e0a0 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -302,7 +302,7 @@ version of this proposal, hence `"schemaVersion": "3.0"`: "swiftResourcesPaths": [""], "includeSearchPaths": [""], "librarySearchPaths": [""], - "toolsetPath": [""] + "toolsetPaths": [""] } // more triples can be supported by a single destination if needed, primarily for sharing files between them. ] @@ -325,13 +325,13 @@ Here's `destination.json` file for the `ubuntu_jammy` artifact previously introd "swiftResourcesPaths": ["aarch64-unknown-linux-gnu/usr/lib/swift"], "includeSearchPaths": ["aarch64-unknown-linux-gnu/usr/include"], "librarySearchPaths": ["aarch64-unknown-linux-gnu/usr/lib"], - "toolsetPath": "aarch64-unknown-linux-gnu/toolset.json" + "toolsetPaths": "aarch64-unknown-linux-gnu/toolset.json" }, "x86_64-apple-darwin": { "swiftResourcesPaths": ["x86_64-unknown-linux-gnu/usr/lib/swift"], "includeSearchPaths": ["x86_64-unknown-linux-gnu/usr/include"], "librarySearchPaths": ["x86_64-unknown-linux-gnu/usr/lib"], - "toolsetPath": "x86_64-unknown-linux-gnu/toolset.json" + "toolsetPaths": "x86_64-unknown-linux-gnu/toolset.json" } ], } @@ -485,7 +485,7 @@ publish their own artifact bundles with executables, as defined in SE-0305. Platform triples are not specific enough in certain cases. For example, `aarch64-unknown-linux` host triple can’t prevent a user from installing a CC destination bundle on an unsupported Linux distribution. In the future we could -deprecate `buildTimeTriples` and `runTimeTriples` JSON properties in favor of dictionaries with keys and values that +deprecate `supportedTriples` and `runTimeTriples` JSON properties in favor of dictionaries with keys and values that describe aspects of platforms that are important for destinations. Such dictionaries could look like this: ```json5 @@ -539,5 +539,5 @@ However, this requires reducing the number of dependencies that Swift runtime an ### Destination Bundles and Package Registries Since `info.json` manifest files contained within bundles contain versions, it would make sense to host destination -bundles at package registries. Althouhg, it remains to be seen whether it makes sense for an arbitrary SwiftPM package +bundles at package registries. Although, it remains to be seen whether it makes sense for an arbitrary SwiftPM package to specify a destination bundle within its list of dependencies. From 0b5259596a6a36139d16d8edc5c0de2bad20a2c6 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Tue, 7 Feb 2023 16:49:02 +0000 Subject: [PATCH 05/22] Address PR feedback --- proposals/0387-cross-compilation-destinations.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 73ed37e0a0..46e5e0a4df 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -292,14 +292,14 @@ version of this proposal, hence `"schemaVersion": "3.0"`: "schemaVersion": "3.0", "runTimeTriples": [ "": { - "swiftResourcesPaths": [""], + "swiftResourcesPath": "", "includeSearchPaths": [""], "librarySearchPaths": [""], "toolsetPaths": [""] }, // a destination can support more than one run-time triple: "": { - "swiftResourcesPaths": [""], + "swiftResourcesPath": "", "includeSearchPaths": [""], "librarySearchPaths": [""], "toolsetPaths": [""] @@ -322,16 +322,16 @@ Here's `destination.json` file for the `ubuntu_jammy` artifact previously introd "schemaVersion": "3.0", "runTimeTriples": [ "aarch64-unknown-linux-gnu": { - "swiftResourcesPaths": ["aarch64-unknown-linux-gnu/usr/lib/swift"], + "swiftResourcesPath": "aarch64-unknown-linux-gnu/usr/lib/swift", "includeSearchPaths": ["aarch64-unknown-linux-gnu/usr/include"], "librarySearchPaths": ["aarch64-unknown-linux-gnu/usr/lib"], - "toolsetPaths": "aarch64-unknown-linux-gnu/toolset.json" + "toolsetPaths": ["aarch64-unknown-linux-gnu/toolset.json"] }, "x86_64-apple-darwin": { - "swiftResourcesPaths": ["x86_64-unknown-linux-gnu/usr/lib/swift"], + "swiftResourcesPath": "x86_64-unknown-linux-gnu/usr/lib/swift", "includeSearchPaths": ["x86_64-unknown-linux-gnu/usr/include"], "librarySearchPaths": ["x86_64-unknown-linux-gnu/usr/lib"], - "toolsetPaths": "x86_64-unknown-linux-gnu/toolset.json" + "toolsetPaths": ["x86_64-unknown-linux-gnu/toolset.json"] } ], } From 3d2a1c2be03d5702c5327c8d0c3718fcbf1b78f0 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Thu, 9 Feb 2023 16:50:38 +0000 Subject: [PATCH 06/22] Address PR feedback --- .../0387-cross-compilation-destinations.md | 28 ++++++++++--------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 46e5e0a4df..2205ed72a0 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -124,7 +124,7 @@ swift-5.8_ubuntu.artifactbundle ``` Here each artifact directory is dedicated to a specific CC destination, while files specific to each triple are placed -in `arm64-apple-darwin` and `x86_64-apple-darwin` subdirectories. +in `aarch64-unknown-linux-gnu` and `x86_64-unknown-linux-gnu` subdirectories. `info.json` bundle manifests at the root of artifact bundles should specify `"type": "crossCompilationDestination"` for corresponding artifacts. Artifact identifiers in this manifest file uniquely identify a CC destination, and @@ -292,17 +292,19 @@ version of this proposal, hence `"schemaVersion": "3.0"`: "schemaVersion": "3.0", "runTimeTriples": [ "": { + "sdkRootPath": "", "swiftResourcesPath": "", - "includeSearchPaths": [""], - "librarySearchPaths": [""], - "toolsetPaths": [""] + "includeSearchPaths": [""], + "librarySearchPaths": [""], + "toolsetPaths": [""] }, // a destination can support more than one run-time triple: "": { + "sdkRootPath": "", "swiftResourcesPath": "", - "includeSearchPaths": [""], - "librarySearchPaths": [""], - "toolsetPaths": [""] + "includeSearchPaths": [""], + "librarySearchPaths": [""], + "toolsetPaths": [""] } // more triples can be supported by a single destination if needed, primarily for sharing files between them. ] @@ -314,6 +316,10 @@ bundle for security reasons, in the same way that `toolset.json` files are valid bundles. That is, `../` components, if present in paths, will not be allowed to reference files and directories outside of a corresponding destination bundle. Symlinks will also be validated to prevent them from escaping out of the bundle. + +If `sdkRootPath` is specified and `swiftResourcesPath` is not, the latter is inferred to be +`"\(sdkRootPath)/usr/lib/swift"`. Similarly `includeSearchPaths` is inferred as `["\(sdkRootPath)/usr/include"]`, +`librarySearchPaths` as `["\(sdkRootPath)/usr/lib"]`. Here's `destination.json` file for the `ubuntu_jammy` artifact previously introduced as an example: @@ -322,15 +328,11 @@ Here's `destination.json` file for the `ubuntu_jammy` artifact previously introd "schemaVersion": "3.0", "runTimeTriples": [ "aarch64-unknown-linux-gnu": { - "swiftResourcesPath": "aarch64-unknown-linux-gnu/usr/lib/swift", - "includeSearchPaths": ["aarch64-unknown-linux-gnu/usr/include"], - "librarySearchPaths": ["aarch64-unknown-linux-gnu/usr/lib"], + "sdkRootPath": "aarch64-unknown-linux-gnu/ubuntu-jammy.sdk", "toolsetPaths": ["aarch64-unknown-linux-gnu/toolset.json"] }, "x86_64-apple-darwin": { - "swiftResourcesPath": "x86_64-unknown-linux-gnu/usr/lib/swift", - "includeSearchPaths": ["x86_64-unknown-linux-gnu/usr/include"], - "librarySearchPaths": ["x86_64-unknown-linux-gnu/usr/lib"], + "sdkRootPath": "x86_64-unknown-linux-gnu/ubuntu-jammy.sdk", "toolsetPaths": ["x86_64-unknown-linux-gnu/toolset.json"] } ], From a9a77c10f2ae948b888980dc48cbce949b69065b Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Thu, 9 Feb 2023 17:02:58 +0000 Subject: [PATCH 07/22] Make `sdkRootPath` required --- proposals/0387-cross-compilation-destinations.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 2205ed72a0..51ae291e0d 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -292,7 +292,7 @@ version of this proposal, hence `"schemaVersion": "3.0"`: "schemaVersion": "3.0", "runTimeTriples": [ "": { - "sdkRootPath": "", + "sdkRootPath": "", "swiftResourcesPath": "", "includeSearchPaths": [""], "librarySearchPaths": [""], @@ -300,7 +300,7 @@ version of this proposal, hence `"schemaVersion": "3.0"`: }, // a destination can support more than one run-time triple: "": { - "sdkRootPath": "", + "sdkRootPath": "", "swiftResourcesPath": "", "includeSearchPaths": [""], "librarySearchPaths": [""], From f1851cf68b912403bafd25e12fdd5ae58d19531c Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Fri, 10 Feb 2023 12:04:40 +0000 Subject: [PATCH 08/22] Address PR feedback --- proposals/0387-cross-compilation-destinations.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 51ae291e0d..6b4774f7ec 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -140,7 +140,6 @@ above: "swift-5.8_ubuntu22.04" : { "type" : "crossCompilationDestination", "version" : "0.0.1", - "toolset": "toolset.json", "variants" : [ { "path" : "ubuntu_jammy", @@ -285,7 +284,9 @@ provided within the bundle. Note the presence of `destination.json` files in each `` subdirectory. These files should contain a JSON dictionary with an evolved version of the schema of [existing `destination.json` files that SwiftPM already supports](https://github.com/apple/swift-package-manager/pull/1098) and `destination.json` files presented in the pitch -version of this proposal, hence `"schemaVersion": "3.0"`: +version of this proposal, hence `"schemaVersion": "3.0"`. We'll keep parsing `"version": 1` and `"version": 2` for +backward compatibility, but for consistency with `info.json` this field is renamed to `"schemaVersion"`. Here's an +informally defined schema for these files: ```json5 { @@ -331,7 +332,7 @@ Here's `destination.json` file for the `ubuntu_jammy` artifact previously introd "sdkRootPath": "aarch64-unknown-linux-gnu/ubuntu-jammy.sdk", "toolsetPaths": ["aarch64-unknown-linux-gnu/toolset.json"] }, - "x86_64-apple-darwin": { + "x86_64-unknown-linux-gnu": { "sdkRootPath": "x86_64-unknown-linux-gnu/ubuntu-jammy.sdk", "toolsetPaths": ["x86_64-unknown-linux-gnu/toolset.json"] } From be0b74b21f2cc75bc8af8c150adb7186671ab016 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Fri, 10 Feb 2023 14:58:46 +0000 Subject: [PATCH 09/22] Disambiguate `swift build --destination` invocation example --- proposals/0387-cross-compilation-destinations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 6b4774f7ec..f2eda13987 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -373,7 +373,7 @@ which then will be stored as configuration for this destination. After a destination is installed, users can refer to it via its identifier passed to the `--destination` option, e.g. ``` -swift build --destination ubuntu_jammy +swift build --destination ubuntu_focal ``` We'd also like to make `--destination` flexible enough to recognize run-time triples when there's only a single CC From cf2deb6e3d5ee5ba4e3ed8cccfbaf91177bd8226 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Fri, 10 Feb 2023 16:28:10 +0000 Subject: [PATCH 10/22] Add `swiftStaticResourcesPath` field to `destination.json` --- .../0387-cross-compilation-destinations.md | 27 +++++++++++-------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index f2eda13987..e307c5c20e 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -294,18 +294,22 @@ informally defined schema for these files: "runTimeTriples": [ "": { "sdkRootPath": "", - "swiftResourcesPath": "", - "includeSearchPaths": [""], - "librarySearchPaths": [""], - "toolsetPaths": [""] + // all of the properties listed below are optional: + "swiftResourcesPath": "", + "swiftStaticResourcesPath": "", + "includeSearchPaths": [""], + "librarySearchPaths": [""], + "toolsetPaths": [""] }, // a destination can support more than one run-time triple: "": { "sdkRootPath": "", - "swiftResourcesPath": "", - "includeSearchPaths": [""], - "librarySearchPaths": [""], - "toolsetPaths": [""] + // all of the properties listed below are optional: + "swiftResourcesPath": "", + "swiftStaticResourcesPath": "", + "includeSearchPaths": [""], + "librarySearchPaths": [""], + "toolsetPaths": [""] } // more triples can be supported by a single destination if needed, primarily for sharing files between them. ] @@ -318,9 +322,10 @@ bundles. That is, `../` components, if present in paths, will not be allowed to directories outside of a corresponding destination bundle. Symlinks will also be validated to prevent them from escaping out of the bundle. -If `sdkRootPath` is specified and `swiftResourcesPath` is not, the latter is inferred to be -`"\(sdkRootPath)/usr/lib/swift"`. Similarly `includeSearchPaths` is inferred as `["\(sdkRootPath)/usr/include"]`, -`librarySearchPaths` as `["\(sdkRootPath)/usr/lib"]`. +If `sdkRootPath` is specified and `swiftResourcesPath` is not, the latter is inferred to be +`"\(sdkRootPath)/usr/lib/swift"` when linking the Swift standard library dynamically, `"swiftStaticResourcesPath"` is +inferred to be `"\(sdkRootPath)/usr/lib/swift_static"` when linking it dynamically. Similarly, `includeSearchPaths` is +inferred as `["\(sdkRootPath)/usr/include"]`, `librarySearchPaths` as `["\(sdkRootPath)/usr/lib"]`. Here's `destination.json` file for the `ubuntu_jammy` artifact previously introduced as an example: From 5f39655195ebec929e45c67aa7eefdb0d50d3da0 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Fri, 10 Feb 2023 16:34:58 +0000 Subject: [PATCH 11/22] Clean up formatting --- proposals/0387-cross-compilation-destinations.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index e307c5c20e..20842f4439 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -293,7 +293,7 @@ informally defined schema for these files: "schemaVersion": "3.0", "runTimeTriples": [ "": { - "sdkRootPath": "", + "sdkRootPath": "", // all of the properties listed below are optional: "swiftResourcesPath": "", "swiftStaticResourcesPath": "", @@ -303,7 +303,7 @@ informally defined schema for these files: }, // a destination can support more than one run-time triple: "": { - "sdkRootPath": "", + "sdkRootPath": "", // all of the properties listed below are optional: "swiftResourcesPath": "", "swiftStaticResourcesPath": "", From e8cd5450e1c772ea7965e384c04f2ef2e484d724 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Fri, 10 Feb 2023 17:33:11 +0000 Subject: [PATCH 12/22] Fix typo --- proposals/0387-cross-compilation-destinations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 20842f4439..91b32765a0 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -324,7 +324,7 @@ out of the bundle. If `sdkRootPath` is specified and `swiftResourcesPath` is not, the latter is inferred to be `"\(sdkRootPath)/usr/lib/swift"` when linking the Swift standard library dynamically, `"swiftStaticResourcesPath"` is -inferred to be `"\(sdkRootPath)/usr/lib/swift_static"` when linking it dynamically. Similarly, `includeSearchPaths` is +inferred to be `"\(sdkRootPath)/usr/lib/swift_static"` when linking it statically. Similarly, `includeSearchPaths` is inferred as `["\(sdkRootPath)/usr/include"]`, `librarySearchPaths` as `["\(sdkRootPath)/usr/lib"]`. Here's `destination.json` file for the `ubuntu_jammy` artifact previously introduced as an example: From 5d8bec28024849a9d7b3de22a45c7e10df25d229 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Tue, 21 Feb 2023 13:45:19 +0000 Subject: [PATCH 13/22] Expand "Alternatives Considered" --- proposals/0387-cross-compilation-destinations.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 91b32765a0..7c9580505b 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -487,6 +487,18 @@ Different formats of destination bundles can be considered, but we don't think t from the proposed one. If they were different, this would complicate bundle distribution scenarios for users who want to publish their own artifact bundles with executables, as defined in SE-0305. +## Making Destination Bundles Fully Self-Contained + +Some users expressed interest in self-contained destination bundles that ignore the value of `PATH` environment variable +and prevent launching any executables from outside of a bundle. So far in our practice we haven't seen any problems +caused by the use of executables from `PATH`. Quite the opposite, we think most destinations would want to reuse as many +tools from `PATH` as possible, which would allow making destination bundles much smaller. For example as of Swift 5.7, +on macOS `clang-13` binary takes ~360 MB, `clangd` ~150 MB, and `swift-frontend` ~420 MB. Keeping copies of these +binaries in every destination bundle seems quite redundant when existing binaries from `PATH` can be easily reused. +Additionally, we find that preventing tools from being launched from arbitrary paths can't be technically enforced +without sandboxing, and there's no cross-platform sandboxing solution available for SwiftPM. Until such sandboxing +solution is available, we'd like to keep the existing approach, where setting `PATH` behaves in a predictable way. + ## Future Directions ### Identifying Platforms with Dictionaries of Properties From d1f98e754f3311d9a284992c828e63e8008acc5d Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Tue, 21 Feb 2023 15:01:26 +0000 Subject: [PATCH 14/22] Refine wording --- proposals/0387-cross-compilation-destinations.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 7c9580505b..274edbae30 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -497,7 +497,8 @@ on macOS `clang-13` binary takes ~360 MB, `clangd` ~150 MB, and `swift-frontend` binaries in every destination bundle seems quite redundant when existing binaries from `PATH` can be easily reused. Additionally, we find that preventing tools from being launched from arbitrary paths can't be technically enforced without sandboxing, and there's no cross-platform sandboxing solution available for SwiftPM. Until such sandboxing -solution is available, we'd like to keep the existing approach, where setting `PATH` behaves in a predictable way. +solution is available, we'd like to keep the existing approach where setting `PATH` environment variable behaves in a +predictable way and is consistent with established CLI conventions. ## Future Directions From c7f5c7ce99a0d23d1e0ebbdc3fb2340193cfa7f6 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Tue, 28 Feb 2023 09:42:17 +0000 Subject: [PATCH 15/22] Add a link to implementation PR --- proposals/0387-cross-compilation-destinations.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 274edbae30..c1edc44928 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -7,7 +7,8 @@ - Status: **Active Review (January 31st...Feburary 14th, 2023**) - Implementation: [apple/swift-package-manager#5911](https://github.com/apple/swift-package-manager/pull/5911), [apple/swift-package-manager#5922](https://github.com/apple/swift-package-manager/pull/5922), - [apple/swift-package-manager#6023](https://github.com/apple/swift-package-manager/pull/6023) + [apple/swift-package-manager#6023](https://github.com/apple/swift-package-manager/pull/6023), + [apple/swift-package-manager#6186](https://github.com/apple/swift-package-manager/pull/6186) - Review: ([pitch](https://forums.swift.org/t/pitch-cross-compilation-destination-bundles/61777)) ([review](https://forums.swift.org/t/se-0387-cross-compilation-destination-bundles/62875)) From 58b2c5a3141d159874261a86a9da053e8e5361eb Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Tue, 28 Feb 2023 09:54:24 +0000 Subject: [PATCH 16/22] Fix toolset file reference in CLI invocation --- proposals/0387-cross-compilation-destinations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index c1edc44928..953e60da6c 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -273,7 +273,7 @@ will be appended to the default tool invocation. For example `pedanticCCompiler. } ``` -in `swift build --toolset pedanticCCompiler` will pass `-pedantic` to the C compiler located at a default path. +in `swift build --toolset pedanticCCompiler.json` will pass `-pedantic` to the C compiler located at a default path. When cross-compiling, paths in `toolset.json` files supplied in destination artifact bundles should be self-contained: no absolute paths and no escaping symlinks are allowed. Users are still able to provide their own `toolset.json` files From 515aac33e764a2f44114919031836dff2c146df7 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Thu, 2 Mar 2023 12:07:12 +0000 Subject: [PATCH 17/22] Add table of contents --- .../0387-cross-compilation-destinations.md | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 953e60da6c..3f71589a16 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -12,6 +12,38 @@ - Review: ([pitch](https://forums.swift.org/t/pitch-cross-compilation-destination-bundles/61777)) ([review](https://forums.swift.org/t/se-0387-cross-compilation-destination-bundles/62875)) +## Table of Contents + +- [Cross-Compilation Destination Bundles](#cross-compilation-destination-bundles) + - [Table of Contents](#table-of-contents) + - [Introduction](#introduction) + - [Motivation](#motivation) + - [Proposed Solution](#proposed-solution) + - [Detailed Design](#detailed-design) + - [CC Destination Artifact Bundles](#cc-destination-artifact-bundles) + - [`toolset.json` Files](#toolsetjson-files) + - [`destination.json` Files](#destinationjson-files) + - [Destination Bundle Installation and Configuration](#destination-bundle-installation-and-configuration) + - [Using a CC Destination](#using-a-cc-destination) + - [CC Destination Bundle Generation](#cc-destination-bundle-generation) + - [Security](#security) + - [Impact on Existing Packages](#impact-on-existing-packages) + - [Prior Art](#prior-art) + - [Rust](#rust) + - [Go](#go) + - [Alternatives Considered](#alternatives-considered) + - [Extensions Other Than `.artifactbundle`](#extensions-other-than-artifactbundle) + - [Building Applications in Docker Containers](#building-applications-in-docker-containers) + - [Alternative Bundle Formats](#alternative-bundle-formats) + - [Making Destination Bundles Fully Self-Contained](#making-destination-bundles-fully-self-contained) + - [Future Directions](#future-directions) + - [Identifying Platforms with Dictionaries of Properties](#identifying-platforms-with-dictionaries-of-properties) + - [SwiftPM Plugins for Remote Running, Testing, Deployment, and Debugging](#swiftpm-plugins-for-remote-running-testing-deployment-and-debugging) + - [`swift destination select` Subcommand](#swift-destination-select-subcommand) + - [SwiftPM and SourceKit-LSP Improvements](#swiftpm-and-sourcekit-lsp-improvements) + - [Source-Based CC Destinations](#source-based-cc-destinations) + - [Destination Bundles and Package Registries](#destination-bundles-and-package-registries) + ## Introduction Cross-compilation is a common development use case. When cross-compiling, we need to refer to these concepts: From 6ce9bc197b6f5bec42945e3b19a3891978d3cf9d Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Thu, 2 Mar 2023 12:08:00 +0000 Subject: [PATCH 18/22] Clean up table of contents --- .../0387-cross-compilation-destinations.md | 56 +++++++++---------- 1 file changed, 27 insertions(+), 29 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 3f71589a16..98de0f5c8c 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -14,35 +14,33 @@ ## Table of Contents -- [Cross-Compilation Destination Bundles](#cross-compilation-destination-bundles) - - [Table of Contents](#table-of-contents) - - [Introduction](#introduction) - - [Motivation](#motivation) - - [Proposed Solution](#proposed-solution) - - [Detailed Design](#detailed-design) - - [CC Destination Artifact Bundles](#cc-destination-artifact-bundles) - - [`toolset.json` Files](#toolsetjson-files) - - [`destination.json` Files](#destinationjson-files) - - [Destination Bundle Installation and Configuration](#destination-bundle-installation-and-configuration) - - [Using a CC Destination](#using-a-cc-destination) - - [CC Destination Bundle Generation](#cc-destination-bundle-generation) - - [Security](#security) - - [Impact on Existing Packages](#impact-on-existing-packages) - - [Prior Art](#prior-art) - - [Rust](#rust) - - [Go](#go) - - [Alternatives Considered](#alternatives-considered) - - [Extensions Other Than `.artifactbundle`](#extensions-other-than-artifactbundle) - - [Building Applications in Docker Containers](#building-applications-in-docker-containers) - - [Alternative Bundle Formats](#alternative-bundle-formats) - - [Making Destination Bundles Fully Self-Contained](#making-destination-bundles-fully-self-contained) - - [Future Directions](#future-directions) - - [Identifying Platforms with Dictionaries of Properties](#identifying-platforms-with-dictionaries-of-properties) - - [SwiftPM Plugins for Remote Running, Testing, Deployment, and Debugging](#swiftpm-plugins-for-remote-running-testing-deployment-and-debugging) - - [`swift destination select` Subcommand](#swift-destination-select-subcommand) - - [SwiftPM and SourceKit-LSP Improvements](#swiftpm-and-sourcekit-lsp-improvements) - - [Source-Based CC Destinations](#source-based-cc-destinations) - - [Destination Bundles and Package Registries](#destination-bundles-and-package-registries) +- [Introduction](#introduction) +- [Motivation](#motivation) +- [Proposed Solution](#proposed-solution) +- [Detailed Design](#detailed-design) + - [CC Destination Artifact Bundles](#cc-destination-artifact-bundles) + - [`toolset.json` Files](#toolsetjson-files) + - [`destination.json` Files](#destinationjson-files) + - [Destination Bundle Installation and Configuration](#destination-bundle-installation-and-configuration) + - [Using a CC Destination](#using-a-cc-destination) + - [CC Destination Bundle Generation](#cc-destination-bundle-generation) +- [Security](#security) +- [Impact on Existing Packages](#impact-on-existing-packages) +- [Prior Art](#prior-art) + - [Rust](#rust) + - [Go](#go) +- [Alternatives Considered](#alternatives-considered) + - [Extensions Other Than `.artifactbundle`](#extensions-other-than-artifactbundle) + - [Building Applications in Docker Containers](#building-applications-in-docker-containers) + - [Alternative Bundle Formats](#alternative-bundle-formats) +- [Making Destination Bundles Fully Self-Contained](#making-destination-bundles-fully-self-contained) +- [Future Directions](#future-directions) + - [Identifying Platforms with Dictionaries of Properties](#identifying-platforms-with-dictionaries-of-properties) + - [SwiftPM Plugins for Remote Running, Testing, Deployment, and Debugging](#swiftpm-plugins-for-remote-running-testing-deployment-and-debugging) + - [`swift destination select` Subcommand](#swift-destination-select-subcommand) + - [SwiftPM and SourceKit-LSP Improvements](#swiftpm-and-sourcekit-lsp-improvements) + - [Source-Based CC Destinations](#source-based-cc-destinations) + - [Destination Bundles and Package Registries](#destination-bundles-and-package-registries) ## Introduction From 139e51f120b20e2d6aa9a61d44474ec698d9ec8c Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Thu, 2 Mar 2023 12:11:27 +0000 Subject: [PATCH 19/22] Change `swift destination delete` to `swift destination remove` for consistency --- proposals/0387-cross-compilation-destinations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 98de0f5c8c..03f4ab2f14 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -402,7 +402,7 @@ used subsequently when building with a given destination. Specifically, multiple which then will be stored as configuration for this destination. `swift destination configure --show-configuration` will print currently set paths, while `swift destination configure --reset` will reset all of those at once. -- `swift destination delete ` will delete a given destination from the filesystem. +- `swift destination remove ` will remove a given destination from the filesystem. ### Using a CC Destination From 775b427f995ba53221596dd33b94c05655f1a5b4 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Fri, 3 Mar 2023 14:17:17 +0000 Subject: [PATCH 20/22] Update `swift destination configure` subcommand --- proposals/0387-cross-compilation-destinations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 03f4ab2f14..53e23185e0 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -396,7 +396,7 @@ To manage CC destinations, we'd like to introduce a new `swift destination` comm `install` subcommand with `--update` flag to allow updating an already installed destination artifact to a new version. - `swift destination list`, which prints a list of already installed CC destinations with their identifiers. -- `swift destination configure `, which allows users to provide additional search paths and toolsets to be +- `swift destination configure `, which allows users to provide additional search paths and toolsets to be used subsequently when building with a given destination. Specifically, multiple `--swift-resources-path`, `--include-search-path`, `--library-search-path`, and `--toolset` options with corresponding paths can be provided, which then will be stored as configuration for this destination. From db299df325cbbd0707af33ea0855ea6e0e49065f Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Tue, 7 Mar 2023 14:49:37 +0000 Subject: [PATCH 21/22] Clarify command-line argument for `destination configure` subcommand --- proposals/0387-cross-compilation-destinations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 53e23185e0..8c3c44329f 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -396,7 +396,7 @@ To manage CC destinations, we'd like to introduce a new `swift destination` comm `install` subcommand with `--update` flag to allow updating an already installed destination artifact to a new version. - `swift destination list`, which prints a list of already installed CC destinations with their identifiers. -- `swift destination configure `, which allows users to provide additional search paths and toolsets to be +- `swift destination configure `, which allows users to provide additional search paths and toolsets to be used subsequently when building with a given destination. Specifically, multiple `--swift-resources-path`, `--include-search-path`, `--library-search-path`, and `--toolset` options with corresponding paths can be provided, which then will be stored as configuration for this destination. From 2d34fda588a08193ef6d002c755668e6f2337296 Mon Sep 17 00:00:00 2001 From: Max Desiatov Date: Tue, 4 Apr 2023 16:54:09 +0100 Subject: [PATCH 22/22] Fix `Authors` field formatting --- proposals/0387-cross-compilation-destinations.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/proposals/0387-cross-compilation-destinations.md b/proposals/0387-cross-compilation-destinations.md index 8c3c44329f..17a346fc54 100644 --- a/proposals/0387-cross-compilation-destinations.md +++ b/proposals/0387-cross-compilation-destinations.md @@ -1,8 +1,7 @@ # Cross-Compilation Destination Bundles - Proposal: [SE-0387](0387-cross-compilation-destinations.md) -- Authors: [Max Desiatov](https://github.com/MaxDesiatov), [Saleem Abdulrasool](https://github.com/compnerd/), [Evan - Wilde](https://github.com/etcwilde) +- Authors: [Max Desiatov](https://github.com/MaxDesiatov), [Saleem Abdulrasool](https://github.com/compnerd/), [Evan Wilde](https://github.com/etcwilde) - Review Manager: [Mishal Shah](https://github.com/shahmishal) - Status: **Active Review (January 31st...Feburary 14th, 2023**) - Implementation: [apple/swift-package-manager#5911](https://github.com/apple/swift-package-manager/pull/5911),