Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build platform independent rust binary wheel through wasi #1107

Merged
merged 8 commits into from
Oct 9, 2022
Merged

Conversation

konstin
Copy link
Member

@konstin konstin commented Sep 15, 2022

I had this idea that besides having to build a wheel for each platform and then still there are platforms without manylinux supports, we could have a single wasm wheel as a generic fallback. Currently more theoretical since wasmtime supports less platforms than manylinux. Also we could have isolation and such in the future.

We build the rust binaries to wasi and put them in a wheel with a wasmtime launcher. The resulting wheels are platform independent (py3-none-any) and run on all wasmtime platforms.

TODO:

@netlify
Copy link

netlify bot commented Sep 15, 2022

Deploy Preview for maturin-guide ready!

Name Link
🔨 Latest commit 18e00ae
🔍 Latest deploy log https://app.netlify.com/sites/maturin-guide/deploys/63428d86f7861700085b9e22
😎 Deploy Preview https://deploy-preview-1107--maturin-guide.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

Copy link
Member

@messense messense left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting idea!

# TODO: this is taken from https://docs.wasmtime.dev/api/wasmtime/struct.Linker.html#method.get_default
# is this always correct?
start = linking1.exports(store).get("") or linking1.exports(store)["_start"]
start(store)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember that wasmtime execution is sandboxed, so you might need to give access to file systems explicitly somehow?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think so, let's see if i get a response in bytecodealliance/wasmtime-py#94 and otherwise i'll have to collect some real world use cases compatible with wasm

tests/run.rs Outdated Show resolved Hide resolved
@messense
Copy link
Member

wasm32-wasi std library also needs to be installed via rustup.

@konstin
Copy link
Member Author

konstin commented Sep 16, 2022

yet that's the next step

@messense

This comment was marked as outdated.

@messense messense added bindings/bin bin bindings target/wasm wasm build target target/wasi wasi build target labels Sep 17, 2022
We build the rust binaries to wasi and put them in a wheel with a wasmtime launcher. The resulting wheels are platform independent (py3-none-any) and run on all wasmtime platforms. Those are currently less than maturin natively, but you still only have to build one wheel and i hope for some nice things in the future (isolation capabilities, wasm runtimes on less-supported platforms)
@konstin konstin changed the title WIP: Build platform independent rust binary wheel through wasi Build platform independent rust binary wheel through wasi Oct 9, 2022
@konstin konstin marked this pull request as ready for review October 9, 2022 09:00
@konstin
Copy link
Member Author

konstin commented Oct 9, 2022

ready to merge from my end

@messense messense enabled auto-merge (squash) October 9, 2022 09:16
@messense messense merged commit fe8a120 into main Oct 9, 2022
@messense messense deleted the wasi branch October 9, 2022 09:17
@messense messense added this to the 0.14.0 milestone Oct 14, 2022
bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this pull request Nov 20, 2022
https://build.opensuse.org/request/show/1036879
by user mia + dimstar_suse
- Update to v0.14.0:
* Breaking Change: Remove support for specifying python package
  metadata in `Cargo.toml`
  Python package metadata should be specified in the `project`
  section of `pyproject.toml` instead as PEP 621 specifies.
  gh#PyO3/maturin#1200
* Initial support for shipping bin targets as wasm32-wasi binaries
  that are run through wasmtime
  Note that wasmtime currently only support the five most popular
  platforms and that wasi binaries have restrictions when
  interacting with the host.
  Usage is by setting `--target wasm32-wasi`.
  gh#PyO3/maturin#1107
* Add support for python first `src` project layout
  gh#PyO3/maturin#1185
* Add `--src` option to generate src layout for mixed Python/Rust
  projects
  gh#PyO3/maturin#1189
* Add Python metadata support for `license-file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bindings/bin bin bindings target/wasi wasi build target target/wasm wasm build target
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants