Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal/wasmtools: run wasm-tools.wasm in embedded Wazero runtime #252

Closed
wants to merge 11 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,9 @@ jobs:
with:
go-version-file: go.mod

- name: Set up wams-tools.wasm
run: make internal/wasmtools/wasm-tools.wasm
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to do this?

The artifact should already be committed, and this adds a Rust dependency / build step to the release workflow.


- name: Run Go tests
run: go test ${{ env.go-modules }}

Expand Down
13 changes: 12 additions & 1 deletion .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,13 @@ jobs:
with:
go-version-file: go.mod

- name: Set up wams-tools.wasm
run: make internal/wasmtools/wasm-tools.wasm

- name: Vet Go code
run: go vet ${{ env.go-modules }} ./tests/...
run: |
make build
go vet ${{ env.go-modules }} ./tests/...

# Test with Go
test-go:
Expand All @@ -61,6 +66,9 @@ jobs:
with:
version: ${{ env.wasm-tools-version }}

- name: Set up wams-tools.wasm
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Necessary? Delete this and the step above?

run: make internal/wasmtools/wasm-tools.wasm

- name: Run Go tests
run: go test -v ${{ env.go-modules }}

Expand Down Expand Up @@ -147,6 +155,9 @@ jobs:
with:
version: ${{ env.wasm-tools-version }}

- name: Set up wams-tools.wasm
run: make internal/wasmtools/wasm-tools.wasm
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this necessary?

Should we delete the step above?


- name: Set up Wasmtime
uses: bytecodealliance/actions/wasmtime/setup@v1
with:
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added

- `internal/wasmtools` now loads compiled `wasm-tools` Wasm module from the `wasm-tools` crate and executes it using wazero. This allows `wit-bindgen-go` to run on any platform without needing to install `wasm-tools` natively.

## [v0.5.0] — 2024-12-14

### Changed
Expand Down
18 changes: 16 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ wit_files = $(sort $(shell find testdata -name '*.wit' ! -name '*.golden.*'))
json: $(wit_files)

.PHONY: $(wit_files)
$(wit_files):
wasm-tools component wit -j --all-features $@ > [email protected]
$(wit_files): internal/wasmtools/wasm-tools.wasm
wasmtime --dir . internal/wasmtools/wasm-tools.wasm component wit -j --all-features $@ > [email protected]

# golden recompiles the .golden.wit test files.
.PHONY: golden
Expand All @@ -21,12 +21,26 @@ generated: clean json
.PHONY: clean
clean:
rm -rf ./generated/*
rm -f internal/wasmtools/wasm-tools.wasm.gz

# tests/generated writes generated Go code to the tests directory
.PHONY: tests/generated
tests/generated: json
go generate ./tests

# build builds the cmd/wit-bindgen-go binary
.PHONY: build
build: internal/wasmtools/wasm-tools.wasm
go build -o wit-bindgen-go ./cmd/wit-bindgen-go

internal/wasmtools/wasm-tools.wasm: internal/wasmtools/wasm-tools.wasm.gz
gzip -dc $< > $@

internal/wasmtools/wasm-tools.wasm.gz:
cd internal/wasmtools && \
cargo build --target wasm32-wasip1 --release -p wasm-tools
Mossaka marked this conversation as resolved.
Show resolved Hide resolved
gzip -c internal/wasmtools/target/wasm32-wasip1/release/wasm-tools.wasm > $@

# test runs Go and TinyGo tests
GOTESTARGS :=
GOTESTMODULES := ./... ./cm/...
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ require (
github.com/klauspost/compress v1.17.11 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/tetratelabs/wazero v1.8.2 // indirect
github.com/ulikunitz/xz v0.5.12 // indirect
golang.org/x/sync v0.10.0 // indirect
golang.org/x/sys v0.28.0 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@ github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81P
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/tetratelabs/wazero v1.8.2 h1:yIgLR/b2bN31bjxwXHD8a3d+BogigR952csSDdLYEv4=
github.com/tetratelabs/wazero v1.8.2/go.mod h1:yAI0XTsMBhREkM/YDAK/zNou3GoiAce1P6+rp/wQhjs=
github.com/ulikunitz/xz v0.5.12 h1:37Nm15o69RwBkXM0J6A5OlE67RZTfzUxTj8fB3dfcsc=
github.com/ulikunitz/xz v0.5.12/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/urfave/cli/v3 v3.0.0-beta1 h1:6DTaaUarcM0wX7qj5Hcvs+5Dm3dyUTBbEwIWAjcw9Zg=
Expand Down
2 changes: 2 additions & 0 deletions internal/wasmtools/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
target
Mossaka marked this conversation as resolved.
Show resolved Hide resolved
wasm-tools.wasm
Loading
Loading