Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(wamr): add wasm-micro-runtime shim implementation #716

Merged
merged 20 commits into from
Nov 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
strategy:
matrix:
os: ["ubuntu-latest", "windows-latest"]
runtime: ["common", "wasmedge", "wasmtime", "wasmer"]
runtime: ["common", "wasmedge", "wasmtime", "wasmer", "wamr"]
uses: ./.github/workflows/action-check.yml
with:
os: ${{ matrix.os }}
Expand Down Expand Up @@ -53,7 +53,7 @@ jobs:
strategy:
matrix:
os: ["ubuntu-22.04"]
runtime: ["common", "wasmtime", "wasmedge", "wasmer"]
runtime: ["common", "wasmtime", "wasmedge", "wasmer", "wamr"]
libc: ["musl", "gnu"]
arch: ["x86_64", "aarch64"]
uses: ./.github/workflows/action-build.yml
Expand Down Expand Up @@ -85,7 +85,7 @@ jobs:
matrix:
# 20.04 uses cgroupv1, 22.04 uses cgroupv2
os: ["ubuntu-20.04", "ubuntu-22.04"]
runtime: ["wasmtime", "wasmedge", "wasmer"]
runtime: ["wasmtime", "wasmedge", "wasmer", "wamr"]
uses: ./.github/workflows/action-test-smoke.yml
with:
os: ${{ matrix.os }}
Expand All @@ -99,7 +99,7 @@ jobs:
matrix:
# 20.04 uses cgroupv1, 22.04 uses cgroupv2
os: ["ubuntu-20.04", "ubuntu-22.04"]
runtime: ["wasmtime", "wasmedge", "wasmer"]
runtime: ["wasmtime", "wasmedge", "wasmer", "wamr"]
uses: ./.github/workflows/action-test-kind.yml
with:
os: ${{ matrix.os }}
Expand All @@ -112,7 +112,7 @@ jobs:
strategy:
matrix:
os: ["ubuntu-22.04"]
runtime: ["wasmtime", "wasmedge", "wasmer"]
runtime: ["wasmtime", "wasmedge", "wasmer", "wamr"]
uses: ./.github/workflows/action-test-kind.yml
with:
os: ${{ matrix.os }}
Expand All @@ -127,7 +127,7 @@ jobs:
fail-fast: false
matrix:
os: ["ubuntu-20.04", "ubuntu-22.04"]
runtime: ["wasmtime", "wasmedge", "wasmer"]
runtime: ["wasmtime", "wasmedge", "wasmer", "wamr"]
uses: ./.github/workflows/action-test-k3s.yml
with:
os: ${{ matrix.os }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ on:
- containerd-shim-wasmer
- containerd-shim-wasmedge
- containerd-shim-wasmtime
- containerd-shim-wamr
version:
description: "The version of the crate to release. (e.g., 1.2.3)"
type: string
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ There are several projects in the repository:

- `containerd-shim-wasm` - main library that is used by runtimes to create shims. Most of the shared code lives here.
- `containerd-shim-wasm-test-modules` - library with wasm test modules used in testing framework
- `containerd-shim-<runtime>` - shims per runtime (wasmtime, wasmedge, wasmer, etc). These produce binaries that are the shims which containerd talks too.
- `containerd-shim-<runtime>` - shims per runtime (wasmtime, wasmedge, wasmer, wamr, etc). These produce binaries that are the shims which containerd talks too.
- `oci-tar-builder` - library and executable that helps build OCI tar files.
- `wasi-demo-app` - wasm application that is used for demos and testing.

Expand All @@ -55,7 +55,7 @@ To build all the shims in this repository:
make build
```

To build a shim for specific runtime (wasmtime, wasmer, wasmedge, etc):
To build a shim for specific runtime (wasmtime, wasmer, wasmedge, wamr, etc):

```
make build-<runtime>
Expand Down
51 changes: 50 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ members = [
"crates/containerd-shim-wasmedge",
"crates/containerd-shim-wasmtime",
"crates/containerd-shim-wasmer",
"crates/containerd-shim-wamr",
"benches/containerd-shim-benchmarks",
]
resolver = "2"
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ PREFIX ?= /usr/local
INSTALL ?= install
CARGO ?= cargo
TEST_IMG_NAME ?= wasmtest:latest
RUNTIMES ?= wasmedge wasmtime wasmer
RUNTIMES ?= wasmedge wasmtime wasmer wamr
CONTAINERD_NAMESPACE ?= default
RUSTC ?= rustc

Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,10 +121,10 @@ Check out these projects that build on top of runwasi:

### Components

- **containerd-shim-[ wasmedge | wasmtime | wasmer ]-v1**
- **containerd-shim-[ wasmedge | wasmtime | wasmer | wamr ]-v1**

This is a containerd shim which runs wasm workloads in [WasmEdge](https://github.com/WasmEdge/WasmEdge) or [Wasmtime](https://github.com/bytecodealliance/wasmtime) or [Wasmer](https://github.com/wasmerio/wasmer).
You can use it with containerd's `ctr` by specifying `--runtime=io.containerd.[ wasmedge | wasmtime | wasmer ].v1` when creating the container.
You can use it with containerd's `ctr` by specifying `--runtime=io.containerd.[ wasmedge | wasmtime | wasmer | wamr ].v1` when creating the container.
And make sure the shim binary must be in $PATH (that is the $PATH that containerd sees). Usually you just run `make install` after `make build`.
> build shim with wasmedge we need install library first

Expand Down Expand Up @@ -156,7 +156,7 @@ make load

### Demo 1 using container image that contains a Wasm module.

Run it with `sudo ctr run --rm --runtime=io.containerd.[ wasmedge | wasmtime | wasmer ].v1 ghcr.io/containerd/runwasi/wasi-demo-app:latest testwasm /wasi-demo-app.wasm echo 'hello'`. You should see some output repeated like:
Run it with `sudo ctr run --rm --runtime=io.containerd.[ wasmedge | wasmtime | wasmer | wamr ].v1 ghcr.io/containerd/runwasi/wasi-demo-app:latest testwasm /wasi-demo-app.wasm echo 'hello'`. You should see some output repeated like:

```terminal
sudo ctr run --rm --runtime=io.containerd.wasmtime.v1 ghcr.io/containerd/runwasi/wasi-demo-app:latest testwasm
Expand Down Expand Up @@ -193,7 +193,7 @@ make test-image/oci
make load/oci
```

Run the image with `sudo ctr run --rm --runtime=io.containerd.[ wasmedge | wasmtime | wasmer ].v1 ghcr.io/containerd/runwasi/wasi-demo-oci:latest testwasmoci`
Run the image with `sudo ctr run --rm --runtime=io.containerd.[ wasmedge | wasmtime | wasmer | wamr ].v1 ghcr.io/containerd/runwasi/wasi-demo-oci:latest testwasmoci`

```
sudo ctr run --rm --runtime=io.containerd.wasmtime.v1 ghcr.io/containerd/runwasi/wasi-demo-oci:latest testwasmoci wasi-demo-oci.wasm echo 'hello'
Expand Down
21 changes: 21 additions & 0 deletions crates/containerd-shim-wamr/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[package]
name = "containerd-shim-wamr"
version.workspace = true
edition.workspace = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
anyhow = { workspace = true }
containerd-shim-wasm = { workspace = true }
log = { workspace = true }

[target.'cfg(unix)'.dependencies]
wamr-rust-sdk = { git = "https://github.com/bytecodealliance/wamr-rust-sdk", tag = "v1.1.0" }
Mossaka marked this conversation as resolved.
Show resolved Hide resolved

[dev-dependencies]
containerd-shim-wasm = { workspace = true, features = ["testing"] }
serial_test = { workspace = true }

[[bin]]
name = "containerd-shim-wamr-v1"
path = "src/main.rs"
93 changes: 93 additions & 0 deletions crates/containerd-shim-wamr/src/instance.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
use anyhow::{Context, Result};
use containerd_shim_wasm::container::{Engine, Entrypoint, Instance, RuntimeContext, Stdio};
use wamr_rust_sdk::function::Function;
use wamr_rust_sdk::instance::Instance as WamrInst;
use wamr_rust_sdk::module::Module;
use wamr_rust_sdk::runtime::Runtime;
use wamr_rust_sdk::wasi_context::WasiCtxBuilder;

pub type WamrInstance = Instance<WamrEngine>;

pub struct WamrEngine {
runtime: Runtime,
}

unsafe impl Send for WamrEngine {}
unsafe impl Sync for WamrEngine {}
Comment on lines +15 to +16
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Could you add a comment as to why this is safe to do? I guess Runtime is not Send/Sync, should Runtime be made Send/Sync upstream?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the Runtime should be made Send / Sync upstream

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lum1n0us is this the case? Does Wamr Runtime contain any thread-local storage? Does it have any shared internal states?


// TODO: wasmr_rust_sdk::runtime::Runtime should implement Clone

impl Default for WamrEngine {
fn default() -> Self {
let runtime = Runtime::new().unwrap();
Self { runtime }
}
}

impl Clone for WamrEngine {
fn clone(&self) -> Self {
let runtime = Runtime::new().unwrap();
Self { runtime }
}
}
Comment on lines +27 to +32
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Is this correct? doesn't Runtime hold any state? I don't remember why we require the engine to be Clone :-\

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want the Engine to be Clone because we want cheap copies of the Runtime for each container in the pod. This is the case for Wasmtime, WasmEdge and Wasmer. I believe it should be the case for Wamr as well.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we don't clone the engine for each container, we clone the process...
In any case, if we create a new runtime with each clone, we might as well use a marker struct for the Engine and create a new runtime inside the run_wasi method.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I agree with you that we clone the entire process so there is no point to clone the engine.

In any case, if we create a new runtime with each clone,

at least for wasmtime, Engine::clone() is a cheap operation: https://docs.wasmtime.dev/api/wasmtime/struct.Engine.html#engines-and-clone


impl Engine for WamrEngine {
fn name() -> &'static str {
"wamr"
}

fn run_wasi(&self, ctx: &impl RuntimeContext, stdio: Stdio) -> Result<i32> {
let args = ctx.args();
let envs = ctx.envs();
let Entrypoint {
source, func, name, ..
} = ctx.entrypoint();

let wasm_bytes = source
.as_bytes()
.context("Failed to get bytes from source")?;

log::info!("Create a WAMR module");

// TODO: error handling isn't ideal

let mod_name = name.unwrap_or_else(|| "main".to_string());

let mut module = Module::from_buf(&self.runtime, &wasm_bytes, &mod_name)
.context("Failed to create module from bytes")?;

log::info!("Create a WASI context");

let wasi_ctx = WasiCtxBuilder::new()
.set_pre_open_path(vec!["/"], vec![])
.set_env_vars(envs.iter().map(String::as_str).collect())
.set_arguments(args.iter().map(String::as_str).collect())
.build();

module.set_wasi_context(wasi_ctx);

// TODO: no way to register a named module with bytes?

log::info!("Create a WAMR instance");

let instance = WamrInst::new(&self.runtime, &module, 1024 * 64)
.context("Failed to create instance")?;

log::info!("redirect stdio");
stdio.redirect()?;

log::info!("Running {func:?}");
let function =
Function::find_export_func(&instance, &func).context("Failed to find function")?;
let status = function
.call(&instance, &vec![])
.map(|_| 0)
.map_err(|err| {
log::error!("Error: {:?}", err);
err
})
.context("Failed to call function")?;

Ok(status)
}
}
10 changes: 10 additions & 0 deletions crates/containerd-shim-wamr/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#[cfg(unix)]
pub mod instance;

#[cfg(unix)]
pub use instance::WamrInstance;

#[cfg(unix)]
#[cfg(test)]
#[path = "tests.rs"]
mod wamr_tests;
13 changes: 13 additions & 0 deletions crates/containerd-shim-wamr/src/main.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#[cfg(not(target_os = "windows"))]
use containerd_shim_wamr::WamrInstance;
use containerd_shim_wasm::sandbox::cli::{revision, shim_main, version};

#[cfg(target_os = "windows")]
fn main() {
panic!("WAMR shim is not supported on Windows");
}

#[cfg(not(target_os = "windows"))]
fn main() {
shim_main::<WamrInstance>("wamr", version!(), revision!(), "v1", None);
}
Loading
Loading