Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use youki's libcontainer APIs to implement wasmtime shim. #142

Merged
merged 36 commits into from
Jul 10, 2023
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
407b883
Update youki to the latest main branch and oci-spec to 0.6.1.
Mossaka Jun 12, 2023
8ca83f2
Merge remote-tracking branch 'upstream/main' into oci-0.6.1
Mossaka Jun 13, 2023
c555e3f
Removed some uses of unsafe in wasmedge shim
Mossaka Jun 13, 2023
fc1bb2e
Use youki's libcontainer APIs to implement wasmtime shim.
Mossaka Jun 13, 2023
83ed1a2
Fix some clippy warnings
Mossaka Jun 14, 2023
d8e4c1a
Merge remote-tracking branch 'upstream/main' into wasmtime-youki
Mossaka Jun 14, 2023
d3b7f69
Passed the test_delete_after_create test
Mossaka Jun 21, 2023
d998250
Merge remote-tracking branch 'upstream/main' into wasmtime-youki
Mossaka Jun 21, 2023
feb93fc
This commit adds a new crate called "containerd-shim-common".
Mossaka Jun 21, 2023
47ce17d
Apply suggestions from code review
Mossaka Jun 23, 2023
363f1cd
Resolved some review comments
Mossaka Jun 23, 2023
6882dd5
Removed common crate and moved the library to containerd-shim-wasm and
Mossaka Jun 23, 2023
fc33358
Rollback accidentally deleted deps
Mossaka Jun 23, 2023
b606b25
Removed the libcontainer feature
Mossaka Jun 23, 2023
8c9e43d
Fixed the test_wasi test
Mossaka Jun 28, 2023
11c2e20
Add trace to "cargo test" in CI
Mossaka Jun 28, 2023
839442a
Merge remote-tracking branch 'upstream/main' into wasmtime-youki
Mossaka Jun 28, 2023
80cd742
Temp commit to run tests on ubuntu 22.04 only
Mossaka Jun 28, 2023
ebad282
Rollback ubuntu 20.02 test in CI
Mossaka Jun 29, 2023
560356f
Use the same determine root logic from wasmedge shim
Mossaka Jun 29, 2023
4b106b2
Fixed clippy issues
Mossaka Jun 29, 2023
51182c6
Reset stdio at the end of the tests
Mossaka Jun 29, 2023
7ca42a1
Need rollback: commenting out the stdio thing
Mossaka Jun 29, 2023
8d5c858
Use inherit_stdio()
Mossaka Jun 29, 2023
65177c3
Setup CI build env for wasmtimie e2e
Mossaka Jun 30, 2023
bb1e940
Add inpsection step to e2e-wasmtime
Mossaka Jun 30, 2023
e120093
Add deps to kind nodes that run k8s tests
Mossaka Jul 5, 2023
76fa9cb
Merge remote-tracking branch 'upstream/main' into wasmtime-youki
Mossaka Jul 5, 2023
f7472fb
Handle Result types from dup and dup2 in both wasmtime and wasmedge s…
Mossaka Jul 6, 2023
d807601
Small refactoring
Mossaka Jul 6, 2023
3f31397
Update crates/containerd-shim-wasmtime/src/executor.rs
Mossaka Jul 6, 2023
2c64834
Merge branch 'wasmtime-youki' of https://github.com/Mossaka/runwasi i…
Mossaka Jul 6, 2023
729c3b4
Add a log to when Errno::ECHILD was returned
Mossaka Jul 6, 2023
a28fad9
Merge remote-tracking branch 'upstream/main' into wasmtime-youki
Mossaka Jul 7, 2023
aaaea73
Resolve comments
Mossaka Jul 9, 2023
dfc680d
Rustfmt
Mossaka Jul 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ members = [
"crates/containerd-shim-wasmedge",
"crates/containerd-shim-wasmtime",
"benches/containerd-shim-benchmarks",
"crates/common",
]

[workspace.package]
Expand All @@ -18,6 +19,8 @@ homepage = "https://github.com/containerd/runwasi"

[workspace.dependencies]
anyhow = "1.0"
containerd-shim-common = { path = "crates/common" }
containerd-shim-wasm = { path = "crates/containerd-shim-wasm" }
serde = "1.0"
serde_json = "1.0"
env_logger = "0.10"
Expand Down
11 changes: 11 additions & 0 deletions crates/common/Cargo.toml
Mossaka marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
[package]
name = "containerd-shim-common"
version.workspace = true
edition.workspace = true

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
anyhow = "1"
containerd-shim-wasm = { workspace = true }
libcontainer = { workspace = true }
55 changes: 55 additions & 0 deletions crates/common/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
//! Common utilities for the containerd shims, including wasmtime and wasmedge shim.

use anyhow::{bail, Context, Result};
use containerd_shim_wasm::sandbox::error::Error;
use libcontainer::container::Container;
use std::{
fs::{self, OpenOptions},
io::ErrorKind,
os::fd::{IntoRawFd, RawFd},
path::{Path, PathBuf},
};

/// Loads the container state from the given root path.
pub fn load_container<P: AsRef<Path>>(root_path: P, container_id: &str) -> Result<Container> {
let container_root = construct_container_root(root_path, container_id)?;
if !container_root.exists() {
bail!("container {} does not exist.", container_id)
}

Container::load(container_root)
.with_context(|| format!("could not load state for container {container_id}"))
}

/// Checks if the container exists.
pub fn container_exists<P: AsRef<Path>>(root_path: P, container_id: &str) -> Result<bool> {
let container_root = construct_container_root(root_path, container_id)?;
Ok(container_root.exists())
}

/// containerd can send an empty path or a non-existant path
/// In both these cases we should just assume that the stdio stream was not setup (intentionally)
/// Any other error is a real error.
pub fn maybe_open_stdio(path: &str) -> Result<Option<RawFd>, Error> {
if path.is_empty() {
return Ok(None);
}
match OpenOptions::new().read(true).write(true).open(path) {
Ok(f) => Ok(Some(f.into_raw_fd())),
Err(err) => match err.kind() {
ErrorKind::NotFound => Ok(None),
_ => Err(err.into()),
},
}
}

fn construct_container_root<P: AsRef<Path>>(root_path: P, container_id: &str) -> Result<PathBuf> {
let root_path = fs::canonicalize(&root_path).with_context(|| {
format!(
"failed to canonicalize {} for container {}",
root_path.as_ref().display(),
container_id
)
})?;
Ok(root_path.join(container_id))
}
5 changes: 3 additions & 2 deletions crates/containerd-shim-wasmedge/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ version.workspace = true
edition.workspace = true

[dependencies]
containerd-shim ={ workspace = true }
containerd-shim-wasm = { path = "../containerd-shim-wasm" }
containerd-shim = { workspace = true }
containerd-shim-wasm = { workspace = true }
containerd-shim-common = { workspace = true }
log = { workspace = true }
ttrpc = { workspace = true }
wasmedge-sdk = "0.8.1"
Expand Down
15 changes: 2 additions & 13 deletions crates/containerd-shim-wasmedge/src/executor.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
use anyhow::Result;
use containerd_shim_wasm::sandbox::oci;
use nix::unistd::{dup, dup2};
use oci_spec::runtime::Spec;

Expand All @@ -22,7 +23,7 @@ pub struct WasmEdgeExecutor {
impl Executor for WasmEdgeExecutor {
fn exec(&self, spec: &Spec) -> Result<(), ExecutorError> {
// parse wasi parameters
let args = get_args(spec);
let args = oci::get_args(spec);
if args.is_empty() {
return Err(ExecutorError::InvalidArg);
}
Expand Down Expand Up @@ -91,18 +92,6 @@ impl Executor for WasmEdgeExecutor {
}
}

fn get_args(spec: &Spec) -> &[String] {
let p = match spec.process() {
None => return &[],
Some(p) => p,
};

match p.args() {
None => &[],
Some(args) => args.as_slice(),
}
}

fn env_to_wasi(spec: &Spec) -> Vec<String> {
let default = vec![];
let env = spec
Expand Down
Loading