[water-api] API v0 (#2)

* update: major revision

Revised WATER design with new C-API from refraction-networking/wasmtime.

* update: add notes for SetReadDeadline

* refactor: move RuntimeConn definition

* update: refactor structs

* update: revise Dialer and Listener procedures

* fix: major crashing bugs

Now example is working.

* fix: abandon windows

File descriptors will not work on Windows anyways...

* new: adding benchmark

* new: dirty fix GO GC bug

* update: doc, ci check, cleanup

* new: v0 final prototype

* update: new v0 specs

* fix: minor bug and redundancy

* fix: external caller can't set WASIConfigFactory

Add (*Config).WASIConfig() to either return the already set WASIConfigFactory or create a new one.

* fix: typo

* fix: broken interface

.deepsource.toml

@@ -0,0 +1,13 @@
+version = 1
+
+[[analyzers]]
+name = "test-coverage"
+
+[[analyzers]]
+name = "go"
+
+  [analyzers.meta]
+  import_root = "github.com/gaukas/water"
+
+[[transformers]]
+name = "gofumpt"

.github/workflows/go.yml

@@ -14,16 +14,17 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        os: [ "ubuntu-latest", "windows-latest", "macos-latest" ]
-        # go: [ "1.20.x", "1.21.x" ]
+        # os: [ "ubuntu-latest", "windows-latest", "macos-latest" ] # Windows is not supported until net library implements Fd() for Windows
+        os: [ "ubuntu-latest", "macos-latest" ]
+        go: [ "1.20.x", "1.21.x" ]
     runs-on: ${{ matrix.os }}
     steps:
     - uses: actions/checkout@v3
     - uses: actions/setup-go@v4
       with:
-        go-version: "1.21.x"
+        go-version: ${{ matrix.go }}
     - run: go version
     - name: Build
       run: go build -v ./...
     - name: Test
-      run: go test ./...
+      run: go test -failfast ./...

README.md

@@ -5,39 +5,21 @@ W.A.T.E.R. provides a runtime environment for WebAssembly modules to run in and
 
 ## API 
 
-Currently, W.A.T.E.R. provides a set of APIs relying on **WASI Preview 1 (wasip1)** snapshot. 
+Currently, W.A.T.E.R. provides a set of APIs based on **WASI Preview 1 (wasip1)** snapshot. 
 
 ### Config
 A `Config` is a struct that contains the configuration for a WASI instance. It is used to configure the WASI reactor before starting it. 
 
-### RuntimeConn
-A `RuntimeConn` is a `Conn` that represents a connection from the local user to a remote peer. Each living `RuntimeConn` encapsulates a running WASI instance. 
-It process the data sent from the local user and send it to the remote peer, and vice versa.
+### Dialer 
 
-A `RuntimeConn` interfaces `io.ReadWriteCloser` and is always and only spawned by a `RuntimeConnDialer`.
+A `Dialer` could be used to dial a remote address upon `Dial()` and return a `net.Conn` back to the caller once the connection is established. Caller could use the `net.Conn` to read and write data to the remote address and the data will be processed by a WebAssembly instance.
 
-#### RuntimeConnDialer
-A `RuntimeConnDialer` is a `Dialer` loaded with a `Config` that can dial for `RuntimeConn` as abstracted connections. Currently, it is just a wrapper around a `Config`. **It does not contain any running WASI instance.**
+### Listener
 
-### RuntimeDialer _(TODO)_
-A `RuntimeDialer` is a `Dialer` that dials for `RuntimeDialerConn`. Each living `RuntimeDialer` encapsulates a running WASI instance. It manages multiple `RuntimeDialerConn` instances created upon caller's request.
+A `Listener` could be used to listen on a local address. Upon `Accept()`, it returns a `net.Conn` back once an incoming connection is accepted from the wrapped listener. Caller could use the `net.Conn` to read and write data to the remote address and the data will be processed by a WebAssembly instance.
 
-\* Not to be confused with [`RuntimeConnDialer`](#runtimeconndialer), a static dialer which creates `RuntimeConn` instances from `Config`.
+### Server
 
-#### RuntimeDialerConn
-A `RuntimeDialerConn` is a sub-`Conn` spawned by a `RuntimeDialer` upon caller's request. It is a `Conn` that is dialed by a `RuntimeDialer` and is used to communicate with a remote peer. Multiple `RuntimeDialerConn` instances can be created from a single `RuntimeDialer`, which means they could be related to one single WASI instance.
+A `Server` somewhat combines the role of `Dialer` and `Listener`. It could be used to listen on a local address and dial a remote address and automatically `Accept()` the incoming connections, feed them into the WebAssembly instance and `Dial()` the pre-defined remote address. Without any caller interaction, the `Server` will automatically* handle the data transmission between the two ends.
 
-\* Not to be confused with [`RuntimeConn`](#runtimeconn), an `io.ReadWriteCloser` that encapsulates a running WASI instance each.
-
-## TODOs
-
-- W.A.T.E.R. API
-    - [x] `Config`
-    - [x] `RuntimeConn`
-        - [x] `RuntimeConnDialer`
-    - [ ] `RuntimeDialer`
-        - [ ] `RuntimeDialerConn`
-- [x] Minimal W.A.T.E.R. WASI example
-    - No background worker threads
-- [ ] Multi-threaded W.A.T.E.R. WASI example
-    - [ ] Background worker threads working 
+***TODO: Server could not be realistic until WASI multi-threading or blocking mainloop is supported**

config.go

@@ -1,21 +1,124 @@
 package water
 
+import (
+	"net"
+	"os"
+
+	"github.com/gaukas/water/internal/log"
+	"github.com/gaukas/water/internal/wasm"
+)
+
 type Config struct {
-	// WASI contains the compiled WASI binary in bytes.
-	WASI []byte
+	// WATMBin contains the binary format of the WebAssembly Transport Module.
+	// In a typical use case, this mandatory field is populated by loading
+	// from a .wasm file, downloaded from a remote target, or generated from
+	// a .wat (WebAssembly Text Format) file.
+	WATMBin []byte
+
+	// DialerFunc specifies a func that dials the specified address on the
+	// named network. This optional field can be set to override the Go
+	// default dialer func:
+	// 	net.Dial(network, address)
+	DialerFunc func(network, address string) (net.Conn, error)
+
+	// NetworkListener specifies a net.listener implementation that listens
+	// on the specified address on the named network. This optional field
+	// will be used to provide (incoming) network connections from a
+	// presumably remote source to the WASM instance. Required by
+	// ListenConfig().
+	NetworkListener net.Listener
+
+	// Feature specifies a series of experimental features for the WASM
+	// runtime.
+	//
+	// Each feature flag is bit-masked and version-dependent, and flags
+	// are independent of each other. This means that a particular
+	// feature flag may be supported in one version of the runtime but
+	// not in another. If a feature flag is not supported or not recognized
+	// by the runtime, it will be silently ignored.
+	Feature Feature
+
+	// WATMConfig optionally provides a configuration file to be pushed into
+	// the WASM Transport Module.
+	WATMConfig WATMConfig
+
+	// wasiConfigFactory is used to replicate the WASI config for each WASM
+	// instance created. This field is for advanced use cases and/or debugging
+	// purposes only.
+	//
+	// Caller is supposed to call c.WASIConfig() to get the pointer to the
+	// WASIConfigFactory. If the pointer is nil, a new WASIConfigFactory will
+	// be created and returned.
+	wasiConfigFactory *wasm.WASIConfigFactory
+}
 
-	// Dialer is used to dial a network connection.
-	Dialer Dialer
+func (c *Config) Clone() *Config {
+	if c == nil {
+		return nil
+	}
+
+	wasmClone := make([]byte, len(c.WATMBin))
+	copy(wasmClone, c.WATMBin)
+
+	return &Config{
+		WATMBin:           c.WATMBin,
+		DialerFunc:        c.DialerFunc,
+		NetworkListener:   c.NetworkListener,
+		Feature:           c.Feature,
+		WATMConfig:        c.WATMConfig,
+		wasiConfigFactory: c.wasiConfigFactory.Clone(),
+	}
 }
 
-// init() checks if the Config is valid and initializes
-// the Config with default values if optional fields are not provided.
-func (c *Config) init() {
-	if len(c.WASI) == 0 {
-		panic("water: WASI binary is not provided")
+func (c *Config) DialerFuncOrDefault() func(network, address string) (net.Conn, error) {
+	if c.DialerFunc == nil {
+		return net.Dial
 	}
 
-	if c.Dialer == nil {
-		c.Dialer = DefaultDialer()
+	return c.DialerFunc
+}
+
+func (c *Config) NetworkListenerOrPanic() net.Listener {
+	if c.NetworkListener == nil {
+		panic("water: network listener is not provided in config")
 	}
+
+	return c.NetworkListener
+}
+
+func (c *Config) WATMBinOrPanic() []byte {
+	if len(c.WATMBin) == 0 {
+		panic("water: WebAssembly Transport Module binary is not provided in config")
+	}
+
+	return c.WATMBin
+}
+
+func (c *Config) WASIConfig() *wasm.WASIConfigFactory {
+	if c.wasiConfigFactory == nil {
+		c.wasiConfigFactory = wasm.NewWasiConfigFactory()
+	}
+
+	return c.wasiConfigFactory
+}
+
+// WATMConfig defines the configuration file used by the WebAssembly Transport Module.
+type WATMConfig struct {
+	FilePath string // Path to the config file.
+}
+
+// File opens the config file and returns the file descriptor.
+func (c *WATMConfig) File() *os.File {
+	if c.FilePath == "" {
+		log.Errorf("water: WASM config file path is not provided in config")
+		return nil
+	}
+
+	f, err := os.Open(c.FilePath)
+	if err != nil {
+		log.Errorf("water: failed to open WATM config file: %v", err)
+		return nil
+	}
+
+	return f
 }

conn_generic.go

@@ -0,0 +1,80 @@
+package water
+
+import (
+	"fmt"
+	"net"
+	"time"
+)
+
+var mapCoreDialContext = make(map[string]func(core *core, network, address string) (Conn, error))
+var mapCoreAccept = make(map[string]func(*core) (Conn, error))
+
+// Conn is an abstracted connection interface which encapsulates
+// a WASM runtime core.
+type Conn interface {
+	net.Conn
+
+	// For forward compatibility with any new methods added to the
+	// interface, all Conn implementations MUST embed the
+	// UnimplementedConn in order to make sure they could be used
+	// in the future without any code change.
+	mustEmbedUnimplementedConn()
+}
+
+func RegisterDial(version string, dialContext func(core *core, network, address string) (Conn, error)) error {
+	if _, ok := mapCoreDialContext[version]; ok {
+		return fmt.Errorf("water: core dial context already registered for version %s", version)
+	}
+	mapCoreDialContext[version] = dialContext
+	return nil
+}
+
+func RegisterAccept(version string, accept func(*core) (Conn, error)) error {
+	if _, ok := mapCoreAccept[version]; ok {
+		return fmt.Errorf("water: core accept already registered for version %s", version)
+	}
+	mapCoreAccept[version] = accept
+	return nil
+}
+
+// UnimplementedConn is used to provide forward compatibility for
+// implementations of Conn, such that if new methods are added
+// to the interface, old implementations will not be required to implement
+// each of them.
+type UnimplementedConn struct{}
+
+func (*UnimplementedConn) Read([]byte) (int, error) {
+	return 0, fmt.Errorf("water: Read() is not implemented")
+}
+
+func (*UnimplementedConn) Write([]byte) (int, error) {
+	return 0, fmt.Errorf("water: Write() is not implemented")
+}
+
+func (*UnimplementedConn) Close() error {
+	return fmt.Errorf("water: Close() is not implemented")
+}
+
+func (*UnimplementedConn) LocalAddr() net.Addr {
+	return nil
+}
+
+func (*UnimplementedConn) RemoteAddr() net.Addr {
+	return nil
+}
+
+func (*UnimplementedConn) SetDeadline(_ time.Time) error {
+	return fmt.Errorf("water: SetDeadline() is not implemented")
+}
+
+func (*UnimplementedConn) SetReadDeadline(_ time.Time) error {
+	return fmt.Errorf("water: SetReadDeadline() is not implemented")
+}
+
+func (*UnimplementedConn) SetWriteDeadline(_ time.Time) error {
+	return fmt.Errorf("water: SetWriteDeadline() is not implemented")
+}
+
+func (*UnimplementedConn) mustEmbedUnimplementedConn() {}
+
+var _ Conn = (*UnimplementedConn)(nil)