feat: Layotto Faas engine integrate with wasmtime

[rayowang]

What this PR does:
Layotto Faas engine supports wasmtime
Which issue(s) this PR fixes:

Fixes #732

[zu1k]

I got a crash if continue use wasmer.

"engine": "wasmer"

fatal error: unexpected signal during runtime execution                                                                          
[signal SIGSEGV: segmentation violation code=0x80 addr=0x0 pc=0x7f819b4f66ea]
...

goroutine 1 [syscall]:
runtime.cgocall(0x5103790, 0xc000b1cb28)
        /usr/local/go/src/runtime/cgocall.go:158 +0x5c fp=0xc000b1cb00 sp=0xc000b1cac8 pc=0x1f06f5c
github.com/wasmerio/wasmer-go/wasmer._Cfunc_wasi_get_wasi_version(0xad47fa0)
        _cgo_gotypes.go:758 +0x4c fp=0xc000b1cb28 sp=0xc000b1cb00 pc=0x4c302ec
github.com/wasmerio/wasmer-go/wasmer.GetWasiVersion.func1(0x10000000000?)
        /go/pkg/mod/github.com/wasmerio/[email protected]/wasmer/wasi.go:106 +0x46 fp=0xc000b1cb60 sp=0xc000b1cb28 pc=0x4c3f2e6
github.com/wasmerio/wasmer-go/wasmer.GetWasiVersion(0x7faa5ba85a68?)
        /go/pkg/mod/github.com/wasmerio/[email protected]/wasmer/wasi.go:106 +0x19 fp=0xc000b1cb78 sp=0xc000b1cb60 pc=0x4c3f279
mosn.io/mosn/pkg/wasm/runtime/wasmer.(*Module).Init(0xc0008a2050)
        /go/pkg/mod/mosn.io/[email protected]/pkg/wasm/runtime/wasmer/module.go:52 +0x29 fp=0xc000b1cbe0 sp=0xc000b1cb78 pc=0x4c46689
mosn.io/mosn/pkg/wasm/runtime/wasmer.NewWasmerModule(...)
        /go/pkg/mod/mosn.io/[email protected]/pkg/wasm/runtime/wasmer/module.go:46
mosn.io/mosn/pkg/wasm/runtime/wasmer.(*VM).NewModule(0xc00088c3c0, {0xc000f00000, 0x1f493, 0x1f494})
        /go/pkg/mod/mosn.io/[email protected]/pkg/wasm/runtime/wasmer/vm.go:65 +0x16f fp=0xc000b1cc40 sp=0xc000b1cbe0 pc=0x4c4752f
mosn.io/mosn/pkg/wasm.NewWasmPlugin({{0xc000ea60c0?, 0x5d6e420?}, 0xc0005fc0f0, 0x259fabf?})
        /go/pkg/mod/mosn.io/[email protected]/pkg/wasm/plugin.go:172 +0x382 fp=0xc000b1cd40 sp=0xc000b1cc40 pc=0x2832e22
mosn.io/mosn/pkg/wasm.(*wasmManagerImpl).AddOrUpdateWasm(0x97e52a0, {{0xc000ea60c0?, 0x60?}, 0xc0005fc0f0?, 0xc0000843c0?})
        /go/pkg/mod/mosn.io/[email protected]/pkg/wasm/manager.go:122 +0x145 fp=0xc000b1cdf0 sp=0xc000b1cd40 pc=0x2831c05
mosn.io/layotto/pkg/wasm.(*FilterConfigFactory).Install(0x9626080, 0x1f25c67?)
        /workspaces/layotto/pkg/wasm/factory.go:114 +0x97 fp=0xc000b1ce90 sp=0xc000b1cdf0 pc=0x4c4c217
mosn.io/layotto/pkg/wasm.createProxyWasmFilterFactory(0x5f49820?)
        /workspaces/layotto/pkg/wasm/factory.go:73 +0xbf fp=0xc000b1cf58 sp=0xc000b1ce90 pc=0x4c4be5f
mosn.io/api.CreateStreamFilterChainFactory({0xc000dc8880, 0x7}, 0x1f10b3f?)
        /go/pkg/mod/mosn.io/[email protected]/filter_factory.go:71 +0x57 fp=0xc000b1cfa8 sp=0xc000b1cf58 pc=0x219a2b7
mosn.io/mosn/pkg/streamfilter.createStreamFilterFactoryFromConfig({0xc000da4380?, 0x1, 0x5d6e420?})
        /go/pkg/mod/mosn.io/[email protected]/pkg/streamfilter/config.go:120 +0x127 fp=0xc000b1d060 sp=0xc000b1cfa8 pc=0x2bf8367
mosn.io/mosn/pkg/streamfilter.NewStreamFilterFactory({0xc000da4380, 0x1, 0x4})
        /go/pkg/mod/mosn.io/[email protected]/pkg/streamfilter/factory.go:45 +0x4c fp=0xc000b1d090 sp=0xc000b1d060 pc=0x2bf85ec
mosn.io/mosn/pkg/streamfilter.(*StreamFilterManagerImpl).AddOrUpdateStreamFilterConfig(0x97e50e0, {0xc000dc8830, 0x4}, {0xc000da4380, 0x1, 0x4})
        /go/pkg/mod/mosn.io/[email protected]/pkg/streamfilter/manager.go:75 +0x231 fp=0xc000b1d108 sp=0xc000b1d090 pc=0x2bf8d11
mosn.io/mosn/pkg/server.(*connHandler).AddOrUpdateListener(0xc000ddca20, 0xc000dc6590)
        /go/pkg/mod/mosn.io/[email protected]/pkg/server/handler.go:122 +0x11d fp=0xc000b1d440 sp=0xc000b1d108 pc=0x2bf9b7d
mosn.io/mosn/pkg/server.(*server).AddListener(0xc000dc6590?, 0x0?)
        /go/pkg/mod/mosn.io/[email protected]/pkg/server/server.go:92 +0x22 fp=0xc000b1d460 sp=0xc000b1d440 pc=0x2c033c2
mosn.io/mosn/pkg/mosn.(*Mosn).initServer(0xc0007b1950)
        /go/pkg/mod/mosn.io/[email protected]/pkg/mosn/mosn.go:219 +0x5db fp=0xc000b1d628 sp=0xc000b1d460 pc=0x2c05a5b
mosn.io/mosn/pkg/mosn.(*Mosn).Init(0xc0007b1950, 0x2?)
        /go/pkg/mod/mosn.io/[email protected]/pkg/mosn/mosn.go:83 +0x7d fp=0xc000b1d668 sp=0xc000b1d628 pc=0x2c04d7d
mosn.io/mosn/pkg/stagemanager.(*StageManager).runInitStage(0x962e020)
        /go/pkg/mod/mosn.io/[email protected]/pkg/stagemanager/stage_manager.go:212 +0xb0 fp=0xc000b1d6e8 sp=0xc000b1d668 pc=0x27b16f0
mosn.io/mosn/pkg/stagemanager.(*StageManager).Run(0x1?)
        /go/pkg/mod/mosn.io/[email protected]/pkg/stagemanager/stage_manager.go:300 +0x28 fp=0xc000b1d710 sp=0xc000b1d6e8 pc=0x27b20c8
mosn.io/mosn/pkg/stagemanager.(*StageManager).RunAll(0x962e020)
        /go/pkg/mod/mosn.io/[email protected]/pkg/stagemanager/stage_manager.go:589 +0x1e fp=0xc000b1d728 sp=0xc000b1d710 pc=0x27b39de
main.glob..func1(0xc000d0a840)
        /workspaces/layotto/cmd/layotto/cmd.go:201 +0x165 fp=0xc000b1d778 sp=0xc000b1d728 pc=0x50fa3c5
github.com/urfave/cli.HandleAction({0x5db7cc0?, 0x689af48?}, 0x5?)
        /go/pkg/mod/github.com/urfave/[email protected]/app.go:523 +0x50 fp=0xc000b1d790 sp=0xc000b1d778 pc=0x2799470
github.com/urfave/cli.Command.Run({{0x66e7f89, 0x5}, {0x0, 0x0}, {0x0, 0x0, 0x0}, {0x6801226, 0x4e}, {0x0, ...}, ...}, ...)
        /go/pkg/mod/github.com/urfave/[email protected]/command.go:174 +0x65b fp=0xc000b1d970 sp=0xc000b1d790 pc=0x279a2db
github.com/urfave/cli.(*App).Run(0xc000d16a80, {0xc000126000, 0x4, 0x4})
        /go/pkg/mod/github.com/urfave/[email protected]/app.go:276 +0x907 fp=0xc000b1df48 sp=0xc000b1d970 pc=0x2796847
main.main()
        /workspaces/layotto/cmd/layotto/main.go:487 +0x4e fp=0xc000b1df80 sp=0xc000b1df48 pc=0x510122e
runtime.main()
        /usr/local/go/src/runtime/proc.go:250 +0x212 fp=0xc000b1dfe0 sp=0xc000b1df80 pc=0x1f3f8f2
runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:1594 +0x1 fp=0xc000b1dfe8 sp=0xc000b1dfe0 pc=0x1f70161
...

[zu1k]

I got a crash if continue use wasmer.

I think this is due to the fact that we linked in two wasm libraries with the same symbol name, since they both use the standard wasm-c-api.

WebAssembly/wasm-c-api#165

[zu1k]

I think this is due to the fact that we linked in two wasm libraries with the same symbol name, since they both use the standard wasm-c-api.

I created a demo using [email protected] and [email protected] and both engines work, so i think the problem is not here.

[rayowang]

I think this is due to the fact that we linked in two wasm libraries with the same symbol name, since they both use the standard wasm-c-api.

I created a demo using [email protected] and [email protected] and both engines work, so i think the problem is not here.

👍 How to do it specifically, wasmtime static library and wasmer dynamic library? Indeed, it may also be caused by some dependencies of dapr, layotto, especially mosn. I feel that we can raise a special issue to track this problem. We can discuss whether this is feasible or not. @seeflood @zhenjunMa Take a look when you are free~

[zu1k]

I created a demo using [email protected] and [email protected] and both engines work, so i think the problem is not here.

Very strange, yesterday it worked fine, today it does not work properly, encountered the same problem.

I can confirm that the problem is still caused by using the same symbols. I traced wasm_module_new() was called in wasmer's cgo func to_wasm_module_new(), but was linked with wasmtime static library function, the two are not compatible. The problem is triggered anywhere in libwasmer.so where the wasm_module_t pointer is passed.

[zu1k]

And using dynamic libraries does not solve this problem.

 # ldd wt
        linux-vdso.so.1 (0x00007ffde79fc000)
        libwasmer.so => /home/zu1k/go/pkg/mod/github.com/wasmerio/[email protected]/wasmer/packaged/lib/linux-amd64/libwasmer.so (0x00007efcb9000000)
        libwasmtime.so => /home/zu1k/go/pkg/mod/github.com/bytecodealliance/[email protected]/build/linux-x86_64/libwasmtime.so (0x00007efcb8600000)
        libm.so.6 => /usr/lib/libm.so.6 (0x00007efcb9d2b000)
        libc.so.6 => /usr/lib/libc.so.6 (0x00007efcb8419000)
        libdl.so.2 => /usr/lib/libdl.so.2 (0x00007efcb9d26000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007efcb9d04000)
        librt.so.1 => /usr/lib/librt.so.1 (0x00007efcb9cff000)
        libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007efcb9cfa000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007efcb9e5c000)

This time my wasmer worked fine, instead wasmtime had a problem, I found that wasmtime's wasm_module_new() was not linked and used wasmer's.

WebAssembly/wasm-c-api#165 (comment)

[zhenjunMa]

@zu1k
According to my test, if we use -tags wasmer to build, we can use wasmer as WebAssembly Runtime, but if we use -tags wasmcomm,wasmtime or -tags wasmcomm,wasmtime, wasmer to build, we can only use wasmtime, or it will throw an exception as you mentioned above. So I think we can ignore this annoying problem cause I have updated the doc.

@rayowang
Thanks for this wonderful contribution, I only have one question, is it possible to replace the build tag wasmcomm in pkg/wasm/update/init.go with wasmtime ?

[rayowang]

@zu1k According to my test, if we use -tags wasmer to build, we can use wasmer as WebAssembly Runtime, but if we use -tags wasmcomm,wasmtime or -tags wasmcomm,wasmtime, wasmer to build, we can only use wasmtime, or it will throw an exception as you mentioned above. So I think we can ignore this annoying problem cause I have updated the doc.

@rayowang Thanks for this wonderful contribution, I only have one question, is it possible to replace the build tag wasmcomm in pkg/wasm/update/init.go with wasmtime ?

@zhenjunMa I want to confirm, do you mean remove the -tags wasmcomm build tag, use -tags wasmtime by default, then if we need to use wasmer, use -tags wamtime,wasmer? if so, it's really easier.

But I thought about it again, pkg/wasm/update/init.go, pkg/wasm/uninstall/init.go, and pkg/wasm/install/init.go correspond to the external restful operation interface of wasm plugin. Is it better to separate them?

[zhenjunMa]

@rayowang ok, you are right, just keep it this way. -tags wasmcomm,wasmer or -tags wasmcomm,wasmtime

[zhenjunMa]

LGTM

[zu1k]

LGTM