Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compiled function with arguments has wrong signature #3010

Closed
alexec opened this issue Jul 21, 2022 · 13 comments
Closed

Compiled function with arguments has wrong signature #3010

alexec opened this issue Jul 21, 2022 · 13 comments
Labels
wasm WebAssembly

Comments

@alexec
Copy link

alexec commented Jul 21, 2022

This does not make any sense to me:

package main

func main() {}

//export HelloWorld
func HelloWorld() {
}

//export HelloArg
func HelloArg(arg string) {
}

//export HelloReturn
func HelloReturn() string {
	return ""
}
tinygo version 0.24.0 darwin/amd64 (using go version go1.18.4 and LLVM version 14.0.0)
cd wasm && tinygo build -o ../module.wasm -wasm-abi=generic -target=wasi .
wasmer inspect module.wasm
Type: wasm
Size: 36.8 KB
Imports:
  Functions:
    "wasi_snapshot_preview1"."fd_write": [I32, I32, I32, I32] -> [I32]
  Memories:
  Tables:
  Globals:
Exports:
  Functions:
    "malloc": [I32] -> [I32]
    "free": [I32] -> []
    "calloc": [I32, I32] -> [I32]
    "realloc": [I32, I32] -> [I32]
    "_start": [] -> []
    "HelloWorld": [] -> []
    "HelloArg": [I32, I32] -> []
    "HelloReturn": [I32] -> []
    "asyncify_start_unwind": [I32] -> []
    "asyncify_stop_unwind": [] -> []
    "asyncify_start_rewind": [I32] -> []
    "asyncify_stop_rewind": [] -> []
    "asyncify_get_state": [] -> [I32]
  Memories:
    "memory": not shared (2 pages..)
  Tables:
  Globals:

Can someone please explain why HelloArg and HelloReturn has these weird parameters?

@dgryski
Copy link
Member

dgryski commented Jul 21, 2022

In Go, a string is actually passed around as a pointer to the string data and a length. For wasm, this is a pair of (int32, int32).

For returns values, wasm functions can only return a single value. So for functions that need to return a string the destination for the multiple return values is passed as a function argument as a pointer to the location to write the multiple values.

Does this make sense?

@alexec
Copy link
Author

alexec commented Jul 22, 2022

Ah. Yet this is not going to work for my use case.

Let me explain what I want to do. I want to allow other teams to implement an interface, and implement in any language, Golang orJavascript. This will then be executed by importing the module into a Go app. The interface I want is this:

func SayHello(person string) string

Can tinygo do this? Or do I need to look for other solutions?

@kenbell kenbell added the wasm WebAssembly label Jul 28, 2022
@codefromthecrypt
Copy link
Contributor

codefromthecrypt commented Aug 16, 2022

my 2p:

I think string and []byte could be special cased somehow, though when (if) strings are supported natively in wasm, special rules like this will create ambiguity.

I might instead keep your signature the same except implement it differently based on arch.

Ex. func SayHello(person string) string can be implemented using WebAssembly types like so:

func SayHello(person string) string {
	ptr, size := stringToPtr(person)
	ptrSize := _SayHello(ptr, size)
	ptr = uint32(ptrSize >> 32)
	size = uint32(ptrSize)
	return ptrToString(ptr, size)
}

// _SayHello implements SayHello using types present in WebAssembly
// core specification 1.0
//
// Note: In TinyGo "//export" on a func is actually an import!
//
//go:wasm-module env
//export SayHello
func _SayHello(ptr uint32, size uint32) (ptrSize uint64)


// ptrToString returns a string from WebAssembly compatible numeric types
// representing its pointer and length.
func ptrToString(ptr uint32, size uint32) string {
	// Get a slice view of the underlying bytes in the stream. We use SliceHeader, not StringHeader
	// as it allows us to fix the capacity to what was allocated.
	return *(*string)(unsafe.Pointer(&reflect.SliceHeader{
		Data: uintptr(ptr),
		Len:  uintptr(size), // Tinygo requires these as uintptrs even if they are int fields.
		Cap:  uintptr(size), // ^^ See https://github.com/tinygo-org/tinygo/issues/1284
	}))
}

// stringToPtr returns a pointer and size pair for the given string in a way
// compatible with WebAssembly numeric types.
func stringToPtr(s string) (uint32, uint32) {
	buf := []byte(s)
	ptr := &buf[0]
	unsafePtr := uintptr(unsafe.Pointer(ptr))
	return uint32(unsafePtr), uint32(len(buf))
}

As you'll notice this code looks generatable even if not done in tinygo compiler.

You can also look at https://github.com/inkeliz/karmem

@codefromthecrypt
Copy link
Contributor

ps above was basically cobbled from https://github.com/tetratelabs/wazero/tree/main/examples/allocation/tinygo but I didn't make the source files arch specific (ex to have SayHello defined in a different source file if target=wasi)

@alexec
Copy link
Author

alexec commented Aug 16, 2022

Wow. The answer appears to be “WASM does not support strings”.

I’m actually pretty 🤯because it’s such a common language feature I’d just assumed it’s be supported.

It’s like buying a car only to find out it does not have windscreen wipers. Yes, you can drive it, but only if you are happy to drive only in the sunshine.

@codefromthecrypt
Copy link
Contributor

@alexec yeah that actually is a better way to frame it, basically it doesn't currently and even 2.0 (draft) doesn't support strings. You aren't alone in surprise, basically what some end up realizing is that wasm is mostly focused on core numeric types still though there is hope in the future.

There are two specifications some compilers may support ahead of passing the requisite w3c phases, as mentioned in the proposals repo: component-model and stringref. What that means is some compilers may opt-in experimental support ahead of actually ending up in the spec.

Meanwhile, what's typical is either a compiler or a code generator translates strings to offset/length pairs, noting that even multiple results aren't supported until WebAssembly 2.0. That said, most runtimes support features needed for multiple results.

@alexec
Copy link
Author

alexec commented Aug 16, 2022

This is such a shame. I had strong expectation about using WASM as an fast (i.e millions of evaluations a second) and portable (i.e. many different language runtime) embedded scripting language and have two strong use cases for it.

But lack of string support makes it unusable.

🤷

@codefromthecrypt
Copy link
Contributor

@alexec I suspect it is ok to close this, as tinygo has limited influence on changing the WebAssembly spec ;)

@alexec alexec closed this as completed Sep 7, 2022
@aykevl
Copy link
Member

aykevl commented Sep 8, 2022

Yeah, we can't do a lot about it. I hope that the stringref proposal gets some traction, that would allow us to use strings in exported functions. Until then, they are represented as a (pointer, length) pair like in many other languages.

@HarikrishnanBalagopal
Copy link

In Go, a string is actually passed around as a pointer to the string data and a length. For wasm, this is a pair of (int32, int32).

For returns values, wasm functions can only return a single value. So for functions that need to return a string the destination for the multiple return values is passed as a function argument as a pointer to the location to write the multiple values.

Does this make sense?

Wasm functions can return multiple values now.

@codefromthecrypt
Copy link
Contributor

Wasm functions can return multiple values now.

Yep, but that isn't a part of a REC spec. Right now, 2.0 which supports that is still a draft. It doesn't mean TinyGo couldn't enable features scheduled for 2.0, but it is a distinct choice to do that. I'm not sure even rust by default will do multiple returns either fwiw.

@DeeStarks
Copy link

Not sure if this was fixed, but as @dgryski mentioned "a string is actually passed around as a pointer to the string data". By creating a view into the WebAssembly memory buffer starting at the address passed, with a maximum string length of 12 bytes (adjust as needed), and decoding it as a UTF-8 string, it worked for me.

package main

func main() {
    HelloArg("Hello World!")
}

//export HelloArg
func HelloArg(arg string)

JS example:

// assuming the wasm module has been instantiated
go.importObject.env = {
    'HelloArg': function(arg) {
        result = new TextDecoder('utf-8').decode(new Uint8Array(wasm.exports.memory.buffer, arg, 12));
        console.log(result) // logs "Hello World!"
    }
}

@aykevl
Copy link
Member

aykevl commented Aug 26, 2024

"a string is actually passed around as a pointer to the string data"

That's only half true. A string is basically struct{data *byte; len int}, which gets expanded in parameters. So in your example, you can use the following (untested):

    'HelloArg': function(ptr, len) {
        result = new TextDecoder('utf-8').decode(new Uint8Array(wasm.exports.memory.buffer, ptr, len));
        console.log(result) // logs "Hello World!"
    }

Note ptr, len instead of arg.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wasm WebAssembly
Projects
None yet
Development

No branches or pull requests

7 participants