Support non-fs inputs

[kentosugama]

Hi!

This tool is very helpful. I had one gripe I was wondering about.

Is there any plans to allow modules to be programmatically passed to wasm-opt (even if it's unstructured binary being passed as an argument to the API) without first writing out to a file on the file system?

wasm-opt-rs/components/wasm-opt/src/run.rs

Line 57 in 66b161e

/// The Rust API does not support reading a module on stdin, as the CLI

seems to be related.

The motivation for my request is that I may want to use this tool on a platform where I do not have fs access.

Thank you!

[kentosugama]

Or maybe using a memfile to act as a bridge

https://docs.rs/memfile/latest/memfile/#

[brson]

@kentosugama I think that is a feature we can add. The reason the crate doesn't support it currently is because the underlying binaryen APIs don't, so it will take some upstream work to make it possible in binaryen itself. I can get started on it this month.

Just some notes to myself here:

It looks to me like both the binaryen ModuleWriter and ModuleReader would need to get new functions, like

void ModuleReader::readBinaryDataMem(std::vector<char>& input,
                      Module& wasm,
                      std::vector<char>& sourceMap);

void writeTextMem(Module& wasm, std::vector<char>& output);
void writeBinaryMem(Module& wasm, std::vector<char>& output);

Then we can bind those as we do others. The surface API can add two methods to OptimizationOptions:

    pub fn run_mem(
        &self,
        infile: &[u8],
        outfile: &mut Vec<u8>,
    ) -> Result<(), OptimizationError>;
    
    pub fn run_with_sourcemaps_mem(
        &self,
        infile: &[u8],
        infile_sourcemap: Option<&[u8]>,
        outfile: &mut Vec<u8>,
        outfile_sourcemap: Option<&mut Vec<u8>>,
        sourcemap_url: Option<impl AsRef<str>>,
    ) -> Result<(), OptimizationError>

I am not sure offhand how to deal with in-memory output sourcemaps in the binaryen api.

[kentosugama]

Okay, that would be awesome. The API you sketched out looks exactly like what I need.

[brson]

I've filed an issue upstream to see how they feel about adding the APIs to binaryen.

@kentosugama In your use case would you still be compiling the wasm-opt crate for a "normal" host environment, or do you need to run on a more exotic target like wasm32-unknown-unknown? I don't know to what extent binaryen supports targets that don't have typical OS.

[kentosugama]

Hey Brian. Thanks for the update!

wasm32-unknown-unknown was the target I had in mind. I guess if binaryen doesn't support it then I would have to take it up with them directly.