Add ability to refresh wasm middleware without restart

[codefromthecrypt]

Describe the feature

WebAssembly has an interesting ability to be safely reloadable. I suspect we should have either an implicit file watch, or another control to allow refreshing the wasm without restarting the sidecar.

Release Note

ADD HTTP wasm middleware can now be configured with watchPath: true to reload the wasm binary upon change.

[codefromthecrypt]

cc @daixiang0 @ItalyPaleAle @anuraaga for ideas.. maybe swap out this function with some go atomic ref? https://github.com/dapr/components-contrib/blob/master/middleware/http/wasm/httpwasm.go#L148

[codefromthecrypt]

prior art might be helpful too..

@antJack do you know if mosn does hot-reloading of proxy-wasm plugins? @anuraaga do you know on envoy?

[codefromthecrypt]

the primary use case I'm thinking of right now is just development, as it is intuitive to have the server reload if you re-compile the wasm path dapr/samples#122

[anuraaga]

@codefromthecrypt If I'm not mistaken, Envoy doesn't have anything specific for wasm, so not based on file change but a hot restart can be triggered manually with some complexity (it's a process swap though, not a in-process reload)

envoyproxy/envoy#156

[ItalyPaleAle]

I have mixed opinions about this feature.

On one hand, I can see how this could be helpful during development

On the other hand, if used in production, it could lead to unintended consequences as it would essentially allow changing the code that's being executed on-the-fly. When your app scales horizontally, you also need to ensure consistency there.

What do maintainers (CC: @yaron2 @berndverst ) think?

As an aside, you can use dapr/kit/fswatcher to implement a inotify-based watcher.

[dapr-bot]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged (pinned, good first issue, help wanted or triaged/resolved) or other activity occurs. Thank you for your contributions.

[codefromthecrypt]

I think it is ok until users request this