JS: Support javascript extension to SRS

[winlinvip]

We believe that if a streaming server supports an extension language (or scripting language), such as Nginx supporting the Lua scripting language OpenResty, it will cause a disaster of script misuse. This issue has been previously described in comparing Nginx.

Video (or streaming) servers are very complex business servers. If there are scripts that can write complex logic, users generally tend to use scripts for customization, resulting in a huge amount of unmaintainable script code.

Of course, users can also write large and unmaintainable C/C++ code, which ultimately makes our support for scripting languages seem not so evil. If the script is strictly limited, it can be considered.

More importantly, the current situation has changed. SRS's code is increasing, and the core protocols supported are becoming more and more diverse, from RTMP+HLS to the addition of SRT, GB28181, and WebRTC. SRS has no intention of using scripts to implement new protocols. We still believe that video servers should be carefully implemented in C++ and ensure quality, while scripting languages seem inherently difficult to achieve high quality.

As for Lua or WASM, I prefer WASM, so that multiple languages can be used to extend SRS, rather than being limited to Lua.

As for WASM or JS, I prefer JS because the main target group is JS rather than all languages. WASM is mainly for supporting all language extensions, generally for adapting legacy system migration, such as in cloud-native scenarios, Envoy supports WASM extensions, and any backend language can connect its existing language to Envoy. Obviously, SRS's main target is JS, not all languages, nor legacy systems.

Of course, some research needs to be done, especially the degree of fit between ST and JS Runtime, thread model, etc.

[winlinvip]

Why WASM?

Why WASM filters

With WASM filter implementations we get:

• Agility - filters can be dynamically loaded into the running Envoy process without the need to stop or re-compile.
• Maintainability - we don’t have to change the Envoy’s codebase to extend its functionality.
• Diversity - popular programming languages such as C/C++ and Rust can be compiled into WASM, thus developers can implement filters using their programming language of choice.
• Reliability and isolation - filters are deployed into a VM (sandbox), therefore are isolated from the hosting Envoy process itself (e.g. when the WASM filter crashes it will not impact the Envoy process).
• Security - since filters communicate with the host (Envoy Proxy) through a well-defined API, they have access to and can modify only a limited number of connection or request properties.

It also has a few drawbacks that need to be taken into consideration:

• Performance is ~70% as fast as native C++.
• Higher memory usage due to the need to start one or more WASM virtual machines.

Backgroup for WASM in Envoy

As of early 2019, Envoy is a statically compiled binary, with all its extensions compiled at build-time. This means that projects that provide custom extensions (e.g. Istio), must maintain and distribute their own binaries, instead of using the official and unmodified Envoy binary.

For projects that don’t control their deployment, this is even more problematic, because any updates and/or bug fixes to extensions require building new binary, producing release, distributing it, and more importantly, re-deploying it in production.

This also means that there is often a version drift between deployed extensions and the control plane configuring them.

While parts of the issue could be solved using dynamically loadable C++ extensions, this isn’t feasible solution at this time, because, due to the rapid pace of Envoy development, there is no stable ABI, or even API, for the extensions, and more often than not, updating Envoy requires code changes, which makes updating a manual process.

Instead, we’ve decided to address this problem by writing and delivering Envoy extensions in WebAssembly with stable ABI, since it brings many additional benefits (described below).

Benefits for WASM in Envoy

• Agility. Extensions can be delivered and reloaded at runtime, directly from the control plane.
  This means not only that everybody can use the official and unmodified version of a proxy to load
  custom extensions, but also that any bug fixes and/or updates can be pushed and/or tested at
  runtime, without the need to update and/or re-deploy a new binary.

• Reliability and Isolation. Because extensions are deployed inside a sandbox with resource
  constraints, they can crash and/or leak memory, without bringing the whole proxy down.
  Furthermore, CPU and memory usage can be limited.

• Security. Because extensions are deployed inside a sandbox with clearly defined API for
  communicating with a proxy, they have access and can modify only a limited number of properties of
  the connection and/or request. Furthermore, because proxy mediates this interaction, it can hide
  or sanitize sensitive information from the extension (e.g. “Authorization” and “Cookie” HTTP
  headers, or client’s IP address).

• Diversity. Over 30 programming languages can be compiled to WebAssembly modules, allowing
  developers from all backgrounds (C, Go, Rust, Java, TypeScript, etc.) to write Proxy-Wasm
  extensions in their language of choice.

• Maintainability. Thanks to the fact that extensions are written using standard libraries,
  independent from proxy’s codebase, we can provide a stable ABI.

• Portability. Because the interface between host environment and extensions is proxy-agnostic,
  extensions written using Proxy-Wasm can be executed in various proxies, e.g. Envoy, NGINX, ATS or
  even inside the gRPC library (assuming they all implement the standard).

Drawbacks for WASM in Envoy

• Higher memory usage, due to the need to start many virtual machines, each with its own memory
  block.

• Lower performance for extensions transcoding the payload, due to the need to copy significant
  amounts of data in and out of the sandbox.

• Lower performance for CPU-bound extensions. The slowdown is expected to be less than 2x, compared
  to the native code.

• Increased binary size, due to the need to include Wasm runtime. It’s ~20MB for WAVM and ~10MB for
  V8.

• The WebAssembly ecosystem is still young, and the development is currently focused on the
  in-browser usage, where JavaScript is considered to be the host environment.

[winlinvip]

About WASM runtime in Envoy.

Wasm VMs

When a Wasm module is loaded into Envoy, it’s run in its own sandboxed Wasm VM. There are a couple different options currently available:

• Null VM: Uses the same ABI but gets compiled natively into envoy. No Wasm code is created. The null VM API is similar to the C++ SDK’s api.
• V8: The Wasm VM from Chrome; loads fast, but doesn’t compile to native. Tests done by google estimate performance here as 50% of native performance. This sounds like a lot, but might be a win for services that would otherwise do a call-out, like ext auth, etc
• WAVM: A vm that pre-compiles Wasm to native assembly. Loads slower but presumably runs faster.

WAVM

WAVM is also meant to be used in this kind of scenario where it is embedded in another application, though I think with quite different trade-offs from wasmjit. It uses libc and LLVM.

LLVM is quite heavy-weight in terms of both code size and the time it takes to translate the WebAssembly code to native code. It wouldn't make sense in a browser VM, but may be well-suited to an application that compiles a WebAssembly module once at startup, and spends a lot of time running the generated code.

WAVM uses signal handling to make bounds-checking of memory accesses as fast as possible. I believe that v8 supports both signal handling and a slower fallback if installing a signal handler isn't possible. I am very curious to know what your constraints around signal handling are:

• Is it possible for WAVM to install a SIGSEGV/SIGBUS/SIGFPE handler?
• Do you have an existing signal handler for SIGSEGV/SIGBUS/SIGFPE that would need to cooperate with WAVM's signal handler?
• Do you need to be able to handle WebAssembly traps? If so, do you prefer an exception or a return code?

WASM Runtimes in Envoy

• null
• v8
• wasmtime, 4.5K star, Standalone JIT-style runtime for WebAssembly, using Cranelift
• wavm, 1.7K star, WAVM is a WebAssembly virtual machine, designed for use in non-web applications.

Another one, wasmer, 7.5K star, The leading WebAssembly Runtime supporting WASI and Emscripten.

High-level overview For WASM in Envoy

With Proxy-Wasm, developers can write proxy extensions using their programming language of choice,
ideally using language-specific libraries provided by us. Those extensions are then compiled to
portable Wasm modules, and distributed in that format.

On the proxy side, once a Wasm module is loaded (either directly from disk or pushed from the
control plane over xDS), it’s validated for conformance with the defined Proxy-Wasm interface and
instantiated using the embedded Wasm runtime, which creates a new Wasm virtual machine in each
worker thread.

For each of Envoy’s extension types, we’ve created a shim that translates the extension’s interface
to Proxy-Wasm calls, so the interfaces are very similar to those used in native (C++) Envoy
extensions, embracing the event-driven programming model.

Virtual machines for WASM in Envoy

When the Wasm runtime instantiates a Wasm module, it creates a Wasm virtual machine (VM instance)
for it.

There are a few models for mapping between VM instances and Proxy-Wasm extensions. Ultimately, it’s
a trade-off between: startup latency & resource usage, and isolation & security.

• Persistent in-process VM per worker thread per Wasm module (shared across multiple configured
  uses of Wasm extension).
  A single Wasm module can contain multiple extensions (e.g. listener filter and transport socket,
  both in a single package). For each Wasm module, a single persistent in-process VM instance is
  created, and it can (but doesn’t have to) be shared by all Proxy-Wasm extensions referencing that
  Wasm module in the configuration.

• Persistent in-process VM per worker thread per Wasm extension.
  A single persistent in-process VM instance is created for each Wasm extension and it’s shared by
  all Proxy-Wasm extensions referencing the given Wasm module in the configuration, similarly to how
  the native (C++) extensions are instantiated today.

• Persistent in-process VM per worker thread per configured use of Wasm extension.
  A single persistent in-process VM instance is created for each configured use of a Proxy-Wasm
  extension referencing given Wasm module in the configuration. This model provides stronger
  isolation guarantees than the previous models, and it should be preferred in the multi-tenant
  environments.

• Ephemeral (per request) in-process VM.
  A new ephemeral in-process VM instance is created for each request, for each Proxy-Wasm extension,
  and it’s destroyed immediately after the request is finished. This is expected to be prohibitively
  expensive.

• Out-of-process VM.
  This is out of scope for this document, but for deployments loading untrusted (and potentially
  malicious) Wasm modules in multi-tenant environments, that require strong security guarantees and
  want to protect against Spectre-like attacks, proxy should communicate with an out-of-process Wasm
  sandbox (e.g. using Filters-over-gRPC or shared memory) that implements Proxy-Wasm, which would
  execute Wasm modules on its behalf and stream results back to the proxy.

[winlinvip]

Based on WebAssembly Envoy Extension —— GetEnvoy Extension Toolkit Introduction

The getenvoy project provides a convenient way to obtain Envoy and WASM SDK and create extensions, which is worth learning from.

This way, you can deeply customize without modifying Envoy's source code. Unlike scripts, WASM extensions are also written in Rust or Go, with complete testing and control, and can achieve high quality.

[winlinvip]

Refer to Nginx supports JS extension.

To involve JS developers in audio and video, it can only be done through the JS extension method (like Nginx), not the Node.js method.

Using Node.js is like rewriting and has the same idea as Pion, which is not a good path because the main technology stack for audio and video is still C and cannot be changed.

JS extensions allow JS developers to implement their own business under the existing audio and video framework, rather than implementing the entire audio and video framework.

In the early days, Adobe AMS server supported AS extensions, which were similar to JS but not exactly the same. The main problem was that AMS itself was not open source, so features that should have been done in AMS could only be implemented in AS, resulting in extremely bloated and unmaintainable AS. Therefore, SRS has not done extensions to avoid what should have been done in SRS being implemented in extension languages, which is also a disaster.

SRS later researched WASM and found that it is also an extension mechanism, but it is more about solving the problem of programmers writing extensions in multiple languages, such as cloud-native languages that need to be supported. Most of them are for existing frameworks to be able to access. SRS's main goal is not to connect to existing systems but to connect to Node.js community developers.

Therefore, the problem is clear: SRS must and only needs to support JS extensions, just like Nginx supports JS.

The main problem with LUA extension is that it is too niche, JS is too popular, why not JS? It must be JS.

Next, some research needs to be done, including JS runtime and the degree of fit with ST coroutines.