Backends

Backends

For use in Browsers, Human library includes pre-bundled CPU, WASM and WebGL backends,
there is no need to load them externally

For use in NodeJS, Human library supports tfjs-node and tfjs-node-gpu implementations of tensorflow backend
which have to be loaded before Human library

Experimental support is added for WebGPU backend in Browsers and WASM backend in NodeJS

Backend Performance Compare

Performance compare using single model:

Backend	Variation	Warmup	Average
CPU		1230 ms	1230 ms
WASM	Default	338 ms	255 ms
WASM	SIMD	125 ms	62 ms
WebGL	Default	6015 ms	18 ms
WebGL	Pack Disable	2698 ms	22 ms
WebGL	Uniforms Enable	2082 ms	20 ms
WebGPU	GSLS	3877 ms	12 ms
WebGPU	WGSL	N/A	N/A
TFlite		N/A	N/A
TFlite	SIMD	N/A	N/A

Tensorflow Backend

When used in NodeJS, load either tfjs-node or tfjs-node-gpu modules before loading Human library
and set Human config.backend to tensorflow

If using tfjs-node-gpu module and there is no available GPU acceleratrion, backend will print warning messages
during execution and will fall back to CPU execution model instead

GPU acceleration in NodeJS is currently supported for nVidia GPU cards

For requirements and how to unable GPU acceleration using tfjs-node-gpu in
native Linux, Windows or Windows WSL2 environments,
see appropriate docs on nVidia CUDA

WebGL Backend

WebGL backend usage is fastest on systems with modern GPU
although startup speed is a bit slower

HumanGL Backend

Human library includes custom version of WebGL backend with specific optimizations
to enable enhanced workflows and set specific WebGL options to increase warmup speed
and overal performance of the WebGL backend

HumanGL is a default backend for Browser implementations

CPU Backend

CPU backend usage is not recommended for performance reasons
as TensorFlow implementation in JavaScript is by far the slowest

WASM Backend

WASM backend has good resuts under specific conditions:

• Enable WASM SIMD support in your browser
  (CPU parallel processing instructions)
• Reduce size of input image (e.g. webcam)
  as WASM does not have optimized image extraction methods

Loading WASM Files

WASM backend requires access to wasm files which are part of @tensorflow/tfjs-backend-wasm package and included in
node_modules/@tensorflow/tfjs-backend-wasm/dist/*.wasm (different WASM file is loaded depending on detected capabilities)

Also, to be able to load WASM files your web server, web server has to set appropriate HTTP headers:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

Or configure Human load WASM files directly from a CDN:

wasmPath: 'https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]/dist/'

Note that version of WASM binaries must match version of TFJS used by Human library

How to enable WASM SIMD support

Chrome:

• Navigate to chrome://flags
• WebAssembly SIMD support set to Enabled

Edge:

• Navigate to chrome://flags
• WebAssembly SIMD support set to Enabled

Firefox:

• Navigate to about:config
• javascript.options.wasm_simd set to True

WebGPU Backend

In the long-term, WebGPU backend will superseed existing WebGL backend - it's intention is to support same workflows and features while having lower overhead while communicating with GPU

Currently, WebGPU support is only present in Canary version of Google Chrome
And must be explicitly enabled via --enable-unsafe-gpu flag

Due to it's experimental nature WebGPU module is not bundled with Human and instead it has to be manually loaded
After WebGPU has been loaded, simply set Human config.backend to WebGPU and it will perform additional checks and initializations as needed

Note that @tensorflow/tfjs-backend-webgpu is currently in alpha phase of development and released packages are often too old
Instead, it is recommended to build @tensorflow/tfjs-backend-webgpu from sources directly from main branch

Known Issues

See <TODO.md>