-
-
Notifications
You must be signed in to change notification settings - Fork 329
Backends
For use in Browsers, Human
library includes pre-bundled CPU
, WASM
and WebGL
backends,
there is no need to load them externally
For use in NodeJS, Human
library supports tfjs-node
and tfjs-node-gpu
implementations of tensorflow
backend
which have to be loaded before Human
library
Experimental support is added for WebGPU
backend in Browsers and WASM
backend in NodeJS
When used in NodeJS
, load either tfjs-node
or tfjs-node-gpu
modules before loading Human
library
and set Human config.backend
to tensorflow
If using tfjs-node-gpu
module and there is no available GPU acceleratrion, backend will print warning messages
during execution and will fall back to CPU execution model instead
GPU acceleration in NodeJS is currently supported for nVidia GPU cards
For requirements and how to unable GPU acceleration using tfjs-node-gpu
in
native Linux, Windows or Windows WSL2 environments,
see appropriate docs on nVidia CUDA
WebGL backend usage is fastest on systems with modern GPU
although startup speed is a bit slower
Human
library includes custom version of WebGL
backend with specific optimizations
to enable enhanced workflows and set specific WebGL options to increase warmup speed
and overal performance of the WebGL backend
HumanGL
is a default backend for Browser implementations
CPU backend usage is not recommended for performance reasons
as TensorFlow implementation in JavaScript is by far the slowest
WASM backend has good resuts under specific conditions:
- Enable WASM SIMD support in your browser
(CPU parallel processing instructions) - Reduce size of input image (e.g. webcam)
as WASM does not have optimized image extraction methods
WASM backend requires access to wasm
files which are part of @tensorflow/tfjs-backend-wasm
package and included in
node_modules/@tensorflow/tfjs-backend-wasm/dist/*.wasm
(different WASM file is loaded depending on detected capabilities)
Also, to be able to load WASM files your web server, web server has to set appropriate HTTP headers:
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
Or configure Human
load WASM files directly from a CDN:
wasmPath: 'https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]/dist/'
Note that version of WASM binaries must match version of TFJS used by Human
library
Chrome:
- Navigate to chrome://flags
-
WebAssembly SIMD support
set to Enabled
Edge:
- Navigate to chrome://flags
-
WebAssembly SIMD support
set to Enabled
Firefox:
- Navigate to about:config
-
javascript.options.wasm_simd
set to True
In the long-term, WebGPU
backend will superseed existing WebGL
backend - it's intention is to support same workflows and features while having lower overhead while communicating with GPU
Currently, WebGPU support is only present in Canary version of Google Chrome
And must be explicitly enabled via --enable-unsafe-gpu
flag
Due to it's experimental nature WebGPU
module is not bundled with Human
and instead it has to be manually loaded
After WebGPU
has been loaded, simply set Human config.backend
to WebGPU
and it will perform additional checks and initializations as needed
Note that @tensorflow/tfjs-backend-webgpu
is currently in alpha phase of development and released packages are often too old
Instead, it is recommended to build @tensorflow/tfjs-backend-webgpu
from sources directly from main
branch
See <TODO.md>
Human Library Wiki Pages
3D Face Detection, Body Pose, Hand & Finger Tracking, Iris Tracking, Age & Gender Prediction, Emotion Prediction & Gesture Recognition