Use C++ callbacks

[beaufortfrancois]

No description provided.

[beaufortfrancois]

@kainino0x I'm looking at using C++ callbacks in this app and while it's working natively with Dawn ToT, it fails with Emscripten ToT with the following error:

$ /Users/fbeaufort/github/emscripten/emcmake cmake -B build-web && cmake --build build-web -j100
configure: cmake -B build-web -DCMAKE_TOOLCHAIN_FILE=/Users/fbeaufort/github/emscripten/cmake/Modules/Platform/Emscripten.cmake -DCMAKE_CROSSCOMPILING_EMULATOR=/opt/homebrew/opt/node/bin/node
-- Configuring done (0.4s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/fbeaufort/github/webgpu-cross-platform-app/build-web
[ 50%] Building CXX object CMakeFiles/app.dir/main.cpp.o
/Users/fbeaufort/github/webgpu-cross-platform-app/main.cpp:35:16: error: cannot initialize a parameter of type 'RequestAdapterCallback' (aka 'void (*)(WGPURequestAdapterStatus, WGPUAdapterImpl *, const char *, void *)') with an rvalue of type 'wgpu::CallbackMode'
   35 |       nullptr, wgpu::CallbackMode::AllowSpontaneous,
      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/fbeaufort/github/emscripten/cache/sysroot/include/webgpu/webgpu_cpp.h:977:91: note: passing argument to parameter 'callback' here
  977 |         void RequestAdapter(RequestAdapterOptions const * options, RequestAdapterCallback callback, void * userdata) const;
      |                                                                                           ^
/Users/fbeaufort/github/webgpu-cross-platform-app/main.cpp:50:16: error: cannot initialize a parameter of type 'RequestDeviceCallback' (aka 'void (*)(WGPURequestDeviceStatus, WGPUDeviceImpl *, const char *, void *)') with an rvalue of type 'wgpu::CallbackMode'
   50 |       nullptr, wgpu::CallbackMode::AllowSpontaneous,
      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/fbeaufort/github/emscripten/cache/sysroot/include/webgpu/webgpu_cpp.h:803:87: note: passing argument to parameter 'callback' here
  803 |         void RequestDevice(DeviceDescriptor const * descriptor, RequestDeviceCallback callback, void * userdata) const;
      |                                                                                       ^
2 errors generated.
make[2]: *** [CMakeFiles/app.dir/main.cpp.o] Error 1
make[1]: *** [CMakeFiles/app.dir/all] Error 2
make: *** [all] Error 2

Do you know what are the steps to fix this issue? Or are you already working on this by any chance?

[kainino0x]

Emscripten doesn't implement any of the Futures-enabled async stuff yet. I'm supposed to implement it but my higher priority right now is temporarily forking the Emscripten bindings so we can deal with the necessary breaking changes without breaking Emscripten users continually.

[beaufortfrancois]

@lokokung With https://dawn-review.googlesource.com/c/dawn/+/202176 now landed, is it a good time to revisit this PR?

[lokokung]

Almost! That CL only landed changes for requestAdapter, but I think this PR requires requestDevice also. I'm currently working on that, so hopefully soon!

[beaufortfrancois]

@lokokung I assume https://dawn-review.googlesource.com/c/dawn/+/203421 is the CL you mentioned.

[lokokung]

Yup! I'm hoping to land that sometime this week!

[beaufortfrancois]

@lokokung @kainino0x I'd like to create a emdawnwebgpu branch in https://github.com/beaufortfrancois/webgpu-cross-platform-app that uses emdawnwebgpu to switch to the C++ callbacks and the Futures.

I assume I'd need to follow steps in https://dawn.googlesource.com/dawn/+/refs/heads/main/src/emdawnwebgpu/README.md but I'm not sure how this applies to the context of this app.

Can you explain how to do that properly?

[beaufortfrancois]

FYI Here's what I'm stuck for now:

out/Debug/args.gn contains

use_remoteexec = true
dawn_emscripten_dir="/Users/fbeaufort/github/emscripten"

emscripten/.emscripten contains

LLVM_ROOT = '/Users/fbeaufort/github/emsdk/upstream/bin'
BINARYEN_ROOT = '/Users/fbeaufort/github/emsdkupstream'
NODE_JS = '/Users/fbeaufort/github/emsdk/node/18.20.3_64bit/bin/node'

And despite this, I get the following when running autoninja -C out/Debug emdawnwebgpu in Dawn ToT:

Proxy started successfully.
ninja: Entering directory `out/Debug'
[1/3] CXX obj/src/emdawnwebgpu/emdawnwebgpu/webgpu.o
FAILED: obj/src/emdawnwebgpu/emdawnwebgpu/webgpu.o 
../../buildtools/reclient/rewrapper -cfg=../../buildtools/reclient_cfgs/chromium-browser-clang/rewrapper_mac.cfg -exec_root=/Users/fbeaufort/dawn/ ../../third_party/llvm-build/Release+Asserts/bin/clang++ -MMD -MF obj/src/emdawnwebgpu/emdawnwebgpu/webgpu.o.d -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_LIBCPP_HARDENING_MODE=_LIBCPP_HARDENING_MODE_EXTENSIVE -D__ARM_NEON__=1 -DCR_XCODE_VERSION=1500 -DCR_CLANG_REVISION=\"llvmorg-20-init-1009-g7088a5ed-10\" -DCOMPONENT_BUILD -DCR_LIBCXX_REVISION=450ae0d29766e87ea12148e8c6c3352053f78e15 -DTEMP_REBUILD_HACK -D_DEBUG -DDYNAMIC_ANNOTATIONS_ENABLED=1 -I../.. -Igen -I../../buildtools/third_party/libc++ -Igen/src/emdawnwebgpu/include -Wall -Wextra -Wimplicit-fallthrough -Wextra-semi -Wunreachable-code-aggressive -Wthread-safety -Wunguarded-availability -Wno-missing-field-initializers -Wno-unused-parameter -Wno-psabi -Wloop-analysis -Wno-unneeded-internal-declaration -Wno-cast-function-type -Wno-deprecated-this-capture -Wno-invalid-offsetof -Wno-vla-extension -Wno-thread-safety-reference-return -Wshadow -Werror -fno-delete-null-pointer-checks -fno-ident -fno-strict-aliasing -fstack-protector-strong -fcolor-diagnostics -fmerge-all-constants -fno-sized-deallocation -fcrash-diagnostics-dir=../../tools/clang/crashreports -mllvm -instcombine-lower-dbg-declare=0 -mllvm -split-threshold-for-reg-with-hint=0 -ffp-contract=off -fcomplete-member-pointers --target=arm64-apple-macos -mno-outline -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -ffile-compilation-dir=. -no-canonical-prefixes -ftrivial-auto-var-init=pattern -O0 -fno-omit-frame-pointer -g2 -gdwarf-aranges -Xclang -debug-info-kind=limited -isysroot ../../build/mac_files/xcode_binaries/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX14.0.sdk -mmacos-version-min=11.0 -fvisibility=hidden -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -Wenum-compare-conditional -Wno-c++11-narrowing-const-reference -Wno-missing-template-arg-list-after-template-kw -Wno-dangling-assignment-gsl -Wno-undefined-bool-conversion -Wno-tautological-undefined-compare -std=c++20 -Wno-trigraphs -fno-exceptions -fno-rtti -nostdinc++ -isystem../../third_party/libc++/src/include -isystem../../third_party/libc++abi/src/include -fvisibility-inlines-hidden -c ../../third_party/emdawnwebgpu/webgpu.cpp -o obj/src/emdawnwebgpu/emdawnwebgpu/webgpu.o
../../third_party/emdawnwebgpu/webgpu.cpp:10:10: fatal error: 'emscripten/emscripten.h' file not found
   10 | #include <emscripten/emscripten.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
ninja: build stopped: subcommand failed.
RBE Stats: down 0 B, up 0 B, 1 non zero exit

[lokokung]

Hmm, so it seems like right now the CMake config is trying to build Dawn via the gn build and then using CMake to use the results to build the demo. I haven't ever tried this path so I wouldn't be surprised if it doesn't actually work... My immediate suggestion to see if it would work would be to try using the CMake version of the build for Emscripten for now so something like:

if(EMSCRIPTEN)
  set_target_properties(app PROPERTIES SUFFIX ".html")
  target_link_libraries(app webgpu_cpp)
else()
  set(DAWN_FETCH_DEPENDENCIES ON)
  add_subdirectory("dawn" EXCLUDE_FROM_ALL)
  target_link_libraries(app PRIVATE dawn::webgpu_dawn glfw webgpu_glfw)
endif()

Note that the webgpu_cpp target should resolve to the Emscripten stuff as long as you set up the directory with emcmake as per the instructions in the README. Let me know if that helps! (If not I can try to checkout this repo and see if I can get it to work later on)

[beaufortfrancois]

@lokokung FWIW The instructions for "Building Dawn Emscripten bindings and samples with CMake" don't work as well:

fbeaufort-macbookpro:cmake-wasm fbeaufort$ emcmake cmake -GNinja -DDAWN_EMSCRIPTEN_TOOLCHAIN="/Users/fbeaufort/github/emscripten" ../..
configure: cmake -GNinja -DDAWN_EMSCRIPTEN_TOOLCHAIN=/Users/fbeaufort/github/emscripten ../.. -DCMAKE_TOOLCHAIN_FILE=/opt/homebrew/Cellar/emscripten/3.1.65/libexec/cmake/Modules/Platform/Emscripten.cmake -DCMAKE_CROSSCOMPILING_EMULATOR=/opt/homebrew/opt/node/bin/node
...
-- Build files have been written to: /Users/fbeaufort/dawn/out/cmake-wasm

fbeaufort-macbookpro:cmake-wasm fbeaufort$ ninja
[1/3] Linking CXX executable ComputeBoids.html
FAILED: ComputeBoids.html 
: && /opt/homebrew/Cellar/emscripten/3.1.65/libexec/em++ -g -sUSE_WEBGPU=0 -sASYNCIFY=1 --js-library=/Users/fbeaufort/dawn/out/cmake-wasm/gen/src/emdawnwebgpu/library_webgpu_enum_tables.js --js-library=/Users/fbeaufort/dawn/out/cmake-wasm/gen/src/emdawnwebgpu/library_webgpu_generated_struct_info.js --js-library=/Users/fbeaufort/dawn/out/cmake-wasm/gen/src/emdawnwebgpu/library_webgpu_generated_sig_info.js --js-library=/Users/fbeaufort/dawn/third_party/emdawnwebgpu/library_webgpu.js --js-library=/Users/fbeaufort/dawn/third_party/emdawnwebgpu/library_html5_webgpu.js --closure-args=--externs=/Users/fbeaufort/github/emscripten/src/closure-externs/webgpu-externs.js src/dawn/samples/CMakeFiles/ComputeBoids.dir/ComputeBoids.cpp.o -o ComputeBoids.html  src/dawn/samples/libdawn_sample_utils.a  src/dawn/utils/libdawn_system_utils.a  third_party/abseil/absl/strings/libabsl_cord.a  third_party/abseil/absl/strings/libabsl_cordz_info.a  third_party/abseil/absl/strings/libabsl_cord_internal.a  third_party/abseil/absl/strings/libabsl_cordz_functions.a  third_party/abseil/absl/strings/libabsl_cordz_handle.a  third_party/abseil/absl/crc/libabsl_crc_cord_state.a  third_party/abseil/absl/crc/libabsl_crc32c.a  third_party/abseil/absl/crc/libabsl_crc_internal.a  third_party/abseil/absl/crc/libabsl_crc_cpu_detect.a  third_party/abseil/absl/strings/libabsl_str_format_internal.a  third_party/abseil/absl/container/libabsl_raw_hash_set.a  third_party/abseil/absl/hash/libabsl_hash.a  third_party/abseil/absl/hash/libabsl_city.a  third_party/abseil/absl/types/libabsl_bad_variant_access.a  third_party/abseil/absl/hash/libabsl_low_level_hash.a  third_party/abseil/absl/container/libabsl_hashtablez_sampler.a  third_party/abseil/absl/profiling/libabsl_exponential_biased.a  third_party/abseil/absl/synchronization/libabsl_synchronization.a  third_party/abseil/absl/debugging/libabsl_stacktrace.a  third_party/abseil/absl/synchronization/libabsl_graphcycles_internal.a  third_party/abseil/absl/synchronization/libabsl_kernel_timeout_internal.a  third_party/abseil/absl/debugging/libabsl_symbolize.a  third_party/abseil/absl/base/libabsl_malloc_internal.a  third_party/abseil/absl/debugging/libabsl_debugging_internal.a  third_party/abseil/absl/debugging/libabsl_demangle_internal.a  third_party/abseil/absl/time/libabsl_time.a  third_party/abseil/absl/strings/libabsl_strings.a  third_party/abseil/absl/strings/libabsl_strings_internal.a  third_party/abseil/absl/strings/libabsl_string_view.a  third_party/abseil/absl/base/libabsl_base.a  third_party/abseil/absl/base/libabsl_spinlock_wait.a  third_party/abseil/absl/numeric/libabsl_int128.a  third_party/abseil/absl/time/libabsl_civil_time.a  third_party/abseil/absl/time/libabsl_time_zone.a  third_party/abseil/absl/types/libabsl_bad_optional_access.a  src/dawn/utils/libdawn_wgpu_utils.a  src/dawn/common/libdawn_common.a  src/emdawnwebgpu/libemdawnwebgpu_c.a  third_party/abseil/absl/base/libabsl_throw_delegate.a  third_party/abseil/absl/base/libabsl_raw_logging_internal.a  third_party/abseil/absl/base/libabsl_log_severity.a && :
cache:INFO: generating system asset: symbol_lists/91b060de6e469d8de22be3204cb50853754d5385.json... (this will be cached in "/opt/homebrew/Cellar/emscripten/3.1.65/libexec/cache/symbol_lists/91b060de6e469d8de22be3204cb50853754d5385.json" for subsequent builds)
error: /Users/fbeaufort/dawn/third_party/emdawnwebgpu/library_webgpu.js: failure to execute js library "/Users/fbeaufort/dawn/third_party/emdawnwebgpu/library_webgpu.js":
error: /Users/fbeaufort/dawn/third_party/emdawnwebgpu/library_webgpu.js: use -sVERBOSE to see more details
Internal compiler error in src/compiler.js!
Please create a bug report at https://github.com/emscripten-core/emscripten/issues/
with a log of the build and the input files used to run. Exception message: "TypeError: Cannot read properties of undefined (reading 'toString')
    at file:///opt/homebrew/Cellar/emscripten/3.1.65/libexec/src/parseTools.mjs:38:31
    at String.replace (<anonymous>)
    at processMacros (file:///opt/homebrew/Cellar/emscripten/3.1.65/libexec/src/parseTools.mjs:36:15)
    at Object.load (file:///opt/homebrew/Cellar/emscripten/3.1.65/libexec/src/modules.mjs:248:21)
    at Module.runJSify (file:///opt/homebrew/Cellar/emscripten/3.1.65/libexec/src/jsifier.mjs:169:18)
    at file:///opt/homebrew/Cellar/emscripten/3.1.65/libexec/src/compiler.mjs:89:11
em++: error: '/opt/homebrew/opt/node/bin/node /opt/homebrew/Cellar/emscripten/3.1.65/libexec/src/compiler.mjs /var/folders/v7/ws9m3q151hdb_pfby31y1lx0005z_t/T/tmpamkm9vda.json --symbols-only' failed (returned 1)
[2/3] Linking CXX executable HelloTriangle.html
FAILED: HelloTriangle.html 
...

I'd appreciate any help you may offer as I'm stuck now.

[kainino0x]

@beaufortfrancois I put up a fix for that here: https://dawn-review.googlesource.com/c/dawn/+/205614

[lokokung]

You'll also need this https://dawn-review.googlesource.com/c/dawn/+/205814 after https://dawn-review.googlesource.com/c/dawn/+/197476 was landed.

[kainino0x]

One more important fix to land: https://dawn-review.googlesource.com/c/dawn/+/205874

Please see https://github.com/kainino0x/webgpu-cross-platform-demo/tree/dawnwasm for how I've made the emdawnwebgpu build work. It's a little messy to set up but it is working! (No longer using the gn build, now using the cmake build since Loko set that up)

[beaufortfrancois]

Thank you both for landing all the necessary changes! I'll give it a try!