Automated tests fail on Mac OS 10.13

[mcclure]

Last time I checked (early Summer) wgpu was informally supported on MacOS back to 10.13. This made me happy as I am trying to maintain one machine on 10.13. I asked today in the wgpu Matrix what would have to happen for wgpu to formally ("documented in README") support MacOS back to 10.13. cwfitzgerald said at minimum we would need to successfully run cargo xtask test on 10.13. I volunteered to try this.

I installed nextest, checked out the 0.18.0 tag and ran the test command. Nearly all tests failed, including basic tests like test_api. This was a surprise to me, to the extent I suspect something is wrong with the test runner, because on the same machine (MacBookPro12,1 with integrated Intel Iris Graphics 6100, macOS 10.13.6 17G14042) I can successfully run any wgpu examples I like (for example, hello-triangle).

Expected behavior: Unless you have made a conscious decision to stop supporting 10.13, the tests should work on 10.13

0.18.TEST.TXT

[cwfitzgerald]

There are two issues here:

• We unconditionally use MTLComputePassDescriptor but that was introduced in 11.0
• VERTEX_WRITABLE_STORAGE is apparently not unconditionally supported on mac, as this machine doesn't support it.

[mcclure]

For comparison here's the same test run on trunk (last commit Nov 04)

trunk.dfa7e2f8fbc0.TEST.TXT

[teoxoy]

Looking at the logs we get the following errors:

Class with name MTLComputePassDescriptor could not be found

TRY 3 FAIL [   0.249s] player::test test_api
TRY 3 FAIL [   0.298s] wgpu-boids-example::bin/boids [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] boids
TRY 3 FAIL [   0.226s] wgpu-hello-compute-example::bin/hello-compute [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] hello_compute::tests::compute_1
TRY 3 FAIL [   0.282s] wgpu-hello-compute-example::bin/hello-compute [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] hello_compute::tests::compute_2
TRY 3 FAIL [   0.238s] wgpu-hello-compute-example::bin/hello-compute [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] hello_compute::tests::compute_overflow
TRY 3 FAIL [  10.277s] wgpu-hello-compute-example::bin/hello-compute [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] hello_compute::tests::multithreaded_compute
TRY 3 FAIL [   0.223s] wgpu-hello-synchronization-example::bin/hello-synchronization [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] hello_synchronization::tests::sync
TRY 3 FAIL [   0.205s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::bgra8unorm_storage::bgra8_unorm_storage
TRY 3 FAIL [   0.268s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::poll::double_wait
TRY 3 FAIL [   0.267s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::poll::double_wait_on_submission
TRY 3 FAIL [   0.195s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::poll::wait
TRY 3 FAIL [   0.205s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::poll::wait_on_submission
TRY 3 FAIL [   0.242s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::poll::wait_out_of_order
TRY 3 FAIL [   0.182s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::shader::numeric_builtins::numeric_builtins
TRY 3 FAIL [   0.212s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::shader::struct_layout::push_constant_input
TRY 3 FAIL [   0.201s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::shader::struct_layout::storage_input
TRY 3 FAIL [   0.194s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::shader::struct_layout::uniform_input
TRY 3 FAIL [   0.269s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::shader::zero_init_workgroup_mem::zero_init_workgroup_memory
TRY 3 FAIL [   0.236s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::zero_init_texture_after_discard::discarding_depth_target_resets_texture_init_state_check_visible_on_copy_in_same_encoder
TRY 3 FAIL [   0.245s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::zero_init_texture_after_discard::discarding_either_depth_or_stencil_aspect_test

illegal write to device memory for non-void vertex function 'vs_main'

TRY 3 FAIL [   0.292s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::vertex_indices::draw
TRY 3 FAIL [   0.226s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::vertex_indices::draw_instanced
TRY 3 FAIL [   0.286s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::vertex_indices::draw_instanced_offset
TRY 3 FAIL [   0.262s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::vertex_indices::draw_vertex

Expected Mean (0.559973) to be under expected maximum (0.02)

TRY 3 FAIL [   2.703s] wgpu-shadow-example::bin/shadow [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] shadow

texture with format X was not fully cleared

TRY 3 FAIL [   0.239s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::clear_texture::clear_texture_depth
TRY 3 FAIL [   0.312s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::clear_texture::clear_texture_depth32_stencil8

assertion failed: readback_buffer.check_buffer_contents(&ctx.device, &expected_data)

TRY 3 FAIL [   0.203s] wgpu-test::gpu-tests [Executed] [Metal/Intel(R) Iris(TM) Graphics 6100/0] gpu_tests::scissor_tests::scissor_test_empty_rect

[teoxoy]

illegal write to device memory for non-void vertex function 'vs_main'

Seems to be:

from https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf

But it's unclear if this limitation has been lifted in some of the newer Metal versions since we haven't gotten any other reports of this.

https://developer.apple.com/forums/thread/665272 is asking the same thing.

[mcclure]

Upgraded to 1d4fa81 to get PR #4643 and reran tests
Reduced from
"28 failed" to "8 failed"
trunk.1d4fa812ef4d.TEST.2.TXT

[cwfitzgerald]

Any chance you can try this again on trunk?

[mcclure]

Hello,

• MacBookPro12,1 as described in first post, same OS version
• rustup'd to 1.83.0
• updated git¹ checkout to 3cc63af
• cargo show, your rust-toolchain.toml kicked me back down to 1.76. okay well that's fine
• cargo xtask test

I saw no failures but a number of timeouts on the larger benchmarks (I somewhat suspect that these were real timeouts because the laptop is slow). But then…

Somewhere around 220/473 ("Renderpass: Single Threaded/1 renderpasses x 10000 draws (RenderPass Time)") the entire laptop became unresponsive. I could move the cursor freely I could not activate the dock, or select text, or move windows or menus, I could not ctrl-C or bring up the force quit dialog. I had to longpress the power to get it to accept the reboot command.

When this happened, the bottom of the test display was frozen at "Running [ 00:08:36] … 220/473: 4 running, 209 passed (2 slow), 2 failed, 9 timed out, 0 skipped". (I saw a lot more than two "slow"s in the scroll. The only two "fails" i noticed were a 90 second timeout that was for some reason listed as TERMINATING, and another that ended as SIGABRT at 47.162. When I rebooted I found Terminal had saved a complete backscroll.

2024-12-15-xtask-test.txt

It seems possible to me that you either have a bug, or tickled a bug in Apple's drivers, which caused one stresstest test to terminate abnormally with a SIGABRT and another to lock up the machine. However I think what is more likely is that because you are intentionally running many tests at the same time, you used up all the RAM. The machine only has 8GB.

I am happy to try the test again, but before I do, is there a way to force the test runner to limit itself to one test at a time, and/or to extend the timeouts for my poor decade-old integrated GPU's sake?

¹ hg-git. don't worry about it

[cwfitzgerald]

Thanks! So to run one at a time you can pass cargo xtask test -j1 and it should go one at a time. You can also reduce the cost of the benchmarks at test time if you check out this PR: #6725. They were way too high for testing.

[mcclure]

I bumped to 21ff968 . I had a somewhat substantial number of failures:

484 tests run: 459 passed (1 slow), 25 failed, 0 skipped

And unlike yesterday, the failures did not look spurious. I'm not certain, but I think some of the ones that failed were ones that were passing yesterday.

2024-12-16-1103-xtask-test.txt

Let me know if you need anything else.

[cwfitzgerald]

Nice, thanks, that's all we need for now!