Running tests in headless mode for Firefox is failing with 500 code response but ok in web browser

[arn-the-long-beard]

Describe the Bug

Hello guys !

First, I want to say thank you so much for wasm-bindgen. It feel soooo good to use Rust to do web stuff <3 !!!

Now, since today my command wasm-pack test --firefox --headless is failing this way on fresh clone from the tutorial repos and older project when I followed the nice tutorial.

Running headless tests in Firefox on `http://127.0.0.1:35995/`
Try find `webdriver.json` for configure browser's capabilities:
Not found
driver status: signal: 9                          
driver stdout:
    1596024416809	geckodriver	INFO	Listening on 127.0.0.1:35995

Error: non-200 response code: 500                 
{"value":{"error":"unknown error","message":"Invalid Content-Type","stacktrace":""}}
error: test failed, to rerun pass '--test web'
Error: Running Wasm tests with wasm-bindgen-test failed
Caused by: failed to execute `cargo test`: exited with exit code: 1
  full command: "cargo" "test" "--target" "wasm32-unknown-unknown"

If I run just wasm-pack test --firefox and go in my Firefox, everything works as expected.

Am I missing something ? I read the tutorial over and over and I am not sure what is happening or if I should post this issue there : https://github.com/mozilla/geckodriver/issues

Steps to Reproduce

1. -> git clone https://github.com/rustwasm/wasm_game_of_life.git
2. -> cd wasm_game_of_life
3. -> wasm-pack test --firefox --headless

Expected Behavior

Should :

• detect tests
• run tests
• pass tests

example from wasm-pack test --chrome --headless

INFO]: Checking for the Wasm target...
    Finished dev [unoptimized + debuginfo] target(s) in 0.02s
[INFO]: Installing wasm-bindgen...
    Finished test [unoptimized + debuginfo] target(s) in 0.01s
     Running target/wasm32-unknown-unknown/debug/deps/wasm_game_of_life-8e78d1069e23af04.wasm
no tests to run!
     Running target/wasm32-unknown-unknown/debug/deps/web-ceee830f7a9d7373.wasm
Set timeout to 20 seconds...
Running headless tests in Chrome on `http://127.0.0.1:34345/`
Try find `webdriver.json` for configure browser's capabilities:
Not found
running 2 tests                                   

test web::test_tick ... ok
test web::pass ... ok

test result: ok. 2 passed; 0 failed; 0 ignored

Actual Behavior

Running headless tests in Firefox on `http://127.0.0.1:35995/`
Try find `webdriver.json` for configure browser's capabilities:
Not found
driver status: signal: 9                          
driver stdout:
    1596024416809	geckodriver	INFO	Listening on 127.0.0.1:35995

Error: non-200 response code: 500                 
{"value":{"error":"unknown error","message":"Invalid Content-Type","stacktrace":""}}
error: test failed, to rerun pass '--test web'
Error: Running Wasm tests with wasm-bindgen-test failed
Caused by: failed to execute `cargo test`: exited with exit code: 1
  full command: "cargo" "test" "--target" "wasm32-unknown-unknown"

Additional Context

On my linux my mozilla is 78.0.2 (64-bit)

Linux Mint 19.3
Ubuntu Bionic

Additional Details

Everything was working good last week. I see that geckodriver got an update on Monday , should I post the issue there maybe ? Is it maybe related ?

Thank you for your feedback & help guys !!!

PS: this is maybe the first big issue I ever posted, every feedback is welcomed to improve my writing style :)

[alexcrichton]

Thanks for the report! Can you try using the latest geckodriver and see if it works? If that's the case then you'll need to either configure that or wasm-pack will need to update.

[arn-the-long-beard]

Hello !
Thank you for your answer.

Well, I succeed to install the latest geckodriver as you can see there

geckodriver 0.27.0 (7b8c4f32cdde 2020-07-28 18:16 +0000) 
WebDriver implementation for Firefox

USAGE:
    geckodriver [FLAGS] [OPTIONS]

FLAGS:
        --connect-existing    Connect to an existing Firefox instance
    -h, --help                Prints this message
        --jsdebugger          Attach browser toolbox debugger for Firefox
    -v                        Log level verbosity (-v for debug and -vv for trace level)
    -V, --version             Prints version and copying information

OPTIONS:
    -b, --binary <BINARY>           Path to the Firefox binary
        --log <LEVEL>               Set Gecko log level [possible values: fatal, error, warn, info, config, debug, trace]
        --marionette-host <HOST>    Host to use to connect to Gecko [default: 127.0.0.1]
        --marionette-port <PORT>    Port to use to connect to Gecko [default: system-allocated port]
        --host <HOST>               Host IP to use for WebDriver server [default: 127.0.0.1]
    -p, --port <PORT>               Port to use for WebDriver server [default: 4444]

I am not sure how to run my tests with it to check for the bug. Is there a way somewhere to specify wasm-pack to use this one ?

[alexcrichton]

Sorry I'm not sure about wasm-pack, but if you run wasm-bindgen-test manually you can set the GECKODRIVER env var.

[arn-the-long-beard]

Hello :) What do you mean by running wasm-bindgen-test manually ?

I have it in my Cargo.toml and I tagged tests with it like in https://rustwasm.github.io/docs/book/game-of-life/testing.html

How should I set GECKODRIVER env variable ? Should I just set the version or the path ? I already have the path.

I am very new to this environment, I need some help because I do not know what to do :P

Thank you !

[chinedufn]

There are a few sections in the guide about wasm-bindgen-test https://rustwasm.github.io/wasm-bindgen/wasm-bindgen-test/usage.html#appendix-using-wasm-bindgen-test-without-wasm-pack

You can follow those instructions and then try

GECKODRIVER=geckodriver cargo test --target wasm32-unknown-unknown

[arn-the-long-beard]

Thank you!

I did spend some time on the doc :)

GECKODRIVER=geckodriver cargo test --target wasm32-unknown-unknown fails with Syntax error: end of file unexpected

I am not sure what's is happening there because the file that it is complaining about works fine.

[alexcrichton]

When testing you'll need to also configure CARGO_TARGET_WASM32_UNKNOWN_UNKNOWN_RUNNER to wasm-bindgen-test as an env var.

[alexlapa]

I've also encountered this issue, i've done some tests, and it seems that it is geckodriver v0.27 issue. Test complete if i use geckodriver 0.26 + Firefox 79/78.02, but fail with geckodriver 0.27 + Firefox 79/78.02.

[arn-the-long-beard]

Thank you for your message ! I am sure that is the same issue for me because everything was working good until the 27/07 .

[MartinKavik]

Hey guys, if somebody wants to dive deeper into wasm-pack and resolve the problem by dynamically choosing the required/compatible/whitelisted geckodriver and install it automatically, I would start probably here - https://github.com/rustwasm/wasm-pack/blob/master/src/test/webdriver/geckodriver.rs.
P.S. I've written the most of this code in this PR a year ago, so I don't know how much I will be able to help.

[arn-the-long-beard]

I opened a PR.

By the way, should we report he bug about geckodriver there as well ?
https://github.com/mozilla/geckodriver/issues

[alexlapa]

Being able to select geckodriver version is a good thing, however im not sure that it fixes this issue.

AFAIK, it is still unclear whether it is a bug in geckodriver v.0.27, and we have nothing to do here, or v0.27 introduced some BC break, and wasm-bindgen / wasm-pack should be updated.

I will dive deeper into this later this week, if no one else wants to.

[arn-the-long-beard]

Hello :)
Thank you for your message.
I agree with you. But I do not have the skills to investigate. It would be very nice if you do it and then we can post together if needed 👍

[josephlr]

Would it be possible to get a release including this fix? We had to disable Firefox CI testing, but this change would let use reenable it.

[alexcrichton]

Sure thing! I'll be doing that in #2305