Use Chrome Headless instead of Electron

[mramato]

Since Chrome added an officially supported headless mode, there's no reason to use Electron for our unit tests. This will be both more accurate (uses actual Chrome) and also gets rid of the rather large Electron
dependency.

We also want to start testing FirefoxHeadless on CI as well, but need to wait until #6539 is either worked around or fixed.

[cesium-concierge]

@mramato, thanks for the pull request! Maintainers, we have a signed CLA from @mramato, so you can review this at any time.

⚠️ I noticed that CHANGES.md has not been updated. If this change updates the public API in any way, fixes a bug, or makes any non-trivial update, please add a bullet point to CHANGES.md and comment on this pull request so we know it was updated. For more info, see the Pull Request Guidelines.

---

I am a bot who helps you make Cesium awesome! Contributions to my configuration are welcome.

🌍 🌎 🌏

[mramato]

Apparently this is having some issues on Windows, so hold off until I bump it again.

[mramato]

Nevermind, I'm just a moron who forgot to run build before testing on Windows 😄

This is ready.

[ggetz]

Looks good! I think there are a few places the Testing Guide may need updates.

[hpinkos]

@ggetz did you run the tests locally? I couldn't get them to finish on this branch but they finish in master. I ran npm install and npm run build before trying to run the tests and they still failed

[ggetz]

@hpinkos I was able to run locally, but I'll blow away node_modules and double check.

[mramato]

@ggetz did you run the tests locally? I couldn't get them to finish on this branch but they finish in master. I ran npm install and npm run build before trying to run the tests and they still failed

@hpinkos, what exactly failed for you? What OS and version of Chrome are you running? Did you do a clean npm install?

[hpinkos]

@mramato it said it timed out. Windows 10, Chrome 67. I didn't try blowing away node_modules but I did npm install. I'll try again.

[hpinkos]

It's probably just my computer that's weird. If it passes on travis and passes for both of you guys it's probably good to go

[hpinkos]

Yeah, it just failed again after a clean npm install

HeadlessChrome 0.0.0 (Windows 10 0.0.0) ERROR
  {
    "message": "An error was thrown in afterAll\nError: Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.\n    at <Jasmine>",
    "str": "An error was thrown in afterAll\nError: Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.\n    at <Jasmine>"
  }

HeadlessChrome 0.0.0 (Windows 10 0.0.0): Executed 7699 of 9975 (51 FAILED) (skipped 8) ERROR (7 mins 14.436 secs / 6 mins 43.495 secs)

I guess my computer is just too slow

[mramato]

Well, we shouldn't just dismiss your failure, but weird things have been going on with our Unit tests for the past few releases, particularly related to WebGL content. Does it work if you run:

npm run test -- --webgl-stub

[mramato]

Let's not merge this until I track down the problem @hpinkos is seeing, since I'm sure it will affect more users.

@hpinkos Also, I assume running the tests through the "old" web page interface works for you?

[hpinkos]

@mramato no from the command line. npm run test

[hpinkos]

Whoops, totally misread that. Yeah, running from the website interface works fine. I'm running with the webgl stub now to see what works

[hpinkos]

Yes, everything passed with npm run test -- --webgl-stub. It's just npm run test that's failing a bunch of tests and timing out

[ggetz]

I'm getting the same results as @hpinkos. Everything is passing with -- --webgl-stub, but running the normal test target results in 55 failures.

[mramato]

So I think ChromeHeadless is running with SwiftShader WebGL (i.e. software rendering) on Windows, which is why it seems to have WebGL related failures. I need to figure out if there's a good workaround. If not, we can still use ChromeHeadless for travis and Chrome on dev (instead of Electron), but it will probably be a few days before I get back to this. I'll bump when it's ready for another look.

[mramato]

OK, ready for another look. WebGL is disabled in headless, so we only use headless on CI (because we also disable webGL tests there). When running locally, it just shows an actual Chrome window. Chrome has an issue and plans to address this: https://bugs.chromium.org/p/chromium/issues/detail?id=765284 so once WebGL is available in headless we can choose to run the tests headless too (but now that I think about it, most of the time you probably want to debug with the browser anyway).

[hpinkos]

Works great @mramato, thanks! I'll merge as soon as CI passes

[ggetz]

I think the Test Guide still needs to be updated to remove references to Electron.

[hpinkos]

I updated the doc. This looks good, thanks again @mramato!

[mramato]

Thanks @hpinkos, the doc actually just needed to be removed (and the Webstorm configuration updated to use Chrome), we can't run Chrome headless locally until it supports WebGL. I submitted the small change directly into master with da79edb