Cover Several Environments As Part of Testing

[D4nte]

Planned start date:
Due date:

Summary

js-waku library is meant for browser usage.
Most of the testing is currently done via NodeJS, which is not ideal especially that cryptographic functions uses the Web Crypto API or NodeJS's crypto modules or bindings. Meaning testing in NodeJS does not mean it would work in the Browser.

#230 laid some basis on how to get browser testing as part of the CI. However, it is not complete:

1. No Firefox run (only chrome)
2. No test done against nim-waku for js-waku in the browser

Acceptance Criteria

• js-waku tests run against both Chrome and Firefox web engines
• js-waku test include bundling and running a simple apps written in several framework (TBD, at least React and Vite)

Tasks

• Add FirefoxHeadless to the browser list and make it work feat: add Firefox and Webkit to karma  #1598
• Orchestrate the spin up of a nim-waku instance Add interoperability tests with nwaku to browser suite #887 (most likely needed to achieve the above)
• Bundlers support? Do a bundler run as part of the CI #1418

Note: aegir uses Karma for browser tests so it should be possible to test it. It could make orchestration easier thanks to the before and after hooks.

RAID (Risks, Assumptions, Issues and Dependencies)

• Need to avoid scope creep for this issue. In terms of bundler testing, the most demanded framework so far are React and Svelte
• It seems that migration to Jest may be needed to more easily achieve this task

[littlealex003]

You want to deploy the code, fire up a browser and connect to it with something like Selenium, to replicate a real deployment, correct?

[D4nte]

I do not have much knowledge about testing JS code against a browser. What are the options?

Yes, the idea is to ensure that the library is tested in an environment closest to real deployment.

I was going to do something similar to what js-libp2p does as I assume it should be good enough.

[littlealex003]

I will look at the js-libp2p; I mean if we want to we can do a full automated deployment, launch browsers with automated control, and run tests against the actual deployed application, on as many different OS / Browser combinations as we like. It really depends how many platforms we want to support and thorough we want to be about testing the JavaScript on different platforms and browsers.

The tests I saw before looked like they were using mocked objects, I'm not sure what automated tests there are but I'll look more.

[littlealex003]

It looks like js-libp2p is using https://ipfs.github.io/aegir/ I'm not familiar with it; it has a "browser" test mode, though it's not entirely clear what that is doing. We could implement this same library

[D4nte]

Hum, I am enjoy the integration of Webstorm (my IDE) with mocha/chai. Not really keen to move to another test framework that does not integrate well (I moved away from ava for this reason exactly).

Isn't selenium about testing UI component and simulating user actions? We don't really do that as js-waku does not have an UI element.

[littlealex003]

Yeah Mocha/Chai is pretty much the standard I find, so good to stay with that, I was just looking at what the other project was doing for "browser" testing based on their package.json and it seems that this tool provides to solution. What exactly it is doing, and what exactly "browser" testing means in that environment or in general seems unclear. To me browser testing is deploying a full end to end application with a Web UI component and testing that UI in the browser, as that is how end users will end up using it, so it may be a matter of semantics; I'm not exactly sure how anything else is actually testing the browser though. We could certainly deploy our code and load the example App in a browser and make sure it works as a way of testing to make sure the JavaScript works in actual browsers on actual OSes, that is what I thought you had in mind; I'm more used to working with independent CI/CD systems like Jenkins or others than what is integrated into GitHub but I think we could use Docker to build a real deployment scenario and they offer both Windows and Mac OS resources as well (though I'm not sure how the monetization model looks). I will look through more of the tests, the other tasks were a bit more clear so I was holding off on this until I got more info from you.

…

On Thu, Apr 29, 2021 at 3:07 AM Franck Royer ***@***.***> wrote: Hum, I am enjoy the integration of Webstorm (my IDE) with mocha/chai. Not really keen to move to another test framework that does not integrate well (I moved away from ava for this reason exactly). Isn't selenium about testing UI component and simulating user actions? We don't really do that as js-waku does not have an UI element. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AP6HJBFPLSEUN4KVXWD5DXDTLCWOBANCNFSM43L2SGJA> .

[weboko]

Weekly Update

• achieved: created front-end app to be run in a pipeline
• next: complete app and run in the pipeline, figure out next steps to run Firefox

[weboko]

Weekly update:

• achieved: finishing testing against chrome and react;
• next: investigate other adding browsers;

[weboko]

Weekly update:

• achieved: browser testing is redone and opening for review
• next: integrate with release process - rather quick follow up, revisit current epic

[weboko]

Weekly update:

• achieved: got reviews on playwrights tests
• next: maybe add bounty, check Karma testing

[weboko]

Weekly update:

• achieved: improved karma testing, added testing in browser

[weboko]

Since #887 is not urgent delaying the ticket.

[fryorcraken]

@weboko I think we can close thsi issue as we now test in Firefox and webkit. #887 can be a ticket on its own.

[weboko]

@fryorcraken agree, closing