Add jest-runner

[bmealhouse]

PR to support: microsoft/vscode-docs#3390

[DonJayamanne]

Looking forward to this,
I managed to get it working for VSCode, however not without a hack https://github.com/DonJayamanne/gitHistoryVSCode/blob/master/test/extension/testRunner.ts

[DonJayamanne]

@bmealhouse
Does this mock VSCode API or use the real thing?

[bmealhouse]

@bmealhouse
Does this mock VSCode API or use the real thing?

By default, this does not mock on the VS Code API. However, it is possible to mock individual VS Code APIs as needed. This will be documented in the jest-runner README after this PR merges.

[bmealhouse]

@octref - I rebased this PR after everything was passing in CI. However, the final build failed. It doesn't seem to be related to failing tests. Is this something I need to address?

[octref]

Retry the build

[jasonwilliams]

@octref @bmealhouse what’s preventing this being merged?

[octref]

I've left Microsoft. /cc @eamodio

[bmealhouse]

I've left Microsoft. /cc @eamodio

@eamodio - Is there anything else we need to do to get this merged?

[connor4312]

Hi, I am the lucky new owner of vscode-test 👋

I would ideally like vscode-test to be ambivalent towards different test runners; this is largely the case today with mocha. Can you clarify why we need changes in vscode-test for jest? It looks like we just reference files in the jest-runner folder, which could be included in a separate npm module just as easily.

If there are snags or features you dicovered that would be useful to make swapping out test runners easier, please let me know!

[bmealhouse]

I respect that @connor4312.

The main goal of all this was to document how to use jest for testing VS code extensions. I started doing this as a separate npm module (vscode-jest-test-runner) but was asked to move it to vscode-test. I believe the reasoning behind this was to avoid referencing a non-Microsoft owned npm module in VS code documentation.

NVM. I just realized that you published your own test-runner, and it makes sense to have documentation there.
Would you be interested in merging the code to vscode-test repo?

Originally posted by @octref in microsoft/vscode-extension-samples#251 (comment)

I'm happy to head in whatever direction you think is best @connor4312. Please let me know your thoughts.

Maybe this is something I should just throw on my own blog and we avoid adding to VS code docs?

[jasonwilliams]

To add to what @bmealhouse was saying, I ended up here because I wanted to use jest (my projects were already using jest and I didn’t want to mix test frameworks), and there was no documentation about how. If we’re being honest this will only become more of a thing now that jest is becoming the de-facto for JavaScript testing over mocha https://npmcharts.com/compare/Jest,jest,mocha

I understand the wish to remain ambivalent but pragmatically it would make sense to have some official documentation on how to do this with tooling people are using today, as this is what many of us are looking for. Having this in a wiki (GitHub) format also helps us keep it up to date (compared to a blog post that will be outdated in a year).

VSCode already has official docs on how to get set up with Webpack, so I don’t see this being much different.

in summary I understand staying neutral, but I think not including documentation/examples for certain frameworks is more harmful in the long run.

[connor4312]

Thanks for the feedback! I pushed an update to your branch (which github isn't or is being slow to show for some reason).

The commit does a couple things:

• Moves sources into an "src" folder.

• I removed most of the default configuration. This is something that
  the user should choose to use. For instance, ts-jest uses
  transpileModule, which (like many other loaders and all of those
  based on babel) lacks some features like const enum support and
  therefore might not be appropriate for all users.

  Instead we can suggest ts-jest and include it in Jest config files
  in our samples.

• Adds a code-jest cli that is effectively a jest cli and passes
  its arguments through. This means that you don't need a runTests.ts
  and can run tests more like you would any 'normal' TS/JS project.

• Jest is quite large so I have made it an optionalDependency. The
  CLI with exit with a helpful error if jest is not available.

Not here yet, I want to make code-jest work with debugging and launch
configs, so you could do something like this -- again, like you would
mocha or any other test runner, just with type: extensionHost:

{
	"name": "Run Extension Tests",
	"type": "extensionHost",
	"request": "launch",
	"runtimeExecutable": "${workspaceFolder}/node_modules/.bin/code-jest",
	"args": ["--some-jest-args=here"],
}

I think I can make this work in a generic way.

[aeschli]

vscode-test provides helpers that can be used with any test runner of any flavor and version and currently has no dependency on any testing framework (also not mocha).
The PR would bring in jest (@jest/core) as a dependency to vscode-test. @jest/core will become a dev dependency to all extensions created by yo code, even the extension doesn't need it.
Keeping a separate node module https://github.com/bmealhouse/vscode-jest-test-runner is the right way to go.
I see no problem for us to mention the node module in the VSCode documentation.

[bmealhouse]

@conner4312 - Looks great. I like the API changes!

It looks like the colorizing stuff stuck around. Is there any way to get rid of this? It was a giant pain to set up and would be awesome if the colors came through automatically.

@aeschli - If there is a concern introducing jest to this repository, could the jest stuff live in a separate Microsoft-owned npm module?

[aeschli]

could the jest stuff live in a separate Microsoft-owned npm module?

That would mean we own it and have to maintain it. Not sure if @connor4312 want's to do that.
Maintaining is quite some work as it means keeping up-to-date with the latest developments in Jest but we don't use Jest on a daily basis.

For me what's important is that the vscode-test npm module stays lean and simple and its APIs are useful for any type of test framework.

[bmealhouse]

Makes sense, @aeschli. I'll leave it to you and @connor4312 to figure out the direction of this PR.

If you all decide to keep this functionality in my npm module (vscode-jest-test-runner), @connor4312, would you be able to open a PR with your suggested changes? I could also implement these changes myself, but I don't want to steal the credit for your amazing work!

[ssbarnea]

Can someone from Microsoft give some insights on this PR? Not a single update in more than year and AFAIK, jest is the most popular js based test framework, twice as popular as mocha.

[connor4312]

Hey there. After discussing this internally a bit last year after my initial comment above, I don't think we want to take this change yet -- and as commenters have shared above, there are ways to run with Jest without requiring integration with this module.

However, at some point in the future, we will integrate extension tests with the VS Code testing API. This is likely to result in having a 'blessed' testing framework which we hook into or provide in some way. We may reevaluate adopting Jest at that point.

[ssbarnea]

@connor4312 Thanks for taking time to update on this, it really helps other extension developers.

I was a little bit disappointed about the lack of active community on mochajs, combined with missing features from it (xskip, retries,...). Due to this I was looking for better alternative and jest came as being far more active and ~2x as popular.

Still, when I dug a little bit more I discovered that the request to add an API to it in 2017 never got resolved. So, in less than a day of investigation I discovered that mochajs was at least from that point of view better.

I a really looking forward to hear about the testing framework plans as I want to be able to run a single test from vscode UI, debug code and things like this.