Build Test Runner

[ErikSchierboom]

This issue is part of the migration to v3. You can read full details about the various changes here.

In Exercism v3, one of the biggest changes is that we'll automatically check if a submitted solution passes all the tests.

We'll check this via a new, track-specific tool: the Test Runner. Each test runner is track-specific. When a new solution is submitted, we run the track's test runner, which outputs a JSON file that describes the test results.

The test runner must be able to run the tests suites of both Concept Exercises and Practice Exercises. Depending on the test runner implementation, this could mean having to update the Practice Exercises to the format expected by the test runner.

Goal

Build a test runner for your track according to the spec.

If you are building a test runner from scratch, we have a starting guide and a generic test runner that can be used as the base for the new test runner.

If a test runner has already been built for this track, please check if it works on both Concept Exercises and Practice Exercises.

It can be very useful to check how other tracks have implemented their test runner.

Tracking

exercism/v3-launch#4

[MeerKatDev]

I'll take this

[ErikSchierboom]

@MeerKatDev There is an existing repository for that: https://github.com/exercism/scala-test-runner

[jcranky]

What is the state of this? Judging from the (lack of) activity on https://github.com/exercism/scala-test-runner I imagine this is not progressing at the moment?

[ErikSchierboom]

That is correct. The problem that we're having is that the Scala test runner is functional, but too slow at the moment to finish within our time constraints for test runners. If you have any idea how to speed up the test runner, help would be much appreciated!

[jcranky]

Ah, I see. I'll have a look, just our of curiosity. No promises 😅

[jcranky]

I saw the exercism/scala-test-runner#13 issue - I'll follow up there 😄

[LarsWestergren]

Since exercism/scala-test-runner#13 is closed thanks to the hard work of @jcranky this could be closed as well right?