Test Runners for Launch

[iHiD]

Please add a comment to this issue when you have a test-runner ready to launch, and I'll add it to the top of this, and @ccare can work through getting them deployed. I'll also add the handle of the person who adds the comment, as @ccare's point of contact for any issues :)

• Ruby (@iHiD)
• Bash (@bkhl)
• Go (@tehsphinx)
• Python (@yawpitch and/or @cmccandless)
• x86-64 Assembly (@bergjohan)
• Rust (@coriolinus)
• C# (@ErikSchierboom)
• F# (@ErikSchierboom)
• C (@bergjohan)
• Elm (@ceddlyburge)

[bkhl]

I think this one is ready to go: https://github.com/exercism/bash-test-runner

[iHiD]

@bkhl Thanks. This is probably the one we're most wary of (in terms of people running other stuff) so will be good for you and @ccare to chat this one over on Slack at some point! :)

[tehsphinx]

Go Test Runner is ready.

[cmccandless]

Python test runner is ready.

*Edit: https://github.com/exercism/python-test-runner

[bergjohan]

x86-64 assembly test runner is ready.

[coriolinus]

rust test runner is ready

[ErikSchierboom]

The C# test runner and F# test runner are ready to go.

[bergjohan]

The C test runner is ready. C is not participating in experiment-1, so this is less prioritized.

[ceddlyburge]

I believe the Elm Test Runner is ready. Please let me know if you have any comments.

[iHiD]

Hey everyone. Just a quick update. @ccare is working away trying to get these deployed. We've hit lots of unexpected problems with deploying the C# one and are trying to get that one deployed fully before setting up everyone else's, so that we can give you a more reliable set of feedback if it doesn't work, rather than you all having to shave the same yak with us.

There have been a few tweaks to the spec over the last few weeks which you might have missed. One of them is the addition of user output and another is ensuring that your test-runner returns with an exit code of 0.

We will also be requiring exercises to have their own config.json file to work with test-runners. You can see info about this in the Experiment README. If you're hoping to test out your test runner in the experiment, please check that out too.

If you'd like the Ruby one in action, you can check out https://research.exerism.io (the basic-auth username/password are in the #experiment-1 channel on Slack.

Finally, thanks you all for your patience!

[ccare]

We've had a good couple of days deploying the containers. There have been a few things to fix on the way, but they're all pretty minor things - getting environment variables correct has been coming up a lot.

I've been able to get decent test results end-to-end using the exercism research site. The following runners appear to be working nicely end-to-end.

• C#
• Go
• Python
• Ruby
• x86 Assembly

In addition, we're also really close with

• JavaScript
• Elm
• Rust

The next step will be to apply some extra ops work to cleanup old solutions and keep our disks empty, we also have some work to do to make scaling up and down a bit more slick, but otherwise, we're really close. Thanks everyone for your patience and effort! It's all looking great!

[iHiD]

Fantastic. Thanks @ccare. Great work :)

[kytrinyx]

This is awesome! So much progress!

[TheLostLambda]

The Common Lisp Test Runner is up and running!