Auto-retry option to weed out volatile/flaky tests

[calebdw]

Hello!

The ability to automatically retry a failing test a certain number of times would be a very useful feature! Occasionally, some tests can be rather flaky and debugging the flakyness can be a time consuming process. When this happens, developers will usually restart the CI job to see if the test re-fails but this a lengthy process re-running thousands of tests when there's only one test that really needs to be retried.

Playwright has something similar with its --retries=3 cli and config options.

Thanks!