RSpec::Retry

RSpec::Retry adds a :retry option for intermittently failing rspec examples. If an example has the :retry option, rspec will retry the example the specified number of times until the example succeeds.

Compatibility

Rspec Version	Rspec-Retry Version
> 3.8	0.6.2 but untested
> 3.3, <= 3.8	0.6.2
3.2	0.4.6
2.14.8	0.4.4

Installation

Add this line to your application's Gemfile:

gem 'rspec-retry', group: :test # Unlike rspec, this doesn't need to be included in development group

And then execute:

$ bundle

Or install it yourself as:

$ gem install rspec-retry

require in spec_helper.rb

# spec/spec_helper.rb
require 'rspec/retry'
require 'rspec/core/formatters/base_text_formatter'

RSpec.configure do |config|
  # (Optional) show retry status in spec process
  config.verbose_retry = true
  # (Optional) show exception that triggers a retry if verbose_retry is set to true
  config.display_try_failure_messages = true
  # (Optional) configure a reporter & formatter for tests which fail then pass in the alloted number of retries (a.k.a. flakey tests).
  reporter = RSpec::Core::Reporter.new(config)
  formatter = RSpec::Core::Formatters::BaseTextFormatter.new(File.open('tmp/rspec-retry-flakey-specs.log', 'ab')
  reporter.register_listener(formatter), 'message')
  config.retry_reporter = reporter

  # run retry 3 times on all specs
  config.around do |ex|
    ex.run_with_retry retry: 3
  end

  # callback to be run between retries
  config.retry_callback = proc do |ex|
    # run some additional clean up task - can be filtered by example metadata
    if ex.metadata[:js]
      Capybara.reset!
    end
  end
end

Usage

it 'should randomly succeed', :retry => 3 do
  expect(rand(2)).to eq(1)
end

it 'should succeed after a while', :retry => 3, :retry_wait => 10 do
  expect(command('service myservice status')).to eq('started')
end
# run spec (following log is shown if verbose_retry options is true)
# RSpec::Retry: 2nd try ./spec/lib/random_spec.rb:49
# RSpec::Retry: 3rd try ./spec/lib/random_spec.rb:49

Calling run_with_retry programmatically

You can call ex.run_with_retry(opts) on an individual example.

Configuration

• :verbose_retry(default: false) Print retry status
• :display_try_failure_messages (default: false) If verbose retry is enabled, print what reason forced the retry
• :default_retry_count(default: 1) If retry count is not set in an example, this value is used by default. Note: If this is changed from the default of 0, all examples will be retried.
• :default_sleep_interval(default: 0) Seconds to wait between retries
• :clear_lets_on_failure(default: true) Clear memoized values for lets before retrying
• :exceptions_to_hard_fail(default: []) List of exceptions that will trigger an immediate test failure without retry. Takes precedence over :exceptions_to_retry
• :exceptions_to_retry(default: []) List of exceptions that will trigger a retry (when empty, all exceptions will)
• :retry_callback(default: nil) Callback function to be called between retries
• :retry_reporter(default: nil) Reporter which logs comma delimited data for indeterminate specs in the format: [attempts, max_retries, path_and_line_number_of_failed_spec, error_messages]

Environment Variables

• RSPEC_RETRY_RETRY_COUNT can override the retry counts even if a retry count is set in an example or default_retry_count is set in a configuration.

Contributing

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Added some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a pull request