Ruby Feature Request: rspec-core

rspec-core is a gem that provides the structure for writing executable examples of how your code should behave, and an rspec command with tools to constrain which examples get run and tailor the output.

Your goal is to implement the following feature request:

It would be nice to be able to configure the number of failures before rspec exits out, so something like --fail-fast 5, so that I can get 5 failing tests at a time instead of having to choose between either one or all of them.

The documentation below is for the rspec-core gem itself.

Happy coding!

Install

gem install rspec      # for rspec-core, rspec-expectations, rspec-mocks
gem install rspec-core # for rspec-core only
rspec --help

Want to run against the master branch? You'll need to include the dependent RSpec repos as well. Add the following to your Gemfile:

%w[rspec rspec-core rspec-expectations rspec-mocks rspec-support].each do |lib|
  gem lib, :git => "git://github.com/rspec/#{lib}.git", :branch => 'master'
end

Basic Structure

RSpec uses the words "describe" and "it" so we can express concepts like a conversation:

"Describe an order."
"It sums the prices of its line items."

RSpec.describe Order do
  it "sums the prices of its line items" do
    order = Order.new

    order.add_entry(LineItem.new(:item => Item.new(
      :price => Money.new(1.11, :USD)
    )))
    order.add_entry(LineItem.new(:item => Item.new(
      :price => Money.new(2.22, :USD),
      :quantity => 2
    )))

    expect(order.total).to eq(Money.new(5.55, :USD))
  end
end

The describe method creates an ExampleGroup. Within the block passed to describe you can declare examples using the it method.

Under the hood, an example group is a class in which the block passed to describe is evaluated. The blocks passed to it are evaluated in the context of an instance of that class.

Nested Groups

You can also declare nested nested groups using the describe or context methods:

RSpec.describe Order do
  context "with no items" do
    it "behaves one way" do
      # ...
    end
  end

  context "with one item" do
    it "behaves another way" do
      # ...
    end
  end
end

Nested groups are subclasses of the outer example group class, providing the inheritance semantics you'd want for free.

Aliases

You can declare example groups using either describe or context. For a top level example group, describe and context are available off of RSpec. For backwards compatibility, they are also available off of the main object and Module unless you disable monkey patching.

You can declare examples within a group using any of it, specify, or example.

Shared Examples and Contexts

Declare a shared example group using shared_examples, and then include it in any group using include_examples.

RSpec.shared_examples "collections" do |collection_class|
  it "is empty when first created" do
    expect(collection_class.new).to be_empty
  end
end

RSpec.describe Array do
  include_examples "collections", Array
end

RSpec.describe Hash do
  include_examples "collections", Hash
end

Nearly anything that can be declared within an example group can be declared within a shared example group. This includes before, after, and around hooks, let declarations, and nested groups/contexts.

You can also use the names shared_context and include_context. These are pretty much the same as shared_examples and include_examples, providing more accurate naming when you share hooks, let declarations, helper methods, etc, but no examples.

Metadata

rspec-core stores a metadata hash with every example and group, which contains their descriptions, the locations at which they were declared, etc, etc. This hash powers many of rspec-core's features, including output formatters (which access descriptions and locations), and filtering before and after hooks.

Although you probably won't ever need this unless you are writing an extension, you can access it from an example like this:

it "does something" do |example|
  expect(example.metadata[:description]).to eq("does something")
end

described_class

When a class is passed to describe, you can access it from an example using the described_class method, which is a wrapper for example.metadata[:described_class].

RSpec.describe Widget do
  example do
    expect(described_class).to equal(Widget)
  end
end

This is useful in extensions or shared example groups in which the specific class is unknown. Taking the collections shared example group from above, we can clean it up a bit using described_class:

RSpec.shared_examples "collections" do
  it "is empty when first created" do
    expect(described_class.new).to be_empty
  end
end

RSpec.describe Array do
  include_examples "collections"
end

RSpec.describe Hash do
  include_examples "collections"
end

A Word on Scope

RSpec has two scopes:

• Example Group: Example groups are defined by a describe or context block, which is eagerly evaluated when the spec file is loaded. The block is evaluated in the context of a subclass of RSpec::Core::ExampleGroup, or a subclass of the parent example group when you're nesting them.
• Example: Examples -- typically defined by an it block -- and any other blocks with per-example semantics -- such as a before(:example) hook -- are evaluated in the context of an instance of the example group class to which the example belongs. Examples are not executed when the spec file is loaded; instead, RSpec waits to run any examples until all spec files have been loaded, at which point it can apply filtering, randomization, etc.

To make this more concrete, consider this code snippet:

RSpec.describe "Using an array as a stack" do
  def build_stack
    []
  end

  before(:example) do
    @stack = build_stack
  end

  it 'is initially empty' do
    expect(@stack).to be_empty
  end

  context "after an item has been pushed" do
    before(:example) do
      @stack.push :item
    end

    it 'allows the pushed item to be popped' do
      expect(@stack.pop).to eq(:item)
    end
  end
end

Under the covers, this is (roughly) equivalent to:

class UsingAnArrayAsAStack < RSpec::Core::ExampleGroup
  def build_stack
    []
  end

  def before_example_1
    @stack = build_stack
  end

  def it_is_initially_empty
    expect(@stack).to be_empty
  end

  class AfterAnItemHasBeenPushed < self
    def before_example_2
      @stack.push :item
    end

    def it_allows_the_pushed_item_to_be_popped
      expect(@stack.pop).to eq(:item)
    end
  end
end

To run these examples, RSpec would (roughly) do the following:

example_1 = UsingAnArrayAsAStack.new
example_1.before_example_1
example_1.it_is_initially_empty

example_2 = UsingAnArrayAsAStack::AfterAnItemHasBeenPushed.new
example_2.before_example_1
example_2.before_example_2
example_2.it_allows_the_pushed_item_to_be_popped

The rspec Command

When you install the rspec-core gem, it installs the rspec executable, which you'll use to run rspec. The rspec command comes with many useful options. Run rspec --help to see the complete list.

Store Command Line Options .rspec

You can store command line options in a .rspec file in the project's root directory, and the rspec command will read them as though you typed them on the command line.

Get Started

Start with a simple example of behavior you expect from your system. Do this before you write any implementation code:

# in spec/calculator_spec.rb
RSpec.describe Calculator do
  describe '#add' do
    it 'returns the sum of its arguments' do
      expect(Calculator.new.add(1, 2)).to eq(3)
    end
  end
end

Run this with the rspec command, and watch it fail:

$ rspec spec/calculator_spec.rb
./spec/calculator_spec.rb:1: uninitialized constant Calculator

Address the failure by defining a skeleton of the Calculator class:

# in lib/calculator.rb
class Calculator
  def add(a, b)
  end
end

Be sure to require the implementation file in the spec:

# in spec/calculator_spec.rb
# - RSpec adds ./lib to the $LOAD_PATH
require "calculator"

Now run the spec again, and watch the expectation fail:

$ rspec spec/calculator_spec.rb
F

Failures:

  1) Calculator#add returns the sum of its arguments
     Failure/Error: expect(Calculator.new.add(1, 2)).to eq(3)

       expected: 3
            got: nil

       (compared using ==)
     # ./spec/calcalator_spec.rb:6:in `block (3 levels) in <top (required)>'

Finished in 0.00131 seconds (files took 0.10968 seconds to load)
1 example, 1 failure

Failed examples:

rspec ./spec/calcalator_spec.rb:5 # Calculator#add returns the sum of its arguments

Implement the simplest solution, by changing the definition of Calculator#add to:

def add(a, b)
  a + b
end

Now run the spec again, and watch it pass:

$ rspec spec/calculator_spec.rb
.

Finished in 0.000315 seconds
1 example, 0 failures

Use the documentation formatter to see the resulting spec:

$ rspec spec/calculator_spec.rb --format doc
Calculator
  #add
    returns the sum of its arguments

Finished in 0.000379 seconds
1 example, 0 failures

Also see

• http://github.com/rspec/rspec
• http://github.com/rspec/rspec-expectations
• http://github.com/rspec/rspec-mocks