module Flores::RSpec::Analyze

RSpec helpers for stress testing examples

Setting it up in rspec:

RSpec.configure do |c|
  c.extend RSpec::StressIt
end

TODO(sissel): Show an example of stress_it and analyze_it

Public Instance Methods

analyze_results() click to toggle source

Save state after each example so it can be used in analysis after specs are completed.

If you use this, you'll want to set your RSpec formatter to Flores::RSpec::Formatter::Analyze

Let's show an example that fails sometimes.

describe "Addition of two numbers" do
  context "positive numbers" do
    analyze_results
    let(:a) { Flores::Random.number(1..1000) }

    # Here we make negative numbers possible to cause failure in our test.
    let(:b) { Flores::Random.number(-200..1000) }
    subject { a + b }

    stress_it "should be positive" do
      expect(subject).to(be > 0)
    end
  end
end

And running it:

% rspec -f Flores::RSpec::Formatter::Analyze
Addition of two numbers positive numbers should be positive
  98.20% tests successful of 3675 tests
  Failure analysis:
    1.80% -> [66] RSpec::Expectations::ExpectationNotMetError
      Sample exception for {:a=>126.21705882478048, :b=>-139.54814492675024, :subject=>-13.33108610196976}
        expected: > 0
             got:   -13.33108610196976
      Samples causing RSpec::Expectations::ExpectationNotMetError:
        {:a=>90.67298249206425, :b=>-136.6237821353908, :subject=>-45.95079964332655}
        {:a=>20.35865155878871, :b=>-39.592417377658876, :subject=>-19.233765818870165}
        {:a=>158.07905166101787, :b=>-177.5864470909581, :subject=>-19.50739542994023}
        {:a=>31.80445518715138, :b=>-188.51942190504894, :subject=>-156.71496671789757}
        {:a=>116.1479954937354, :b=>-146.18477887927958, :subject=>-30.036783385544183}
# File lib/flores/rspec/analyze.rb, line 67
def analyze_results
  # TODO(sissel): Would be lovely to figure out how to inject an 'after' for
  # all examples if we are using the Analyze formatter.
  # Then this method could be implied by using the right formatter, or something.
  after do |example|
    example.metadata[:values] = __memoized.clone
  end
end