module Flores::RSpec::Analyze
RSpec helpers for stress testing examples
Setting it up in rspec:
RSpec.configure do |c| c.extend RSpec::StressIt end
TODO(sissel): Show an example of stress_it and analyze_it
Public Instance Methods
analyze_results()
click to toggle source
Save state after each example so it can be used in analysis after specs are completed.
If you use this, you'll want to set your RSpec formatter to Flores::RSpec::Formatter::Analyze
Let's show an example that fails sometimes.
describe "Addition of two numbers" do context "positive numbers" do analyze_results let(:a) { Flores::Random.number(1..1000) } # Here we make negative numbers possible to cause failure in our test. let(:b) { Flores::Random.number(-200..1000) } subject { a + b } stress_it "should be positive" do expect(subject).to(be > 0) end end end
And running it:
% rspec -f Flores::RSpec::Formatter::Analyze Addition of two numbers positive numbers should be positive 98.20% tests successful of 3675 tests Failure analysis: 1.80% -> [66] RSpec::Expectations::ExpectationNotMetError Sample exception for {:a=>126.21705882478048, :b=>-139.54814492675024, :subject=>-13.33108610196976} expected: > 0 got: -13.33108610196976 Samples causing RSpec::Expectations::ExpectationNotMetError: {:a=>90.67298249206425, :b=>-136.6237821353908, :subject=>-45.95079964332655} {:a=>20.35865155878871, :b=>-39.592417377658876, :subject=>-19.233765818870165} {:a=>158.07905166101787, :b=>-177.5864470909581, :subject=>-19.50739542994023} {:a=>31.80445518715138, :b=>-188.51942190504894, :subject=>-156.71496671789757} {:a=>116.1479954937354, :b=>-146.18477887927958, :subject=>-30.036783385544183}
# File lib/flores/rspec/analyze.rb, line 67 def analyze_results # TODO(sissel): Would be lovely to figure out how to inject an 'after' for # all examples if we are using the Analyze formatter. # Then this method could be implied by using the right formatter, or something. after do |example| example.metadata[:values] = __memoized.clone end end