Back to Blog

Get in Touch

Hunting Down Execution Order Test Failures

By Jerry Cheung January 11, 2012 in rails, ruby, tests, tip

Medium

Unit tests should pass when run in random order. But for an existing legacy project certain tests might depend on the execution order. One test might run perfectly fine by itself, but fail miserably when run after another test. Rather than running different combinations manually, RSpec 2.8 has the option to run specs in random order with the --order random flag. But even with this it can be hard to determine which specific test is causing the dependency. For example:

    rspec spec/controllers  # succeeds
    rspec spec/lib/my_lib_spec.rb  # succeeds
    rspec spec/controllers spec/lib/my_lib_spec.rb  # fails

In this scenario you know that one of the spec files in spec/controllers is not jiving with your lib spec, but if you have hundreds of spec files, it's hard to tell which. Never fear! There's a Ruby one-liner for that:

    ls spec/controllers/*.rb | ruby -pe '$_=`rspec #{$_} spec/lib/my_lib_spec.rb`'

Let's break this command down into its components:

    ls spec/controllers/*.rb

gives you a list of spec files to run alongside your lib spec

    ruby -pe

'e' for execute, and 'p' means wrap the code in a loop and assign each line of STDIN to $_. We're piping in STDIN from the ls command.

    $_=`rspec #{$_} spec/lib/my_lib_spec.rb

The 'p' flag also prints out the value of $_ at the end of each loop. So we assign the output of running rspec with the 2 files (one from ls alongside my_lib_spec).

My bash buddies would wag their fingers at me for using a ruby one-liner here, but it's a familiar syntax and it's easier for me than remembering other shell commands and regex flags. If there's something another unix program is better at processing, then I can then take the output of the ruby one-liner and pipe it into another command. It's a very simple and versatile way to munge on text.

Medium

Jerry Cheung

Jerry loves creating software. He started experimenting with Ruby on Rails in 2007 and got hooked on it after creating several personal projects. Upon graduating from Berkeley, he joined Coupa and later went on to start his own company Outspokes with several friends from Berkley. When he's not furiously typing, Jerry might be out running, brewing beer, or enjoying a BBQ and getting a serious sunburn.

More posts by Jerry Cheung

Jerry Cheung

Rails views are typically rendered after some controller action is executed. ...

Jerry Cheung

Twitter popularized the term "firehose API", to mean a realtime stream of dat...

Jerry Cheung

Rails ActiveRecord models have a lifecycle that developers are allowed to hoo...