July 20, 2010 20:37
Posted by Jeremy Durham
Webrat or Capybara? With thousands of steps, it’s not a question you ask lightly. We’ve been using Webrat for a long time, but it definitely has it’s issues. It’s dependence on selenium RC to run in Selenium mode being one of them. How many times have you copied the selenium-server.jar into Webrat so that you can run your tests on Firefox 3.6? Also, it’s ability to see and click hidden elements, a lack of case sensitivity (until recently), and other annoying little things. Don’t even get me started about having to run Selenium and non-standard separately.
A few months back, we started pondering the idea of using Capybara instead of Webrat. We even decided to start a new project with Capybara instead of Webrat; that way, if we had to convert it to Webrat because of problems, it would be less work. As luck would have it, at the time I was testing parallel_tests with Webrat and doing some hacking to see if I could get the two to play nicely. Unfortunately, I wasn’t successful. After a few weeks of successfully using Capybara on the other project and my failed attempt to get parallel_tests to work with Webrat in selenium mode, I decided that something needed to be done.
I started the tedious process of converting all of the steps in one of my projects to Capybara, and removing custom steps by adding pickle. I had used pickle in a few pet projects, and found that for creating test data and limiting the amounts of custom paths, it standardized and greatly simplified the process. It took a few weeks of slowly changing in my downtime, rebasing against master, resolving merge conflicts, and converting some more.
I also decided that resurrecting parallel_tests needed to be done; we were spending a significant amount of time trying to optimize one of our projects build times, which had ballooned to around 45 minutes. Through some clever rework by some of my co-workers, they managed to get the build down to the mid 30 minutes.
After creating a small sandbox, I was able to get Capybara running on parallel_tests with very little effort. Although the results were not astronomical, I was able to save approximately 30-35% on my dual core laptop. With our build machine being quad core, I could only imagine the time savings.
The end result of running the old and new features on the build machine:
- Non-parallel (build machine): 11m27.796s
- Dual core parallel_tests: 8m28.36s
- Quad core parallel_tests (build machine): 4m8.45s
So, I succeeded in shaving off 3 minutes locally by running parallel_tests, and 7 minutes off my build by hacking up Capybara, and here’s the hack:
@port = 9887 + ENV['TEST_ENV_NUMBER'].to_i
@port += 1 while is_port_open?(@port) and not is_running_on_port?(@port)
Drop this into a file in features/support, setup parallel_tests as specified and enjoy faster Selenium features. You may also want to create a rake task to run your tests in parallel:
namespace :ci do
task :parallel_all do
RAILS_ENV = 'test'
We’re slowly working on converting the project that’s build takes over 30 minutes to Capybara, and will report our results when we’re completed. We’re shooting for under 10 minutes.