Nick Gauthier was recently sharing how he was running his Jasmine test using Capybara with the Capybara Webkit headless driver. You can check out his capybara-jasmine project on GitHub.

In my recent blog post on TDD I mentioned that I had just put together a Javascript test suite using Jasmine. In my case, I’ve used Phantom.js for headless fast tests as well as Selenium Webdriver for real browser testing (I usually use headless during development to get quick feedback and use the real browsers as a safety check before commit). Nick Gauthier’s project inspired me to share my setup.

Test suite

For this blog post, I can not share the code I already built as it belongs to my employer. Instead, I will start from scratch using the Jasmine 1.2.0 standalone release. It includes a sample test suite which will be a perfect fit for this example. It includes the main test runner (an HTML page called SpecRunner.html) and the Jasmine distribution (under the lib/ directory). It also includes example code under spec and src. If you unzip the file and load SpecRunner.html in your browser, you should see the test suite pass without errors.

Phantom.js

To run the test suite using Phantom.js, I’ve used Joshua Carver’s Jasmine runner for Phantom.js. I copied the lib directory from the project under lib/phantom-jasmine in my source tree.

Next I changed the SpecRunner.html to add an additional reporter called ConsoleReporter - which is part of Joshua’s project.

1

<script type="text/javascript" src="lib/phantom-jasmine/console-runner.js"></script>

1
2

window.consoleReporter = new jasmine.ConsoleReporter();
jasmineEnv.addReporter(consoleReporter);

Third step: patch Jasmine to get a stack trace on test failure. The current version of Jasmine (1.2.0) does not show stack traces when executed throught Phantom.js. Thanks to a tip found on Stackoverflow I was able to fix that with a very simple patch and submitted a pull request.

1
2
3
4
5
6
7
8
9
10

@@ -106,7 +106,9 @@ jasmine.ExpectationResult = function(params) {
   this.actual = params.actual;
   this.message = this.passed_ ? 'Passed.' : params.message;

-  var trace = (params.trace || new Error(this.message));
+  var err;
+  try { throw new Error(this.message); } catch(e) { err = e };
+  var trace = (params.trace || err);
   this.trace = this.passed_ ? '' : trace;
 };

Assuming that Phantom.js is installed and available in the $PATH, the test suite can be executed using the following command:

1

phantomjs lib/phantom-jasmine/run_jasmine_test.coffee SpecRunner.html

Finally, I wrapped the above command in a Rake task so that I can just type rake phantomjs to run the test suite:

1
2
3
4
5

desc "Run Jasmine tests with Phantomjs"
task :phantomjs do
  # Note: we are assuming phantomjs is installed and available in $PATH
  sh %{phantomjs lib/phantom-jasmine/run_jasmine_test.coffee SpecRunner.html}
end

This is what the output looks like:

Webdriver

In order to execute the Jasmine test suite in real browsers, I am using Selenium Webdriver. I am using the selenium-webdriver gem to interact with it. In just a minute, I will show how I have added an additional task to my Rakefile to run the tests.

Also note that a webserver is required to serve the SpecRunner.html file and its dependencies. One could setup a webserver such as Apache but I think it is too much work and creates an external unnecessary dependency for the test runner (every environment where the tests are executed would need to configure an instance of Apache). Instead, I prefer to use the service_manager gem to spin up a simple web server on demand using the python -m SimpleHTTPServer command. With Service Manager, you define services that will be automatically started when the tests begin and stopped at the end. Because Python is widely available, no extra dependencies should be required. For more information, see service_manager documentation on GitHub.

The Rakefile in the end is fairly straightforward:

• Require bundler and some gems
• Start ServiceManager which will spawn the webserver process
• Detect the browser passed as a command line argument
• Instantiate Webdriver and load the SpecRunner.html file
• Capture the output of the ConsoleReporter and display it
• Finally, check the status for success or failure

The complete Rakefile looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

require 'rubygems'
require 'bundler/setup'

require 'selenium-webdriver'
require 'service_manager'


desc 'Run Jasmine tests with Phantomjs'
task :phantomjs do
  # Note: we are assuming phantomjs is installed and available in $PATH
  sh %{phantomjs lib/phantom-jasmine/run_jasmine_test.coffee SpecRunner.html}
end


desc 'Run Jasmine tests with Webdriver'
task :webdriver do
  # Use service_manager to start a simple web server
  ServiceManager.start

  # Detect which browser we want to run
  case ENV['browser']
  when 'chromium'
    # TODO: detect chromium path? I use Linux, this is the path on my machine
    Selenium::WebDriver::Chrome.path = '/usr/lib/chromium-browser/chromium-browser'
    driver = Selenium::WebDriver.for :chrome
  when 'firefox'
    driver = Selenium::WebDriver.for :firefox
  else
    raise ArgumentError, 'Unknown browser requested'
  end

  # Finally, we load the spec runner HTML page with WebDriver
  driver.navigate.to 'http://localhost:8000/SpecRunner.html'
  status = driver.execute_script('return consoleReporter.status;')
  output = driver.execute_script('return consoleReporter.getLogsAsString();')
  driver.quit

  print output

  # Make sure to exit with code > 0 if there is a test failure
  raise RuntimeError, 'Failure' unless status === 'success'

end

To run the webdriver task, just execute rake webdriver browser=firefox. You should see Firefox start and go through the tests. This is what the output looks like:

A remaining step is that I will need to add support for browsers other than Firefox or Chromium, as well as support for remote Webdriver instances (such as Internet Explorer running in a virtual machine).

Breaking stuff

Before wrapping up, I’d like to show a failure example. Let’s break some code by commenting out line 17 of src/Player.js. Running rake phantomjs fails as expected. Notice how I can see the full stack trace which points me to line 33 of spec/PlayerSpec.js.

Wrap up

Phantom.js, Webdriver and Jasmine are a really powerful combo to test drive code. Not only that, they are also really easy to set up. Kudos to the various Open Source contributors behind these projects for providing a great developer experience!

The next step of this adventure is continuous integration. I need to integrate the tests with our Jenkins instance. Hopefully, that should be trivial since I have my tests runner command ready to go.

If you want to see the code, it’s available on GitHub.

What do you think? Do you find this technique useful? What other tips do you use for running your Javascript tests? If you have feedback on this post, feel free to comment below or to contact me via Twitter (@jphpsf).