Test driven development with JavaScript, part two

It's been a while since part one, but I guess late is better than never. In this second part I'll walk you through some available testing frameworks for JavaScript.

Testing frameworks

If you're looking for info on unit testing in JavaScript you might find that search engines give you less than satisfactory results. Sure, you'll find something, but in general I find the amount of test related material out there disapointing in comparison to the amount of JavaScript being written.

Through the past couple of years I've been looking out for a good testing framework for JavaScript. There is no one holy grail, but several fully featured frameworks have popped up, and some documentation is available as well. I'd say the situation is rapidly improving, even though it's not nearly as good as for other technologies.

JsUnit

JsUnit is the framework I've personally had the most experience with. I've used it for Validatious, as well as other web projects. JsUnit is a port of JUnit, the defacto testing framework for Java.

JsUnit uses a test runner, which is basically a web page with a form allowing you to input an URL to your test case (or suite) and then run it. It's possible also to pass the test case URL as a parameter on the test runner URL, but you still have to start the runner manually.

JsUnit gives you a set of assertions which you may run against your code:

The example from part one of this article could be expressed with JsUnit as such:

function add(a, b) {
    return parseInt(a) + parseInt(b);
}

function testAdd() {
    assertEquals("adding 1 and 2 should yield 3", 3, add(1, 2));
    assertEquals("adding '2' and 2 should yield 4", 4, add("2", 2));
}

Test case.

JsUnit is a full featured solution, however not without its drawbacks

JsUnit also provides a test server (Java) which allows you to automatically run tests in all available browsers issuing a single command. Pretty cool!

A final feature worth mentioning is JsUnits test suites. A test suite is an HTML file including some JavaScript that lists URLs to test case HTML files. These can all be run in a single operation from the runner through the suite. As far as I know, JsUnit is the only framework provding this feature.

JsUnit

YUI Test

YUI Test is the framework coming out of Yahoo!, and - compared to JsUnit from 2001 - is fairly new. YUI Test offers the following assertions:

As you can see, YUI Test offers a toolset better suited for the language it's testing than what JsUnit does. YUI Test also provides user event simulation:

Like any other YUI feature, you access these through the YAHOO namespace, as such: YAHOO.util.UserAction.click(mittElement);

I have only two complaints when it comes to YUI Test, none of which are good enough to not try it out:

The example using YUI Test:

function add(a, b) {
    return parseInt(a) + parseInt(b);
}

var addTestCase = new YAHOO.tool.TestCase({
    testNumbersAndStrings: function() {
        YAHOO.util.Assert.areEqual(3, add(1, 2), "adding 1 and 2 should yield 3");
        YAHOO.util.Assert.areEqual(4, add("2", 2), "adding '2' and 2 should yield 4");
    }
});

YAHOO.util.Event.onDOMReady(function () {
    //create the logger
    var logger = new YAHOO.tool.TestLogger("testLogger");
    YAHOO.tool.TestRunner.add(addTestCase);

    //run the tests
    YAHOO.tool.TestRunner.run();
});

Run test.

As I mentioned, the logger is barely legible. The following is a minimal fix which fixes the readability slightly (who wants to read test results in 9px monospace fonts?):

.yui-log {
    font-size: 110%;
}

.yui-log .yui-log-bd {
    height: auto;
}

YUI Test

JsSpec

JsSpec provides a different flavour of testing. Those familiar with behaviour driven development, using for instance RSpec (for Ruby) will feel right at home. The syntax differs quite a bit from the assertions of JsUnit and YUI Test. An example is available from the JsSpec documentation. The MooTools team uses JsSpec.

Yehuda Katz, known from Merb (and now Rails 3) also has a JsSpec project which is not related to the previously mentioned framework. A quick peek at it gives a cleaner impression. If anyone has any experience with either of these two frameworks I'd love to hear back from you!

QUnit

QUnit is the unit testing framework developed and used by the jQuery team. I'd been looking for this for quite some time when I finally stumbled across it late 2008. It seemed to have been available all the way since march that year without catching my eyes. Oh well, I guess this might have something to do with it:

QUnit has only 4 assertions:

These'll get the job done, but it seems a bit scarse. The YUI arsenal of assertions potentially gives you alot more expressiveness in your tests, and makes alot of common JavaScript tasks easier to do. Compared to YUI Test I'd have to say that QUnit still has some ground to cover.

Through jQuery and it's event module, QUnit also has the ability to simulate user events. The trigger method can be called on jQuery objects with all kinds of events as the first parameter: $("a#minLenke").trigger("click", { mock: "objekt" });

Our running example using QUnit:

function add(a, b) {
    return parseInt(a) + parseInt(b);
}

// Test funksjonen
test("addisjon med tall og strenger", function() {
    equals(3, add(1, 2), "adding 1 and 2 should yield 3");
    equals(4, add("2", 2), "adding '2' and 2 should yield 4");
});

Run test case.

jQuery test suite (using QUnit) (lots of tests, will consume your browser for a little bit).

QUnit

jsunittest.js

jsunittest.js is the testing framework coming out of the Prototype.js/Scriptaculous community. Documentation is very scarse here. In fact, the only "official" documentation I could find was the slides from Thomas Fuchs' presentation on it.

The framework relies on prototypejs, meaning - given prototypejs' numerous contributions to the global scope - it's probably not well suited to test applications that don't rely on prototype.js.

Luckily, there is a version of jstestunit.js which doesn't rely on prototypejs, and works quite well (if you don't mind the lack of documentation). Dr Nic has created a command line tool for generating code stubs for JavaScript projects called newjs. This tool includes a modified version of jsunittest.js which doesn't rely on prototype. You can get the two files directly from me if you want to:

Assertions and helpers:

Using Ruby, Dr Nic also wrote a way to run tests from the shell. The application fires up browsers with test cases, checks the results and then closes down the browsers. Finally it reports the results in the original shell. newjs also supports auto testing through this mechanism, a feature none of the other frameworks have.

Dr Nics newjs is a good start and a good idea. However, I find it to be a little bit too opinionated and unstable. It provides (or, provided, I haven't checked recently) very limited ways of configuring directory layout and such, and was a bit hard to incorporate in an existing project.

Our example using jsunittest.js:

function add(a, b) {
    return parseInt(a) + parseInt(b);
}

new Test.Unit.Runner({
    testAddStringsAndNumbers: function() { with(this) {
        assertEqual(3, add(1, 2), "adding 1 and 2 should yield 3");
        assertEqual(4, add("2", 2), "adding '2' and 2 should yield 4");
    }}
}, { testLog: "enumerablelog" });

Run test case.

Crosscheck

The last framework I'll mention is Crosscheck. Seeing how the webpage touts Firefox 1.5 and IE6 support I haven't looked closely at this. If anyone have any experience or know anything about this do let me know! It seems that the strength of Crosscheck is that it can run tests without actually starting up the browsers. That would be sweet.

Challenges when testing JavaScript

The biggest challenge when unit testing JavaScript is depending on browsers to run tests. Having to run tests in 4, 5, 6 or maybe even more browsers is cumbersome. Both JsUnit and jsunittest.js offers automation for this, but they still require the browsers to run to do their job.

Hopefully, we'll be able to run tests on the command line some day. John Resigs env.js looks very promising in this regard, but until you can actually run the environment of all popular browsers this way you really can't be helped. You will have to open those browsers.

More on testing

Some more resources I stumbled across when researching this post:

What's your favourite JavaScript testing framework?

Published 23. January 2009 in javascript og test driven development.

Possibly related