Skip to content

cjohansen.no

Higer-level Stubbing and Mocking Tools in Sinon

Having covered the basic Sinon interface in one and a half posts I want to show you some of the higher-level tools Sinon brings to the table to reduce the amount of ceremony required to add stubs and mocks to your JavaScript tests. Hopefully, these tools will make it dead simple to properly integrate Sinon with any xUnit style test framework.

Collections

As previously discussed, stubbing and mocking shared/global objects requires some scaffolding to avoid ripple effects from stubs and mocks leaking between tests. Basically this means having to add a call to restore() on each stub in the tearDown method.

Additionally, if you set expectations on more than a single mock in one test you have a possibly unsolvable problem on your hands: If the first mock fails its verify() call, it will throw an exception, meaning that further mocks are never verified and thus not restored. This problem can be circumvented by restoring mocks in tearDown as well, but this quickly becomes bothersome.

A final problem is this: If not all tests stub and/or mock the same methods, you have to check that a given method has a restore property before calling it from tearDown. That's an if statement in your test case, and it should make you throw up.

Sinon offers collections to deal with these issues.

Stub collections

A stub collection is created by calling sinon.collection.create(). The resulting object has a stub() method which works exactly like sinon.stub(), only a reference to the resulting stub function is kept by the collection. This means you can create the collection in setUp, and call restore() on it in tearDown to restore all stubs back to normal.

The following is an example from the previous post, updated to use a collection: 

setUp: function () {
  this.fakes = sinon.collection.create();
},

tearDown: function () {
  this.fakes.restore();
},

"test using XMLHttpRequest open should be called before send": function () {
  var open = this.fakes.stub(XMLHttpRequest.prototype, "open");
  var send = this.fakes.stub(XMLHttpRequest.prototype, "send");

  jQuery.ajax("/some/url");

  assert(open.calledBefore(send));
  assert(send.calledAfter(open));
}

The collections can pretty easily be built into your test framework of choice, but as you will see shortly, Sinon has more in stock.

Mock collections

Mocks need to have their verify() method called. Collections support this as well: 

setUp: function () {
  this.fakes = sinon.collection.create();
},

tearDown: function () {
  this.fakes.verify();
},

"test should always pass argument to send": function () {
  this.fakes.mock(XMLHttpRequest.prototype).expects("send").withArgs(null);

  jQuery.ajax("/some/url");
}

Now that the mock is created through the collection the test no longer needs a local reference to it - it's verified "automatically" in the tearDown method.

Mix and match stubs and mocks

Stubs need to be reset while mocks need to be verified. Actually, mocks need to be reset as well, but verify does it for them. Collections have a third method, verifyAndRestore, which verifies all mocks, catches any exceptions, restores all fakes and then throws any expectation failures. Observe: 

setUp: function () {
  this.fakes = sinon.collection.create();
},

tearDown: function () {
  this.fakes.verifyAndRestore();
},

"test should always pass argument to send": function () {
  this.fakes.stub(XMLHttpRequest.prototype);
  this.fakes.mock(XMLHttpRequest.prototype).expects("send").withArgs(null);

  jQuery.ajax("/some/url");
}

No matter what happens to this test, the XMLHttpRequest object is restored to its native state when it's done.

Automating collections

While a lot better than manually juggling all the stubs and mocks, collections still require a tiny ceremony. Sinon can take care of this for you by wrapping the test method in a call to sinon.test. When you do so, Sinon creates a collection, binds the stub and mock methods on it and passes them as arguments to the test method. When the test is done (or if it fails), Sinon verifies-and-restores all the fakes before returning control to the test runner.

Let's revisit the ajax example: 

"test should always pass argument to send": sinon.test(function (stub, mock) {
  stub(XMLHttpRequest.prototype);
  mock(XMLHttpRequest.prototype).expects("send").withArgs(null);

  jQuery.ajax("/some/url");
})

Now we're getting somewhere. We're down to a single method call and two arguments to completely automate mock verification and global stub restoration.

Sinon test cases

One unfortunate consequence of using sinon.test is that fakes cannot be shared from setUp, as it does not have access to the bound stub and mock functions.

Sinon has an answer to this problem as well. By wrapping the entire test case in a call to sinon.testCase, Sinon gives every test method the sinon.test() treatment, and takes over running setUp and tearDown. This way, stubs and mocks can be created in set up to share between tests, but still be dismantled between each run.

The following example is our running example used with JsTestDriver: 

TestCase("AjaxTest", sinon.testCase({
  setUp: function (stub, mock) {
    stub(XMLHttpRequest.prototype);
  },

  "test should always pass argument to send": function (stub, mock) {
    mock(XMLHttpRequest.prototype).expects("send").withArgs(null);

    jQuery.ajax("/some/url");
  },

  "test using XMLHttpRequest open should be called before send": function () {
    jQuery.ajax("/some/url");

    var xhr = XMLHttpRequest.prototype;
    assert(xhr.open.calledBefore(xhr.send)); // or...
    assert(xhr.send.calledAfter(xhr.open));
  }
}));

What needs work?

One of the goals of Sinon is to be completely test framework agnostic - at least within the xUnit style frameworks. I've tried to satisfy this requirement by separating concerns as much as possible, and by layering high-level helpers such that any given framework can choose its entry level. Currently I've employed defaults that favor JsTestDriver, simply because it's the test runner I'm currently using.

Going forward I imagine making some of the behavior of the above tools configurable to adapt to a wider range of frameworks. For instance, there could possibly be some kind of configuration telling Sinon where to expose the bound stub and mock functions. Passed as arguments to tests is one way to do it - binding them to the test case object another. Hell, maybe even offer to expose them into the global environment for the folks who really want that.

Any feedback on these things would be greatly appreciated! I'm very open to feedback, both good and bad - but preferably constructive - so give it your best.

Assertions

The final piece of cosmetics for spy behavior verification is Sinon's assertions. Using custom assertions in place of the spy interface has two main benefits:

The following assertions mirror the methods available on spies, and I won't explain them in detail again. Please refer to the original post for detailed explanations. When an assertion accepts a "spy" as argument, it means you can pass it spies, stubs and mocks as they all implement the spy interface. In practice, you'll be using assertions with stubs.

Let's revisit our ajax example again, this time using assertions to verify the test: 

"test open should be called before send": sinon.test(function (stub, mock) {
  var open = stub(XMLHttpRequest.prototype, "open");
  var send = stub(XMLHttpRequest.prototype, "send");

  jQuery.ajax("/some/url");

  sinon.assert.callOrder(open, send);
})

Mixing assertions in on other objects

The only thing sticking out in this test is the namespace - JsTestDriver uses global assertions. To better fit in, we can instruct Sinon to expose the assertions onto another object, optionally renaming assertions by prefixing them with "assert". Observe: 


// Before loading tests
// this - global object in global scope
// true - map names to include the assert prefix
sinon.assert.expose(this, true);

TestCase("AjaxTest", sinon.testCase({
  "test open should be called before send": function (stub, mock) {
    var open = stub(XMLHttpRequest.prototype, "open");
    var send = stub(XMLHttpRequest.prototype, "send");

    jQuery.ajax("/some/url");

    assertCallOrder(open, send);
  }
}));

Beautiful - looks entirely like a native JsTestDriver assertion.

Failing assertions

In order for assertion failures to blend properly with the test framework's assertions, they need a common concept of failure. Sinon currently offers two ways to tweak this. The default behavior is to throw an AssertError exception. Incidentally, this is exactly what JsTestDriver does, meaning that the above example already works smoothly with it.

Other runners may define failure differently. To change the kind of assertion thrown, set the string property sinon.assert.failException to the desired type. If throwing an exception isn't compatible with the test runner's way of failing you can override sinon.assert.fail which is a function. The default one looks like this: 

function fail(message) {
  var error = new Error(message);
  error.name = this.failException || assert.failException;
  throw error;
}

As I mentioned this already matches the way JsTestDriver fails assertions. What's more is that JsTestDriver already has a global fail function. To avoid overwriting it, you can provide a third argument to sinon.assert.expose, which is a boolean specifying whether or not failException and fail should be exported as well. Pass false to leave them behind.

In conclusion

That's about it for now. I think, and hope that these tools will make integrating Sinon with your favorite test framework a breeze. Once I've gotten the first release out the door I will start actually integrating it with more than the one test framework I've tried so far to discover areas where my assumptions won't hold.

If you have any ideas on how to improve the interface or just general comments about Sinon, keep it coming! In the next and last (for now) post on Sinon I'll cover its utilities for testing timers and ajax requests.

As always, I'm @cjno on Twitter.

Published 27. May 2010 in javascript, test driven development, open source og sinon.js.

Possibly related

2006 - 2012 Christian Johansen Creative Commons License