ContractTesting

Contract testing is a means of verifying that a TestDouble adheres to the same BehavioralContract as the UnitOfCode it stands in for.

The general principle of contract testing is that the same suite of tests ought to pass when run against the production implementation of a system or against the Fake implementation used in testing.

If you use TestDoubles but don't contract-test them or write IntegrationTests, it's more likely that you'll find yourself in a situation where all your UnitTests pass but the system as a whole doesn't work.

Contract tests have many of the same benefits as IntegrationTests, since they can alert you to cases where the pieces of the system won't work together.

vs. J.B. Rainsberger's Definition

J.B. Rainsberger uses the term "contract test" to refer to a suite of test cases that one can run against any object to validate that it correctly implements an interface. However, my impression from reading J.B.'s blog is that he stops short of actually running the contract tests against his test doubles, because he doesn't build test doubles that correctly implement interfaces—he just uses mocks and stubs.

His solution to this is, apparently, to manually verify that every interaction he programs into a mock or stub has a corresponding contract test that runs against the real implementation. So, borrowing from his countWords example, if he wants to write a test for an editor plugin that calls countWords, he does something like this:

// pseudocode
const countWords = stub().given("text with four words").returns(4)
const plugin = new MyPlugin(countWords)
// ... assert some stuff about the plugin here

And then he makes sure that the contract suite for countWords contains a test like this:

assert(countWords("text with four words"), is, 4)

That seems like a lot of work to avoid calling a real reference implementation of the interface. Note that by constraining his stubs to only return values that the real countWords would return, he's essentially coupling his tests for MyPlugin to the real behavior of countWords anyway.

In this case, I wouldn't use test doubles at all. countWords is a pure freaking function, for crying out loud! You don't need to stub it! Perhaps you're worried that the CPU time required to call a potentially deep stack of library functions will slow down your tests, but let me remind you of an important point:

AnyCodeIsFastForSmallInputs

Algorithms are not slow. Or, more precisely, it is meaningless to ask how fast or slow an algorithm is, in WallClockTime. Algorithms have a time complexity that is a function of input size. If your UnitTests are slow, the fix is almost always to use smaller data. If your data is already as small as possible, refactor to flatten the dependency hierarchy, and find boundaries at which you can make dependencies more abstract so real collaborators can be replaced with Fakes (and contract-test those fakes!)

Tools

RSpec's shared example groups let one easily run a suite of tests against two different subjects. This makes contract testing easier.

In JUnit, you can create an abstract test class and put your contract test cases there. Then you can inherit from that abstract class to create a test suite for each class that must implement the contract. Here is an example: the abstract contract test class, and two subclasses: one, two.