Unit Testing With Hand Crafted Mocks

Unit Testing With Hand Crafted Mocks

Last updated: september 06 2004

sven.gorts@refactoring.be

Abstract: This article gives an overview of my personal experiences with hand written mocks. It also tries to capture and document the mocking idioms I've found most helpful so far.

Introduction

Many of the unit tests I wrote over the last couple of years use mock objects in order to test the behavior of a component in isolation of the rest of the system. So far, despite the availability of various mocking frameworks, each of the mock classes I've used has been hand written. In this article I do some retrospection and try to wrap up the mocking idioms I've found most usefull.

When I started experimenting with mock objects, I chose to write them by hand. Doing so allowed me to quickly gain some hands-on experience without the need to study any of the available mocking frameworks. At the time I expected to shift to use a mocking framework soon.

Those early mocking experiments really worked out. However, the codebase I was working on was in a bad state and needed urgent treatment so I deferred studying the available mocking frameworks and continued writing them by hand. For a while, the mocks happily evolved with the production code.

As the test harness grew, and ever more mocking behavior was required to support the different testcases. The mock classes got bloated and messy. I was in trouble: due to this code clutter the mocks themselves had become fragile. Testing got difficult.

Fortunately, the Single Responsibility Principle brought outcome. In general, the single responsibility principle states that a class should have only one reason to change. Applied to mocks this principle says that different kind of mocking behavior need to be carried out by different mock classes. Realizing this, all of a sudden entire families of mock classes arose, each family having its unique specialization: Counters, Saboteurs, Stores

As a result of this new insight much of the testcode as well as the mocks themselves could greatly be simplified. However, some tests required a combination of the behavior provided by two different mock classes. Here the Decorator pattern lead to a second breakthrough. mocks could be used to decorate each other.

Defining Mocks

There has been quite some discussion about the exact definition of the term mock object, even within the mock object community. For the mocking idioms presented here we require mocks to be implementations of an interface, rather than subclasses of an existing class. Further we don't require the mocks to be able to validate themselves, as opposed the the mainstream school of thought. Our mock implementations may provide helper methods to do so but generally we consider validation to be the responsibility of the test class.

So for the scope of this article we'll define a Mock Object as follows: a mock object is an instance that is intended for unit testing, and for which the substitution with production code instances is being enabled by a common interface for the mock and the production code classes.

Testing Mocks

Creating mocks by hand implies that the mocks we create themselves need to be validated. We certainly don't want to fool ourselves by testing with a buggy mock implementation, so we also write unit tests for the mocks themselves. In a sense unittesting a handcrafted mock is like the calibration of any measurement tool: if we skip the calibration step we can't trust our measurements to be accurate.

Many developers consider this need to test hand crafted mocks as an argument in favor of the available mock frameworks. However in contrast to this apparent disadvantage writing hand-crafted mocks also has its advantages:

Roles

When unit testing with mock objects we can distinguish three different roles: The Validator, the class that performs the tests. The Subject, the class being tested. And the Substitute, a class which instances can be plugged into the subject instances in order to mock some production code behavior.

Validator

The Validator role refers to the class or set of classes that are responsible for testing the behavior of a given Subject. In order to do so a Validator is allowed to interact with a Subject, as well as the Substitute the Subject's behavior relies upon. For instance: a TestCase within the JUnit framework.

Subject

The Subject role maps to the class that is to be tested. A Subject class which behavior is implemented in terms of a lower level interface and that allows its instances to be configured with different implementations. Using these interface a Subject class can be tested by means of a Substitute class rather than actual production code.

Substitute

The Substitute role refers to the class or classes that are used to test and validate the Subject. A Substitute class is a special purpose implementation of the interface the Subject is programmed to that provides behaviour for simulation and diagnosing of test conditions.

Substitutes can be used are used for the following purposes:


validator, subject and substitute are roles, not classes


In the catalog we'll name the concrete role of the substitute in terms of its responsibility, thereby forming a language in which we can express our idea's.

Mocking Idioms

Name:Counter
Problem:How to verify that the Subject calls a given method of the Substitute?
Solution:Have the Substitute role performed by a Counter class that keeps track of the number of calls to the methods defined on its interface.
Example: 
 
public class CounterCommunicator implements Communicator { 
  private int _opens; 
  private int _communicates; 
  private int _closes; 
 
  private Communicator _decoree; 
 
  public CounterCommunicator(Communicator decoree) { 
    _decoree = decoree; 
  } 
 
  public CounterCommunicator() { 
    this(null); 
  } 
 
  public void open() throws CommunicationException { 
    if (_decoree != null) 
      _decoree.open(); 
 
    _opens++; 
  } 
 
  public String communicate(String message) throws CommunicationException { 
    String reply = null; 
 
    if (_decoree != null) 
      reply = _decoree.communicate(message); 
 
    _communicates++; 
 
    return reply; 
  } 
 
  public void close() throws CommunicationException { 
    if (_decoree != null) 
      _decoree.close(); 
 
    _closes++; 
  } 
 
  public int opens() { 
    return _opens; 
  } 
 
  public int communicates() { 
    return _communicates; 
  } 
 
  public int closes() { 
    return _closes; 
  } 
}
Context: Counting the number of calls made to a method can be implemented by keeping an integer field for each method that's being incremented for each method call. For each method on the Substitute's interface a Counter will typically provide an accompanying methodCall which returns the number of times the method has been called or the binary version isMethodCalled that returns true when the method has been called at least once.

* * *
Name:Store
Problem:How to verify that Subject calls to Substitute methods happen with the expected argument values ?
Solution:Have the Substitute role performed by a Store class that stores the argument associated with each given method.
Example: 
 
public class StoreCommunicator implements Communicator { 
  private Communicator _decoree; 
 
  private String _lastMessage = ""; 
  private String _lastResponse; 
 
  public StoreCommunicator() { 
  } 
 
  public StoreCommunicator(Communicator decoree) { 
    _decoree = decoree; 
  } 
 
  public void open() throws CommunicationException { } 
 
  public String communicate(String message) throws CommunicationException { 
    String response = null; 
 
    if ( _decoree != null) 
      response = _decoree.communicate(message); 
 
    _lastMessage  = message; 
    _lastResponse = response; 
 
    return response; 
  } 
 
  public void close() throws CommunicationException { } 
 
  public String lastMessage() { 
    return _lastMessage; 
  } 
 
  public String lastResponse() { 
    return _lastResponse; 
  } 
}
Context: Storing method arguments works fine for methods that accept a single argument of a generic data type. For methods dealing with multiple arguments storing method calls is more cumbersome. For each method on the Substitute's interface a Store 'll typically provide an accompanying wasMethodCalledWith which passes in the expected values for the stored arguments and returns true when both match. When the Substitute's interface accepts a generic data type a methodCalledWith can be useful.

* * *
Name:Responder
Problem: How to verify the Subject when in order to write a test the Subject requires meaningfull return values from the Substitute.
Solution: Have the Substitute role performed by a class that can be configured to respond with a Responder for each given method calls.
Example: 
 
import java.util.ArrayList; 
import java.util.List; 
 
public class ResponderCommunicator implements Communicator { 
  private List _responses = new ArrayList(); 
 
  public void open() throws CommunicationException { } 
 
  public void close() throws CommunicationException { } 
 
  public String communicate(String message) throws CommunicationException { 
    if ( !_responses.isEmpty() ) 
      return (String)_responses.remove(0); 
 
    throw new CommunicationException("use setResponse to define responses"); 
  } 
 
  public void setResponse(String response) { 
    _responses.add(response); 
  } 
}
Context: Setting a fixed reply for given calls works pretty well for methods that need to be called only once. For methods that need to be called multiple times a variation the theme that allows to add a sequence of replies is also an option. For each method on the Substitute's interface a Responder 'll typically provide a setMethodReply method that allows to set the expected reply. For the sequence variation an addMethodReply method is more common.

* * *
Name:History
Problem: How to verify the sequence of calls the Subject makes against the Substitute ?
Solution: Implement the Substitute role with a class that logs the sequence of method calls made by Subject thus providing a history that can be validated by the Validator.
Example: 
 
import java.util.List; 
import java.util.ArrayList; 
 
public class HistoryCommunicator implements Communicator { 
  public static final String OPEN   = "OPEN"; 
  public static final String CLOSE  = "CLOSE"; 
  public static final Object COMMUNICATE = "COMMUNICATE"; 
 
  private List _history = new ArrayList(); 
 
  public void open() throws CommunicationException { 
    _history.add(OPEN); 
  } 
 
  public String communicate(String message) throws CommunicationException { 
    _history.add(COMMUNICATE); 
 
    return null; 
  } 
 
  public void close() throws CommunicationException { 
    _history.add(CLOSE); 
  } 
 
  public List list() { 
    return _history; 
  } 
}
Context: For each method on the Substitute's interface a History 'll provide an accompanying public constant METHOD thus allowing to compose a list of methods called. Whenever method is being called the corresponding METHOD constant is added to the History 's internal list. A history method allows the Validator to validate the list.

* * *
Name:Saboteur
Problem:How to verify the behavior of the Subject under error conditions.
Solution:Use a Saboteur for the Substitute role and have it simulate error conditions.
Example: 
 
public class SaboteurCommunicator implements Communicator { 
  private boolean _openIsSabotaged; 
  private boolean _closeIsSabotaged; 
  private boolean _communicateIsSabotaged; 
 
  public void open() throws CommunicationException { 
    if ( _openIsSabotaged ) 
      throw new CommunicationException("open() sabotaged"); 
  } 
 
  public String communicate(String message) throws CommunicationException { 
    if ( _communicateIsSabotaged ) 
      throw new CommunicationException("communicate() sabotaged"); 
 
    return null; 
  } 
 
  public void close() throws CommunicationException { 
    if ( _closeIsSabotaged ) 
      throw new CommunicationException("close() sabotaged"); 
  } 
 
  public void sabotageOpen() { 
    _openIsSabotaged = true; 
  } 
 
  public void sabotageClose() { 
    _closeIsSabotaged = true; 
  } 
 
  public void sabotageCommunicate() { 
    _communicateIsSabotaged = true; 
  } 
}
Context:A Saboteur class is an instance of the Substitute role that has the sole responsibility of simulating error conditions in response to its method calls. For each method of the Substitute interface a Saboteur class typically provides a corresponding sabotageMethod method that allows to set whether or not the given call needs to be sabotaged. Variations on the theme are a sabotageNextMethod that only sabotages the first call made to method.

* * *
Name:Printer
Problem:Interactions between Subject and Substitute need to be visualized
Solution:Decorate the Substitute with a Printer that provides a visual trace of the runtime interactions between the Subject and the Substitute.
Example: 
 
import java.io.Writer; 
import java.io.PrintWriter; 
 
public class PrinterCommunicator implements Communicator{ 
  private PrintWriter _output; 
  private Communicator _decoree; 
 
  public PrinterCommunicator(Communicator communicator) { 
    this(new PrintWriter(System.out), communicator); 
  } 
 
  public PrinterCommunicator(Writer output, Communicator communicator) { 
    _decoree = communicator; 
    _output = new PrintWriter(output); 
  } 
 
  public void open() throws CommunicationException { 
    _output.println("open()"); 
    _output.flush(); 
 
    if ( _decoree != null ); 
      _decoree.open(); 
  } 
 
  public String communicate(String message) throws CommunicationException { 
    _output.println("-> communicate("+ message + ")"); 
    _output.flush(); 
 
    String response = ""; 
 
    if ( _decoree != null ) 
      response = _decoree.communicate(message); 
 
    _output.println("<- ["+ response + "]"); 
    _output.flush(); 
 
    return response; 
  } 
 
  public void close() throws CommunicationException { 
    if ( _decoree != null ); 
      _decoree.close(); 
 
    _output.println("close()"); 
    _output.flush(); 
  } 
}
Context:Reasoning about the flow of interactions between Subject and Substitute can be difficult and confusing. In such cases having a visual trace of what is going on is helpful as sanity check for our reasoning. A Printers responsibility is limited to visualizing and forwarding incoming calls from the Subject and forwarding them to the Substitute it decorates. By implementing it as a Decorator tracing is separated from production code and can easily be toggled by decorating the Substitute. Extra methods on the public interface of the Printer class are typically limited to formatting rules.

* * *

Conclusion

After more than two years of experience in unit testing with hand written mocks I still haven't felt the need for using a more advanced mocking framework For most of the unit tests I've written the simple specialized mocks described in this paper where sufficient and in the rare occasions they weren't I found I could combine mocks together by means of decoration.

References