How to Test Drive Time Dependent Code Continued
philippe.tseyen@refactoring.be
Abstract
One of the more difficult parts of software development is how to test drive time dependent code. In part one we already explained how to handle test code that is dependent on timing. But sometimes, the timing dependency itself is inherent to the software being developed. What follows is a simplified example of a story that we implemented a couple of weeks ago.
First off all a bit of domain context. We were working on an application that uses a database. In order to do something with
the database we need to ask the database for a connection. Our database is being used by multiple users concurrently; so it
might happen that the database already has too many connections. The database indicates this by throwing an exception. The
database is used to upload a batch of customer data (name, adresses). There is no real time processing needed. So it makes
makes sense to wait a while, and then retry to obtain a connection, instead of directly failing the whole batch upload.
Story: linear retry mechanism for establishing a database connection. The delay and the number of retries can be specified.
Ok, here is the code that was already available.
public class TooManyConnectionsException extends Exception {
}
public interface Database {
Connection connect() throws TooManyConnectionsException;
}
public class Connection {
}
Ok, let's start writing our first test.
We need a testcase, DatabaseConnectorTestCase is the first name that pops up, we will rename it later if we find a better name. The test itself is rather trivial: if we can connect to the database immediately we need to have a connection object. We provide necessary classes and mocks to make it compile.
import junit.framework.TestCase;
public class DatabaseConnectorTestCase extends TestCase {
public void testImmediateConnection() {
DatabaseMock database = new DatabaseMock();
DatabaseConnector databaseConnector = new DatabaseConnector(database);
assertNotNull(databaseConnector.connect());
}
}
public class DatabaseConnector {
public DatabaseConnector(Database database) {
}
public Connection connect() {
return null;
}
}
public class DatabaseMock implements Database {
public Connection connect() throws TooManyConnectionsException {
return null;
}
}
Run our test ...
OK, red bar. Let's implement the mock and use obvious implementation for the production class. The DatabaseConnector
should ask the Database for a connection.
public class DatabaseMock implements Database {
public Connection connect() {
return new Connection();
}
}
public class DatabaseConnector {
private Database database;
public DatabaseConnector(Database database) {
this.database = database;
}
public Connection connect() {
try {
return database.connect();
} catch (TooManyConnectionsException tmce) {
throw new Error(tmce);
}
}
}
We catch the TooManyConnectionsException and throw an Error, this is probably not the behaviour we want, but apparently
its enough to get our test working. We put it on our todo list to make sure, we come back to it later.
Green bar.
Our next test case communicates the behaviour we expect when we can not connect to the database the first time and we don't have a retry scheme.
import junit.framework.TestCase;
public class DatabaseConnectorTestCase extends TestCase {
public void testImmediateConnection() throws ConnectionFailedException {
DatabaseMock database = new DatabaseMock();
DatabaseConnector databaseConnector = new DatabaseConnector(database);
assertNotNull(databaseConnector.connect());
}
public void testNoRetries()
{
DatabaseMock database = new DatabaseMock();
database.throwTooManyConnectionsException = true;
DatabaseConnector databaseConnector = new DatabaseConnector(database);
try
{
databaseConnector.connect();
fail();
}
catch (ConnectionFailedException expected)
{
}
}
}
We create the ConnectionFailedException, implement our mock, let connect throw ConnectionFailedException and adapt our previous testcase to throw ConnectionFailedException.
public class ConnectionFailedException extends Exception{
}
public class DatabaseMock implements Database {
public boolean throwTooManyConnectionsException;
public Connection connect() throws TooManyConnectionsException {
if (throwTooManyConnectionsException) throw new TooManyConnectionsException();
return connectReturnValue;
}
}
public class DatabaseConnector {
private Database database;
public DatabaseConnector(Database database) {
this.database = database;
}
public Connection connect() throws ConnectionFailedException{
try {
return database.connect();
} catch (TooManyConnectionsException tmce) {
throw new Error(tmce);
}
}
}
Compiles and red bar: great ! Let's implement.
public class DatabaseConnector {
private Database database;
public DatabaseConnector(Database database) {
this.database = database;
}
public Connection connect() throws ConnectionFailedException{
try {
return database.connect();
} catch (TooManyConnectionsException tmce) {
throw new ConnectionFailedException();
}
}
}
Ok, green bar. Next testcase. We set up our databaseconnector for one retry and with an initial delay of 10 seconds. Then we assert that the time it takes to set up the connection takes atleast 10 seconds.
import junit.framework.TestCase;
public class DatabaseConnectorTestCase extends TestCase {
private DatabaseMock database = new DatabaseMock();
public void testImmediateConnection() throws ConnectionFailedException {
DatabaseConnector databaseConnector = new DatabaseConnector(database);
assertNotNull(databaseConnector.connect());
}
public void testNoRetries() {
database.throwTooManyConnectionsException = true;
DatabaseConnector databaseConnector = new DatabaseConnector(database);
try {
databaseConnector.connect();
fail();
} catch (ConnectionFailedException expected) {
}
}
public void testConnectionAfterOneRetry() throws ConnectionFailedException {
final int delay = 10000;
DatabaseConnector databaseConnector = new DatabaseConnector(database, 1, delay);
long begin = System.currentTimeMillis();
assertNotNull(databaseConnector.connect());
long end = System.currentTimeMillis();
assertTrue(end - begin >= delay);
}
}
We see that our mock does not fit our needs anymore and we refactor it.
public class DatabaseMock implements Database {
private int numberOfAttempts = 0;
public Connection connect() throws TooManyConnectionsException {
if (numberOfAttempts == 1) {
numberOfAttempts--;
return new Connection();
} else {
numberOfAttempts--;
throw new TooManyConnectionsException();
}
}
public void setImmediateConnection() {
numberOfAttempts = 1;
}
public void setConnectionAfterRetry() {
numberOfAttempts = 2;
}
}
Run all tests, still green (off course, we commented out the one test we were working during the refactoring ;-)). Ok, let's get back to our failing test now and provide the implementation.
public class DatabaseConnector {
private Database database;
private int numberOfRetries;
private long delay;
public DatabaseConnector(Database database) {
this(database, 0, 0);
}
public DatabaseConnector(Database database, int numberOfRetries, long delay) {
this.database = database;
this.numberOfRetries = numberOfRetries;
this.delay = delay;
}
public Connection connect() throws ConnectionFailedException {
try {
return database.connect();
} catch (TooManyConnectionsException tmce) {
for (int i = 1; i <= numberOfRetries; i++) {
try {
Thread.sleep(delay);
} catch (InterruptedException e) {
throw new Error(e);
}
try {
return database.connect();
} catch (TooManyConnectionsException ignore) {
}
}
throw new ConnectionFailedException();
}
}
}
Okay, time to look back on what we have now. We see several problems.
First, our tests our undeterministic due to the Thread.sleep in our production code.Furthermore our test takes too long. We could reduce the run time to e.g. 2 seconds, but even then it is a huge cost, which is often underestimated.
run tests 60 times an hour * 40 hour man week * 30 developers working on code = 60 * 40 * 30 * 2 = 40 hours
Off course this is a rather simplistic way off calculating, but just think about it.
We have catched the InterruptedException and thrown an Error, how should we handle the InteruptedException and how can we test it ? We add this to our todo list, to make sure we do not forget about this.
Suppose we leave the code like this, how can we implement our test to test for a linear increase in delay between retries. That will even be harder than our current testcases.
First of all we clean up our current DatabaseConnector: just some simple extract methods to improve readability.
public class DatabaseConnector {
private Database database;
private int numberOfRetries;
private long delay;
public DatabaseConnector(Database database) {
this(database, 0, 0);
}
public DatabaseConnector(Database database, int numberOfRetries, long delay) {
this.database = database;
this.numberOfRetries = numberOfRetries;
this.delay = delay;
}
public Connection connect() throws ConnectionFailedException {
try {
return database.connect();
} catch (TooManyConnectionsException tmce) {
return retry();
}
}
private Connection retry() throws ConnectionFailedException {
for (int i = 1; i <= numberOfRetries; i++) {
sleep();
try {
return database.connect();
} catch (TooManyConnectionsException ignore) {
}
}
throw new ConnectionFailedException();
}
private void sleep() {
try {
Thread.sleep(delay);
} catch (InterruptedException e) {
throw new Error(e);
}
}
}
Remember our TimeServer from the first part: here it is again. We add a method sleepMillis to the TimeServer, provide an appropriate mock implementation and rewrite the testcase in function of the TimeServer. Our testcase becomes much more simpler and clear: we can just assert that the TimeServer sleepMillis method is called one with a given delay.
public interface TimeServer
{
/**
* Returns the current time in milliseconds. Implementations may vary in
* accuracy. Define t1, t2 two moments in time; define T the
* currentTimeMillis function. If t1 comes before t2 than T(t1) <= T(t2).
*
* @return the current time in milliseconds.
*/
long currentTimeMillis();
/**
* Causes the currently executing thread to sleep (temporarily cease
* execution) for the specified number of milliseconds.
*
* @param delay number of milliseconds to sleep [0, Integer.MAX_VALUE]
* @throws InterruptedException if another threads interrupts this thread
* while sleeping
*/
void sleepMillis(long delay) throws InterruptedException;
}
public class TimeServerMock implements TimeServer {
public long currentTimeMillis;
private StringBuffer stringBuffer = new StringBuffer();
public long currentTimeMillis() {
return currentTimeMillis;
}
public void sleepMillis(long delay) throws InterruptedException {
if (stringBuffer.length() != 0) {
stringBuffer.append(" > ");
}
stringBuffer.append("sleepMillis " + delay);
}
public String toString() {
return stringBuffer.toString();
}
}
import junit.framework.TestCase;
public class DatabaseConnectorTestCase extends TestCase {
private TimeServerMock timeServer = new TimeServerMock();
private DatabaseMock database = new DatabaseMock();
public void testImmediateConnection() throws ConnectionFailedException {
database.setImmediateConnection();
DatabaseConnector databaseConnector = new DatabaseConnector(database, timeServer);
assertNotNull(databaseConnector.connect());
}
public void testNoRetries() {
database.setConnectionAfterRetry();
DatabaseConnector databaseConnector = new DatabaseConnector(database, timeServer);
try {
databaseConnector.connect();
fail();
} catch (ConnectionFailedException expected) {
}
}
public void testConnectionAfterOneRetry() throws ConnectionFailedException {
database.setConnectionAfterRetry();
DatabaseConnector databaseConnector = new DatabaseConnector(database, 1, 10000, timeServer);
assertNotNull(databaseConnector.connect());
assertEquals("sleep 10000", timeServer.toString());
}
}
It is important to notice that we recognize the TimeServer interface as a core abstraction within the problem domain. Otherwise we would define a separate interface only containing a sleepMillis method, since that is the only one we need.
Red bar for the third testcase. Off course DatabaseConnector should now call the TimeServer sleepMillis method.
public class DatabaseConnector {
private Database database;
private int numberOfRetries;
private long delay;
private TimeServer timeServer;
public DatabaseConnector(Database database, TimeServer timeServer) {
this(database, 0, 0, timeServer);
}
public DatabaseConnector(Database database, int numberOfRetries, long delay, TimeServer timeServer) {
this.database = database;
this.numberOfRetries = numberOfRetries;
this.delay = delay;
this.timeServer = timeServer;
}
public Connection connect() throws ConnectionFailedException {
try {
return database.connect();
} catch (TooManyConnectionsException tmce) {
return retry();
}
}
private Connection retry() throws ConnectionFailedException {
for (int i = 1; i <= numberOfRetries; i++) {
sleep();
try {
return database.connect();
} catch (TooManyConnectionsException ignore) {
}
}
throw new ConnectionFailedException();
}
private void sleep() {
try {
timeServer.sleepMillis(delay);
} catch (InterruptedException e) {
throw new Error(e);
}
}
}
Green !
We are still uncomfortable with the throw new Error in the sleep method. We have three options here.
We could leave it as it is, which means that we are sure that the Thread where the sleep is executed on will never be interupted. If this thread is interrupted, then this is a bug. We do not write a test for this.
We could also catch the Interupted exception and not handle it. This makes sence if we want to go on with retrying connections. We test this by letting TimeServerMock throw InteruptedException and see that it does not influence our flow.
Last possibility is to catch the interrupted exception, give up retrying and throw a ConnectionFailedException.
After some discussion we concluded that the last option is most suited for our application, since interrupts only happen when the application is shut down. We define the testcase and adapt our mock.
import junit.framework.TestCase;
public class DatabaseConnectorTestCase extends TestCase {
private TimeServerMock timeServer = new TimeServerMock();
private DatabaseMock database = new DatabaseMock();
public void testImmediateConnection() throws ConnectionFailedException {
database.setImmediateConnection();
DatabaseConnector databaseConnector = new DatabaseConnector(database, timeServer);
assertNotNull(databaseConnector.connect());
}
public void testNoRetries() {
database.setConnectionAfterRetry();
DatabaseConnector databaseConnector = new DatabaseConnector(database, timeServer);
try {
databaseConnector.connect();
fail();
} catch (ConnectionFailedException expected) {
}
}
public void testConnectionAfterOneRetry() throws ConnectionFailedException {
database.setConnectionAfterRetry();
DatabaseConnector databaseConnector = new DatabaseConnector(database, 1, 10000, timeServer);
assertNotNull(databaseConnector.connect());
assertEquals("sleepMillis 10000", timeServer.toString());
}
public void testInterruptedDuringRetry()
{
timeServer.interruptSleepMillis = true;
database.setConnectionAfterRetry();
DatabaseConnector databaseConnector = new DatabaseConnector(database, 1, 10000, timeServer);
try {
databaseConnector.connect();
fail();
} catch (ConnectionFailedException expected) {
}
}
}
public class TimeServerMock implements TimeServer {
public long currentTimeMillis;
private StringBuffer stringBuffer = new StringBuffer();
public boolean interruptSleepMillis;
public long currentTimeMillis() {
return currentTimeMillis;
}
public void sleepMillis(long delay) throws InterruptedException {
if (stringBuffer.length() != 0) {
stringBuffer.append(" > ");
}
stringBuffer.append("sleepMillis " + delay);
if (interruptSleepMillis) throw new InterruptedException();
}
public String toString() {
return stringBuffer.toString();
}
}
Red bar. Let's implement.
public class DatabaseConnector {
private Database database;
private int numberOfRetries;
private long delay;
private TimeServer timeServer;
public DatabaseConnector(Database database, TimeServer timeServer) {
this(database, 0, 0, timeServer);
}
public DatabaseConnector(Database database, int numberOfRetries, long delay, TimeServer timeServer) {
this.database = database;
this.numberOfRetries = numberOfRetries;
this.delay = delay;
this.timeServer = timeServer;
}
public Connection connect() throws ConnectionFailedException {
try {
return database.connect();
} catch (TooManyConnectionsException tmce) {
return retry();
}
}
private Connection retry() throws ConnectionFailedException {
for (int i = 1; i <= numberOfRetries; i++) {
try {
timeServer.sleepMillis(delay);
return database.connect();
} catch (TooManyConnectionsException ignore) {
} catch (InterruptedException e) {
throw new ConnectionFailedException();
}
}
throw new ConnectionFailedException();
}
}
Green bar.
We remove the 2 argument constructor from DatabaseConnector since, we only call the other constructor in production code.
We move the default settings to the TestCase.
Last thing we need to cover is that the delay between retries increases linearly.
import junit.framework.TestCase;
public class DatabaseConnectorTestCase extends TestCase {
private TimeServerMock timeServer = new TimeServerMock();
private DatabaseMock database = new DatabaseMock();
public void testImmediateConnection() throws ConnectionFailedException {
database.setImmediateConnection();
DatabaseConnector databaseConnector = new DatabaseConnector(database, 0, 0, timeServer);
assertNotNull(databaseConnector.connect());
}
public void testNoRetries() {
database.setConnectionAfterRetry();
DatabaseConnector databaseConnector = new DatabaseConnector(database, 0, 0, timeServer);
try {
databaseConnector.connect();
fail();
} catch (ConnectionFailedException expected) {
}
}
public void testConnectionAfterOneRetry() throws ConnectionFailedException {
database.setConnectionAfterRetry();
DatabaseConnector databaseConnector = new DatabaseConnector(database, 1, 10000, timeServer);
assertNotNull(databaseConnector.connect());
assertEquals("sleepMillis 10000", timeServer.toString());
}
public void testInterruptedDuringRetry() {
timeServer.interruptSleepMillis = true;
database.setConnectionAfterRetry();
DatabaseConnector databaseConnector = new DatabaseConnector(database, 1, 10000, timeServer);
try {
databaseConnector.connect();
fail();
} catch (ConnectionFailedException expected) {
}
}
public void testLinearDelayBetweenRetries() throws ConnectionFailedException {
database.setConnectionAfterTwoRetries();
DatabaseConnector databaseConnector = new DatabaseConnector(database, 2, 10000, timeServer);
assertNotNull(databaseConnector.connect());
assertEquals("sleepMillis 10000 > sleepMillis 20000", timeServer.toString());
}
}
Red bar, and an easy fix.
public class DatabaseConnector {
private Database database;
private int numberOfRetries;
private long delay;
private TimeServer timeServer;
public DatabaseConnector(Database database, int numberOfRetries, long delay, TimeServer timeServer) {
this.database = database;
this.numberOfRetries = numberOfRetries;
this.delay = delay;
this.timeServer = timeServer;
}
public Connection connect() throws ConnectionFailedException {
try {
return database.connect();
} catch (TooManyConnectionsException tmce) {
return retry();
}
}
private Connection retry() throws ConnectionFailedException {
for (int i = 1; i <= numberOfRetries; i++) {
try {
timeServer.sleepMillis(i*delay);
return database.connect();
} catch (TooManyConnectionsException ignore) {
} catch (InterruptedException e) {
throw new ConnectionFailedException();
}
}
throw new ConnectionFailedException();
}
}
We still have the return database.connect(); statement that is duplicated in the DatabaseConnector. Let's try to get rid of it.
public Connection connect() throws ConnectionFailedException {
for (int i = 0; i <= numberOfRetries; i++) {
try {
if (i != 0)
timeServer.sleepMillis(i * delay);
} catch (InterruptedException e) {
throw new ConnectionFailedException();
}
try {
return database.connect();
} catch (TooManyConnectionsException e) {
}
}
throw new ConnectionFailedException();
}
mmm, we are not convinced that our latest change will make the code more readable: we obtained only a minor simplification at the cost of less explicit code. Let's rollback to the previous version.
We are both pretty satisfied with our code quality now: no duplication, the simplest thing that could possibly work, testcases communicate their intent.We check our todo list to see whether we handled all issues: OK. We run all tests, watch the green bar, we check in.
Drink ?