How to Test Drive Time Dependent Code

April 2004

philippe.tseyen@refactoring.be


Abstract

One of the more difficult parts of software development is how to test drive time dependent code. Based on an easy example I will explain how to solve some of the problems that pop up during development of time dependent code.




A few weeks ago I was pair programming on a project called ProjectTimer. I will use a simplified version of the ProjectTimer (only the first story) to illustrate the challenges that we are facing when developing time depedent code.

Story 1: keep track of the time spent on a project

Ok, let's start with a simple TestCase. 

import junit.framework.TestCase;

public class ProjectTestCase extends TestCase
{
    public void testElapsedTime() throws InterruptedException {
        Project project = new Project();
        project.start();
        Thread.sleep(5000);
        project.stop();
        assertEquals(5000, project.elapsedTime());
    }
}

We create a new project and work for it during 5 seconds. Then we can assert the total time spent on the project equals 5 seconds.

Ok, let's make it compile. 

public class Project
{
    public void start()
    {

    }

    public void stop()
    {

    }

    public long elapsedTime()
    {
        return 0;
    }
}

Red bar, great. Let's use obvious implementation. 

public class Project
{
    private long beginTime;
    private long endTime;

    public void start()
    {
        beginTime = System.currentTimeMillis();
    }

    public void stop()
    {
        endTime = System.currentTimeMillis();
    }

    public long elapsedTime()
    {
        return endTime - beginTime;
    }
}

Even better, green bar ... or not ? Run it a few times ... 

junit.framework.AssertionFailedError: expected:<5000> but was:<5007>
	at junit.framework.Assert.fail(Assert.java:47)
	at junit.framework.Assert.failNotEquals(Assert.java:282)
	at junit.framework.Assert.assertEquals(Assert.java:64)
	at junit.framework.Assert.assertEquals(Assert.java:136)
	at junit.framework.Assert.assertEquals(Assert.java:142)
	at ProjectTestCase.testElapsedTime(ProjectTestCase.java:10)

junit.framework.AssertionFailedError: expected:<5000> but was:<4997>
	at junit.framework.Assert.fail(Assert.java:47)
	at junit.framework.Assert.failNotEquals(Assert.java:282)
	at junit.framework.Assert.assertEquals(Assert.java:64)
	at junit.framework.Assert.assertEquals(Assert.java:136)
	at junit.framework.Assert.assertEquals(Assert.java:142)
	at ProjectTestCase.testElapsedTime(ProjectTestCase.java:10)

It looks like our TestCase is non deterministic. We sleep for 5 seconds, so the elapsed time is at least 5 seconds. What is even worse is that although we sleep for 5000 millis, the elapsed time is sometimes less than 5000 millis. Apparently the combination of Thread.sleep(int) and System.currentTimeMillis() is not reliable.

From the javadoc of System.currentTimeMillis():
Returns the current time in milliseconds. Note that while the unit of time of the return value is a millisecond, the granularity of the value depends on the underlying operating system and may be larger. For example, many operating systems measure time in units of tens of milliseconds.

Ok, let's fix the TestCase.

We will build in a safety margin. If we sleep for five seconds, we can assert that the elapsed time is more than four seconds. Run it a few times: all green. 

import junit.framework.TestCase;

public class ProjectTestCase extends TestCase
{
    public void testElapsedTime() throws InterruptedException {
        Project project = new Project();
        project.start();
        Thread.sleep(5000);
        project.stop();
        assertTrue(project.elapsedTime() > 4000);
    }
}

Great ?

Although we have a green bar, a few things are bothering us. First, we haven't really solved the undeterministic behaviour. We introduced a safety margin of one second but we don't know if this is enough to guarantee that the test will always be green. Secondly, our TestCase does not communicate clearly it's intent anymore.

Ok, let's stub out the problematic behaviour.

We extract the System.currentTimeMillis() in a separate method which is made package private so it can be overridden. 

public class Project
{
    private long beginTime;
    private long endTime;

    public void start()
    {
        beginTime = getCurrentTimeMillis();
    }

    public void stop()
    {
        endTime = getCurrentTimeMillis();
    }

    public long elapsedTime()
    {
        return endTime - beginTime;
    }

    long getCurrentTimeMillis() {
        return System.currentTimeMillis();
    }
}

We adapt our TestCase and create a stub for the Project class, where we can set the return value of the getCurrentTimeMillis(). 

import junit.framework.TestCase;

public class ProjectTestCase extends TestCase {
    public void testElapsedTime() throws InterruptedException {
        ProjectStub project = new ProjectStub();
        project.setCurrentTime(0);
        project.start();
        project.setCurrentTime(5000);
        project.stop();
        assertEquals(5000, project.elapsedTime());
    }

    private static class ProjectStub extends Project {
        private long currentTime;

        public void setCurrentTime(long currentTime) {
            this.currentTime = currentTime;
        }

        long getCurrentTimeMillis() {
            return currentTime;
        }
    }
}

Great, we have deterministic behaviour now and the TestCase communicates it's intent.

But we are still not satisfied with the result. From our experience we know that the need for stubs in newly written code is a smell.

We think about the Single Responsibility Principle (SRP)[1]: "A class should have only one reason to change".

Let's take a closer look at the Project class. The Project class is responsible for keeping track of how much time is spent on a project. Secondly Project also knows how to retrieve the current time. If we change the way we calculate the time spent, Project must change. If we change the way we retrieve the current time, Project must change.

Looks like our design is not SRP compliant.

Let's move to a SRP compliant solution, by separating these two responsibilities. We shield the retrieval of the current time behind an interface TimeServer. In our TestCase we use a mock implementation of this interface. 

import junit.framework.TestCase;

public class ProjectTestCase extends TestCase {
    public void testElapsedTime() throws InterruptedException {
        TimeServerMock timeServer = new TimeServerMock();
        Project project = new Project(timeServer);
        timeServer.currentTimeMillis = 0;
        project.start();
        timeServer.currentTimeMillis = 5000;
        project.stop();
        assertEquals(5000, project.elapsedTime());
    }

    public static class TimeServerMock implements TimeServer {
        public long currentTimeMillis;

        public long currentTimeMillis() {
            return currentTimeMillis;
        }
    }
}

Ok, let's adapt the production code. 

public class Project
{
    private long beginTime;
    private long endTime;

    private TimeServer timeServer;

    public Project(TimeServer timeServer) {
        this.timeServer = timeServer;
    }

    public void start()
    {
        beginTime = timeServer.currentTimeMillis();
    }

    public void stop()
    {
        endTime = timeServer.currentTimeMillis();
    }

    public long elapsedTime()
    {
        return endTime - beginTime;
    }
}

public interface TimeServer
{
    /**
     * Returns the current time in milliseconds. Implementations may vary in
     * accuracy. Define t1, t2 two moments in time; define T the
     * currentTimeMillis function. If t1 comes before t2 than T(t1) <= T(t2).
     *
     * @return	the current time in milliseconds.
     */
    long currentTimeMillis();
}

Okay, time for a quick retrospective. We didn't like the stubbing stuff because it revealed a design problem. So we introduced an extra interface to come to a SRP compliant solution. We defined the contract for the TimeServer.

Hey, we didn't test the TimeServer implementation ! That's right, we are not forced by our TestCase to provide a production implementation of our TimeServer and that is exactly what we want: our TestCase only defines the functionality of the elapsedTime() method and does not care about how the current time is retrieved. Our design is "open for extension and closed for modification"[1], applied to the way we retrieve the current time.

When the need for a TimeServer implementation arises we make the TimeServer contract explicit with a unit test. 

import junit.framework.TestCase;

public class DefaultTimeServerTestCase extends TestCase
{
    public void testCurrentTimeMillis() throws InterruptedException {
        TimeServer timeServer = new DefaultTimeServer();
        long firstTime = timeServer.currentTimeMillis();
        long secondTime = timeServer.currentTimeMillis();
        Thread.sleep(10);
        long thirdTime = timeServer.currentTimeMillis();
        assertTrue(firstTime <= secondTime);
        assertTrue(secondTime <= thirdTime);
    }
}

And we provide an implementation.

public class DefaultTimeServer implements TimeServer{
    public long currentTimeMillis() {
        return System.currentTimeMillis();
    }
}

Great, we finished our first story.

References

[1] Robert C. Martin: Agile Software Development Principles, Patterns, and Practices