Analyzing Testability, Part 2.

This post will expand on points made in Part 1. Previously we identified four categories of methods with different characteristics concerning direct observability. The first category of methods contains parameters and a non void return type in its signature, such as:

int Abs(int x)

All other things being equal, this category of methods is more easily tested. Since we can observe a result directly and control the test input directly. If the method is side effects free we have a pure function which is the cream of the crop regarding testability. The same input will always produce the same output. You should be writing the most important business logic in pure functions for maximum certainty of its behaviour.

The second category by example:

int Next()

Necessarily requires a more involved test setup since we must control its input by other means than direct parameterization. Of course we might be so lucky that Next() is part of a create-set-call pattern that allows us to control the parameterization indirectly. E.g

var c = new SomeCollection() // create

c.SetElementAt(0, 4) // set
c.SetElementAt(1, 7) // set

var result = c.Next() // call

Assert.AreEqual(4, result)

The following category doesn’t provide a directly observable result but lets us parameterize the method invocation:

void SetStateExecuting(Task task)

So how do we observe the result of an experiment? Easy as pie, we just write this (example is from stackoverflow and slightly edited..):

public void SetStateExecuting_Should_Set_State_To_Pause()
var task = new Task { ID = 1, TimeZone = -660, GlobalState = TaskState.Paused };
task.Expect(p => p.StateUpdate(task.ID, task.TimeZone, TaskState.Paused));
_task.AssertWasNotCalled(p => p.GlobalStateUpdate(task.ID, TaskState.Executing));

I’m sure you can spot the problem that this stackoverflow user is combatting? He is mixing the old Record/Replay pattern and the new AAA pattern in Rhino Mocks. This test style tends to be fragile since we concern ourselves with how SetStateExecuting is implemented. For example in the form of a call to AssertWasNotCalled that asserts that a specific method was called. If the method in question changes it signature, the test will break.

The last category of methods, where no parameters are defined and the return type is void, we have no way of observing directly. E.g. void DoSomething(). We have to know where to look for changes in state in the surrounding class, or use a mocking framework as the previous example. In other words, knowledge of the behavior is harder to come by and the risk of fragile tests has increased.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s