software development

How I do TDD

TL;DR: The specification I write are based on domain-level Ubiquitous Language-based specifications of system behaviour. These spec tests describe the functional behaviour of the system, as it would be specified by a Product Owner (PO), and as it would be experienced by the user; i.e. user-level functional specifications.


I get asked frequently how I do Test-Driven Development (TDD), especially with example code. In my previous post I discussed reasons why I do TDD. In this post I’ll discuss how I do TDD.

First-off, in my opinion, there is no difference between TDD, Behaviour-Driven Development (BDD) and Acceptance-Test-Driven Development (ATDD). The fundamental concept is identical: write a failing test, then write the implementation that makes that test pass. The only difference is that these terms are used to describe the ‘amount’ of the system being specified. Typically, with BDD, at a user level, with ATDD, at a user acceptance level (like BDD), and TDD for much finer-grained components or parts of the system. I see very little value in these distinctions: they’re all just TDD to me. (I do find it valuable to discuss the ROI of TDD specifications at different levels, as well as their cost and feedback speed. I’m currently working on a post discussing these ideas.) Like many practices and techniques, I see TDD as fractal. We can get the biggest ROI for a spec test that covers as much as possible.

The test that covers the most of the system is a user-level functional test. That is, a test written at the level of a user’s interaction with the system – i.e. outside of the system – which describes the functional behaviour of the system. The ‘user-level’ part is important, since this level is by definition outside of the system, and covers the entirety of the system being specified.

Its time for an example. The first example I’ll use is specifying Conway’s Game of Life (GoL).

These images represent an (incomplete) specification of GoL, and can be used to specify and validate any implementation of Conway’s Game of Life, from the user’s (i.e. a functional) point of view. These specifications make complete sense to a PO specifying GoL – in fact, this is often how GoL is presented.

These images translate to the following code:

[Test]
public void Test_Barge()
{
    var initialGrid = new char[,]
    {
        {'.', '.', '.', '.', '.', '.'},
        {'.', '.', '*', '.', '.', '.'},
        {'.', '*', '.', '*', '.', '.'},
        {'.', '.', '*', '.', '*', '.'},
        {'.', '.', '.', '*', '.', '.'},
        {'.', '.', '.', '.', '.', '.'},
    };

    Game.PrintGrid(initialGrid);

    var game = CreateGame(initialGrid);
    game.Tick();
    char[,] generation = game.Grid;

    Assert.That(generation, Is.EqualTo(initialGrid));
}

[Test]
public void Test_Blinker()
{
    var initialGrid = new char[,]
    {
        {'.', '.', '.'},
        {'*', '*', '*'},
        {'.', '.', '.'},
    };

    var expectedGeneration1 = new char[,]
    {
        {'.', '*', '.'},
        {'.', '*', '.'},
        {'.', '*', '.'},
    };

    var expectedGeneration2 = new char[,]
    {
        {'.', '.', '.'},
        {'*', '*', '*'},
        {'.', '.', '.'},
    };

    var game = CreateGame(initialGrid);
    Game.PrintGrid(initialGrid);
    game.Tick();

    char[,] actualGeneration = game.Grid;
    Assert.That(actualGeneration, Is.EqualTo(expectedGeneration1));

    game.Tick();
    actualGeneration = game.Grid;
    Assert.That(actualGeneration, Is.EqualTo(expectedGeneration2));
}

[Test]
public void Test_Glider()
{
    var initialGrid = new char[,]
    {
        {'.', '*', '.'},
        {'.', '.', '*'},
        {'*', '*', '*'},
    };

    var expectedGeneration1 = new char[,]
    {
        {'.', '.', '.'},
        {'*', '.', '*'},
        {'.', '*', '*'},
        {'.', '*', '.'},
    };

    var expectedGeneration2 = new char[,]
    {
        {'.', '.', '.'},
        {'.', '.', '*'},
        {'*', '.', '*'},
        {'.', '*', '*'},
    };

    var game = CreateGame(initialGrid);
    Game.PrintGrid(initialGrid);

    game.Tick();
    char[,] actualGeneration = game.Grid;

    Assert.That(actualGeneration, Is.EqualTo(expectedGeneration1));

    game.Tick();
    actualGeneration = game.Grid;
    Assert.That(actualGeneration, Is.EqualTo(expectedGeneration2));
}

All I’ve done to make the visual specification above executable is transcode it into a programming language – C# in this case. Note that the tests do not influence or mandate anything of the implementation, besides the data structures used for input and output.

I’d like to introduce another example, this one a little more complicated and real-world. I’m currently developing a book discovery and lending app, called Lend2Me, the source code for which is available on Github. The app is built using Event Sourcing (and thus Domain-Driven Design and Command-Query Responsibility Segregation). The only tests in this codebase are user-level functional tests. Because I’m using DDD, the spec tests are written in the Ubiquitous Language of the domain, and actually describe the domain, and not a system implementation. Since user interaction with the system is in the form of Command-Result and Query-Result pairs, these are what are used to specify the system. Spec tests for a Command generally take the following form:

GIVEN:  a sequence of previously handled Commands
WHEN:   a particular Command is issued
THEN:   a particular result is returned

In addition to this basic functionality, I also want to capture the domain events I expect to be persisted (to the event store), which is still a domain-level concern, and of interest to the PO. Including event persistence, the tests, in general, look like:

GIVEN:  a sequence of previously handled Commands
WHEN:   a particular Command is issued
THEN:   a particular result is returned
And:    a particular sequence of events is persisted

A concrete example from Lend2Me:

User Story: AddBookToLibrary – As a User I want to Add Books to my Library so that my Connections can see what Books I own.
Scenario: AddingPreviouslyRemovedBookToLibraryShouldSucceed

GIVEN:  Joshua is a Registered User AND Joshua has Added and Removed Oliver Twist from his Library
WHEN:   Joshua Adds Oliver to his Library
THEN:   Oliver Twist is Added to Joshua's Library

This spec test is written at the domain level, in the Ubiquitous Language. The code for this test is:

[Test]
public void AddingPreviouslyRemovedBookToLibraryShouldSucceed()
{
    RegisterUser joshuaRegisters = new RegisterUser(processId, user1Id, 1, "Joshua Lewis", "Email address");
    AddBookToLibrary joshuaAddsOliverTwistToLibrary = new AddBookToLibrary(processId, user1Id, user1Id, title, author, isbnnumber);
    RemoveBookFromLibrary joshuaRemovesOliverTwistFromLibrary = new RemoveBookFromLibrary(processId, user1Id, user1Id, title, author, isbnnumber);

    UserRegistered joshuaRegistered = new UserRegistered(processId, user1Id, 1, joshuaRegisters.UserName, joshuaRegisters.PrimaryEmail);
    BookAddedToLibrary oliverTwistAddedToJoshuasLibrary = new BookAddedToLibrary(processId, user1Id, title, author, isbnnumber);
    BookRemovedFromLibrary oliverTwistRemovedFromJoshuaLibrary = new BookRemovedFromLibrary(processId, user1Id, title, author, isbnnumber);

    Given(joshuaRegisters, joshuaAddsOliverTwistToLibrary, joshuaRemovesOliverTwistFromLibrary);
    When(joshuaAddsOliverTwistToLibrary);
    Then(succeed);
    AndEventsSavedForAggregate(user1Id, joshuaRegistered, oliverTwistAddedToJoshuasLibrary, oliverTwistRemovedFromJoshuaLibrary, oliverTwistAddedToJoshuasLibrary);
}

The definitions of the Commands in this code (e.g. RegisterUser) would be specified as part of the domain:

public class RegisterUser : AuthenticatedCommand
{
    public long AuthUserId { get; set; }
    public string UserName { get; set; }
    public string PrimaryEmail { get; set; }

    public RegisterUser(Guid processId, Guid newUserId, long authUserId, string userName, string primaryEmail)
    : base(processId, newUserId, newUserId)
    {
        AuthUserId = authUserId;
        UserName = userName;
        PrimaryEmail = primaryEmail;
    }

}

Again, the executable test is just a transcoding of the Ubiquitous Language spec test, into C#. Note how closely the code follows the Ubiquitous Language used in the natural language specification. Again, the test does not make any assumptions about the implementation of the system. It would be very easy, for example, to change the test so that instead of calling CommandHandlers and QueryHandlers directly, HTTP requests are issued.

It could be argued that the design of this system makes it very easy to write user-level functional spec tets. However these patterns can be used to specify any system any system, since it is possible to automatically interact with any system through the same interface a user uses. Similarly, it is possible to automatically assert any state in any boundary-interfacing system, e.g. databases, 3rd party webservices, message busses etc. It may not be easy or cheap or robust, but it is possible. For example, a spec test could be:

GIVEN:  Pre-conditions
WHEN:   This HTTP request is issued to the system
THEN:   Return this response
AND:    This HTTP request was made to this endpoint
AND:    This data is in the database
AND:    This message was put onto this message bus

This is how I do TDD: the spec tests are user-level functional tests. If its difficult, expensive or brittle to use the same interface the user uses, e.g. GUIs, then test as close to the system boundary, i.e. as close to the user, as you can.

For interest’s sake, all the tests in the Lend2Me codebase hit a live PostgreSQL database, and an embedded eventstore client. There is no mocking anywhere. All interactions and assertions are outside the system.

software development

Why I do TDD

People give many reasons for why they do Test-Driven Development (TDD). The benefits I get from TDD, and thus my reasons for practicing it, are given below, in order of most important to least important. The order is based on whether the benefit is available without test-first, and the ease with which the benefit is gained without a test-first approach.

1. Executable specification

The first step in building the correct thing is knowing what the correct thing is (or what it does). I find it very difficult to be confident that I’m building the correct thing without TDD. If we can specify the expectations of our system so unambiguously that a computer can understand and execute that specification, there’s no room for ambiguity or misinterpretation of what we expect from the system. TDD tests are executable specifications. This is pretty much impossible without test-first.

I also believe Specification by Example, which is necessary for TDD, is a great way to get ‘Product Owners’ to think about that they want (without having to first build software).

2. Living documentation

The problem with most other forms of documentation is there is no guarantee that it is current. Its easy to change the code without changing the natural-language documentation, including comments. There’s no way to force the documentation to be current. However, with TDD tests, as long as the tests are passing, and assuming the tests are good and valid, the tests act as perfectly up-to-date documentation for and of the code, (from an intent and expectation point of view, not necessarily from an implementation PoV). The documenation is thus considered ‘living’. This is possible with a test-after approach but is harder than with test-first.

3. Focus

TDD done properly, i.e. true Red-Green-Refactor, forces me to focus on one area of work at a time, and not to worry about the future. The future may be as soon as the next test scenario or case, but I don’t worry about that. The system gets built one failing test at a time. I don’t care about the other tests, I’ll get there in good time. This is possible without test-first, but harder and nigh-impossible for me.

4. As a design tool

This is the usual primary benefit touted for TDD. My understanding of this thinking is thus: in general, well-designed systems are decoupled. Decoupling means its possible to introduce a seam for testing components in isolation. Therefore, in general, well-designed systems are testable. Since TDD forces me to design components that are testable, inherently I’m forced to design decoupled components, which are in general considered a ‘better design’. This is low down on my list because this is relatively easy to do without a test-first approach, and even without tests at all. I’m pretty confident I can build what I (and I think others) consider to be well-designed systems without going test-first.

5. Suite of automated regression tests

Another popular selling point for TDD is that the tests provide a suite of automated regression tests. This is absolutely true. However I consider this a very pleasant by-product of doing TDD. A full suite of automated regression tests is very possible with a test-after approach. Another concern I have with this selling point is that TDD is not about testing, and touting regression tests as a benefit muddies this point.

Why do you do TDD? Do you disagree with my reasons or my order? Let me know in the comments!

Uncategorized

Applying TDD in principle if not in practice

My previous post was about describing the essence of TDD as a specification and direction tool, using non-technical examples. A quick recap: the point of TDD is to know where you going, and how to know when you’ve achieved your goal.

Several people have told me that TDD is all well and good, but it doesn’t apply to them, for various reasons. One chap told me he can’t to TDD because he’s a ‘SQL developer’ (his words). A friend told me he works on a vendor system, which is configured with XSLT files, and there is no tooling to support TDD. Its all very well to use whatever flavour of xUnit you want, but what about those who don’t have frameworks? What can the SQL, ASP and XSLT progammers and configurers out there do?

This is why it is important to understand the principle of TDD, and not only its practice (and why I think most demonstrations of TDD that I’ve seen focus too much on the practice).

As long as you have some way to set up a desired outcome, and can compare your actual outcome to your desired, you can perform TDD. You may not get some side benefits like automated regression testing and test coverage, but as mentioned in my previous post, those aren’t primary benefits. You also might not get the benefits of a decoupled design, since this may not make sense in your application.

You can still convey your intent. You will still have direction and focus. You will still be able to answer the question ‘am I done?’

A nice example of not getting the primary benefit of a decoupled design is the case of TDD in the SQL context. The team in which I’m currently working is still strongly focused on stored procedure and dataset-based programming. Therefore I’ve done a fair amount of thinking and playing in the TDD for SQL space. I’ve come up with a framework based on nUnit and TSQLUnit, but that’s another point.

I’ve used this framework to do TDD for stored procedures. One major shortcoming of T-SQL as a language (I feel dirty now) is that there’s no concept of abstraction. There is no way to break dependencies. If stored proceudre 1 (SP1) depends on user-defined function a (UDFa), there is no way I can test SP1 in isolation to UDFa. I therefore don’t get the benefit of a decoupled design, but I can still express the intent of my stored procedure, and I can prove when I’ve satisfied the requirment.

This is a clear case of the tooling failing the technique, but the technique can still be applied in principle. The same should be applicable in many other instances where there either isn’t tooling for the product (e.g. a vendor’s product which is configured).

Again, as long as you can create an expectation, and compare your actual outcome to that expectation, you can apply TDD in principle. So if you’ve previously had a look at TDD and wanted to apply it, but felt a lack of tooling for your particular case, think about how you can apply it in principle.

Uncategorized

Test-Driven Development – Part 1

I’ve found Test-Driven Development (TDD) to be pretty much revolutionary in my own software development efforts. Yet I find it is often very poorly explained, and explanations focus too much on the technical aspects of the technique, instead of the ‘philosophy’ or principle behind it. I place a lot more value on principles and philosophies over practices, since often practices can’t be implemented exactly as described, whereas there can still be a takeaway from the philosophy.

In my opinion, TDD is unfortunately named, since it confuses people. The intent of TDD really has nothing to do with testing. Testing is merely the mechanism through which one gains the primary benefits.

My primary TDD mantra is ‘TDD’s not about testing, its about specification’. In my interpretation, TDD has 2 primary benefits:

  1. It forces you to specify the problem you’re trying to solve very narrowly, and very explicitly.
  2. It promotes a loosely-coupled, modular design, since testable designs are generally loosely-coupled by nature.

Most people will state that the aim of TDD is a large proportion of test coverage and automated regression tests. This is not true. Both of these goals are readily achievable without doing test-first TDD; i.e. they can be easily achieved by writing tests after writing code. Test coverage and automated regression tests are very nice by-products of TDD (when its done correctly), but to my mind they are certainly not the primary benefit nor aim of TDD.

I’d like to explain the first of these primary benefits of  TDD with a non-software related, everyday (somewhat contrived) example. (I’ll need to think of another contrived example to explain the second.) The reason for a non-software related example is I want you to be able to explain the concept to a non-technical person, like your mother or your CEO (or maybe even your team lead 😛 ).

Imagine that I get a crazy idea in my head that I want to become more ‘eco friendly’ in my day-to-day life, and contribute less to global warming. My statement of intent, or vision may be something like ‘I want to make a personal contribution to the lessening of global warming.’ This statement makes one warm and fuzzy, but on its own, its quite difficult to implement. I call this my strategy.

Good management textbooks and courses will tell you one has to implement strategy through strategic objectives. So in order to implement my strategy of being more eco friendly, I come up with the ‘strategic objective’ of reducing my daily carbon emissions.

After some thought about this objective being a little ‘hand-wavy’ and hard to measure, I can refine it by phrasing it as ‘I will decrease my daily commute petrol usage by 10% by the end of this month’. The objective now is measurable, and has a concrete time frame. I have a concrete goal towards which I can work, and measure daily progress (my car has a trip computer which indicates current and average fuel consumption). I can easily see whether I have met (or exceeded) this goal next time I fill up my car.

I have progressed from the warm-and-fuzzy but somewhat hand-wavy statement of ‘I want to do my bit to decrease global warming’ to something tangible to which I can work towards. I know when I have achieved my goal, without a shadow of a doubt, and I can prove it. I can now brag to my friends about how wonderful I am, and maybe I can even get a tax break or something.

Now that the example is concluded, I’d like to point out some things about strategic objectives that I learned at university. Strategic objectives (SOs) must be phrased so that they are measurable, and have a time frame. It says only what must be achieved, but not how. The fact that its measurable means that you know when its done.  You don’t know how to do it (that comes later, at the tactical and operations levels), but you know when its achieved.

If an SO is not measurable, it is considered bad and must be rephrased and rethough. This is akin to designing the SO and implicitly the fulfilment of the SO.

So, an SO is a statement of intent, its a description of a requirement. In x months’ time, we must have sold y units. The fact that it is measurable means  you know when you’re done, but you can also manage it, analyse it, report on it etc. If its not measurable, rephrase it.

If an SO is not a strong enough statement of intent,will those members at the tactical and operations levels  – who don’t operate at all at the strategic level – be able to interpret it and implement it?

If an SO is not measurable, how can you prove to your board that the strategy has been fulfilled?

The testing aspect of this example is simply asking oneself: ‘its now x months down the line, have I sold y units? Its a yes or no answer. Its comparing the actual outcome to the desired outcome.

This example is not test oriented at all. We don’t create strategic objectives in order to run a somewhat trivial test at the end of the prescribed period. The SO exists as a direction, as a statement of intent, as a specification of the where the business needs to be at some point. Sure the measurment aspect is a big part of the SO, but we don’t write SOs for the sake of measuring them.

TDD is no different. The point of TDD is not to write tests. TDD uses the test as a mechanism for feedback. Are we done yet? You can know if you’re done only you know where you’re supposed to be. I don’t know if I’ve made a meaningful contribution to decreasing my emissions until I’m at the petrol pump. A company doesn’t know if its fulfiled its strategy until it can prove it sold x units by a certain date.

TDD forces you to make that end-goal explicit, and to start with that end-goal. Businesses don’t write down strategic sales targets after they’ve made the sales. They start with the target, then put measures in place to achieve that target. TDD in software development is the same.

Start with your end-goal, stated in the form of tests, usually unit tests. You then know where you going. You can then start to decide how you’re going to get there.

To use another example, developing without TDD is like driving around with a sat-nav device, ending up somewhere, and post-haste setting that point as your destination.

Unfortunately, the examples I’ve discussed here don’t really explain the second primary goal of TDD, a loosely-coupled design. I will try think of an example for this benefit and blog it.

Some more notes:

1: As mentioned, businesses start with strategy and strategic objectives, and implement through tactical and operational implementations. They don’t run for a couple of months then retrofit their strategy to where they land up. One can state that ‘its more important to know where you’re going, than how to get there’.

One might also be able to state that ‘its more important to know where you’re going, than actually getting there’. This statement arises from the following: according to agile principles, one delivers the highest value tasks first. Since TDD tests are written before the code to satisfy them, one might argue that the test is more important than the code written to pass the test.

This can be reinforced by considering software product maintenance. The majority of these efforts is spent in understanding the code written by other developers. TDD tests can be a shortcut to understanding the intent of the code.

Also, as businesses changed, an application’s code is likely to change too (as should the TDD tests). Essentially, the intent of the code (conveyed by the test) is likely to outlive the code itself.

2: Behaviour-Driven Design (BDD) is an evolution of TDD, which basically has the notion of ‘executable specifications’ as its holy grail. Instead of developers having to interpret the end-goal presented to them in some type of specification and translating that into code, the customer of the software (business) should be able to codify their specifications in a language as close to their own natural language which can be executed as code.

3: Good TDD tests should be statements of intent. This is often what is meant as ‘tests documenting the code’.