software development

Pull promotes quality

In this post I contend that pull systems enable and promote working at a higher level of quality than is allowed by working according to a push system.

In traditional software development scenarios, project plans and timelines are imposed on development work. In addition, work is pre-assigned to specific developers. It is fact that any type of worker can only do so much work per hour, and there are so many hours in a day. When timelines (and a set number of human resources) are imposed on a creative, innovative process like software development, either the quality of the work decreases, or the amount of work decreases (if quality is kept constant).

If you give Michaelangelo 10 days to paint the Sistine Chapel, he will either paint only a corner of the ceiling to the desired quality (which is why you hired him in the first place), or he will paint the entire ceiling but it will look like rubbish.

This is a push system. The deadline is pushed onto the work. Work items are pushed onto specific developers.

However, if you don’t impose a deadline, and you don’t pre-allocate tasks to developers, the developer has the time available to him to ‘do a proper job’, i.e. complete the task to the level of quality required by the system. He is not under pressure to cut corners in either scope or quality. He is also not inclined to optimise to the incentive.

The challenge of these systems then becomes managing the uncertainties around such a project, like ‘when will it be complete’, ‘how much will it cost etc’. But in general, the quality of the finished product should be higher.

software development

TDD is a pull system

In my previous post, I described how pull systems work. I made the statement that work is only done in a stage in a pipeline when there is a request from the downstream stage. I.e. I only develop some code when QA has asked me for some artifacts to test. I also made the statement that if there is a request from a downstream stage, and no work needs to be done to fulfil that request, then the ‘conversation’ should ‘return’ immediately.

I contend that test-driven development (TDD) is a pull system. I make a request, and I then fulfill that request. I write a test as a specification of what I want to achieve, and I then write code to fulfill that request.

According to TDD best practice, I write code only when I have a failing test. This is indicative of a pull system. I write the code only when I need to. This need is represented by a failing test.

TDD ensures that you don’t waste time writing unneeded code. All code written must be justified by the ‘failing test’ rule. Otherwise it is considered superfluous, and does not contribute to satisfying the request from the downstream process.

If you write more code after all your tests have passed, you are not adding value and you are being wasteful.

Think of TDD according to this example: instead of producing an engine and then trying to fit it into a car (a set of requirements), rather start off with the car, and have the test ‘does my car go?’ You only need an engine to make a car go. If the car goes without an engine (highly unlikely), you don’t need the engine. Production of the engine can be justified (ie represent value) only when there is an explicit need for an engine. Code is only needed when you have a failing test.

You may refine your test to include economy and performance requirements etc: “My car needs a certain maximum speed. To achieve that, I need an engine which produces 100kW. Does my engine produce 100kW?” If your car needs an engine that produces a maximum of 100kW, it is wasteful to design an engine that produces 120kW. When designing the engine, you have explicit requirements  you need to satisfy. Producing more than what is required is wasteful, and is not ‘pull’. If your tests aren’t failing, don’t write more code.

Similarly, if you are designing a new car that needs an engine that produces 100kW and  you already have one sitting on the shelf, don’t design another one – if you add a test which passes without any new code, don’t write more code.

TDD simply provides a philosophy for specifying your requirements, and a mechanism for evaluating whether you have satisfied those requirements (i.e. does my engine satisfy all my requirements?). It provides a means for simulating requests that the code’s production clients will expect to be satisfied (e.g. drive-train and throttle interfaces). TDD makes those requirements explicit and measurable.

To conclude, TDD is a pull system. When practicing TDD correctly, code is only written when there is a failing test, signifying an explicit need for some work to take place. There should be direct traceability from each line of code to a TDD test, from each TDD test to a user story/MMF/BDD test etc, and so on up the line. The value stream then becomes explicit. Any code which can’t be traced back to a test (requirement) can then be considered waste and should be eliminated. (Obviously there will be some infrastructure code etc).

Some more notes on TDD as a pull system:

– A failing test is usually signified by ‘red’. This red serves as a visual trigger for action: we need to write some code. ‘Red’ is therefore a Kanban.

– The ideas discussed here don’t address the problem of how to arrive at TDD tests. In the car example, the tests are put forth as natural langauge: “Does my car go”?, “Does my engine make at least 100kW?” In reality, it is often difficult to translate user requirements into low-level unit tests required for TDD. Techniques like Doman Driven Design (DDD)’s ubiquitous language and Behaviour-Driven Design (BDD) can help towards this.

If we’re talking about engines, unless we can agree what ‘100kW’ means and how we measure it, we can never be sure that we’re actually satisfying our requirements, and we will fail only when we try put the engine in the car and drive away. At this point, it becomes very costly to try change the engine or the car.

– We are lucky that we can use these self-same tests for automated regression testing.

software development

Kanban Board and Pull

These are my thoughts on Lean systems from a pull perspective, and how Kanban Boards can be used to facilitate a pull instead of a push system in software development. I don’t go into the pros and cons of pull systems in software development.

Imaging you want to order a  new from Toyota. In a push system, Toyota would try anticipate customer demand, and would produce a certain number of cars per period. These cars would then sit in a lot somewhere until they’re sold. This is wasteful. In the ideal, what should happen is that your order gets passed down the line, through successive levels in the business until the order is fulfilled. It should look something like this: The sales person passes on the order to the manufacturing plant;  the order is directed to the appropriate manufacturing line (car model) etc etc until it gets to the person making the wheels, doors etc for the car (including the suppliers).

This can be illustrated by the following conversations:

Me: “Salesperson, I want a 1.6 liter Corolla!”
Saleperson: “Factory, we need a new Corolla for this customer!”
Factory foreman: “Corolla line, we need a car with these specs for this customer!”
Corolla Line Manager: “Engine supplier, we need a new 1.6 liter engine for this car!”

Engine Supplier: “Corolla Line Manager, here’s that engine you asked for!”
Corolla Line Manager: “Factory foreman, here’s that car  you asked for!”
Factory Foreman: “Salesperson, here’s that Corolla you asked for!”
Salesperson: “Customer, here’s your car!”

Each lower level in the process is only asked for something when there is a need. It is obvious that the conversation at a specific level cannot be ‘returned’ until it is satisfied. That is, a car cannot be produced without an engine. Note also the symmetrical nature of this series of  conversations, it starts and ends with the customer.

What does not happen is that there is a pile of engines, gearboxes, doors etc waiting around to be used when an order comes through. (Obviously there may be lead times associated with each part, but this can be handled with Kanban and order points). The process to create each item is triggered or initiated only when there is a firm requirement for that part to be made. Parts are created just-in-time (JIT), only created when they are needed (and delivered to where they are needed).

The same process should be followed in software development. The conversation might be something like this:

Business (our customer): “Deployment team, give me this feature I want!”
Deployment team: “Quality assurance team, give me this feature to deploy!”
QA team: “Development, give me this feature to test!”
Development: “Business analysts, specify this feature for me to develop!”
BAs: “Business, explain this feature to me so I can specify it!”
Business thinks about it then, Business: “BAs, it needs to do this and this and  that!”
BAs specify it then, BAs: “Development, here is your specification to develop off of!”
Development develops the feature then: “QA, here is the feature to test!”
QA test then, QA: “Deployment team, here is the artifact to deploy!”
Deployment deployes then, Deployment: “Business, here is your feature in the live system!”

Each team in the process only does work if there is a request from the downstream team. The downstream team is notified when the work item is ready for their attention. If there is no request from the downstream team, no work happens. If there is a request from a downstream team, and no work needs to be done to fulfil this request, then it is simply moved downstream, and started on its return conversation journey. (If this case seems cryptic, I will explain it more in a later post.)

The term ‘Kanban’ has become invested with the meaning of a ‘visual trigger for action’. A task board becomes a Kanban board only when task cards signify some action, otherwise it serves only the purpose of tracking where in the process a task is. The typical trigger for action is the return path in the conversation above, something like: “QA, here is that feature for you to test!”. This is manifest by a developer moving a task card into the ‘Done’ or ‘Waiting for QA’ column on the task board. This is a visual trigger for someone in the QA team to pull this task into their personal queue. This can further be improved by imposing work-in-progress limits for each resource at each stage in the process.

The following occurred to me while writing this post:

At the moment, I don’t know if the forward conversation should be represented on a Kanban board, and if so, how. Perhaps with a different coloured card. I think it might add value to track this forward conversation, since it could still serve as a trigger for action. As in the case with the car and engine, a QA engineer can test a feature until it has been developed by a developers. Similarly, a developer cannot begin development on a feature until it has been specified by a BA. Perhaps it is necessary to represent this on the Kanban board. I.e. a visual trigger to a BA that he needs to do some work.

This could be achieved by having a ‘reverse’ stream on a Kanban board, representing the forward conversation. Thus, when QA is asked to present a feature to give to the deployment team to deploy, QA should put a card in the ‘waiting for development’ column on the board. This card is then passed upstream in the process until the first stage in the process (business in our case) fulfils it, then starts passing it back downstream. This could be used to discover bottlenecks in the upstream direction. I.e. our developers are sitting idle because the BAs cannot develop specification fast enough.

This can be discovered on a ‘normal’ Kanban board by the ‘to do’ column for development remaining empty for some time. Perhaps then the absence of work items in the ‘to do’ column for any stage indicates a bottleneck in the stage before.