January 31st, 2016

TDD, continuous delivery and inductive thinking

Approaches to using TDD in your development process without ruining your code or velocity.

— Matthew Fellows —

So TDD isn’t dead, or maybe it is. Or perhaps we are just doing it wrong.

You’ve probably read DHH’s rant on the subject, and for the most part I agree with what he is saying: it was never meant to be taken as literally as the industry has, and following dogmatic TDD will result in terrible design. I would point that at least since the Bible, people have been taking metaphors more literally than they should – so this shouldn’t be a surprise to anyone.

I did however take issue with one of David’s comments, or at least as I understood it to be, advice to look toward frameworks like Capybara for the ‘future’ of testing. The reason this raised my eyebrows is the potential religious effect from above – will people come away from this article thinking ‘I need to browser test everything – it’s the future!’? We’ll address this after quick digression…

The Test Pyramid and Continuous Delivery

The test pyramid is a staple in everyone’s testing diet, if you don’t know what it is take the time to read Martin Fowler’s article on the matter. In essence, it’s the food pyramid applied to testing: Lot’s of hearty unit tests form the base (~80%), some integration tests (~15%) to keep you entertained, and topped off with a small amount of E2E/UI tests for dessert.

If you buy into this thinking, as I do, you might see why I’m afraid of the ‘advice’ to increase the use of browser testing. But why do I care so much, and who the hell am I to question DHH of all people (fair call)?

When you add continuous delivery into the mix, some of these problems are pronounced, in particular the use of end-to-end tests. In my own experience, it is much better to have a faster continuous delivery pipeline with less functional tests, than a lengthy pipeline with 100%* coverage. In fact, I would argue that you want your entire continuous delivery pipeline – from build, test to deploy – to take no more than 5 – 10 minutes. The reasons for this are simple:

  • Faster deployment cycles mean smaller change sets will make their way to Production
  • Smaller change sets reduce the risk of Production issues
  • Smaller change sets make it easier to identify the root cause of an issue
  • Smaller change sets increase the ability to rollback in the event of an issue

Browser tests, and to some extent integration tests, tend to slow this process down significantly. Of course, you might argue that you could speed up those tests, using computer power/parallelization etc., and if you know what you’re doing then go for it. But browser tests do come at another cost though: complexity.

In any case, there are no commandments set in stone here.

Deductive vs Inductive thinking

So back to the TDD discussion – is it valuable at all? In my view, yes, in the metaphorical sense of the term. In my opinion however, it only makes sense, once you have a fairly good understanding of what and how you’re going to build something. That is to say that I don’t use TDD to drive design and I certainly don’t start writing unit tests before I write code for anything but the most mundane things.

The reason for this is that development, for me, is as much of a creative endeavor as it is a precise, mathematical and engineering-driven one. If we’re constantly solving problems that have been solved before, then maybe TDD can work – but who the hell is solving problems that have already been solved?

When solving a real, novel problem, inductive reasoning is a better starting point. You get yourself into the problem domain, possibly within an existing code base, and ask a bunch of questions to tease out some plausible paths to a solution; “has this been solved before? Do we have code structures that can support this? What if I did X or Y, can I get to the things I need?”. At this point, you might hack around with some code to validate as you go – you’re not going to write unit tests for these questions.

At a certain point, you’ve explored a number of options and possibly have a half-baked solution that proves a path could be cleared for your problem to be completed. This is the point where you can now start to write the real thing – using deductive reasoning – and where writing tests is going to assist you with writing a more detailed, correct solution.

I find this approach allows me to first see the forest from the trees, and then get buried in the weeds of detail at appropriate times in the development cycle, yet still maintain a healthy level of test coverage across the application.


* Hint: You’re barking up the wrong tree, sorry. You’ll never get to 100%. Is that line coverage? Statement coverage? Branch coverage? Scenario coverage? Correctness…?