TITLE: Kent Beck on Developer Testing and System Health AUTHOR: Eugene Wallingford DATE: December 15, 2004 10:46 AM DESC: Kent reminds us that a system can be bug-free by not healthy -- and that healthy systems are what we really want. That includes developers. ----- BODY: Kent Beck While my students are taking their object-oriented programming final exam, I'm listening to Kent Beck's talk on developer testing at the recent Developer Testing Forum held at PARC. The big theme of the talk is that developer testing is a way that an individual programmer can take control of his own accountability -- whether or not he adopts other agile practices, whether or not anyone else in his organization goes agile. Kent's been talking a lot about accountability lately, and I think it captures one of those very human values that underlie the agile methods, one of the non-techie reasons that I am drawn to the agile community. My favorite idea in the talk is one that Kent introduces right away: the distinction between quality (an instantaneous measure of a system's goodness) and health (a measure over time). I'm not sure I like the use of "quality" to mean an instantaneous measure, but I love the distinction. Many developers, including students, mistake "there are no bugs in my code" with "this is good code". In one sense, I suppose this is true. The code runs. It performs as desired. That is a Good Thing. But in another sense the implication is just wrong. Can people extend the program? Change it? Use it? Port it? Kent's turns to his health metaphor to explain. In my words: A person may have a good heart rate and normal blood pressure, but if he can't walk around the block without keeling over then he's probably not all that healthy. Refactoring mercilessly is a practice that recognizes the importance of the distinction between quality and health. Just because my code passes all the tests does not mean that the code is healthy. At least it's not as healthy as it can be. Refactoring is an exercise regimen for my system. It seeks to improve the long-term health of my program by drawing on its strength at the moment. Rigorous developer testing also recognizes the importance of this distinction. Having tests means that I can extend and change my code -- work akin to walking around the block -- with some confidence that I won't break the system. And, if I do, the system helps me recover from any errors I introduce. The tests are the immune system of my program! I really like this "system health" metaphor and think it extends quite nicely to many of the principles and practices of the agile methods. Continuous feedback and sustainable pace spring to mind immediately. Others, such as continuous integration and pair programming, require some thought. That will make for a future blog entry! Oh, and one other thing I liked from the talk... In the Q-n-A session after the talk, someone asked Kent how we know that our tests are correct. They are, after all, software, too. Kent said that you can't ever know for certain, but you can be more or less confident, depending on how thoroughly you test and refactor. It's like cross-checking sums in arithmetic or accounting. Over the long run, the chance that you make the offsetting mistakes in the code and the tests He made an analogy to mathematical proof, saying something to this effect:
There's no such thing as proof in software. Proof of correctness isn't proof of correctness; it's proof of equivalence. "Here is one expression of what I'm trying to compute, and here's another expression of what I'm trying to compute, and they match." That's what you do with a proof of correctness. Tests and code are the same way. You're saying these two expressions are equivalent in some sense. ... That means my confidence in the answer is much, much higher.
Well said. Scientific reasoning, even in the artificial world of mathematics, is about confidence, not certainty; evidence, not proof. We should not expect more of the messy, human enterprise of building software. (I have written about tests and proofs before, and of course there is the wonderful Donald Knuth quote in Alan Kay's talks at OOPSLA.) -----