TITLE: Thinking About Testing and Software Engineering AUTHOR: Eugene Wallingford DATE: July 03, 2009 8:31 AM DESC: ----- BODY: I've been buried in a big project on campus for the last few months. Yesterday, we delivered our report to the president. Ah, time to breathe, heading into a holiday weekend! Of course, next week I'll get back to my regular work. Department stuff. Cleaning my desk. And thinking about teaching software engineering this fall. A bit of side reading found via my Twitter friends has me thinking about testing, and the role it will play in the course. In the old-style software engineering course, testing is a "stage" in the "process", which betrays a waterfall view of the world even when the instructor and textbook say that they encourage iterative development. But testing usually doesn't get much attention in such courses, maybe one chapter that describes the theory of testing and a few of the kinds of testing we need to do. It seems to me that testing can take a bigger place in the course, if only because it exemplifies the sort of empiricism that we should all engage in as software developers. When we test, we run experiments to gather evidence that our program works as specified. We should adopt a similar mindset about how we build our programs. How do we know that our design is a good one? Or that our team is functioning well? Or that we are investing enough time and energy in writing tests and refactoring our code? That's one reason I like Joakim Karlsson's post about the principle of locality in code changes. There may be ways that he can improve his analysis, but the most important thing about this post is that he analyzed code at all. He had a question about how code edits work, so he wrote a program to ask subversion repositories for the answer. That's so much better than assuming that his own hypothesis was correct, or that conventional wisdom was. In regard to the testing process itself, Michael Feathers wrote a piece on "canalizing" design that points out a flaw in how we usually test our code. We write tests that are independent of one another in principle but that our test engines always run in the same order. This inadvertent weakness of sequential code creates an opportunity for programmers to write code that takes advantage of the implicit relationship between tests. But it's not really an advantage at all, because we then have dependencies in our code that we may not be aware of and which should not exist at all. Feathers suggests putting the tests in a set data structure and executing them them from there. At least then the code makes explicit that there is no implied order to the tests, which reminds the programmers who modify the code later that they should not depend on the order of test execution. (I also like this idea for its suggestion that programs can and other should be dynamic structures, not dead sequences of text. Using a set of tests also moves us a step closer to making our code work well in a parallel environment. Explicit and implicit sequencing in programs makes it hard to employ the full power of multicore systems, and we need to re-think how we structure our programs if we want to break away from purely sequential machines. The languages guy in me sees some interesting applications of this idea in how write our compilers.) Finally, I enjoyed reading Gojko Adzic's description of Keith Braithwaite's "TDD as if you mean it" exercise. Like the programming challenges I have described, it asks developers to take an idea to its extreme to break out of habits and to learn just how the idea feels and what it can give. Using tests to drive how the writing of code is more different from what most of us do than we usually realize. This exercise can help you to see just how different -- if you have an exercise leader like Keith to keep you honest. However, I disagree with something Keith said in response to a comment about the relationship between TDD and functional programming:
I'm firmly convinced that sincere TDD leads one towards a functional style.
TDD will drive you to the style whose language you think. There will be functional components to your solution to support the tests, and some good OOP has a functional feel. But in my experience you can end up with very nice objects in an object-oriented program as a result of faithfully-executed TDD. Another of Braithwaite's comments saved the day, though. He credits Allan Watts for this line that captures his intent in designing exercises like this:
I came not as a salesman but as an entertainer. I want you to enjoy these ideas because I enjoy them.
Love this! He has a scholar's heart. There is a lot more to testing that unit tests or regression testing. Finding ways to introduce students to the full set of ideas while also giving them a visceral sense of testing in the trenches is a challenge. I have to teach enough to prepare a general audience and also prepare students who will go on to take our follow-up course, Software Testing. That's a course that undergraduates at most schools don't have the opportunity to take, a strong point of our program. But that course can't be an excuse not to do testing well in the software engineering course. It's not a backstop; it's new ballgame. -----