TITLE: Thinking About Testing and Software Engineering
AUTHOR: Eugene Wallingford
DATE: July 03, 2009  8:31 AM
DESC: 
-----
BODY:
I've been buried in a big project on campus for the
last few months.  Yesterday, we delivered our report
to the president.  Ah, time to breathe, heading into
a holiday weekend!  Of course, next week I'll get
back to my regular work.  Department stuff.  Cleaning
my desk.  And thinking about
<A HREF="http://www.cs.uni.edu/~wallingf/blog/archives/monthly/2009-03.html#e2009-03-31T06_43_02.htm">
   teaching software engineering</A>
this fall.

A bit of side reading found via my Twitter friends
has me thinking about testing, and the role it will
play in the course.  In the old-style software
engineering course, testing is a "stage" in the
"process", which betrays a waterfall view of the
world even when the instructor and textbook say that
they encourage iterative development.  But testing
usually doesn't get much attention in such courses,
maybe one chapter that describes the theory of
testing and a few of the kinds of testing we need
to do.

It seems to me that testing can take a bigger place
in the course, if only because it exemplifies the
sort of empiricism that we should all engage in as
software developers.  When we test, we run experiments
to gather evidence that our program works as
specified.  We should adopt a similar mindset about
how we build our programs.  How do we know that
our design is a good one?  Or that our team is
functioning well?  Or that we are investing enough
time and energy in writing tests and refactoring
our code?

That's one reason I like Joakim Karlsson's post
about the
<A HREF="http://jkarlsson.com/blog/2009/06/24/the-locality-of-code-changes/">
   principle of locality in code changes</A>.
There may be ways that he can improve his analysis,
but the most important thing about this post is that
he analyzed code at all.  He had a question about
how code edits work, so he wrote a program to ask
subversion repositories for the answer.  That's so
much better than assuming that his own hypothesis
was correct, or that conventional wisdom was.

In regard to the testing process itself, Michael
Feathers wrote a piece on
<A HREF="http://michaelfeathers.typepad.com/michael_feathers_blog/2009/06/canalizing-design.html">
   "canalizing" design</A>
that points out a flaw in how we usually test our
code.  We write tests that are independent of one
another in principle but that our test engines
always run in the same order.  This inadvertent
weakness of sequential code creates an opportunity
for programmers to write code that takes advantage
of the implicit relationship between tests.  But
it's not really an advantage at all, because we
then have dependencies in our code that we may not
be aware of and which should not exist at all.
Feathers suggests putting the tests in a set data
structure and executing them them from there.  At
least then the code makes explicit that there is
no implied order to the tests, which reminds the
programmers who modify the code later that they
should not depend on the order of test execution.

(I also like this idea for its suggestion that
programs can and other should be dynamic structures,
not dead sequences of text.  Using a set of tests
also moves us a step closer to making our code
work well in a parallel environment.  Explicit
and implicit sequencing in programs makes it hard
to employ the full power of multicore systems,
and we need to re-think how we structure our
programs if we want to break away from purely
sequential machines.  The languages guy in me sees
some interesting applications of this idea in how
write our compilers.)

Finally, I enjoyed reading
<A HREF="http://gojko.net/2009/02/27/thought-provoking-tdd-exercise-at-the-software-craftsmanship-conference/">
   Gojko Adzic's description</A>
of
<A HREF="http://peripateticaxiom.blogspot.com/">
   Keith Braithwaite</A>'s
"TDD as if you mean it" exercise.  Like the
<A HREF="http://www.cs.uni.edu/~wallingf/blog/archives/monthly/2007-11.html#e2007-11-10T16_21_19.htm">
   programming challenges</A>
I have described, it asks developers to take an
idea to its extreme to break out of habits and
to learn just how the idea feels and what it can
give.  Using tests to drive how the writing of
code is more different from what most of us do
than we usually realize.  This exercise can help
you to see just how different -- if you have an
exercise leader like Keith to keep you honest.

However, I disagree with something Keith said
in response to a comment about the relationship
between TDD and functional programming:

<BLOCKQUOTE><EM>
I'm firmly convinced that sincere TDD leads one
towards a functional style.
</EM></BLOCKQUOTE>

TDD will drive you to the style whose language
you think.

There will be functional components to your solution
to support the tests, and some good OOP has a
functional feel.  But in my experience you can end
up with very nice objects in an object-oriented
program as a result of faithfully-executed TDD.

Another of Braithwaite's comments saved the day,
though.  He credits Allan Watts for this line
that captures his intent in designing exercises
like this:

<BLOCKQUOTE><EM>
I came not as a salesman but as an entertainer.
I want you to enjoy these ideas because I enjoy them.
</EM></BLOCKQUOTE>

Love this!  He has a scholar's heart.

There is a lot more to testing that unit tests or
regression testing.  Finding ways to introduce
students to the full set of ideas while also
giving them a visceral sense of testing in the
trenches is a challenge.  I have to teach enough
to prepare a general audience and also prepare
students who will go on to take our follow-up
course,
<A HREF="http://www.cs.uni.edu/courses.php#175">
   Software Testing</A>.
That's a course that undergraduates at most schools
don't have the opportunity to take, a strong point
of our program.  But that course can't be an
excuse not to do testing well in the software
engineering course.  It's not a backstop; it's
new ballgame.
-----