TITLE: Practice and Dogma in Testing
AUTHOR: Eugene Wallingford
DATE: May 18, 2009  8:58 PM
DESC: 
-----
BODY:
<EM>Shh</EM>.

<EM>I have a secret</EM>.

<EM>When I am writing a program, I will on occasion
add a new piece of functionality without writing a
test</EM>.

I am whispering because I have seen the reaction
on the XP mailing list and on a number of blogs
that Kent Beck received to his recent article,
<A HREF="http://www.threeriversinstitute.org/blog/?p=187">
   To Test or Not to Test?
   That's a Good Question</A>.
In this short piece, Kent describes his current
thinking that, like golf, software development may
have "long game" and "short game", which call for
different tools and especially mentalities.  One
of the differences might be whether one is willing
to trade automated testing for some other value,
such as delivering a piece of software sooner.

Note that Kent did <EM>not</EM> say that in the
long game he chooses not to test his code; he
simply tested manually.  He also didn't say that
he plans never to write the automated tests he
needs later; he said he would write them later,
either when he has more time or, perhaps, when he
has learned enough to turn 8 hours of writing a
test into something much shorter.

Many peoples' public reactions to Kent's admission
have been along these lines:  "We test you to make
this decision, Kent, but we don't trust everyone
else.  And by saying this is okay, you will
contribute to the delinquency of many programmers."
Now you know why I need to whisper...  I am
certainly not in the handful of programmers so
good that these folks would be willing to excuse
my apostasy.  Kent himself is taking a lot of
abuse for it.

I have to admit that Kent's argument doesn't seem
that big a deal to me.  I may not agree with
everything he says in his article, but at its core
he is claiming only that there is a particular
context in which programmers might choose to use
their judgment and not write tests before or
immediately after writing some code.  Shocking:
A programmer should use his or her judgment in
the course of acting professionally.  Where is
the surprise?

One of the things I like about Kent's piece is
that he helps us to think about when it might be
useful to break a particular rule.  I know that
I'll be breaking rules occasionally, but I often
worry that I am surrendering to laziness or
sloppiness.  Kent is describing a candidate
pattern:  In this context, with these goals, you
are justified in breaking this rule consciously.
We are balancing forces, as we do all the time
when building anything.  We might disagree with
the pattern he proposes, but I don't understand
why developers would attack the very notion of
making a trade-off that results in breaking a
rule.

In practice, I often play a little loose with
the rules of XP.  There are a variety of reasons
that lead me to do so.  Sometimes I pay for not
writing a test, and when I do I reflect on what
about the situation made the omission so dangerous.
If the only answer I can offer is "You <B>must</B>
write the test, always.", then I worry that I have
moved from behaving like a professional to
behaving like a zealot.  I suspect that a lot of
developers make similar trade-offs.

I do appreciate the difficulty this raises for
those of us who teach XP, whether at universities
or in industry.  If we teach a set of principles
as valuable, what happens to our students'
confidence in the principles when we admit that
we don't follow the rules slavishly?  Well, I
hope that my students are learning to think, and
that they realize any principle or rule is
subject to our professional judgment in any given
circumstance.

Of course, in the context of a course, I often
ask students to follow the rules "slavishly",
especially when the principles in question require
a substantial change in how they think and behave.
TDD is an example, as is pair programming.  More
broadly, this idea applies when we teach OOP or
functional programming or any other new practice.
(No assignment statements or sequences until Week
10 of Programming Languages!)  Often, the best way
to learn a new practice is to live it for a while.
You understand it better then than you can from
any description, especially how it can transform
the way you think.  You can use this understanding
later when it comes to apply your judgment about
potential trade-offs.

Even still, I know that, no matter how much an
instructor encourages a new practice and strives
to get students to live inside it for a while,
some students simply won't do it.  Some want to
but struggle changing their habits.  I feel for
them.  Others willfully choose not to try the
something new and deny themselves the opportunity
to grow.  I feel for them, too, but in a different
way.

Once students have had a chance to learn a set of
principles and to practice them for a while, I
love to talk with them about choices, judgment,
and trade-offs.  They are capable of having a
meaningful discussion then.

It's important to remember that Kent is not teaching
novices.  His primary audience is professional
programmers, with whom he ought to be able to
have a coherent conversation about choices, judgment,
and trade-offs.  Fortunately, a few folks on the
XP list have entertained the "long game versus
short game" claim and related their own experiences
making these kind of decisions on a daily basis.

If we in the agile world rely on unthinking
adherence to rules, then we are guilty of
proselytizing, not educating.  Lots of folks who
don't buy the agile approaches love when they see
examples of this rigidity.  It gives them evidence
to support their tenuous position about the whole
community.  From all of my full-time years in the
classroom, I have learned that perhaps the most
valuable asset I can possess is my students'
trust in my goals and attitudes.  Without that,
little I do is likely to have any positive effect
on them.

Kent's article has brought to the surfaced another
choice agilistas face most every day: the choice
between dogma and judgment.  We tend to lose people
when we opt for unthinking adherence to a rule or
a practice.  Besides, dogmatic adherence is rarely
the best path to getting better every day at what
we do, which is, I think one of the principles
that motivate the agile methods.
-----