TITLE: OOPSLA Day 2: Jim Waldo "On System Design"
AUTHOR: Eugene Wallingford
DATE: October 26, 2006  4:28 PM
DESC: 
-----
BODY:
Last year, OOPSLA introduced another new track called Essays.
This track shared a motivation from another recent introduction,
Onward!, in providing an avenue for people to write and present
important ideas that cannot find a home in the research-oriented
technical program of the usual academic conference.  But not all
advances are in the form of novelty, of results from
narrowly-defined scientific experiments.  Some empirical results
are the fruit of experience, reflection, and writing.  The Essays
track offers an avenue for sharing this sort of learning.  The
author writes an essay in the spirit of
<A HREF="http://en.wikipedia.org/wiki/Montaigne">
   Montaigne</A>
and
<A HREF="http://en.wikipedia.org/wiki/Francis_Bacon">
   Bacon</A>,
using the writing to work out an understanding of his experience.
He then presents the essay to the OOPSLA audience, followed by a
thoughtful response from a knowledge person who has read the
essay and thought about the ideas.  (Essays crossed my path in
a different arena last week, when I blogged a bit on the idea of
<A HREF="http://www.cs.uni.edu/~wallingf/blog/archives/monthly/2006-10.html#e2006-10-19T18_26_51.htm">
   blog as essay</A>.)

Jim Waldo, a distinguished engineer at Sun, presented the first
essay of OOPSLA 2006, titled "On System Design".  He reflected
on his many years as a software developer and lead, trying to
get a handle on what he now believes about the design of software.

What is a "system"?  To Waldo, it is not "just a program" in the
sense he thinks meant by the
<A HREF="http://www.mcs.vuw.ac.nz/comp/Publications/CS-TR-02-9.abs.html">
   postmodern programming</A>
crowd, but a collection of programs.  It exists at many levels
of scale that must be created and related; hence the need for us
to define and manage abstractions.

Software design is a craft, in the classical sense.  Many of
Waldo's engineer friends are appalled at what software engineers
think of as engineering (essentially the application of patterns
to roll out product in a reliable, replicable way) because what
engineers <EM>really</EM> do involves a lot of intuition and craft.

Software design is about technique, not facts.  Learning technique
takes time, because it requires <STRONG>trial and error</STRONG>
and <STRONG>criticism</STRONG>.  It requires patience -- and faith.

Waldo paraphrase Grady Booch as having said that the best benefit
of the Rational toolset was that it gives developers "a way to look
like they were doing something while they had time to think".

The traditional way to learn system design is via apprenticeship.
Waldo usually asks designers he respects who they apprenticed
with.  At first he feared that at least a few would look at him
oddly, not understanding the question.  But he was surprised to
find that every person answered without batting an eye.  They
all not only understood the question but had an immediate
answer.  He was also surprised to hear the same few names over
and over.  This may reflect Waldo moving in particular circles,
or only a small set of master software developers out there!

In recent years, Waldo has despaired of the lack of time,
patience, and faith shown in industry for developing developers.
Is all lost?  No.  In reflecting on this topic and discussing
with readers of his early drafts, Waldo sees hope in two parts
of the software world: open source and extreme programming.

Consider open source.  It has a built-in meritocracy, with
masters at the top of the pyramid, controlling the growth and
design of their systems.  New developers learn from example --
the full source of the system being built.  Developers face
real criticism and have the time and opportunity to learn and
improve.

Consider extreme programming.  Waldo is not a fan of the agile
approaches and doesn't think that the features on which they
are sold are where they offer most.  It isn't the illusion of
short-term cycles or of the incrementalism that grows a big
ball of mud which give him hope.  In reality, the agile
approaches are based in a communal process that builds systems
over time, giving people the time to think and share, mentor
and learn.  Criticism is built into the process.  The system
is an open, growing example.

Waldo concludes that we can't teach system design in a class.
As an adjunct professor, he believes that system design skills
aren't a curriculum <EM>component</EM> but a curricular
<STRONG>outcome</STRONG>.  Brian Marick, the discussant on
Waldo's took a cynical turn:  No one should be allowed to
teach students system design if they haven't been a committer
to a large open-source project.  (Presumably, having had
experience building "real" big systems in another context
would suffice.)  More seriously, Marick suggested that it is
only for recent historical reasons that we would turn to
academia to solve the problem of producing software designers.

I've long been a proponent of apprenticeship as a way of
learning to program, but Waldo is right that doing this as
a part of the typical university structure is hard, if not
impossible.  We heard about a
<A HREF="http://www.cs.uni.edu/~wallingf/blog/archives/monthly/2005-10.html#e2005-10-17T21_54_33.htm">
  short-lived attempt to do this</A>
at last year's OOPSLA, but a lot of work remains.  Perhaps
if more people like Waldo, not just the more provocative folks
at OOPSLA, start talking openly about this we might be able to
make some progress.

Bonus reading reference:  Waldo is trained more broadly as a
philosopher, and made perhaps a surprising recommendation for
a great document on the act of designing a new sort of large
system:
<A HREF="http://thomas.loc.gov/home/histdox/fedpapers.html">
   The Federalist Papers</A>.
This recommendation is a beautiful example of the value in
a broad education.  The Federalist Papers are often taught
in American political science courses, but from a different
perspective.  A computer scientist or other thinker about the
design of systems can open a whole new vista on this sort of
document.  Here's a neat idea: a system design course team
taught by a computer scientist and a political scientist,
with The Federalist Papers as a major reading!

Now, how to make system design as a skill an inextricable
element of all our courses, so that an outcome of our major
is that students know the technique?  ("Know how", not "know
that".)
-----