Knowing and Doing

In recent years, my favorite conference, OOPSLA, has been suffering a strange identity crisis. When the conference began in the mid-1980s, object-oriented programming was a relatively new idea, known to some academics but unheard of by most in industry. By the early 2000s, it had gone mainstream. Most developers understands OOP now, or think they do. Our languages and tools make it seem second nature. We teach it to freshmen in many universities. Who needs a big conference devoted to objects?

As I've written before, say, here, this reflects a fundamental misunderstanding about OOPSLA has always been about. It's a natural misunderstanding, given the name and the marketing hype around it through the 1990s, but a misunderstanding nonetheless. It's hard to unbrand a well-known conference and re-brand it once its existing brand loses its purpose or zip. So OOPSLA struggles a bit. I hear that next year it will become one attraction under the umbrella of a new event, "SPLASH". (More on that later!)

In the last few years, people have begun to notice something similar going on with a younger spinoff from OOPSLA. Agile software development has started to become mainstream. It is over the initial hump of awareness and acceptance. Most everyone knows what 'agile' is, or thinks he does. Its practices have begun to seep into a large number of software houses, whether from Scrum or XP. Our languages and tools now make it possible to work in shorter iterations with continuous feedback and more confidence. We teach it in universities, sometimes even to freshmen. Agile has become old school.

Maybe I'm overstating things a bit, but the buzz has certainly died off. Now we are more likely to hear grumbling. I don't know that many people are asking whether we need conferences devoted to agile approaches yet. But I am starting to read articles that second guess the "movement" and to see a lot of revisionist history. Perhaps this is what happens when the excitement of newness fades into the blandness of same-old, same-old. The excitement passes, and people begin to pay attention more to the chinks in the armor than the strengths that gave the idea birth.

Let's not forget what made agile development and especially XP so galvanizing. William Caputo hasn't forgotten:

But why Agile sells isn't why I liked it. I liked it *because* it put the programmer at the center of the effort of programming (crazy notion that), I didn't need the manifesto to tell me that I had to find ways to make the person paying the money delighted with my efforts, what I needed was a way to tell them that I would gladly do so, if they would just let me.

(The asterisks are his. The bold is mine.)

This is what made XP and the other agile approaches so very important. Focusing on the buyers' needs is easy to do. They pay for software and so have an opportunity to affect strongly the nature of software development. What was missing pre-agile was attention to the developer. I still prefer the term "programmer", with all it implies. It was easy to find ways in which programmers were marginalized, buried beneath processes, documentation, and tools that seemed primarily to serve purposes other than to trust and help programmers to make great software.

The agile movement helped to change that, to shift the perspective from inhuman processes that expected the worst from programmers to practices that celebrate programmers and their love for making software. That some people are now even talking about restating the Agile Manifesto's values toward maximizing shareholder value is a tribute to agile's success at changing the overarching story of the industry.

All this reminds me of an essay by Brian Marick on ease and joy. Agile is about joy at work. We programmers love to make software -- software that improves the lives of its users. Great software. Let us love writing programs, and we can do great things for you.

I enjoyed Reg Braithwaite's talk Ruby.rewrite(Ruby) (slides available on-line). It gives a nice survey of some metaprogramming hacks related to Ruby's syntactic and semantic structure.

To me, one of the most thought-provoking things Reg says is actually a rather small point in the overall message of the talk. Object-oriented programming is, he summarizes, basically a matter a matter of nouns and verbs, objects and their behaviors. What about other parts of speech? He gives a simple example of an adverb:

blitz.not.blank?

In this expression, not is an adverb that modifies the behavior of blank?. At the syntactic level, we are really telling blitz to behave differently in response to the next message, which happens to be blank?, but from the programmer's semantic level not modifies the predicate blank?. It is an adverb!

Reg notes that some purists might flag this code as a violation of the Law of Demeter, because it sends a message to an object received from another message send. But it doesn't! It just looks that way at the syntax level. We aren't chaining two requests together; we are modifying how one of the requests works, how its result is to be interpreted. While this may look like a violation of the Law of Demeter, it isn't. Being able to talk about adverbs, and thus to distinguish among different kinds of message, helps to make this clear.

It also helps us to program better in at least two ways. First, we are able to use our tools without unnecessary guilt at breaking the letter of a law that doesn't really apply. Second, we are freed to think more creatively about how our programs can say what we mean. I love that Ruby allows me to create constructs such as not and weave them seamlessly into my code. Many of my favorite gems and apps use this feature to create domain-specific languages that look and feel like what they are and look and feel like Ruby -- at the same time. Treetop is an example. I'd love to hear about your favorite examples.

So, our OO programs have nouns and verbs and adverbs. What about other parts of speech? I can think of at least two from Java. One is pronouns. In English, this is a demonstrative pronoun. It is in Java, too. I think that super is also demonstrative pronoun, though it's not a word we use similarly in English. As an object, I consist of this part of me and that (super) part of me.

Another is adjectives. When I teach Java to students, I usually make an analogy from access modifiers -- public, private, and protected -- to adjectives. They modify the variables and methods which they accompany. So do synchronized and volatile.

Once we free ourselves to think this way, though, I think there is something more powerful afoot. We can begin to think about creating and using our own pronouns and adjectives in code. Do we need to say something in which another part of speech helps us to communicate better? If so, how can we make it so? We shouldn't be limited to the keywords defined for us five or fifteen or fifty years ago.

Thinking about adverbs in programming languages reminds me of a wonderful Onward! talk I heard at the OOPSLA 2003 conference. Cristina Lopes talked about naturalistic programming. She suggested that this was a natural step in the evolution from aspect-oriented programming, which had delocalized references within programs in a new way, to code that is concise, effective, and understandable. Naturalistic programming would seek to take advantage of elements in natural language that humans have been using to think about and describe complex systems for thousand of years. I don't remember many of the details of the talk, but I recall discussion of how we could use anaphora (repetition for the sake of emphasis) and temporal references in programs. Now that my mind is tuned to this wavelength, I'll go back to read the paper and see what other connections it might trigger. What other parts of speech might we make a natural part of our programs?

(While writing this essay, I have felt a strong sense of deja vu. Have I written a previous blog entry on this before? If so, I haven't found it yet. I'll keep looking.)

One of my colleagues is an old-school C programmer. He can make the machine dance using C. When C++ came along, he tried it for a while, but many of the newly-available features seemed like overkill to him. I think templates fell into that category. Other features disturbed him. I remember him reporting some particularly bad experiences with operator overloading. They made code unreadable! Unmaintainable! You could never be sure what + was doing, let alone operators like () and casts. His verdict: Operator overloading and its ilk are too powerful. They are fine in theory, but real languages should not provide so much freedom.

Some people don't like languages with features that allow them to reconfigure how the language looks and works. I may have been in that group once, long ago, but then I met Lisp and Smalltalk. What wonderful friends they were. They opened themselves completely to me; almost nothing was off limits. In Lisp, most everything was open to inspection, code was data that I could process, and macros let me define my own syntax. In Smalltalk, everything was an object, including the integers and the classes and the methods. Even better, most of Smalltalk was implemented in Smalltalk, right there for me to browse and mimic... and change.

Once I was shown a world bigger than Fortran, PL/I, and Pascal, I came to learn something important, something Giles Bowkett captures in his inimitable, colorful style:

There is no such thing as metaprogramming. It's all just programming.

(Note: "Colorful" is a euphemism for "not safe to read aloud at work, nor to be read by those with tender sensibilities".)

Ruby fits nicely with languages such as Common Lisp, Scheme, and Smalltalk. It doesn't erect too many boundaries around what you can do. The result can be disorienting to someone coming from a more mainstream language such as Java or C, where boundaries between "my program" and "the language" are so much more common. But to Lispers, Schemers, and Smalltalkers, the freedom feels... free. It empowers them to express their ideas in code that is direct, succinct, and powerful.

Actually, when you program in C, you learn the same lesson, only in a different way. It's all just programming. Good C programmers often implement their own little interpreters and their own higher-order procedures as a part of larger programs. To do so, they simply create their own data structures and code to manipulate them. This truth is the raw material out of which Greenspun's Tenth Rule of Programming springs. And that's the point. In languages like C, if you want to use more powerful features, and you will, you have to roll them for yourself. My friends who are "C weenies" -- including the aforementioned colleague -- take great pride in their ability to solve any problem with just a bit more programming, and they love to tell us the stories

Metaprogramming is not magic. It is simply another tool in the prepared programmer's toolbox. It's awfully nice when that tool is also part of the programming language we use. Otherwise, we are limited in what we can say conveniently in our programs by the somewhat arbitrary lines drawn between real and meta.

You know what? Almost everything in programming looks like magic to me. That may seem like an overstatement, but it's not. When I see a program of a few thousand lines or more generate music, play chess, or even do mundane tasks like display text, images, and video in a web browser, I am amazed. When I see one program convert another into the language of a particular machine, I am amazed. When people show me shorter programs that can do these things, I am even more amazed.

The beauty of computer science is that we dig deeper into these programs, learn their ideas, and come to understand how they work. We also learn how to write them ourselves.

It may still feel like magic to me, but in my mind I know better.

Whenever I bump into a new bit of sorcery, a new illusion or a new incantation, I know what I need to do. I need to learn more about how to write programs.

I know all about the idea of "writing to learn". It is one of the most valuable aspects of this blog for me. When I first got into academia, though, I was surprised to find how many books in the software world are written by people who are far from experts on the topic. Over the years, I have met several serial authors who pick a topic in conjunction with their publishers and go. Some of these folks write books that are successful and useful to people. Still the idea has always seemed odd.

In the last few months, I've seen several articles in which authors talk about how they set out to write a book on a topic they didn't know well or even much at all. Last summer, Alex Payne wrote this about writing the tapir book:

I took on the book in part to develop a mastery of Scala, and I've looked forward to learning something new every time I sit down to write, week after week. Though I understand more of the language than I did when I started, I still don't feel that I'm on the level of folks like David Pollak, Jorge Ortiz, Daniel Spiewak, and the rest of the Scala gurus who dove into the language well before Dean or I. Still, it's been an incredible learning experience ...

Then today I ran across Noel Rappin's essay about PragProWriMo:

I'm also completely confident in this statement -- if you are willing to learn new things, and learn them quickly, you don't need to be the lead maintainer and overlord to write a good technical book on a topic. (Though it does help tremendously to have a trusted super-expert as a technical reference.)

Pick something that you are genuinely curious about and that you want to understand really, really well. It's painful to write even a chapter about something that doesn't interest you.

This kind of writing to learn is still not a part of my mentality. I've certainly chosen to teach courses in order to learn -- to have to learn -- something I want to know, or know better. For example, I didn't know any PHP to speak of, so I gladly took on a 5-week course introducing PHP as a scripting language. But I have a respect for books, perhaps even a reverence, that makes the idea of publishing one on a subject I am not expert in unpalatable. I have to much respect for the people who might read it to waste their time.

I'm coming to learn that this probably places an unnecessary limit on myself. Articles like Payne's and Rappin's remind me that I can study something and become expert enough to write a book that is useful to others. Maybe it's time to set out on that path.

Getting people to take this step is one good reason to heed the call of Pragmatic Programmers Writing Month (PragProWriMo), which is patterned after the more generic NaNoWriMo (NaNoWriMo). Writing is like anything else: we can develop a habit that helps us to produce material regularly, which is a first and necessary step to ever producing good material regularly. And if research results on forming habits is right, we probably need a couple of months of daily repetitions to form a habit we can rely on.

So, whether it's a book or blog you have in mind, get to writing.

(Oh, and you really should click through the link in Rappin's essay to Merlin Mann's Making the Clackity Noise for a provocative -- if salty -- essay on why you should write. From there, follow the link to Buffering, where you will find a video of drummer Sonny Payne playing an extended solo for Count Basie's orchestra. It is simply remarkable.)

I was happy to come across Greg Wilson's talk, Bits of Evidence, on empirical data in software engineering. When I started preparing to teach software engineering last summer, I looked for empirical data to support some of the claims that we make about building software. I wasn't all that successful. I figured that either I wasn't looking hard enough, or there wasn't much. The answer probably lies somewhere in the middle.

Someone could do the SE world a great service by gathering, organizing, and providing links to all the good work that has been done. Wilson is one of the people I turn to for pointers to empirical SE results.

I did have fun reading some classic old work in this area. One is McCabe's original article on cyclomatic complexity. This is very cool and has a nice tie to theory, but it simply describes a metric. It does not gather present data from real programs against which the metric has applied, and it doesn't provide any base line for comparison. When he speaks of 10 as a reasonable upper bound for a module's cyclomatic complexity, or of code with cyclomatic complexity in the 3-7 range as "quite well structured", I wonder "Why?" He drew these values from experience looking at production code available to him, but these numbers feel a bit unreliable.

I'm not a stickler who needs statistically significant analysis of carefully collected data, though. I like to learn from experience that is gathered and presented qualitatively. I enjoyed reading about an informal experiment into the complexity of code developed test-first. We need to be careful to take the results with a grain of salt, given the informality of the definitions and methodology used, but the experiment seems to say something useful. I also liked Martin Fowler's reflective analysis of dynamic type checking in production Ruby code from ThoughtWorks. He also wrote an interesting reflection on Ruby at ThoughtWorks that I learned from.

Still, we need more of the sort of empirical evidence that Wilson offers in his talk. As a discipline, we can do a better job of paying attention to the validity of our claims, and of more frequently asking, "Data, please!"

I really enjoyed reading the text of William Cook's banquet speech at ECOOP 2009. When I served as tutorials chair for OOPSLA 2006, Cook was program chair, and that gave me a chance to meet him and learn a bit about his background. He has an interesting career story to tell. In this talk, he tells us this story as a way to compare and contrast computer science in academia and in industry. It's well worth a read.

As a doctoral student, I thought my career path might look similar to Cook's. I applied for research positions in industry, with two of the Big Six accounting firms and with an automobile manufacturer, at the same time I applied for academic positions. In the end, after years of leaning toward industry, I decided to accept a faculty position. As a result, my experience with industrial CS research and development is limited to summer positions and to interactions throughout grad school.

Cook's talk needs no summary; you should read it for yourself. Here are a few points that stood out to me as I read:

Venture capitalists talk about pain killers versus vitamins. A vitamin is a product that might make you healthier over the long run, but it's hard to tell. Pain killers are products where the customer screams "give it to me now" and asks what it costs later. Venture capitalists only want to fund pain killers.

This is true not only of venture capitalists, but of lots of people. As a professor, I recognize this in myself and in my students all of the time. Cook points out that most software development tools are vitamins. So are many of the best development practices. We need to learn tools and practices that will make us most productive and powerful in the long run, but without short-term pain we may not generate the resolve to do so.

We read all the standard references on OO for business applications. It didn't make sense to us. We started investigating model-driven development styles. We created modeling languages for data, user interfaces, security and workflow. These four aspects are the core of any enterprise application. We created interpreters to run the languages, and our interpreters did many powerful optimizations by weaving together the different aspects.

To me, this part of the talk exemplifies best how a computer scientist thinks differently than a non-computer scientist, whether experienced in software development or not. Languages are tools we create to help us solve problems, not merely someone else's solutions we pluck off the shelf. Language processors are tools we create to make our languages come to life in solving instances of actual problems.

The way I see it is that industry generally has more problems than they do solutions, but academia often has more solutions than problems.

Cook makes a great case for a bidirectional flow between industry, with its challenging problems in context, and academia, with its solutions built of theory, abstraction, and design. This transfer can be mutually beneficial, but it is complicated by context:

Industrial problems are often messy and tied to specific technical situations or domains. It is not easy to translate these complex problems into something that can be worked on in academia. This translation involves abstraction and selection.

The challenge is greatest when we then take solutions to problems abstracted from real-world details and selected for interestingness more than business value and try to re-inject them into the wild. Too often, these solutions fail to take hold, not because people in industry are "stupid or timid" but because the solution doesn't solve their problem. It solves an ideal problem, not a real one. The translation process from research to development requires a finishing step that people in the research lab often have little interest in doing and that people in the development studio have little time to implement. The result is a disconnect that can sour the relationship unnecessarily.

Finally, the talk is full of pithy lines that I hope to steal and use to good effect sometime soon. Here is my favorite:

Simplicity is not where things start. ... It is where they end.

Computer scientists seek simplicity, whether in academia or in industry. Cook gives us some clues in this talk about how people in these spheres can understand one another better and, perhaps, work better together toward their common goal.

I didn't write this comment on the blog entry Wa: The key to clear, harmonious design:

I know that you are talking about visual design, but I am struck by how this approach applies to many other domains.

But I could have.

I started university life intending to become an architect, and my interest in visual design has remained strong through the years. I was delighted when I learned of Christopher Alexander's influence on some in the software world, because it gave me more opportunities to read and think about architectural design -- and to think about how its ideas relate to how we design software. I am quite interested in the notion that there are universal truths about design, and even if not what we can learn from designers in other disciplines.

Garr Reynolds identifies seven principles of the Zen aesthetic of harmony. Like the commenter, my thoughts turned quickly from the visual world to another domain. For me, the domain is software. How well do these principles of harmony apply to software? Several are staples of software design. Others require more thought.

(1) Embrace economy of materials and means
(3) Keep things clean and clutter-free

These are no-brainers. Programmers want to keep their code clean, and most prefer an economical style, even when using a language that requires more overhead than they would like.

(6) Think not only of yourself, but of the other (e.g., the viewer).

When we develop software, we have several kinds of others to consider. The most obvious are our users. We have disciplines, such as human-computer interaction, and development styles, such as user-centered design, focused squarely on the people who will use our programs.

We also need to think of other programmers. These are the people who will read our code. Software usually spends much more time in maintenance than in creation, so readability pays off in a huge way over time. We can help our readers by writing good documentation, an essential complement to our programs. However, the best way to help our readers is to write readable code. In this we are more like Reynolds's presenters. We need to focus on the clarity and beauty of our primary product.

Finally, let's not forget our customers and our clients, the people who pay us to write software. To me, one of the most encouraging contributions of XP was its emphasis on delivering tangible value to our customers every day.

(7) Remain humble and modest.

This is not technical advice. It is human advice. And I think it is underrated in too many contexts.

I have worked with programmers who were not humble enough. Sadly, I have been that programmer, too.

A lack of humility almost always hurts the project and the team. Reynolds is right in saying that true confidence follows from humility and modesty. Without humility, a programmer is prone to drift into arrogance, and arrogance is more dangerous than incompetence.

A programmer needs to balance humility against gumption, the hubris that empowers us to tackle problems which seem insurmountable. I have always found that humility is a great asset when I have the gumption to tackle a big problem. Humility keeps me alert to things I don't understand or might not see otherwise, and it encourages me to take care at each step.

... Now come a couple of principles that cause me to thing harder.

(2) Repeat design elements.

Duplication is a bane to software developers. We long ago recognized that repetition of the same code creates so many problems for writing and modifying software that we have coined maxims such as "Don't repeat yourself" and "Say it one once and only once." We even create acronyms such as DRY to get the idea across in three spare letters.

However, at another level, repetition is unavoidable. A stack is a powerful way to organize and manipulate data, so we want to use one whenever it helps. Rather than copy and paste the code, we create an abstract data type or a class and reuse the component by instantiating it.

Software reuse of this sort is how programmers repeat design elements. Indeed, one of the most basic ideas in all of programming is the procedure, an abstraction of a repeated behavioral element. It is fundamental to all programming, and one of the contributions that computer science made as we moved away from our roots in mathematics.

In the last two decades, programmers have begun to embrace repeatable design units at higher levels. Design patterns recur across contexts, and so now we do our best to document them and share them with others. Architectures and protocols and, yes, even our languages are ways to reify recurring patterns in a way that makes using them as convenient as possible.

(4) Avoid symmetry.

Some programmers may look at this principle and say, "Huh? How can this apply? I'm not even sure what it means in the context of software."

When linguistic structures and data structures repeat, they repeat just as they are, bringing a natural symmetry to the algorithms we use and the code we write. But at the level of design patterns and architectures, things are not so simple. Christopher Alexander, the building architect who is the intellectual forefather of the software patterns community, famously said that a pattern appears a million times, but never exactly the same. The pattern is molded to fit the peculiar forces at play in each system. This seems to me a form of breaking symmetry.

But we can take the idea of avoiding symmetry farther. In the mathematical and scientific communities, there has long been a technical definition of symmetry in groups, as well as a corresponding definition of breaking symmetry in patterns. Only a few people in the software community have taken this formal step with design patterns. Chief among them are Jim Coplien and Liping Zhao. Check out their book chapter, Symmetry Breaking in Software Patterns, if you'd like to learn more.

A few years ago I was able to spend some time looking at this paper and at some of the scientific literature on patterns and symmetry breaking. Unfortunately, I have not been able to return to it since. I don't yet fully understand these ideas, but I think I understand enough to see that there is something important here. This glimmer convinces me that avoiding symmetry is perhaps an important principle for us software designers, one worthy of deeper investigation.

... This leaves us with one more principle from the Presentation Zen article:

(5) Avoid the obvious in favor of the subtle

This is the one principle out of the seven that I think does not apply to writing software. All other things being equal, we should prefer the obvious to the subtle. Doing something that isn't obvious is the single best reason to write a comment in our code. When we must do something unexpected by our readers, we must tell them what we have done and why. Subtlety is an impediment to understanding code.

Perhaps this is a way in which we who work in software differ from creative artists. Subtlety can enhance a work of art, by letting -- even requiring -- the mind to sense, explore, and discover something beyond the surface. As much art as there is in good code, code is at its core a functional document. Requiring maintenance programmers to mull over a procedure and to explore its hidden treasures only slows them down and increases the chances that they will make errors while changing it.

I love subtlety in algorithms and designs, and I think I've learned a lot from reading code that engages me in a way I've not experienced before. But there is something dangerous about code in which subtlety becomes more important than what the program does.

Blaine Buxton recently wrote a nice entry on the idea of devilishly clever code:

But, it got me thinking about clever and production code. In my opinion, clever is never good or wanted in production code. It's great to learn and understand clever code, though. It's a great mental workout to keep you sharp.

Maybe I am missing something subtle here; I've been accused of not seeing nuance before. This may be like the principle of avoiding symmetry, but I haven't reached the glimmer of understanding yet. Certainly, many people speak of Apple's great success with subtle design that engages and appeals to users in a way that other design companies do not. Others, though attribute its success to creating products that are intuitive to use. To me, intuitiveness points more to obviousness and clarity than to subtlety. And besides, Apple's user experience is at the level of design Reynolds is talking about, not at the level of code.

I would love to hear examples, pro and con, of subtlety in code. I'd love to learn something new!

Once again, quick hits, for different values of x and different values of day.

Blog Title of the Day

I love the title of this one:

C:\ONGRTLNS.OSX

If you're too old to remember DOS, this may not mean much to you. Ironically, the article is about a vestige of old Mac OS that is disappearing. It's an interesting read, even for non-Mac guys, as an example of software evolution.

Tweet of the Day

The simplest advice is often the best:

i've said it before, @jbogard : never take software advice from a bug tracking system salesman

Duct tape that.

Reader and occasional writer that I am, Michael Nielsen's Six Rules for Rewriting seemed familiar in an instant. I recognize their results in good writing, and even when I don't practice them successfully in my own writing I know they would often make it better.

Occasional programmer that I am, they immediately had me thinking... How well do they apply to refactoring? Programming is writing, and refactoring is one of our common forms of rewriting... So let's see.

First of all, let's acknowledge up front that a writer's rewriting is not identical to a programmer's refactoring. First of all, the writer does not have automated tests to help her ensure that the rewrite doesn't break anything. It's not clear to me exactly what not breaking anything means for a writer, though I have a vague sense that it is meaningful for most writing.

Also, the term "refactoring" does not refer to any old rewrite of a code base. It has a technical meaning: to modify code without changing its essential functionality. There are rewrites of a code base that are not refactoring. I think that's true of writing in general, though, and I also think that Nielsen is clearly talking about rewrites that do not change the essential content or purpose of a text. His rules are about how to say the same things more effectively. That seems close enough to our technical sense of refactoring to make this exercise worth an effort.

Striking

Every sentence should grab the reader and propel them forward.

Can we say that every line of code should grab the reader and propel her forward?! I certainly prefer to read programs in which every statement or expression tells me something important about what the program is and does. Some programming languages make this harder to do, with boilerplate and often more noise than signal.

Perhaps we could say that every line of code should propel the program forward, not get in the way of its functionality? This says more about the conciseness with which the programmer writes, and fits the spirit of Nielsen's rule nicely.

Every paragraph should contain a striking idea, originally expressed.

Can we say that every function or class should contain a striking idea, originally expressed? Functions and classes that do not usually get in the reader's way. In programming, though, we often write "helpers", auxiliary functions or classes that assist another in expressing an essential, even striking, idea. The best helpers capture an idea of deep value, but it's may be the nature of decomposition that we sometimes create ones that are striking only in the context of the larger system.

The most significant ideas should be distilled into the most potent sentences possible.

Yes! The most significant ideas in our programs should be distilled into the most potent code possible: expressions, statements, functions, classes, whatever the abstractions our language and style provide.

Style

Use the strongest appropriate verb.

Of course. Names matter. Use the strongest, most clearly named primitives and library functions possible. When we create new functions, give them strong, clear names. This rule applies to our nouns, too. Our variables and classes should carry strong names that clearly name their concept. No more "manager" or "process" nouns. They avoid naming the concept. What do those objects do?

This rule also applies more broadly to coding style. It seems to me that Tell, Don't Ask is about strength in our function calls.

Beware of nominalization.

In code, this guideline prescribes a straightforward idea: Don't make a class when a function will do. You Aren't Gonna Need It.

Meta

None of the above rules should be consciously applied while drafting material.

Anyone who writes a lot knows how paralyzing it can be to worry about writing good prose before getting words down onto paper, or into an emacs buffer. Often we don't know what to write until we write it; why try to write that something perfect before we know what it is?

This rule fits nicely with most lightweight approaches to programming. I even encourage novice programmers to write code this way, much to the chagrin of my more engineering-oriented colleagues. Don't be paralyzed by the blank screen. Write freely. Make something work, anything on the path to a solution, and only then worry about making it right and fast. Do the simplest thing that will work. Only after your code works do you rewrite to make it better.

Not all rewriting is refactoring, but all refactoring is rewriting. Write. Pass the test. Refactor.

Many people find that refactoring provides the most valuable use of design patterns, as a target toward which one moves the code. This is perhaps a more important use of patterns than initial design, at which time many of us tend to overdesign our programs. Joshua Kerievsky's Refactoring to Patterns book makes shows programmers how to do this safely and reliably. I wonder if there is any analogue to this book in the writing world, or if there even could be such a book?

I once wrote a post on writing in an agile style, and rewriting played a key role in that idea. Some authors like rewriting more than writing, and I think you can say the same thing of many, many programmers. Refactoring brings a different kind of joy, at getting something right that was before almost right -- which is, of course, another way of saying not yet right.

I recall once talking with a novelist over lunch about tools for writers. Yet even the most humble word processor has done so much to change how authors write and rewrite. One of the comments on Nielsen's entry asks whether new tools for writing have changed the way writers think. We might also ask whether new tools -- the ability to edit and rewrite so much more easily and with so much less= technical effort -- has changed the product created by most writers. If not, could it?

New tools also change how we rewrite code. The refactoring browser has escaped the confines of the Smalltalk image and now graces IDEs for Java, C++, and C## programmers; indeed, refactoring tools exist for so many languages these days. Is that good or bad? Many of my colleagues lament that the ease of rewriting has led to an endemic sloppiness, to a rash of random programming in which students keep making seemingly random changes to their code until something compiles. Back in the good old days, we had to think hard about our code before we carved it into clay tablets... It seems clear to me that making rewriting and refactoring easier is a huge win, even as it changes how we need to teach and practice writing.

In retrospect, a lot of Nielsen's rules generalize to dicta we programmers will accept eagerly. Eliminate boilerplate. Write concise, focused code. Use strong, direct, and clear language. Certainly when we abstract the tasks to a certain level, writing and rewriting really are much the same in text and code.

... AKA Test-Driven X: Teaching

Writing a test before writing code provides a wonderful level of accountability to self and world. The test helps me know what code to write and when we are done. I am often weak and like being able to keep myself honest. Tests also enable me to tell my colleagues and my boss.

These days, I usually think in test-first terms whenever I am creating something. More and more finding myself wondering whether a test-driven approach might work even better. In a recent blog entry, Ron Jeffries asked for analogues of test-driven development outside the software world. My first thought was, what can't be done test-first or even test-driven? Jeffries is, in many ways, better grounded than I am, so rather than talk about accountability, he writes about clarity and concreteness as virtues of TDD. Clarity, concreteness, and accountability seem like good features to build into most processes that create useful artifacts.

I once wrote about student outcomes assessment as a source of accountability and continuous feedback in the university. I quoted Ward Cunningham at the top of that entry, " It's all talk until the tests run.", to suggest to myself a connection to test-first and test-driven development.

Tests are often used to measure student outcomes from courses that we teach at all levels of education. Many people worry about placing too much emphasis on a specific test as a way to evaluate student learning. Among other things, they worry about "teaching to the test". The implication is that we will focus all of our instruction and learning efforts on that test and miss out on genuine learning. Done poorly, teaching to the test limits learning in the way people worry it will. But we can make a similar mistake when using tests to drive our programming, by never generalizing our code beyond a specific set of input values. We don't want to do that in TDD, and we don't want to do that when teaching. The point of the test is to hold us accountable: Can our students actually do what we claim to teach them?

Before the student learning outcomes craze, the common syllabus was the closest thing most departments had to a set of tests for a course. The department could enumerate a set of topics and maybe even a set of skills expected of the course. Faculty new to the course could learn a lot about what to teach by studying the syllabus. Many departments create common final exams for courses with many students spread across many sections and many instructors. The common final isn't exactly like our software tests, though. An instructor may well have done a great job teaching the course, but students have to invest time and energy to pass the test. Conversely, students may well work hard to make sense of what they are taught in class, but the instructor may have done a poor or incomplete job of covering the assigned topics.

I thought a lot about TDD as I was designing what is for me a new course this semester, Software Engineering. My department does not have common syllabi for courses (yet), so I worked from a big binder of material given to me by the person who has taught the course for the last few years. The material was quite useful, but it stopped short of enumerating the specific outcomes of the course as it has been taught. Besides, I wanted to put my stamp on the course, too... I thought about what the outcomes should be and how I might help students reach them. I didn't get much farther than identifying a set of skills for students to begin learning and a set of tools with which they should be familiar, if not facile.

Greg Wilson has done a very nice job of designing his Software Carpentry course in the open and using user stories and other target outcomes to drive his course design. In modern parlance, my efforts in this regard can be tagged #fail. I'm not too surprised, though. For me, teaching a course the first time is more akin to an architectural spike than a first iteration. I have to scope out the neighborhood before I know how to build.

Ideally, perhaps I should have done the spike prior to this semester, but neither the world nor I are ideal. Doing the course this way doesn't work all that badly for the students, and usually no worse than taking a course that has been designed up front by someone who hasn't benefitted from the feedback of teaching the course. In the latter case, the expected outcomes and tests to know they have been met will be imperfect. I tend to be like other faculty in expecting too much from a course the first few times I teach it. If I design the new course all the way to the bottom, either the course is painful for students in expecting too much too fast, or the it is painful for me as I un-design and re-design huge portions of the course.

Ultimately, writing and running tests come back to accountability. Accountability is in short supply in many circles, university curricula included, and tests help us to have it. We owe it to our users, and we owe it to ourselves.

The last two class days in my Software Engineering course, I have introduced students to agile software development. Here is a recent Twitter exchange between me and student in the course:

Student: Agile development is the hippy approach to programming.

Eugene: @student: "Agile development is the hippy approach to programming." Perhaps... If valuing people makes me a hippie, so be it.

Student: @wallingf I leave it up to the individual whether they interpret that as good or bad. Personally, I prefer agile, but am also a free spirit

Eugene: @student It's just rare that anyone would mistake me for a hippie...

Student: @wallingf you do like the phrase "roll my own" a lot

I guess I can never be sure what to expect when I tell a story. I've been talking about agile approaches with colleagues for a decade, and teaching them to students off and on since 2002. No student has ever described agile to me as hippie-style development development.

As I think about it from the newcomer's perspective, though, it makes sense. Consider Change, and keep changing, a recent article by Liz Keogh:

Sometimes people ask me, "When we've gone Agile ... what will it look like?" ...

Things will be changing. You'll be in a better place to respond to change. Your people will have a culture of courage and respect, and will seek continuous improvement, feedback and learning. ...

I have no idea what skills your people will need. The people you have are good people; start with them. ...

There is no end-state with Agile or Lean. You'll be improving, and continue to improve, trying new things out and discarding the ones which don't work. ...

I love Liz Keogh's stuff. Is this article representative of the agile community? I think so, at least of a vibrant part of it. Reading this article as I did right after participating in the above exchange, I can see what the student means.

I do not think there is a necessary connection between hippie culture and a preference for agile development. I am so not a hippie. Ask anyone who knows me. I see agile ideas from a perspective of pragmatism. Change happens. Responding to it, if possible, is better than trying to ignore it while it happens before my eyes.

Then again, a lot of my friends in the agile community probably were hippies, or could have been. The Lisp and Smalltalk worlds have long been sources of agile practice, and I am willing to speculate that the counterculture has long been over-represented there. Certainly, those worlds are countercultural amid the laced up corporate world of Cobol, PL/I, and Java.

I decided to google "agile hippie" to see who else has talked about this connection. The top hit talks about common code ownership as "hippie socialism" and highlights the need for a strong team leader to make P work. Hmm. Then the second hit is an essay by Lisp guru and countercultural thinker Richard Gabriel. It refers to agile methodologies as rising from the world of paid consultants and open-source software as the hippie thing.

One of Gabriel's points in this essay echoes something I told my students today: the difference between traditional and agile approaches to software is a matter of degree and style, not a difference in kind. I told my students that nearly all software development these days is iterative development, more or less. The agile community simply prefers and encourages an even tighter spiral through the stages of development. Feedback is good.

Folks may see a hippie connection in agile's emphasis on conversation, working together, and close teamwork. One thing I like about the Gabriel essay is its mention of transparency -- being more open about the process to the rest of the world, especially the client. I need to bring this element out explicitly in my presentation, too,

Maybe even the most straight-laced of agile developers have a touch of hippie radical inside them.

Yesterday evening, in between volleyball games, I had a chance to do some reading. I marked several one-liners to blog on. I planned a disconnected list of short notes, but after I started writing I realized that they revolve around a common theme: change.

Over the last few months, Kent Beck has been blogging about his experiences creating a new product and trying to promote a new way to think about his design. In his most recent piece, Turning Skills into Money, he talks about how difficult it can be to create change in software service companies, because the economic model under which they operates actually encourages them to have a large cohort of relatively inexperienced and undertrained workers.

The best line on that page, though, is a much-tweeted line from a comment by Niklas Bjørnerstedt:

A good team can learn a new domain much faster than a bad one can learn good practices.

I can't help thinking about the change we would like to create in our students through our software engineering course. Skills and good practices matter. We cannot overemphasize the importance of proficiency, driven by curiosity and a desire to get better.

Then I ran across Jason Fried's The Next Generation Bends Over, a salty and angry lament about the sale of Mint to Intuit. My favorite line, with one symbolic editorial substitution:

Is that the best the next generation can do? Become part of the old generation? How about kicking the $%^& out of the old guys? What ever happened to that?

I experimented with Mint and liked it, though I never convinced myself to go all the way it. I have tried Quicken, too. It seemed at the same time too little and too much for me, so I've been rolling my own. But I love the idea of Mint and hope to see the idea survive. As the industry leader, Intuit has the leverage to accelerate the change in how people manage their finances, compared to the smaller upstart it purchased.

For those of us who use these products and services, the nature of the risk has just changed. The risk with the small guy is that it might fold up before it spreads the change widely enough to take root. The risk with the big power is that it doesn't really get it and wastes an opportunity to create change (and wealth). I suspect that Intuit gets it and so hold out hope.

Still... I love the feistiness that Fried shows. People with big ideas and need not settle. I've been trying to encourage the young people with whom I work, students and recent alumni, to shoot for the moon, whether in business or in grad school.

This story meshed nicely with Paul Graham's Post-Medium Publishing, in which Graham joins in the discussion of what it will be like for creators no longer constrained by the printed page and the firms that have controlled publication in the past. The money line was:

... the really interesting question is not what will happen to existing forms, but what new forms will appear.

Change will happen. It is natural that we all want to think about our esteemed institutions and what the change means for them. But the real excitement lies in what will grow up to replace them. That's where the wealth lies, too. That's true for every discipline that traffics in knowledge and ideas, including our universities.

Finally, Mark Guzdial ruminates on what changes CS education. He concludes:

My first pass analysis suggests that, to make change in CS, invent a language or tool at a well-known institution. Textbooks or curricula rarely make change, and it's really hard to get attention when you're not at a "name" institution.

I think I'll have more to say about this article later, but I certainly know what Mark must be feeling. In addition to his analysis of tools and textbooks and pedagogies, he has his own experience creating a new way to teach computing to non-majors and major alike. He and his team have developed a promising idea, built the infrastructure to support it, and run experiments to show how well it works. Yet... The CS ed world looks much like it always has, as people keep doing what they've always been doing, for as many reasons as you can imagine. And inertia works against even those with the advantages Mark enumerates. Education is a remarkably conservative place, even our universities.

There's been a long-running thread on the XP discussion list about a variation of pair programming suggested by a list member, on the basis of an experiment run in his shop and documented in a simple research report. One reader is skeptical; he characterized the research as 'only' an experience report and asked for more evidence. Dave Nicolette responded:

Point me to the study that proves...point me to the study that proves...point me to the study that proves...<yawn>

There is no answer that will satisfy the person who demands studies as proof.

Studies aren't proof. Studies are analyses of observed phenomena. The thing a study analyzes has already happened before the study is performed.

... The proof is in the doing.

This is one of those issues where each perspective has value, and even dominates the other under the right conditions.

Skepticism is good. Asking for data before changing how you, your team, or your organization behaves is reasonable. Change is hard enough on people when it succeeds; when it fails, it wastes time and can dispirit people. At times, skepticism is an essential defense mechanism.

Besides, if you are happy with how things work now, why change at all?

The bigger the changes, the more costly the change, the more valuable is skepticism.

In the case of the XP list discussion, though, we see a different set of conditions. The practice being suggested has been tried at one company, so its research report really is "just" an experience report. But that's fine. We can't draw broadly applicable conclusions from an experiment of one anyway, at least not if we want the conclusions to be reliable. This sort of suggestion is really an invitation to experiment: We tried this, it worked for us, and you might want to try it.

If you are dissatisfied with your current practice, then you might try the idea as a way to break out of a suboptimal situation. If you are satisfied enough but have the freedom and inclination to try to improve, then you might try the idea on a lark. When the cost of giving the practice a test drive is small enough, it doesn't take much of a push to do so.

What a practice like this needs is to have a lot of people try it out, under similar conditions and different conditions, to find out if the first trial's success indicates a practice that can be useful generally or was a false positive. That's where the data for a real study comes from!

This sort of practice is one that professional software developers must try out. I could run an experiment with my undergraduates, but they are not (yet) like the people who will use the practice in industry, and the conditions under which they develop are rarely like the conditions in industry. We could gain useful information from such an experiment (especially about how undergrads work and think), but the real proof will come when teams in industry use the practice and we see what happens.

Academics can add value after the fact, by collecting information about the experiments, analyzing the data, and reporting it to the world. That is one of the things that academics do well. I am reminded of Jim Coplien's exhortations a decade and more ago for academic computer scientists to study existing software artifacts, in conjunction with practitioners, to identify useful patterns and document the pattern languages that give rise to good and beautiful programs. While some CS academics continue to do work of this sort -- the area of parallel programming patterns is justifiably hot right now -- I think we in academic CS have missed an opportunity to contribute as much as we might otherwise to software development.

We can't document a practice until it has been tried. We can't document a pattern until it recurs. First we do, then we document.

This reminds me of a blog entry written by Perryn Fowler about a similar relationship between "best practices" and success:

Practices document success; they don't create it.

A tweet before its time...

As I have scanned through this thread on the discussion list, I have found it interesting that some agile people are as skeptical about new agile practices as non-agile folks were (and still are) about agile practices. You would think that we should be practicing XP as it was writ nearly a decade ago. Even agile behavior tends toward calcification if we don't remain aware of why we do what we do.

Greg Wilson wrote a nice piece recently on the big picture for his Software Carpentry course. His first paragraph captures an idea that is key to the notion of "programming for all" that I and others have been talking about lately:

One of the lessons we learned ... is that most scientists don't actually want to learn how to program--they want solve scientific problems. To many, programming is a tax they have to pay in order to do their research. To the rest, it's something they really would find interesting, but they have a grant deadline coming up and a paper to finish.

Most people don't care about programming. If they do care, they don't like it, or think they don't. They want to do something. Programming has to be a compelling tool, something that makes their lives better as they do the thing they like to do. If it changes how they think about problems and solutions, all the better. When we teach programming to people who are non-CS types, we should present it as such: a compelling tool that makes their lives better. If we teach programming qua programming, we will lose them before we have a chance to make a difference for them.

It turns out that all this is true of CS students, too. Most of them want to do something. And several of the lessons Wilson describes for working with scientists learning to program apply to CS students, such as:

• When students have never experienced the pain of working with big or long-lived programs that benefit from good abstractions, effective interfaces, and tools such as version control systems, "no amount of handwaving is going to get the idea across".
• The best way to reach students is to give them programming skills that will pay off for them in the short term. This motivates better than deferred benefit and encourages them to dig in deeper on their own. That's where the best learning happens anyway.
• Students are often surprised when a course is able to "convey the fundamental ideas needed to make sensible decisions about software without explicitly appearing to do so".

This issue is close to the surface for me as I work on my work on my software engineering course. Undergraduate courses don't usually expose students to the kind of software development projects that are likely to give the right context for learning many of the "big picture" ideas. An in-course project can help, but contemporaneous experience often runs out of sync with the software engineering content of the course. Next semester, they will take a project course in which they can apply what they learn in this course, but little good that does us now!

(Idea for later: Why not teach the project course first and follow with course that teaches techniques that depend on the experience?)

Fortunately, most students trust their professors and give them some leeway when learning skills that are beyond their experience to really grok -- especially when the skills and ideas are a known part of the milieu in which they will work.

Software Engineering this semester offers another complication. There is a wide spread in the level of software and programming experience between the most and least experienced students in the course. What appeals to students at one end of the spectrum often won't appeal students on the other. More fun for the professor...

In closing, I note that many of specific ideas from Wilson's course and Udell's conception of computational thinking apply to a good Software Engineering course for majors. (Several of Wilson's extensions to the body of CT apply less well, in part because they are part of other courses in the CS curriculum and in part because they go beyond the CT that every person probably needs.) I should not be surprised that these basic ideas apply. Wilson is teaching a form of software engineering aimed at a specific audience of developers, and computational thinking is really just a distillation of what all CS students should be learning throughout their undergraduate careers!

Mark Guzdial blogged this morning about a bug slipped through into the latest release of JES, the Jython IDE that his group has created to support courses and workshops in media computation. The bug was the result of a simple variable renaming in which n-1 of the uses was changed. (That nth always comes back to haunt us.) Mark uses this bug as an opportunity to discuss some bigger issues involving process, language, and the nature of CS1. He is taking some heat from a couple of pro-Python commenters, but I'm glad he wrote the article. People too rarely share mistakes when they are made, and besides Mark offers some interesting ideas both in his post and in his responses to comments.

Should we use a different kind of language when we program for others than when we program for ourselves? A compiler would have found this simple error at the time JES was built and saved everyone the grief of a new release. But so would running the test suite that the JES team has built. The difference is that developers have a choice of running the test suite or not, whether they use a compiled language or an interpreted one, but they have to compile their program when they use a compiled language.

People make mistakes. I am certainly not in a position to look down on the poor developer who opted not to run the unit tests after making what seemed like a trivial change. I have made similar mistakes before, and I'm sure I'll make another like it soon. And at least one thing is true about teaching university courses: students will find my mistakes!

I like it when a compiler catches errors like this for me, but... Still, I'm uneasy to give up the freedom that a language like Smalltalk, Scheme, or Ruby gives me when I am writing code. Even when the value of Mark's f() rises. What is someone like me to do?

The compiler is a tool that helps me find a certain kind of error. But there are other tools that help me find this and other kinds of error. The unit tests are one. A refactoring browser is another. If the JES developer had had at his or her disposal a refactoring browser for Python, it would have at least been able to point out possible effects of renaming the variable behind this bug. Dynamic languages pose a particular challenge for building really useful static analysis tools, but we can still build tools to help.

But the developer might not use the refactoring browser. The change is so simple, a quick romp in emacs or vi is all we need, right? The compiler is a build tool; the programmer has to run it to create the executable. One approach that developers using dynamic languages can take is to use a build system that always runs the test suite. Another is to tool the version control system to run the tests before accepting the code at check-in. These tools more closely fulfill the role played by the compiler.

In response to another comment on his entry, Mark mentions a trade-off between process and language, a lá the familiar space-time trade-off among algorithms and data structures. This, I think, is the most interesting implication of his post. But the language in which we write code is just one tool that we use in building software. The compiler is another. So are editors, browsers, version control systems, build systems, and testing frameworks. The trade-off is between processes and tools.

This trade-off is one that seems to fly under most people's radar. The authors of the Agile Manifesto wrestle with the trade-off between caring for "individuals and interactions" and caring for "processes and tools". They come down on the side of individuals and interactions. But as I wrote a few weeks ago, valuing people over tools makes having the right tools even more important. I should probably have said "the right tools and process", because the process by which one works is just as important a part of the infrastructure that any programmer or creator needs. The question then becomes: In supporting individuals and interactions, how do we find the right balance between tools and process?

People characterize the different kinds of process available to programmers in a number of ways:

• high ceremony versus low ceremony
• heavy versus lightweight
• sturdy versus agile

Some people assume that the righthand side of these choices require more discipline than the left, because the lefthand-style processes provide more rules to follow. That's not right. Discipline is, at best, orthogonal to this distinction. I think that the agile, lightweight, and low-ceremony approaches usually require more discipline, not less. In XP, you have to code in pairs, even if you think you don't need to. You have to run the tests. You have to refactor frequently and, on occasion, vigorously. You have to integrate code frequently. You have to take what you learn from all the practices and feed that back into the code. Easy? Not at all. Undisciplined? Not if you are practicing what you preach.

This sort of disciplined approach only works when the practices are at the level of granularity where they can become habit, part of one's muscle memory. It also works well only when supported by tools that make the process comfortable, almost disappear from conscious thought. Good tools shift as much of the burden of details and administrivia off of the programmer's mind and onto the tools that we use to write, build, test, share, and deliver our code.

I understand why many people want to use a compiler to support their process. But I still prefer to use languages that give me other kinds of freedom -- along with other tools that support my creative process.

I've been thinking a lot about the Software Engineering course I'm teaching this fall, which commences a week from Tuesday. Along the way, I have been looking at a lot of books and articles with "software engineering" in the title. It's surprising how few of them get any where near code. I know I shouldn't be surprised. I remember taking courses on software engineering, and I've stayed close enough to the area all these years to know what matters. There are issues other than programming that we need to think about while building big systems. And there is plenty of material out there about programming and programming tools and the nuts and bolts of programming.

Still, I think it is important when talking about software engineering to keep in mind what the goal is: a working program, or collection of programs. When we forget that, it's too easy to spin off into a land of un-reality. It's also important to keep in mind that someone has to actually write code, or no software will ever be engineered. I hope that the course I teach can strike a good balance.

In the interest of keeping code in mind, I share with you an assortment of programming news. Good, bad, ugly, or fun? You decide.

Hiding the Rainforest. Mark Guzdial reports that Georgia Tech is eliminating yet another language from its computing curriculum. Sigh. Thought experiments notwithstanding, variety in language and style is good for programmers. On a pragmatic note, someone might want to tell the GT folks that programming for the JVM may soon look more like Lisp and feel more like ML than Java or C++.

Programming meets the Age of Twitter. A Processing programming contest with a twist: all programs must be 200-characters or less. I'll give extra credit for any program that is a legal tweet.

Power to the Programmer! Phil Windley enjoys saving $60 by writing his own QIF->CSV converter. But the real hero is the person who wrote Finance::QIF.

Why Johnny Can't Read Perl. Courtesy of Lambda the Ultimate comes news we all figured had to be true: a formal proof that Perl cannot be parsed. Who said the Halting Theorem wasn't useful? I guess I'll stop working on my refactoring browser for Perl.

Via Jason Yip, this dandy story from a hospital president who favors process improvement Lean-style. It features Hideshi Yokoi, who is president of a Toyota production system support center in Kentucky:

... At one point, we pointed out a new information system that we were thinking of putting into place to monitor and control the flow of certain inventory. Mr. Yokoi's wise response, suggesting otherwise, was:

"When you put problem in computer, box hide answer. Problem must be visible!"

I have had many experiences, both personal and professional, in which putting something in a computer hides the problem from view and complicates finding the answer. Too often, when I decide to improve my process by putting data into my computer, I get so caught up in the rest of my tasks that I forget all about the problem. Tasks to do get lost in an ever-growing text file. Concerns about how we allocate merit pay get lost behind complex formulas in a spreadsheet that implement our current method. Scheduling and budget issues are lost in the last-minute details of readying this semester's spreadsheet for the next step in a university we don't control.

When you lose sight of a problem, it's hard to solve it. Moving things out of the daily clutter too easily becomes moving them out of our attention altogether.

I agree with Mr. Yokoi. A problem must be visible! But that cannot mean not using technology to automate and improve a process. Instead it means that we have to find ways to make the problem visible. JUnit was, for me, a great model. We needed to improve how we tested our programs, so we make the tests code. Rather than leave that code sitting lonely on disk, no better than textual documentation, we use a tool that runs the tests for us and tells what passes and what fails. Rather than leave that tool sitting lonely on disk, gathering dust because we forget to run the tests, we build running the tool into our development tools, where the tests can be run every time we save our changes or build our program. Make things visible.

We can avoid much of Mr. Yokoi's valid concern by changing our tools and our practices to ensure that problems don't disappear when we automate processes. That allows us to use computers to make us better.

Last month, in honor of the Apollo 11 mission's fortieth anniversary, Google Code announced the open-sourcing of code for the mission's command and lunar modules. Very cool indeed. This project creates opportunities for many people. CS historians can add this code to their record of computing and can now study computing at NASA in a new way. Grady Booch will be proud. He has been working for many years on the task of preserving code and other historic artifacts of our discipline that revolve around software.

Software archeologists can study the code to find patterns and architectural decisions that will help us understand the design of software better. What we learn can help us do more, just as the Apollo 11 mission prepared the way for future visitors to the moon, such as Charles Duke of Apollo 16 (pictured here).

This code could help CS educators convince a few students to assembly language programming seriously. This code isn't Java or even C, folks. Surely some young people are still mesmerized enough by space travel that they would want to dig in to this code?

As a person who appreciates assembly-level programming but prefers working at a higher level, I can't help but think that it would be fun to reverse-engineer these programs to code at a more abstract level and then write compilers that could produce equivalent assembly that runs on the simulator. The higher-level programs created in this way would be a great way for us to record the architecture and patterns we find in the raw source.

Reading this code and about the project that surrounds it, I am in awe of the scale of the programming achievement. For a computer scientist, this achievement is beautiful. I'd love to use this code to share the excitement of computing with non-computer scientists, but I don't know how. It's assembly, after all. I'm afraid that most people would look at this code and say, "Um, wow, very impressive" while thinking, "Yet another example of how computer science is beyond me."

If only those people knew that many computer scientists feel the same way. We are in awe. At one level, we feel like this is way over our heads, too. How could these programmers done so much with so little? Wow. But then we take a breath and realize that we have the tools we need to dig in and understand how this stuff works. Having some training and experience, we can step back from our awe and approach the code in a different way. Like a scientist. And anyone can have the outlook of a scientist.

When I wonder how could the programmers of the 1960s could have done so much with so little, I feel another emotion, too: sheepishness. How far have we as a discipline progressed in forty years? Stepping back from the sheepishness, I can see that since that time programmers have created some astoundingly complex systems in environments that are as harsh or harsher than the Apollo programmers faced. It's not fair to glorify the past more than it deserves.

But still... Wow. Revisiting this project forty years later ought to motivate all of us involved with computer science in 2009 -- software professionals, academics, and students -- to dream bigger dreams and tackle more challenging projects.

Evolution

The software engineering tip Every line of code is a user interface expresses several important ideas. Design matters. Names matter. Separating business logic from system wiring matters. Programs are read by people, so the code you write matters.

As I think about it, though, it embodies another important idea that is right up agile development's alley. We all know that programs tend to evolve over time, as user's needs and wants change. As developers, we also know that, as we build more systems in the same domain, we can evolve one-off programs toward a common framework. The framework generalizes over the separate programs, pulling the wiring out for maintenance in one place and for use in creating new programs in the same domain later.

What's interesting to me about the police database system described in that article is that it needed to evolve away from its built-in wiring and toward a more generic framework created in the outside world:

Specifically it wasn't Microsoft Access we had to worry about, but Ruby on Rails that provided 99% of all the infrastructure we'd built ourselves for an HTML-fronted database application.

It didn't have to be Rails, because web frameworks in other languages would probably have done as well for their shop. The point is, when the frameworks came along, their code was too tangled to easily take advantage of them. On the business side, another company could come along, replicate the business logic on top of one of the frameworks that provided the 99% infrastructure, and compete right away.

Is your design clean enough?

Time

I mentioned David Ogilvy's Confessions of an Advertising Man in a recent entry. Another passage from the book set of the alarm in my brain that watches for examples of shorter-than-usual iterations and continuous feedback:

Most young men in big corporations behave as if profit were not a function of time. When Jerry Lambert scored his first breakthrough with Listerine, he speeded up the whole process of marketing by dividing time into months. Instead of locking himself into annual plans, Lambert reviewed his advertising and his profits every month.

I guess that Lambert wasn't satisfied with halving his project length; he went straight to a factor of twelve. Feedback matters in the advertising game. In the same chapter of Confessions, Ogilvy talks about the importance of testing campaigns in front of real users. He claims that 24 out of 25 ideas will fail, so it's better to find out in testing than in production -- which both costs more and exposes the company to public failure. (Reputation matters a lot in the advertising game.)

Lambert's approach went further by monitoring projects that had already been deployed on a much shorter cycle and making decisions based on more frequent feedback about the projects. This reminded me of the approach suggested by Ricardo Semler in his talk Leading by Omission. Semler says that long-term plans, even yearly plans, are simply too large-grained to be useful. He advocates continuous feedback and monitoring.

Time is related to evolution, of course. When we operate at smaller units of time, we can evolve -- through both failure and success -- that much faster.

You don't need (or have to be) a special kind of leader, which Lambert and Semler seem to be, to benefit from this style. A disciplined process such as XP can guide you, and a team of developers operating from a set of agile principles can work to achieve similar goals.

An Addition to My News Aggregator

Thanks to John Cook, I came across the blog of Dan Meyers, a high school math teacher. Cook pointed to an entry with a video of Meyer speaking pecha kucha-style at OSCON. One of the important messages for teachers conveyed in this five minutes is Be less helpful. Learning happens more often when people think and do than when they follow orders in a well-defined script.

While browsing his archive I came across this personal revelation about the value of the time he was spending on his class outside of the business day:

I realize now that the return on that investment of thirty minutes of my personal time isn't the promise of more personal time later. ... Rather it's the promise of easier and more satisfying work time now.

Time saved later is a bonus. If you depend on that return, you will often be disappointed, and that feeds the emotional grind that is teaching. Kinda like running in the middle. I think it also applies more than we first realize to reuse and development speed in software.

Learning and Doing

One of the underlying themes in Meyers's writing seems to be the same idea in this line from Gerd Binnig, which I found at Physics Quote of Day:

Doing physics is much more enjoyable than just learning it. Maybe 'doing it' is the right way of learning ....

Programming can be a lot more fun than learning to program, at least the way we often try to teach it. I'm glad that so many people are working on ways to teach it better. In one sense, the path to better seems clear.

Knowing and Doing

One of the reasons I named by blog "Knowing and Doing" was that I wanted to explore the connection between learning, knowing, and doing. Having committed to that name so many years ago, I decided to stake its claim at Posterous, which I learned about via Jake Good. Given some technical issues with using NanoBlogger, at least an old version of it, I've again been giving some thought to upgrading or changing platforms. Like Jake, I'm always tempted to roll my own, but...

I don't know if I'll do much or anything more with Knowing and Doing at Posterous, but it's there if I decide that it looks promising.

A Poignant Convergence

Finally, a little levity laced with truth. Several people have written to say they liked the name of my recent entry, Sometimes, Students Have an Itch to Scratch. On a whim, I typed it into Translation Party, which alternately translates a phrase from English into Japanese and back until it reaches equilibrium. In only six steps, my catchphrase settles onto:

Sometimes I fear for the students.

Knowing how few students will try to scratch their own itches with their new-found power as a programmer, and how few of them will be given a chance to do so in their courses on the way to learning something valuable, I chuckled. Then I took a few moments to mourn.

An excerpt from an interview at Gaping Void:

Some days, the work is tedious, labour-intensive and as repetitive as a production line in a factory. ... The key is having a good infrastructure. ...

But none of it works without discipline. Early on in my career, I was told that success demanded one thing above all others: turning up. Turning up every bloody day, regardless of everything.

This was said by artist Hazel Dooney, but it could just as well been said by a programmer -- or a university professor. One thing I love about the agile software world is its willingness to build new kinds of tools to support the work of programmers. Isn't it ironic? Valuing people over tools makes having the right tools even more important.

Thanks to everyone who responded to my call for advice on what advice I should give to a student interested in studying at the university with a goal of becoming a web developer. People interpreted the article in lots of different ways and so were able to offer suggestions in many different ways.

In general, most confirmed the gist of my advice. Learn a broad set of technical skills from the spectrum of computer science, because that prepares one to contribute to the biggest part of the web space, or study design, because that's they part the techies tend to do poorly. A couple of readers filled me in on the many different kinds of web development programs being offered by community colleges and technical institutes. We at the university could never compete in this market, at least not as a university.

Mike Holmes wrote a bit about the confusion people have about computer science, with a tip of the hat to Douglas Adams. This confusion does play a role in prospective students' indecision about pursuing CS in college. People go through phases where they think of the computer as replacing an existing technology or medium: calculator, typewriter and printing press, sketchpad, stereo, television. Most of us in computer science seem to do one of two things: latch onto the current craze, or stand aloof from the latest trend and stick with theory. The former underestimates what computing can be and do, while the latter is so general that we appear not to care about what people want or need. It is tough to balance these forces.

In some twittering around my request, Wade Arnold tweeted about the technical side of the issue:

@wallingf Learn Java for 4 years to really know one language well. Then they will pick up php, ruby, or python for domain specific speed

The claim is that by learning a single language really well, a person really learns how to program. After that, she can learn other languages and fill in the gaps, both language-wise and domain-wise. This advice runs counter to what many, many people say, myself included: students should learn lots of different languages and programming styles in order to really learn how to program. I think Wade agrees with that over the long term of a programmer's career. What's unusual in his advice is the idea that a student could or should spend all four years of undergrad study mastering one language before branching out.

A lot of CS profs will dismiss this idea out of hand; indeed, one of the constant complaints one hears in certain CS ed circles is that too many schools have "gone Java" to the exclusion of all other languages and to the lasting detriment of their students. My department's curriculum has, since before I arrived, required students to study a single language for their entire first year, in an effort to help students learn one language well enough that they learn how to program before moving on to new languages and styles. When that language was, say, Pascal, students could pretty well learn the whole language and get a lot of practice using it. C is simple enough for that purpose, I suppose, but C++, Ada, and Java aren't. If we want students to master those languages at a comparable level, we might well need four years. I think that says more about the languages we use than about students learning enough programming in a year to be ready to generalize and fill in gaps with new languages and styles.

This entry has gotten longer than I expected, yet I have more to say. I'll write more in the days to come.

For the last couple of days, I've been speaking with a prospective non-traditional student who is seeking advice about what to study for a second bachelor's degree. His indecision about careers and school is worthy of its own post; I've seen it in so many other 20- and 30-somethings who are unhappy with their first careers and not sure of where to go next.

He wants to get into web development. What should he study? Well, he thinks he wants to get into web development, but he isn't sure exactly what it is. This is a common issue for prospective CS majors coming out of high school; they rarely know just what computer science is. It is apparently also an issue with the more visible discipline of web development. People see the products, but they are not sure at all how they are created. Is this a problem for budding electrical engineers or civil engineers?

This young man asked me for precise definitions of the terms "web design" and "web development", especially clear distinctions between the two, and a map from the terms onto specific undergrad majors. I explained that these are fuzzy terms, like so many of our best words and phrases. That didn't satisfy him much, because the fuzziness won't easily wipe away his indecision.

He is asking the head of the Computer Science department for advice, so I tell him what I think about web development, as fairly as I can, but with an unmistakeable fondness for my discipline. On the one hand, there is presentation, the web pages themselves and how they appear to users within a browser. On the other, there is the technology that underlies the web, the code and data that make it all work. To be versatile in the web game, and ultimately great, a person should learn both.

We do not offer a program in web development at my university, so a student must go to departments that teach the ideas and tools behind these facets of the web. I suggested graphic design with a little psychology for the presentation side, and computer science for the technology side. We have a excellent graphic design program here, and our CS program introduces students to programming and to the ideas of data structures, databases, algorithms, and information storage and retrieval that drive the web. We even teach an undergrad course in user interface design, which is the closest our major courses gets to presentation.

A freshman coming out of high school would do well to double-major in CS and graphic design. We have at least one such double-major now -- a dynamite student -- and a few others are thinking about taking this route. Majoring in both is a challenge, because it requires skills that are, on the surface, so unlike one another. I think the styles of thinking at the heart of these two disciplines are not so different after all. Besides, these majors don't just require skills; they help to develop them. It is a great double major.

However, most nontraditional students coming back to school are not interested in re-living the four- or five-year undergrad experience. They have lives to support, and often families, so they are usually focused on making a career change and getting on with it. As a result, one of these majors will have to do.

It seems to me that if you want to make a dent in the web world, you really want to know the technology and the ideas that make them work. Philip Greenspun teaches a software engineering course at MIT with web programming as its centerpiece, based in large part on the premise that this particular technology is so central to the experience of current CS grads when they get out in the world.

(I wonder what it would be like if I tried to teach Greenspun's course as my SE course?! My guess is that a lot of students would love it, a few would hate it, and the respective ratios for the faculty would be swapped.)

Not all students want to do the technology. I would rather students make this decision from a position of power -- knowing something about the technology -- than from a position of ignorance, but that's not always how the world works. Fine. Then the student can study graphic design and psychology and apply that knowledge to learning how to make web sites whose appearances sizzle. There is a shortage of great web sites out there, which tells me there may be a shortage of great web designers. Go for it.

I had to be honest. You can learn a lot about how to make websites at the tool level without giving any university a lot of money. With a high school degree and a few books on HTML, CSS, Javascript (three topics we teach in a course for non-majors), and maybe Flash (which we currently teach in an experimental course, also for non-majors), any reasonably intelligent person can become a competent web developer. This student already has a college degree, so with discipline and perseverance he could probably make this career shift on his own. There may be practical limitations on the career potential for some people who choose this path, but it's viable.

In at least two of my exchanges with this young man, I ran into a concern that I expected to encounter. It is the fear that rises to the top of almost every neophyte's mind whenever he or she is confronted with the words "computer science": Will I have to do much programming? I think he was asking as much about the jobs he would get in web development as about the CS courses he would have to take to get a major here.

My part of these conversations can be summed up in a mantra I now keep close to my heart:

You don't have to program; you get to program.

We need to do something to change the default expectation young people have about programming. Seriously.

This particular student is a reasonable young man. He is college-educated and earnest about finding a good career. He is also a little afraid. In my recent experience, I have encountered many like him, and I'd like to help them as best I can. So, let's get back to the original question, "What should a person study in college if she wants to get into web development?" Faithful readers: What should I tell them? What am I missing?

I often find myself at meetings with administrator types, where I am the only person who has spent much time in a classroom teaching university students, at least recently. When talk turns to teaching, I sometimes feel like a pregnant woman at an ob-gyn convention. They may be part of a university and may even have taught a workshop or specialty course, but they don't usually know what it's like to design and teach several courses at a time, over many semesters in a row. That doesn't always stop them from having deeply-held and strong opinions about how to teach. Having been a student once isn't enough to know what it's like to teach.

I have had similar experiences as a university professor among professional software developers. All have been students and have learned something about teaching and learning from their time in school. But their lessons are usually biased by their own experiences. I was a student once, too, but that prepared me only a bit for being a teacher... There are so many different kinds of people in a classroom, so many different kinds of student. Not very many are just like me. (Thank goodness!)

Some software developers have taught. Many give half- or full-day tutorials at conferences. Others teach week-long courses on specific topics. A few have even taught a university course. Even still, I often don't feel like we are talking about the same animal when we talk about teaching. Teaching a training course for professional developers is not the same as teaching a university course to freshmen or even seniors. Teaching an upper-division or graduate seminar bears little resemblance to an elective course for juniors, let alone a class of non-majors. Even teaching such a course as an adjunct can't deliver quite the same experience as teaching a full load of courses across the spectrum of our discipline, day in and day out for a few years in a row. The principle of sustainable pace pops up in a new context.

As with administrators, lack of direct experience doesn't always stop developers from having deeply-held and strong opinions about what we instructors should be doing in the classroom. It creates an interesting dialectic.

That said, I try to learn whatever I can from the developers with whom I'm able to discuss teaching, courses, content, and curricula. One of the reasons I so enjoy PLoP, ChiliPLoP, and OOPSLA is having an opportunity to meet reflective individuals who have thought deeply about their own experiences as students and who are willing to offer advice on how I can do better. But I do try to step back from their advice and put it into the context of what it's like to teach in a real university, not one we've invented. Some ideas sound marvelous in the abstract but die a grisly death on the banks of daily university life. Revolution is easy when the battlefield is in our heads.

When it comes to working with software developers, I am more concerned that they will feel like the pregnant woman when they discuss their area of expertise with me and my university colleagues. One of my goals is not to be "that guy" when talking about software development with developers. I hope and prefer to speak out of personal and professional experience, rather than a theory I read in a book or a blog, or something another professor told me.

What we teach needs to have some connection to what developers do and what our students will need to do when they graduate. There is a lot more to a CS degree than just cutting code, but when we do talk about building software, we should be as accurate and as useful as we can be. This makes teaching a course like software engineering a bigger challenge for most CS profs than the more theoretical or foundational material such as algorithms or even programming languages.

One prescription is the same as above: I listen and try to learn whatever I can from developers when we talk about building software. Conferences like PLoP, ChiliPLoP, and OOPSLA give me opportunities I would not have otherwise, and I listen to alumni tell me about what they do -- and how -- whenever I can. I still have to sift what I learn into the context of the university, but it makes for great raw material.

Another prescription is to write code and use the tools and practices the people in industry use. Staying on top of that fast-moving game gets harder all the time. The world software is alive and changing. We professors have to keep at it. Mostly, it's a matter of us staying alive, too.

A couple of weeks ago, I quoted Glenn Vanderburg as my Tweet of the Day. Now he has made his tweet the set-up line in a full-blown blog entry, Sharp vs. Blunt Instruments:

Weak developers will move heaven and earth to do the wrong thing. You can't limit the damage they do by giving them less powerful tools. They'll just swing the blunt tools harder.

I still agree with the sentiment, as well as the bigger point: Give people powerful tools, teach them what you can, and let them create. The best programmers will do amazing things; the rest will be none the worse off. If you help to create a culture that values learning and encourages responsibility for others, you may even find some of the weak growing into amazing programmers, too.

In trying to understand the role patterns and pattern languages play both in developing software and in learning to develop software, I often look for different angles from which to look at patterns. I've written the idea of patterns as descriptive grammar and the idea of patterns as a source of freedom in design. Both still seem useful to me as perspectives on patterns, and the latter is among the most-read articles on my blog. The notion of patterns-as-grammar also relates closely to one of the most commonly-cited roles that patterns play for the developer or learner, that of vocabulary for describing the meaningful components of a program.

This weekend, I read Brian Hayes's instructive article on compressive sensing, The Best Bits. Hayes talks about how it is becoming possible to imagine that digital cameras and audio recorders could record compressed streams -- say, a 10-megapixel camera storing a 3MB photo directly rather than recording 30MB and then compressing it after the fact. The technique he calls compressive sensing is a beautiful application of some straightforward mathematics and a touch of algorithmic thinking. I highly recommend it.

While reading this article, though, the idea of patterns as vocabulary came to mind in a new way, triggered initially by this passage:

... every kind of signal that people find meaningful has a sparse representation in some domain. This is really just another way of saying that a meaningful signal must have some structure or regularity; it's not a mere jumble of random bits.

Programs are meaningful signals and have structure and regularity beyond the jumble of seemingly random characters at the level of the programming level. The chasm between random language stuff and high-level structure is most obvious when working with beginners. They have to learn that structure can exist and that there are tools for creating it. But I think developers face this chasm all the time, too, whenever they dive into a new language, a new library, or a new framework. Where is the structure? Knowing it is there and seeing it are too different matters.

The idea of a sparse representation is fundamental to compression. We have to find the domain in which a signal, whether image or sound, can be represented in as few bits as possible while losing little or even none of the signal's information. A pattern language of programs does the same thing for a family of programs. It operates at a level (in Hayes' terms, in a domain) at which the signal of the program can be represented sparsely. By describing Java's I/O stream library as a set of decorators on a set of concrete streams, we convey a huge amount of information in very few words. That's compression. If we say nothing else, we have a lossy compression, in that we won't be able to reconstruct the library accurately from the sparse representation. But if we use more patterns to describe the library (such as Abstract Class and "Throw, Don't Catch"), we get a representation that pretty accurately captures the structure of the library, if not the bit-by-bit code that implements it.

This struck me as a useful way to think about what patterns do for us. If you've seen other descriptions of patterns as a means for compression, I'd love to hear from you.

I recently ran across a link to a Dan Bricklin article from a few years ago, Why Johnny can't program. (I love the web!) Bricklin discusses some of the practical reasons why more people don't program. As he points out, it's not so much that people can't program as that they won't or choose not to program. Why? Because the task of writing code in a textual language isn't fun for everyone. What Bricklin calls "typed statement" programming fails all of Don Norman's principles of good design: visibility, good conceptual model, good mappings, and full and continuous feedback. Other programming models do better on these counts -- spreadsheets, rule-based expert system shells, WYSIWYG editors in which users generate HTML through direct manipulation -- and reach a wider audience. Martin Fowler recently talked about this style, calling it illustrative programming.

I had an agile moment as I read this paragraph from Bricklin about why debugging is hard:

One of the problems with "typed-statement" systems is that even though each statement has an effect, you only see the final result. It is often unclear which statement (or interaction of statements) caused a particular problem. With a "Forms" or direct manipulation system, the granularity is often such that each input change has a corresponding result change.

When we write unit tests for our code at about the same time we write the code, we improve our programming experience by creating intermediate results that help us to debug. But there's more. Writing tests helps us to construct a conceptual model of the program we are writing. They make visible the intended state of the program, and help us to map objects and functions in the code onto the behavior of the program at run-time. When we take small steps and run our tests frequently, they give us full and continuous feedback about the state of our program. Best of all, this understanding is recorded in the tests, which are themselves code!

In some ways, test-driven programming may improve on styles where we don't type statements. By writing tests, we participate actively in creating the model of our program. We are not simply passive users of someone else's constraint engine or inference engine. We construct our understanding as we construct our program.

Then again, some people don't need or want to write the reasoning component, so we need to provide access to tools they can use to be productive. Spreadsheets did that for regular folks. Scripting languages do it for programmers. Some people complain about scripting languages because they lack type safety, hide details, and are too slow. But the fact is that programmers are people, too, and they want tools that put them into a flow. They want languages that hit them in their sweet spot.

Bricklin concludes:

From all this you can see that the way a system requires an author to enter instructions into the computer affects the likelihood of acceptance by regular people. The more constrained the instructions the better. The more the instructions are clearly tied to various results the better. The more obvious and quickly the results may be examined the better.

TDD does all this, and more. It makes professional programmers more productive by providing better cognitive support for mental tasks we have to perform anyway. If we use TDD properly as we teach people to program, perhaps it can help us hit the sweet spot for more people, even in a "typed statement" environment.

I've been buried in a big project on campus for the last few months. Yesterday, we delivered our report to the president. Ah, time to breathe, heading into a holiday weekend! Of course, next week I'll get back to my regular work. Department stuff. Cleaning my desk. And thinking about teaching software engineering this fall.

A bit of side reading found via my Twitter friends has me thinking about testing, and the role it will play in the course. In the old-style software engineering course, testing is a "stage" in the "process", which betrays a waterfall view of the world even when the instructor and textbook say that they encourage iterative development. But testing usually doesn't get much attention in such courses, maybe one chapter that describes the theory of testing and a few of the kinds of testing we need to do.

It seems to me that testing can take a bigger place in the course, if only because it exemplifies the sort of empiricism that we should all engage in as software developers. When we test, we run experiments to gather evidence that our program works as specified. We should adopt a similar mindset about how we build our programs. How do we know that our design is a good one? Or that our team is functioning well? Or that we are investing enough time and energy in writing tests and refactoring our code?

That's one reason I like Joakim Karlsson's post about the principle of locality in code changes. There may be ways that he can improve his analysis, but the most important thing about this post is that he analyzed code at all. He had a question about how code edits work, so he wrote a program to ask subversion repositories for the answer. That's so much better than assuming that his own hypothesis was correct, or that conventional wisdom was.

In regard to the testing process itself, Michael Feathers wrote a piece on "canalizing" design that points out a flaw in how we usually test our code. We write tests that are independent of one another in principle but that our test engines always run in the same order. This inadvertent weakness of sequential code creates an opportunity for programmers to write code that takes advantage of the implicit relationship between tests. But it's not really an advantage at all, because we then have dependencies in our code that we may not be aware of and which should not exist at all. Feathers suggests putting the tests in a set data structure and executing them them from there. At least then the code makes explicit that there is no implied order to the tests, which reminds the programmers who modify the code later that they should not depend on the order of test execution.

(I also like this idea for its suggestion that programs can and other should be dynamic structures, not dead sequences of text. Using a set of tests also moves us a step closer to making our code work well in a parallel environment. Explicit and implicit sequencing in programs makes it hard to employ the full power of multicore systems, and we need to re-think how we structure our programs if we want to break away from purely sequential machines. The languages guy in me sees some interesting applications of this idea in how write our compilers.)

Finally, I enjoyed reading Gojko Adzic's description of Keith Braithwaite's "TDD as if you mean it" exercise. Like the programming challenges I have described, it asks developers to take an idea to its extreme to break out of habits and to learn just how the idea feels and what it can give. Using tests to drive how the writing of code is more different from what most of us do than we usually realize. This exercise can help you to see just how different -- if you have an exercise leader like Keith to keep you honest.

However, I disagree with something Keith said in response to a comment about the relationship between TDD and functional programming:

I'm firmly convinced that sincere TDD leads one towards a functional style.

TDD will drive you to the style whose language you think.

There will be functional components to your solution to support the tests, and some good OOP has a functional feel. But in my experience you can end up with very nice objects in an object-oriented program as a result of faithfully-executed TDD.

Another of Braithwaite's comments saved the day, though. He credits Allan Watts for this line that captures his intent in designing exercises like this:

I came not as a salesman but as an entertainer. I want you to enjoy these ideas because I enjoy them.

Love this! He has a scholar's heart.

There is a lot more to testing that unit tests or regression testing. Finding ways to introduce students to the full set of ideas while also giving them a visceral sense of testing in the trenches is a challenge. I have to teach enough to prepare a general audience and also prepare students who will go on to take our follow-up course, Software Testing. That's a course that undergraduates at most schools don't have the opportunity to take, a strong point of our program. But that course can't be an excuse not to do testing well in the software engineering course. It's not a backstop; it's new ballgame.

Quick hits, for different values of x, of course, but also different values of "the day" I encountered them. I'm slow, and busier than I'd like.

Tweet of the Day

Courtesy of Glenn Vanderburg:

Poor programmers will move heaven and earth to do the wrong thing. Weak tools can't limit the damage they'll do.

Vanderburg is likely talking about professional programmers. I have experienced this truth when working with students. At first, it surprised me when students learning OOP would contort their code into the strangest configurations not to use the OO techniques they were learning. Why use a class? A fifty- or hundred-line method will do nicely.

Then, students learning functional programming would seek out arcane language features and workarounds found on the Internet to avoid trying out the functional patterns they had used in class. What could have been ten lines of transparent Scheme code in two mutually recursive functions became fifteen or more of the most painfully tortured C code wrapped in a thin veil of Scheme.

I've seen this phenomenon in other contexts, too, like when students take an elective course called Agile Software Development and go out of their way to do "the wrong thing". Why bother with those unit tests? We don't really need to try pair programming, so we? Refactor -- what's that?

This feature of programmers and learners has made me think harder trying to help them see the value in just trying the techniques they are supposed to learn. I don't succeed as often as I'd like.

Comic of the Day

Hammock dwellers, unite!

If only. If only. When does summer break start?

While reading about the fate of newspapers prior to writing my recent entry on whether universities are next, I ran across a blog entry by Dave Winer called If you don't like the news.... Winer had attended a panel discussion at the UC-Berkeley school of journalism. After hearing what he considered the standard "blanket condemnation of the web" by the journalists there, he was thinking about all the blogs he would love to have shown them -- examples of experts and citizens alike writing about economics, politics, and the world; examples of a new sort of journalism, made possible by the web, which give him hope for the future of ideas on the internet.

Here is the money quote for me:

I would also say to the assembled educators -- you owe it to the next generations, who you serve, to prepare them for the world they will live in as adults, not the world we grew up in. Teach all of them the basics of journalism, no matter what they came to Cal to study. Everyone is now a journalist. You'll see an explosion in your craft, but it will cease to be a profession.

Replace "journalism" with "computer science", and "journalist" with "programmer", and this statement fits perfectly with the theme of much of this blog for the past couple of years. I would be happy to say this to my fellow computer science educators: Everyone should now be a programmer. We'll see an explosion in our craft.

Will programming cease to be a profession? I don't think so, because there is still a kind and level of programming that goes beyond what most people will want to do. Some of us will remain the implementors of certain tools for others to use, but more and more we will empower others to make the tools they need to think, do, and maybe even play.

Are academic computer scientists ready to make this shift in mindset? No more so than academic journalists, I suspect. Are practicing programmers? No more so than practicing journalists, I suspect.

Purely by happenstance, I ran across another quote from Winer this week, one that expresses something about programming from the heart of a programmer:

i wasn't born a programmer.
i became one because i was impatient.
-- @davewiner

I suspect that a lot of you know just what he means. How do we cultivate the right sort of impatience in our students?

The first: Our local paper carries a parenting advice column by John Rosemond, an advocate of traditional parenting. In Wednesday's column, a parent asked how to handle a child who refuses to eat his dinner. Rosemond responded that the parents should calmly, firmly, and persistently expect the child to eat the meal -- even if it meant that the child went hungry that night by refusing.

[Little Johnny] will survive this ordeal -- it may take several weeks from start to finish -- with significantly lower self-esteem and a significantly more liberal palette, meaning that he will be a much happier child.

If you know Rosemond, you'll recognize this advice.

I couldn't help thinking about what happens when we adults learn a new programming style (object-oriented or functional programming), a new programming technique (test-driven development, pair programming), or even a new tool that changes our work flow (say, SVN or JUnit). Calm, firm, persistent self-discipline or coaching are often the path to success. In many ways, Rosemond's advice works more easily with 3- or 5-year-olds than college students or adults, because the adults have the option of leaving the room. Then again, the coach or teacher has less motivation to ensure the change sticks -- that's up to the learner.

I also couldn't help thinking how often college students and adults behave like 3- and 5-year-olds.

The second: Our paper also carries a medical advice column by a Dr. Gott, an older doctor who harkens back to an older day of doctor-patient relations. (There is a pattern here.) In Wednesday's column, the good doctor said about a particular diagnosis:

There is no laboratory or X-ray test to confirm or rule out the condition.

My first thought was, well, then how do we know it exists at all? This a natural reaction for a scientist -- or pragmatist -- to have. I think this means that we don't currently have a laboratory or X-ray test for the presence or absence of this condition. Or there may be another kind of test that will tell us whether the condition exists, such as a stress tests or an MRIs.

Without any test, how can we know that something is? We may find out after it kills the host -- but then we would need a post-mortem test. While the patient lives, there could be a treatment regimen that works reliably in face of the symptoms. This could provide the evidence we need to say that a particular something was present. But if the treatment fails, can we rule out the condition? Not usually, because there are other reasons that the treatment fails.

We face a similar situation in software with bugs. When we can't reproduce a bug, at least not reliably, we have a hard time fixing it. Whether we know the problem exists depends on which side of the software we live... If I am the user who encounters the problem, I know it exists. If I'm the developer, then maybe I don't. It's easy for me as developer to assume that there is something wrong with the user, not my lovingly handcrafted code. When the program involves threading or a complex web of interactions among several systems, we are more inclined to recognize that a problem exists -- but which problem? And where? Oh, to have a test... I can only think of two software examples of reliable treatment regimens that may tell us something was wrong: rebooting the machine and reinstalling the program. (Hey to Microsoft.). But those are such heavy-handed treatments that they can't give us much evidence about a specific bug.

There is, of course, the old saying of TDD wags: Code without a test doesn't exist. Scoff at that if you want, but it is a very nice guideline to live by.

To close, here are my favorite new phrases from stuff I've been reading:

• from the Xark article quoted in my previous article: forward into the now


• from the David Allen book quoted in How To Be Invincible: creative discomfort for progress

Expect to see these jewels used in an article sometime soon.

Brian Marick tweeted about his mini-blog post Pay me until you're done, which got me to thinking. The idea is something like this: Many agile consultants work in an agile way, attacking the highest-value issue they can in a given situation. If the value of the issues to work on decreases with time, there will come a point at which the consultant's weekly stipend exceeds the value of the work he is doing. Maybe the client should stop buying services at that point.

My first thought was, "Yes, but." (I am far too prone to that!)

First, the "yes": In the general case of consulting, as opposed to contract work, the consultant's run will end as his marginal effect on the company approaches 0. Marick is being honest about his value. At some point, the value of his marginal contribution will fall below the price he is charging that week. Why not have the client end the arrangement at that point, or at least have the option to? This is a nice twist on our usual thinking.

Now for the "but". As I tweeted back this feels a bit like Zeno's Paradox. Marick the consultant covers not half the distance from start to finish each week, but the most valuable piece of ground remaining. With each week, he covers increasingly less valuable distance. So our consultant, cast in the role of Achilles, concedes the race and says, okay, so stop paying me.

This sounds noble, but remember: Achilles would win the race. We unwind Zeno's Paradox when we realize that the sum of an infinite series can be a finite number -- and that number may be just small enough for Achilles to catch the tortoise. This works only for infinite series that behave in a particular way.

Crazy, I know, but this is how the qualification of the "yes" arose in my mind. Maybe, the consultant helps to create a change in his client that changes the nature of the series of tasks he is working on. New ideas might create new or qualitatively different tasks to do. The change created may change the value of an existing task, or reorder the priorities of the remaining tasks. If the nature of the series changes, it may cause the value of the series to change, too. If so, then the client may well want to keep the consultant around, but doing something different than the original set of issues would have called for.

Another thought: Assume that the conditions that Marick described do hold. Should the compensation model be revised? He seems to be assuming that the consultant charges the same amount for each week of work, with the value of the tasks performed early being greater than that amount and the value of the tasks performed later being less than that amount. If that is true,then early on the consultant is bringing in substantially more value than he costs. If the client pulls the plug as soon as the value proposition turns in its favor, then the consultant ends up receiving less than the original contract called for yet providing more than average value for the time period. If the consultant thinks that is fair, great. What if not? Perhaps the consultant should charge more in the early weeks, when he is providing more value, than in later week? Or maybe the client could pay a fee to "buy out" the rest of the contract? (I'm not a professional consultant, so take that into account when evaluating my ideas about consultant compensation...)

And another thought: Does this apply to what happens when a professor teaches a class? In a way, I think it does. When I introduce a new area to students, it may well be the case that the biggest return on the time we spend (and the biggest bang for the students' tuition dollars) happens in the first weeks. If the course is successful, then most students will become increasingly self-sufficient in the area as the semester goes on. This is more likely the case for upper-division courses than for freshmen. What would it be like for a student to decide to opt out of the course at the point where she feels like she has stopped receiving fair value for the time being spent? Learning isn't the same as a business transaction, but this does have an appealing feel to it.

The university model for courses doesn't support Marick's opt-out well. The best students in a course often reach a point where they are self-sufficient or nearly so, and they are "stuck". The "but" in our teaching model is that we teach an audience larger than one, and the students can be at quite different levels in background and understanding. Only the best students reach a point where opting out would make sense; the rest need more (and a few need a lot more -- more than one semester can offer!).

The good news is that the unevenness imposed by our course model doesn't hurt most of those best students. They are usually the ones who are able to make value out of their time in the class and with the professor regardless of what is happening in the classroom. They not only survive the latency, but thrive by veering off in their own direction, asking good questions and doing their own programming, reading, thinking outside of class. This way of thinking about the learning "transaction" of a course may help to explain another class of students. We all know students who are quite bright but end up struggling through academic courses and programs. Perhaps these students, despite their intelligence and aptitude for the discipline, don't have the skills or aptitude to make value out of the latency between the point they stop receiving net value and the end of the course. This inability creates a problem for them (among them, boredom and low grades). Some instructors are better able to recognize this situation and address it through one-on-one engagement. Some would like to help but are in a context that limits them. It's hard to find time for a lot of one-on-one instruction when you teach three large sections and are trying to do research and are expected to meet all of the other expectations of a university prof.

Sorry for the digression from Marick's thought experiment, which is intriguing in its own setting. But I have learned a lot from applying agile development ideas to my running. I have found places where the new twist helps me and others where the analogy fails. I'm can't shake the urge to do the same on occasion with how we teach.

One of the challenges every beginner faces is learning the subtle judgments they must make. How much time will it take for us to implement this story? Should I create a new kind of object here? Estimation and other judgments are a challenge because the beginner lacks the "instinct" for making them, and the environment often does provide enough data to make a clear-cut decision.

I've been living with such a beginner's mind this week on my morning runs. Tuesday morning I started with a feeling of torpor and was sure I'd end with a slow time. When I finished, I was surprised to have run an average time. On Wednesday morning, I felt good yet came in with one of my slowest times for the distance ever. This morning, my legs were stiff, making steps seem a chore. I finished in one of my better times at this distance since working my mileage back up.

My inaccurate judgments flow out of bad instincts. Sometimes, legs feel slow and steps a challenge because I am pushing. Sometimes, I run with ease because I'm not running very hard at all!

At this stage in my running, bad instincts are not a major problem. I'm mostly just trying to run enough miles to build my aerobic base. Guessing my pace wrong has little tangible effect. It's mostly just frustrating not to know. Occasionally, though, the result is as bad as the judgment. Last week, I ran too fast on Wednesday after running faster than planned on Tuesday. I ended up sick for the rest of the week and missed out on 8-10 miles I need to build my base. Other times, the result goes the other way, as when I turned in a best-case scenario half-marathon in Indianapolis. Who knew? Certainly not me.

So, inaccurate instincts can give good or bad results. The common factor is unpredictability. That may be okay when running, or not; in any case, it can be a part of the continuous change I seek. But unpredictability in process is not so okay when I am developing software. Continuous learning is still good, but being wrong can wreak havoc on a timeline, and it can cause problems for your customer.

Bad instincts when estimating my pace wasn't a problem two years, though it has been in my deeper past. When I started running, I often felt like an outsider. Runners knew things that I didn't, which made me feel like a pretender. They had instincts about training, eating, racing, and resting that I lacked. But over time I began to feel like I knew more, and soon -- imperceptibly I began to feel like a runner after all. A lot -- all? -- of what we call "instinct" is developed, not inborn. Practice, repetition, time -- they added up to my instincts as a runner.

Time can also erode instinct. A lack of practice, a lack of repetition, and now I am back to where I was several years ago, instinct-wise. This is, I think, a big part of what makes learning to run again uncomfortable, much as beginners are uncomfortable learning the first time.

One of the things I like about agile approaches to software development is their emphasis on the conscious attention to practice. They encourage us to reflect about our practice and look for ways to improve that are supported by experience. The practices we focus on help us to develop good instincts: how much time it will take for us to implement a story, when to write -- and not write -- tests, how far to refactor a system to prepare for the next story. Developing accurate and effective instinct is one way we get better, and that is more important than being agile.

The traditional software engineering community thinks about this challenge, too. Watts Humphrey created the Personal Software Process to help developers get a better sense of how they use their time and to use this sense to get better. But, typically, the result feels so heavy, so onerous on the developers it aims to help, that few people are likely to stick with it when they get into the trenches with their code.

An aside: This reminds me of conversations I had with students in my AI courses back in the day. I asked them to read Alan Turing's classic Computing Machinery and Intelligence, and in class we discussed the Turing Test and the many objections Turing rebuts. Many students clung to the notion that a computer program could never exhibit human-like intelligence because humans lacked "gut instinct" -- instinct. Many students played right into Turing's rebuttal yet remained firm; they felt deeply that to be human was different. Now, I am not at ease with scientific materialism's claim that humans are purely deterministic beings, but the scientist in me tells me to strive for natural explanations of as much of every phenomenon as possible. Why couldn't a program develop a "gut feeling"? To the extent that at least some of our instincts are learned responses, developed through repetition and time, why couldn't a program learn the same instincts? I had fun playing devil's advocate, as I always do, even when I was certain that I was making little progress in opening some students' minds.

In your work and in your play, be aware of the role that practice, repetition, and time play in developing your instincts. Do not despair that you don't have good instincts. Work to develop them. The word missing from your thought is "yet". A little attention to your work, and a lot of practice, will go a long way. Once you have good instincts, cherish them. They give us comfort and confidence. They make us feel powerful. But don't settle. The same attention and practice will help you get better, to grow as a developer or runner or whatever your task.

As for my running, I am certainly glad to be getting stronger and to be able to run faster than I expect. Still, I look forward to the feeling of control I have when my instincts are more reliable. Unpredictable effort leads to unpredictable days.

Sometimes it pays to keep reading. Last time, I commented on breaking rules and mentioned a thread on the XP mailing list. I figured that I had seen all I needed there and was on the verge of skipping the rest. Then I saw a message from Laurent Bossavit and decided to read. I'm not surprised to learn something from Laurent; I have learned from him before.

Laurent's note introduced me to the legal term bright line. In the law, a bright-line rule is...

... a clearly defined rule or standard, composed of objective factors, which leaves little or no room for varying interpretation. The purpose of a bright-line rule is to produce predictable and consistent results in its application.

As Laurent says, Bright lines are important in situations where temptations are strong and the slope particularly steep, a well-known example is alcoholics' high vulnerability to even small exceptions. Test-driven development, or even writing tests soon after code and thus maintaining a complete suite of automated tests, requires a bright line for many developers. It's too easy to slide back into old habits, which for most developers are much older and stronger. Staying on the right side of the line may be the only practical way to Live Right.

This provides a useful name for what teachers often do in class: create bright lines for students. When students are first learning a new concept, they need to develop a new habit. A bright-line rule -- "Thou shalt always write a test first." or "Thou shalt write no line of code outside of a pair." -- removes from the students' minds the need to make a judgment that they are almost always not prepared to make yet: "Is this case an exception?" While learning, it's often better to play Three Bears and overdo it. This gives your mind a chance to develop good judgment through experience.

(For some reason, I am reminded of one way that I used to learn to play a new chess opening. I'd play a bazillion games of speed chess using it. This didn't train my mind to think deeply about the positions the opening created, but it gave me a bazillion repetitions. I soon learned a lot of patterns that allowed me to dismiss many bad alternatives and focus my attention on the more interesting positions.)

I often ask students to start with a bright line, and only later take on the challenge of a balancing test. It's better to evolve toward such complexity, not try to start there.

The psychological benefits of a bright-line test are not limited to beginners. Just as alcoholics have to hold a hard line and consider every choice consciously every day, some of us need a good "Thou shalt.." or "Thou shalt not..." in certain cases. As much as I like to run, I sometimes have to force myself out of bed at 5:00 AM or earlier to do my morning work-out. Why not just skip one? I am a creature of habit, and skipping even one day makes it even harder to get up the next, and the difficulty grows until I have a new habit.

(This has been one of the most challenging parts of trying to get back up to my old mileage after several extended breaks last year. I am proud finally to have done all five of my morning runs last week -- no days off, no PM make-ups. A new habit is in formation.)

If you know you have a particular weakness, draw a bright line for yourself. There is no shame in that; indeed, I'd say that it shows professional maturity to recognize the need and address it. If you need a bright line for everything, that may be a problem...

Sometimes, I adopt a bright line for myself because I want everyone on the team to follow a practice. I may feel comfortable exercising judgment in the gray area but not feel the rest of the team is ready. So we all play by the rules rather than discuss every possible judgment call. As the team develops, we can begin having those discussions. This is similar to how I teach many practices.

This may sound too controlling to you, and occasionally a student will say as much. But nearly everyone in class benefits from taking the more patient road to expertise. Again, from Laurent:

Rules which are more ambiguous and subtle leave more room for various fudge factors, and that of course can turn into an encouragement to fudge, the top of a slippery slope.

Once learners have formed their judgment, they are ready to balance forces. Until then, most are more likely to backslide out of habit than to make an appropriate choice to break the rule. And time spent arguing every case before they are ready is time not spent learning.

Shh.

I have a secret.

When I am writing a program, I will on occasion add a new piece of functionality without writing a test.

I am whispering because I have seen the reaction on the XP mailing list and on a number of blogs that Kent Beck received to his recent article, To Test or Not to Test? That's a Good Question. In this short piece, Kent describes his current thinking that, like golf, software development may have "long game" and "short game", which call for different tools and especially mentalities. One of the differences might be whether one is willing to trade automated testing for some other value, such as delivering a piece of software sooner.

Note that Kent did not say that in the long game he chooses not to test his code; he simply tested manually. He also didn't say that he plans never to write the automated tests he needs later; he said he would write them later, either when he has more time or, perhaps, when he has learned enough to turn 8 hours of writing a test into something much shorter.

Many peoples' public reactions to Kent's admission have been along these lines: "We test you to make this decision, Kent, but we don't trust everyone else. And by saying this is okay, you will contribute to the delinquency of many programmers." Now you know why I need to whisper... I am certainly not in the handful of programmers so good that these folks would be willing to excuse my apostasy. Kent himself is taking a lot of abuse for it.

I have to admit that Kent's argument doesn't seem that big a deal to me. I may not agree with everything he says in his article, but at its core he is claiming only that there is a particular context in which programmers might choose to use their judgment and not write tests before or immediately after writing some code. Shocking: A programmer should use his or her judgment in the course of acting professionally. Where is the surprise?

One of the things I like about Kent's piece is that he helps us to think about when it might be useful to break a particular rule. I know that I'll be breaking rules occasionally, but I often worry that I am surrendering to laziness or sloppiness. Kent is describing a candidate pattern: In this context, with these goals, you are justified in breaking this rule consciously. We are balancing forces, as we do all the time when building anything. We might disagree with the pattern he proposes, but I don't understand why developers would attack the very notion of making a trade-off that results in breaking a rule.

In practice, I often play a little loose with the rules of XP. There are a variety of reasons that lead me to do so. Sometimes I pay for not writing a test, and when I do I reflect on what about the situation made the omission so dangerous. If the only answer I can offer is "You must write the test, always.", then I worry that I have moved from behaving like a professional to behaving like a zealot. I suspect that a lot of developers make similar trade-offs.

I do appreciate the difficulty this raises for those of us who teach XP, whether at universities or in industry. If we teach a set of principles as valuable, what happens to our students' confidence in the principles when we admit that we don't follow the rules slavishly? Well, I hope that my students are learning to think, and that they realize any principle or rule is subject to our professional judgment in any given circumstance.

Of course, in the context of a course, I often ask students to follow the rules "slavishly", especially when the principles in question require a substantial change in how they think and behave. TDD is an example, as is pair programming. More broadly, this idea applies when we teach OOP or functional programming or any other new practice. (No assignment statements or sequences until Week 10 of Programming Languages!) Often, the best way to learn a new practice is to live it for a while. You understand it better then than you can from any description, especially how it can transform the way you think. You can use this understanding later when it comes to apply your judgment about potential trade-offs.

Even still, I know that, no matter how much an instructor encourages a new practice and strives to get students to live inside it for a while, some students simply won't do it. Some want to but struggle changing their habits. I feel for them. Others willfully choose not to try the something new and deny themselves the opportunity to grow. I feel for them, too, but in a different way.

Once students have had a chance to learn a set of principles and to practice them for a while, I love to talk with them about choices, judgment, and trade-offs. They are capable of having a meaningful discussion then.

It's important to remember that Kent is not teaching novices. His primary audience is professional programmers, with whom he ought to be able to have a coherent conversation about choices, judgment, and trade-offs. Fortunately, a few folks on the P list have entertained the "long game versus short game" claim and related their own experiences making these kind of decisions on a daily basis.

If we in the agile world rely on unthinking adherence to rules, then we are guilty of proselytizing, not educating. Lots of folks who don't buy the agile approaches love when they see examples of this rigidity. It gives them evidence to support their tenuous position about the whole community. From all of my full-time years in the classroom, I have learned that perhaps the most valuable asset I can possess is my students' trust in my goals and attitudes. Without that, little I do is likely to have any positive effect on them.

Kent's article has brought to the surfaced another choice agilistas face most every day: the choice between dogma and judgment. We tend to lose people when we opt for unthinking adherence to a rule or a practice. Besides, dogmatic adherence is rarely the best path to getting better every day at what we do, which is, I think one of the principles that motivate the agile methods.

I am coming to a newfound respect for Robert's Rules of Order these days. I've usually shied away from that level of formality whenever chairing a committee, but I've experienced the forces that can drive a group in that direction.

For the last year, I have been chairing a campus-wide task force. Our topic is one on which there are many views on campus and for which there is not currently a shared vision. As a result, we all realized that our first priority was communication: discussing key issues, sharing ideas, and learning what others thought. I'll also say that I have learned a lot about what I think from these discussions. I've learned a lot about the world that lies outside of my corner of campus.

With sharing ideas and building trust as our first goals, I kept our meetings as unstructured as possible, even allowing conversations to drift off topic at times. That turned out well sometimes, when we came to a new question or a new answer unexpectedly.

We are nearing the end of our work, trying to reach closure on our descriptions and recommendations. This is when I see forces pushing us toward more structure. It is easy to keep talking, to talk around a decision so much that we find ourselves doubting a well-considered result, or even contradicting the it. At this point, we are usually cover well-trod ground. A little formality -- motion, second, discussion, vote, repeat -- may help. At least I now have some first hand experience of what might have led Mr. Robert to define his formal set of rules.

It occurs to me that Robert's Rules are a little like the heavyweight methodologies we often see in the software development world. We agile types are sometimes prone to look down on big formal methodologies as obviously wrong: too rigid, too limiting, too unrealistic. But, like the Big Ball of Mud, these methodologies came into being for a reason. Most large organizations would like to ensure some level of consistency and repeatability in their development process over time. That's hard to do when you have a 100 or a 1000 architects, designers, programmers, and testers. A natural tendency is to formalize the process in order more closely to control it. If you think you value safety more than discovery, or if you think you can control the rate of change in requirements, then a big process looks pretty attractive.

Robert's Rules looks like a solution to a similar problem. In a large group, the growth in communication overhead can outpace the value gained by lots of free-form discussion. As a group grows larger, the likelihood of contention grows as well, and that can derail any value the group might gain from free-form discussion. As a group reaches the end of its time together, free-form discussion can diverge from consensus. Robert's Rules seek to ensure that everyone has a chance to talk, but that the discussion more reliably reach a closing point. They opt for safety and lowering the risk of unpredictability, in lieu of discovery.

Smaller teams can manage communication overhead better than large ones. This is one of the key ideas behind agile approaches to software development: keep teams small so that they can learn at the same time they are making steady process toward a goal. Agile approaches can work in large organizations, too, but developers need to take into account the forces at play in larger and perhaps more risk-averse groups. That's where the sort of expertise we find in Jutta Eckstein's Agile Software Development in the Large comes in so handy.

While I sense the value of running a more structured meeting now, I don't intend to run any of my task force or faculty meetings using Robert's Rules any time soon. But I will keep in mind the motivation behind them and try to act in the spirit of a more directed discussion when necessary. I would rather still value people and communication over rules and formalisms, to the greatest extent possible.

Earlier today I listened to a TED talk by Tony Robbins. One snippet stood out. Here is a paraphrase, in part to clean up the language (because that's how I roll):

If I ask you whether you like variety, you'll say yes. Baloney. You like surprises you want. The others, you call problems.

Some people are better than others at accepting the surprises that they don't want. Perhaps that is why Robbins's anecdote reminded me of a story I read last summer in a book by John G. Miller called QBQ! The Question Behind the Question. The story, perhaps fictional, tells of a father and young daughter out for a fun plane ride one day, with dad behind the controls. When the plane's engine dies unexpectedly, dad turns to his daughter and says, calmly, I'm going to need to fly the plane differently.

I don't make generally New Year's resolutions, but when I am next tempted, I'll probably think again of this story. I want to be that guy, and I'm not.

----

(Quick book review: QBQ! is pretty standard for this genre of business self-help lit. It says a lot of things we all should already know, and probably do. But there are days when some of us need a reminder or a little pep talk. This book is full of short pep talks. It's a quick read and good enough at its task, as long as you remember that unless you change your behavior books like these are nothing but empty calories. A bit like software design methodologies.)

Though final grades are not all yet submitted, the semester is over. We made adjustments to the specification in my compilers course, and the students were able to produce a compiler that produced compilable, executable Java code for a variety of source programs. For the most part, the issues we discussed most at their finals week demo dealt with quality control. We found some programs that confounded their parser or code generator, which were evidence of bits of complexity they had not yet mastered. There is a lesson to be learned: theory and testing often take a back seat to team dynamics and practices. Given the complexity of their source language, I was not too disappointed with their software, though I think this team fell short of its promise. I have been part of teams that have fallen similarly short and so can empathize.

So, what is the verdict on a couple of new ideas we tried this semester: letting the team design its own source language and working in a large team, of six? After their demo, we debriefed the project as a group, and then I asked them to evaluate the project and course in writing. So I have some student data on which to draw as well as my own thoughts.

On designing their own language: yes, but. Most everyone enjoyed that part of the project, and for some it was their favorite activity. But the students found themselves still churning on syntax and semantics relatively late into the project, which affected the quality and stability of their parser. We left open the possibility of small changes to the grammar as they learned more about the language by implementing it, but this element of reality complicated their jobs. I did not lock down the language soon enough and left them making decisions too late in the process.

One thing I can do the next time we try this is to put a firmer deadline on language design. One thing thing that the students and I both found helpful was writing programs in the proposed language and discussing syntactic and semantic language issues grounded in real code. I think I'll build a session or two of this into the course early, before the drop-dead date for the grammar, so that we can all learn as much as we can about the language before we proceed on to implementing it.

We also discussed the idea of developing the compiler in a more agile way, implementing beginning-to-end programs for increasing subsets of the language features until we are done. This may well help us get better feedback about language design earlier, but I'm not sure that it addresses the big risks inherent in letting the students design their own language. I'll have to think more on this.

On working is a team of size six: no. The team members and I were unanimous that a team of size six created more problems than it solved. My original thinking was that a larger team would be better equipped to do the extra work introduced by designing their own language, which almost necessarily delayed the start of the compiler implementation. But I think we were bitten by a preemptive variation of Brooks's Law -- more manpower slowed them down. Communication overhead goes up pretty quickly when you move from a team of three to a team of six, and it was much harder for the team to handle all of its members' ideas effectively. This might well be true for a team of experienced developers, but for a team of undergrads working on their first collaborative project of this scale, it was an occasional show-stopper. I'll know better next time.

As an aside, one feature the students included in the language they designed was first-class functions. This clearly complicated their syntax and their implementation. I was pleased that they took the shot. Even after the project was over and they realized just how much extra work first-class functions turned out to be, the team was nearly unanimous in saying that, if they could start over, they would retain that feature. I admire their spunk and their understanding of the programming power this feature gave to their language.

When I teach programming languages, we discuss the concepts of static and dynamic scoping. Scheme, like most languages these days, is statically scoped. This means that a variable refers to the binding that existed when the variable was created. For example,

> (define f
    (let ((i 100))
      (lambda (x)
        (+ x i))))
> (define i 1)
> (f 1)
101

This displays 101, not 2, because the reference to i in the body of function f is to the local variable i that exists when the function was created, not to the i that exists when the function is called. If the interpreter looked to the calling context to find its binding for i, that would be an example of dynamic scope, and the interpreter would display 2 instead.

Most languages use static typic these days for a variety of reasons, not the least of which is that it is easier for programmers to reason about code that is statically scoped. It is also easier to decompose programs and create modules that programmers can understand easily and use reliably.

In my course, when looking for an example of a dynamically-scoped language, I usually refer to Common Lisp. Many old Lisps were scoped dynamically, and Common Lisp gives the programmer the ability to define individual variables as dynamically-scoped. Lisp does not mean much to students these days, though. If I were more of a Perl programmer, I would have known that Perl offers the same ability to choose dynamic scope for a particular variable. But I'm not, so I didn't know about this feature of the language until writing this entry. Besides, Perl itself is beginning to fade from the forefront of students' attention these days, too. I could use an example closer to my students' experience.

A recent post on why Python does not optimize tail calls brought this topic to mind. I've often heard it said that in Python closures are "broken", which is to say that they are not closures at all. Consider this example drawn from the linked article:

IDLE 1.2.1      
>>> def f(x):
    if x > 0:
       return f(x-1)
    return 0;

>>> g = f
>>> def f(x):
    return x

>>> g(5)
4

g is a function defined in terms of f. By the time we call g, f refers to a different function at the top level. The result is something that looks a lot like dynamic scope.

I don't know enough about the history of Python to know whether such dynamic scoping is the result of a conscious decision of the language designer or not. Reading over the Python history blog, I get the impression that it was less a conscious choice and more a side effect of having adopted specific semantics for other parts of the language. Opting for simplicity and transparency as an overarching sometimes means accepting their effects downstream. As my programming languages students learn, it's actually easier to implement dynamic scope in an interpreter, because you get it "for free". To implement static scope, the interpreter must go to the effort of storing the data environment that exists at the time a block, function, or other closure is created. This leads to a trade-off: a simpler interpreter supports programs that can be harder to understand, and a more complex interpreter supports programs that are easier to understand.

So for now I will say that dynamic scope is a feature of Python, not a bug, though it may not have been one of the intended features at the time of the language's design.

If any of your current favorite languages use or allow dynamic scope, I'd love to hear about it -- and especially whether and how you ever put that feature to use.

We have fallen behind in my compilers course. You may recall that before the semester I contemplated some changes in the course, including letting letting the students design their own language. My group of six chose that route, and as a part of that choice decided to work as a team of six, rather than in pairs or threes. This was the first time for me to have either of these situations in class, and I was curious to see how it would turn out.

Designing a language is tough, and even having lots of examples to work from, both languages and documents describing languages, is not enough to make it easy. We took a little longer than I expected. Actually, the team met its design deadline (with no time to spare, but...), but then followed a period of thinking more about the language. We both needed to come to a better understanding of the implications of some of their design decisions. Over time they changed their definition, sometimes refining and sometimes simply making the language different. This slowed the process of starting to implement the language and caused a few false starts in the scanner and parser.

Such bumps are a natural part of taking on the tougher problem of creating the language, so I don't mind that we are behind. I have learned a few things to do differently the next time a compiler class chooses this route. Working as a team of six increases the communication overhead they face, so I need to do a better job preparing them for the management component of such a large development project. It's hard for a team to manage itself, either through specific roles that include a nominal team leader or through town-hall style democracy. As the instructor, I need to watch for moments when the team needs me to take the rudder and guide things a bit more closely. Still, I think that this has been a valuable experience for the students. When they get out into industry, they will see successes and failures of the sort they've created for themselves this semester.

Still, things have gone reasonably well. It's almost inevitable that occasional disagreements about technical detail or team management will arise. People are people, and we are all hard to work with sometimes. But I've been happy with the attitude that all have brought to the project. I think all have shown a reasonable degree of commitment to the project, too, though they may not realize yet just what sort of commitment getting a big project like this done can require.

I have resisted the urge to tell (or re-tell?) the story of my senior team project: a self-selected team of good programmers and students who nonetheless found ways to fall way behind their development schedule. We had no way to change the scope of the system, struggled mightily in the last weeks of the two-term project, and watched our system crash on the day of the acceptance test. The average number of hours I spent on this project during its second term? 62 hours. And that was while taking another CS course, two accounting courses, and a numerical analysis course -- the final exam for which I have literally no recollection of at all, because by that time I was functioning on nearly zero sleep for days on end. This story probably makes me sound crazy -- not committed, but in need of being committed. Sometimes, that's what a project takes.

On the technical side, I will do more next time to accelerate our understanding of the new language and our fixing of the definition. One approach I'm considering is early homework assignments writing programs in the new language, even before we have a scanner or parser. This causes us all to get concrete sooner. Maybe I will offer extra-credit points to students who catch errors in the spec or in others students' programs. I'll definitely give extra-credit for catching errors in my programs. That's always fun, and I make a perfect foil for the class. I am sure both to make mistakes and to find holes in their design or their understanding of it.

But what about this semester? We are behind, with three weeks to D-Day. What is the best solution?

The first thing to recognize is that sometimes this sort of thing happens. I do not have the authority to implement a death march, short of becoming an ineffective martinet. While I could try telling students that they will receive incompletes until the project is finished, I don't really have the authority to push the deadline of the project beyond the end of our semester.

The better option is one not made available to my project team in school, but which we in the software world now recognize as an essential option: reduce the scope of the project. The team and I discussed this early in the week. We can't do much to make the language smaller, because it is already rather sparse in data types, primitive operators, and control structure. The one thing we could drop is higher-order procedures, but I was so please when they included this feature that I would feel bad watching it drop out now. But that would not really solve our problem. I am not sure they could complete a compiler for the rest of the language in time anyway.

We decided instead to change the target language from JVM bytecodes to Java itself. This simplifies what remains for them quite a bit, but not so much that it makes the job easy. The big things we lose are designing and implementing a low-level run-time system and emitting machine-level code. The flip side is that we decided to retain the language's support for higher-order procedures, which is not trivial to implement in generated Java code. They'll still get to think about and implement closures, perhaps using anonymous inner classes to implement function arguments and results.

This results in a different challenge, and a change in the experience the students will have. The object lesson is a good one. We have made a trade-off, and that is the nature of life for programmers. Change happens, and things don't always proceed according to plan. So we adapt and do the best we can. We might even spend 60 hours one week working on our project!

For me, the biggest effect of the change is on our last two and a half weeks of lecture. Given where we are and what they will be doing, what do they most need to learn? What ideas and techniques should they see even if they won't use them in their compilers? I get to have some fun right up to the end, too.

Last evening, Mike Feathers tweeted a provocative idea: The world might be better if all code disappeared at a fix age and we had to constantly rewrite it. Heck, he could even write a tool to seek and destroy all code as it reaches the age of three months. Crazy, huh?

Maybe this idea is not so crazy. At my university and most other places, hardware is on a 3- or 4-year "replacement cycle". Whether we need new computers in our student labs, we replace them on a schedule. Why? Because we recognize that hardware reaches a natural "end of life". Using it beyond that time means that we carry an ever-increasing risk that it will fail. Rather than let it fail and be caught without for a short while, we accept the upfront cost of replacing it with newer, more reliable, better equipment. The benefit is piece of mind, and more reliable performance.

Maybe we should recognize that software can be like hardware. It reaches a natural "end of life" -- not because physical components wear out, but because the weight of changing requirements and growing desires push it farther out of compliance with reality. (This is like when we replace a computer because its processor speed and RAM size fall out of compliance with reality: the demands of new operating systems and application software.) Using software beyond its natural end of life means that we carry an ever-increasing risk of failure -- when it actually breaks, or when we "suddenly" need to spend major time and money to "maintain" it. Rather than risk letting our software fail out from under us, we could accept the cost of replacing it with newer, more reliable, better software.

One of the goals of the agile software development community is to reduce the cost of changing our code. If agile approaches are successful, then we might be more willing to bear the risk of our code falling away from reality, because we are not afraid of changing it. (Agile approaches also value continuous feedback, which allows us to recognize the need for change early, perhaps before it becomes too costly.) But there may be times or environments in which these techniques don't work as well as we like.

Suppose that we committed to rewriting 1/4 of every system every year. This would allow graceful, gradual migration to new technologies. A possible cost of this strategy is increasing complexity, because our systems would come to be amalgams of two, three, or even four technologies and programming languages interoperating in one system. But is this all that much different from life now? How many of our systems already consist of modules in several languages, several technologies, several styles?

Another side of me is skeptical. Shouldn't our programs just keep working? Why not take care to design them really well? Why not change small parts of the system as needed, rather than take on wholesale changes we don't need yet? Doesn't this approach violate the principle of YAGNI?

Another advantage of the approach: It gives us a built-in path to continuous learning. Rewriting part of a system means digging into the code, learning or re-learning what it's made of and how it works. With pair programming, we could bring new people into a deeper understanding of the code. This would help us to increase the feeling of collective code ownership, as well as preserving and replenishing corporate memory.

Another disadvantage of the approach: It is hard to maintain this sort of discipline in hard financial times. We see this with hardware, too. When money is short, we often decide to lengthen or eliminate the replacement cycle. In such times, my colleagues who traffic in computer labs and faculty desktop computers are a little more worried than usual; what happens if... Software development seems always to be under financial pressure, because user demands grow to outpace our capacity to produce meet them. Even if we decided to try this out for software, administration might immediately declare exigency and fall back into the old ways: build new systems, now, now, now.

Even after thinking about the idea for a while now, it still sounds a little crazy. Then again, sometimes crazy ideas have something to teach us. It is not so crazy that I will dismiss it out of hand. Maybe I will try it some time just see how it works. If it does, I'll give Mike the credit!

... for a definition of "the day" that includes when I read them, not when the authors posted them.

Tweet of the Day

Marick's Law: In software, anything of the form "X's Law" is better understood by replacing the word "Law" with "Fervent Desire".
-- Brian Marick

I love definitions that apply to themselves. They are the next best thing to recursion. I will have plenty of opportunities to put Brian's fervent desire into practice while preparing to teach software engineering this fall.

Non-Tech Blog of the Day

I don't usually quote former graffiti vandals or tattoo artists here. But I am an open-minded guy, and this says something that many people prefer not to hear. Courtesy of Michael Berman:

"Am I gifted or especially talented?" Cartoon said. "No. I got all this through hard work. Through respecting my old man. From taking direction from people. From painting when everyone else was asleep. I just found something I really love and practiced at it my whole life."
-- Mister Cartoon

Okay, so I am tickled to have quoted a guy named Mister Cartoon. His work isn't my style, but his attitude is. Work. Respect. Deference. Practice. Most days of the week, I would be well-served by setting aside my hubris and following Mister Cartoon's example.

William Stafford's Writing the Australian Crawl includes several essays on language, words, and diction in poetry. Words and language -- he and others say -- are co-authors of poems. Their shapes and sounds drive the writer in unexpected ways and give rise to unexpected results, which are the poems that they needed to write, whatever they had in mind when they started. This idea seems fundamental to the process of creation for most poets.

We in CS think a lot about language. It is part of the fabric of our discipline, even when we don't deal in software. Some of us in CS education think and talk way too much about programming languages: Pascal! Java! Ada! Scheme!

But even if we grant that there is art in programming and programs, can we say that language drives us as we build our software? That language is the co-author of our programs? That its words and shapes (and sounds?) drive the programmer in unexpected ways and gives rise to unexpected results, which are the programs we need to write, whatever we have in mind when we start? Can the programmer's experience resemble in any way the poet's experience that Stafford describes?

[Language] begins to distort, by congealing parts of the total experience into successive, partially relevant signals.... [It] begins to enhance the experience because of a weird quality of language: the successive distortions of language have their own cumulative potential, and under certain conditions the distortions of language can reverberate into new experiences more various, more powerful, and more revealing than the experiences that set off language in the first place.

Successive distortions with cumulative potential... Programmers tend not to like it when the language they use, or must use, distorts what they want to say, and the cumulative effects of such distortions in a program that can give us something that feels cumbersome, feels wrong, is wrong.

Still... I think of my experiences coding in Smalltalk and Scheme, and recall hearing others tell similar tales. I have felt Smalltalk push me towards objects I wasn't planning to write, even to objects of a kind I had previously been unaware. Null objects, and numbers as control structures; objects as streams of behavior. Patterns of object-oriented programs often give rise to mythical objects that don't exist in the world, which belies OOP's oft-stated intention to build accurate models of the world. I have felt Scheme push me toward abstractions I did not know existed until just that moment, abstractions so abstract that they make me -- and many a programmer already fearful of functional style -- uncomfortable. Yet it is simply the correct code to write.

For me: Smalltalk and Lisp and Scheme, yes. Maybe Ruby. Not Java. C?

Is my question even meaningful? Or am I drowning in my own inability to maintain suitable boundaries between things that don't belong together?

My in-flight and bedtime reading for my ChiliPLoP trip was William Stafford's Writing the Australian Crawl, a book on reading and especially writing poetry, and how these relate to Life. Stafford's musings are crashing into my professional work on the trip, about solving problems and writing programs. The collisions give birth to disjointed thoughts about software, programming, and art. Let's see what putting them into words does to them, and to me.

Intention endangers creation.

An intentional person is too effective to be a good guide in the tentative act of creating.

I often think of programming as art. I've certainly read code that felt poetic to me, such as McCarthy's formulation of Lisp in Lisp (which I discussed way back in an entry on the unity of data and program. But most of the programs we write are intentional: we desire to implement a specific functionality. That isn't the sort of creation that most artists do, or strive to do. If we have a particular artifact in mind, are we really "creating"?

Stafford might think not, and many software people would say "No! We are an engineering discipline, not an artistic one." Thinking as "artists", we are undisciplined; we create bad software: software that breaks, software that doesn't serve its intended purpose, software that is bad internally, software that is hard to maintain and modify.

Yet many people I know who program know... They feel something akin to artistry and creation.

How can we impress both sides of this vision on people, especially students who are just starting out? When we tell only one side of the story, we mislead.

Art is an interaction between object and beholder.

Can programs be art? Can a computer system be art? Yes. Even many people inclined to say 'no' will admit, perhaps grudgingly, that the iPod and the iPhone are objects of art, or at least have elements of artistry in them. I began writing some of these notes on the plane, and all around me I see iPods and iPhones serving people's needs, improving their lives. They have changed us. Who would ever have thought that people would be willing to watch full-length cinematic films on a 2" screen? Our youth, whose experiences are most shaped by the new world of media and technology, take for granted this limitation, as a natural side effect of experiencing music and film and cartoons everywhere.

Yet iPods aren't only about delivering music, and iPhones aren't just ways to talk to our friends. People who own them love the feel of these devices in their hands, and in our lives. They are not just engineered artifacts, created only to meet a purely functional need. They do more, and they are more.

Intention endangers creation.

Art reflects and amplifies experience. We programmers often look for inspirations to write programs by being alert to our personal experience and by recognizing disconnects, things that interrupt our wholeness.

Robert Schumann said, To send light into the darkness of men's hearts -- such is the duty of the artist. Artists deal in truth, though not in the direct, assertional sense we often associate with mathematical or scientific truth. But they must deal in truth if they are to shine light into the darkness of our hearts.

Engineering is sometimes defined as using scientific knowledge and physical resources to create artifacts that achieve a goal or meet a need. Poets use words, not "physical resources", but also shapes and sounds. Their poems meet a need, though perhaps not a narrowly defined one, or even one we realize we had until it was met in the poem. Generously, we might think of poets as playing a role somewhat akin to the engineer.

How about engineers playing a role somewhat akin to the artist? Do engineers and programmers "send light into the darkness of men's hearts"? I've read a lot of Smalltalk code in my life that seemed to fill a dark place in my mind, and my soul, and perhaps even my heart. And some engineered artifacts do, indeed, satisfy a need that we didn't even know we had until we experienced them. And in such cases it is usually experience in the broadest sense, not the mechanics of saving a file or deleting our e-mail. Design, well done, satisfies needs users didn't know they had. This applies as well to the programs we write as to any other artifact that we design with intention.

I have more to write about this, but at this time I feel a strong urge to say "Yes".

Well, Carefree. But it plays the Western theme to the hilt.

This was a shorter conference visit than usual. Due to bad weather on the way here, I arrived on the last flight in on Sunday. Due to work constraints of my workshop colleagues, I am heading out before the Wednesday morning session. Yet it was a productive trip -- like last year, but this time on our own work, as originally planned. We produced

• the beginnings of a catalog of data-driven real-world problems used in CS1 courses across the country, and
• half of a grant proposal to NSF's CPATH program, to fund some of our more ambitious ideas about programming for everyone, including CS majors.

Yesterday over our late afternoon break, we joined with the other workshop group and had an animated discussion started by a guy who has been involved with the agile community. He claimed that XP and other agile approaches tell us that "thinking is not allowed", that no design is allowed. A straw man can be fun and useful for exploring the boundaries of a metaphor. But believing it for real? Sigh.

A passing thought: Will professionals in other disciplines really benefit from knowing how to program? Why can't they "just" use a spreadsheet or a modeling tool like Shazam? This question didn't come to mind as a doubt, but as a realization that I need a variety of compelling stories to tell when I talk about this with people who don't already believe my claim.

While speaking of spreadsheets... My co-conspirator Robert Duvall was poking around Swivel, a web site that collects and shares open data sets, and read about the founders' inspiration. They cited something Dan Bricklin said about his own inspiration for inventing the spreadsheet:

I wanted to create a word processor for data.

Very nice. Notice that Bricklin's word processor for data exposes a powerful form of end-user programming.

When I go to conferences, I usually feel as if the friends and colleagues I meet are doing more, and more interesting, things than I -- in research, in class, in life. It turns out that a lot of my friends and colleagues seem to think the same thing about their friends and colleagues, including me. Huh.

I write this in the air. I was booked on a 100% full 6:50 AM PHX-MSP flight. We arrive at the airport a few minutes later than planned. Rats, I have been assigned a window seat by the airline. Okay, so I get on the plane and take my seat. A family of three gets on and asks me hopefully whether there is any chance I'd like an aisle seat. Sure, I can help. (!) I trade out to the aisle seat across the aisle so that they can sit together. Then the guy booked into the middle seat next to me doesn't show. Surprise: room for my Macbook Pro and my elbows. Some days, the smile on me in small and unexpected ways.

Gus Mueller, creator of one of my favorite tools, VoodooPad, recently wrote a short note on debugging his designs and programs:

I learned a long time ago that the two best debugging tools I own are a nice piece of paper, and a good pencil.

Writing something down is a great way to "think out loud". My Ghostbusters-loving colleague, Mark Jacobson, calls this biction. He doesn't define the term on his web page, though he does have this poetic sequence:

Bic pen, ink flowing, snow falling, writing, thinking, playing, dancing

That sounds fanciful, but biction is a nuts-and-bolts idea. The friction of that Bic pen on the paper is when ideas that are floating fuzzily through the mind confront reality.

Mark and I taught a data structures course together back in the 1990s, and we had a rule: if students wanted to ask one of us a question, they had to show us a picture they had drawn that illustrated their problem: the data structure, pointers, a bit of code, ... If nothing else, this picture helped us to understand their problem better. But it usually offered more. In the process of showing us the problem using their picture, students often figured out the problem in front of our eyes. Other students commented that, while drawing a picture in preparing to ask a question, they saw the answer for themselves. Biction.

Of course, one can also "think out loud" out loud. In response to my post on teaching software engineering, a former student suggested that I should expose my students to pair programming, which he found "hugely beneficial" in another course's major project, or at least to rubber duck debugging. That's biction with your tongue.

It may be that the most important activity happens inside our heads. We just need to create some friction for our thoughts.

We offer a lot of our upper-division courses on a three-semester rotation. For the last few years, I have been teaching Programming Languages and our compilers course as a part of our rotation. Before I became department head, I taught Algorithms -- a course I love -- in the third slot. I passed the course on to someone else my first semester as head, and I've never gotten it back. The person who teaches it now is really good, and this leaves me to be a utility infielder, covering whatever most needs to be covered that semester. I've taught our intro course in this slot, as well as a set of 1-credit courses on individual programming languages.

This fall, I will teach a course I have never taught before: software engineering. This is a traditional introduction to the discipline for juniors and seniors:

Study of software life cycle models and their phases -- planning, requirements, specifications, design, implementation, testing, and maintenance. Emphasis on tools, documentation, and applications.

I have taught non- traditional software engineering before, in the form of two offerings of a course on agile software development. In fact, I taught that course during my first full semester after starting this blog, and it -- along with and sometimes in tandem with training for my second marathon -- provided a wealth of inspiration for my writing.

Long-term readers of this blog know that I have expressed skepticism about the software engineering metaphor, both in general and in some of the specifics, such as gathering requirements. Still, I respect the goals of the discipline and the effort and earnestness of its proponents. I also am able to put aside philosophical concerns in the interest of department concerns. We all agree that developing software, especially in the large, is a challenging endeavor that requires knowledge and skill. That is the goal of our course, and it will be the goal of my offering this fall.

The course I teach needs to stay pretty close to the traditional notion of software engineering, because that's what our curriculum calls for. By most accounts, employers who hire our students are pretty happy with what we teach in the course, in particular the breadth of exposure we give them. This is one of the constraints I face as I plan.

That said, I will certainly teach my own course. I will make sure that students are exposed to the agile approaches. Many of our alumni have told me that their employers are introducing elements of XP or Scrum into their software development processes, and that they wish they had learned a bit about them during their undergraduate studies. This is the right place for them to encounter agile principles and practices, as a different perspective on the phases of the software lifecycle.

I also tend more toward the working-code end of the spectrum, whereas this course has historically tended toward the modeling-and-abstraction end. I'd like to find a way to ensure that students see and experience both. Models are nothing more than pictures on a whiteboard or in a CASE tool until we can implement them in code. Besides, our students seem to graduate with an alarming lack of exposure to modern tools for version control, build management, and testing. I'd like to find a way for them to learn not only what the stages of software development are but also tools and practices for effectively evolving programs through those stages.

I'm looking for papers and ideas that can help me design a good course that balances several of these forces. Karen Reid's and Greg Wilson's Learning by doing is a great exemplar; it talks about how to introduce version control by using a tool like CVS or SVN to deliver, manage, and collect student assignments.

In the end, I want this course to serve well students wherever they end up after graduation, whether at a big-process company such as Rockwell Collins or a mostly-agile shop consisting of a few developers. Most of our graduates will end up working in several different development environments within a few years, regardless of where they land, so broad principles and understanding trump mastery of any particular skill set. But I also know that mastery plays a big role in how they will come to understand the principles.

Whatever I do, I want the course to be real, not merely an academic exercise.

My daily bleg: please send me any thoughts you have on this course, ways to organize it, or tools to use. As always, I value your suggestions and will gladly share the credit!

My slogan is:
computing is too important to be left to men.

-- Karen Sparck-Jones, 1935-2007

We talk a lot about the state of women in computing. Girls have deserted computer science as an academic major in recent years, and female undergrad enrollment is at a historic low relative to boys. Some people say, "Girls don't like to program," but I don't think that explains all of the problem. At least a few women agree with me... During a session of the Rebooting Computing Summit in January, one of the men asserted that girls don't like to program, and one of the women -- Fran Allen, I think -- asked, "Says who?" From the back of the room, a woman's voice called out, "Men!"

A lot of people outside of computer science do not know how much pioneering work in our discipline was done by women. Allen won a Turing Award for her work on languages and compilers, and the most recent Turing Award was given to Barbara Liskov, also for work in programming languages. Karen Sparck-Jones, quoted above, discovered the idea of inverse document frequency, laying the foundation for a generation of advances in information retrieval. And these are just the ones ready at hand; there many more.

When people assert that women don't like (just) to program, they usually mean that women prefer to do computer science in context, where they can see and influence more directly the effects that their creations will have in the world. One of my heroes in computing, Adele Goldberg, has demonstrated that women can like -- and excel -- on both sides of the great divide.

(Note: I am not speaking of this Adele Goldberg, who is, I'm sure, a fine computer scientist in her own right!)

Goldberg is perhaps best known as co-author of several books on Smalltalk. Many of us fortunate enough to come into contact with Smalltalk back in the 1980s cut our teeth on the fabulous "blue book", Smalltalk-80: The Language and Its Implementation. You can check out a portion of the blue book on-line. This book taught many a programmer how to implement a language like Smalltalk. It is still one of the great books about a language implementation, and it still has a lot to teach us a lot about object-oriented languages.

But Goldberg didn't just write about Smalltalk; she was in the lab doing the work that created it. During the 1970s, she was one of the principal researchers at Xerox PARC. The team at PARC not only developed Smalltalk but also created and experimented with graphical user interfaces and other features of the personal computing experience that we all now take for granted.

Goldberg's legacy extends beyond the technical side of Smalltalk. She worked with Alan Kay to develop an idea of computing as a medium for everyone and a new way for young people to learn, using the computer as a dynamic medium. They described their vision in Personal Dynamic Media, a paper that appeared in the March 1977 issue of IEEE Computer. This was a vision that most people did not really grasp until the 1990s, and it inspired many people to consider a world far beyond what existed at the time. But this paper did not just talk about vision; it also showed their ideas implemented in hardware and software, tools that children were already using to create ideas. When I look back at this paper, it reminds me of one reason I admire Goldberg's work: it addresses both the technical and the social, the abstract and the concrete, idea and implementation. She and Kay were thinking Big Thoughts and also then testing them in the world.

(A PDF of this paper is currently available on-line as part of the New Media Reader. Read it!)

After leaving PARC, Goldberg helped found ParcPlace, a company that produced a very nice Smalltalk product suitable for corporate applications and CS research alike. The Intelligent Systems Lab I worked in as a grad student at Michigan State was one of ParcPlace's first clients, and we built all of our lab's software infrastructure on top of its ObjectWorks platform. I still have ObjectWorks on 3.5" floppies, as well as some of the original documentation. (I may want to use it again some day...)

Some academics view founding a business as antithetical to the academic enterprise, or at least as not very interesting, but Goldberg sees it as a natural extension of what computer science is:

The theoretical and practical knowledge embodied in CS is interesting as standalone study. But the real opportunity lies in equipping oneself to partner with scientists or business experts, to learn what they know and, together, to change how research or business is conducted.

(I found this quote as a sidebar in Women in Computing -- Take 2, an article in a recent issue of Communications of the ACM.)

I suppose that the women-don't-like-to-program crowd might point to Goldberg's career in industry as evidence that she prefers computing in its applied context to the hard-core technical work of computer science, but I don't think that is true. Her work on Smalltalk and real tools at PARC was hard-core technical, and her work at ParcPlace on Smalltalk environments was hard-core technical, too. And she has the mentality of a researcher:

Don't ask whether you can do something, but how to do it.

When no one knows the answer, you figure it out for yourself. That's what Goldberg has done throughout her career. And once she knows how, she does it -- both to test the idea and make it better, and to get the idea out into the world where people can benefit from it. She seems to like working on both sides of the divide. No, she would probably tell us that the divide is an artificial barrier of our own making, and that more of us should be doing both kinds of work.

When we are looking for examples of women who have helped invent computer science, we find researchers and practitioners. We find women working in academia and in industry, working in technical laboratories and in social settings where applications dominate theory. We don't have to limit our vision of what women can do in computing to any one kind of work or work place. We can encourage young women who want to be programmers and researchers, working on the most technical of advances. We can encourage young women who want to work out in the world, changing how people do what they do via the dynamic power of software. If you are ever looking for one person to serve as an example of all these possibilities, Adele Goldberg may be the person you seek.

Tim Bray talks about how photography is too easy to learn:

Quoting from About Photography (1949) by American photographer Will Connell (hat tip Brendan MacRae): "Every medium suffers from its own particular handicap. Photography's greatest handicap is the ease with which the medium as such can be learned. As a result, too many budding neophytes learn to speak the language too long before they have anything to say."

Programming doesn't seem to suffer from this problem! Comments to Bray's entry about books like "C for Dummies" notwithstanding, there are not many people walking around who think programming is too easy. Mark Guzdial has described the reaction of students taking a non-majors course with a computational economics theme when they found out they would have to do a little scripting in Python. Most people who do not already have an interest in CS express disdain for programming's complexity, or fear of it. No one likes to feel stupid. Perhaps worst of all, even students who do want to major in CS don't want to program.

We in the business seem almost to have gone out of our way to make programming hard. I am not saying that programming is or can be "easy", but we should stop erecting artificial barriers that make it harder than it needs to be -- or that create an impression that only really smart people can write code. People who have ideas can write. We need to extend that idea to the realm of code. We cannot make professional programmers out of everyone, any more than piano and violin lessons can make professional musicians out of everyone. But we ought to be able to do what music teachers can do: help anyone become a competent, if limited, practitioner -- and come to appreciate the art of programming along the way.

The good news is that we can solve this "problem", such as it is. As Guzdial wrote in another fine piece:

An amazing thing about computing is that there are virtually no ground rules. If we don't like what the activity of programming is like, we can change it.

We need to create tools that expose powerful constructs to novices and hide the details until later, if they ever need to be exposed. Scratch and Alice are currently popular platforms in this vein, but we need more. We also need to connect the ability to program with people's desires and interests. Scripting Facebook seems like the opportunity du jour that is begging to be grasped.

I'm happy to run across good news about programming, even if it is only the backhanded notion that programming is not too easy. Now we need to keep on with the business of making certain that programming is not too hard.

On Learning Curves, I read a lament about calculus students having a hard time putting their skills into practice on some basic word problems. This line stood out:

They can do the calculus. The algebra slays them.

Of late, our CS faculty have been discussing a programming corollary. Students have passed their intro programming course and a data structures course. They get to an intermediate programming course, or a programming languages course, or an AI course, where they learn some more advanced design idea or algorithm. They answer conceptual courses about the material on exams and do well. Then they try to write code... and hit a wall.

Sometimes a new programming language gets in the way, but sometimes not -- students are using a language they used for a year in the intro sequence. And whether it's a new language or an old one, the problems seem ticky-tack: semicolons, declarations, function calls.

They can talk about the advanced concepts, but simple programming tasks to implement the ideas slays them. The programming part looks like attention to detail to me, or effort spent to internalize basic grammar and vocabulary. One prof half-jokingly says his students have gone out of their way to forget what they already knew.

You don't really know programming unless you can write a program from a real problem, and not just a tightly-specified exercise designed by the instructor. And I'm not sure you can really know a concept -- not in the way that a computer scientist needs to know it -- unless you can write a program using it for a real problem. If syntax is in the way, you simply need to buckle down and learn the syntax.

I don't have any answers on this but thought it was interesting that a calculus prof is running into the same kind of problem.

My compiler students are getting to the point where they should be deep in writing a parser for their language. Walking back from lunch, I was thinking about some very simple things they could do to make their lives -- and project -- better.

1.   Start.

Yes, start. If you read the literature of the agile software development world or even of the life hacker world, people talk about the great power that comes just from taking the first step. I've always loved the old Goethe quote about the power of committing to a course of action. But isn't this all cliché?

It is so easy to put off tracing your language grammar, or building those FIRST and FOLLOW sets, or attacking what you know will be a massive parsing table. It is so easy to be afraid of writing the first line of code because you aren't sure what the whole app will look like.

Take the first step, however small and however scary. I'm always amazed how much more motivated I feel once I break the seal on a big task and have some real feedback from my client or my compiler.

2.   Always have live code.

Live code is always better than ideas in your head. Brian Marick tells us so. One of the Gmail guys tells us so:

We did a lot of things wrong during the 2.5 years of pre-launch Gmail development, but one thing we did very right was to always have live code. ...

Of course none of the code from my prototype ever made it near the real product (thankfully), but that code did something that fancy arguments couldn't do (at least not my fancy arguments), it showed that the idea and product had real potential.

Your code can tell which ideas are good ones and which are bad ones. It can teach you about the app you are building. It can help you learn what your user wants.

I hear this all the time from students: "We have a pretty good handle on this, but no code yet." Sounds good, but... Live code can convince your professor that you really have done something. It can also help you ask questions and be submitted on the due date. Don't underestimate the value in that.

As Buchheit says from the Gmail experience, spend less time talking and more time prototyping. You may not be Google, but you can realize the same benefits as those guys. And with version control you don't have to worry about taking the wrong step; you can always back up.

3.   Don't forget what you know.

Okay, I have to admit that this did not occur to me on my walk home form lunch. This afternoon, a former student and local entrepreneur gave a department seminar on web app security. He twice mentioned that many of the people he hires have learned many useful skills in object-oriented design and software engineering, using system languages such as Java and Ada. When they get to his shop, they are programming in a scripting language such as PHP. "And they throw away all they know!" They stop using the OOP principles and patterns they have learned. They stop documenting code and testing. It's as if scripting occurs in a different universe.

As he pointed out after the talk, all of those skills and techniques and practices matter just as much -- no, more -- when using a language with many power tools, few boundaries, and access to all of his and his clients' data and filesystem.

When building a compiler in class, or any other large-scale team project in a capstone course, all of those skills and techniques and practices matter, too, and sometimes for the first time in student's career. This is likely the largest and most sophisticated program they have ever written. It is the first time they have ever had to depend on one or two or five other students to get done, to understand and work with others' code, to turn their own code other for the understanding and use of their teammates.

There is a reason that you are learning all this stuff. It's for the project you are working on right now.

Confluence... The Education column in the February 2009 issue of Communications of the ACM, Human Computing Skills: Rethinking the K-12 Experience, champions computational thinking in lieu of programming:

Through the years, despite our best efforts to articulate that CS is more than "just programming," the misconception that the two are equivalent remains. This equation continues to project a narrow and misleading image of our discipline -- and directly impacts the character and number of students we attract.

I remain sympathetic to this concern. Many people, including lost potential majors, think that CS == programming. I don't know any computer scientists who think that is true. I'd like for people to understand what CS is and for potential majors who end up not wanting to program for a living to know that there is room for them in our discipline. But pitching programming to the aside altogether is the wrong way to do that, and will do more harm than good -- even for non-computer scientists.

It seems to me that the authors of this column conflate CS with programming at some level, because they equate writing a program with "scholarly work" in computer science:

While being educated implies proficiency in basic language and quantitative skills, it does not imply knowledge of or the ability to carry out scholarly English and mathematics. Indeed, for those students interested in pursuing higher-level English and mathematics, there exist milestone courses to help make the critical intellectual leaps necessary to shift from the development of useful skills to the academic study of these subjects. Analogously, we believe the same dichotomy exists between CT, as a skill, and computer science as an academic subject. Our thesis is this: Programming is to CS what proof construction is to mathematics and what literary analysis is to English.

In my mind, it is a big -- and invalid -- step from saying "CT and CS are different" to saying that programming is fundamentally the domain of CS scholars. I doubt that many professional software developers will agree with a claim that they are academic computer scientists!

I am familiar with Peter Naur's Programming as Theory Building, which Alistair Cockburn brought to the attention of the software development world in his book, Agile Software Development. I'm a big fan of this article and am receptive to the analogy; I think it gives us an interesting way to look at professional software development.

But I think there is more to it than what Naur has to say. Programming is writing.

Back to the ACM column. It's certainly true that, at least for many areas of CS, "The shift to the study of CS as an academic subject cannot .. be achieved without intense immersion in crafting programs." In that sense, Naur's thesis is a good fit. But consider the analogy to English. We all write in a less formal, less intense way long before we enter linguistic analysis or even intense immersion in composition courses. We do so as a means of communicating our ideas, and most of us succeed quite well doing so without advanced formal training in composition.

How do we reach that level? We start young and build our skills slowly through our K-12 education. We write every year in school, starting with sentences and growing into larger and larger works as we go.

I recall that in my junior year English class we focused on the paragraph, a small unit of writing. We had written our first term papers the year before, in our sophomore English course. At the time, this seemed to me like a huge step backward, but I now recognize this as part of the Spiral pattern. The previous year, we had written larger works, and now we stepped back to develop further our skills in the small after seeing how important they were in the large.

This is part of what we miss in computing: the K-8 or K-12 preparation (and practice) that we all get as writers, done in the small and across many other learning contexts.

Likewise, I disagree that proof is solely the province of mathematics scholars:

Just as math students come to proofs after 12 or more years of experience with basic math, ...

In my education, we wrote our first proofs in geometry -- as sophomores, the same year we wrote our first term papers.

I do think one idea from the article and from the CT movement merits more thought:

... programming should begin for all students only after they have had substantial practice acting and thinking as computational agents.

Practice is good! Over the years, I have learned from CS colleagues encountered many effective ways to introduce students, whether at the university or earlier, to ideas such as sorting algorithms, parallelism, and object-oriented programming by role play and other active techniques -- through the learner acting as a computational agent. This is an area in which the Computational Thinking community can contribute real value. Projects such as CS Unplugged have already developed some wonderful ways to introduce CT to young people.

Just as we grow into more mature writers and mathematical problem solvers throughout our school years, we should grow into more mature computational thinkers as we develop. I just don't want us to hold programming out of the mix artificially. Instead, let's look for ways to introduce programming naturally where it helps students understand ideas better. Let's create languages and build tools to make this work for students.

As I write this, I am struck by the different nouns phrases we are using in this conversation. We speak of "writers", not "linguistic thinkers". People learn to speak and write, to communicate their ideas. What is it that we are learning to do when we become "computational thinkers"? Astrachan's plea for "computational doing" takes on an even more XXXXX tone.

Alan Kay's dream for Smalltalk has always been the children could learn to program and grow smoothly into great ideas, just as children learn to read and write English and grow smoothly into the language and great ideas of, say, Shakespeare. This is a critical need in computer science. The How to Design Programs crowd have shown us some of the things we might do to accomplish this: language levels, tool support, thinking support, and pedagogical methods.

Deep knowledge of programming is not essential to understand all basic computer science, some knowledge of programming adds so very much even to our basic ideas.

A while back, I clipped this quote from a university publication, figuring it would decorate a blog entry some day:

The thing about a liberal arts education ... is it prepares you to fail successfully and learn from that failure. ... You will all fail. That's OK.

-- Jim Linahon

Linahon is an alumnus of our school who works in the music industry. He gave a talk on campus for students aspiring to careers in the industry, the theme of which was, "Learn to fail. It happens"

More recently, I ran across this as the solution to a word puzzle in our local paper:

You've got to jump off cliffs all the time and build your wings on the way down.

-- Ray Bradbury

Bradbury was one of my favorite authors when I was growing up (The Martian Chronicles mesmerized me!) This quote goes farther than Linahon's: what other people call failure is learning to fly. Do not fear.

A comment made at the Rebooting Computing summit about "embracing failure" brought these quotes back to mind, along with an e-mail message Rich Pattis wrote sometime last year. Rich talked about how hard our discipline must feel to beginners, because it is a discipline learned almost wholly by failure. Learning to program can seem like nothing more than an uninterrupted sequence failures: syntax errors, logic errors, boundary cases, ugly interface, .... I'm not a novice any more, but I still feel the constant drip of failure whenever I work on a piece of code I don't already understand well.

The thing is, I kinda like that feeling -- the challenge of scaling a mountain of code. My friends who program can at least say that they don't mind it, and the best among them seem to thrive in such an environment. I think that's part of what separates programmers from less disturbed people.

Then again, repeated failure is a part of learning many things. Learning to play a musical instrument or a sport require repeated failure for most people. Hitting a serve in tennis, or a free throw on the hardcourt, or a curve ball in baseball -- the only way to learn is by doing it over and over, failing over and over, until the mind and body come together in a memory that make success a repeatable process. This seems to be an accepted part of athletics, even among the duffers who only play for fun. How many people in America are on a golf course this day, playing the game poorly but hoping -- and working -- to get better?

Why don't we feel the same way about academics, and about computer programming in particular? Some small number seem to, maybe the 2% that Knuth said are capable of getting it.

I have heard some people say that in sports we have created mechanisms for "meaningful failure", though I'm not sure exactly what that means, but I suspect that if we could build tools for students and set problems before them that give them a sense of meaningful failure, we'd probably not scare off so many people from our early courses. I suspect that this is part of what some people mean when they say we should make our courses more fun, though thinking in terms of meaningful failures might give us a better start on the issue than simply mantras about games and robots.

I don't think just equating programming to sports is enough. Mitch Wand sent a message to the PLT Scheme mailing list this weekend on a metaphor he has been using to help students want to stick to the design recipes of How to Design Programs:

In martial arts, the first thing you learn is to do simple motions very precisely. Ditto for ballet, where the first thing you learn is the five positions.

Once those are committed to muscle memory, you can go on to combinations and variations.

Same deal for programming via HtDP: first practice using the templates until you can do it without thinking. Then you can go on to combinations and variations.

I like the analogy and have used a similar idea with students in the past. But my experience is that this only works for students who want to learn to programming -- or martial arts or ballet, for that matter. If you start with people who want to go through the process of learning, then lots of things can work. The teacher just needs to motivate the student every once in a while to stick with the dream. But it's already their dream.

Maybe the problem is that people want to play golf and the martial arts -- for whatever social, business, or masochistic reasons -- but that most people don't want to learn to program? Then our problem comes back to a constant theme on this blog: putting a feeling of power in peoples' hands when we show them programming, so they want to endure the pain.

One last quote, in case you ever are looking for a literary way to motivate students to take on tough challenges rather than little ones that acquiesce easily and making us feel good sooner:

What we choose to fight is so tiny!
What fights us is so great!
...
When we win it's with small things,
and the triumph itself makes us small.
...
Winning does not tempt that man.
This is how he grows: by being defeated, decisively,
by constantly greater beings.

This comes from Rainer Maria Rilke's The Man Watching. What a marvelous image, growing strong by being beaten -- decisively, less -- by ever greater opponents. I'm sure you professional programmers who have been tackling functional programming, continuations, Scheme, Haskell, and Erlang these last few years know just the feeling Rilke describes, deep in the marrow of your bones.

I recently read an early version of a paper by a colleague in the agile world that talks about several different ways he approaches design problems. I like that he is writing about design. After all these years, I still encounter many people who think this statement is an axiom:

In those situations, I try to share (gently) my experience as developer and occasional consultant, but the notion is pretty well ingrained. As I wrote last month, it's hard a slog.

As I read the preprint, I thought more about doing design in a reactive way, responding to new features as we add them to our code. My skeptical colleagues think that the sort of design so many of us do in agile settings is not design at all. But I think they are wrong, and the word "reactive" brought to mind a similar notion from my days in AI: reactive planning.

When I first heard that term in a graduate readings course, it sounded like an oxymoron to me. Planning is the opposite of reaction, right? If an agent does not construct a plan, then how is it planning? Reactive planning contrasts with what then came to be known as classical planning. An agile wag wag might coin the term "BPUF" -- big planning up-front.

But I learned about the idea and found it wasn't an oxymoron at all, that reactive planning was a real and valuable way for an agent to prepare its actions. I have learned the same is true of "reactive" design. We folks in the agile world probably haven't done enough to teach agile novices to design in this way, which is one reason I'm hoping to see more papers like the draft I just read.

The latest edition of inroads, the quarterly publication of SIGCSE, arrived in my mailbox yesterday. The invited editorial was timely for me, as I finally begin to think about teaching this spring. Alfred Aho, co-author of the dragon book and creator of several influential languages and programming tools, wrote about Teaching the Compilers Course. He didn't write a theoretical article or a solely historical one, either; he's been teaching compilers every semester for the last many years at Columbia.

As always, I enjoyed reading what an observant mind has to say about his own work. It's comforting to know that he and his students face many of the same challenges with this course as my students and I do, from the proliferation of powerful, practical tools to the broad range of languages and target machines available. (He also faces a challenge I do not -- teaching his course to 50-100 students every term. Let's just say that my section this spring offers ample opportunity for familiarity and one-on-one interaction!)

Two of Aho's ideas are now germinating in my mind as I settle on my course. The first is something that has long been a feature of my senior project courses other than compilers: an explicit focus on the software engineering side of the project. Back when I taught our Knowledge-Based Systems course (now called Intelligent Systems) every year, we paid a lot of attention to the process of writing a large program as a team, from gathering requirements and modeling knowledge to testing and deploying the final system. We often used a supplementary text on managing the process of building KBS. Students produced documents as well as code and evaluated team members on their contributions and efforts.

When I moved to compilers five or six years ago after a year on sabbatical, I de-emphasized the software engineering process. First of all, I had smaller classes and so no teams of three, four, or five. Instead, I had pairs or even individuals flying solo. Managing team interactions became less important, especially when compared to the more intellectually daunting content of the compiler course. Second, the compiler students tended to skew a little higher academically, and they seemed to be able to handle more effectively the challenge of writing a big program. Third, maybe I got a little lazy and threw myself into the fun content of the course, where my own natural inclinations lie.

Aho has his students work in teams of five and, in addition to writing a compiler and demoing at the end of the semester:

• write a white paper on their source language,
• write a tutorial on using it, and
• close with a substantial project report written by every member of the team.

This short article has reawakened my interest in having my students -- many of whom will graduate into professional careers developing software and managing its development -- attend more to process. I'll keep it light, but these three documents (white paper, tutorial, and project report) will provide structure to tasks the students already have to do, such as to understand their source language well and to explore the nooks and crannies of its use.

The second idea from Aho's article is to have students design their own language to compile. This is something I have never done. It is also a feature that brings more value to the writing of a white paper and a tutorial for the language. I've always given students a language loosely of my own design, adapted from the many source languages I've encountered in colleagues' courses and professional experience. When I design the language, I have to write specs and clarifications; I have to code sample programs to demonstrate the semantics of the language and to test the students' compilers at the end of the semester.

I like the potential benefits of having students design their own language. They will encounter some of the issues that the designers of the languages they use, such as Java, C++, and Ada faced. They can focus their language in a domain or a technology niche of interest to them, such as music and gaming or targeting a multi-core machine. They may even care more about their project if they are implementing an engine that makes their own creation come to life.

If I adopt these course features, they will shift the burden between instructor and student in some unusual ways. Students will have to exert more energy into the languages of the course and write more documentation. I will have to learn about their languages and follow their projects much more intimately as the semester proceeds in order to be able to provide the right kind of guidance at the right moments. But this shift, while demanding different kinds of work on both our parts, should benefit both of us. When I design more of the course upfront, I have greater control over how the projects proceed. This gives me a sense of comfort but deprives the students of experiences with language design and evolution that will serve them well in their careers. The sense of comfort also deprives me of something: the opportunity to step into the unknown in real-time and learn. Besides, my students often surprise me with what they have to teach me.

As I said, I'm just now starting to think about my course in earnest after letting Christmas break be a break. And I'm starting none to soon -- classes begin Monday. Our first session will not be until Thursday, however, as I'll begin the week at the Rebooting Computing summit. This is not the best timing for a workshop, but it does offer me the prospect of a couple of travel days away from the office to think more about my course!

An acquaintance of mine sent a link to James Shore's The Decline and Fall of Agile to a mailing list with a warning: "You probably don't want to read it if you're a follower of the religion." His use of the pejorative reveals that he's not a fan. He is a strong believer in design -- if not BDUF, at least more than he perceives agile methods to encourage. I read Shore's articles anyway, so I was happy to now.

I don't think of myself as religious on this matter, but I often speak positively about agile approaches (I might say "fairly") when a colleague expresses what I think os a misconception, so here goes...

Shore is right. He also echoes something that comes up every so often when I am discussing agile approaches with folks who have had a bad experience: When a company does something it calls "agile" but which is unsound (perhaps because they leave something important out), things usually do not go well. But that's because they've done something unsound, not because they call what they did "agile".

I really try not to be a knee-jerk apologist for agile methods. They don't solve every problem in the world; nothing does. So when someone tells me that agile has failed them, I try to listen carefully to what they've been doing. Otherwise, it's too easy to say, "Oh, you did it wrong." That's the sort of behavior that leads people to talk about religion. And it's intellectually sloppy, which bothers me more.

But sometimes people do have a misconception, or do leave out an essential practice, or do something else that counters the desired benefit. When someone tells me,

[Extreme Programming] says you should spend no more than 10 minutes on design during a three-week sprint. 10 minutes!

... I have to say something. Maybe some agile evangelists have said this; maybe not. But it's wrong, and someone has to say so.

For what it's worth, my understanding of XP goes back to the original spirit: If doing something is valuable, then why not do it all the time? As a result, I encourage the teams I work with at the university and in industry to think about design all the time. But I also encourage them not to design too far ahead of their code base, because that is too often untested thought. Refactoring, a practice that some design-oriented people like to deride as "not doing it right the first time", is a key part of the design process. (Teams that don't refactor all the time are usually in big trouble -- and then unhappy with the other agile practices they have adopted.)

As with any idea that becomes more buzzword than idea, there is always a huge risk that the energy of the "movement" will flame out early when things don't go as predicted by eager creators and early adopters. We see the same thing happen in educational settings all the time. Many of you have lived through objects-first in the CS curriculum.

Software patterns encountered the same fate. Hype outran practice. When this happens, it is usually best to let the fad die out. The good news is that some people will persevere with the good part of the idea and do wonderful things. The most recent patterns example I know of is the resurgence of interest in patterns in the parallel programming community, which faces many of the same challenges that faced a world moving to object-oriented programming faced fifteen years ago. The Parallel Computing Laboratory at Cal-Berkeley is undertaking an ambitious pattern language project in this area.

Echoing Shore:

What frustrates me the most is that this situation is entirely avoidable. In a green-field environment, the solid agile engineering practices included in Extreme Programming pay for themselves within the first few months.

When this passage leads the person with whom you are speaking to say,

I didn't know anyone used XP anymore. I feel sorry for them.

... you'll know it's time to stop talking. The religion that is getting in the way of developing software better may not be on the agile side of the conversation.

Today, I was asked the best question ever by a high-school student.

During the fall, we host weekly campus visits by prospective students who are interested in majoring in CS. Most are accompanied by their parents, and most of the dialogue in the sessions is driven by the parents. Today's visitors were buddies from school who attended sans parents. As a part of our talking about careers open to CS grads, I mentioned that some grads like to move into positions where they don't deal much with code. I told them that two of the things I don't like about my current position is that I only get to teach one course each semester and that I don't have much time to cut code. Off-handedly, I said, "I'm a programmer."

Without missing a beat, one the students asked me, "What hobby projects are you working on?"

Score! I talked about a couple of things I work on whenever I can, little utilities I'm growing for myself in Ruby and Scheme, and some refactoring support for myself in Scheme. But the question was much more important than the answer.

Some people like to program. Sometimes we discover the passion in unexpected ways, as we saw in the article I referred to in my recent entry:

[Leah] Culver started out as an art major at the University of Minnesota, but found her calling in a required programming class. "Before that I didn't even know what programming was," she admits. ... She built Pownce from scratch using a programming language called Python.

Programmers find a way to program, just as runners find a way to run. I must admit, though, that I am in awe of the numbers Steve Yegge uses when talking about all the code he has written when you take into account his professional and personal projects:

I've now written at least 30,000 lines of serious code in both Emacs Lisp and JavaScript, which pales next to the 750,000 or so lines of Java I've [spit] out, and doesn't even compare to the amount of C, Python, assembly language or other stuff I've written.

Wow. I'll have to do a back-of-the-envelope estimate of my total output sometime... In any case, I am willing to stipulate to his claim that:

... 30,000 lines is a pretty good hunk of code for getting to know a language. Especially if you're writing an interpreter for one language in another language: you wind up knowing both better than you ever wanted to know them.

The students in our compiler course will get a small taste of this next semester, though even I -- with the reputation of a slave driver -- can't expect them to produce 30 KLOC in a single project! I can assure them that they will make a non-trivial dent in the 10,000 hours of practice they need to master their discipline. And most will be glad for it.

Courtesy of Brian Marick and his Agile Development Practices keynote:

Humans plus running code are smarter than humans plus time.

We can sit around all day thinking, and we may come up with something good. But if we turn our thoughts into code and execute it, we will probably get there faster. Running a piece of code gives us information, and we can use that feedback to work smarter. Besides, the act of writing the code itself tends to make us smarter, because writing code forces us to be honest with ourselves in places where abstract thought can get away with being sloppy.

Brian offers this assertion as an assumption that underlies the agile software value of working software, and specifically as assumption that underlies a guideline he learned from Kent Beck:

No design discussion should last more than 15 minutes without someone turning to a computer to do an experiment.

An experiment gives us facts, and facts have a way of shutting down the paths to fruitless (and often strenuous) argument we all love to follow whenever we don't have facts.

I love Kent Beck. He has a way of capturing great ideas in simple aphorisms that focus my mind. Don't make the mistake that some people make, trying to turn one of his aphorisms into more than it is. This isn't a hard and fast rule, and it probably does not apply in every context. But it does capture an idea that many of us in software development share: execucting a real program usually gives us answers faster and more reliably than a bunch of software developers sitting around pontificating about a theoretical program.

As Brian says:

Rather than spending too much time predicting the future, you take a stab at it and react to what you learn from writing and running code...

This makes for a nice play on Alan Kay's most famous bon mot, "The best way to predict the future is to invent it." The best way to predict the future of a program is to invent it: to write the program, and to see how it works. Agile software development depends on the fact that software is -- or should be -- soft, malleable, workable in our hands. Invent the future of your program, knowing full well that you will get some things wrong, and use what you learn from writing and running the program to make it better. Pretty soon, you will know what the program should look like, because you will have it in hand.

To me, this is one of the best lessons from Brian's keynote, and well worth an Agile Thought.

Just this weekend I learned about Ashleigh Brilliant, a cartoonist and epigrammist. From little I've seen in a couple of days, his cartoons remind me of Hugh MacLeod's business-card doodles Gaping Void -- only with a 1930s graphic style and language that is more likely SFW.

This Brilliant cartoon, #3053, made it into my Agile Development Hall of Fame on first sight:

Doing it wrong fast means that we have a chance to learn sooner, and so have a chance to be less wrong than yesterday.

This lesson was fresh in my mind over the weekend from a small dose of programming. I was working on the financial software I've decided to write for myself, which has me exploring a few corners of PLT Scheme that I don't use daily. As a result, I've been failing more frequently than usual. The best failures come rat-a-tat-tat, because they are often followed closely by an a-ha! Sunday, just before I learned of Brilliant's work, I felt one of those great releases when, for the first time, my code gave me an answer I did not already know and had not wanted to compute by hand: our family net worth. At that moment I was excited to verify the result manually (I'll need that test later!) and enjoy all the wrong wrong work that had brought me to this point. Brilliant has something.

In recent weeks, the financial markets of the world have entered "interesting times". There is a great story to tell here about the role that computational models have played in the financial situation we face, but I have been more intrigued by another connection to computing, more specifically to software development. It turns out that these bad times for the economy are a good time to "be agile".

Paul Graham writes that this is a good time to start a start-up, in part because a start-up can be more nimble and consume fewer resources than a big software shop. A new product can grow in small steps in a market where resources are limited. Tim Bray expands on that idea in his post, A Good Time for Agility. It may be difficult to get major projects and corresponding big budgets approved in tough times, because most execs will be focused on cost containment and surviving to the quarterly report. But...

The classic Agile approach, where you pick two or three key features, spec'em out with test suites that involve the business side, build'em and test'em, and then think about maybe going back for the next two or three, well, that's starting to look awfully attractive.

Small steps and small up-front expense draw less attention than BDUF and multi-month budgets. And if they lead to concrete, measurable improvements, they have a greater chance of sticking. They might even lead to important new software.

The third example I read recently came in a posting to the XP mailing list, the link to which I seem to have lost. The gist was straightforward: The writer worked in the software arm of a major financial institution. Having previously adopted agile practices enabled his shop to shift direction on short notice in response to the market crash. They were not in the middle of a major project with a specific market but in the middle of ongoing creation of loan products. The market for their usual products deteriorated and were able to begin delivering software to a new market relatively quickly. This did not require a new major project, but a twist on their current trajectory.

This shouldn't surprise us. Agile approaches allow us to manage risk and change at finer levels of granularity, and in a time of major change the massive dinosaurs will be at a disadvantage against more nimble species.

Not all news is so rosy. Bureaucracy can still dominate an environment. Last Friday, an alumnus of our department gave a talk for our current students on how not to stink in industry. His advice was uniformly practical, with many of his technical points reminiscent of The Practical Programmer. But in response to a question about XP and agile practices, his comments were not so positive. He and his team have not yet figured out how to do planning for XP projects, so they are left with tool-specific XP practices such as pair programming and testing early and often. I think that I can help him get a grip on XP-style planning, and offered to do so, but I think his problem goes deeper, to something he has little control over: his team's customers expect big project plans and fixed-price "contracts".

This is not a new problem. I was fortunate to visit RoleModel Software back when Ken Auer was first building it, and one topic of discussion within the company was how to educate clients about a new way of planning and budgeting for projects and how to shift the culture when all of its competitors was doing the same old thing. His potential customers had one way of managing the risk they faced, and that was do to things the usual way, even if that led to software that was off target, over budget, and over time. I don't know much more about the issue than this and need to see if anyone has written of positive experiences with it in industry.

My former student works for a government agency, which perhaps makes the bureaucracy hard to move by law or administrative rule, rather than by market forces. I feel for him as my department continues to work on outcomes assessment. University mandates are my job's version of "the customer demands a big project plan". (Outcomes assessment is an academic form of unit testing for its curriculum.) As we try to enact an outcomes assessment plan in small steps, we face a requirement to produce a BDUF plan by the end of this year. It's hard to figure out what will work best for us if we have to predict what is best up front. Some will tell us that the academic world understands outcomes assessment well enough to design a suitable plan from scratch, but many of us in the trenches will disagree. It's certainly possible to design from scratch a plan that looks familiar to other people, but who knows if that is what will help this department and this faculty steer its programs most effectively?

Complete operational plans of this sort often end up being as useful as many of their software design counterparts. Worse, mandates for such also tend to counterproductive, because when the last big plan fails, and when administration doesn't follow through by holding departments accountable for fixing the plans, faculty learn to mistrust both the mandates and the plans. That is how the link in the previous paragraph can be to a post nearly two years old, yet my department still not have an effective assessment plan in place: the faculty have a hard time justifying spending the time and energy to take on such a big project if it is likely to fail or if not developing a plan has no consequences.

I am hoping that we can use the most recent mandate as an opportunity to begin growing an assessment regimen that will serve us well over time. I believe in units tests and continuous feedback. I'm also willing to argue to the administration that an "incomplete" but implementable plan is better than a complete plan with no follow-through.

As the software guys are saying, this is a good time to be agile.

I had one of those agile moments on Wednesday. A colleague stopped by my office to share his good feeling. He had just come from a CS 2 lab. "I love it whenever I design a lab in which students work in pairs. There is such life in the lab!" He went on to explain the interactions within pairs but also across pairs; one group would hear what another was thinking or doing, and would ask about it. So much learning was in the air.

This reminded me of the old joke... Patient: "Doctor, it hurts when I do this." (Demonstrates.) "Can you help me?" Doctor: "Sure. Don't do that."

Of course, it reminded of the negative space around the joke. Patient: "Doctor, life is great when I do this." (Demonstrates.) "Can you help me?" Doctor: "Sure. Do more of that."

"But..." But. We faculty are creatures of habit, both in knowing and in doing. We just know we can't teach all of our material with students working in pairs, so we don't. I think we can, even when I don't follow my own advice. (Doctor, heal thyself!) We design the labs, so if we want students to work in pairs, we can have them work in pairs.

I've had one or two successful experiences with all pair programming all the time in closed labs. Back when we taught CS1 and CS2 in C++, in the mid-1990s, and I was doing our first-year courses a lot, I designed all of my labs for students working in pairs. I wish I could say I had bee visionary, but my motivation was extrinsic: I had 25-30 students in class and 15 computers in the lab. Students worked with different students every week, in pseudo-random assignments of my device.

My C++-based courses probably weren't very good -- I was relatively new to teaching, and we were using C++ after all -- and the paired programming in our lab sessions may have been one of the saving graces: students shared their perplexity and helped each other learn. When they worked on outside programming assignments for the course, they could call on a sturdy network of friends they had built in lab sessions. Without the pairs, I fear that our course would have worked well for very few students.

If something works well, let's try to understand the context in which it works, and then do it more often in those contexts. That's an agile sentiment, whether we apply it to pair programming or not. Whether we apply it at the university or in industry. Whether we apply it to software development or any other practice in which we find ourselves engaged.

When I first came to UNI, a colleague and I talked a lot about programming by novices, graphical programming, programming by example, declarative programming, and the like. Many people saw ideas such as these as a way to broaden participation by making it possible for people to program without "programming". Anyone can draw a flowchart, right? Or program by dragging and dropping widgets, right?

No, said my colleague. He was fond of saying that all of those solutions were really just new kinds of programming language in disguise; ultimately, you had to know how to write a program. That may not be an insanely difficult task, but it is not a trivial task, either -- even if pictures were the lingua franca.

Yet the world goes on, looking for its next way to bypass the hard work of programming and make it disappear with pretty pictures. So I'm glad when people say out loud in public what my colleague has always said in-house. Here is Uncle Bob Martin sounding in on the latest, model-driven architecture:

Some folks have put a great deal of hope in technologies like MDA. I don't. The reason is that I don't see MDA as anything more than a different kind of computer language. To be effective it will still need 'if' and 'for' statements of some kind. And 'programmers' will still need to write programs in that language, because details will still need to be managed. There is no language that can eliminate the programming step, because the programming step is the translation from requirements to systems irrespective of language. MDA does not change this.

Programming is the translation from requirements to systems, whether we write it down in C, Java, or Ruby -- or an MDA model. There will be always be a program, and so there will be always be programmers. Making better programming tools helps, but as Bob says we also need thinkers, practitioners and academics alike, to figure out better what it means "to program" and to help others learn how to do it.

A couple of weeks ago, I mentioned that I might have my students create their own examples for a homework assignment. Among the possible benefits of this were:

• helping the programmers to write down their understanding of the problem in a concrete way early in the process
• giving the programmers a way of to ask concrete questions early in the process -- and reason to ask the questions
• helping the programmers know how much code to write and when to stop

I tried this and, as usual, learned as much or more than my students.

Getting students to think concretely about their tasks is tough, but asking them to write examples seemed to help. Most of them made a pretty good effort and so fleshed out what the one- or two-line text description I gave them meant. I saw lots of the normal cases for each task but also examples at the boundaries of the spec (What if the list is empty?) and on the types of arguments (What if the user passes an integer when the procedure asks for a list? What if the user passes -1 when the procedure expects a non-negative integer?) In class, before the assignment was due, we were able to discuss how much type checking we want our procedures to do, if any, in a language like Scheme without manifest types. Similarly, should we write examples with the wrong number of arguments, which result in an error?

I noticed that most students' examples contrasted cases with different inputs to a procedure, but that few thought about different kinds of output from the procedure. Can filter return an empty list? Well, sure; can you show me an example? I'll know next time to talk to students about this and have them think more broadly about their specs.

Requiring examples part-way through the assignment did motivate questions earlier than usual. On previous assignments, if I received any questions at all, they tended to arrive in my mailbox the night before the programs were due. That was still the case, but now the deadline was halfway through the assignment period, before they had written any code. And most of the class seemed happy to comply with my request that they write their examples before they wrote their code. (They are rarely in a hurry to write their code!)

Did having their own examples in hand help the students know how much code to write and when to stop? Would examples provided by me have helped as much? I don't know, but I guess 'yes' to both. Hmm. I didn't ask students about this! Next time...

Seeing their examples early helped me as much writing their examples early helped them. They got valuable feedback, yes, but so did I. I learned a bit of what they were thinking about the specific problems at hand, but I also learned a bit of what they think about more generally when faced with a programming task.

My first attempt at this also gave me some insight about how to describe the idea of writing examples better, and why it's worth the effort. The examples should clarify the textual description of the problem. They aren't about testing. They may be useful as tests later, but they probably aren't sufficient. (They approximate are a form of black box testing, but not white box testing.) As clarifiers, one might take an extreme position: If the textual description of the problem were missing, would the examples be enough for us to know what procedure to write? At this extreme, examples with the wrong number and type of arguments might be essential; in the more conventional role of clarifying the spec, those examples are unnecessary.

One thing that intrigued me after I made this assignment is that students might use their examples as the source material for test-driven development. (There's that word again.) I doubt many students consider this on their own; a few have an inclination to write and test code in close temporal proximity, but TDD isn't a natural outgrowth of that for most of them. In any case, we are currently learning a pattern-driven style of programming, so they have a pretty good idea of what their simplest piece of code will look like. There is a nice connection, though. Structural recursion relies on mimicking the structure of the input data, and that data definition also gives the programmer an idea about the kinds of input for which she should have examples. That s-list is either an empty list or a pair...

I'm probably reinventing a lot of wheels that the crew behind How to Design Programs smoothed out long ago. But I feel like I'm learning something useful along the way.

I'm a big tennis fan. I like to play and would love to play more, though I've never played well. But I also like to watch tennis -- it is a game of athleticism and strategy. The players are often colorful, yet many of the greatest have been quiet, classy, and respectful of the game. I confess a penchant for the colorful players; Jimmy Connors is my favorite player of all time, and in the 1990s my favorite was Andre Agassi.

Agassi's chief rival throughout his career was one of the game's all-time greats, Pete Sampras. Sampras won a record fourteen Grand Slam titles (a record under assault by the remarkable Roger Federer) and finished six consecutive years as the top-ranked player in the world (a record that no one is likely to break any time soon). He was also one of the quiet, respectful players, much more like me than the loud Agassi, who early in his career seemed to thrive on challenging authority and crossing boundaries just for the attention.

Sampras recently published a tennis memoir, A Champion's Mind, which I gladly read -- a rare treat these days, reading a book purely for pleasure. But even while reading for pleasure I could not help noticing parallels to my professional interest in software development and teaching. I saw in Sampras's experience some lessons that that we in CS have also learned. Here are a few.

Teaching and Humility

After Sampras had made his mark as a great player, one of his first coaches liked to be known as one of the coaches who helped make Sampras the player he was. Sampras gave that coach his due, and gave the two men who coached him for most of his pro career a huge amount of credit for honing specific elements of his game and strategy. But without sounding arrogant, he also was clear that no coach "made" him. He had a certain amount of native talent, and he was also born with the kind of personality that drove him to excel. Sampras would likely have been one of the all-time greats even if he had had different coaches in his youth, and even as a pro.

Great performers have what it takes to succeed. It is rare for a teacher to help create greatness in a student. What made Sampras's pro coaches so great themselves is not that they built Sampras but that they were able to identify the one or two things that he needed at that point in his career and helped him work on those parts of his game -- or his mind. Otherwise, they let the drive within him push him forward.

As a teacher, I try figure out what students need and help them find that. It's tough to do when teaching a class of twenty-five students, because so much of the teaching is done with the group and so cannot be tailored to the individual as much as I might like and as much as each might need. But when mentoring students, whether grad students or undergrads, a dose of humility is in order. As I think back to the very best of my past students, I realize that I was most successful when I helped them get past roadblocks or to remove some source of friction in their thinking or their doing. Their energy often energized me, and I fed off of the relationship as much as they did.

Agile Moments

The secret of greatness is working hard day in and day out. Sampras grew as a player because he had to in order to achieve his goal of finishing six straight years as #1. And the only way to do that was to add value to his game every day. This seems consistent with agile developers' emphasis on adding value to their programs every day, through small steps and daily builds. Being out there every day also makes it possible to get feedback more frequently and so make the next day's work potentially more valuable. For some reason, Sampras's comments on a commitment to being in the arena day in and day out reminded me of one of Kent Beck's early bits of writing on XP, in which he proclaimed that, and the end of the day, if you hadn't produced some code, you probably had not given your customer any value. I think Sampras felt similarly.

Finally, this paragraph from a man who never changed the model of racket he used throughout his career, even as technology made it possible for lesser players to serve bigger and hit more powerful ground strokes. Here he speaks of the court on which his legend grew beyond ordinary proportion, Centre Court at the All England Club:

I enjoyed the relative "softness" of the court; it was terrific to feel the sod give gently beneath my feet with every step. I felt catlike out there, like I was on a soft play mat where I could do as I pleased without worry, fear, or excessive wear and tear. Centre Court always made me feel connected to my craft, and the sophisticated British crowd enhanced that feeling. It was a pleasure to play before them, and they inspired me to play my best. Wimbledon is a shrine, and it was always a joy to perform there.

Whatever else the agile crowd is about, feeling connected to the craft of making software is at its heart. I like to use tools that give gently beneath my feet, that let me make progress without worry and fear. Even ordinary craftsmen such as I appreciate these feelings.

The latest issue of ACM's on-line pub Ubiquity consists of Chauncey Bell's My Problem with Design, an article that first appeared on his blog a year ago. I almost stopped reading it early on, distracted by other things and not enamored with its wordiness. (I'm one to talk about another writer's wordiness!) I'm glad I read the whole article, because Bell has an inspiring take on design for a world that has redefined the word from its classic sense. He echoes a common theme of the software patterns and software craftsmanship crowd, that in separating design from the other tasks involved in making an artifact we diminish the concept of design, and ultimately we diminish the quality of the artifact thus made.

But I was especially struck by these words:

The distinctive character of the designer shapes each design that affects us, and at the same time the designer is shaped by his/her inventions. Successful designs shape those for whom they are designed. The designs alter people's worlds, how they understand those worlds, and the character and possibilities of inhabiting those worlds. ...

Most of our contemporaries tell a different story about designing, in which designers fashion or craft artifacts (including "information") that others "use." One reason that we talk about it this way, I think, is that it can be frightening to contemplate the actual consequences of our actions. Do we dare speak a story in which, in the process of designing structures in which others live, we are designing them, their possibilities, what they attend to, the choices they will make, and so forth?

(The passage I clipped gives the networked computer as the signature example of our era.)

Successful designs shape those for whom they are designed. In designing structures for people, we design them, their possibilities.

I wonder how often we who make software think this sobering thought. How often do we simply string characters together without considering that our product might -- should?! -- change the lives of its users? My experience with software written by small, independent developers for the Mac leads me to think that at least a few programmers believe they are doing something more than "just" cutting code to make a buck.

I have had similar feelings about tools built for the agile world. Even if Ward and Kent were only scratching their own itches when they built their first unit-testing framework in Smalltalk, something tells me they knew they were doing more than "making a tool"; they were changing how they could write Smalltalk. And I believe that Kent and Erich knew that JUnit would redefine the world of the developers who adopted it.

What about educators? I wonder how often we who "design curriculum" think this sobering thought. Our students should become new people after taking even one of our courses. If they don't, then the course wasn't part of their education; it's just a line on their transcripts. How sad. After four years in a degree programs, our students should see and want possibilities that were beyond their ken at the start.

I've been fortunate in my years to come to know many CS educators for whom designing curriculum is more than writing a syllabus and showing up 40 times in a semester. Most educators care much more than that, of course, or they would probably be in industry. (Just showing up out there pays better than just showing up around here, if you can hold the gig.) But even if we care, do we really think all the time about how our courses are creating people, not just degree programs? And even if we think this way in some abstract way, how often do we let it seep down into our daily actions. That's tough. A lot of us are trying.

I know there's nothing new here. Way back, I wrote another entry on the riff that "design, well done, satisfies needs users didn't know they had". Yet it's probably worth reminding ourselves about this every so often, and to keep in mind that what we are doing today, right now, is probably a form of design. Whose world and possibilities are we defining?

This thought fits nicely with another theme among some CS educators these days, context. We should design in context: in the context of implementation and the other acts inherent in making something, yes, but also in the context of our ultimate community of users. Educators such as Owen Astrachan are trying help us think about our computing in the context of problems that matter to people outside of the CS building. Others, such as Mark Guzdial, have been preaching computing in context for a while now. I write occasionally on this topic here. If we think about the context of our students, as we will if we think of design as shaping people, then putting our courses and curricula into context becomes the natural next step.

On the PLT Scheme mailing list, someone asked why the authors of How to Design Programs do not provide unit tests for their exercises. The questioner could understand not giving solutions, but why not give the examples that the students could use to guide their thinking. A list member who is not an HtDP c-author speculated that if the authors provided unit tests then students would not bother to implement their own.

Co-author Matthias Felleisen responded "Yes" and added this stronger assertion:

I have come to believe that being able to make up your own examples (inputs, outputs) is the critical step in solving most problems.

Writing examples is one of the essential elements of the "design recipe" approach on which Felleisen et al. base How to Design Programs. The idea itself isn't new, as I'm sure the book's authors will tell you. Some CS teachers have been requiring students to write test cases or test plans for many years, and the practice is similar to what some engineers learn to do from the start of their education. Heck, test-driven design has gone from being the latest rage in agile development to an accepted (if too infrequently practiced) part of creating software.

What HtDP and TDD do is to remind us all of the importance of the practice and to make it an essential step in the student's or developer's programming process.

What struck me by Matthias's response is that making up examples is the critical step in writing code. It is certainly reasonable, for so many reasons, among them:

• It forces the programmer to write down her understanding of the problem in a concrete way early in the process. Concrete understanding is always preferable to the fuzzy-minded understanding that follows reading a problem statement. Besides, writing a program requires that level of understanding.


• Having examples in hand gives the programmer a way of talking to the client or teacher to see if her understanding matches that of the person who "owns" the problem.


• It gives the programmer a way of knowing how much code to write and so when to stop. Most students can use that guidance.

I usually give my students several examples as a part of specifying problems and ask them to write a few of their own. Most don't do much on their own and, uncharacteristically, I don't hold them accountable often enough. My next programming assignment may look different from the previous ones; I have an idea of how to sneak this little bit of design recipe thinking into the process.

When I was in grad school, my advisor sent me to a series of swank conferences on expert systems in business, finance, and accounting. Among the things these conferences did for me was to give a chance to stay at Ritz Carlton hotels. This Midwestern boy had never been treated so well.

At the 1990 conference, I heard a talk by Gary Ribar from KPMG Peat Marwick, one of the Big Six accounting consulting firms of the time. Ribar described LoanProbe, a program that evaluated the collectibility of commercial loans. LoanProbe was a rule-based system organized in a peculiar way, with its 9000 rules separated into thirty-three separate "knowledge bases". It was a significant application that interacted with sixty external programs and two large data bases. Peat Marwick took LoanProbe a step further and used its organizational technique to build a knowledge acquisition program that enabled non-programmers to create systems with a similar structure.

I was so excited. I recognized this technique as what we in our lab called structured matching, a remarkably common and versatile pattern in knowledge-based systems. LoanProbe looked to me like the largest documented application of structured matching, which I was working on as a part of my research. Naturally, I wanted to make a connection to this work and share experiences with the speaker.

After the talk, I waited in line to speak with him. When my turn came, I gushed that their generic architecture was very cool and that "we do something just like that in our lab!". I expected camaraderie, but all I received back was an icy stare, a curt response, and an end to the conversation.

I didn't understand. Call me naive. For a while, I wondered if Mr. Ribar was simply an unfriendly guy. Then I realized that he probably took my comment not as a compliment -- We do that, too! -- but as a claim that the work he described was less valuable because it was not novel. I realized that I had violated one of the basic courtesies of research by telling him that his work was known already.

These days, I think fondly of Ribar, that talk, and that conference. He was behaving perfectly reasonably, given the culture in which he worked. Novelty is prized.

A few years after that conference, I came across PLoP and the software patterns community. This group of people valued discovering, documenting, and sharing common solutions, the patterns that make our software and our programming lives better. Structured Matcher did that, and it appeared in programs from all sorts of domain.

Jeff Patton has described this feature of the patterns community nicely in his article on emerging best practices in user experience:

If you tell someone a great idea, and they say "Yes, we do something like that too!", that's a pattern.

Documenting and sharing old, proven solutions that expert practitioners use may not get you a publication in the best research journal (though it might, if you are persistent and fortunate), but it will make the world better for programmers and software users. That is valuable, too.

Fall semester is just around the corner. Students will begin to arrive on campus next week, and classes start a week from Monday. I haven't been able to spend much time on my class yet and am looking forward to next week, when I can.

What I have been doing is clearing a backlog of to-dos from the summer and handling standing tasks that come with the start of a new semester and especially a new academic year. This means managing several different to-do lists, crossing priorities, and generally trying to get things done.

As I look at this mound of things to do I can't help being reminded of something Jeff Patton blogged a month or so ago: two secrets of success in software development, courtesy of agile methods pioneer Jim Highsmith: start sooner, and do less.

Time ain't some magical quantity that I can conjure out of the air. It is finite, fixed, and flowing relentlessly by. If I can't seem to get done on time, I need to start sooner. If I can't seem to get it all done, I need to do less. Nifty procedures and good tools can help only so much.

I need to keep this in mind every day of the year.

Oh, and to you students out there: You may not be able to do less work in my class, but you can start sooner. You may have said so yourself at the end of last semester. Heck, you may even want to do more, like read the book...

I once wrote that extreme programming is a self-help system. This generalizes pretty well to other software methodologies, too. As we step away from developing software to personal hygiene, there is an entire ecosystem around the notion of life hacks, self-help for managing information and combatting data overload. Programmers and techies are active players in the lifehacking community because, well, we love to make tools to solve our problems and we love self-help systems. In the end, sometimes, we spend more time making tools and playing with them than actually solving our problems.

One of the popular lifehacking systems among techies is David Allen's Getting Things Done, or GTD. I've never read the book or adopted the system, but I've read about it and borrowed some of its practices in trying to treat my own case of information overload. The practices I have borrowed feel a lot like XP and especially test-driven development. Maybe that's why they appeal to me.

Consider this post on the basic concepts of GTD. Here is why GTD makes me think of TDD:

1. think in terms of outcomes: write a test
2. take the next action: take a simple action
3. review your circumstances regularly: refactor

This is not a perfect match. In GTD, a goal from Step 1 may require many next actions, executed in sequence. In TDD, we decompose such big goals into smaller steps so that we can define a very clear next action to perform. And in GTD, Step 3 isn't really refactoring of a system. It's more a global check of where you are and how your lists of projects and next actions need to be revised or pruned. What resonates, though, is its discipline of regular review of where you are headed and how well your current 'design' can get you there.

It's not a perfect match, but then no metaphor is. Yet the vibe feels undeniably similar to me. Each has a mindset of short-term accountability through tests, small steps to achieve simple, clear goals, and regular review and clean-up of the system. The lifehackers who play with GTD even like to build tools to automate as much as they can so that they stay in the flow of getting things done as much as possible and trust their tools to help them manage performance and progress.

Successful patterns recur. I shouldn't be surprised to find these similarities.

Greg Wilson relates an observation by Michael Feathers: "refactoring pure functional code is a lot easier than refactoring imperative code". In one sense, this ought not to surprise us. When we eliminate side effects from our code, dependencies among functions flow through parameters, which make individual functions more predictably independent of one another. Without side effects, we don't have sequences of statements, which encourages smaller functions, which also makes it easier to understand the functions.

(A function call does involve sequencing, because arguments are evaluated before the function is invoked. But this encourages small functions, too: Deeply-nested expressions can be quite hard to read.)

There is another force counteracting this one, though. Feathers has been playing a lot with Haskell, which is strongly-typed through manifest types and type inferencing. Many functional languages are dynamically-typed, and dynamic typing makes it harder to refactor functional programs -- at least to guarantee that a particular refactoring does not change the program's behavior.

I'm a Scheme programmer when I use a functional language, so I encounter the conflict between these two forces. My suspicion from personal experience is that functional programmers need less support, or at least different kinds of support, when it comes to refactoring tools. The first key step is to identify refactorings from FP practice. From there, we can find ways to automate support for these refactorings. This is a longstanding interest of mine. One downside to my current position is a lack of time to devote to this research project...

Ralph Johnson pointed me to a design idea for very large databases called a shard. This is a neat little essay for several reasons. First, its author, Todd Hoff, explains an architecture for massive, distributed databases that has grown up in support of several well-known, high-performance web sites, including Flickr, Google, and LiveJournal. Second, Hoff also wrote articles that describe the architectures of Flickr, Google, and LiveJournal. Third, all four pages point to external articles that are the source of the information summarized. Collectively, these pages make a wonderful text on building scalable data-based web systems.

I've posted this entry in my Patterns category because this recurring architecture has all the hallmarks of a design pattern. It even has great name and satisfies Rule Of Three, something I've mentioned before -- and what a fine three it is. Each implementation uses the idea of a shard slightly differently, in fitting with the particular forces at play in the three companies' systems.

Buried near the bullet list on the Google page was an item worth repeating:

Don't ignore the Academy. Academia has a lot of good ideas that don't get translated into production environments. Most of what Google has done has prior art, just not prior large scale deployment.

This advice is a bit different from some advice I once shared for entrepreneurs, looking for Unix commands that haven't been implemented on the web yet, but the spirit is similar. Sometimes I hear envious people remark that Google hasn't done anything special; they just used a bunch of ideas others created to build a big system. Now, I don't think that is strictly true, but I do think that many of the ideas they used existed before in the database and networking worlds. And to the extent that is true, good for them! They paid attention in school, read beyond their assignments, and found some cool ideas that they could try in practice. Isn't that the right thing to do?

In any case, I recommend this article and encourage others to write more like it.

A recent entry mentioned that one advantage of short source code for beginners is a smaller space for errors. If a student writes only three lines of code, then any error in the program is probably on one of those three lines. That's better than looking for errors in a 100-line program, at least when the programmer is learning.

This assertion may seem like an oversimplification. What if the students writes a bunch of three-line procedures that call one another? Couldn't an error arise out of the interaction of multiple procedures, and thus lie far from the point at which it is discovered? Sure, but that is usually only a problem if the student doesn't know that each three-line procedure works. If we develop the habit of testing each small piece of code well, or even reasoning formally about its behavior, then we can have confidence in the individual pieces, which focuses the search for an error in the short piece of code that calls them.

This is, of course, one of the motivations behind the agile practices of taking small steps and creating tests for each piece of code as we go along. It is also why programming in a scripting language can help novices. The language provides powerful constructs, which allow the novice programmer to say a lot in a small amount of code. We can trust the language constructs to work correctly, and so focus our search for errors in the small bit of code.

Even still, it's not always as simple as it sounds. I am reminded of an article on a new course proposed by Matthias Felleisen, in which he argues for the use of a limited proof language. Even when we think that a 'real' language is small enough to limit the scope of errors students can make, we are usually surprised. Felleisen comments on the Teach Scheme! experience:

... we used to give students the "simple language of first-order Lisp" and the code we saw was brutal. Students come up with the worst possible solution that you can imagine, even if you take this sentence into account in your predictions.

This led the Teach Scheme! team to create a sequence of language levels that expose students to increasingly richer sets of ideas and primitives, culminating in the complete language. This idea has also been in the Java world, via Dr. Java. Another benefit of using limited teaching languages is that the interpreter or compiler can provide much more specific feedback to students at each level because it, too, can take advantage of the smaller space of possible errors.

Felleisen does not limit the idea of limited language to the programming language. He writes of carefully introducing students to the vocabulary we use to talk about programming:

Freshmen are extremely limited in their vocabulary and "big words" (such as 'specification' and 'implementation') seem to intimidate them. We introduce them slowly and back off often.

When I read this a couple of weeks ago, it troubled me a bit. Not because I disagree with what Felleisen says, but because it seems to conflict with something else I believe and blogged about couple of weeks ago: speak to students in real language, and help the students grow into the language. I have had good experience with children, including my own, when talking about the world in natural language. What makes the experience of our students different.

As I write this, I am less concerned that these conflict. First, Felleisen mentions one feature of the CS1 experience that distinguishes it from my kids' experience growing up: fear. Children don't spend a lot of their time afraid of the world; they are curious and want to know more. They are knowledge sponges. CS1 students come out of a school system that tends to inculcate fear and dampen curiosity, and they tend to think computer science is a little scary -- despite wanting to major in it.

Second, when I speak to children in my usual vocabulary, I take the time to explain what words mean. Sometimes they ask, and sometimes I notice a quizzical curious look on their faces. Elaboration of ideas and words gives us more to talk about (a good thing) and connects to other parts of their knowledge (also good). And I'm sure that I don't speak to kids using only thirteen-letter words; that's not the nature of regular life, at least in my house. In computing jargon words of excessive length are the norm.

So I don't think there's a contradiction in these two ideas. Felleisen is reminding us to speak to students as if they are learners, which they are, and to use language carefully, not simplistically.

Even if there is a contradiction, I don't mind. It would not be the only contradiction I bear. Strange as it may sound, I try to be true to both of these ideas in my teaching. I try not to talk down to my students, instead talking to them about real problems and real solutions and cool ideas. My goal is to help students reach up to the vocabulary and ideas as they need, offering scaffolding in language and tools when they are helpful.

Yesterday, I wrote a bit about scripting languages. It seems odd to have to talk about the value of scripting languages in 2008, as Ronald Loui does in his recent IEEE Computer article, but despite their omnipresence in industry, the academic world largely continues to prefer traditional systems languages. Some of us would like to see this change. First, let's consider the case of novice programmers.

Most scripting languages lack some of the features of systems languages that are considered important for learners, such as static typing. Yet these "safer" languages also get in the way of learning, as Loui writes, by imposing "enterprise-sized correctness" on the beginner.

Early programmers must learn to be creative and inventive, and they need programming tools that support exploration rather than production.

This kind of claim has been made for years by advocates of languages such as Scheme for CS1, but those languages were always dismissed by "practical" academics as toy languages or niche languages. Those people can't dismiss scripting languages so easily. You can call Python and Perl toy languages, but they are used widely in industry for significant tasks. The new ploy of these skeptics is to speak of the "scripting language du jour" and to dismiss them as fads that will disappear while real languages (read: C) remain.

What scripting language would be the best vehicle for CS1? Python has had the buzz in the CS ed community for a while. After having taught a little PHP las semester, I would deem it too haphazard for CS1. Sure, students should be able to do powerful things, but the pocket-protected academic in me prefers a language that at least pretends to embody good design principles, and the pragmatist in me prefers a language that offers a smoother transition into languages beyond scripting. JavaScript is an idea I've seen proposed more frequently of late, and it is a choice with some surprising positives. I don't have enough experience with it to say much, but I am a little concerned about the model that programming in a browser creates for beginning students.

Python and Ruby do seem like the best choices among the scripting languages with the widest and deepest reach. As Loui notes, few people dislike either, and most people respect both, to some level. Both have been designed carefully enough to be learned by beginners and and support a reasonable transition as students move to the next level of the curriculum. Having used both, I prefer Ruby, not only for its OO-ness but also for how free I feel when coding in it. But I certainly respect the attraction many people have to Python, especially for its better developed graphics support.

Some faculty ask whether scripting languages scale to enterprise-level software. My first reaction is: For teaching CS1, why should we care? Really? Students don't write enterprise-level software in CS1; they learn to program. Enabling creativity and supporting exploration are more important than the speed of the interpreter. If students are motivated, they will write code -- a lot of it. Practice makes perfect, not optimized loop unrolling and type hygiene.

My second reaction is that these languages scale quite nicely to real problems in industry. That is why they have been adopted so widely. If you need to process a large web access log, you really don't want to use Java, C, or Ada. You want Perl, Python, or Ruby. This level of scale gives us access to real problems in CS1, and for these tasks scripting languages do more than well enough. Add to that their simplicity and the ability to do a lot with a little code, and student learning is enhanced.

Loui writes, "Indeed, scripting languages are not the answer for long-lasting, CPU-intensive nested loops." But then, Java and C++ and Ada aren't the answer for all the code we write, either. Many of the daily tasks that programmers perform lie in the space better covered by scripting languages. After learning a simpler language that is useful for these daily tasks, students can move on to larger-scale problems and learn the role of a larger-scale language in solving them. That seems more natural to me than going in the other direction.

Now let's consider the case of academic programming languages research. A lot of interesting work is being done in industry on the design and implementation of scripting language, but Loui laments that academic PL research still focus on syntactic and semantic issues of more traditional languages.

Actually, I see a lot of academic work on DSLs -- domain-specific languages -- that is of value. One problem is this research is so theoretical that it is beyond the interest of programmers in the trenches. Then again, it's beyond the mathematical ability and interest of many CS academics, too. (I recently had to comfort a tech entrepreneur friend of mine who was distraught that he couldn't understand even the titles of some PL theory papers on the resume of a programmer he was thinking of hiring. I told him that the lambda calculus does that to people!)

Loui suggest that PL language research might profitably move in a direction taken by linguistics and consider pragmatics rather than syntax and semantics. Instead of proving something more about type systems, perhaps a languages researcher might consider "the disruptive influence that Ruby on Rails might have on web programming". Studying how well "convention over configuration" works in practice might be of as much use as incrementally extending a compiler optimization technique. The effect of pragmatics research would further blur the line between programming languages and software engineering, a line we have seen crossed by some academics from the PLT Scheme community. This has turned out to be practical for PL academics who are interested in tools that support the programming process.

Loui's discussion of programming pragmatics reminds me of my time in studying knowledge-based systems. Our work was pragmatic, in the sense that we sought to model the algorithms and data organization that expert problem solvers used, which we found to be tailored to specific problem types. Other researchers working on such task-specific architectures arrived at models consistent with ours. One particular group went beyond modeling cognitive structures to the sociology of problem solving, John McDermott's lab at Carnegie Mellon. I was impressed by McDermott's focus on understanding problem solvers in an almost anthropological way, but at the time I was hopelessly in love with the algorithm and language side of things to incorporate this kind of observation into my own work. Now, I recognize it as the pragmatics side of knowledge-based systems.

(McDermott was well known in the expert systems community for his work on the pioneering programs R1 and XCON. I googled him to find out what he was up to these days but didn't find much, but through some publications, I infer that he must now be with the Center for High Assurance Computer Systems at the Naval Research Laboratory. I guess that accounts for the sparse web presence.)

Reading Loui's article was an enjoyable repast, though even he admits that much of the piece reflects old arguments from proponents of dynamic languages. It did have, I think, at least one fact off track. He asserts that Java displaced Scheme as the primary language used in CS1. If that is true, it is so only for a slender subset of more elite schools, or perhaps that Scheme made inroads during a brief interregnum between Java and ... Pascal, a traditional procedural language that was small and simple enough to mostly stay out of the way of programmers and learners.

As with so many current papers, one of the best results of reading it is a reminder of a piece of classic literature, in this case Ousterhout's 1998 essay. I usually read this paper again each time I teach programming languages, and with my next offering of that course to begin in three weeks, the timing is perfect to read it again.

Colleague and reader Michael Berman pointed me to the July 2008 issue of IEEE Computer, which includes an article on the virtues of scripting languages, Ronald Loui's In Praise of Scripting: Real Programming Pragmatism. Loui's inspiration is an even more important article in praise of scripting, John Ousterhout's classic Scripting: Higher Level Programming for the 21st Century. Both papers tell us that scripting deserves more respect in the hierarchy of programming and that scripting languages deserve more consideration in the programming language and CS education communities.

New programming languages come from many sources, but most are created to fill some niche. Sometimes the niche is theoretical, but more often the creators want to be able to do something more easily than they can with existing languages. Scripting languages in particular tend to originate in practice, to fill a niche in the trenches, and grow from there. Sometimes, they come to be used just like a so-called general-purpose programming language.

When programmers have a problem that they need solve repeatedly, they want a language that gives them tools that are "ready at hand". For these programming tasks, power comes from the level of abstraction provided by built-in tools. Usually these tools are chosen to fill the needs of a specific niche, but they almost always include the ability to process text conveniently, quickly, and succinctly.

Succinctness is a special virtue of scripting languages. Loui mentions the virtue of short source code, and I'm surprised that more people don't talk about the value of small programs. Loui suggests one advantage that I rarely see discussed: languages that allow and even encourage short programs enable programmers to get done with a task before losing motivation or concentration. I don't know how important this advantage is for professional programmers; perhaps some of my readers who work in the real world can tell me what they think. I can say, though, that, when working with university students, and especially novice programmers, motivation or concentration are huge factors. I sometimes hear colleagues say that students who can't stay motivated and concentrate long enough to solve an assignment in C++, Ada, or Java probably should not be CS majors. This seems to ignore reality, both of human psychology and of past experience with students. Not to mention the fact that teach non-majors, too.

Another advantage of succinctness Loui proposes relates to programmer error. System-level languages include features intended to help programmers make fewer errors, such as static typing, naming schemes, and verbosity. But they also require programmers to spend more time writing code and to write more code, and in that time programmers find other ways to err. This is, too, is an interesting claim if applied to professional software development. One standard answer is that software development is not "just" programming and that such errors would disappear if we simply spent more time up-front in analysis, modeling, and design. Of course, these activities add even more time and more product to the lifecycle, and create more space for error. They also put farther in the future the developers' opportunity to get feedback from customers and users, which in many domains is the best way to eliminate the most important errors that can arise when making software.

Again, my experience is that students, especially CS1 students, find ways to make mistakes, regardless of how safe their language is.

One way to minimize errors and their effects is to shrink the universe of possible errors. Smaller programs -- less code -- is one way to do that. It's harder to make as many or even the same kind of errors in a small piece of code. It's also easier to find and fix errors in a small piece of code. There are exceptions to both of these assertions, but I think that they hold in most circumstances.

Students also have to be able to understand the problem they are trying to solve and the tools they are using to solve it. This places an upper bound on the abstraction level we can allow in the languages we give our novice students and the techniques we teach them. (This has long been an argument made by people who think we should not teach OO techniques in the first year, that they are too abstract for the minds of our typical first-year students.) All other things equal, concrete is good for beginning programmers -- and for learners of all kinds. The fact that scripting languages were designed for concrete tasks means that we are often able to make the connection for students between the languages abstractions and tasks they can appreciate, such as manipulating images, sound, and text.

My biases resonate with this claim in favor of scripting languages:

Students should learn to love their own possibilities before they learn to loathe other people's restrictions.

I've always applied this sentiment to languages such as Smalltalk and Scheme which, while not generally considered scripting languages, share many of the features that make scripting languages attractive.

In this regard, Java and Ada are the poster children in my department's early courses. Students in the C++ track don't suffer from this particular failing as much because they tend not to learn C++ anyway, but a more hygienic C. These students are more likely to lose motivation and concentration while drowning in an ocean of machine details.

When we consider the problem of teaching programming to beginners, this statement by Loui stands out as well:

Students who learn to script early are empowered throughout their college years, especially in the crucial Unix and Web environments.

Non-majors who want to learn a little programming to become more productive in their disciplines of choice don't get much value at all from one semester learning Java, Ada, or C++. (The one exception might be the physics majors, who do use C/C++ later.) But even majors benefit from learning a language that they might use sooner, say, in a summer job. A language like PHP, JavaScript, or even Perl is probably the most valuable in this regard. Java is the one "enterprise" language that many of our students can use in the summer jobs they tend to find, but unfortunately one or two semesters are not enough for most of them to master enough of the language to be able to contribute much in a professional environment.

Over the years, I have come to think that even more important than usefulness for summer jobs is the usefulness a language brings to students in their daily lives, and the mindset it fosters. I want CS students to customize their environments. I want them to automate the tasks they do every day when compiling programs and managing their files. I want them to automate their software testing.

When students learns a big, verbose, picky language, they come to think of writing a program as a major production, one that may well cause more pain in the short term than it relieves in the long term. Even if that is not true, the student looks at the near-term pain and may think, "No, thanks." When students learn a scripting language, they can see that writing a program should be as easy as having a good idea -- "I don't need to keep typing these same three commands over and over", or "A program can reorganize this data file for me." -- and writing it down. A program is an idea, made manifest in an executable form. They can make our lives better. Of all people, computer scientists should be able to harness their power -- even CS students.

This post has grown to cover much more than I had originally planned, and taken more time to write. I'll stop here for now and pick up this thread of thought in my next entry.

The recent rescue of hostages in Colombia relied on a strategy familiar to people interested in computer network security: a man-in-the-middle attack.

... for months, in an operation one army officer likened to a "broken telephone," military intelligence had been able to convince Ms. Betancourt's captor, Gerardo Aguilar, a guerrilla known as "Cesar," that he was communicating with his top bosses in the guerrillas' seven-man secretariat. Army intelligence convinced top guerrilla leaders that they were talking to Cesar. In reality, both were talking to army intelligence.

As Bruce Schneier reports in Wired magazine, this strategy is well-known on the internet, both to would-be system crackers and to security experts. The risk of man-in-the-middle attacks is heightened on-line because the primary safeguard against them -- shared social context -- is so often lacking. Schneier describes some of the technical methods available for reducing the risk of such attacks, but his tone is subdued... Even when people have a protection mechanism available, as they do in SSL, they usually don't take advantage of it. Why? Using the mechanism requires work, and most of us are just too lazy.

Then again, the probability of being victimized by a man-in-the-middle attack may be small enough that many of us can rationalize that the cost is greater than the benefit. That is a convenient thought, until we are victimized!

The problem feature that makes man-in-the-middle attacks possible is unjustified trust. This is not a feature of particular technical systems, but of any social system that relies on mediated communication. One of the neat things about the Colombian hostage story it that shows that some of the problems we study in computer science are relevant in a wider context, and that some of our technical solutions can be relevant, too. A little computer science can amplify the problem solving of almost anyone who deals with "systems", whatever their components.

This story shows a potential influence from computing on the wider world. Just so that you know the relationship runs both ways, I point you to Joshua Kerievsky's announcement of "Programming with the Stars", one of the events on the Developer Jam stage at the upcoming Agile 2008 conference. Programming with the Stars adapts the successful formula of Dancing with the Stars, a hit television show, to the world of programming. On the TV show, non-dancers of renown from other popular disciplines pair with professional dancers for a weekly dance competitions. Programming with the Stars will work similarly, only with (pair) programming plugged in for dancing. Rather than competitions involving samba or tango, the competitions will be in categories such as test-driven development of new code and refactoring a code base.

As in the show, each pair will include an expert and a non-expert, and there will be a panel of three judges:

• Rachel Davies
• Uncle Bob Martin
• Crazy Mike Hill

I've already mentioned Uncle Bob in this blog, even in a humorous vein, and I envision him playing the role of Simon Cowell from "American Idol". How Davies and Hill compare to Paula Abdul and Randy Jackson, I don't know. But I expect plenty of sarcasm, gushing praise, and hip lingo from the panel, dog.

Computer scientists and software developers can draw inspiration from pop culture and have a little fun along the way. Just don't forget that the ideas we play with are real and serious. Ask those rescued hostages.

Most papers presented at SIGCSE and the OOPSLA Educators' Symposium are about teaching methods, not computational methods. When the papers do contain new technical content, it's usually content that isn't really new, just new to the audience or to mainstream use in the classroom. The most prominent example of the latter that comes to mind immediately is the series of papers by Zung Nguyen and Stephen Wong at SIGCSE on design patterns for data structures. Those papers were valuable in principle because they showed that how one conceives of containers changes when one is working with objects. In practice, they sometimes missed their mark because they were so complex that many teachers in the audience said, "Cool! But I can't do that in class."

However, the OOPSLA Educators' Symposium this year received a submission with a cool object-oriented implementation of a common introductory programming topic. Unfortunately, it may not have made the cut for inclusion based on some technical concerns of the committee. Even so, I was so happy to see this paper and to play with the implementation a little on the side! It reminded me of one of the first efforts I saw in a mainstream CS book to show how we think differently about a problem we all know and love when working with objects. That was Tim Budd's implementation of the venerable eight queens problem in An Introduction to Object-Oriented Programming.

Rather than implement the typical procedural algorithm in an object-oriented language, Budd created a solution that allowed each queen to solve the problem for herself by doing some local computation and communicating with the queen to her right. I remember first studying his code to understand how it worked and then showing it to colleagues. Most of them just said, "Huh?" Changing how we think is hard, especially when we already have a perfectly satisfactory solution for the problem in mind. You have to want to get it, and then work until you do.

You can still find Budd's code from the "download area" link on the textbook's page, though you might find a more palatable version in the download area for the book's second edition. I just spent a few minutes creating a Ruby version, which you are welcome to. It is slightly Ruby-ized but mostly follows Budd's solution for now. (Note to self: have fun this weekend refactoring that code!)

Another thing I liked about "An Introduction to Object-Oriented Programming" was its linguistic ecumenism. All examples were given in four languages: Object Pascal, C++, Objective C, and Smalltalk. The reader could learn OOP without tying it to a single language, and Budd could point out subtle differences in how the languages worked. I was already a Smalltalk programmer and used this book as a way to learn some Objective C, a skill which has been useful again this decade.

(Budd's second edition was a step forward in one respect, by adding Java to the roster of languages. But it was also the beginning of the end. Java soon became so popular that the next version of his book used Java only. It was still a good book for its time, but it lost some of its value when it became monolingual.)

In March I talked about a couple of OOPSLA submissions written by our merged ChiliPLoP groups. In May I wrote about the verdict on one but forgot to mention the other. Maybe because the rejection was so much more interesting!

Anyone, our second submission was accepted into the Educators' Symposium. It is not a paper, really, but an extended abstract for an activity we will run: a code review recast as it might happen in a software studio. We hope to give participants a snapshot of what a studio-based course looks, feels, and works like. This is something instructors can try on a small scale in class and, if it works for them, expand throughout their course. Even if code reviews is all the farther they go, we co-authors think that this will be a useful step for many instructors. It draws on experiences in the writers' workshops of PLoP helps students to think about the many design choices they make when writing software, and to make them reflectively rather than subconsciously.

The real trick to this activity will be the homework we give before the symposium:

Before coming to OOPSLA, Educators' Symposium participants will be asked to submit a program, in a language of their choice (though using only standard libraries), which implements the core of a program to generate Tag Clouds from a data set. ...

My experience with many workshops in the past and especially with the Educators' Symposium is that participants never do this kind of homework. Some are well-intentioned but never make time for it, while others figure they can skate by without having written the code. (Sounds as if professors are a lot like their students, huh?) Without code to review, a code review doesn't get very far. We hope that we can find a way to encourage symposium attendees to overcome history and come with some code to workshop.

I've long been a fan of William James, and once wrote briefly about the connection between James's pragmatism and my doctoral work on knowledge-based systems. I was delighted yesterday to run across this quote from James's The Principles of Psychology, courtesy of 43 Folders and Linda Stone:

[Attention] is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. ... It implies withdrawal from some things in order to deal effectively with others....

Prone as I am to agile moments, this message from James struck me in an interesting way. First of all, I occasionally battle the issue that Stone writes about, the to-do list that grows no matter productive I seem to be on a given day. (And on lazy summer June days, well, all bets are off.) James tells me that part of my problem isn't a shortage of time, but a lack of will to focus. I need to make better, more conscious choices about what tasks to add to the list. Kent Beck is fond of saying something to the effect that you may have too many things to do and too little time, but you ultimately have control over only one side of the equation. James would tell us the same thing.

My mind also made a connection from this quote to the agile software and test-driven development practice of working on small stories, on taking small steps. If I pick up a card with a single, atomic, well-defined feature to be added to my program, I am able to focus. What is the shortest step I can take and make this feature part of my code? No distractions, no Zerstreutheit. Though I have an idea in mind toward where my program is evolving, for this moment I attend to one small feature and make it work. Focus. James would be proud.

I think it's ironic in a way that one of the more effective ways to reach the state of flow is to decompose a task into the smallest of tasks and focus on them one at a time. The mind gets into a rhythm of red bar-green bar: select task, write code, refactor, and soon it is deep in its own world. I would like to be more effective at doing this in my non-programming duties. Perhaps if I keep James and his quote in mind, I can be.

This idea applies for me in other areas, in particular in running and training for particular events. Focusing each day on a particular goal -- intervals, Long Slow Distance, hill strength, and so on -- helps the mind to push aside its concerns with other parts of the game and attend to a particular kind of improvement. There is a great sense of relaxation in running super-hard repeats when the problem I've been having is, say, picking up pace late in a run. (I'd love to run super-hard repeats again some day soon, but I'm not there yet.)

I ran across a pattern today that reminded of another I encountered while at ChiliPLoP. Both help developers deal with side effects, changes to a program's state. Let me share these patterns with you all, with a common explanation of their value.

Most programs written these days are in a style where setting and changing the values of one or more variables, via a sequence of imperative statements. Side effects are a common source of programmers' misunderstanding when reading code. A program can change a variable's state almost anywhere, and that makes reading any bit of code uneasy: "Do I really know what this variable means right now?" Using only local variables only in the context of a small procedure is one way to localize effects. Even still, problems can arise.

I won't try write full patterns for these. I'll write short patlets and give you links to the articles I read, which do a good job talking about the ideas.

Mutual Recursion with Side Effects

In Solving Every Sudoku Puzzle, Peter Norvig builds a Python program for the title task. About 40% of the way in, he says:

If you have two mutually-recursive functions that both alter the state of an object, try to move almost all the functionality into just one of the functions. Otherwise you will probably end up duplicating code.

Duplicate code is usually a bad thing, because it creates a problem for programmers keeping the copies in sync. Duplicate code that modifies state is especially bad, because it brings in the extra complication of making another procedure harder to understand.

Side Effects and Python's Default Parameters

Coincidentally, the second pattern involves Python, too. This time the pattern is Python-specific, addressing an issue with a language feature.

Context: You are writing a procedure with a default parameter.

Problem: The procedure needs to modify the default parameter. Python evaluates a default parameter's initial value only on the first call to the procedure.

Solution: Whenever the parameter is missing, initialize it within the method.

Tim Ottinger gives his implementation technique in Python's Mutable Default Problem:

def function( item, stuff=None ):
   stuff = stuff or []
   # ... modify stuff, etc.

This pattern allows the user to take advantage of the default parameter when he has no collection to send. It does so by following a principle laid out in the article Ottinger references: In Python, "don't use mutable objects as function defaults."

This pattern may not be Python-specific, because there may be other languages with this initialize-once behavior. It seems like a bug to me, not a feature, but I'm not a Python programmer and so can't speak authoritatively. But I am glad that Ruby doesn't do this.

~~~~~

Postscript: While preparing this article, I learned something new about Ruby, unrelated to this issue. It is possible to extract from a class a new class that has a subset of the original class's protocol. Very nice indeed!)

The verdict is in on the paper we wrote at ChiliPLoP and submitted to Onward!: rejected. (We are still waiting to hear back on our Educators' Symposium submission.) The reviews of our Onward! paper were mostly on mark, both on surface features (e.g., our list of references was weak) and on the deeper ideas we offer (e.g., questions about the history of studio approaches, and questions about how the costs will scale). We knew that this submission was risky; our time was simply too short to afford enough iterations and legwork to produce a good enough paper for Onward!.

I found it interesting that the most negative reviewer recommended the paper for acceptance. This reviewer was clearly engaged by the idea of our paper and ended up writing the most thorough, thoughtful review, challenging many of our assumptions along the way. I'd love to have the chance to engage this person in conversation at the conference. For now, I'll have to settle for pointing out some of the more colorful and interesting bits of the review.

In at least one regard, this reviewer holds the traditional view about university education. When it comes to the "significant body of knowledge that is more or less standard and that everyone in the field should acquire at some point in time", "the current lecture plus problem sets approach is a substantially more efficient and thorough way to do this."

Agreed. But isn't it more efficient to give the students a book to read? A full prof or even a TA standing in a big room is an expensive way to demonstrate standard bodies of knowledge. Lecture made more sense when books and other written material were scarce and expensive. Most evidence on learning is that lecture is actually much less effective than we professors (and the students who do well in lecture courses) tend to think.

The reviewer does offer one alternative to lecture: "setting up a competition based on mastery of these skills". Actually, this approach is consistent with the spirit of our paper's studio-based, apprenticeship-based, and project-based. Small teams working to improve their skills in order to win a competition could well inhabit the studio. Our paper tended to overemphasize the softer collaboration of an idyllic large-scale team.

This comment fascinated me:

Another issue is that this approach, in comparison with standard approaches, emphasizes work over thinking. In comparison with doing, for example, graph theory or computational complexity proofs, software development has a much lower ratio of thought to work. An undergraduate education should maximize this ratio.

Because I write a blog called Knowing and Doing, you might imagine that I think highly of the interplay between working and thinking. The reviewer has a point: an education centered on projects in a studio must be certain to engage students with the deep theoretical material of the discipline, because it is that material which provides the foundation for everything we do and which enables us to do and create new things. I am skeptical of the notion that an undergrad education should maximize the ratio of thinking to doing, because thinking unfettered by doing tends to drift off into an ether of unreality. However, I do agree that we must try to achieve an appropriate balance between thinking and doing, and that a project-based approach will tend to list toward doing.

One comment by the reviewer reveals that he or she is a researcher, not a practitioner:

In my undergraduate education I tried to avoid any course that involved significant software development (once I had obtained a basic mastery of programming). I believe this is generally appropriate for undergraduates.

Imagine the product of an English department saying, "In my undergraduate education I tried to avoid any course that involved significant composition (once I had obtained a basic mastery of grammar and syntax). I believe this is generally appropriate for undergraduates." I doubt this person would make much of a writer. He or she might be well prepared, though, to teach lit-crit theory at a university.

Most of my students go into industry, and I encourage them to take as many courses as they can in which they will build serious pieces of software with intellectual content. The mixture of thinking and doing stretches them and keeps them honest.

An education system that produces both practitioners and theoreticians must walk a strange line. One of the goals of our paper was to argue that a studio approach could do a better job of producing both researchers and practitioners than our current system, which often seems to do only a middling job by trying to cater to both audiences.

I agree wholeheartedly, though, with this observation:

A great strength of the American system is that it keeps people's options open until very late, maximizing the ability of society to recognize and obtain the benefits of placing able people in positions where they can be maximally productive. In my view this is worth the lack of focus.

My colleagues and I need to sharpen our focus so that we can communicate more effectively the notion that a system based on apprenticeship and projects in a studio can, in fact, help learners develop as researchers and as practitioners better than a traditional classroom approach.

The reviewer's closing comment expresses rather starkly the challenge we face in advocating a new approach to undergraduate education:

In summary, the paper advocates a return to an archaic system that was abandoned in the sciences for good reason, namely the inefficiency and ineffectiveness of the advocated system in transmitting the required basic foundational information to people entering the field. The write-up itself reflects naive assumptions about the group and individual dynamics that are required to make the approach succeed. I would support some of the proposed activities as part of an undergraduate education, but not as the primary approach.

The fact that so many university educators and graduates believe our current system exists in its current form because it is more efficient and effective than the alternatives -- and that it was designed intentionally for these reasons -- is a substantial cultural obstacle to any reform. Such is the challenge. We owe this reviewer our gratitude for laying out the issues so well.

In closing, I can't resist quoting one last passage from this review, for my friends in the other sciences:

The problem with putting students with no mastery of the basics into an apprenticeship position is that, at least in computer science, they are largely useless. (This is less true in sciences such as biology and chemistry, which involve shallower ideas and more menial activities. But even in these sciences, it is more efficient to teach students the basics outside of an apprenticeship situation.)

The serious truth behind this comment is the one that explains why building an effective computer science research program around undergraduates can be so difficult. The jocular truth behind it is that, well, CS is just plain deeper and harder! (I'll duck now.)

I've tried to explain the idea of software patterns in a lot of different ways, to a lot of different kinds of people. Reading James Tauber's Grammar Rules reminds me of one of my favorites: a pattern language is a descriptive grammar. Patterns describe how (good) programmers "really speak" when they are working in the trenches.

Talking about patterns as grammar creates the potential for the sort of misunderstanding that Tauber discusses in his entry. Many people, including many linguists, think of grammar rules as, well, rules. I was taught to "follow the rules" in school and came to think of the rules as beyond human control. Linguists know that the rules of grammar are man-made, yet some still seem to view them as prescriptive:

It is as if these people are viewing rules of grammar like they would road rules--human inventions that one may disagree with, but which are still, in some sense, what is "correct"...

Software patterns are rarely prescriptive in this sense. They describe a construct that programmers use in a particular context to balance the forces at play in the problem. Over time, they have been found useful and so recur in similar contexts. But if a programmer decides not to use a pattern in a situation where it seems to apply, the programmer isn't "wrong" in any absolute sense. But he'll have to resolve the competing forces in some other way.

While the programmer isn't wrong, other programmers might look at him (or, more accurately, his program) funny. They will probably ask "why did you do it that way?", hoping to learn something knew, or at least confirm that the programmer has done something oddly.

This is similar to how human grammar works. If I say, "Me wrote this blog", you would be justified in looking at me funny. You'd probably think that what I speaking incorrectly.

Tauber points out that, while I might be violating the accepted rules of grammar, I'm not wrong in any absolute sense:

... most linguists focus on modeling the tacit intuitions native speakers have about their language, which are very often at odds with the "rules of grammar" learnt at school.

He gives a couple of examples of rules that we hear broken all of the time. For example, native speakers of English almost always say "It's me", not "It's I", though that violates the rules of nominative and accusative case. Are we all wrong? In Sr. Jeanne's 7th-grade English class, perhaps. But English grammar didn't fall from the heavens as incontrovertible rules; it was created by humans as a description of accepted forms of speech.

When a programmer chooses not to use a pattern, other programmers are justified in taking a second look at the program and asking "why?", but they can't really say he's guilty of anything more than doing things differently.

Like grammar rules, some patterns are more "right" than others, in the sense that it's less acceptable to break some than others. I can get away with "It's me", even in more formal settings, but I cannot get away with "Me wrote this blog", even in the most informal settings. An OO programmer might be able get away with not using the Chain of Responsibility pattern in a context where it applies, but not using Strategy or Composite in appropriate contexts just makes him look uninformed, or uneducated.

A few more thoughts:

So, patterns are not like a grammar for programming language, which is prescriptive. To speak Java at all, you have to follow the rules. They are like the grammar of a human language, which model observations about how people speak in the wild.

As a tool for teaching and learning, patterns are so useful precisely because they give us a way to learn accepted usages that go beyond the surface syntactic rules of a language. Even better, the pattern form emphasizes documenting when a construct works and why. Patterns are better than English grammar in this regard, at least better than the way English grammar is typically taught to us as schoolchildren.

There are certainly programmers, software engineers, and programming language theorists who want to tell us how to program, to define prescriptive rules. There can be value in this approach. We can often learn something from a model that has been designed based on theory and experience. But to me prescriptive models for programming are most useful when we don't feel like we have to follow them to the letter! I want to be able to learn something new and then figure out how I can use it to become a better programmer, not a programmer of the model's kind.

But there is also a huge, untapped resource in writing the descriptive grammar of how software is built in practice. It is awfully useful to know what real people do -- smart, creative people; programmers solving real problems under real constraints. We don't understand programming or software development well enough yet not to seek out the lessons learned by folks working in the trenches.

This brings to mind a colorful image, of software linguists venturing into the thick rain forest of a programming ecosystem, uncovering heretofore unexplored grammars and cultures. This may not seem as exotic as studying the Pirahã, but we never know when some remote programming tribe might upend our understanding of programming...

Update: Added a known use contributed by Joe Bergin. -- 05/19/08.

Also Known As    Frequent Releases

From Pattern Language    Graduate Student

You are writing a large document that one or more other people must read and provide feedback on before it is distributed externally.

The archetype is a master's thesis or doctoral dissertation, which must be approved by a faculty committee.

You want to give your reviewers the best document possible. You want them to give you feedback and feel good about approving the document for external distribution.

You want to publish high-quality work. You would also like to have your reviewers see only your best work, for a variety of reasons. First, you respect them and are thankful for their willingness to help you. Second, they often have the ability to help you further in the future, in the form of jobs or recommendations. Good work will impress your reviewers more than weaker work. A more complete and mature document is more likely to resemble the final result than a rougher, earlier version.

In order to produce a document of the highest quality, you need time -- time to research, write, and revise. This delays your opportunity to distribute it to your reviewers. It also delays their opportunity to give you feedback.

Your reviewers are busy. They have their own work to do and are often volunteering their time to serve you.

Big tasks require a lot of time. If a task takes a lot of time to do, some people face a psychological barrier to starting the task.

A big task decomposed into several smaller tasks may take as much time as the big task, or more, but each task takes a smaller time. It is often easier to find time in small chunks, which makes it easier to start on the task in the first place.

The sooner your reviewers are able to read parts of the document, the sooner they will be able to give you feedback. This feedback helps you to improve the document, both in the large (topic, organization) and the small (examples, voice, illustrations, and so on).

Therefore:

Distribute your document to reviewers periodically over a relatively drawn out period. Early versions can be complete parts of the document, or rough versions of the entire document.

In the archetypal thesis, you might give the reviewers one chapter per week, or you might give a whole thesis in which each part is in various stages of completeness.

There is certainly a trade-off between the quality of a document and the timeliness of delivery. Don't worry; this is just a draft. You are always free to improve and extend your work. Keep in mind that there is also a trade-off between the quality of a document and the amount of useful feedback you are able to incorporate.

There is value in distributing even a very rough or incomplete document at regular intervals. If reviewers read the relatively weak version and make suggestions, they will feel valuable. If they don't read it, they won't know or mind that you have made changes in later versions. Furthermore, they may feel wise for not having wasted their time on the earlier draft!

With the widespread availability of networks, we can give our reviewers real-time access to an evolving document in the form of an-line repository. In such a case, Small Doses may take the form of comments recorded as each change is committed to the repository. It is often better for your reviewers if you give them periodic descriptions of changes made to the document, so that they don't have to wade through minutiae and can focus on discrete meaningful jumps in the document.

I have seen Small Doses work effectively in a variety of academic contexts, from graduate students writing theses to instructors writing lecture notes and textbooks for students. I've seen it work for master's students and doctoral students, for text and code and web sites. Joe Bergin says that this pattern is an essential feature of the Doctor of Professional Studies program at Pace University. Joe has documented patterns for succeeding in the DPS program. (If you know of particularly salient examples of Small Doses, I'd love to hear them.)

Often times, the value of Small Doses is demonstrated best in the results of its applying its antipattern, Avalanche. Suppose you are nearing a landmark date, say, the deadline for graduation or the end of spring semester, when faculty will scatter for the summer. You dump a 50-, 70-, or 100+-page document on your thesis committee. In the busy time before the magic date, committee members have a difficulty finding time to read the whole document. Naturally, they put off reading it. Soon they fall way behind and have a hard time meeting the deadline -- and they blame you for the predicament, for unrealistic expectations. Of course, had you given the committee a chapter at a time for 5 or 6 weeks leading up to the magic date, some committee members would still have fallen behind reading it, because they are distracted with their own work. But then you could blame the them for not getting done!

Related Patterns    Extreme Programming and other agile software methodologies encourage Short Iterations and Frequent Releases. Such frequent releases are the Small Doses of the software development world. They enable more continuous feedback and allow the programmers to improve the quality of the code based on what they learn. The Avalanche antipattern is the source of many woes in the world of software, in large part due to the lack of feedback they afford from users and clients.

I learned the Small Doses pattern from Carl Page, my Artificial Intelligence II professor at Michigan State University in 1987. Professor Page was a bright guy and very good computer scientist, with an often dark sense of humor. He mentored his students and advisees with sardonic stories like Small Doses. He was also father to another famous Page, Larry.

Yesterday I was listening to comedian Rodney Laney do a bit called "Old Jobs". He explained that the best kind of job to have is a one-word job that everyone understands. Manager, accountant, lawyer -- that's good. But if you have to explain what you do, then you don't have a good job. "You know the inside of the pin has these springs? I put the springs on the inside of the pins." And then you have to explain why you matter... "Without me, the pins wouldn't go click." Bad job.

Okay, computer scientists and software developers, raise your hands if you've had to explain what you do to a new acquaintance at a party. To a curious relative? I should say "tried to explain", because my own attempts come up short far too often.

I think good Rodney has nailed a major flaw in being a computer scientist.

Sadly, going with the one-word job title of "programmer" doesn't help, and the people who think know what a programmer is often don't really know.

Even still, I like what I do and know why it's a great job.

(Thanks to the wonders of the web, you can watch another version of Laney's routine, Good Jobs, on-line at Comedy Central. I offer no assurance that you'll like it, but I did.)

Owen is reputed to have said something like "Don't give as a programming assignment something the student could just as easily do by hand." (I am still doing penance, even though Lent ended two weeks ago.) This has been dubbed Astrachan's Law, perhaps by Nick Parlante. In the linked paper, Parlante says that showmanship is the key to the Law, that

A trivial bit of code is fine for the introductory in-lecture example, but such simplicity can take the fun out of an assignment. As jaded old programmers, it's too easy to forget the magical quality that software can have, especially when it's churning out an unimaginable result. Astrachan's Law reminds us to do a little showing off with our computation. A program with impressive output is more fun to work on.

I think of this Astrachan's Law in a particular way. First, I think that it reaches beyond showmanship: Not only do students have less fun working on trivial programs, they don't think that trivial programs are worth doing at all -- which means they may not practice enough or at all. Second, I most often think of Astrachan's Law as talking about data. When we ask students to convert Fahrenheit to Celsius, or to sum ten numbers entered at the keyboard, we waste the value of a program on something that can be done faster with a calculator or -- gasp! -- a pencil and paper. Even if students want to know the answer to our trivial assignment, they won't see a need to master Java syntax to find it. You don't have to go all the way to data-intensive computing, but we really should use data sets that matter.

Yesterday, I encountered what might be a variant or extension of Astrachan's Law.

John Zelle of Wartburg College gave a seminar for our department on how to do virtual reality "on a shoestring" -- for $2000 or less. He demonstrated some of his equipment, some of the software he and his students have written, and some of the programs written by students in his classes. His presentation impressed me immensely. The quality of the experience produced by a couple of standard projects, a couple of polarizing filters, and a dollar pair of paper 3D glasses was remarkable. On top of that, John and his students wrote much of the code driving the VR, including the VR-savvy presentation software.

Toward the end of his talk, John was saying something about the quality of the VR and student motivation. He commented that it was hard to motivate many students when it came to 3D animation and filmmaking these days because (I paraphrase) "they grow up accustomed to Pixar, and nothing we do can approach that quality". In response to another question, he said that a particular something they had done in class had been quite successful, at least in part because it was something students could not have done with off-the-shelf software.

These comments made me think about how, in the typical media computation programming course, students spend a lot of time writing code to imitate what programs such as Photoshop and Audacity do. To me, this seems empowering: the idea that a freshman can write code for a common Photoshop filter in a few lines of Java or Python, at passable quality, tells me how powerful being able to write programs makes us.

But maybe to my students, Photoshop filters have been done, so that problem is solved and not worthy of being done again. Like so much of computing, such programs are so much a part of the background noise of their lives that learning how to make them work is as appealing to them as making a ball-point pen is to people of my age. I'd hope that some CS-leaning students do want to learn such trivialities, on the way to learning more and pushing the boundaries, but there may not be enough folks of that bent any more.

On only one day's thought, this is merely a conjecture in search of supporting evidence. I'd love to here what you think, whether pro, con, or other.

I do have some anecdotal experience that is consistent in part with my conjecture, in the world of 2D graphics. When we first started teaching Java in a third-semester object-oriented programming course, some of the faculty were excited by what we could do graphically in that course. It was so much more interesting than some of our other courses! But many students yawned. Even back in 1997 or 1998, college students came to us having experienced graphics much cooler than what they could do in a first Java course. Over time, fewer and fewer students found the examples knock-out compelling; the graphics became just another example.

If this holds, I suppose that we might view it as a new law, but it seems to me a natural extension of Astrachan's Law, a corollary, if you will, that applies the basic idea into the realm of application, rather than data.

My working title for this conjecture is the Pixar Effect, from the Zelle comment that crystallized it in my mind. However, I am open to someone else dubbing it the Wallingford Conjecture or the Wallingford Corollary. My humility is at battle with my ego.

Is anybody home? After a flurry of writing from SIGCSE, I returned home to family time and plenty of work at the office. The result has been one entry in ten days. I look forward to finishing up my SIGCSE reports, but they appear to lie forward a bit, as the next week or so are busy. I have a few new topics in the hopper waiting for a few minutes to write as well.

One bit of good news is that part of my busy-ness this week and next is launching the third iteration of my language topics course. We've done bash and PHP and are now moving on to Ruby, one of my favorite languages. Shell scripting is great, but its tools are too limited to make writing bigger programs fun. PHP was better than I expected, but in the end it is really about building web sites, not writing more general programs. (And after a few weeks of using the language, PHP's warts started to grate on me.)

Ruby is... sublime. It isn't perfect, of course, but even its idiosyncrasies seem to get out of my way when I am deep in code. I looked up the definition of 'sublime', as I sometimes do when I use a word which is outside my daily working vocabulary or is misused enough in conversation that I worry about misusing it myself. The first set of definitions have a subtlety reminiscent of Ruby. To "vaporize and then condense right back again" sounds just like Ruby getting out of my way, only for me to find that I've just written a substantial program in a few lines. (My favorite, though, is "well-meaning ineptitude that rises to empyreal absurdity"!)

This is my first time to teach Ruby formally in a course. I hope to use this new course beginning as a prompt to write a few entries on Ruby and what teaching it is like.

There are many wonderful resources for learning about and programming in Ruby. I've suggested that my students use the pickaxe book as a primary reference, even if they use the first edition, a complete version of which is available on-line. In today's class, though, I used a simple evolutionary example from Brian Marick's book Everyday Scripting with Ruby. I hesitated to use this book as the student's primary source because it was originally written for tester's without any programming background, and my course is for upper-division CS majors with several languages under their belts. But Brian works through several examples in a way that I find compelling, and I think I can base a few solid sessions on one or two of them.

This book makes me wonder how easy it would be to re-target a book from an audience like non-programming testers to an audience of scripting-savvy programmers who want to learn Ruby's particular yumminess. I know that in the course of writing the book Brian generalized his target audience from testers to the union of three different audiences (testers, business analysts, and programmers). Maybe after I've lived with the book and an audience of student programmers I'll have a better sense of how well the re-targeting worked. If it works for my class, then I'll be inclined to adopt it for the next offering of this course.

Anyway, today we evolved a script for diffing to directories of files for a tester. I liked the flow of development and the simple script that resulted. Now we will move on to explore language features and their use in greater depth. One example I hope to work through soon, perhaps in conjunction with Ruby's regular expressions, is "Finding Things", Tim Bray's chapter in Beautiful Code.

Oh, and I must say that this is the first time that one of my courses has a theme song -- and a fine theme song, indeed. Now, if only someone would create a new programming language called "Angie", I would be in heaven.

[A transcript of the SIGCSE 2008 conference: Table of Contents]

The second day of the conference opened with the keynote address by Google's VP of Search Products and User Experience, Marissa Mayer. She was one of the early hires as the company expanded beyond the founders, and from her talk it's clear that she has been involved with a lot of different products in her time there. She is also something the discipline of computer science could use more of, a young woman in a highly-visible technical and leadership and roles. Mayer is a great ambassador for CS as it seeks to expand the number of female high-school and college students.

This talk was called Innovation, Design, and Simplicity at Google and illustrated some of the ways that Google encourages creativity in its employees and gets leverage from their ideas and energy. I'll drop some of her themes into this report, though I imagine that the stories I relate in between may not always sync up. Such is the price of a fast-moving talk and five days of receding memory.

Creativity loves constraint.

I have written on this topic a few times, notably in the context of patterns, and it is a mantra for Google, whose home page remains among the least adorned on the web. Mayer said that she likes to protect its minimalist feel even when others would like to jazz it up. The constraints of a simple page force the company to be more creative in how it presents results. I suspect it also played a role in Google developing its cute practice of customizing the company logo in honor of holidays and other special events. Mayer said that minimalism may be a guide now, but it was not necessarily a reason for simplicity in the beginning. Co-founder Sergey Brin created the first Google home page, and he famously said, "I don't do HTML."

Mayer has a strong background both in CS and in CS education, having worked with the undergrad education folks at Stanford as a TA while an undergrad. (She said that it was Eric Roberts who first recommended Google to her, though at the time he could not remember the company's name!) One of her first acts as an employee was to run a user study on doing search from the Google homepage. She said that when users first sat down and brought up the page, they just sat there. And sat there. They were "waiting for the rest of it"! Already, users of the web were already accustomed to fancy pages and lots of graphics and text. She said Google added its copyright tag line at the bottom of the page to serve as punctutation, to tell the user that that's all there was.

Search is a fixed focus at Google, not a fancy user interface. Having a simple UI helps to harness the company's creativity.

Work on big problems, things users do every day.

Work on things that are easy to explain and understand.

Mayer described in some detail the path that a user's query follows from her browser to Google and back again with search results. Much of that story was as expected, though I was surprised by the fact that there are load balancers to balance the load on the load balancers that hand off queries to processors! Though I might have thought that another level of indirection would slow the process it down, indeed it is necessary in order to ensure that the system doesn't slow down. Even with the web server and the ad server and the mixers, users generally see their results in about 0.2 seconds. How is that for a real-time constraint to encourage technical creativity?

Search is the #2 task performed on the web. E-mail is (still) #1. Though some talking heads have begun to say that search is a mature business in need of consolidation, Google believes that search is just getting started. We know so little about how to do it well, how to meet the user's needs, and how to uncover untapped needs. Mayer mentioned a problem familiar to this old AI guy: determining the meaning of the words used in a query so that they can serve pages that match the user's conceptual intent. She used a nice example that I'll probably borrow the next time I teach AI. When a user asks for an "overhead view", he almost always wants to see a picture!

This points in the direction of another open research area, universal search. The age in which users want to search for text pages, images, videos, etc., as separate entities has likely passed. That partitioning is a technical issue, not a user's issue. The search companies will have to find a way to mix in links to all kinds of media when they serve results. For Google, this also means figuring out how to maintain or boost ad revenue when doing so.

Ideas come from everywhere.

Mayer gave a few examples. One possible source is office hours, which are usually thought of as an academic concept but which she has found useful in the corporate world. She said that the idea for Froogle walked through her office door one day with the scientist who had it.

Another source is experiments. Mayer told a longer story about Gmail. The company was testing it in-house and began to discuss how they could make many from it. She suggested the industry-standard model of giving a small amount of storage for free and then charging for more. This might well have worked, because Google's cost structure would allow it to offer much larger amounts at both pricing levels. But a guy named Paul -- he may be famous, but I don't know his last name -- suggested advertising. Mayer pulled back much as she expected users to do; do people really want Google to read their e-mail and serve ads? Won't that creep them out?

She left the office late that night believing that the discussion was on hold. She came back to work the next morning to find that Paul had implemented an experimental version in a few hours. She was skeptical, but the idea won her over when the system began to serve ads that were perfectly on-spot. Some folks still prefer to read e-mail without ads, but the history of Gmail has shown just how successful the ad model can be.

The insight here goes beyond e-mail. The search ad data base can be used on any paged on the web. This is huge... Search pages account for about 5% of the pages served on the web. Now Google knew that they could reach the other 95%. How's that for a business model?

To me, the intellectual lesson is this:

If you have an idea, try it out.

This is a power that computer programmers have. It is one of the reasons that I want everyone to be able to program, if only a little bit. If you have an idea, you ought to be able to try it out.

Not every idea will lead to a Google, but you never know which ones will.

Google Books started as a simple idea, too. A person, a scanner, and a book. Oh, and a metronome -- Mayer said that when she was scanning pages she would get out of rhythm with the scanner and end up photocopying her thumbs. Adding a metronome to the system smoothed the process out.

... "You're smart. We're hiring." worked remarkably well attracting job candidates. We programmers have big egos! Google is one of the companies that has made it okay again to talk about hiring smart people, not just an army of competent folks armed with a software development process, and giving them the resources they need to do big things.

Innovation, not instant perfection.

Google is also famous for not buying into the hype of agile software development. But that doesn't mean that Google doesn't encourage a lot of agile practices. For example, at the product level, it has long practiced a "start simple, then grow" philosophy.

Mayer contrasted two kinds of programmers, castle builders and nightly builders. Companies are like that, too. Apple -- at least to outside appearances -- is a castle-building company. Every once in a while, Steve Jobs et al. go off for a few years, and then come back with an iPod or an iPhone. This is great if you can do it, but only a few companies can make it work. Google is more of a nightly builder. Mayer offered Google News as a prime example -- it went through 64 iterations before it reached its current state. Building nightly and learning from each iteration is often a safer approach, and even companies that are "only good" can make it work. Sometimes, great companies are the result.

Data is a-political.

Mayer didn't mean Republican versus Democrat here, rather that well-collected data provide a more objective basis for making decisions than the preferences of a manager or the guesses of a designer or programmer. Designing an experiment that will distinguish the characteristics you are in interested, running it, and analyzing the data dispassionately are a reliable way to make good decisions. Especially when a leader's intuition is wrong, such as Mayer's was on Gmail advertising.

She gave a small plug for using Google Trends as a way to observe patterns in search behavior when they might give an idea about a question of interest. Query volume may not not change much, but the content of the queries does.

Users, users, users.

What if some users want more than the minimalist white front page offered by Google? In response to requests from a relatively small minority of users -- and the insistent voices of a few Google designers -- iGoogle is an experiment in offering a more feature-filled portal experience. How well will it play? As is often the case, the data will tell the story.

Give license to dream.

Mayer spent some time talking about the fruits of Google's well-known policy of 20% Time, whereby every employee is expected to spend 1/5 of his or her time working on projects of personal interest. While Google is most famous for this policy these days, like most other ideas it isn't new. At ChiliPLoP this week, Richard Gabriel reported that Eric Schmidt took this idea to Google with him when he left Sun, and Pam Rostal reported that 3M had a similar policy many years ago.

But Google has rightly earned its reputation for the fruits of 20% Time. Google News. Google Scholar. Google Alerts. Orkut. Froogle Wireless. Much of Google Labs. Mayer said that 50% of Google's new products come from these projects, which sounds like a big gain in productivity, not the loss of productivity that skeptics expect.

I have to think that the success Google has had with this policy is tied pretty strongly with the quality of its employees, though. This is not meant to diss the rest of us regular guys, but you have to have good ideas and the talent to carry them out in order for this to work well. That said, these projects all resulted from the passions of individual developers, and we all have passions. We just need the confidence to believe in our passions, and a willingness to do the work necessary to implement them.

Most of the rest of Mayer's talk was a litany of these projects, which one wag in the audience called a long ad for the goodness of Google. I wasn't so cynical, but I did eventually tire of the list. One fact that stuck with me was the description of just how physical the bits of Google Earth are. She described how each image of the Earth's surface needs to be photographed at three or four different elevations, which requires three or four planes passing over every region. Then there are the cars driving around taking surface-level shots, and cameras mounted to take fixed-location shots. A lot of physical equipment is at work -- and a lot of money.

Share the data.

This was the last thematic slogan Mayer showed, though based on the rest of the talk I might rephrase it as the less pithy "Share everything you can, especially the data." Much of Google's success seems based in a pervasive corporate culture of sharing. This extends beyond data to ideas. It also extends beyond Google campus walls to include users.

The data-sharing talk led Mayer to an Easter Egg she could leave us. If you check Google's Language Tools page, you will see Bork, Bork, Bork, a language spoken (only?) by the Swedish chef on the Muppets. Nonetheless, the Bork, Bork, Bork page gets a million hits a year (or was it a day?). Google programmers aren't the only ones having fun, I guess.

Mayer closed with suggestions for computer science educators. How might we prepare students better to work in the current world of computing? Most of her recommendations are things we have heard before: build and use applications, work on large projects and in teams, work with legacy code, understand and test at a large scale, and finally pay more attention to reliability and robustness. Two of her suggestions, though, are ones we don't hear as often and link back to key points in her talk: work with and understand messy data, and understand how to use statistics to analyze the data you collect.

After the talk, folks in the audience asked a few questions. One asked how Google conducts user studies. Mayer described how they can analyze data of live users by modding the key in user cookies to select 1/10 or 1/1000 of the user population, give those users a different experience, and then look at characteristics such as click rate and time spent on page by these users compared to the control group.

The best question was in fact a suggestion for Google. Throughout her talk, Mayer referred repeatedly to "Google engineers", the folks who come up with neat ideas, implement them in code, test them, and then play a big role in selling them to the public. The questioner pointed out that most of those "engineers" are graduates of computer science programs, including herself, Sergey Brin, and Larry Page. He then offered that Google could do a lot to improve the public perception of our discipline if it referred to its employees as computer scientists.

I think this suggestion caught Mayer a little off-guard, which surprised me. But I hope that she and the rest of Google's leadership will take it to heart. In a time when it is true both that we need more computer science students and that public perception of CS as a discipline is down, we should be trumpeting the very cool stuff that computer scientists are doing at places like Google.

All in all, I enjoyed Mayer's talk quite a bit. We should try to create a similarly creativity-friendly environment for our students and ourselves. (And maybe work at a place like Google every so often!)

As I mentioned in my last SIGCSE entry, I have moved from carefree Portland to Carefree, Arizona, for ChiliPLoP 2008. The elementary patterns group spent yesterday, its first, working on the idea of integrating large datasets into the CS curriculum. After a few years of working on specific examples, both stand-alone and running, we started this year thinking about how CS students can work on real problems from many different domains. In the sciences, that often means larger data sets, but more important it means authentic data sets, and data sets that inspire students to go deeper. On the pedagogical side of the ledger, much of the challenge lies in finding and configuring data sets so that they can used reliably and without unnecessary overhead placed on the adopting instructor.

This morning, we volunteered to listen to a presentation by the other hot topic group on its work from yesterday: a "green field" thought experiment designing an undergrad CS program outside of any constraints from the existing university structure. This group consists of Dave West and Pam Rostal, who presented an earlier version of this work at the OOPSLA 2005 Educators' Symposium, and Richard Gabriel, who brings to the discussion not only an academic background in CS and a career in computer science research and industry but also an MFA in poetry. Perhaps the key motivation for their hot topic is that most CS grads go on to be professional software developers or CS researchers, and that our current way of educating them doesn't do an ideal job of preparing grads for either career path.

Their proposal is much bigger than I can report here. They started by describing a three-dimensional characterization of different kinds of CS professionals, including provocative and non-traditional labels as "creative builder", "imaginative researcher", and "ordinologist". The core of the proposal is the sort of competency-based curriculum that West and Rostal talked about at OOPSLA, but I might also describe it as studio-based, apprenticeship-based, and project-based. One of their more novel ideas is that students would learn everything they need for a liberal arts, undergraduate computer science education through their software projects -- including history, English, writing, math, and social science. For example, students might study the mathematics underlying a theorem prover while building a inference engine, study a period of history in order to build a zoomable timeline on the web for an instructional web site, or build a Second Life for a whole world in ancient Rome.

In the course of our discussion, the devil's advocates in the room raised several challenging issues, most of which the presenters had anticipated. For example, how do the instructors (or mentors, as they called them) balance the busy work involved in, say, the students implementing some Second Life chunk with the content the students need to learn? Or how does the instructional environment ensure that students learn the intellectual process of, say, history, and not just impose a computer scientist's worldview on history? Anticipating these concerns does not mean that they have answers, only that they know the issues exist and will have to be addressed at some point. But this isn't the time for self-censorship... When trying to create something unlike anything we see around us, the bigger challenge is trying to let the mind imagine the new thing without prior restraint from the imperfect implementations we already know.

We all thought that this thought experiment was worth carrying forward, which is where the change of direction comes in. While our group will continue to work on the dataset idea from yesterday, we decided in the short term to throw our energies into the wild idea for reinventing CS education. The result will be two proposals to OOPSLA 2008: one an activity at the Educators' Symposium, and the other an Onward! paper. This will be my first time as part of a proposal to the Onward! track, which is both a cool feeling and an intimidating prospect. We'll see what happens.

This set of entries records my experiences at SIGCSE 2008, in Portland, Oregon, March 12-16. I'll update it as I post new pieces about the conference. One of those entries will explain why my posts on SIGCSE may come more slowly than they might.

Primary entries:

• SIGCSE Day 1
  • Randy Pausch and Alice
    ... on Pausch's contributions to CS education
  • The Mystery Problem
    ... on correlations and students' mental models
  • Nifty Assignments
    ... a few cool things to ask students to do
  • Innovating Our Image
    ... how two NSF Distinguished Educational Fellows are trying
  • This and That
    ... miscellaneous observations from the day
• SIGCSE Day 2
  • Plenary Address by Marissa Mayer
    ... on Google's approach to creativity and simplicity
  • Rediscovering the Passion and Beauty
    ... on ways to share our passion and awe with others
  • Recreating the Passion and Beauty
    ... because maybe it's just not that much fun any more
  • This and That
    ... miscellaneous observations from the day
• SIGCSE Day 3
  • Expectation and Outcome
    ... a tale of two sessions
  • CS Past, Present, and Future
    ... on computing's contributions and coming challenges

Ancillary entries:

• Not an Example of Problem-Based Learning
  -- no matter what people say
• On the Roads Back in Portland
  -- running in an old new place

The Prometheus Awards are the Iowa tech industry equivalent of the Academy Awards. At last night's 2008 award ceremony, CS alumni from my school made a good showing.

T8Design won Software Company of the Year in the Small Company Division. Alumnus Wade Arnold is one of T8's co-founders, its former CTO, and its current CEO. Wade accepted the statuette and gave a fine acceptance speech. At least two other alumni were there in the T8 delegation, and it was good to be able to congratulate them all in person.

Alliance Technologies won the award for IT Service Provider of the Year. Alumnus and UNI CS advisory board member Mike Lang, CEO and President of Alliance, represented the company on the dais.

There were two other software connections to our program among the winners. Alumnus Tony Bibbs was part of the team that wrote one of the pieces of software nominated for Best Innovation in Government, which was won by ABC Virtual. And the most distant connection involved Express Auto, which won Top Growth Company of the Year in the Under $25M Division. One of the first programs that got Express Auto off the ground back in the mid-1990s was written by a UNI alumnus and CS advisory board member Fred Zelhart.

It was a good evening for UNI Computer Science alumni, and a good night for this CS prof. With the exception of Mike, who graduated the year before I came here, I had all of these entrepreneurs and developers as students in class, and now I work with Mike, Fred, and Wade on an ongoing basis. I feel a little paternal -- now fraternal -- pride!

This provocative statement appears in the Roly Perera essay that I mentioned recently:

All I know is that if there is a place for post-modernist, lit-crit, social constructivist thinking in the modern world, it's nowhere near the field of computing.

Robert Biddle and James Noble may have a duel on their hands (PDF)! I think that we have the makings of an outstanding OOPSLA 2008 panel.

I'd certainly like to hear more from Perera if he has more ideas of this sort:

... emergent behaviours are in a sense dual to the requirements on a solution. Requirements are known and obligate the system in certain ways, whereas emergent behaviours ("emergents", one could call them) are those which are permitted by the system, but which were not known a priori.

This is an interesting analogy, one I'd like to think more about. For some reason, it reminds of Jon Postel's dictum about protocol design, "Be liberal in what you accept, and conservative in what you send" and Bertrand Meyer's ideas about design by contract.

This week I ran across Jonathan Edwards's review of Gregor Kiczales's OOPSLA 2007 keynote address, "Context, Perspective and Programs" (which is available from the conference podcast page). Having recently commented on Peter Turchi's keynote, I figured this was a good time to go back and listen to all of Gregor's again. I first listened to part of it back when I posted it to the page, but I have to admit that I didn't understand it all that well then. So a second listen was warranted. This time I had access to his slides, which made a huge difference.

In his talk, Kiczales tries to bring together ideas from philosophy, language, and our collective experience writing software to tackle a problem that he has been working around his whole career: programs are abstractions, and any abstraction represents a particular point of view. Over time, the point of view changes, which means that the program is out of sync. Software folks have been thinking about ways to make programs capable of responding naturally as their contexts evolve. Biological systems have offered some inspiration in the last decade or so. Kiczales suggests that computer science's focus on formality gets in the way of us finding a good answer to our problem.

Some folks took this suggestion as meaning that we would surrender all formalism and take up models of social negotiation as our foundation. Roly Perera wrote a detailed and pointed review of this sort. While I think Perera does a nice job of placing Kiczales's issues in their philosophical context, I do not think Kiczales was saying that we should go from being formal to being informal. He was suggesting that we shouldn't have to go from being completely formal to being completely informal; there should be a middle ground.

Our way of thinking about formality is binary -- is that any surprise? -- but perhaps we can define a continuum between the two. If so, we could write our program at an equilibrium point for the particular context it is in and then, as the context shifts, allow the program to move along the continuum in response.

Now that I understand a little better what Kiczales is saying, his message resonates well with me. It sounds a lot like the way a pattern balances the forces that affect a system. As the forces change, a new structure may need to emerge to keep the code in balance. We programmers refactor our code in response to such changes. What would it be like for the system to recognize changes in context and evolve? That's how natural systems work.

As usual, Kiczales is thinking thoughts worth thinking.

While catching up on some work at the office yesterday -- a rare Saturday indeed -- I listened to Peter Turchi's OOPSLA 2007 keynote address, available from the conference podcast page. Turchi is a writer with whom conference chair Richard Gabriel studied while pursuing his MFA at Warren Wilson College. I would not put this talk in the same class as Robert Hass's OOPSLA 2005 keynote, but perhaps that has more to do with my listening to an audio recording of it and not being there in the moment. Still, I found it to be worth listening as Turchi encouraged us to "get lost" when we want to create. We usually think of getting lost as something that happens to us when we are trying to get somewhere else. That makes getting lost something we wish wouldn't happen at all. But when we get lost in a new land inside our minds, we discover something new that we could not have seen before, at least not in the same way.

As I listened, I heard three ideas that captured much of the essence of Turchi's keynote. First was that we should strive to avoid preconception. This can be tough to do, because ultimately it means that we must work without knowing what is good or bad! The notions of good and bad are themselves preconceptions. They are valuable to scientists and engineers as they polish up a solution, but they often are impediments to discovering or creating a solution in the first place.

Second was the warning that a failure to get lost is a failure of imagination. Often, when we work deeply in an area for a while, we sometimes feel as if we can't see anything new and creative because we know and understand the landscape so well. We have become "experts", which isn't always as dandy a status as it may seem. It limits what we see. In such times, we need to step off the easy path and exercise our imaginations in a new way. What must I do in order to see something new?

This leads to the third theme I pulled from Turchi's talk: getting lost takes work and preparation. When we get stuck, we have to work to imagine our way out of the rut. For the creative person, though, it's about more about getting out of a rut. The creative person needs to get lost in a new place all the time, in order to see something new. For many of us, getting lost may seem like as something that just happens, but the person who wants to be lost has to prepare to start.

Turchi mentioned Robert Louis Stevenson as someone with a particular appreciation for "the happy accident that planning can produce". But artists are not the only folks who benefit from these happy accidents or who should work to produce the conditions in which they can occur. Scientific research operates on a similar plane. I am reminded again of Robert Root-Bernstein's ideas for actively engaging the unexpected. Writers can't leave getting lost to chance, and neither can scientists.

Turchi comes from the world of writing, not the world of science. Do his ideas apply to the computer scientist's form of writing, programming? I think so. A couple of years ago, I described a structured form of getting lost called air-drop programming, which adventurous programmers use to learn a legacy code base. One can use the same idea to learn a new framework or API, or even to learn a new programming language. Cut all ties to the familiar, jump right in, and see what you learn!

What about teaching? Yes. A colleague stopped by my office late last week to describe a great day of class in which he had covered almost none of what he had planned. A student had asked a question whose answer led to another, and then another, and pretty soon the class was deep in a discussion that was as valuable, or more, than the planned activities. My colleague couldn't have planned this unexpectedly good discussion, but his and the class's work put them in a position where it could happen. Of course, unexpected exploration takes time... When will they cover all the material of the course? I suspect the students will be just fine as they make adjustments downstream this semester.

What about running? Well, of course. The topic of air-drop programming came up during a conversation about a general tourist pattern for learning a new town. Running in a new town is a great way to learn the lay of the land. Sometimes I have to work not to remember landmarks along the way, so that I can see new things on my way back to the hotel. As I wrote after a glorious morning run at ChiliPLoP three years ago, sometimes you run to get from Point A to Point B; sometimes, you should just run. That applies to your hometown, too. I once read about an elite women's runner who recommended being dropped off far from your usual running routes and working your way back home through unfamiliar streets and terrain. I've done something like this myself, though not often enough, and it is a great way to revitalize my running whenever the trails start look like the same old same old.

It seems that getting lost is a universal pattern, which made it a perfect topic for an OOPSLA keynote talk.

... though I can't in good conscience say, "for I know not what I do."

1. Write a script named simple-interest.php that defines a function to compute the simple interest on an amount, given the amount, annual interest rate, and number of months. Your script should apply the function to its command-line arguments, which are those same values.

The rest of my PHP class's first homework is more reasonable, with a couple of problems repeated from the bash class's first assignment as a way for students to get a sense of trade-offs between shell programming and scripting in a more general-purpose language.

Still, I had to squeeze my eyes shut tight to hit the key that published this assignment. I keep telling myself that this is just an ice-breaking assignment for students who have never written any PHP before, or learned how to access command-line arguments, or convert strings to integers. That such a simple, context-free task is a nice way for them to succeed on their first effort. That our future assignments will be engaging, challenging, worthwhile. But... Ick.

The first time I teach a course, there always seem to be clunkers like this. Starting from scratch, relying on textbooks for inspiration, and working under time pressure all work against my goal of making everything students do in the class worth their time and energy. I suppose that problems such as this one are my opportunities to improve next time out.

My despair notwithstanding, I suspect that many students are happy enough to have at least one problem that is a gift, however uninteresting it may be. Maybe I can find solace in that while I'm working on exercises for my next problem set.

Are all the open jobs in computing that we keep hearing about going unfilled?

Actually -- they're not. Companies do fill those jobs. They fill them with less expensive workers, without computing degrees, and then train them to program.

Mark Guzdial is concerned that some American CEOs and legislators are unconcerned -- "So? Where's the problem?" -- and wonders how we make the case that degrees in CS matter.

I wonder if the US would be better off if we addressed a shortage of medical doctors by starting with less expensive workers, without medical degrees, and then trained them to practice medicine? We currently do face a shortage of medical professionals willing to practice in rural and underprivileged areas.

The analogy is not a perfect one, of course. A fair amount of the software we produce in the world is life-critical, but a lot is not. But I'm not sure whether we want to live in a world where our financial, commercial, communication, educational, and entertainment systems depend on software to run, and that software is written by folks with a shallow understanding of software and computing more generally.

Maybe an analogy to the law or education is more on-point. For example, would the US would be better off if we addressed a shortage of lawyers or teachers by starting with less expensive workers, without degrees in those areas, and then trained them? A shortage of lawyers -- ha! But there is indeed a critical shortage of teachers in many disciplines looming in the near future, especially in math and science. This might lead to an interesting conversation, because many folks advocate loosening the restrictions on professional training for folks who teach in our K-12 classrooms.

I do not mean to say that folks who are trained "on the job" to write software necessarily have a shallow understanding of software or programming. Much valuable learning occurs on the job, and there are many folks who believe strongly in a craftsmanship approach to developing developers. My friend and colleague Ken Auer built his company on the model of software apprenticeship. I think that our university system should adopt more of a project-based and apprenticeship-based approach to educating software developers. But I wonder about the practicality of a system that develops all of its programmers on the job. Maybe my view is colored by self-preservation, but I think there is an important role for university computing education.

Speaking of practicality, perhaps the best way to speak to the CEOs and legislators who doubt the value of academic CS degrees is in their language of supply and productivity. First, decentralized apprenticeship programs are probably how people really became programmers, but they operate on a small scale. A university program is able to operate on a slightly larger scale, producing more folks who are ready for apprenticeship in industry sooner than industry can grow them from scratch. Second, the best-prepared folks coming out of university programs are much more productive than the folks being retrained, at least while the brightest trainees catch. That lack of productivity is at best an opportunity cost, and at worst an invitation for another company to eat your lunch.

Of course, I also think that in the future more and more programmers will be scientists and engineers who have learned how to program. I'm inclined to think that these folks and the software world will be better off being educated by folks with a deep understanding of computing. Artists, too. And not only for immediately obvious economic reasons.

I don't usually play meme games in my web, but as I am winding down for the week I ran across this one on Brian Marick's blog: grab the nearest book, open it to page 123, go to the 5th sentence, and type up the three sentences beginning there.

With my mind worn out from a week in which I caught something worse than a meme, I fell prey and swung my arm around. The nearest book was Beautiful Code. Technically, I suppose that a stack of PHP textbooks is a couple of inches closer to me, but does anyone really want to know what is on Page 123 of any of them?

Here is the output:

The resultant index (which was called iSrc in FilterMethodCS) might be outside the bounds of the array. The following code loads an integer 0 on the stack and branches if iSrc is less than 0, effectively popping both operands from the stack. This is a partial equivalent of the if statement conditional in line 19 of Example 8-2:

Okay, that may not be much more interesting than what a PHP book might have to say, at least out of context. I'm a compiler junkie, though, and I was happy to find a compiler-style chapter in the book. So I turned to the beginning of the chapter, which turns out to be "On-the-Fly Code Generation for Image Processing" by Charles Petzold. I must admit that this sounds pretty interesting to me. The chapter opens with something that may be of interest to others, too:

Among the pearls of wisdom and wackiness chronicled in Steve Levy's classic "Hackers: Heroes of the Computer Revolution" (Doubleday), my favorite is this one by Bill Gosper, who once said, "Data is just a dumb kind of programming."

Petzold then goes on to discuss the interplay between code and data, which is something I've written about as one of the big ideas computer science has taught the world.

What a nice way for me to end the week. Now I have a new something to read over the weekend. Of course, I should probably spend most of my time with those PHP textbooks; that language is iteration 2 in my course this semester. But I've avoided "real work" for a lot less in the past.

That is the title of a blog post that I planned to write five or six weeks ago. Here it is over a month later, and the course just ended. Well, it ended, and it begins again on Tuesday. So now I am thinking agile thoughts as I think back over my course, and still thinking agile thoughts as I prepare for the course. Let me explain.

810:151 is a different sort of course for us. We try to expose students to several different programming languages in the course of their undergraduate study. Even so, it is common for students to graduate thinking, "I wish I'd learned X." Sometimes X is a relatively new language, such as Scala or Groovy. Sometimes it's a language that is now more mainstream but has not yet made it into one of our courses, such as Ruby. Sometimes it is even a language we do emphasize in a course, such as Scheme, but in a course they didn't have an opportunity to take. We always have told students who express this wish that they should be well-equipped to learn a new language on their own, and they are. But...

While taking a full load of courses and working part-time (or taking a part-time load of courses and working full-time), it is often hard for students to set aside time to learn completely optional. People talk about "the real world" as if it is tougher than being in school, but students face a lot of competing demands for their time. Besides, isn't it often more fun to learn something from an expert who can help you learn the tricks of the trade faster and miss a few of the potholes that lie along the way?

I sometimes think of this course, which we call "Topics in Programming Languages", as a "make time" course for students who want to learn a new language, or perhaps a broader topic related to language, but who want or need the incentive that a regular course, assigned readings, and graded work provides. The support provided by the prof's guidance also is a good safety net for the less seasoned and less confident. For these folks, one of the desired outcomes is for them to realize, hey, I really can learn a language on my own.

We usually offer each section of 810:151 as a 1-credit course. The content reason is that the course has the relatively straightforward purpose of teaching a single language, without a lot of fluff. The practical purpose is that we can offer three 1-credit courses in place of a single 3-credit course. Rather than meet one hour per week for the entire semester, the course can meet 3 hours per week for 5 weeks. This works nicely for students who want to take all three, as they look and feel like a regular course. It also works nicely for students who choose to take only one or two of the courses, as they need not commit an entire semester's worth of attention to them.

This is my first semester assigned (by me) to teach this odd three-headed course. The topics this semester are Unix shell programming in bash, PHP, and Ruby.

I've been thinking of the three courses as three 5-week iterations. Though the topics of the three courses are different, they share a lot in terms of being focused on learning a language in five weeks. How much material can I cover in a course? How can students best use their time? How can I best evaluate their work and provide feedback? Teaching three iterations of a similar course in one semester is so much better for me when it comes to taking what I learn and trying to improve the next offering. With an ordinary course taught every semester, I would have to wait until next fall to begin implementing improvements; with an ordinary course three-course rotation, I would have to wait until Fall 2009!

I opted to dispense with all examinations and evaluate students solely in terms of the bash scripts they wrote. The goal of the course is for students to learn how to program in bash, so that is where I wanted the students' attention to be. One side effect of this decision is that the course is not really over yet; students will work on their final problem set in the coming week, and I'll have to grade it next Friday. The problem sets have consisted mostly in small-ish scripts that exercise the features of bash as we encounter them. We did have one larger task that students solved in three parts over the course of the semester, a processor for a Markdown-like mark-up language that produces HTML. This project scratched one of my own itches, as I like to use simple text-based, e-mail-friendly mark-up, and now I have a simple bash script that does the job!

One thing I did not do this semester that I thought I might, and which perhaps I should, is to work with them through a non-trivial shell script or two. I had thought that the fifth week would be devoted to examining and extending larger scripts, but I kept uncovering more techniques and ideas that I wanted them to see. Perhaps I could use a real script as a primary source for learning the many features of bash, instead of building their skills from the bottom up. That is how many of them have to come to know what little they know about shell scripting, by confronting a non-trivial script for building or configuring an open-source application that interests them. To be honest, though, I think that the bottom-up style that we used this semester may prepare them better for digging into a more complex script than starting with a large program first. This is one of the issues I hope to gain some insight into from student feedback on the course.

Making this "short iterations" more interesting is the fact that some students will be in all three of the iterations, but there will be a significant turnover in the class rosters. The client base evolves, but there should be enough overlap that I can get some comparative feedback as I try to implement improvements.

I tried to follow a few other agile principles as I started teaching this new prep. I tend to start each new course with a template from my past courses, from the way I organize sessions and lecture notes to the look-and-feel of the web site. This semester, I tried to maintain a YAGNI mindset: start as simple as I can, and add new elements only as I use them -- not when I think I need them tomorrow. By and large I have succeeded in this regard. My web site is bare-bones in comparison to my past sites, and lecture notes are plain text records of in-class activities and short messages to remind me and the students of what we discussed. I saved a lot of time not trying to produce attractive and complete lecture notes in HTML. Maybe some day, but this time around I just didn't need them.

One agile practice that I didn't think to encourage soon enough was unit testing. Shame on me. Some students suffered far more than I from this oversight. Many did a substandard job of testing their scripts, in part I think because they were biting off too much of a task to start. Unix pipelines are almost perfectly suited to unit testing, as one can test the heck out of each stage in isolation, growing the pipeline one stage at a time until the task is solved. The fact that each component is reading from stdin and writing to stdout means that later stages can be tested independent of the stages that precede it before we add it to the end.

For whatever reason, it didn't occur to me that there would exist an shUnit. It's too late for me to use it this semester, but I'll be sure to put phpUnit to good use in the next five weeks. And I've always known that I would use a unit testing framework such as this one for Ruby. Heck, we may even roll our own as we learn the language!

I've really enjoyed teaching a course with the Unix philosophy at the forefront of our minds. Simple components, the universal interface of plain text streams, and a mandate to make tools to work together -- the result is an amazingly "agile" programming environment. The best way to help students see the value of agile practices is to let them live in an environment where that is natural, and let them feel the difference from the programming environments in which they other times find themselves. I just hope that my course did the mindset justice.

The tool-builder philosophy that pervaded this course reminded me of this passage from Jason Marshall's Something to Say:

There's an old saying, "A good craftsman never blames his tools." Many people take this to mean "Don't make excuses," or even, "Real men don't whine when their tools break." But I take it to mean, "A good craftsperson does not abide inferior tools."

A good craftsman never blames his tools, because if his tools are blameworthy, he finds better tools. I associate this idea more directly with the pragmatic programmers than with the agile community, but it seems woven into the fabric of the agile approaches. The Agile Manifesto declares that we value "individuals and interactions over processes and tools", but I do not take this to mean that tools don't matter. I think it means that we should use tools (and processes) that empower us to focus our energy on people and interactions. We should let programs do what they do best so that we programmers can do what we do best. That's why we have unit testing frameworks, refactoring tools, automatic build tools, and the like. It's also why Unix is far more human-affirming than it is sometimes given credit for.

As I told students to close the lecture notes for this course: Don't settle for inferior tools. You are a computer programmer. Make the tools that make you better.

I recently came across a SIGCSE paper from 1991 called Integrating Writing into Computer Science Courses, by Linda Hutz Pesante, who at the time was affiliated with the Software Engineering Institute at Carnegie Mellon. This paper describes both content and technique for teaching writing within a CS program, a topic that cycles back into the CS education community's radar every few years. (CS academics know that it is important even during trough periods, but their attention is on some other, also often cyclic, attention-getter.)

What caught my attention about Pesante's paper is that she tries help software engineers to use their engineering expertise to the task of writing technical prose. One of her other publications, a video, even has the enticing title, Applying Software Engineering Skills to Writing. I so often think about applying ideas from other disciplines to programming, the thought of applying ideas from software development to another discipline sounded like a new twist.

Pesante's advice on how to teach writing reflects common practice in teaching software development:

• Motivate students so that they know what to expect.
• Attend to the writing process, as well as the final product.
• Use analogies to programming, such as debugging and code reviews.
• Have students practice, and give them feedback.

Given Pesante's affiliation with the SEI, her suggestions for what to teach about writing made a bigger impression on me. The software engineering community certainly embraces a broad range of development "methodologies" and styles but, underlying its acceptance even of iterative methods, there seems to be a strong emphasis on planning and "getting things right" the first time.

Her content advice relies on the notion that "writing and software development have something in common", from user analysis through the writing itself to evaluation. As such, a novice writer can probably learn a lot from how programmers write code. Programmers like to be creative and explore when they write, too, but they also know that thinking about the development process can add structure to a potentially unbounded activity. They use tools to help them manage their growing base of documents and to track revisions over time. That part of the engineer's mindset can come in handy for writers. For the programmer who already has that mindset, applying it to the perhaps scary of writing prose can put the inexperienced writer at ease.

Pesante enumerates a few other key content points:

• Writing takes place in a context.
• The writing process is neither linear nor algorithmic.
• The writing process is iterative.
• Correct is not the same as effective.

The middle two of these especially feel more agile than the typical software engineering discussion. I think that the agile software community's emphasis on short iterations with frequent releases of working software to the client also matches quite nicely the last of the bullets. It's all too easy to do a good job of analysis and planning, produce a piece of software that is correct by the standards of the analysis and plan, and find that it does not meet the user's needs effectively. With user feedback every few weeks, the development team has many more opportunities to ensure that software stays on a trajectory toward effectiveness.

Most people readily accept the idea that creative writing is iterative, non-linear, and exploratory. But I have heard many technical writers and other writers of non-creative prose say that their writing also has these features -- that they often do not know what they had to say, or what their ultimate product would be, until they wrote it. My experience as a writer, however limited, supports the notion that almost all writing is exploratory, even when writing something so pedestrian as a help sheet for students using an IDE or other piece of software.

As a result, I am quite open to the notion that programming -- what many view as the creating of an "engineered" artifact -- also iterative, non-linear, and exploratory. This is true, I think, not only for what we actually call "exploratory programming" but also for more ordinary programming tasks, where both the client and I think we have a pretty good handle on what the resulting programming should do and look like. Often, over the course of writing an ever-larger program, our individual and shared understanding of the program evolves. Certainly my idea of the internal shape of the program -- the structure, the components, the code itself -- changes as the program grows. So I think that there is a lot to be learned going both directions in the writing:programming metaphor.

... about teaching, and about software development.

The February 2008 issue of Smithsonian Magazine contains an article called Being Funny, by comedian, writer, and actor Steve Martin, that has received a fair amount of discussion on-line already. When I read it this weekend, I was surprised by how similar some of the lessons Martin learned as he grew into a professional comedian are to lessons that software developers and teachers learn. I enjoyed being reminded of them.

I gave myself a rule [for dealing with the audience]. Never let them know I was bombing: this is funny, you just haven't gotten it yet.

This is about not showing doubt. Now, I think it's essential for an instructor to be honest -- something I wrote about a while back, in the context of comedy as well. So I don't mean that I as teacher should try to bluff my way through something I don't know or know but have botched. Martin is talking about the audience getting it, and the doubt that enters my mind when a classroom of students seem not to. I experience this occasionally when I teach a course like Programming Languages to a new group of students. Some classes don't warm to me or the material in quite the same way as others, but I know that the material I'm teaching and the basic approach I am using are sound. When this semester's crowd takes a while to catch on -- or if they are openly skeptical of the material or approach -- it's essential that I remain confident. Sure, I'll make adjustments to the presentation to account for my current students' needs, but I should remain steadfast: This is good stuff; they'll get it soon.

I'm not sure this applies as well for software developers. Often, when my users don't get it yet and I feel compelled to bull on through, I have gone beyond the requirements, at least as my users understand them. In those cases, it's usually better to say to myself "You aren't gonna need it" and simplify.

Everything was learned in practice, and the lonely road, with no critical eyes watching, was the place to dig up my boldest, or dumbest, ideas and put them onstage.

This is hard to do as a teacher. I don't get to go on the road to a hundred musty nightclubs and try out my new lecture on continuation-passing style, as Martin did with his bits. He was practicing on small stages in order to prepare for the big ones, such as The Tonight Show. It's next to impossible for me to try a lecture out on a representative audience that isn't my regular audience: my students. I can practice my research talks before a small local audience before taking them to a conference, but the number of repetitions available to me is rather small even in that scenario. For class sessions, I pretty much have to try them out live, see what happens, and feed the results back into my next performance.

Fortunately, I'm not often trying radically new ideas out in a lecture, so fewer and live repetitions may suffice. I have tried teaching methods that quite different than the norm for me and for my students, such as a Software Systems course or gen-ed capstone course with no lecture and all in-class exercises. In those scenarios, I had to follow the advice discussed above: This is going to work; they just haven't gotten it yet...

This piece of advice applies perfectly to programming. The lonely road is my office or my room at home, where I can try out every new idea that occurs to me by writing a program (or ten) and seeing how it works. No critical eyes but my eye, which I turn off. Bold ideas, dumb ideas -- I find out which are which through practice.

The consistent work enhanced my act. I learned a lesson: it was easy to be great. Every entertainer has a night when everything is clicking. These nights are accidental and statistical: like lucky cards in poker, you can count on them occurring over time. What was hard was to be good, consistently good, night after night, no matter what the circumstances.

This is so true of teaching that I have nothing more to say. Read Martin's line again.

I think this is also true of programming, and writing more generally. Every so often, a great idea comes to mind; a great turn of phrase; a cool little hack that solves the problem at hand. To be a good programmer, you need to practice, so that each time you sit down to code you can produce something of value. That kind of skill is earned by practice, and, I think, attainable by everyone.

On one of my appearances [on The Tonight Show, after [Johnny Carson] had done a solid impression of Goofy the cartoon dog, he leaned over to me during a commercial and whispered prophetically, "You'll use everything you ever knew."

When working with students, I find myself borrowing from every good teacher I've ever had, and drawing on any experience I can recall. I've borrowed standards and attitudes from one of my favorite undergrad profs, who taught me the value of meeting deadlines. I've used phrases and jokes spoken by my high school chemistry teacher, who showed us that studying a difficult subject could be fun, with the right mindset and a supportive group of classmates. Whatever works, works. Use it. Adapt it.

Likewise, this advice is true when programming. In the last few years, the notion of scrapheap programming has become quite popular. In this style, a programmer looks for old pieces of code that do part of the job at hand, adapts them, and glues them together to get the job done. But this is how all writers and programmers work, drawing on all of the old bits of code rolling around their heads. In addition to practice, you can improve as a programmer by exposing yourself to as many different programs as possible. That way, you will see the data structures, the idioms, the design patterns, and, yes, the snippets of code that you will use twenty years from now when the circumstance is right. That may seem goofy, but it's not.

Finally:

I believed it was important to be funny now, while the audience was watching, but it was also important to be funny later, when the audience was home and thinking about it.

As a teacher, I would like for what my students see, hear, and do in class today to make an impression today. That is what makes the material memorable and, besides, it's a lot more fun for both of us that way than the alternative. But perhaps more important, I would like for the ideas to make a lasting impression. When the student is home thinking about the course, or an assignment, or computer science in general, I want them to realize how much depth the idea or technique has, or breadth. Today's memorability can be ephemeral. When the idea sticks later, it can affect how the student programs forever.

The hard part is trusting when I don't see students getting it in real-time. Martin says that he learned not to worry if a gag got no response from his audience, "as long as I believed it had enough strangeness to linger". Strangeness may not be what I hope for in my class sessions, but I know what he means.

As a programmer, I think this advice applies, too. When I was an undergrad, I was once on a team building a large system as part of our senior project course. One of our teammates loved to write code that was clever, that would catch our eye on first read and recognize his creativity and skill. But we soon learned that much of this code was inscrutable in a day or two, which made modifying and maintaining his modules impossible. Design and code that makes a module easy to read and understand in a few weeks are what the team needed. Sometimes, the cleverness of a solution shone through then, too, but it impressed us all the more when it had "staying power".

Steve Martin is wacky comedian, not a programmer or teacher per se. Does his advice apply only in that context? I don't think so. Comedians are writers and creators, and many of the traits that make them successful apply to other tasks that require creating and communicating.

Readers of this blog know that programming is one of the topics I most like to write about. In recent months I've had something of a "programming for everyone" theme, with programming as a medium of expression, as a way to create new forms, ideas, and solutions. But programming is also for computer scientists, being the primary mode for communicating their ideas.

To the non-CS folks reading this, that may seem odd. Isn't CS about programming? Most non-CS folks seem to take as a given that computer science is, but these days it is de rigeur for us in the discipline to talk about "computing" and how much bigger it is than "just programming".