Knowing and Doing

In March I talked about a couple of OOPSLA submissions written by our merged ChiliPLoP groups. In May I wrote about the verdict on one but forgot to mention the other. Maybe because the rejection was so much more interesting!

Anyone, our second submission was accepted into the Educators' Symposium. It is not a paper, really, but an extended abstract for an activity we will run: a code review recast as it might happen in a software studio. We hope to give participants a snapshot of what a studio-based course looks, feels, and works like. This is something instructors can try on a small scale in class and, if it works for them, expand throughout their course. Even if code reviews is all the farther they go, we co-authors think that this will be a useful step for many instructors. It draws on experiences in the writers' workshops of PLoP helps students to think about the many design choices they make when writing software, and to make them reflectively rather than subconsciously.

The real trick to this activity will be the homework we give before the symposium:

Before coming to OOPSLA, Educators' Symposium participants will be asked to submit a program, in a language of their choice (though using only standard libraries), which implements the core of a program to generate Tag Clouds from a data set. ...

My experience with many workshops in the past and especially with the Educators' Symposium is that participants never do this kind of homework. Some are well-intentioned but never make time for it, while others figure they can skate by without having written the code. (Sounds as if professors are a lot like their students, huh?) Without code to review, a code review doesn't get very far. We hope that we can find a way to encourage symposium attendees to overcome history and come with some code to workshop.

I've long been a fan of William James, and once wrote briefly about the connection between James's pragmatism and my doctoral work on knowledge-based systems. I was delighted yesterday to run across this quote from James's The Principles of Psychology, courtesy of 43 Folders and Linda Stone:

[Attention] is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. ... It implies withdrawal from some things in order to deal effectively with others....

Prone as I am to agile moments, this message from James struck me in an interesting way. First of all, I occasionally battle the issue that Stone writes about, the to-do list that grows no matter productive I seem to be on a given day. (And on lazy summer June days, well, all bets are off.) James tells me that part of my problem isn't a shortage of time, but a lack of will to focus. I need to make better, more conscious choices about what tasks to add to the list. Kent Beck is fond of saying something to the effect that you may have too many things to do and too little time, but you ultimately have control over only one side of the equation. James would tell us the same thing.

My mind also made a connection from this quote to the agile software and test-driven development practice of working on small stories, on taking small steps. If I pick up a card with a single, atomic, well-defined feature to be added to my program, I am able to focus. What is the shortest step I can take and make this feature part of my code? No distractions, no Zerstreutheit. Though I have an idea in mind toward where my program is evolving, for this moment I attend to one small feature and make it work. Focus. James would be proud.

I think it's ironic in a way that one of the more effective ways to reach the state of flow is to decompose a task into the smallest of tasks and focus on them one at a time. The mind gets into a rhythm of red bar-green bar: select task, write code, refactor, and soon it is deep in its own world. I would like to be more effective at doing this in my non-programming duties. Perhaps if I keep James and his quote in mind, I can be.

This idea applies for me in other areas, in particular in running and training for particular events. Focusing each day on a particular goal -- intervals, Long Slow Distance, hill strength, and so on -- helps the mind to push aside its concerns with other parts of the game and attend to a particular kind of improvement. There is a great sense of relaxation in running super-hard repeats when the problem I've been having is, say, picking up pace late in a run. (I'd love to run super-hard repeats again some day soon, but I'm not there yet.)

I ran across a pattern today that reminded of another I encountered while at ChiliPLoP. Both help developers deal with side effects, changes to a program's state. Let me share these patterns with you all, with a common explanation of their value.

Most programs written these days are in a style where setting and changing the values of one or more variables, via a sequence of imperative statements. Side effects are a common source of programmers' misunderstanding when reading code. A program can change a variable's state almost anywhere, and that makes reading any bit of code uneasy: "Do I really know what this variable means right now?" Using only local variables only in the context of a small procedure is one way to localize effects. Even still, problems can arise.

I won't try write full patterns for these. I'll write short patlets and give you links to the articles I read, which do a good job talking about the ideas.

Mutual Recursion with Side Effects

In Solving Every Sudoku Puzzle, Peter Norvig builds a Python program for the title task. About 40% of the way in, he says:

If you have two mutually-recursive functions that both alter the state of an object, try to move almost all the functionality into just one of the functions. Otherwise you will probably end up duplicating code.

Duplicate code is usually a bad thing, because it creates a problem for programmers keeping the copies in sync. Duplicate code that modifies state is especially bad, because it brings in the extra complication of making another procedure harder to understand.

Side Effects and Python's Default Parameters

Coincidentally, the second pattern involves Python, too. This time the pattern is Python-specific, addressing an issue with a language feature.

Context: You are writing a procedure with a default parameter.

Problem: The procedure needs to modify the default parameter. Python evaluates a default parameter's initial value only on the first call to the procedure.

Solution: Whenever the parameter is missing, initialize it within the method.

Tim Ottinger gives his implementation technique in Python's Mutable Default Problem:

def function( item, stuff=None ):
   stuff = stuff or []
   # ... modify stuff, etc.

This pattern allows the user to take advantage of the default parameter when he has no collection to send. It does so by following a principle laid out in the article Ottinger references: In Python, "don't use mutable objects as function defaults."

This pattern may not be Python-specific, because there may be other languages with this initialize-once behavior. It seems like a bug to me, not a feature, but I'm not a Python programmer and so can't speak authoritatively. But I am glad that Ruby doesn't do this.

~~~~~

Postscript: While preparing this article, I learned something new about Ruby, unrelated to this issue. It is possible to extract from a class a new class that has a subset of the original class's protocol. Very nice indeed!)

The verdict is in on the paper we wrote at ChiliPLoP and submitted to Onward!: rejected. (We are still waiting to hear back on our Educators' Symposium submission.) The reviews of our Onward! paper were mostly on mark, both on surface features (e.g., our list of references was weak) and on the deeper ideas we offer (e.g., questions about the history of studio approaches, and questions about how the costs will scale). We knew that this submission was risky; our time was simply too short to afford enough iterations and legwork to produce a good enough paper for Onward!.

I found it interesting that the most negative reviewer recommended the paper for acceptance. This reviewer was clearly engaged by the idea of our paper and ended up writing the most thorough, thoughtful review, challenging many of our assumptions along the way. I'd love to have the chance to engage this person in conversation at the conference. For now, I'll have to settle for pointing out some of the more colorful and interesting bits of the review.

In at least one regard, this reviewer holds the traditional view about university education. When it comes to the "significant body of knowledge that is more or less standard and that everyone in the field should acquire at some point in time", "the current lecture plus problem sets approach is a substantially more efficient and thorough way to do this."

Agreed. But isn't it more efficient to give the students a book to read? A full prof or even a TA standing in a big room is an expensive way to demonstrate standard bodies of knowledge. Lecture made more sense when books and other written material were scarce and expensive. Most evidence on learning is that lecture is actually much less effective than we professors (and the students who do well in lecture courses) tend to think.

The reviewer does offer one alternative to lecture: "setting up a competition based on mastery of these skills". Actually, this approach is consistent with the spirit of our paper's studio-based, apprenticeship-based, and project-based. Small teams working to improve their skills in order to win a competition could well inhabit the studio. Our paper tended to overemphasize the softer collaboration of an idyllic large-scale team.

This comment fascinated me:

Another issue is that this approach, in comparison with standard approaches, emphasizes work over thinking. In comparison with doing, for example, graph theory or computational complexity proofs, software development has a much lower ratio of thought to work. An undergraduate education should maximize this ratio.

Because I write a blog called Knowing and Doing, you might imagine that I think highly of the interplay between working and thinking. The reviewer has a point: an education centered on projects in a studio must be certain to engage students with the deep theoretical material of the discipline, because it is that material which provides the foundation for everything we do and which enables us to do and create new things. I am skeptical of the notion that an undergrad education should maximize the ratio of thinking to doing, because thinking unfettered by doing tends to drift off into an ether of unreality. However, I do agree that we must try to achieve an appropriate balance between thinking and doing, and that a project-based approach will tend to list toward doing.

One comment by the reviewer reveals that he or she is a researcher, not a practitioner:

In my undergraduate education I tried to avoid any course that involved significant software development (once I had obtained a basic mastery of programming). I believe this is generally appropriate for undergraduates.

Imagine the product of an English department saying, "In my undergraduate education I tried to avoid any course that involved significant composition (once I had obtained a basic mastery of grammar and syntax). I believe this is generally appropriate for undergraduates." I doubt this person would make much of a writer. He or she might be well prepared, though, to teach lit-crit theory at a university.

Most of my students go into industry, and I encourage them to take as many courses as they can in which they will build serious pieces of software with intellectual content. The mixture of thinking and doing stretches them and keeps them honest.

An education system that produces both practitioners and theoreticians must walk a strange line. One of the goals of our paper was to argue that a studio approach could do a better job of producing both researchers and practitioners than our current system, which often seems to do only a middling job by trying to cater to both audiences.

I agree wholeheartedly, though, with this observation:

A great strength of the American system is that it keeps people's options open until very late, maximizing the ability of society to recognize and obtain the benefits of placing able people in positions where they can be maximally productive. In my view this is worth the lack of focus.

My colleagues and I need to sharpen our focus so that we can communicate more effectively the notion that a system based on apprenticeship and projects in a studio can, in fact, help learners develop as researchers and as practitioners better than a traditional classroom approach.

The reviewer's closing comment expresses rather starkly the challenge we face in advocating a new approach to undergraduate education:

In summary, the paper advocates a return to an archaic system that was abandoned in the sciences for good reason, namely the inefficiency and ineffectiveness of the advocated system in transmitting the required basic foundational information to people entering the field. The write-up itself reflects naive assumptions about the group and individual dynamics that are required to make the approach succeed. I would support some of the proposed activities as part of an undergraduate education, but not as the primary approach.

The fact that so many university educators and graduates believe our current system exists in its current form because it is more efficient and effective than the alternatives -- and that it was designed intentionally for these reasons -- is a substantial cultural obstacle to any reform. Such is the challenge. We owe this reviewer our gratitude for laying out the issues so well.

In closing, I can't resist quoting one last passage from this review, for my friends in the other sciences:

The problem with putting students with no mastery of the basics into an apprenticeship position is that, at least in computer science, they are largely useless. (This is less true in sciences such as biology and chemistry, which involve shallower ideas and more menial activities. But even in these sciences, it is more efficient to teach students the basics outside of an apprenticeship situation.)

The serious truth behind this comment is the one that explains why building an effective computer science research program around undergraduates can be so difficult. The jocular truth behind it is that, well, CS is just plain deeper and harder! (I'll duck now.)

I've tried to explain the idea of software patterns in a lot of different ways, to a lot of different kinds of people. Reading James Tauber's Grammar Rules reminds me of one of my favorites: a pattern language is a descriptive grammar. Patterns describe how (good) programmers "really speak" when they are working in the trenches.

Talking about patterns as grammar creates the potential for the sort of misunderstanding that Tauber discusses in his entry. Many people, including many linguists, think of grammar rules as, well, rules. I was taught to "follow the rules" in school and came to think of the rules as beyond human control. Linguists know that the rules of grammar are man-made, yet some still seem to view them as prescriptive:

It is as if these people are viewing rules of grammar like they would road rules--human inventions that one may disagree with, but which are still, in some sense, what is "correct"...

Software patterns are rarely prescriptive in this sense. They describe a construct that programmers use in a particular context to balance the forces at play in the problem. Over time, they have been found useful and so recur in similar contexts. But if a programmer decides not to use a pattern in a situation where it seems to apply, the programmer isn't "wrong" in any absolute sense. But he'll have to resolve the competing forces in some other way.

While the programmer isn't wrong, other programmers might look at him (or, more accurately, his program) funny. They will probably ask "why did you do it that way?", hoping to learn something knew, or at least confirm that the programmer has done something oddly.

This is similar to how human grammar works. If I say, "Me wrote this blog", you would be justified in looking at me funny. You'd probably think that what I speaking incorrectly.

Tauber points out that, while I might be violating the accepted rules of grammar, I'm not wrong in any absolute sense:

... most linguists focus on modeling the tacit intuitions native speakers have about their language, which are very often at odds with the "rules of grammar" learnt at school.

He gives a couple of examples of rules that we hear broken all of the time. For example, native speakers of English almost always say "It's me", not "It's I", though that violates the rules of nominative and accusative case. Are we all wrong? In Sr. Jeanne's 7th-grade English class, perhaps. But English grammar didn't fall from the heavens as incontrovertible rules; it was created by humans as a description of accepted forms of speech.

When a programmer chooses not to use a pattern, other programmers are justified in taking a second look at the program and asking "why?", but they can't really say he's guilty of anything more than doing things differently.

Like grammar rules, some patterns are more "right" than others, in the sense that it's less acceptable to break some than others. I can get away with "It's me", even in more formal settings, but I cannot get away with "Me wrote this blog", even in the most informal settings. An OO programmer might be able get away with not using the Chain of Responsibility pattern in a context where it applies, but not using Strategy or Composite in appropriate contexts just makes him look uninformed, or uneducated.

A few more thoughts:

So, patterns are not like a grammar for programming language, which is prescriptive. To speak Java at all, you have to follow the rules. They are like the grammar of a human language, which model observations about how people speak in the wild.

As a tool for teaching and learning, patterns are so useful precisely because they give us a way to learn accepted usages that go beyond the surface syntactic rules of a language. Even better, the pattern form emphasizes documenting when a construct works and why. Patterns are better than English grammar in this regard, at least better than the way English grammar is typically taught to us as schoolchildren.

There are certainly programmers, software engineers, and programming language theorists who want to tell us how to program, to define prescriptive rules. There can be value in this approach. We can often learn something from a model that has been designed based on theory and experience. But to me prescriptive models for programming are most useful when we don't feel like we have to follow them to the letter! I want to be able to learn something new and then figure out how I can use it to become a better programmer, not a programmer of the model's kind.

But there is also a huge, untapped resource in writing the descriptive grammar of how software is built in practice. It is awfully useful to know what real people do -- smart, creative people; programmers solving real problems under real constraints. We don't understand programming or software development well enough yet not to seek out the lessons learned by folks working in the trenches.

This brings to mind a colorful image, of software linguists venturing into the thick rain forest of a programming ecosystem, uncovering heretofore unexplored grammars and cultures. This may not seem as exotic as studying the Pirahã, but we never know when some remote programming tribe might upend our understanding of programming...

Update: Added a known use contributed by Joe Bergin. -- 05/19/08.

Also Known As    Frequent Releases

From Pattern Language    Graduate Student

You are writing a large document that one or more other people must read and provide feedback on before it is distributed externally.

The archetype is a master's thesis or doctoral dissertation, which must be approved by a faculty committee.

You want to give your reviewers the best document possible. You want them to give you feedback and feel good about approving the document for external distribution.

You want to publish high-quality work. You would also like to have your reviewers see only your best work, for a variety of reasons. First, you respect them and are thankful for their willingness to help you. Second, they often have the ability to help you further in the future, in the form of jobs or recommendations. Good work will impress your reviewers more than weaker work. A more complete and mature document is more likely to resemble the final result than a rougher, earlier version.

In order to produce a document of the highest quality, you need time -- time to research, write, and revise. This delays your opportunity to distribute it to your reviewers. It also delays their opportunity to give you feedback.

Your reviewers are busy. They have their own work to do and are often volunteering their time to serve you.

Big tasks require a lot of time. If a task takes a lot of time to do, some people face a psychological barrier to starting the task.

A big task decomposed into several smaller tasks may take as much time as the big task, or more, but each task takes a smaller time. It is often easier to find time in small chunks, which makes it easier to start on the task in the first place.

The sooner your reviewers are able to read parts of the document, the sooner they will be able to give you feedback. This feedback helps you to improve the document, both in the large (topic, organization) and the small (examples, voice, illustrations, and so on).

Therefore:

Distribute your document to reviewers periodically over a relatively drawn out period. Early versions can be complete parts of the document, or rough versions of the entire document.

In the archetypal thesis, you might give the reviewers one chapter per week, or you might give a whole thesis in which each part is in various stages of completeness.

There is certainly a trade-off between the quality of a document and the timeliness of delivery. Don't worry; this is just a draft. You are always free to improve and extend your work. Keep in mind that there is also a trade-off between the quality of a document and the amount of useful feedback you are able to incorporate.

There is value in distributing even a very rough or incomplete document at regular intervals. If reviewers read the relatively weak version and make suggestions, they will feel valuable. If they don't read it, they won't know or mind that you have made changes in later versions. Furthermore, they may feel wise for not having wasted their time on the earlier draft!

With the widespread availability of networks, we can give our reviewers real-time access to an evolving document in the form of an-line repository. In such a case, Small Doses may take the form of comments recorded as each change is committed to the repository. It is often better for your reviewers if you give them periodic descriptions of changes made to the document, so that they don't have to wade through minutiae and can focus on discrete meaningful jumps in the document.

I have seen Small Doses work effectively in a variety of academic contexts, from graduate students writing theses to instructors writing lecture notes and textbooks for students. I've seen it work for master's students and doctoral students, for text and code and web sites. Joe Bergin says that this pattern is an essential feature of the Doctor of Professional Studies program at Pace University. Joe has documented patterns for succeeding in the DPS program. (If you know of particularly salient examples of Small Doses, I'd love to hear them.)

Often times, the value of Small Doses is demonstrated best in the results of its applying its antipattern, Avalanche. Suppose you are nearing a landmark date, say, the deadline for graduation or the end of spring semester, when faculty will scatter for the summer. You dump a 50-, 70-, or 100+-page document on your thesis committee. In the busy time before the magic date, committee members have a difficulty finding time to read the whole document. Naturally, they put off reading it. Soon they fall way behind and have a hard time meeting the deadline -- and they blame you for the predicament, for unrealistic expectations. Of course, had you given the committee a chapter at a time for 5 or 6 weeks leading up to the magic date, some committee members would still have fallen behind reading it, because they are distracted with their own work. But then you could blame the them for not getting done!

Related Patterns    Extreme Programming and other agile software methodologies encourage Short Iterations and Frequent Releases. Such frequent releases are the Small Doses of the software development world. They enable more continuous feedback and allow the programmers to improve the quality of the code based on what they learn. The Avalanche antipattern is the source of many woes in the world of software, in large part due to the lack of feedback they afford from users and clients.

I learned the Small Doses pattern from Carl Page, my Artificial Intelligence II professor at Michigan State University in 1987. Professor Page was a bright guy and very good computer scientist, with an often dark sense of humor. He mentored his students and advisees with sardonic stories like Small Doses. He was also father to another famous Page, Larry.

Yesterday I was listening to comedian Rodney Laney do a bit called "Old Jobs". He explained that the best kind of job to have is a one-word job that everyone understands. Manager, accountant, lawyer -- that's good. But if you have to explain what you do, then you don't have a good job. "You know the inside of the pin has these springs? I put the springs on the inside of the pins." And then you have to explain why you matter... "Without me, the pins wouldn't go click." Bad job.

Okay, computer scientists and software developers, raise your hands if you've had to explain what you do to a new acquaintance at a party. To a curious relative? I should say "tried to explain", because my own attempts come up short far too often.

I think good Rodney has nailed a major flaw in being a computer scientist.

Sadly, going with the one-word job title of "programmer" doesn't help, and the people who think know what a programmer is often don't really know.

Even still, I like what I do and know why it's a great job.

(Thanks to the wonders of the web, you can watch another version of Laney's routine, Good Jobs, on-line at Comedy Central. I offer no assurance that you'll like it, but I did.)

Owen is reputed to have said something like "Don't give as a programming assignment something the student could just as easily do by hand." (I am still doing penance, even though Lent ended two weeks ago.) This has been dubbed Astrachan's Law, perhaps by Nick Parlante. In the linked paper, Parlante says that showmanship is the key to the Law, that

A trivial bit of code is fine for the introductory in-lecture example, but such simplicity can take the fun out of an assignment. As jaded old programmers, it's too easy to forget the magical quality that software can have, especially when it's churning out an unimaginable result. Astrachan's Law reminds us to do a little showing off with our computation. A program with impressive output is more fun to work on.

I think of this Astrachan's Law in a particular way. First, I think that it reaches beyond showmanship: Not only do students have less fun working on trivial programs, they don't think that trivial programs are worth doing at all -- which means they may not practice enough or at all. Second, I most often think of Astrachan's Law as talking about data. When we ask students to convert Fahrenheit to Celsius, or to sum ten numbers entered at the keyboard, we waste the value of a program on something that can be done faster with a calculator or -- gasp! -- a pencil and paper. Even if students want to know the answer to our trivial assignment, they won't see a need to master Java syntax to find it. You don't have to go all the way to data-intensive computing, but we really should use data sets that matter.

Yesterday, I encountered what might be a variant or extension of Astrachan's Law.

John Zelle of Wartburg College gave a seminar for our department on how to do virtual reality "on a shoestring" -- for $2000 or less. He demonstrated some of his equipment, some of the software he and his students have written, and some of the programs written by students in his classes. His presentation impressed me immensely. The quality of the experience produced by a couple of standard projects, a couple of polarizing filters, and a dollar pair of paper 3D glasses was remarkable. On top of that, John and his students wrote much of the code driving the VR, including the VR-savvy presentation software.

Toward the end of his talk, John was saying something about the quality of the VR and student motivation. He commented that it was hard to motivate many students when it came to 3D animation and filmmaking these days because (I paraphrase) "they grow up accustomed to Pixar, and nothing we do can approach that quality". In response to another question, he said that a particular something they had done in class had been quite successful, at least in part because it was something students could not have done with off-the-shelf software.

These comments made me think about how, in the typical media computation programming course, students spend a lot of time writing code to imitate what programs such as Photoshop and Audacity do. To me, this seems empowering: the idea that a freshman can write code for a common Photoshop filter in a few lines of Java or Python, at passable quality, tells me how powerful being able to write programs makes us.

But maybe to my students, Photoshop filters have been done, so that problem is solved and not worthy of being done again. Like so much of computing, such programs are so much a part of the background noise of their lives that learning how to make them work is as appealing to them as making a ball-point pen is to people of my age. I'd hope that some CS-leaning students do want to learn such trivialities, on the way to learning more and pushing the boundaries, but there may not be enough folks of that bent any more.

On only one day's thought, this is merely a conjecture in search of supporting evidence. I'd love to here what you think, whether pro, con, or other.

I do have some anecdotal experience that is consistent in part with my conjecture, in the world of 2D graphics. When we first started teaching Java in a third-semester object-oriented programming course, some of the faculty were excited by what we could do graphically in that course. It was so much more interesting than some of our other courses! But many students yawned. Even back in 1997 or 1998, college students came to us having experienced graphics much cooler than what they could do in a first Java course. Over time, fewer and fewer students found the examples knock-out compelling; the graphics became just another example.

If this holds, I suppose that we might view it as a new law, but it seems to me a natural extension of Astrachan's Law, a corollary, if you will, that applies the basic idea into the realm of application, rather than data.

My working title for this conjecture is the Pixar Effect, from the Zelle comment that crystallized it in my mind. However, I am open to someone else dubbing it the Wallingford Conjecture or the Wallingford Corollary. My humility is at battle with my ego.

Is anybody home? After a flurry of writing from SIGCSE, I returned home to family time and plenty of work at the office. The result has been one entry in ten days. I look forward to finishing up my SIGCSE reports, but they appear to lie forward a bit, as the next week or so are busy. I have a few new topics in the hopper waiting for a few minutes to write as well.

One bit of good news is that part of my busy-ness this week and next is launching the third iteration of my language topics course. We've done bash and PHP and are now moving on to Ruby, one of my favorite languages. Shell scripting is great, but its tools are too limited to make writing bigger programs fun. PHP was better than I expected, but in the end it is really about building web sites, not writing more general programs. (And after a few weeks of using the language, PHP's warts started to grate on me.)

Ruby is... sublime. It isn't perfect, of course, but even its idiosyncrasies seem to get out of my way when I am deep in code. I looked up the definition of 'sublime', as I sometimes do when I use a word which is outside my daily working vocabulary or is misused enough in conversation that I worry about misusing it myself. The first set of definitions have a subtlety reminiscent of Ruby. To "vaporize and then condense right back again" sounds just like Ruby getting out of my way, only for me to find that I've just written a substantial program in a few lines. (My favorite, though, is "well-meaning ineptitude that rises to empyreal absurdity"!)

This is my first time to teach Ruby formally in a course. I hope to use this new course beginning as a prompt to write a few entries on Ruby and what teaching it is like.

There are many wonderful resources for learning about and programming in Ruby. I've suggested that my students use the pickaxe book as a primary reference, even if they use the first edition, a complete version of which is available on-line. In today's class, though, I used a simple evolutionary example from Brian Marick's book Everyday Scripting with Ruby. I hesitated to use this book as the student's primary source because it was originally written for tester's without any programming background, and my course is for upper-division CS majors with several languages under their belts. But Brian works through several examples in a way that I find compelling, and I think I can base a few solid sessions on one or two of them.

This book makes me wonder how easy it would be to re-target a book from an audience like non-programming testers to an audience of scripting-savvy programmers who want to learn Ruby's particular yumminess. I know that in the course of writing the book Brian generalized his target audience from testers to the union of three different audiences (testers, business analysts, and programmers). Maybe after I've lived with the book and an audience of student programmers I'll have a better sense of how well the re-targeting worked. If it works for my class, then I'll be inclined to adopt it for the next offering of this course.

Anyway, today we evolved a script for diffing to directories of files for a tester. I liked the flow of development and the simple script that resulted. Now we will move on to explore language features and their use in greater depth. One example I hope to work through soon, perhaps in conjunction with Ruby's regular expressions, is "Finding Things", Tim Bray's chapter in Beautiful Code.

Oh, and I must say that this is the first time that one of my courses has a theme song -- and a fine theme song, indeed. Now, if only someone would create a new programming language called "Angie", I would be in heaven.

[A transcript of the SIGCSE 2008 conference: Table of Contents]

The second day of the conference opened with the keynote address by Google's VP of Search Products and User Experience, Marissa Mayer. She was one of the early hires as the company expanded beyond the founders, and from her talk it's clear that she has been involved with a lot of different products in her time there. She is also something the discipline of computer science could use more of, a young woman in a highly-visible technical and leadership and roles. Mayer is a great ambassador for CS as it seeks to expand the number of female high-school and college students.

This talk was called Innovation, Design, and Simplicity at Google and illustrated some of the ways that Google encourages creativity in its employees and gets leverage from their ideas and energy. I'll drop some of her themes into this report, though I imagine that the stories I relate in between may not always sync up. Such is the price of a fast-moving talk and five days of receding memory.

Creativity loves constraint.

I have written on this topic a few times, notably in the context of patterns, and it is a mantra for Google, whose home page remains among the least adorned on the web. Mayer said that she likes to protect its minimalist feel even when others would like to jazz it up. The constraints of a simple page force the company to be more creative in how it presents results. I suspect it also played a role in Google developing its cute practice of customizing the company logo in honor of holidays and other special events. Mayer said that minimalism may be a guide now, but it was not necessarily a reason for simplicity in the beginning. Co-founder Sergey Brin created the first Google home page, and he famously said, "I don't do HTML."

Mayer has a strong background both in CS and in CS education, having worked with the undergrad education folks at Stanford as a TA while an undergrad. (She said that it was Eric Roberts who first recommended Google to her, though at the time he could not remember the company's name!) One of her first acts as an employee was to run a user study on doing search from the Google homepage. She said that when users first sat down and brought up the page, they just sat there. And sat there. They were "waiting for the rest of it"! Already, users of the web were already accustomed to fancy pages and lots of graphics and text. She said Google added its copyright tag line at the bottom of the page to serve as punctutation, to tell the user that that's all there was.

Search is a fixed focus at Google, not a fancy user interface. Having a simple UI helps to harness the company's creativity.

Work on big problems, things users do every day.

Work on things that are easy to explain and understand.

Mayer described in some detail the path that a user's query follows from her browser to Google and back again with search results. Much of that story was as expected, though I was surprised by the fact that there are load balancers to balance the load on the load balancers that hand off queries to processors! Though I might have thought that another level of indirection would slow the process it down, indeed it is necessary in order to ensure that the system doesn't slow down. Even with the web server and the ad server and the mixers, users generally see their results in about 0.2 seconds. How is that for a real-time constraint to encourage technical creativity?

Search is the #2 task performed on the web. E-mail is (still) #1. Though some talking heads have begun to say that search is a mature business in need of consolidation, Google believes that search is just getting started. We know so little about how to do it well, how to meet the user's needs, and how to uncover untapped needs. Mayer mentioned a problem familiar to this old AI guy: determining the meaning of the words used in a query so that they can serve pages that match the user's conceptual intent. She used a nice example that I'll probably borrow the next time I teach AI. When a user asks for an "overhead view", he almost always wants to see a picture!

This points in the direction of another open research area, universal search. The age in which users want to search for text pages, images, videos, etc., as separate entities has likely passed. That partitioning is a technical issue, not a user's issue. The search companies will have to find a way to mix in links to all kinds of media when they serve results. For Google, this also means figuring out how to maintain or boost ad revenue when doing so.

Ideas come from everywhere.

Mayer gave a few examples. One possible source is office hours, which are usually thought of as an academic concept but which she has found useful in the corporate world. She said that the idea for Froogle walked through her office door one day with the scientist who had it.

Another source is experiments. Mayer told a longer story about Gmail. The company was testing it in-house and began to discuss how they could make many from it. She suggested the industry-standard model of giving a small amount of storage for free and then charging for more. This might well have worked, because Google's cost structure would allow it to offer much larger amounts at both pricing levels. But a guy named Paul -- he may be famous, but I don't know his last name -- suggested advertising. Mayer pulled back much as she expected users to do; do people really want Google to read their e-mail and serve ads? Won't that creep them out?

She left the office late that night believing that the discussion was on hold. She came back to work the next morning to find that Paul had implemented an experimental version in a few hours. She was skeptical, but the idea won her over when the system began to serve ads that were perfectly on-spot. Some folks still prefer to read e-mail without ads, but the history of Gmail has shown just how successful the ad model can be.

The insight here goes beyond e-mail. The search ad data base can be used on any paged on the web. This is huge... Search pages account for about 5% of the pages served on the web. Now Google knew that they could reach the other 95%. How's that for a business model?

To me, the intellectual lesson is this:

If you have an idea, try it out.

This is a power that computer programmers have. It is one of the reasons that I want everyone to be able to program, if only a little bit. If you have an idea, you ought to be able to try it out.

Not every idea will lead to a Google, but you never know which ones will.

Google Books started as a simple idea, too. A person, a scanner, and a book. Oh, and a metronome -- Mayer said that when she was scanning pages she would get out of rhythm with the scanner and end up photocopying her thumbs. Adding a metronome to the system smoothed the process out.

... "You're smart. We're hiring." worked remarkably well attracting job candidates. We programmers have big egos! Google is one of the companies that has made it okay again to talk about hiring smart people, not just an army of competent folks armed with a software development process, and giving them the resources they need to do big things.

Innovation, not instant perfection.

Google is also famous for not buying into the hype of agile software development. But that doesn't mean that Google doesn't encourage a lot of agile practices. For example, at the product level, it has long practiced a "start simple, then grow" philosophy.

Mayer contrasted two kinds of programmers, castle builders and nightly builders. Companies are like that, too. Apple -- at least to outside appearances -- is a castle-building company. Every once in a while, Steve Jobs et al. go off for a few years, and then come back with an iPod or an iPhone. This is great if you can do it, but only a few companies can make it work. Google is more of a nightly builder. Mayer offered Google News as a prime example -- it went through 64 iterations before it reached its current state. Building nightly and learning from each iteration is often a safer approach, and even companies that are "only good" can make it work. Sometimes, great companies are the result.

Data is a-political.

Mayer didn't mean Republican versus Democrat here, rather that well-collected data provide a more objective basis for making decisions than the preferences of a manager or the guesses of a designer or programmer. Designing an experiment that will distinguish the characteristics you are in interested, running it, and analyzing the data dispassionately are a reliable way to make good decisions. Especially when a leader's intuition is wrong, such as Mayer's was on Gmail advertising.

She gave a small plug for using Google Trends as a way to observe patterns in search behavior when they might give an idea about a question of interest. Query volume may not not change much, but the content of the queries does.

Users, users, users.

What if some users want more than the minimalist white front page offered by Google? In response to requests from a relatively small minority of users -- and the insistent voices of a few Google designers -- iGoogle is an experiment in offering a more feature-filled portal experience. How well will it play? As is often the case, the data will tell the story.

Give license to dream.

Mayer spent some time talking about the fruits of Google's well-known policy of 20% Time, whereby every employee is expected to spend 1/5 of his or her time working on projects of personal interest. While Google is most famous for this policy these days, like most other ideas it isn't new. At ChiliPLoP this week, Richard Gabriel reported that Eric Schmidt took this idea to Google with him when he left Sun, and Pam Rostal reported that 3M had a similar policy many years ago.

But Google has rightly earned its reputation for the fruits of 20% Time. Google News. Google Scholar. Google Alerts. Orkut. Froogle Wireless. Much of Google Labs. Mayer said that 50% of Google's new products come from these projects, which sounds like a big gain in productivity, not the loss of productivity that skeptics expect.

I have to think that the success Google has had with this policy is tied pretty strongly with the quality of its employees, though. This is not meant to diss the rest of us regular guys, but you have to have good ideas and the talent to carry them out in order for this to work well. That said, these projects all resulted from the passions of individual developers, and we all have passions. We just need the confidence to believe in our passions, and a willingness to do the work necessary to implement them.

Most of the rest of Mayer's talk was a litany of these projects, which one wag in the audience called a long ad for the goodness of Google. I wasn't so cynical, but I did eventually tire of the list. One fact that stuck with me was the description of just how physical the bits of Google Earth are. She described how each image of the Earth's surface needs to be photographed at three or four different elevations, which requires three or four planes passing over every region. Then there are the cars driving around taking surface-level shots, and cameras mounted to take fixed-location shots. A lot of physical equipment is at work -- and a lot of money.

Share the data.

This was the last thematic slogan Mayer showed, though based on the rest of the talk I might rephrase it as the less pithy "Share everything you can, especially the data." Much of Google's success seems based in a pervasive corporate culture of sharing. This extends beyond data to ideas. It also extends beyond Google campus walls to include users.

The data-sharing talk led Mayer to an Easter Egg she could leave us. If you check Google's Language Tools page, you will see Bork, Bork, Bork, a language spoken (only?) by the Swedish chef on the Muppets. Nonetheless, the Bork, Bork, Bork page gets a million hits a year (or was it a day?). Google programmers aren't the only ones having fun, I guess.

Mayer closed with suggestions for computer science educators. How might we prepare students better to work in the current world of computing? Most of her recommendations are things we have heard before: build and use applications, work on large projects and in teams, work with legacy code, understand and test at a large scale, and finally pay more attention to reliability and robustness. Two of her suggestions, though, are ones we don't hear as often and link back to key points in her talk: work with and understand messy data, and understand how to use statistics to analyze the data you collect.

After the talk, folks in the audience asked a few questions. One asked how Google conducts user studies. Mayer described how they can analyze data of live users by modding the key in user cookies to select 1/10 or 1/1000 of the user population, give those users a different experience, and then look at characteristics such as click rate and time spent on page by these users compared to the control group.

The best question was in fact a suggestion for Google. Throughout her talk, Mayer referred repeatedly to "Google engineers", the folks who come up with neat ideas, implement them in code, test them, and then play a big role in selling them to the public. The questioner pointed out that most of those "engineers" are graduates of computer science programs, including herself, Sergey Brin, and Larry Page. He then offered that Google could do a lot to improve the public perception of our discipline if it referred to its employees as computer scientists.

I think this suggestion caught Mayer a little off-guard, which surprised me. But I hope that she and the rest of Google's leadership will take it to heart. In a time when it is true both that we need more computer science students and that public perception of CS as a discipline is down, we should be trumpeting the very cool stuff that computer scientists are doing at places like Google.

All in all, I enjoyed Mayer's talk quite a bit. We should try to create a similarly creativity-friendly environment for our students and ourselves. (And maybe work at a place like Google every so often!)

As I mentioned in my last SIGCSE entry, I have moved from carefree Portland to Carefree, Arizona, for ChiliPLoP 2008. The elementary patterns group spent yesterday, its first, working on the idea of integrating large datasets into the CS curriculum. After a few years of working on specific examples, both stand-alone and running, we started this year thinking about how CS students can work on real problems from many different domains. In the sciences, that often means larger data sets, but more important it means authentic data sets, and data sets that inspire students to go deeper. On the pedagogical side of the ledger, much of the challenge lies in finding and configuring data sets so that they can used reliably and without unnecessary overhead placed on the adopting instructor.

This morning, we volunteered to listen to a presentation by the other hot topic group on its work from yesterday: a "green field" thought experiment designing an undergrad CS program outside of any constraints from the existing university structure. This group consists of Dave West and Pam Rostal, who presented an earlier version of this work at the OOPSLA 2005 Educators' Symposium, and Richard Gabriel, who brings to the discussion not only an academic background in CS and a career in computer science research and industry but also an MFA in poetry. Perhaps the key motivation for their hot topic is that most CS grads go on to be professional software developers or CS researchers, and that our current way of educating them doesn't do an ideal job of preparing grads for either career path.

Their proposal is much bigger than I can report here. They started by describing a three-dimensional characterization of different kinds of CS professionals, including provocative and non-traditional labels as "creative builder", "imaginative researcher", and "ordinologist". The core of the proposal is the sort of competency-based curriculum that West and Rostal talked about at OOPSLA, but I might also describe it as studio-based, apprenticeship-based, and project-based. One of their more novel ideas is that students would learn everything they need for a liberal arts, undergraduate computer science education through their software projects -- including history, English, writing, math, and social science. For example, students might study the mathematics underlying a theorem prover while building a inference engine, study a period of history in order to build a zoomable timeline on the web for an instructional web site, or build a Second Life for a whole world in ancient Rome.

In the course of our discussion, the devil's advocates in the room raised several challenging issues, most of which the presenters had anticipated. For example, how do the instructors (or mentors, as they called them) balance the busy work involved in, say, the students implementing some Second Life chunk with the content the students need to learn? Or how does the instructional environment ensure that students learn the intellectual process of, say, history, and not just impose a computer scientist's worldview on history? Anticipating these concerns does not mean that they have answers, only that they know the issues exist and will have to be addressed at some point. But this isn't the time for self-censorship... When trying to create something unlike anything we see around us, the bigger challenge is trying to let the mind imagine the new thing without prior restraint from the imperfect implementations we already know.

We all thought that this thought experiment was worth carrying forward, which is where the change of direction comes in. While our group will continue to work on the dataset idea from yesterday, we decided in the short term to throw our energies into the wild idea for reinventing CS education. The result will be two proposals to OOPSLA 2008: one an activity at the Educators' Symposium, and the other an Onward! paper. This will be my first time as part of a proposal to the Onward! track, which is both a cool feeling and an intimidating prospect. We'll see what happens.

This set of entries records my experiences at SIGCSE 2008, in Portland, Oregon, March 12-16. I'll update it as I post new pieces about the conference. One of those entries will explain why my posts on SIGCSE may come more slowly than they might.

Primary entries:

• SIGCSE Day 1
  • Randy Pausch and Alice
    ... on Pausch's contributions to CS education
  • The Mystery Problem
    ... on correlations and students' mental models
  • Nifty Assignments
    ... a few cool things to ask students to do
  • Innovating Our Image
    ... how two NSF Distinguished Educational Fellows are trying
  • This and That
    ... miscellaneous observations from the day
• SIGCSE Day 2 -- coming
  • Plenary Address by Marissa Mayer
    ... on Google's approach to creativity and simplicity
• SIGCSE Day 3
  • Expectation and Outcome
    ... a tale of two sessions
  • CS Past, Present, and Future
    ... on computing's contributions and coming challenges

Ancillary entries:

• Not an Example of Problem-Based Learning
  -- no matter what people say
• On the Roads Back in Portland
  -- running in an old new place

The Prometheus Awards are the Iowa tech industry equivalent of the Academy Awards. At last night's 2008 award ceremony, CS alumni from my school made a good showing.

T8Design won Software Company of the Year in the Small Company Division. Alumnus Wade Arnold is one of T8's co-founders, its former CTO, and its current CEO. Wade accepted the statuette and gave a fine acceptance speech. At least two other alumni were there in the T8 delegation, and it was good to be able to congratulate them all in person.

Alliance Technologies won the award for IT Service Provider of the Year. Alumnus and UNI CS advisory board member Mike Lang, CEO and President of Alliance, represented the company on the dais.

There were two other software connections to our program among the winners. Alumnus Tony Bibbs was part of the team that wrote one of the pieces of software nominated for Best Innovation in Government, which was won by ABC Virtual. And the most distant connection involved Express Auto, which won Top Growth Company of the Year in the Under $25M Division. One of the first programs that got Express Auto off the ground back in the mid-1990s was written by a UNI alumnus and CS advisory board member Fred Zelhart.

It was a good evening for UNI Computer Science alumni, and a good night for this CS prof. With the exception of Mike, who graduated the year before I came here, I had all of these entrepreneurs and developers as students in class, and now I work with Mike, Fred, and Wade on an ongoing basis. I feel a little paternal -- now fraternal -- pride!

This provocative statement appears in the Roly Perera essay that I mentioned recently:

All I know is that if there is a place for post-modernist, lit-crit, social constructivist thinking in the modern world, it's nowhere near the field of computing.

Robert Biddle and James Noble may have a duel on their hands (PDF)! I think that we have the makings of an outstanding OOPSLA 2008 panel.

I'd certainly like to hear more from Perera if he has more ideas of this sort:

... emergent behaviours are in a sense dual to the requirements on a solution. Requirements are known and obligate the system in certain ways, whereas emergent behaviours ("emergents", one could call them) are those which are permitted by the system, but which were not known a priori.

This is an interesting analogy, one I'd like to think more about. For some reason, it reminds of Jon Postel's dictum about protocol design, "Be liberal in what you accept, and conservative in what you send" and Bertrand Meyer's ideas about design by contract.

This week I ran across Jonathan Edwards's review of Gregor Kiczales's OOPSLA 2007 keynote address, "Context, Perspective and Programs" (which is available from the conference podcast page). Having recently commented on Peter Turchi's keynote, I figured this was a good time to go back and listen to all of Gregor's again. I first listened to part of it back when I posted it to the page, but I have to admit that I didn't understand it all that well then. So a second listen was warranted. This time I had access to his slides, which made a huge difference.

In his talk, Kiczales tries to bring together ideas from philosophy, language, and our collective experience writing software to tackle a problem that he has been working around his whole career: programs are abstractions, and any abstraction represents a particular point of view. Over time, the point of view changes, which means that the program is out of sync. Software folks have been thinking about ways to make programs capable of responding naturally as their contexts evolve. Biological systems have offered some inspiration in the last decade or so. Kiczales suggests that computer science's focus on formality gets in the way of us finding a good answer to our problem.

Some folks took this suggestion as meaning that we would surrender all formalism and take up models of social negotiation as our foundation. Roly Perera wrote a detailed and pointed review of this sort. While I think Perera does a nice job of placing Kiczales's issues in their philosophical context, I do not think Kiczales was saying that we should go from being formal to being informal. He was suggesting that we shouldn't have to go from being completely formal to being completely informal; there should be a middle ground.

Our way of thinking about formality is binary -- is that any surprise? -- but perhaps we can define a continuum between the two. If so, we could write our program at an equilibrium point for the particular context it is in and then, as the context shifts, allow the program to move along the continuum in response.

Now that I understand a little better what Kiczales is saying, his message resonates well with me. It sounds a lot like the way a pattern balances the forces that affect a system. As the forces change, a new structure may need to emerge to keep the code in balance. We programmers refactor our code in response to such changes. What would it be like for the system to recognize changes in context and evolve? That's how natural systems work.

As usual, Kiczales is thinking thoughts worth thinking.

While catching up on some work at the office yesterday -- a rare Saturday indeed -- I listened to Peter Turchi's OOPSLA 2007 keynote address, available from the conference podcast page. Turchi is a writer with whom conference chair Richard Gabriel studied while pursuing his MFA at Warren Wilson College. I would not put this talk in the same class as Robert Hass's OOPSLA 2005 keynote, but perhaps that has more to do with my listening to an audio recording of it and not being there in the moment. Still, I found it to be worth listening as Turchi encouraged us to "get lost" when we want to create. We usually think of getting lost as something that happens to us when we are trying to get somewhere else. That makes getting lost something we wish wouldn't happen at all. But when we get lost in a new land inside our minds, we discover something new that we could not have seen before, at least not in the same way.

As I listened, I heard three ideas that captured much of the essence of Turchi's keynote. First was that we should strive to avoid preconception. This can be tough to do, because ultimately it means that we must work without knowing what is good or bad! The notions of good and bad are themselves preconceptions. They are valuable to scientists and engineers as they polish up a solution, but they often are impediments to discovering or creating a solution in the first place.

Second was the warning that a failure to get lost is a failure of imagination. Often, when we work deeply in an area for a while, we sometimes feel as if we can't see anything new and creative because we know and understand the landscape so well. We have become "experts", which isn't always as dandy a status as it may seem. It limits what we see. In such times, we need to step off the easy path and exercise our imaginations in a new way. What must I do in order to see something new?

This leads to the third theme I pulled from Turchi's talk: getting lost takes work and preparation. When we get stuck, we have to work to imagine our way out of the rut. For the creative person, though, it's about more about getting out of a rut. The creative person needs to get lost in a new place all the time, in order to see something new. For many of us, getting lost may seem like as something that just happens, but the person who wants to be lost has to prepare to start.

Turchi mentioned Robert Louis Stevenson as someone with a particular appreciation for "the happy accident that planning can produce". But artists are not the only folks who benefit from these happy accidents or who should work to produce the conditions in which they can occur. Scientific research operates on a similar plane. I am reminded again of Robert Root-Bernstein's ideas for actively engaging the unexpected. Writers can't leave getting lost to chance, and neither can scientists.

Turchi comes from the world of writing, not the world of science. Do his ideas apply to the computer scientist's form of writing, programming? I think so. A couple of years ago, I described a structured form of getting lost called air-drop programming, which adventurous programmers use to learn a legacy code base. One can use the same idea to learn a new framework or API, or even to learn a new programming language. Cut all ties to the familiar, jump right in, and see what you learn!

What about teaching? Yes. A colleague stopped by my office late last week to describe a great day of class in which he had covered almost none of what he had planned. A student had asked a question whose answer led to another, and then another, and pretty soon the class was deep in a discussion that was as valuable, or more, than the planned activities. My colleague couldn't have planned this unexpectedly good discussion, but his and the class's work put them in a position where it could happen. Of course, unexpected exploration takes time... When will they cover all the material of the course? I suspect the students will be just fine as they make adjustments downstream this semester.

What about running? Well, of course. The topic of air-drop programming came up during a conversation about a general tourist pattern for learning a new town. Running in a new town is a great way to learn the lay of the land. Sometimes I have to work not to remember landmarks along the way, so that I can see new things on my way back to the hotel. As I wrote after a glorious morning run at ChiliPLoP three years ago, sometimes you run to get from Point A to Point B; sometimes, you should just run. That applies to your hometown, too. I once read about an elite women's runner who recommended being dropped off far from your usual running routes and working your way back home through unfamiliar streets and terrain. I've done something like this myself, though not often enough, and it is a great way to revitalize my running whenever the trails start look like the same old same old.

It seems that getting lost is a universal pattern, which made it a perfect topic for an OOPSLA keynote talk.

... though I can't in good conscience say, "for I know not what I do."

1. Write a script named simple-interest.php that defines a function to compute the simple interest on an amount, given the amount, annual interest rate, and number of months. Your script should apply the function to its command-line arguments, which are those same values.

The rest of my PHP class's first homework is more reasonable, with a couple of problems repeated from the bash class's first assignment as a way for students to get a sense of trade-offs between shell programming and scripting in a more general-purpose language.

Still, I had to squeeze my eyes shut tight to hit the key that published this assignment. I keep telling myself that this is just an ice-breaking assignment for students who have never written any PHP before, or learned how to access command-line arguments, or convert strings to integers. That such a simple, context-free task is a nice way for them to succeed on their first effort. That our future assignments will be engaging, challenging, worthwhile. But... Ick.

The first time I teach a course, there always seem to be clunkers like this. Starting from scratch, relying on textbooks for inspiration, and working under time pressure all work against my goal of making everything students do in the class worth their time and energy. I suppose that problems such as this one are my opportunities to improve next time out.

My despair notwithstanding, I suspect that many students are happy enough to have at least one problem that is a gift, however uninteresting it may be. Maybe I can find solace in that while I'm working on exercises for my next problem set.

Are all the open jobs in computing that we keep hearing about going unfilled?

Actually -- they're not. Companies do fill those jobs. They fill them with less expensive workers, without computing degrees, and then train them to program.

Mark Guzdial is concerned that some American CEOs and legislators are unconcerned -- "So? Where's the problem?" -- and wonders how we make the case that degrees in CS matter.

I wonder if the US would be better off if we addressed a shortage of medical doctors by starting with less expensive workers, without medical degrees, and then trained them to practice medicine? We currently do face a shortage of medical professionals willing to practice in rural and underprivileged areas.

The analogy is not a perfect one, of course. A fair amount of the software we produce in the world is life-critical, but a lot is not. But I'm not sure whether we want to live in a world where our financial, commercial, communication, educational, and entertainment systems depend on software to run, and that software is written by folks with a shallow understanding of software and computing more generally.

Maybe an analogy to the law or education is more on-point. For example, would the US would be better off if we addressed a shortage of lawyers or teachers by starting with less expensive workers, without degrees in those areas, and then trained them? A shortage of lawyers -- ha! But there is indeed a critical shortage of teachers in many disciplines looming in the near future, especially in math and science. This might lead to an interesting conversation, because many folks advocate loosening the restrictions on professional training for folks who teach in our K-12 classrooms.

I do not mean to say that folks who are trained "on the job" to write software necessarily have a shallow understanding of software or programming. Much valuable learning occurs on the job, and there are many folks who believe strongly in a craftsmanship approach to developing developers. My friend and colleague Ken Auer built his company on the model of software apprenticeship. I think that our university system should adopt more of a project-based and apprenticeship-based approach to educating software developers. But I wonder about the practicality of a system that develops all of its programmers on the job. Maybe my view is colored by self-preservation, but I think there is an important role for university computing education.

Speaking of practicality, perhaps the best way to speak to the CEOs and legislators who doubt the value of academic CS degrees is in their language of supply and productivity. First, decentralized apprenticeship programs are probably how people really became programmers, but they operate on a small scale. A university program is able to operate on a slightly larger scale, producing more folks who are ready for apprenticeship in industry sooner than industry can grow them from scratch. Second, the best-prepared folks coming out of university programs are much more productive than the folks being retrained, at least while the brightest trainees catch. That lack of productivity is at best an opportunity cost, and at worst an invitation for another company to eat your lunch.

Of course, I also think that in the future more and more programmers will be scientists and engineers who have learned how to program. I'm inclined to think that these folks and the software world will be better off being educated by folks with a deep understanding of computing. Artists, too. And not only for immediately obvious economic reasons.

I don't usually play meme games in my web, but as I am winding down for the week I ran across this one on Brian Marick's blog: grab the nearest book, open it to page 123, go to the 5th sentence, and type up the three sentences beginning there.

With my mind worn out from a week in which I caught something worse than a meme, I fell prey and swung my arm around. The nearest book was Beautiful Code. Technically, I suppose that a stack of PHP textbooks is a couple of inches closer to me, but does anyone really want to know what is on Page 123 of any of them?

Here is the output:

The resultant index (which was called iSrc in FilterMethodCS) might be outside the bounds of the array. The following code loads an integer 0 on the stack and branches if iSrc is less than 0, effectively popping both operands from the stack. This is a partial equivalent of the if statement conditional in line 19 of Example 8-2:

Okay, that may not be much more interesting than what a PHP book might have to say, at least out of context. I'm a compiler junkie, though, and I was happy to find a compiler-style chapter in the book. So I turned to the beginning of the chapter, which turns out to be "On-the-Fly Code Generation for Image Processing" by Charles Petzold. I must admit that this sounds pretty interesting to me. The chapter opens with something that may be of interest to others, too:

Among the pearls of wisdom and wackiness chronicled in Steve Levy's classic "Hackers: Heroes of the Computer Revolution" (Doubleday), my favorite is this one by Bill Gosper, who once said, "Data is just a dumb kind of programming."

Petzold then goes on to discuss the interplay between code and data, which is something I've written about as one of the big ideas computer science has taught the world.

What a nice way for me to end the week. Now I have a new something to read over the weekend. Of course, I should probably spend most of my time with those PHP textbooks; that language is iteration 2 in my course this semester. But I've avoided "real work" for a lot less in the past.

That is the title of a blog post that I planned to write five or six weeks ago. Here it is over a month later, and the course just ended. Well, it ended, and it begins again on Tuesday. So now I am thinking agile thoughts as I think back over my course, and still thinking agile thoughts as I prepare for the course. Let me explain.

810:151 is a different sort of course for us. We try to expose students to several different programming languages in the course of their undergraduate study. Even so, it is common for students to graduate thinking, "I wish I'd learned X." Sometimes X is a relatively new language, such as Scala or Groovy. Sometimes it's a language that is now more mainstream but has not yet made it into one of our courses, such as Ruby. Sometimes it is even a language we do emphasize in a course, such as Scheme, but in a course they didn't have an opportunity to take. We always have told students who express this wish that they should be well-equipped to learn a new language on their own, and they are. But...

While taking a full load of courses and working part-time (or taking a part-time load of courses and working full-time), it is often hard for students to set aside time to learn completely optional. People talk about "the real world" as if it is tougher than being in school, but students face a lot of competing demands for their time. Besides, isn't it often more fun to learn something from an expert who can help you learn the tricks of the trade faster and miss a few of the potholes that lie along the way?

I sometimes think of this course, which we call "Topics in Programming Languages", as a "make time" course for students who want to learn a new language, or perhaps a broader topic related to language, but who want or need the incentive that a regular course, assigned readings, and graded work provides. The support provided by the prof's guidance also is a good safety net for the less seasoned and less confident. For these folks, one of the desired outcomes is for them to realize, hey, I really can learn a language on my own.

We usually offer each section of 810:151 as a 1-credit course. The content reason is that the course has the relatively straightforward purpose of teaching a single language, without a lot of fluff. The practical purpose is that we can offer three 1-credit courses in place of a single 3-credit course. Rather than meet one hour per week for the entire semester, the course can meet 3 hours per week for 5 weeks. This works nicely for students who want to take all three, as they look and feel like a regular course. It also works nicely for students who choose to take only one or two of the courses, as they need not commit an entire semester's worth of attention to them.

This is my first semester assigned (by me) to teach this odd three-headed course. The topics this semester are Unix shell programming in bash, PHP, and Ruby.

I've been thinking of the three courses as three 5-week iterations. Though the topics of the three courses are different, they share a lot in terms of being focused on learning a language in five weeks. How much material can I cover in a course? How can students best use their time? How can I best evaluate their work and provide feedback? Teaching three iterations of a similar course in one semester is so much better for me when it comes to taking what I learn and trying to improve the next offering. With an ordinary course taught every semester, I would have to wait until next fall to begin implementing improvements; with an ordinary course three-course rotation, I would have to wait until Fall 2009!

I opted to dispense with all examinations and evaluate students solely in terms of the bash scripts they wrote. The goal of the course is for students to learn how to program in bash, so that is where I wanted the students' attention to be. One side effect of this decision is that the course is not really over yet; students will work on their final problem set in the coming week, and I'll have to grade it next Friday. The problem sets have consisted mostly in small-ish scripts that exercise the features of bash as we encounter them. We did have one larger task that students solved in three parts over the course of the semester, a processor for a Markdown-like mark-up language that produces HTML. This project scratched one of my own itches, as I like to use simple text-based, e-mail-friendly mark-up, and now I have a simple bash script that does the job!

One thing I did not do this semester that I thought I might, and which perhaps I should, is to work with them through a non-trivial shell script or two. I had thought that the fifth week would be devoted to examining and extending larger scripts, but I kept uncovering more techniques and ideas that I wanted them to see. Perhaps I could use a real script as a primary source for learning the many features of bash, instead of building their skills from the bottom up. That is how many of them have to come to know what little they know about shell scripting, by confronting a non-trivial script for building or configuring an open-source application that interests them. To be honest, though, I think that the bottom-up style that we used this semester may prepare them better for digging into a more complex script than starting with a large program first. This is one of the issues I hope to gain some insight into from student feedback on the course.

Making this "short iterations" more interesting is the fact that some students will be in all three of the iterations, but there will be a significant turnover in the class rosters. The client base evolves, but there should be enough overlap that I can get some comparative feedback as I try to implement improvements.

I tried to follow a few other agile principles as I started teaching this new prep. I tend to start each new course with a template from my past courses, from the way I organize sessions and lecture notes to the look-and-feel of the web site. This semester, I tried to maintain a YAGNI mindset: start as simple as I can, and add new elements only as I use them -- not when I think I need them tomorrow. By and large I have succeeded in this regard. My web site is bare-bones in comparison to my past sites, and lecture notes are plain text records of in-class activities and short messages to remind me and the students of what we discussed. I saved a lot of time not trying to produce attractive and complete lecture notes in HTML. Maybe some day, but this time around I just didn't need them.

One agile practice that I didn't think to encourage soon enough was unit testing. Shame on me. Some students suffered far more than I from this oversight. Many did a substandard job of testing their scripts, in part I think because they were biting off too much of a task to start. Unix pipelines are almost perfectly suited to unit testing, as one can test the heck out of each stage in isolation, growing the pipeline one stage at a time until the task is solved. The fact that each component is reading from stdin and writing to stdout means that later stages can be tested independent of the stages that precede it before we add it to the end.

For whatever reason, it didn't occur to me that there would exist an shUnit. It's too late for me to use it this semester, but I'll be sure to put phpUnit to good use in the next five weeks. And I've always known that I would use a unit testing framework such as this one for Ruby. Heck, we may even roll our own as we learn the language!

I've really enjoyed teaching a course with the Unix philosophy at the forefront of our minds. Simple components, the universal interface of plain text streams, and a mandate to make tools to work together -- the result is an amazingly "agile" programming environment. The best way to help students see the value of agile practices is to let them live in an environment where that is natural, and let them feel the difference from the programming environments in which they other times find themselves. I just hope that my course did the mindset justice.

The tool-builder philosophy that pervaded this course reminded me of this passage from Jason Marshall's Something to Say:

There's an old saying, "A good craftsman never blames his tools." Many people take this to mean "Don't make excuses," or even, "Real men don't whine when their tools break." But I take it to mean, "A good craftsperson does not abide inferior tools."

A good craftsman never blames his tools, because if his tools are blameworthy, he finds better tools. I associate this idea more directly with the pragmatic programmers than with the agile community, but it seems woven into the fabric of the agile approaches. The Agile Manifesto declares that we value "individuals and interactions over processes and tools", but I do not take this to mean that tools don't matter. I think it means that we should use tools (and processes) that empower us to focus our energy on people and interactions. We should let programs do what they do best so that we programmers can do what we do best. That's why we have unit testing frameworks, refactoring tools, automatic build tools, and the like. It's also why Unix is far more human-affirming than it is sometimes given credit for.

As I told students to close the lecture notes for this course: Don't settle for inferior tools. You are a computer programmer. Make the tools that make you better.

I recently came across a SIGCSE paper from 1991 called Integrating Writing into Computer Science Courses, by Linda Hutz Pesante, who at the time was affiliated with the Software Engineering Institute at Carnegie Mellon. This paper describes both content and technique for teaching writing within a CS program, a topic that cycles back into the CS education community's radar every few years. (CS academics know that it is important even during trough periods, but their attention is on some other, also often cyclic, attention-getter.)

What caught my attention about Pesante's paper is that she tries help software engineers to use their engineering expertise to the task of writing technical prose. One of her other publications, a video, even has the enticing title, Applying Software Engineering Skills to Writing. I so often think about applying ideas from other disciplines to programming, the thought of applying ideas from software development to another discipline sounded like a new twist.

Pesante's advice on how to teach writing reflects common practice in teaching software development:

• Motivate students so that they know what to expect.
• Attend to the writing process, as well as the final product.
• Use analogies to programming, such as debugging and code reviews.
• Have students practice, and give them feedback.

Given Pesante's affiliation with the SEI, her suggestions for what to teach about writing made a bigger impression on me. The software engineering community certainly embraces a broad range of development "methodologies" and styles but, underlying its acceptance even of iterative methods, there seems to be a strong emphasis on planning and "getting things right" the first time.

Her content advice relies on the notion that "writing and software development have something in common", from user analysis through the writing itself to evaluation. As such, a novice writer can probably learn a lot from how programmers write code. Programmers like to be creative and explore when they write, too, but they also know that thinking about the development process can add structure to a potentially unbounded activity. They use tools to help them manage their growing base of documents and to track revisions over time. That part of the engineer's mindset can come in handy for writers. For the programmer who already has that mindset, applying it to the perhaps scary of writing prose can put the inexperienced writer at ease.

Pesante enumerates a few other key content points:

• Writing takes place in a context.
• The writing process is neither linear nor algorithmic.
• The writing process is iterative.
• Correct is not the same as effective.

The middle two of these especially feel more agile than the typical software engineering discussion. I think that the agile software community's emphasis on short iterations with frequent releases of working software to the client also matches quite nicely the last of the bullets. It's all too easy to do a good job of analysis and planning, produce a piece of software that is correct by the standards of the analysis and plan, and find that it does not meet the user's needs effectively. With user feedback every few weeks, the development team has many more opportunities to ensure that software stays on a trajectory toward effectiveness.

Most people readily accept the idea that creative writing is iterative, non-linear, and exploratory. But I have heard many technical writers and other writers of non-creative prose say that their writing also has these features -- that they often do not know what they had to say, or what their ultimate product would be, until they wrote it. My experience as a writer, however limited, supports the notion that almost all writing is exploratory, even when writing something so pedestrian as a help sheet for students using an IDE or other piece of software.

As a result, I am quite open to the notion that programming -- what many view as the creating of an "engineered" artifact -- also iterative, non-linear, and exploratory. This is true, I think, not only for what we actually call "exploratory programming" but also for more ordinary programming tasks, where both the client and I think we have a pretty good handle on what the resulting programming should do and look like. Often, over the course of writing an ever-larger program, our individual and shared understanding of the program evolves. Certainly my idea of the internal shape of the program -- the structure, the components, the code itself -- changes as the program grows. So I think that there is a lot to be learned going both directions in the writing:programming metaphor.

... about teaching, and about software development.

The February 2008 issue of Smithsonian Magazine contains an article called Being Funny, by comedian, writer, and actor Steve Martin, that has received a fair amount of discussion on-line already. When I read it this weekend, I was surprised by how similar some of the lessons Martin learned as he grew into a professional comedian are to lessons that software developers and teachers learn. I enjoyed being reminded of them.

I gave myself a rule [for dealing with the audience]. Never let them know I was bombing: this is funny, you just haven't gotten it yet.

This is about not showing doubt. Now, I think it's essential for an instructor to be honest -- something I wrote about a while back, in the context of comedy as well. So I don't mean that I as teacher should try to bluff my way through something I don't know or know but have botched. Martin is talking about the audience getting it, and the doubt that enters my mind when a classroom of students seem not to. I experience this occasionally when I teach a course like Programming Languages to a new group of students. Some classes don't warm to me or the material in quite the same way as others, but I know that the material I'm teaching and the basic approach I am using are sound. When this semester's crowd takes a while to catch on -- or if they are openly skeptical of the material or approach -- it's essential that I remain confident. Sure, I'll make adjustments to the presentation to account for my current students' needs, but I should remain steadfast: This is good stuff; they'll get it soon.

I'm not sure this applies as well for software developers. Often, when my users don't get it yet and I feel compelled to bull on through, I have gone beyond the requirements, at least as my users understand them. In those cases, it's usually better to say to myself "You aren't gonna need it" and simplify.

Everything was learned in practice, and the lonely road, with no critical eyes watching, was the place to dig up my boldest, or dumbest, ideas and put them onstage.

This is hard to do as a teacher. I don't get to go on the road to a hundred musty nightclubs and try out my new lecture on continuation-passing style, as Martin did with his bits. He was practicing on small stages in order to prepare for the big ones, such as The Tonight Show. It's next to impossible for me to try a lecture out on a representative audience that isn't my regular audience: my students. I can practice my research talks before a small local audience before taking them to a conference, but the number of repetitions available to me is rather small even in that scenario. For class sessions, I pretty much have to try them out live, see what happens, and feed the results back into my next performance.

Fortunately, I'm not often trying radically new ideas out in a lecture, so fewer and live repetitions may suffice. I have tried teaching methods that quite different than the norm for me and for my students, such as a Software Systems course or gen-ed capstone course with no lecture and all in-class exercises. In those scenarios, I had to follow the advice discussed above: This is going to work; they just haven't gotten it yet...

This piece of advice applies perfectly to programming. The lonely road is my office or my room at home, where I can try out every new idea that occurs to me by writing a program (or ten) and seeing how it works. No critical eyes but my eye, which I turn off. Bold ideas, dumb ideas -- I find out which are which through practice.

The consistent work enhanced my act. I learned a lesson: it was easy to be great. Every entertainer has a night when everything is clicking. These nights are accidental and statistical: like lucky cards in poker, you can count on them occurring over time. What was hard was to be good, consistently good, night after night, no matter what the circumstances.

This is so true of teaching that I have nothing more to say. Read Martin's line again.

I think this is also true of programming, and writing more generally. Every so often, a great idea comes to mind; a great turn of phrase; a cool little hack that solves the problem at hand. To be a good programmer, you need to practice, so that each time you sit down to code you can produce something of value. That kind of skill is earned by practice, and, I think, attainable by everyone.

On one of my appearances [on The Tonight Show, after [Johnny Carson] had done a solid impression of Goofy the cartoon dog, he leaned over to me during a commercial and whispered prophetically, "You'll use everything you ever knew."

When working with students, I find myself borrowing from every good teacher I've ever had, and drawing on any experience I can recall. I've borrowed standards and attitudes from one of my favorite undergrad profs, who taught me the value of meeting deadlines. I've used phrases and jokes spoken by my high school chemistry teacher, who showed us that studying a difficult subject could be fun, with the right mindset and a supportive group of classmates. Whatever works, works. Use it. Adapt it.

Likewise, this advice is true when programming. In the last few years, the notion of scrapheap programming has become quite popular. In this style, a programmer looks for old pieces of code that do part of the job at hand, adapts them, and glues them together to get the job done. But this is how all writers and programmers work, drawing on all of the old bits of code rolling around their heads. In addition to practice, you can improve as a programmer by exposing yourself to as many different programs as possible. That way, you will see the data structures, the idioms, the design patterns, and, yes, the snippets of code that you will use twenty years from now when the circumstance is right. That may seem goofy, but it's not.

Finally:

I believed it was important to be funny now, while the audience was watching, but it was also important to be funny later, when the audience was home and thinking about it.

As a teacher, I would like for what my students see, hear, and do in class today to make an impression today. That is what makes the material memorable and, besides, it's a lot more fun for both of us that way than the alternative. But perhaps more important, I would like for the ideas to make a lasting impression. When the student is home thinking about the course, or an assignment, or computer science in general, I want them to realize how much depth the idea or technique has, or breadth. Today's memorability can be ephemeral. When the idea sticks later, it can affect how the student programs forever.

The hard part is trusting when I don't see students getting it in real-time. Martin says that he learned not to worry if a gag got no response from his audience, "as long as I believed it had enough strangeness to linger". Strangeness may not be what I hope for in my class sessions, but I know what he means.

As a programmer, I think this advice applies, too. When I was an undergrad, I was once on a team building a large system as part of our senior project course. One of our teammates loved to write code that was clever, that would catch our eye on first read and recognize his creativity and skill. But we soon learned that much of this code was inscrutable in a day or two, which made modifying and maintaining his modules impossible. Design and code that makes a module easy to read and understand in a few weeks are what the team needed. Sometimes, the cleverness of a solution shone through then, too, but it impressed us all the more when it had "staying power".

Steve Martin is wacky comedian, not a programmer or teacher per se. Does his advice apply only in that context? I don't think so. Comedians are writers and creators, and many of the traits that make them successful apply to other tasks that require creating and communicating.

Readers of this blog know that programming is one of the topics I most like to write about. In recent months I've had something of a "programming for everyone" theme, with programming as a medium of expression, as a way to create new forms, ideas, and solutions. But programming is also for computer scientists, being the primary mode for communicating their ideas.