Knowing and Doing

... with folks in university administration, colleagues whom I respect and with whom I like to work.

[earlier]

Them: These reports can't be combined. Please make separate ones.

Me: They will be identical.

Them: That's okay.

Me: Fine.

[later]

Them: These reports are identical. That's a problem.

Me: That's what you asked me to do.

Them: They can't be identical. Please make them different.

Me: But the committee in charge told us long ago to combine the tasks, so we did. Most of the resulting document really is one item, so we have a common report.

Them: But the reports can't be the same. Please make them different.

Me: But they aren't different.

Them: If they aren't different, the board will be unhappy.

Me: But if I make reports that differ artificially, they'll see that, too. And if they differ only a little, then it will be hard to tell where they differ.

Them: If they aren't different, the board will be unhappy. Please make them different.

Me: But...

Them: Please.

~~~~~

This is the sort of situation that makes me wonder (1) if the board really understands how universities work and (2) what I am doing in The Office.

The verdict is in on the paper we wrote at ChiliPLoP and submitted to Onward!: rejected. (We are still waiting to hear back on our Educators' Symposium submission.) The reviews of our Onward! paper were mostly on mark, both on surface features (e.g., our list of references was weak) and on the deeper ideas we offer (e.g., questions about the history of studio approaches, and questions about how the costs will scale). We knew that this submission was risky; our time was simply too short to afford enough iterations and legwork to produce a good enough paper for Onward!.

I found it interesting that the most negative reviewer recommended the paper for acceptance. This reviewer was clearly engaged by the idea of our paper and ended up writing the most thorough, thoughtful review, challenging many of our assumptions along the way. I'd love to have the chance to engage this person in conversation at the conference. For now, I'll have to settle for pointing out some of the more colorful and interesting bits of the review.

In at least one regard, this reviewer holds the traditional view about university education. When it comes to the "significant body of knowledge that is more or less standard and that everyone in the field should acquire at some point in time", "the current lecture plus problem sets approach is a substantially more efficient and thorough way to do this."

Agreed. But isn't it more efficient to give the students a book to read? A full prof or even a TA standing in a big room is an expensive way to demonstrate standard bodies of knowledge. Lecture made more sense when books and other written material were scarce and expensive. Most evidence on learning is that lecture is actually much less effective than we professors (and the students who do well in lecture courses) tend to think.

The reviewer does offer one alternative to lecture: "setting up a competition based on mastery of these skills". Actually, this approach is consistent with the spirit of our paper's studio-based, apprenticeship-based, and project-based. Small teams working to improve their skills in order to win a competition could well inhabit the studio. Our paper tended to overemphasize the softer collaboration of an idyllic large-scale team.

This comment fascinated me:

Another issue is that this approach, in comparison with standard approaches, emphasizes work over thinking. In comparison with doing, for example, graph theory or computational complexity proofs, software development has a much lower ratio of thought to work. An undergraduate education should maximize this ratio.

Because I write a blog called Knowing and Doing, you might imagine that I think highly of the interplay between working and thinking. The reviewer has a point: an education centered on projects in a studio must be certain to engage students with the deep theoretical material of the discipline, because it is that material which provides the foundation for everything we do and which enables us to do and create new things. I am skeptical of the notion that an undergrad education should maximize the ratio of thinking to doing, because thinking unfettered by doing tends to drift off into an ether of unreality. However, I do agree that we must try to achieve an appropriate balance between thinking and doing, and that a project-based approach will tend to list toward doing.

One comment by the reviewer reveals that he or she is a researcher, not a practitioner:

In my undergraduate education I tried to avoid any course that involved significant software development (once I had obtained a basic mastery of programming). I believe this is generally appropriate for undergraduates.

Imagine the product of an English department saying, "In my undergraduate education I tried to avoid any course that involved significant composition (once I had obtained a basic mastery of grammar and syntax). I believe this is generally appropriate for undergraduates." I doubt this person would make much of a writer. He or she might be well prepared, though, to teach lit-crit theory at a university.

Most of my students go into industry, and I encourage them to take as many courses as they can in which they will build serious pieces of software with intellectual content. The mixture of thinking and doing stretches them and keeps them honest.

An education system that produces both practitioners and theoreticians must walk a strange line. One of the goals of our paper was to argue that a studio approach could do a better job of producing both researchers and practitioners than our current system, which often seems to do only a middling job by trying to cater to both audiences.

I agree wholeheartedly, though, with this observation:

A great strength of the American system is that it keeps people's options open until very late, maximizing the ability of society to recognize and obtain the benefits of placing able people in positions where they can be maximally productive. In my view this is worth the lack of focus.

My colleagues and I need to sharpen our focus so that we can communicate more effectively the notion that a system based on apprenticeship and projects in a studio can, in fact, help learners develop as researchers and as practitioners better than a traditional classroom approach.

The reviewer's closing comment expresses rather starkly the challenge we face in advocating a new approach to undergraduate education:

In summary, the paper advocates a return to an archaic system that was abandoned in the sciences for good reason, namely the inefficiency and ineffectiveness of the advocated system in transmitting the required basic foundational information to people entering the field. The write-up itself reflects naive assumptions about the group and individual dynamics that are required to make the approach succeed. I would support some of the proposed activities as part of an undergraduate education, but not as the primary approach.

The fact that so many university educators and graduates believe our current system exists in its current form because it is more efficient and effective than the alternatives -- and that it was designed intentionally for these reasons -- is a substantial cultural obstacle to any reform. Such is the challenge. We owe this reviewer our gratitude for laying out the issues so well.

In closing, I can't resist quoting one last passage from this review, for my friends in the other sciences:

The problem with putting students with no mastery of the basics into an apprenticeship position is that, at least in computer science, they are largely useless. (This is less true in sciences such as biology and chemistry, which involve shallower ideas and more menial activities. But even in these sciences, it is more efficient to teach students the basics outside of an apprenticeship situation.)

The serious truth behind this comment is the one that explains why building an effective computer science research program around undergraduates can be so difficult. The jocular truth behind it is that, well, CS is just plain deeper and harder! (I'll duck now.)

Not from me, but from The Common School Journal, an early-1800s primer on teaching:

Make no effort to simplify language.

Also: Treat all students as equals, with expectation that all participate and learn. Teach where each student lacks.

I think I have a more detailed essay on this topic in me, in terms of how we teach programming and software development. But it will have to wait for another day. In any case, my agedness seems to be growing asymptotically faster than my wisdom.

I've tried to explain the idea of software patterns in a lot of different ways, to a lot of different kinds of people. Reading James Tauber's Grammar Rules reminds me of one of my favorites: a pattern language is a descriptive grammar. Patterns describe how (good) programmers "really speak" when they are working in the trenches.

Talking about patterns as grammar creates the potential for the sort of misunderstanding that Tauber discusses in his entry. Many people, including many linguists, think of grammar rules as, well, rules. I was taught to "follow the rules" in school and came to think of the rules as beyond human control. Linguists know that the rules of grammar are man-made, yet some still seem to view them as prescriptive:

It is as if these people are viewing rules of grammar like they would road rules--human inventions that one may disagree with, but which are still, in some sense, what is "correct"...

Software patterns are rarely prescriptive in this sense. They describe a construct that programmers use in a particular context to balance the forces at play in the problem. Over time, they have been found useful and so recur in similar contexts. But if a programmer decides not to use a pattern in a situation where it seems to apply, the programmer isn't "wrong" in any absolute sense. But he'll have to resolve the competing forces in some other way.

While the programmer isn't wrong, other programmers might look at him (or, more accurately, his program) funny. They will probably ask "why did you do it that way?", hoping to learn something knew, or at least confirm that the programmer has done something oddly.

This is similar to how human grammar works. If I say, "Me wrote this blog", you would be justified in looking at me funny. You'd probably think that what I speaking incorrectly.

Tauber points out that, while I might be violating the accepted rules of grammar, I'm not wrong in any absolute sense:

... most linguists focus on modeling the tacit intuitions native speakers have about their language, which are very often at odds with the "rules of grammar" learnt at school.

He gives a couple of examples of rules that we hear broken all of the time. For example, native speakers of English almost always say "It's me", not "It's I", though that violates the rules of nominative and accusative case. Are we all wrong? In Sr. Jeanne's 7th-grade English class, perhaps. But English grammar didn't fall from the heavens as incontrovertible rules; it was created by humans as a description of accepted forms of speech.

When a programmer chooses not to use a pattern, other programmers are justified in taking a second look at the program and asking "why?", but they can't really say he's guilty of anything more than doing things differently.

Like grammar rules, some patterns are more "right" than others, in the sense that it's less acceptable to break some than others. I can get away with "It's me", even in more formal settings, but I cannot get away with "Me wrote this blog", even in the most informal settings. An OO programmer might be able get away with not using the Chain of Responsibility pattern in a context where it applies, but not using Strategy or Composite in appropriate contexts just makes him look uninformed, or uneducated.

A few more thoughts:

So, patterns are not like a grammar for programming language, which is prescriptive. To speak Java at all, you have to follow the rules. They are like the grammar of a human language, which model observations about how people speak in the wild.

As a tool for teaching and learning, patterns are so useful precisely because they give us a way to learn accepted usages that go beyond the surface syntactic rules of a language. Even better, the pattern form emphasizes documenting when a construct works and why. Patterns are better than English grammar in this regard, at least better than the way English grammar is typically taught to us as schoolchildren.

There are certainly programmers, software engineers, and programming language theorists who want to tell us how to program, to define prescriptive rules. There can be value in this approach. We can often learn something from a model that has been designed based on theory and experience. But to me prescriptive models for programming are most useful when we don't feel like we have to follow them to the letter! I want to be able to learn something new and then figure out how I can use it to become a better programmer, not a programmer of the model's kind.

But there is also a huge, untapped resource in writing the descriptive grammar of how software is built in practice. It is awfully useful to know what real people do -- smart, creative people; programmers solving real problems under real constraints. We don't understand programming or software development well enough yet not to seek out the lessons learned by folks working in the trenches.

This brings to mind a colorful image, of software linguists venturing into the thick rain forest of a programming ecosystem, uncovering heretofore unexplored grammars and cultures. This may not seem as exotic as studying the Pirahã, but we never know when some remote programming tribe might upend our understanding of programming...

Brian Marick lamented recently that his daughter's homework probably wasn't affecting her future in the same way that some of his school experiences affected his. I've had that feeling, too, but sometimes wonder whether (1) my memory is good enough to draw such conclusions and (2) my daughters will remember key experiences from their school days anyway. After teaching for all these years I am sometimes surprised by what former students remember from their time in my courses, and how those memories affect them.

Brian's mention of New Math elicited some interesting comments. Kevin Lawrence hit on a point that has been on my mind in two contexts lately:

A big decision point in education is whether you are optimizing for people who will go on to be very good at a subject or for people who find it difficult.

In the context of university CS curricula, I often field complaints from colleagues here and everywhere about how the use of (graphics | games | anything post-1980 | non-scientific applications) in CS courses is dumbing down of our curriculum. These folks claim that we are spending too much time catering to folks who won't succeed in the discipline, or at least excel, and that at the same time we drive away the folks who would be good at CS but dislike the "softness" of the new approach.

In the context of reaching out to pre-university students, to show folks cool and glitzy things that they might do in computer science, I sometimes hear the same sort of thing. Be careful, folks say, not to popularize the science too much. We might mislead students into thinking that CS is not serious, or that it is easy.

I fully agree that we don't want to mislead middle schoolers or CS majors about the content or rigor of our discipline, or to give the impression that we cannot do serious and important work. But physics students and math geeks are not the only folks who can or should use computing. They are most definitely not the only folks who can make vital contributions to the discipline. (We can even learn from people who quote "King Lear".)

By not reaching out to students with different views and interests, we do computer science a disservice. Once they are attracted to the discipline and excited to learn, we can teach them all about rigor and science and math. Some of those folks won't succeed in CS, but then again neither do some of the folks who come in with the more traditional "geeky" interests.

If this topic interests you, follow the trail from Brian's blog to two blog entries by Kevin Lawrence, one old and one new. Both are worth a read. (I always knew there was a really good reason to enable comments on my blog -- Alan Kay might drop by!)

I may not be a web guy, but some of my students are -- and very good ones. Back in December, I wrote about one of my students, Sergei Golitsinski, defending an MA thesis in Communications, which used computing to elucidate a problem in that discipline. For that study, he wrote tools that allowed him to trace the threads of influence in a prominent blog-driven controversy.

Sergei finally defended his MS thesis in computer science yesterday. Its title -- "Specification and Automatic Code Generation of the Data Layer for Data-Intensive Web-Based Applications" -- sounds like the usual thesis title, but as is often the case the idea behind it is really quite accessible. This thesis shows how you can use knowledge about your web application to generate much of the code you need for your site.

I like this work for several reasons. First, it was all about finding patterns in real applications and using them to inform software development. Second, it focused on how to use domain knowledge to get leverage from the patterns. Third, it used standard language-processing ideas to create a modeling language and then use models written in it to generate code. This thesis demonstrates how several areas of computer science -- database, information storage and retrieval, and programming languages among them -- can work together to help us write programs to do work for us. I also like it because Sergei applied his ideas to his own professional work and took a critical look at what the outcome means for his own practice.

Listening to the defense, I had two favorite phrases. The first was recursive weakness. He used this term in reference to weak entities in a database that are themselves parents to weak entities. But it brought to mind so many images for the functional programmer in me. (I'm almost certainly recursively weak myself, but where is the base case?) The second arose arose while discussing alternative approaches to a particular problem. Referring to one, he said trivial approach; non-trivial implementation. It occurred to me that so many ideas fall into this category, and part of understanding your domain well is recognizing them. Sometimes we need to avoid their black holes; other times, we need their challenges. Another big part of becoming a master is knowing which path to choose once you have recognized them.

Sergei is a master, and soon he will have a CS degree that says so. But like all masters, he has much to learn. When I wrote about his previous defense, his plan was up in the air but pointing toward applying CS in the world of communications. Since then, he has accepted admission to a Ph.D. program in communications at the University of Maryland, where he hopes to be in the vanguard of a new discipline he calls computational communications. I look forward to watching his progress.

You can read his CS thesis on-line, and all of the code used in his study will be, too, soon.

I am not a web guy. If I intend to teach languages (PHP) or frameworks (Rails) with the web as their natural home, I need to do a lot more practice myself. It's too easy to know how to do something and still not know it well enough to teach it well. Unexpected results under time pressure create too much trouble.

MySQL, no. PostgreSQL, yes. This, twenty years after cutting my database teeth on Ingres.

Ruby's dynamic features are so, so nice. Still, I occasionally find myself wishing for Smalltalk. First love never dies.

Fifteen hours of instruction -- 5 weeks at 3 hours per week -- is plenty of time to teach most or all of the ideas in a language like bash, PHP, or Ruby. But the instructor still needs to select specific examples and parts of the class library carefully. It's too easy to start down a path of "Now look at this [class, method, primitive]..."

When I succeeded in selecting carefully, I suffered from persistent omitter's resource. "But I wish I could have covered that... Sometimes that is what students wanted to see. But most of the time they can figure that out. What they want is some insight. What insight could I have shared had I covered that instead of this?

If you want to know what students want, ask them. Easy to say, hard to do unless I slow down occasionally to reflect.

Practice, practice, practice. That's where students learn. It's also where students who don't learn don't learn.

Oh, and professor: That "Practice, practice, practice" thing -- it applies to you, too. You'll remember just how much fun programming can be.

I remember learning in courses on simulation, operating systems, and networking that, for a given period, the number of events such as cars arriving at an intersection or processes arriving at a scheduler is often best modeled using the Poisson distribution. Mostly, I recall being surprised that these events often occur in clumps, rather than uniformly distributed over a larger time period. Sometimes, it feels like ideas work this way... When I encounter an idea once during the day, I often seem to bump into it again and again. I'm sure that it's just that my mind is sensitized to the idea and recognize -- or project -- it more easily, much as magic books affect us. In any case, yesterday was such a day.

At 3:30 PM I attended a department seminar on bioinformatics by a colleague. I asked him what sort of questions he and his students could ask about bacteriophages in a data-rich environment that they could not ask before. He said that they could now quantify the notions of similarity and difference between phages in ways inaccessible to them before and write programs to apply their metrics. Eventually, he talked about how digital processing of large data sets enforced a more disciplined approach on the approach to problems, in order to battle complexity. Now, they convert big questions into a sequence of smaller, well-defined steps that can be tackled in a clear way. For him as a biologist, this was a surprising and wonderful phenomenon.

I stayed in the same room for a 5:00 PM class taught by one of our adjuncts, whose teaching I was to evaluate. He was teaching a "skills and concepts" course for non-majors, and the day's topic was databases. They talked about the similarities and differences between spreadsheets and databases, especially on how the structural integrity of a database makes it possible to formulate concise queries that can find useful answers. He some of the ideas using an Access database, first using a wizard to query the system and then looking at a raw SQL query. For many queries, he told them, the wizard does all you need. But there will times when you want to ask a question the wizard doesn't support, and then the ability to write your own select statements in SQL becomes a valuable skill.

After class, I caught upon some paperwork in my office until 7:00 PM, when I attended a panel presentation entitled "Visual Art, the Big Screen, and Orchestral Performance". (Here is a poster for the talk, in PDF.) Three local artists -- illustrator Gary Kelley, conductor Jason Weinberger, and videographer Scott Smith -- shared parts of their recent multimedia presentation of Gustav Holst's The Planets and discussed the creative forces that drove them individually and collectively to produce the work. I learned that multimedia presentations of The Planets are relatively common but that this show differed in significant ways from the usual, not the least of which was Kelley's creation of thirty new paintings and monotypes for the show.

(You may recall Smith's name from an earlier post... He had a small acting role in the play I did last winter!)

The panel ended with a discussion of how changes in technology were fundamentally changing how artist work are created and distributed. Not long ago, Hollywood and other media centers produced the entertainment that we all consumed, but now it is possible for folks in the middle of nowhere -- Iowa! -- to create and export their work to a global audience. This is, of course, nothing new in the age of the Internet and YouTube, but it is still cause for marvel to artists who recently lived and worked in a different world.

One of the central themes of the panel was the level of trust and surrender that this kind of presentation required, especially of the symphony members and conductor Weinberger. The timing of the video required the orchestra to hit certain marks in the music on a dot, and Weinberger, who usually controls tempo and shapes the sound of the performance, had to give up control the artwork produced by Kelley and Smith. The visual artists expressed a willingness to turn the tables and find a way to cede control to Weinberger in a future collaboration.

This set me to thinking... The reason that the musicians had to surrender control was essentially technological. Once a video is produced, it is set. Performance of the music was the more malleable medium, as the players could speed up or slow down in real-time to stay in sync. Ideally, of course, they would play a steady predefined pace, but that is quite difficult. But these days, "video" is much more malleable because it is digital. Why not let the musicians play however they and the conductor see fit, and adjust the pace of the video playback to keep in sync with the music? I don't know if such a digital tool exists already, but if not, what fun it would be to write! Then in performance, the videographer could "play" the video by reacting in real-time to the music.

All three of these stories had me thinking the same thing: "Now there's programming." I know the feeling well the feeling my biologist colleague expressed, because both of his answers come down to programming as discipline and medium. When our adjunct instructor told his non-CS students from all over campus about the power of knowing a little SQL, I smiled at the thought of non-programmers writing programs, albeit small ones, to scratch their own itches. Likewise, the ability to imagine how the orchestra might turn the tables on the visual artists in their multimedia collaborations, and then implement the vision in a working tool, is nothing more or less than programming.

The Small Doses pattern I wrote up in my previous entry was triggered almost exclusively by the story I heard from Carl Page. The trigger lives on in the text that runs from "Often times, the value of Small Doses..." to the end, and in the paragraph beginning "There is value in distributing...". The story was light and humorous, just the sort of story that will stick with a person for twenty or more years.

As I finally wrote the pattern, it grew. That happens all the time when I write. It grew both in size and in seriousness. At first I resisted getting too serious, but increasingly I realized that the more serious kernel of truth needed telling. So I gave it a shot.

The result of this change in tone and scope means that the pattern you read is not yet ready for prime time. Rather than wait until it was ready, though, I decided to let the pattern be a self-illustration. I have put it out now, in its rough form. It is rough both in completeness and in quality. Perhaps my readers will help me improve. Perhaps I will have time and inspiration soon to tackle the next version.

In my fantasies, I have time to write more patterns in a Graduate Student pattern language (code name: Chrysalis), even a complete language, and cross-reference it with other pattern languages such as XP. Fantasies are what they are.

Also Known As    Frequent Releases

From Pattern Language    Graduate Student

You are writing a large document that one or more other people must read and provide feedback on before it is distributed externally.

The archetype is a master's thesis or doctoral dissertation, which must be approved by a faculty committee.

You want to give your reviewers the best document possible. You want them to give you feedback and feel good about approving the document for external distribution.

You want to publish high-quality work. You would also like to have your reviewers see only your best work, for a variety of reasons. First, you respect them and are thankful for their willingness to help you. Second, they often have the ability to help you further in the future, in the form of jobs or recommendations. Good work will impress your reviewers more than weaker work. A more complete and mature document is more likely to resemble the final result than a rougher, earlier version.

In order to produce a document of the highest quality, you need time -- time to research, write, and revise. This delays your opportunity to distribute it to your reviewers. It also delays their opportunity to give you feedback.

Your reviewers are busy. They have their own work to do and are often volunteering their time to serve you.

Big tasks require a lot of time. If a task takes a lot of time to do, some people face a psychological barrier to starting the task.

A big task decomposed into several smaller tasks may take as much time as the big task, or more, but each task takes a smaller time. It is often easier to find time in small chunks, which makes it easier to start on the task in the first place.

The sooner your reviewers are able to read parts of the document, the sooner they will be able to give you feedback. This feedback helps you to improve the document, both in the large (topic, organization) and the small (examples, voice, illustrations, and so on).

Therefore:

Distribute your document to reviewers periodically over a relatively drawn out period. Early versions can be complete parts of the document, or rough versions of the entire document.

In the archetypal thesis, you might give the reviewers one chapter per week, or you might give a whole thesis in which each part is in various stages of completeness.

There is certainly a trade-off between the quality of a document and the timeliness of delivery. Don't worry; this is just a draft. You are always free to improve and extend your work. Keep in mind that there is also a trade-off between the quality of a document and the amount of useful feedback you are able to incorporate.

There is value in distributing even a very rough or incomplete document at regular intervals. If reviewers read the relatively weak version and make suggestions, they will feel valuable. If they don't read it, they won't know or mind that you have made changes in later versions. Furthermore, they may feel wise for not having wasted their time on the earlier draft!

With the widespread availability of networks, we can give our reviewers real-time access to an evolving document in the form of an-line repository. In such a case, Small Doses may take the form of comments recorded as each change is committed to the repository. It is often better for your reviewers if you give them periodic descriptions of changes made to the document, so that they don't have to wade through minutiae and can focus on discrete meaningful jumps in the document.

I have seen Small Doses work effectively in a variety of academic contexts, from graduate students writing theses to instructors writing lecture notes and textbooks for students. I've seen it work for master's students and doctoral students, for text and code and web sites. (If you know of particularly salient examples of this pattern, I'd love to hear them.)

Often times, the value of Small Doses is demonstrated best in the results of its applying its antipattern, Avalanche. Suppose you are nearing a landmark date, say, the deadline for graduation or the end of spring semester, when faculty will scatter for the summer. You dump a 50-, 70-, or 100+-page document on your thesis committee. In the busy time before the magic date, committee members have a difficulty finding time to read the whole document. Naturally, they put off reading it. Soon they fall way behind and have a hard time meeting the deadline -- and they blame you for the predicament, for unrealistic expectations. Of course, had you given the committee a chapter at a time for 5 or 6 weeks leading up to the magic date, some committee members would still have fallen behind reading it, because they are distracted with their own work. But then you could blame the them for not getting done!

Related Patterns    Extreme Programming and other agile software methodologies encourage Short Iterations and Frequent Releases. Such frequent releases are the Small Doses of the software development world. They enable more continuous feedback and allow the programmers to improve the quality of the code based on what they learn. The Avalanche antipattern is the source of many woes in the world of software, in large part due to the lack of feedback they afford from users and clients.

I learned the Small Doses pattern from Carl Page, my Artificial Intelligence II professor at Michigan State University in 1987. Professor Page was a bright guy and very good computer scientist, with an often dark sense of humor. He mentored his students and advisees with sardonic stories like Small Doses. He was also father to another famous Page, Larry.