A student stopped in for a chat late last week to discuss the code he was writing for a Programming Languages assignment. This was the sort of visit a professor enjoys most. The student had clearly put in plenty of time on his interpreter and had studied the code we had built in class. His code already worked. He wanted to talk about ways to make his code better.
Some students never reach this point before graduation. In Coders at Work, Bernie Cosell tells a story about leading teams of new hires at BBN:
I would get people -- bright, really good people, right out of college, tops of their classes -- on one of my projects. And they would know all about programming and I would give them some piece of the project to work on. And we would start crossing swords at our project-review meetings. They would say, "Why are you complaining about the fact that I have my global variables here, that I'm not doing this, that you don't like the way the subroutines are laid out? The program works."
They'd be stunned when I tell them, "I don't care that the program works. The fact that you're working here at all means that I expect you to be able to write programs that work. Writing programs that work is a skilled craft and you're good at it. Now, you have to learn how to program.
I always feel that we have done our students well if we can get them to the point of caring about their craft before they leave us. Some students come to us already having this mindset, which makes for a very different undergraduate experience. Professors enjoy working these students, too.
But what stood out to me most from this particular conversation was something the student said, something to this effect:
When we built the lexical addresser in class a few weeks ago, I didn't understand the idea and I couldn't write it. So I studied it over and over until I could write it myself and understand exactly why it worked. We haven't looked at lexical addressing since then, but the work I did has paid off every time we've written code to process programs in our little languages, including this assignment. And I write code more quickly on the exams now, too.
When he finished speaking, I could hardly contain myself. I wish I could bottle this attitude and give to every student who ever thinks that easy material is an opportunity to take it easy in a course for a while. Or who thinks that the best response to difficult material is to wait for something easier to come along next chapter.
Both situations are opportunities to invest energy in the course. The returns on investment are deeper understanding of the material, sharper programming skills, and the ability to get stuff done.
This student is reaping now the benefits of an investment he made five weeks ago. It's a gift that will keep on giving long after this course is over.
I encourage students to approach their courses and jobs in this way, but the message doesn't always stick. As Clay Stone from City Slickers might say, I'm happy as a puppy with two peters whenever it does.
While walking this morning, I coined a word for this effect: exceleration. It's a portmanteau combining "excellence" and "acceleration", which fits this phenomenon well. As with compound interest and reinvested dividends, this sort of investment builds on its self over time. It accelerates learners on their path to mastering their craft.
Whatever you call it, that conversation made my week.
Yesterday, I tweeted absent-mindedly:
It would be cool to teach a course called "Reading Code".
Reading code has been on mind for a few months now, as I've watched my students read relatively small pieces of code in my Programming Languages course and as I've read a couple of small libraries while reading the exercise bike. Then I ran across John Regehr's short brainstorm on the topic, and something clicked. So I tweeted.
Reading code, or learning to do it, must be on the minds of a lot people, because my tweet elicited quite a few questions and suggestions. It is an under-appreciated skill. Computer science programs rarely teach students how to do it, and then usually only implicitly, by hearing a prof or other students talk about code they've read.
Several readers wanted to know what the course outline would be. I don't know. That's one of the things about Twitter or even a blog: it is easy to think out loud absent-mindedly without having much content in mind yet. It's also easier to express an interest in teaching a course than to design a good one.
Right now, I have only a few ideas about how I'd start. Several readers suggested Code Reading by Spinellis, which is the only textbook I know on then topic. It may be getting a little old these days, but many of the core techniques are still sound.
I was especially pleased that someone recommended Richard Gabriel's idea for an MFA in Software, in which reading plays a big role. I've used some of Dick's ideas in my courses before. Ironically, the last time I mentioned the MFA in Software idea in my blog was in the context of a "writing code" course, at the beginning of a previous iteration of Programming Languages!
That's particularly funny to me because someone replied to my tweet about teaching a course called "Reading Code" with:
... followed by a course "Writing Readable Code".
Anyone who has tried to grade thirty evolving language interpreters each week appreciates this under-appreciated skill.
Chris Demwell responded to my initial tweet with direct encouragement: Write the course, or at least an outline, and post it. I begged indulgence for lack of time as the school year ends and said that maybe I can take a stab this summer. Chris's next tweet attempted to pull me into the 2010s:
1. Write an outline. 2. Post on github. 3. Accept pull requests. Congrats, you're an editor!
The world has indeed changed. This I will do. Watch for more soon. In the meantime, feel free to e-mail me your suggestions. (That's an Old School pull request.)
In this interview with The Setup, Patric King talks about his current work:
Right now, my focus is bringing well-designed marketing to industries I believe in, to help them develop more financing. ... It is not glamorous, but it is the right thing to do. Designing pretty things is nice, but it's time for me to do something bigger than me.
That's a pretty good position to be in: bringing value to a company or industry you believe in. Sometimes, we find such positions by virtue of the career path we choose. Those of us who teach as a part of our jobs are lucky in this regard.
Other times, we have to make a conscious decision to seek positions of this sort, or create the company we want to be in. That's what King has done. His skill set gives him more latitude than many people have. Those of us who can create software have more freedom than most other people, too. What an opportunity.
King's ellipsis is filled with the work that matters to him. As much as possible, when the time is right, we all should find the work that replaces our own ellipses with something that really matters to us, and to the world.
This morning, Maciej Ceglowski of Pinboard introduced me to the Matasano crypto challenges, a set of exercises created by Thomas Ptacek and his team as a tool for teaching programmers a little about cryptography, some of its challenges, and the need for more awareness of how easy it is to do it wrong. With the coming break from the grind of the academic year, I plan on giving them a try.
After having completed the exercises himself, Ceglowski observes:
Crypto is like catnip for programmers. It is hard to keep us away from it, because it's challenging and fun to play with. And programmers respond very badly to the insinuation that they're not clever enough to do something. We see the F-16 just sitting there, keys in the ignition, no one watching, lights blinking, ladder extended. And some infosec nerd is telling us we can't climb in there, even though we just want to taxi around a little and we've totally read the manual.
I've noticed this with a number of topics in computing. In addition to cryptography, data compression and sorting/searching are sirens to the best programmers among our students. "What do you mean we can't do better?"
For many undergrads, the idea of writing a compiler seems a mystery. Heck, I admit to my students that even after years of teaching the course I remain in awe of my language tools and the people who build them. This challenge keeps a steady if relatively small stream of programmers flowing into our "Translation of Programming Languages" project course.
One of the great things about all these challenges is that after we tackle them, we have not only the finished product in hand but also what we learn about the topic -- and ourselves -- along the way. Then we are ready for a bigger challenge, and another program to write.
For CS faculty, catnip topics are invaluable ways to draw more students into the spell of computing, and more deeply. We are always on the lookout.
A friend put a copy of GameInformer magazine in my box yesterday with a pointer to an interview with the Great and Powerful Woz, Steve Wozniak. It's a short interview, only two pages, but it reminded me just how many cool things Wozniak (and so many others) did in the mid-1970s. It also reminded me of my younger days, coming into contact with the idea of games and machine learning for the first time.
Woz described how, after seeing Pong in a video arcade, he went home and built his own Pong game out of twenty-eight $1 chips. Steve Jobs took the game to Atari, where he encountered Nolan Bushnell, who had an idea for a single-player version of Pong. Thus did Woz design Breakout, a game with an especially apt name. It helped define Apple Computer.
The thought of building a computer game out of chips still amazes me. I was never a hardware guy growing up. I never had access to computer chips or that culture, and I had little inclination to fiddle with electronics, save for a few attempts to take apart radios and put them back together. When I designed things as a kid, they were houses or office buildings. I was going to be an architect. But Woz's story reminded me of one experience that foreshadowed my career as a computer scientist.
One year in school, I won a math contest. First prize was a copy of The Unexpected Hanging and Other Mathematical Diversions, a collection of Martin Gardner's columns from Scientific American. Chapter 8 was called "A Matchbox Game-Learning Machine". It described Hexapawn, a game played on a 3x3 board with chess pawns. The game was no more complex than Tic Tac Toe, but it was new. And I loved board games.
Gardner's article had more in store for me, though, than simply another game to study. He described how to create a "computer" -- a system of matchboxes -- that learns how to play the game! Here's how:
You make one box for each possible board position. In the box, you put different colored marbles corresponding to the moves that can be played in the position. Then you play a bunch of games against the matchbox computer. When it is the computer's turn to move, you pick up the box for that board position, shake it, and see which marble lands in the lower-right corner of the box. That's the computer's move.
When the game is over, the computer gets feedback. If it won the game, then put all the marbles back in their boxes. If it lost, punish it by keeping the marble responsible for its last move; put all the rest back in their boxes. Gardner claimed that by following this strategy, the matchbox computer would learn to play a perfect game in something under fifty moves.
This can't possibly work, can it? So I built it. And it did learn. I was happy, and amazed.
I remember experimenting a bit. Maybe a move wasn't always a loser? So I seeded the computer with more than one marble for each candidate move, so that the computer could overcome bad luck. Hexapawn is so simple that this wasn't necessary -- losing moves are losing moves -- but the computer still learned to play a perfect game, just a bit slower than before.
This is one of the earliest experiences I remember that started me down the road of studying artificial intelligence. Reading copious amounts of science fiction pushed me in that direction, too, but this was different. I had made something, and it learned. I was hooked.
So, I wasn't a hardware kid, but I had a hardware experience. It just wasn't digital hardware. But my inclination was always more toward ideas than gadgets. My interests quickly turned to writing programs, which made it so much easier to tinker with variations and to try brand-new ideas.
(Not so quickly, though, that I turned away from my dream of being an architect. The time I spent in college studying architecture turned out to be valuable in many ways.)
Wozniak was a hardware guy, but he quickly saw the potential of software. "Games were not yet software, and [the rise of the microprocessor] triggered in my mind: microprocessors can actually program games." He called the BASIC interpreter he wrote "Game BASIC". Ever the engineer, he designed the Apple II with integrated hardware and software so that programmers could write cool games.
I don't have a lot in common with Steve Wozniak, but one thing we share is the fun we have playing games. And, in very different ways, we once made computers that changed our lives.
The GameInformer interview is on-line for subscribers only, but there is a cool video of Wozniak playing Tetris -- and talking about George H.W. Bush and Mikhail Gorbachev!
Watson and Crick announced their discovery of the double helix structure of DNA in Molecular Structure of Nucleic Acids, a marvel of concise science writing. It has been widely extolled for how much information it packs into a single page, including the wonderfully understated line, "It has not escaped our notice that the specific pairing we have postulated immediately suggests a possible copying mechanism for the genetic material."
As I read this paper again recently, though, this passage stood out:
The previously published X-ray data on deoxyribose nucleic acid are insufficient for a rigorous test of our structure. So far as we can tell, it is roughly compatible with the experimental data, but it must be regarded as unproved until it has been checked against more exact results.
They are unpretentious sentences. They do nothing special, stating simply that more and better data are needed to test their hypothesis. This is not a time for hyperbole. It is a time to get back to the lab.
Just scientists being scientists.
First, there was this recent piece in the New York Times about software that grades essays. Such software is probably essential for MOOCs in many disciplines, but it would also be useful in large lecture sections of traditional courses at many universities. The software isn't perfect, and skeptics abound. But the creator of the EdX software discussed in the article says:
This is machine learning and there is a long way to go, but it's good enough and the upside is huge.
It's good enough, and the upside is huge. Entrenched players scoff. Classic disruption at work.
Then there was this piece from the Nieman Journalism Lab about an online Dutch news company that wants readers to subscribe to individual journalists. Is this really news in 2013? I read a lot of technical and non-technical material these days via RSS feeds from individual journalists and bloggers. Of course, that's not the model yet for traditional newspapers and magazines.
... but that's the news business. What about the revolution in universities? The Nieman Lab piece reminded me of an old article in Vanity Fair about Politico, a news site founded by a small group of well-known political journalists who left their traditional employers to start the company. They all had strong "personal brands" and journalistic credentials. Their readers followed them to their new medium. Which got me to thinking...
What would happen if the top 10% of the teachers at Stanford or Harvard or Williams College just walked out to start their own university?
Of course, in the time since that article was published, we have seen something akin to this, with the spin-off of companies like Coursera and Udacity. However, these new education companies are partnering with traditional universities and building off the brands of their partners. At this point in time, the brand of a great school still trumps the individual brands of most all its faculty. But one can imagine a bolder break from tradition.
What happens when technology gives a platform to a new kind of teacher who bypasses the academic mainstream to create and grow a personal brand? What happens when this new kind of teacher bands together with a few like-minded renegades to use the same technology to scale up to the size of a traditional university, or more?
That will never happen, or so many of us in the academy are saying. This sort of thinking is what makes the Dutch news company mentioned above seem like such a novelty in the world of journalism. Many journalists and media companies, though, now recognize the change that has happened around them.
Which leads to a final piece I read this morning, a short blog entry by Dave Winer about Ezra Klein's epiphany on how blogging and journalism are now part of a single fabric. Winer says:
It's tragic that it took a smart guy like Klein so long to understand such a basic structural truth about how news, his own profession, has been working for the last 15 years.
I hope we aren't saying the same thing about the majority of university professors fifteen or twenty years from now. As we see in computers that grade essays, sometimes a new idea is good enough, and the upside is huge. More and more people will experiment with good-enough ideas, and even ideas that aren't good enough yet, and as they do the chance of someone riding the upside of the wave to something really different increases. I don't think MOOCs are a long-term answer to any particular educational problem now or in the future, but they are one of the laboratories in which these experiments can be played out.
I also hope that fifteen or twenty years from now someone isn't saying about skeptical university professors what Winer says so colorfully about journalists skeptical of the revolution that has redefined their discipline while they worked in it:
The arrogance is impressive, but they're still wrong.
I've always liked this quote from the preface of Pragmatic Ajax, by Gehtland, Galbraith, and Almaer:
Writing a book is a lot like (we imagine) flying a spaceship too close to a black hole. One second you're thinking "Hey, there's something interesting over there," and a picosecond later, everything you know and love has been sucked inside and crushed.
Programming can be like that, too, in a good way. Just be sure to exit the black hole on the other side.
This week I bumped into a familiar tension that arises whenever I assign a new homework problem to students.
Path 1. I assign a new problem before I solve it myself. The problem turns out to too difficult, or includes a complication or two that I didn't anticipate. Students become frustrated, especially the weaker ones. Because of the distraction, most everyone misses the goal of the assignment.
Path 2. I solve a new problem before I assign it. I run into a couple of unexpected wrinkles, things that make the problem less straightforward than I had planned. "This will distract my students," I think, so I iron out a wrinkle here and eliminate a distraction there. Then I assign the problem to my students. The result feels antiseptic, unchallenging. Some students are bored with the problem, especially the stronger ones.
In this week's case, I followed the second path. I assigned a new problem in my Programming Languages course before solving it myself. When I sat down to whip up my solutions, I realized the problem held a couple of surprises for me. I had a lot of fun with those surprises and was happy with the code that resulted. But I also realized that my students will have to do more fancy thinking than I had planned on. Nothing too tricky, just a couple of non-obvious steps along the way to an obvious solution.
Will that defeat the point of assigning the problem? Will my stronger students be happy for the challenge? Will my weaker students be frustrated, or angry at their inability to solve a problem that looks so much like another they have already solved? Will the problem help students learn something new about the topic at hand?
I ultimately believe that students benefit strongly from the challenge of problems that have not been pre-digested for them by the instructor or textbook. When we take too much care in preparing assignments ahead of time, we rob students of the joy of solving a problem that has rough edges.
Joy, and the sense of accomplishment that comes from taking on a real challenge. Problems in the world typically have rough edges, or throw wrinkles at us when we aren't looking for them.
Affording students the joy of solving a real problem is perhaps especially important for the stronger students, who often have to live with a course aimed at the middle of the curve. But it's just as important for the rest of the class. Skill and confidence grow out of doing something worth doing, even if it takes a little help from the professor.
I continue to use both approaches when I create assignments, sometimes solving a new problem first and sometimes trusting my instinct. The blend keeps assignments from veering too far in one direction or the other, I think, which gives students some balance.
However, I am usually most happy when I let a new problem surprise us all. I try to keep these problems on track by paying closer attention to students as they begin to work on them. When we run into an unexpected rough edge, I try to intervene with just enough assistance to get them over the complexity, but not so much as to sterilize the problem.
Finding the right balance between too clean and too rough is a tough problem for a teacher to solve. It is a problem worth solving, and a source of both disappointment and joy.
I teach compilers again this fall. I'm looking forward to summer, when I'll have a chance to write some code and play with some ideas for the course.
This morning I thought a bit about a topic that pops up every time I prep the course. The thoughts were prompted by a tweet from James Coglan, which said "Really wish this Compilers course weren't half about parsing. ... get on with semantics." The ellipsis is mine; James's tweet said something about using lex/yacc to get something up and running fast. Then, presumably, we could get on to the fun of semantics.
This is a challenge for my compilers course, too. I know I don't want to rush through scanning and parsing, yet I also wish I had more time for static analysis, optimization, and code generation. Even though I know the value of parsing, I wish I had equal time for a lot of other cool topics.
Geoff Wozniak's response expressed one of the reasons parsing still has such a large role in my compilers course, and so many others:
Parsing is like assembly language: it seems superfluous at the time, but provides deep understanding later. It's worth it.
That's part of what keeps me from de-emphasizing it in my course. Former students often report back to me that they have used their skill at writing parsers frequently in their careers, whether for parsing DSLs they whip up or for making sense of a mess of data they want to process.
A current student is doing an undergrad research project that involves finding patterns in several years of professional basketball play-by-play data, and his ability to parse several files of similar but just-different-enough data proved invaluable. Of course, he was a bit surprised that corralling the data took as much effort as it did. Kind of like how scanning and parsing are such a big part of a compiler project.
I see now that James has tweeted a retraction:
... ppl are RTing something I said about wishing the Compilers course would get done with parsing ASAP. Don't believe this any more.
I understand the change of opinion. After going writing a compiler for a big language and learning the intricacies that are possible, it's easy to reach Geoff's position: a deep understanding comes from the experience.
That doesn't mean I don't wish my semester were twenty weeks instead of fifteen, so that I could go deeper on some other topics, too. I figure there will always be some tension in the design of the course for just that reason.
I've read a couple of interesting papers recently that included memorable sentences related to program state.
First, Stuart Sierra in On the Perils of Dynamic Scope:
Global state is the zombie in the closet of every Clojure program.
This essay explains the difference between scope and extent, a distinction that affects how easy it is to some of what happens in a program with closures and first-order functions with free variables. Sierra also shows the tension between variables of different kinds, using examples from Clojure. An informative read.
Next, Rob Pike in Go at Google: Language Design in the Service of Software Engineering, a write-up of his SPLASH 2012 keynote address:
The motto [of the Go language] is, "Don't communicate by sharing memory, share memory by communicating."
Imperative programmers who internalize this simple idea are on their way to understanding and using functional programming style effectively. The inversion of sharing and communication turns a lot of design and programming patterns inside out.
Pike's notes provide a comprehensive example of how a new language can grow out of the needs of a particular set of applications, rather than out of programming language theory. The result can look a little hodgepodge, but using such a language often feels just fine. (This reminds me of a different classification of languages with similar practical implications.)
(These papers weren't published April Fool's Day, so I don't think I've been punked.)