March 30, 2015 3:33 PM

Reminiscing on the Effects of Photoshop

Thomas Knoll, one of the creators of Adobe Photoshop, reminisces on the insight that gave rise to the program. His brother, John, worked on analog image composition at Industrial Light and Magic, where they had just begun to experiment with digital processing.

[ILM] had a scanner that could scan in frames from a movie, digitally process them, and then write the images out to film again.

My brother saw that and had a revelation. He said, "If we convert the movie footage into numbers, and we can convert the numbers back into movie footage, then once it's in the numerical form we could do anything to it. We'd have complete power."

I bought my first copy of Photoshop in the summer of 1992, as part of my start-up package for new faculty. In addition to the hardware and software I needed to do my knowledge-based systems research, we also outfitted the lab with a number of other tools, including Aldus Persuasion, a LaCie digital scanner, OmniPage Pro software for OCR, Adobe Premiere, and Adobe Photoshop. I felt like I could do anything I wanted with text, images, and video. It was a great power.

In truth, I barely scratched the surface of what was possible. Others took Photoshop and went places that even Adobe didn't expect them to go. The Knoll brothers sensed what was possible, but it must have been quite something to watch professionals and amateurs alike use the program to reinvent our relationship with images. Here is Thomas Knoll again:

Photoshop has so many features that make it extremely versatile, and there are artists in the world who do things with it that are incredible. I suppose that's the nature of writing a versatile tool with some low-level features that you can combine with anything and everything else.

Digital representation opens new doors for manipulation. When you give users control at both the highest levels and the lowest, who knows what they will do. Stand back and wait.

Posted by Eugene Wallingford | Permalink | Categories: Computing

March 13, 2015 3:07 PM

Two Forms of Irrelevance

When companies become irrelevant to consumers.
From The Power of Marginal, by Paul Graham:

The big media companies shouldn't worry that people will post their copyrighted material on YouTube. They should worry that people will post their own stuff on YouTube, and audiences will watch that instead.

You mean Grey's Anatomy is still on the air? (Or, as today's teenagers say, "Grey's what?")

When people become irrelevant to intelligent machines.
From Outing A.I.: Beyond the Turing Test, by Benjamin Bratton:

I argue that we should abandon the conceit that a "true" Artificial Intelligence must care deeply about humanity -- us specifically -- as its focus and motivation. Perhaps what we really fear, even more than a Big Machine that wants to kill us, is one that sees us as irrelevant. Worse than being seen as an enemy is not being seen at all.

Our new computer overlords indeed. This calls for a different sort of preparation than studying lists of presidents and state capitals.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

March 04, 2015 3:28 PM

Code as a Form of Expression, Even Spreadsheets

Even formulas in spreadsheets, even back in the early 1980s:

Spreadsheet models have become a form of expression, and the very act of creating them seem to yield a pleasure unrelated to their utility. Unusual models are duplicated and passed around; these templates are sometimes used by other modelers and sometimes only admired for their elegance.

People love to make and share things. Computation has given us another medium in which to work, and the things people make with it are often very cool.

The above passage comes from Stephen Levy's A Spreadsheet Way of Knowledge, which appeared originally in Harper's magazine in November 1984. He re-published it on Medium this week in belated honor of Spreadsheet Day last October 17, which was the 35th anniversary of VisiCalc, "the Apple II program that started it all". It's a great read, both as history and as a look at how new technologies create unexpected benefits and dangers.

Posted by Eugene Wallingford | Permalink | Categories: Computing

February 27, 2015 3:37 PM

Bad Habits and Haphazard Design

With an expressive type system for its teaching
languages, HtDP could avoid this problem to some
extent, but adding such rich types would also take
the fun out of programming.

As we approach the midpoint of the semester, Matthias Felleisen's Turing Is Useless strikes a chord in me. My students have spent the last two months learning a little Racket, a little functional programming, and a little about how to write data-driven recursive programs. Yet bad habits learned in their previous courses, or at least unchecked by what they learned there, have made the task harder for many of them than it needed to be.

The essay's title plays off the Church-Turing thesis, which asserts that all programming languages have the same expressive power. This powerful claim is not good news for students who are learning to program, though:

Pragmatically speaking, the thesis is completely useless at best -- because it provides no guideline whatsoever as to how to construct programs -- and misleading at worst -- because it suggests any program is a good program.

With a Turing-universal language, a clever student can find a way to solve any problem with some program. Even uninspired but persistent students can tinker their way to a program that produces the right answers. Unfortunately, they don't understand that the right answers aren't the point; the right program is. Trolling StackOverflow will get them a program, but too often the students don't understand whether it is a good or bad program in their current situation. It just works.

I have not been as faithful to the HtDP approach this semester as I probably should have been, but I share its desire to help students to design programs systematically. We have looked at design patterns that implement specific strategies, not language features. Each strategy focuses on the definition of the data being processed and the definition of the value being produced. This has great value for me as the instructor, because I can usually see right away why a function isn't working for the student the way he or she intended: they have strayed from the data as defined by the problem.

This is also of great value to some of my students. They want to learn how to program in a reliable way, and having tools that guide their thinking is more important than finding yet another primitive Racket procedure to try. For others, though "garage programming" is good enough; they just want get the job done right now, regardless of which muscles they use. Design is not part of their attitude, and that's a hard habit to break. How use doth breed a habit in a student!

Last semester, I taught intro CS from what Felleisen calls a traditional text. Coupled that experience with my experience so far this semester, I'm thinking a lot these days about how we can help students develop a design-centered attitude at the outset of their undergrad courses. I have several blog entries in draft form about last semester, but one thing that stands out is the extent to which every step in the instruction is driven by the next cool programming construct. Put them all on the table, fiddle around for a while, and you'll make something that works. One conclusion we can draw from the Church-Turing thesis is that this isn't surprising. Unfortunately, odds are any program created this way is not a very good program.


(The sentence near the end that sounds like Shakespeare is. It's from The Two Gentlemen of Verona, with a suitable change in noun.)

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development, Teaching and Learning

February 06, 2015 3:11 PM

What It Feels Like To Do Research

In one sentence:

Unless you tackle a problem that's already solved, which is boring, or one whose solution is clear from the beginning, mostly you are stuck.

This is from Alec Wilkinson's The Pursuit of Beauty, about mathematician Yitang Zhang, who worked a decade on the problem of bounded gaps between prime numbers. As another researcher says in the article,

When you try to prove a theorem, you can almost be totally lost to knowing exactly where you want to go. Often, when you find your way, it happens in a moment, then you live to do it again.

Programmers get used to never feeling normal, but tackling the twin prime problem is on a different level altogether. The same is true for any deep open question in math or computing.

I strongly recommend Wilkinson's article. It describes what life for untenured mathematicians is like, and how a single researcher can manage to solve an important problem.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

January 28, 2015 3:38 PM

The Relationship Between Coding and Literacy

Many people have been discussing Chris Granger's recent essay Coding is not the New Literacy, and most seem to approve of his argument. Reading it brought to my mind this sentence from Alan Kay in VPRI Memo M-2007-007a, The Real Computer Revolution Hasn't Happened Yet:

Literacy is not just being able to read and write, but being able to deal fluently with the kind of ideas that are important enough to write about and discuss.

Literacy requires both the low-level skills of reading and writing and the higher-order capacity for using them on important ideas.

That is one thing that makes me uneasy about Granger's argument. It is true that teaching people only low-level coding skills won't empower them if they don't know how to use them to use them fluently to build models that matter. But neither will teaching them how to build models without giving them access to the programming skills they need to express their ideas beyond what some tool gives them.

Like Granger, though, I am also uneasy about many of the learn-to-code efforts. Teaching people enough Javascript or Ruby to implement a web site out of the box skips past the critical thinking skills that people need to use computation effectively in their world. They may be "productive" in the short term, but they are also likely to hit a ceiling pretty soon. What then? My guess: they become frustrated and stop coding altogether.

the Scratch logo

We sometimes do a better job introducing programming to kids, because we use tools that allow students to build models they care about and can understand. In the VPRI memo, Kay describes experiences teaching elementary school, students to use eToys to model physical phenomena. In the end, they learn physics and the key ideas underlying calculus. But they also learn the fundamentals of programming, in an environment that opens up into Squeak, a flavor of Smalltalk.

I've seen teachers introduce students to Scratch in a similar way. Scratch is a drag-and-drop programming environment, but it really is a open-ended and lightweight modeling tool. Students can learn low-level coding skills and higher-level thinking skills in tandem.

That is the key to making Granger's idea work in the best way possible. We need to teach people how to think about and build models in a way that naturally evolves into programming. I am reminded of another quote from Alan Kay that I heard back in the 1990s. He reminded us that kindergarteners learn and use the same language that Shakespeare used It is possible for their fluency in the language to grow to the point where they can comprehend some of the greatest literature ever created -- and, if they possess some of Shakepeare's genius, to write their own great literature. English starts small for children, and as they grow, it grows with them. We should aspire to do the same thing for programming.

the logo for Eve

Granger reminds us that literacy is really about composition and comprehension. But it doesn't do much good to teach people how to solidify their thoughts so that they can be written if they don't know how to write. You can't teach composition until your students know basic reading and writing.

Maybe we can find a way to teach people how to think in terms of models and how to implement models in programs at the same time, in a language system that grows along with their understanding. Granger's latest project, Eve, may be a step in that direction. There are plenty of steps left for us to take in the direction of languages like Scratch, too.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

January 18, 2015 10:26 AM

The Infinite Horizon

In Mathematics, Live: A Conversation with Laura DeMarco and Amie Wilkinson, Amie Wilkinson recounts the pivotal moment when she knew she wanted to be a mathematician. Insecure about her abilities in mathematics, unsure about what she wanted to do for a career, and with no encouragement, she hadn't applied to grad school. So:

I came back home to Chicago, and I got a job as an actuary. I enjoyed my work, but I started to feel like there was a hole in my existence. There was something missing. I realized that suddenly my universe had become finite. Anything I had to learn for this job, I could learn eventually. I could easily see the limits of this job, and I realized that with math there were so many things I could imagine that I would never know. That's why I wanted to go back and do math. I love that feeling of this infinite horizon.

After having written software for an insurance company during the summers before and after my senior year in college, I knew all too well the "hole in my existence" that Wilkinson talks about, the shrinking universe of many industry jobs. I was deeply interested in the ideas I had found in Gödel, Escher, Bach, and in the idea of creating an intelligent machine. There seemed no room for those ideas in the corporate world I saw.

I'm not sure when the thought of graduate school first occurred to me, though. My family was blue collar, and I didn't have much exposure to academia until I got to Ball State University. Most of my friends went out to get jobs, just like Wilkinson. I recall applying for a few jobs myself, but I never took the job search all that seriously.

At least some of the credit belongs to one of my CS professors, Dr. William Brown. Dr. Brown was an old IBM guy who seemed to know so much about how to make computers do things, from the lowest-level details of IBM System/360 assembly language and JCL up to the software engineering principles needed to write systems software. When I asked him about graduate school, he talked to me about how to select a school and a Ph.D. advisor. He also talked about the strengths and weaknesses of my preparation, and let me know that even though I had some work to do, I would be able to succeed.

These days, I am lucky even to have such conversations with my students.

For Wilkinson, DeMarco and me, academia was a natural next step in our pursuit of the infinite horizon. But I now know that we are fortunate to work in disciplines where a lot of the interesting questions are being asked and answers by people working in "the industry". I watch with admiration as many of my colleagues do amazing things while working for companies large and small. Computer science offers so many opportunities to explore the unknown.

Reading Wilkinson's recollection brought a flood of memories to mind. I'm sure I wasn't alone in smiling at her nod to finite worlds and infinite horizons. We have a lot to be thankful for.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal, Teaching and Learning

January 16, 2015 2:59 PM

Programming Language As Artistic Medium

Says Ramsey Nasser:

I have always been fascinated by esolangs. They are the such an amazing intersection of technical and formal rigor on one hand and nerdy inside humor on the other. The fact that they are not just ideas, but *actual working languages* is incredible. Its something that could only exist in a field as malleable and accessible as code. NASA engineers cannot build a space station as a joke.

Because we can create programming languages as a joke, or for any other reason, a programming language can be both message and medium.

a Hello, World program in Piet

Esolang is enthusiast shorthand for esoteric programming language. I'm not an enthusiast on par with many, but I've written a few Ook! interpreters and played around with others. Piet is the most visually appealing of the esoteric languages I've encountered. The image to the right is a "Hello, World" program written in Piet, courtesy of the Wikimedia Commons.

Recently I have been reading more about the work of Nasser, a computer scientist and artist formerly at the Eyebeam Art + Technology Center. In 2010, he created the Zajal programming language as his MFA thesis project at the Parsons School of Design. Zajal was inspired by Processing and runs on top of Ruby. A couple of years ago, he received widespread coverage for Qalb, a language with Arabic script characters and a Scheme-like syntax. Zajal enables programmers to write programs with beautiful output; Qalb enables programmers to write programs that are themselves quite beautiful.

I wouldn't call Zajal or Qalb esoteric programming languages. They are, in an important way, quite serious, exploring the boundary between "creative vision" and software. As he says at the close of the interview quoted above, we now live in a world in which "code runs constantly in our pockets":

Code is a driving element of culture and politics, which means that code that is difficult to reason about or inaccessible makes for a culture and politics that are difficult to reason about and inaccessible. The conversation about programming languages has never been more human than it is now, and I believe this kind of work will only become more so as software spreads.

As someone who teaches computer science students to think more deeply about programming languages, I would love to see more and different kinds of people entering the conversation.

Posted by Eugene Wallingford | Permalink | Categories: Computing

January 12, 2015 10:26 AM

WTF Problems and Answers for Questions Unasked

Dan Meyer quotes Scott Farrand in WTF Math Problems:

Anything that makes students ask the question that you plan to answer in the lesson is good, because answering questions that haven't been asked is inherently uninteresting.

My challenge this semester: getting students to ask questions about the programming languages they use and how they work. I myself have many questions about languages! My experience teaching our intro course last semester reminded me that what interests me (and textbook authors) doesn't always interest my students.

If you have any WTF? problems for a programming languages course, please share.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

January 09, 2015 3:40 PM

Computer Science Everywhere, Military Edition

Military Operations Orders are programs that are executed by units. Code re-use and other software engineering principles applied regularly to these.

An alumnus of my department, a CS major-turned-military officer, wrote those lines in an e-mail responding to my recent post, A Little CS Would Help a Lot of College Grads. Contrary to what many people might imagine, he has found what he learned in computer science to be quite useful to him as an Army captain. And he wasn't even a programmer:

One of the biggest skills I had over my peers was organizing information. I wasn't writing code, but I was handling lots of data and designing systems for that data. Organizing information in a way that was easy to present to my superiors was a breeze and having all the supporting data easily accessible came naturally to me.

Skills and principles from software engineering and project development apply to systems other than software. They also provide a vocabulary for talking about ideas that non-programmers encounter every day:

I did introduce my units to the terms border cases, special cases, and layers of abstraction. I cracked a smile every time I heard those terms used in a meeting.

Excel may not be a "real programming language", but knowing the ways in which it is a language can make managers of people and resources more effective at what they do.

For more about how a CS background has been useful to this officer, check out CS Degree to Army Officer, a blog entry that expands on his experiences.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

December 31, 2014 10:15 AM

Reinventing Education by Reinventing Explanation

One of the more important essays I read in 2014 was Michael Nielsen's Reinventing Explanation. In it, Nielsen explores how we might design media that help us explain scientific ideas better than we are able with our existing tools.

... it's worth taking non-traditional media seriously not just as a vehicle for popularization or education, which is how they are often viewed, but as an opportunity for explanations which can be, in important ways, deeper.

This essay struck me deep. Nielsen wants us to consider how we might take what we have learned using non-traditional media to popularize and educate and use it to think about how to explain more deeply. I think that learning how to use non-traditional media to explain more deeply will help us change the way we teach and learn.

In too many cases, new technologies are used merely as substitutes for old technology. The web has led to an explosion of instructional video aimed at all levels of learners. No matter how valuable these videos are, most merely replace reading a textbook or a paper. But computational technology enables us to change the task at hand and even redefine what we do. Alan Kay has been telling this story for decades, pointing us to the work of Ivan Sutherland and many others from the early days of computing.

Nielsen points to Bret Victor as an example of someone trying to develop tools that redefine how we think. As Victor himself says, he is following in the grand tradition of Kay, Sutherland, et al. Victor's An Ill-Advised Personal Note about "Media for Thinking the Unthinkable" is an especially direct telling of his story.

Vi Hart is another. Consider her recent Parable of the Polygons, created with Nicky Case, which explains dynamically how local choices and create systemic bias. This simulation uses computation to help people think differently about an idea they might not understand as viscerally from a traditional explanation. Hart has a long body of working using visualization to explain differently, and the introduction of computing extends the depth of her approach.

Over the last few weeks, I have felt myself being pulled by Nielsen's essay and the example of people such as Victor and Hart to think more about how we might design media that help us to teach and explain scientific ideas more deeply. Reinventing explanation might help us reinvent education in a way that actually matters. I don't have a research agenda yet, but looking again at Victor's work is a start.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

December 28, 2014 11:12 AM

A Little CS Would Help a Lot of College Grads

I would love to see more CS majors, but not everyone should major in CS. I do think that most university students could benefit from learning a little programming. There are plenty of jobs not only for CS and math grads, but also for other majors who have CS and math skills:

"If you're an anthropology major and you want to get a marketing job, well, guess what? The toughest marketing jobs to fill require SQL skills," Sigelman says. "If you can ... along the peripheries of your academic program accrue some strong quantitative skills, you'll still have the advantage [in the job market]." Likewise, some legal occupations (such as intellectual property law) and maintenance and repair jobs stay open for long periods of time, according to the Brookings report, if they require particular STEM skills.

There is much noise these days about the importance of STEM, both for educated citizens and for jobs, jobs, jobs. STEM isn't an especially cohesive category, though, as the quoted Vox article reminds us, and even when we look just at economic opportunity, it misleads. We don't need more college science graduates from every STEM discipline. We do need more people with the math and CS skills that now pervade the workplace, regardless of discipline. As Kurtzleben says in the article, "... characterizing these skill shortages as a broad STEM crisis is misleading to students, and has distorted the policy debate."

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

December 27, 2014 8:47 AM

Let's Not Forget: CS 1 Is Hard For Most Students

... software is hard. It's harder than
anything else I've ever had to do.
-- Donald Knuth

As students were leaving my final CS 1 lab session of the semester, I overheard two talking about their future plans. One student mentioned that he was changing his major to actuarial science. I thought, wow, that's a tough major. How is a student who is struggling with basic programming going to succeed there?

When I checked on his grades, though, I found that he was doing fine in my course, about average. I also remembered that he had enjoyed best the programming exercises that computed terms of infinite arithmetic series and other crazy mathematical values that his classmates often found impenetrable. Maybe actuarial science, even with some hard math, will be a good fit for him.

It really shouldn't surprise us that some students try computer science and decide to major in something else, even something that looks hard to most people. Teaching CS 1 again this semester after a long break reminded me just how much we expect from the students in our introductory course:

  • Details. Lots and lots of details. Syntax. Grammar. Vocabulary, both in a programming language and about programming more generally. Tools for writing, editing, compiling, and running programs.

  • Experimentation. Students have to design and execute experiments in order to figure out how language constructs work and to debug the programs they write. Much of what they learn is by trial and error, and most students have not yet developed skills for doing that in a controlled fashion.

  • Design. Students have to decompose problems and combine parts into wholes. They have to name things. They have to connect the names they see with ideas from class, the text, and their own experience.

  • Abstraction. Part of the challenge in design comes from abstraction, but abstract ideas are everywhere in learning about CS and how to program. Variables, choices, loops and recursion, functions and arguments and scope, ... all come not just as concrete forms but also as theoretical notions. These notions can sometimes be connected to the students' experience of the physical world, but the computing ideas are often just different enough to disorient the student. Other CS abstractions are so different as to appear unique.

In a single course, we expect students to perform tasks in all three of these modes, while mastering a heavy load of details. We expect them to learn by deduction, induction, and abduction, covering many abstract ideas and many concrete details. Many disciplines have challenging first courses, but CS 1 requires an unusual breadth of intellectual tools.

Yes, we can improve our students' experience with careful pedagogy. Over the last few decades we've seen many strong efforts. And yes, we can help students through the process with structural support, emotional support, and empathy. In the end, though, we must keep this in mind: CS 1 is going to be a challenge for most students. For many, the rewards will be worth the struggle, but that doesn't mean it won't take work, patience, and persistence along the way -- by both the students and the teachers.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

December 26, 2014 8:32 AM

Editing and the Illusion of Thought

Martin Amis, in The Paris Review, The Art of Fiction No. 151:

By the way, it's all nonsense about how wonderful computers are because you can shift things around. Nothing compares with the fluidity of longhand. You shift things around without shifting them around--in that you merely indicate a possibility while your original thought is still there. The trouble with a computer is that what you come out with has no memory, no provenance, no history--the little cursor, or whatever it's called, that wobbles around the middle of the screen falsely gives you the impression that you're thinking. Even when you're not.

My immediate reaction was that Mr. Amis needs version control, but there is something more here.

When writing with pencil and paper, we work on an artifact that embodies the changes it has gone through. We see the marks and erasures; we see the sentence where it once was once at the same time we see the arrow telling us where it now belongs. When writing in a word processor, our work appears complete, even timeless, though we know it isn't. Mark-up mode lets us see some of the document's evolution, but the changes feel more distant from our minds. They live out there.

I empathize with writers like Amis, whose experience predates the computer. Longhand feels different. Teasing out what what was valuable, even essential, in previous experience and what was merely the limitation of our tools is one of the great challenges of any time. How do we make new tools that are worth the change, that enable us to do more and better?

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

December 24, 2014 2:05 PM

Computer Science Everywhere, Christmas Eve Edition

Urmson says Google is better positioned than a traditional automaker to crack the riddle of self-driving, because it's more about software than hardware: "When you look at what we're doing, on the surface, you see a vehicle. But the heart of it is computer science.

That is Chris Urmson, the head of Google's self-driving car program, quoted in this article. (Apparently, senior citizens are a natural market for driverless cars.)

Everywhere we look these days, we see gadgets. Increasingly, though, at the heart of them is computer science.

Posted by Eugene Wallingford | Permalink | Categories: Computing

November 25, 2014 1:43 PM

Concrete Play Trumps All

Areschenko-Johannessen, Bundesliga 2006-2007

One of the lessons taught by the computer is that concrete play trumps all.

This comment appeared in the review of a book of chess analysis [ paywalled ]. The reviewer is taking the author to task for talking about the positional factors that give one player "a stable advantage" in a particular position, when a commercially-available chess program shows the other player can equalize easily, and perhaps even gain an advantage.

It is also a fitting comment on our relationship with computers these days more generally. In areas such as search and language translation, Google helped us see that conventional wisdom can often be upended by a lot of data and many processors. In AI, statistical techniques and neural networks solve problems in ways that models of human cognition cannot. Everywhere we turn, it seems, big data and powerful computers are helping us to redefine our understanding of the world.

We humans need not lose all hope, though. There is still room for building models of the world and using them to reason, just as there is room for human analysis of chess games. In chess, computer analysis is pushing grandmasters to think differently about the game. The result is a different kind of understanding for the more ordinary of us, too. We just have to be careful to check our abstract understanding against computer analysis. Concrete play trumps all, and it tests our hypotheses. That's good science, and good thinking.


(The chess position is from Areschenko-Johannessen 2006-2007, used as an example in Chess Training for Post-Beginners by Yaroslav Srokovski and cited in John Hartmann's review of the book in the November 2014 issue of Chess Life.)

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

November 23, 2014 8:50 AM

Supply, Demand, and K-12 CS

When I meet with prospective students and their parents, we often end up discussing why most high schools don't teach computer science. I tell them that, when I started as a new prof here, about a quarter of incoming freshmen had taken a year of programming in high school, and many other students had had the opportunity to do so. My colleagues and I figured that this percentage would go way up, so we began to think about how we might structure our first-year courses when most or all students already knew how to program.

However, the percentage of incoming students with programming experience didn't go up. It went way down. These days, about 10% of our freshman know how to program when they start our intro course. Many of those learned what they know on their own. What happened, today's parents ask?

A lot of things happened, including the dot-com bubble, a drop in the supply of available teachers, a narrowing of the high school curriculum in many districts, and the introduction of high-stakes testing. I'm not sure how much each contributed to the change, or whether other factors may have played a bigger role. Whatever the causes, the result is that our intro course still expects no previous programming experience.

Yesterday, I saw a post by a K-12 teacher on the Racket users mailing list that illustrates the powerful pull of economics. He is leaving teaching for software development industry, though reluctantly. "The thing I will miss the most," he says, "is the enjoyment I get out of seeing youngsters' brains come to life." He also loves seeing them succeed in the careers that knowing how to program makes possible. But in that success lies the seed of his own career change:

Speaking of my students working in the field, I simply grew too tired of hearing about their salaries which, with a couple of years experience, was typically twice what I was earning with 25+ years of experience. Ultimately that just became too much to take.

He notes that college professors probably know the feeling, too. The pull must be much stronger on him and his colleagues, though; college CS professors are generally paid much better than K-12 teachers. A love of teaching can go only so far. At one level, we should probably be surprised that anyone who knows how to program well enough to teach thirteen- or seventeen-year-olds to do it stays in the schools. If not surprised, we should at least be deeply appreciative of the people who do.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

November 20, 2014 3:23 PM

When I Procrastinate, I Write Code

I procrastinated one day with my intro students in mind. This is the bedtime story I told them as a result. Yes, I know that I can write shorter Python code to do this. They are intro students, after all.


Once upon a time, a buddy of mine, Chad, sent out a tweet. Chad is a physics prof, and he was procrastinating. How many people would I need to have in class, he wondered, to have a 50-50 chance that my class roster will contain people whose last names start with every letter of the alphabet?


This is a lot like the old trivia about how we only need to have 23 people in the room to have a 50-50 chance that two people share a birthday. The math for calculating that is straightforward enough, once you know it. But last names are much more unevenly distributed across the alphabet than birthdays are across the days of the year. To do this right, we need to know rough percentages for each letter of the alphabet.

I can procrastinate, too. So I surfed over to the US Census Bureau, rummaged around for a while, and finally found a page on Frequently Occurring Surnames from the Census 2000. It provides a little summary information and then links to a couple of data files, including a spreadsheet of data on all surnames that occurred at least 100 times in the 2000 census. This should, I figure, cover enough of the US population to give us a reasonable picture of how peoples' last names are distributed across the alphabet. So I grabbed it.

(We live in a wonderful time. Between open government, open research, and open source projects, we have access to so much cool data!)

The spreadsheet has columns with these headers:

    name,rank,count,prop100k,cum_prop100k,      \
                    pctwhite,pctblack,pctapi,   \

The first and third columns are what we want. After thirteen weeks, we know how to do compute the percentages we need: Use the running total pattern to count the number of people whose name starts with 'a', 'b', ..., 'z', as well as how many people there are altogether. Then loop through our collection of letter counts and compute the percentages.

Now, how should we represent the data in our program? We need twenty-six counters for the letter counts, and one more for the overall total. We could make twenty-seven unique variables, but then our program would be so-o-o-o-o-o long, and tedious to write. We can do better.

For the letter counts, we might use a list, where slot 0 holds a's count, slot 1 holds b's count, and so one, through slot 25, which holds z's count. But then we would have to translate letters into slots, and back, which would make our code harder to write. It would also make our data harder to inspect directly.

    ----  ----  ----  ...  ----  ----  ----    slots in the list

0 1 2 ... 23 24 25 indices into the list

The downside of this approach is that lists are indexed by integer values, while we are working with letters. Python has another kind of data structure that solves just this problem, the dictionary. A dictionary maps keys onto values. The keys and values can be of just about any data type. What we want to do is map letters (characters) onto numbers of people (integers):

    ----  ----  ----  ...  ----  ----  ----    slots in the dictionary

'a' 'b' 'c' ... 'x' 'y' 'z' indices into the dictionary

With this new tool in hand, we are ready to solve our problem. First, we build a dictionary of counters, initialized to 0.

    count_all_names = 0
    total_names = {}
    for letter in 'abcdefghijklmnopqrstuvwxyz':
        total_names[letter] = 0

(Note two bits of syntax here. We use {} for dictionary literals, and we use the familiar [] for accessing entries in the dictionary.)

Next, we loop through the file and update the running total for corresponding letter, as well as the counter of all names.

    source = open('app_c.csv', 'r')
    for entry in source:
        field  = entry.split(',')        # split the line
        name   = field[0].lower()        # pull out lowercase name
        letter = name[0]                 # grab its first character
        count  = int( field[2] )         # pull out number of people
        total_names[letter] += count     # update letter counter
        count_all_names     += count     # update global counter

Finally, we print the letter → count pairs.

    for (letter, count_for_letter) in total_names.items():
        print(letter, '->', count_for_letter/count_all_names)

(Note the items method for dictionaries. It returns a collection of key/value tuples. Recall that tuples are simply immutable lists.)

We have converted the data file into the percentages we need.

    q -> 0.002206197888442366
    c -> 0.07694634659082318
    h -> 0.0726864447688946
    f -> 0.03450702533438715
    x -> 0.0002412718532764804
    k -> 0.03294646311104032

(The entries are not printed in alphabetical order. Can you find out why?)

I dumped the output to a text file and used Unix's built-in sort to create my final result. I tweet Chad, Here are your percentages. You do the math.

Hey, I'm a programmer. When I procrastinate, I write code.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

November 11, 2014 7:53 AM

The Internet Era in One Sentence

I just love this:

When a 14-year-old kid can blow up your business in his spare time, not because he hates you but because he loves you, then you have a problem.

Clay Shirky attributes it to Gordy Thompson, who managed internet services at the New York Times in the early 1990s. Back then, it was insightful prognostication; today, it serves as an epitaph for many an old business model.

Are 14-year-old kids making YouTube videos to replace me yet?

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

October 31, 2014 2:52 PM

Ada Lovelace, AI Visionary

We hear a lot about Ada Lovelace being the first computer programmer, but that may not be her most impressive computing first. When I read Steven Johnson's The Tech Innovators of the Victorian Age I learned that she may have been the first modern person to envision the digital computer as a vehicle for an intelligent machine.

Though I have heard about Ada's work with Charles Babbage before, I didn't know any of the details. An engineer had written an essay about the Analytical Engine in Italian, and Lovelace set out to translate it into English. But she also added her own comments to the text as footnotes. It was in a footnote that she recorded "a series of elemental instruction sets that could be used to direct the calculations of the Analytical Engine". When people say Lovelace was the first computer programmer, they are referring to this footnote.

Some people contend that Lovelace did not write this program; rather, Babbage had outlined some procedures and that she refined them. If that is true, then Lovelace and Babbage still conspired on a noteworthy act: they were the first people to collaborate on a program. How fitting that the first computer program was a team effort.

That is only the beginning. Writes Johnson,

But her greatest contribution lay not in writing instruction sets but, rather, in envisioning a range of utility for the machine that Babbage himself had not considered. "Many persons," she wrote, "imagine that because the business of the engine is to give its results in numerical notation, the nature of its processes must consequently be arithmetical and numerical, rather than algebraical and analytical. This is an error. The engine can arrange and combine its numerical quantities exactly as if they were letters or any other general symbols."

Lovelace foresaw the use of computation for symbol manipulation, analytical reasoning, and even the arts:

"Supposing, for instance, that the fundamental relations of pitched sounds in the science of harmony and musical composition were susceptible of such expressions and adaptations, the Engine might compose elaborate and scientific pieces of music of any degree of complexity or extent."

The Analytical Engine could be used to simulate intelligent behavior. Lovelace imagined artificial intelligence.

Johnson calls this perhaps the most visionary footnote in the history of print. That may be a bit over the top, but can you blame him? Most people of the 19th century could hardly conceive of the idea of a programmable computer. By the middle of the 20th century, many people understood that computers could implement arithmetic processes that would change many areas of life. But for most people, the idea of an "intelligent machine" was fantastic, not realistic.

In 1956, a group of visionary scientists organized the Dartmouth conferences to brainstorm from the belief that "every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it". The Darmouth summer project may have been a seminal event in the history of AI. However, over a century earlier, Ada Lovelace saw the potential that a computing machine could partake in language and art. That may have been the first seminal moment in AI history.

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 29, 2014 3:56 PM

Computing Future and Computing Past:

Administrative and teaching duties have been keeping me busy of late, but I've enjoyed following along with, a throwback, shell-based, Unix community started by Paul Ford and blogged about by him on his ~ford page there. feels like 1986 to me, or maybe 2036. In one sense, it is much less than today's social networks. In many other ways, it is so much more. The spirit of learning and adventure and connecting are more important there than glitzy interface and data anlytics and posturing for a public that consists of hundreds of Facebook 'friends' and Twitter 'followers'.

Ford mentions the trade-off in his long Medium article:

It's not like you can build the next Facebook or Twitter or Google on top of a huge number of Internet-connected Linux servers. Sure, Facebook, Twitter, and Google are built on top of a huge number of loosely connected Linux servers. But you know what I mean.

This project brings to mind a recent interview with writer William Gibson, in which he talks about the future and the past. In particular, this passage expresses a refreshingly different idea of what knowledge from the future would be most interesting -- and useful -- today:

If there were somehow a way for me to get one body of knowledge from the future -- one volume of the great shelf of knowledge of a couple of hundred years from now -- I would want to get a history. I would want to get a history book. I would want to know what they think of us.

I often wonder what the future will think of this era of computing, in which we dream too small and set the bar of achievement too low. We can still see the 1960s and 1970s in our rearview mirror, yet the dreams and accomplishments of that era are forgotten by so many people today -- even computer scientists, who rarely ever think about that time at all. is the sort of project that looks backward and yet enables us to look forward. Eliminate as much noise as possible and see what evolves next. I'm curious to see where it goes.

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 17, 2014 3:05 PM

Assorted Quotes

... on how the world evolves.

On the evolution of education in the Age of the Web. Tyler Cowen, in Average Is Over, via The Atlantic:

It will become increasingly apparent how much of current education is driven by human weakness, namely the inability of most students to simply sit down and try to learn something on their own.

I'm curious whether we'll ever see a significant change in the number of students who can and do take the reins for themselves.

On the evolution of the Web. Jon Udell, in A Web of Agreements and Disagreements:

The web works as well as it does because we mostly agree on a set of tools and practices. But it evolves when we disagree, try different approaches, and test them against one another in a marketplace of ideas. Citizens of a web-literate planet should appreciate both the agreements and the disagreements.

Some disagreements are easier to appreciate after they fade into history.

On the evolution of software. Nat Pryce on the Twitter, via The Problematic Culture of "Worse is Better":

Eventually a software project becomes a small amount of useful logic hidden among code that copies data between incompatible JSON libraries

Not all citizens of a web-literate planet appreciate disagreements between JSON libraries. Or Ruby gems.

On the evolution of start-ups. Rands, in The Old Guard:

... when [the Old Guard] say, "It feels off..." what they are poorly articulating is, "This process that you're building does not support one (or more) of the key values of the company."

I suspect the presence of incompatible JSON libraries means that our software no longer supports the key values of our company.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Managing and Leading, Software Development, Teaching and Learning

October 16, 2014 3:54 PM

For Programmers, There Is No "Normal Person" Feeling

I see this in the lab every week. One minute, my students sit peering at their monitors, their heads buried in their hands. They can't do anything right. The next minute, I hear shouts of exultation and turn to see them, arms thrust in the air, celebrating their latest victory over the Gods of Programming. Moments later I look up and see their heads again in their hands. They are despondent. "When will this madness end?"

Last week, I ran across a tweet from Christina Cacioppo that expresses nicely a feeling that has been vexing so many of my intro CS students this semester:

I still find programming odd, in part, because I'm either amazed by how brilliant or how idiotic I am. There's no normal-person feeling.

Christina is no beginner, and neither am I. Yet we know this feeling well. Most programmers do, because it's a natural part of tackling problems that challenge us. If we didn't bounce between feeling puzzlement and exultation, we wouldn't be tackling hard-enough problems.

What seems strange to my students, and even to programmers with years of experience, is that there doesn't seem to be a middle ground. It's up or down. The only time we feel like normal people is when we aren't programming at all. (Even then, I don't have many normal-person feelings, but that's probably just me.)

I've always been comfortable with this bipolarity, which is part of why I have always felt comfortable as a programmer. I don't know how much of this comfort is natural inclination -- a personality trait -- and how much of it is learned attitude. I am sure it's a mixture of both. I've always liked solving puzzles, which inspired me to struggle with them, which helped me get better struggling with them.

Part of the job in teaching beginners to program is to convince them that this is a habit they can learn. Whatever their natural inclination, persistence and practice will help them develop the stamina they need to stick with hard problems and the emotional balance they need to handle the oscillations between exultation and despondency.

I try to help my students see that persistence and practice are the answer to most questions involving missing skills or bad habits. A big part of helping them this is coaching and cheerleading, not teaching programming language syntax and computational concepts. Coaching and cheerleading are not always tasks that come naturally to computer science PhDs, who are often most comfortable with syntax and abstractions. As a result, many CS profs are uncomfortable performing them, even when that's what our students need most. How do we get better at performing them? Persistence and practice.

The "no normal-person feeling" feature of programming is an instance of a more general feature of doing science. Martin Schwartz, a microbiologist at the University of Virginia, wrote a marvelous one-page article called The importance of stupidity in scientific research that discusses this element of being a scientist. Here's a representative sentence:

One of the beautiful things about science is that it allows us to bumble along, getting it wrong time after time, and feel perfectly fine as long as we learn something each time.

Scientists get used to this feeling. My students can, too. I already see the resilience growing in many of them. After the moment of exultation passes following their latest conquest, they dive into the next task. I see a gleam in their eyes as they realize they have no idea what to do. It's time to bury their heads in their hands and think.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

October 15, 2014 3:54 PM

Maybe We Just Need to Teach Better

Maybe We Just Need to Teach Better

A couple of weeks ago, I wrote Skills We Can Learn in response to a thread on the SIGCSE mailing list. Mark Guzdial has now written a series of posts in response to that thread, most recently Teaching Computer Science Better To Get Better Results. Here is one of the key paragraphs in his latest piece:

I watch my children taking CS classes, along with English, Chemistry, Physics, and Biology classes. In the CS classes, they code. In the other classes, they do on-line interactive exercises, they write papers, they use simulations, they solve problems by-hand. Back in CS, the only activity is coding with feedback. If we only have one technique for teaching, we shouldn't be surprised if it doesn't always work.

Mark then offers a reasonable hypothesis: We get poor results because we use ineffective teaching methods.

That's worthy of a new maxim of the sort found in my previous post: If things aren't going well in my course, it's probably my fault. Mark's hypothesis sounds more professional.

A skeptic might say that learning to program is like learning to speak a new human language, and when we learn new human languages we spend most of our time reading, writing, and speaking, and getting feedback from these activities. In an introductory programming course, the programming exercises are where students read, write, and get feedback. Isn't that enough?

For some students, yes, but not for all. This is also true in introductory foreign language courses, which is why teachers in those courses usually include games and other activities to engage the students and provide different kinds of feedback. Many of us do more than just programming exercises in computer science courses, too. In courses with theory and analysis, we give homework that asks students to solve problems, compute results, or give proofs for assertions about computation.

In my algorithms course, I open most days with a game. Students play the game for a while, and then we discuss strategies for playing the game well. I choose games whose playing strategies illustrate some algorithm design technique we are studying. This is a lot more fun than yet another Design an algorithm to... exercise. Some students seem to understand the ideas better, or at least differently, when they experience the ideas in a wider context.

I'm teaching our intro course right now, and over the last few weeks I have come to appreciate the paucity of different teaching techniques and methods used by a typical textbook. This is my first time to teach the course in ten years, and I'm creating a lot of my own materials from scratch. The quality and diversity of the materials are limited by my time and recent experience, with the result being... a lot of reading and writing of code.

What of the other kinds of activities that Mark mentions? Some code reading can be turned into problems that the students solve by hand. I have tried a couple of debugging exercises that students seemed to find useful. I'm only now beginning to see the ways in which those exercises succeeded and failed, as the students take on bigger tasks.

I can imagine all sorts of on-line interactive exercises and simulations that would help in this course. In particular, a visual simulator for various types of loops could help students see a program's repetitive behavior more immediately than watching the output of a simple program. Many of my students would likely benefit from a Bret Victor-like interactive document that exposes the internal working of, say, a for loop. Still others could use assistance with even simpler concepts, such as sequences of statements, assignment to variables, and choices.

In any case, I second Mark's calls to action. We need to find more and better methods for teaching CS topics. We need to find better ways to make proven methods available to CS instructors. Most importantly, we need to expect more of ourselves and demand more from our profession.

When things go poorly in my classroom, it's usually my fault.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

October 06, 2014 4:02 PM

A New Programming Language Can Inspire Us

In A Fresh Look at Rust, Armin Ronacher tells us that some of what inspires him about Rust:

For me programming in Rust is pure joy. Yes I still don't agree with everything the language currently forces me to do but I can't say I have enjoyed programming that much in a long time. It gives me new ideas how to solve problems and I can't wait for the language to get stable.

Rust is inspiring for many reasons. The biggest reason I like it is because it's practical. I tried Haskell, I tried Erlang and neither of those languages spoke "I am a practical language" to me. I know there are many programmers that adore them, but they are not for me. Even if I could love those languages, other programmers would never do and that takes a lot of enjoyment away.

I enjoy reading personal blog entries from people excited by a new language, or newly excited by a language they are visiting again after a while away. I've only read Rust code, not written it, but I know just how Ronacher feels. These two paragraphs touch on several truths about how languages excite us:

  • Programmers are often most inspired when a language shows them new ideas how to solve problems.
  • Even if we love a language, we won't necessarily love every feature of the language.
  • What inspires us is personal. Other people can be inspired by languages that do not excite us.
  • Community matters.

Many programmers make a point of learning a new language periodically. When we do, we are often most struck by a language that teaches us new ways to think about problems and how to solve them. These are usually the languages that have the most teach us at the moment.

As Kevin Kelly says, progress sometimes demands that we let go of problems. We occasionally have to seek new problems, in order to be excited by new ways to answer them.

This all is very context-specific, other. How wonderful it is to live in a time with so many languages available to learn from. Let them all flourish, I say.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

October 02, 2014 3:46 PM

Skills We Can Learn

In a thread on motivating students on the SIGCSE mailing list, a longtime CS prof and textbook author wrote:

Over the years, I have come to believe that those of us who can become successful programmers have different internal wiring than most in the population. We know you need problem solving, mathematical, and intellectual skills but beyond that you need to be persistent, diligent, patient, and willing to deal with failure and learn from it.

These are necessary skills, indeed. Many of our students come to us without these skills and struggle to learn how to think like a computer scientist. And without persistence, diligence, patience, and a willingness to deal with failure and learn from it, anyone will likely have a difficult time learning to program.

Over time, it's natural to begin to think that these attributes are prerequisites -- things a person must have before he or she can learn to write programs. But I think that's wrong.

As someone else pointed out in the thread, too many people believe that to succeed in certain disciplines, one must be gifted, to possess an inherent talent for doing that kind of thing. Science, math, and computer science fit firmly in that set of disciplines for most people. Carol Dweck has shown that having such a "fixed" mindset of this sort prevents many people from sticking with these disciplines when they hit challenges, or even trying to learn them in the first place.

The attitude expressed in the quote above is counterproductive for teachers, whose job it is to help students learn things even when the students don't think they can.

When I talk to my students, I acknowledge that, to succeed in CS, you need to be persistent, diligent, patient, and willing to deal with failure and learn from it. But I approach these attributes from a growth mindset:

Persistence, diligence, patience, and willingness to learn from failure are habits anyone can develop with practice. Students can develop these habits regardless of their natural gifts or their previous education.

Aristotle said that excellence is not an act, but a habit. So are most of the attributes we need to succeed in CS. They are habits, not traits we are born with or actions we take.

Donald Knuth once said that only about 2 per cent of the population "resonates" with programming the way he does. That may be true. But even if most of us will never be part of Knuth's 2%, we can all develop the habits we need to program at a basic level. And a lot more than 2% are capable of building successful careers in the discipline.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

September 23, 2014 4:37 PM

The Obstacles in the Way of Teaching More Students to Program

All students should learn to program? Not so fast, says Larry Cuban in this Washington Post blog entry. History, including the Logo movement, illustrates several ways in which such a requirement can fail. I've discussed Cuban's article with a couple of colleagues, and all are skeptical. They acknowledge that he raises important issues, but in the end they offer a "yeah, but...". It is easy to imagine that things are different now, and the result will be similarly different.

I am willing to believe that things may be different this time. They always are. I've written favorably here in the past of the value of more students learning to program, but I've also been skeptical of requiring it. Student motivations change when they "have to take that class". And where will all the teachers come from?

In any case, it is wise to be alert to how efforts to increase the reach of programming instruction have fared. Cuban reminds us of some of the risks. One line in his article expresses what is, to my mind, the biggest challenge facing this effort:

Traditional schools adapt reforms to meet institutional needs.

Our K-12 school system is a big, complex organism (actually, fifty-one of them). It tends to keep moving in the direction of its own inertia. If a proposed reform fits its needs, the system may well adopt it. If it doesn't, but external forces push the new idea onto system, the idea is adapted -- assimilated into what the institution already wants to be, not what the reform actually promises.

We see this in the university all the time, too. Consider accountability measures such as student outcomes assessment. Many schools have adopted the language of SOA, but rarely do faculty and programs change all that much how they behave. They just find ways to generate reports that keep the external pressures at bay. The university and its faculty may well care about accountability, but they tend to keep on doing it the way they want to do it.

So, how can we maximize the possibility of substantive change in the effort to teach more students how to program, and not simply create a new "initiative" with frequent mentions in brochures and annual reports? Mark Guzdial has been pointing us in the right direction. Perhaps the most effective way to change K-12 schools is to change the teachers we send into the schools. We teach more people to be computing teachers, or prepare more teachers in the traditional subjects to teach computing. We prepare them to recognize opportunities to introduce computing into their courses and curricula.

In this sense, universities have an irreplaceable role to play in the revolution. We teach the teachers.

Big companies can fund programs such as and help us reach younger students directly. But that isn't enough. Google's CS4HS program has been invaluable in helping universities reach current K-12 teachers, but they are a small percentage of the installed base of teachers. In our schools of education, we can reach every future teacher -- if we all work together within and across university boundaries.

Of course, this creates a challenge at the meta-level. Universities are big, complex organisms, too. They tends to keep moving in the direction of their own inertia. Simply pushing the idea of programming instruction onto system from the outside is more likely to result in harmless assimilation than in substantive change. We are back to Cuban's square one.

Still, against all these forces, many people are working to make a change. Perhaps this time will be different after all.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

September 22, 2014 10:33 AM

Strange Loop 2014 Videos Are Up

generic Strange Loop logo

Wow. Strange Loop just ended Friday evening, and already videos of nearly all the talks are available on a YouTube channel. (A few have been delayed at the speaker's request.)

I regret missing the conference this year. I've been a regular attendee over the years and much enjoyed last year's edition. But it's probably just as well that the tickets sold out before I bought mine. My intro course has kept me pedaling full speed since school started, and I would have regretted missing a lab day and a class session just as we are getting to the meat of the course. I followed along with the conference on Twitter as time permitted.

The video titles foreshadow the usual treasure trove of Strange Loop content. It would be easier to list the talks I don't want to watch than the ones I do. A few I'll watch early on include Stephen Kell's "Liberating the Smalltalk Lurking in C and Unix", Stefanie Schirmer's "Dynamic Programming At Ease", Mark Allen's "All Of This Has Happened Before, and It Will All Happen Again", Julia Evans's "You Can Be a Kernel Hacker!", and Michael Nygard's "Simulation Testing".

An underrated advantage of actually attending a conference is not being able to be in two places at one time. Having to make a choice is sometimes a good thing; it helps us to preserve limited resources. The downside to the wonderfulness of having all the videos available on-line, for viewing at my leisure, is that I want to watch them all -- and I don't have enough leisure!

Posted by Eugene Wallingford | Permalink | Categories: Computing

September 12, 2014 1:49 PM

The Suffocating Gerbils Problem

I had never heard of the "suffocating gerbils" problem until I ran across this comment in a Lambda the Ultimate thread on mixing declarative and imperative approaches to GUI design. Peter Van Roy explained the problem this way:

A space rocket, like the Saturn V, is a complex piece of engineering with many layered subsystems, each of which is often pushed to the limits. Each subsystem depends on some others. Suppose that subsystem A depends on subsystem B. If A uses B in a way that was not intended by B's designers, even though formally B's specification is being followed by A, then we have a suffocating gerbils problem. The mental image is that B is implemented by a bunch of gerbils running to exhaustion in their hoops. A is pushing them to do too much.

I first came to appreciate the interrelated and overlapping functionality of engineered subsystems in graduate school, when I helped a fellow student build a software model of the fuel and motive systems of an F-18 fighter plane. It was quite a challenge for our modeling language, because the functions and behaviors of the systems were intertwined and did not follow obviously from the specification of components and connections. This challenge motivated the project. McDonnell Douglas was trying to understand the systems in a new way, in order to better monitor performance and diagnose failures. (I'm not sure how the project turned out...)

We suffocate gerbils at the university sometimes, too. Some functions depend on tenure-track faculty teaching occasional overloads, or the hiring of temporary faculty as adjuncts. When money is good, all is well. As budgets tighten, we find ourselves putting demands on these subsystems to meet other essential functions, such as advising, recruiting, and external engagement. It's hard to anticipate looming problems before they arrive in full failure; everything is being done according to specification.

Now there's a mental image: faculty gerbils running to exhaustion.

If you are looking for something new to read, check out some of Van Roy's work. His Concepts, Techniques, and Models of Computer Programming offers all kinds of cool ideas about programming language design and use. I happily second the sentiment of this tweet:

Note to self: read all Peter Van Roy's LtU comments in chronological order and build the things that don't exist yet:

There are probably a few PhD dissertations lurking in those comments.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

September 04, 2014 3:32 PM

Language Isn't Just for Experts

Stephen Ramsey wrote The Mythical Man-Finger, in defense of an earlier piece on the virtues of the command line. The gist of his argument is this:

... the idea that language is for power users and pictures and index fingers are for those poor besotted fools who just want toast in the morning is an extremely retrograde idea from which we should strive to emancipate ourselves.

Ramsay is an English professor who works in digital humanities. From the writings posted on his web site, it seems that he spends nearly as much time teaching and doing computing these days as he spends on the humanities. This opens him to objections from his colleagues, some of whom minimize the relevance of his perspective for other humanists by reminding him that he is a geek. He is one of those experts who can't see past his own expertise. We see this sort of rhetorical move in tech world all the time.

I think the case is quite the opposite. Ramsay is an expert on language. He knows that language is powerful, that language is more powerful than the alternatives in many contexts. When we hide language from our users, we limit them. Other tools can optimize for a small set of particular use cases, but they generally make it harder to step outside of those lines drawn by the creator of the tools: to combine tasks in novel ways, to extend them, to integrate them with other tools.

Many of my intro students are just beginning to see what knowing a programming language can mean. Giving someone language is one of the best ways to empower them, and also a great way to help them even see what is possible.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

August 30, 2014 7:43 AM

A Monad Sighting in Pop Literature

Lab experiments are invaluable in the hard sciences, in part because neutrinos and monads don't change their behavior when they are being watched; but humans do.

Several things ran through my mind when I read this sentence.

  • "Monads don't change their behavior when watched." Wow. The authors of this book must know a little functional programming.

  • Monads mentioned in the same sentence as neutrinos, which are fundamental particles of the universe? Oh, no. This will only make the smug functional programming weenies more smug.

  • Monads are part of the "hard sciences"? These authors really do get functional programming!

  • This sentence appears in a chapter called "The Three Hardest Words in the English Language". That joke writes itself.

  • Maybe I shouldn't be surprised to see this sentence. The book called Think Like a Freak.

I kid my monad-loving friends; I kid. The rest of the book is pretty good, too.

Posted by Eugene Wallingford | Permalink | Categories: Computing

July 21, 2014 10:52 AM

Wesley's Quoted Quote

My recent post Burn All Your Sermons was triggered by a quote taken out of context. Theologian John Wesley did not say:

Once in seven years I burn all my sermons...

He said:

"Once in seven years I burn all my sermons..."

Those "" make all the difference. Wesley wasn't saying that he himself burns all his sermons every seven years; he was talking about the practice doing so. Imagine the assistant of Wesley who, upon seeing this passage in the theologian's diary, burned all of Wesley's old sermons in an effort to ingratiate himself with the boss, only later to find out that Wesley very much intended to use them again. Fiery furnace, indeed.

This sort of indirection isn't important only for human communication. It is a key idea in computing. I wrote a blog post last year about such quotations and how this distinction is an important element in Jon Udell's notion of "thinking like the web". Thinking like the web isn't always foreign to the way most of us already think and work; sometimes it simply emphasizes a particular human practice that until now has been less common.

Studying a little computer science can help, though. Programmers have multiple ways of speaking indirectly about an action such as "burn all the sermons". In Scheme, I might express the program to burn all the sermons in a collection as:

(burn sermons)

We can quote this program, in much the same way that the "" above do, as:

'(burn sermons)

This is actually shorthand for (quote (burn sermons)). The result is a piece of data, much like Wesley's quotation of another person's utterance, that we can manipulate a variety of ways.

This sort of quotation trades on the distinction between data and process. In a post a few years back, I talked a bit about how this distinction is only a matter of perspective, that at a higher level data and program are two sides of the same coin.

However, we can also "quote" our sermon-burning program in a way that stays on the side of process. Consider this program:

(lambda () (burn sermons))

The result is a program that, when executed, will execute the sermon-burning program. Like the data version of the quote, it turns the original statement into something that we can talk about, pass around as a value, and manipulate in a variety of ways. But it does so by creating another program.

This technique, quite simple at its heart, plays a helpful role in the way many of computer language processors work.

Both techniques insert a level of indirection between a piece of advice -- burn all your sermons -- and its execution. That is a crucial distinction when we want to talk about an idea without asserting the idea's truth at that moment. John Wesley knew that, and so should we.

Posted by Eugene Wallingford | Permalink | Categories: Computing

July 16, 2014 2:11 PM

Burn All Your Sermons

Marketers and bridge players have their Rules of Seven. Teachers and preachers might, too, if they believe this old saw:

Once in seven years I burn all my sermons; for it is a shame if I cannot write better sermons now than I did seven years ago.

I don't have many courses in which I lecture uninterrupted for long periods of time. Most of my courses are a mixture of short lectures, student exercises, and other activities that explore or build upon whatever we are studying. Even when I have a set of materials I really like, which have been successful for me and my students in the past, I am forever reinventing them, tweaking and improving as we move through the course. This is in the same spirit as the rule of seven: surely I can make something better since the last time I taught the course.

Having a complete set of materials for a course to start from can be a great comfort. It can also be a straitjacket. The high-level structure of a course design limits how we think about the essential goals and topics of the course. The low-level structure generally optimizes for specific transitions and connections, which limits how easily we can swap in new examples and exercises.

Even as an inveterate tinkerer, I occasionally desire to break out of the straitjacket of old material and make a fresh start. Burn it all and start over. Freedom! What I need to remember will come back to me.

The adage quoted above tells us to do this regularly even if we don't feel the urge. The world changes around us. Our understanding grows. Our skills as a writer and storyteller grow. We can do better.

Of course, starting over requires time. It's a lot quicker to prep a course by pulling a prepped course out of an old directory of courses and cleaning it up around the edges. When I decide to redesign a course from bottom up, I usually have to set aside part of a summer to allow for long hours writing from scratch. This is a cost you have to take into account any time you create a new course.

Being in computer science makes it easier to force ourselves to start from scratch. While many of the principles of CS remain the same across decades, the practices and details of the discipline change all the time. And whatever we want to say about timeless principles, the undergrads in my courses care deeply about having some currency when they graduate.

In Fall 2006, I taught our intro course. The course used Java, which was the first language in our curriculum at that time. Before that, the last time I had taught the course, our first language was Pascal. I had to teach an entirely new course, even though many of the principles of programming I wanted to teach were the same.

I'm teaching our intro course again this fall for the first time since 2006. Python is the language of choice now. I suppose I could dress my old Java course in a Python suit, but that would not serve my students well. It also wouldn't do justice to the important ideas of the course, or Python. Add to this that I am a different -- and I hope better -- teacher and programmer now than I was eight years ago, and I have all the reasons I need to design a new course.

So, I am getting busy. Burn all the sermons.

Of course, we should approach the seven-year advice with some caution. The above passage is often attributed to theologian John Wesley. And indeed he did write it. However, as is so often the case, it has been taken out of context. This is what Wesley actually wrote in his journal:

Tuesday, September 1.--I went to Tiverton. I was musing here on what I heard a good man say long since--"Once in seven years I burn all my sermons; for it is a shame if I cannot write better sermons now than I could seven years ago." Whatever others can do, I really cannot. I cannot write a better sermon on the Good Steward than I did seven years ago; I cannot write a better on the Great Assize than I did twenty years ago; I cannot write a better on the Use of Money, than I did nearly thirty years ago; nay, I know not that I can write a better on the Circumcision of the Heart than I did five-and-forty years ago. Perhaps, indeed, I may have read five or six hundred books more than I had then, and may know a little more history, or natural philosophy, than I did; but I am not sensible that this has made any essential addition to my knowledge in divinity. Forty years ago I knew and preached every Christian doctrine which I preach now.

Note that Wesley attributes the passage to someone else -- and then proceeds to deny its validity in his own preaching! We may choose to adopt the Rule of Seven in our teaching, but we cannot do so with Wesley as our prophet.

I'll stick with my longstanding practice of building on proven material when that seems best, and starting from scratch whenever the freedom to tell a new story outweighs the value of what has worked for me and my students in the past.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

July 10, 2014 3:08 PM

The Passing of the Postage Stamp

In this New York Times article on James Baldwin's ninetieth birthday, scholar Henry Louis Gates laments:

On one hand, he's on a U.S. postage stamp; on the other hand, he's not in the Common Core.

I'm not qualified to comment on Baldwin and his place in the Common Core. In the last few months, I read several articles about and including Baldwin, and from those I have come to appreciate better his role in twentieth-century literature. But I also empathize with anyone trying to create a list of things that every American should learn in school.

What struck me in Gates's comment was the reference to the postage stamp. I'm old enough to have grown up in a world where the postage stamp held a position of singular importance in our culture. It enabled communication at a distance, whether geographical or personal. Stamps were a staple of daily life.

In such a world, appearing on a stamp was an honor. It indicated a widespread acknowledgment of a person's (or organization's, or event's) cultural impact. In this sense, the Postal Service's decision to include James Baldwin on a stamp was a sign of his importance to our culture, and a way to honor his contributions to our literature.

Alas, this would have been a much more significant and visible honor in the 1980s or even the 1990s. In the span of the last decade or so, the postage stamp has gone from relevant and essential to archaic.

When I was a boy, I collected stamps. It was a fun hobby. I still have my collection, even if it's many years out of date now. Back then, stamp collecting was a popular activity with a vibrant community of hobbyists. For all I know, that's still true. There's certainly still a vibrant market for some stamps!

But these days, whenever I use a new stamp, I feel as if I'm holding an anachronism in my hands. Computing technology played a central role in the obsolescence of the stamp, at least for personal and social communication.

Sometimes people say that we in CS need to a better job helping potential majors see the ways in which our discipline can be used to effect change in the world. We never have to look far to find examples. If a young person wants to be able to participate in how our culture changes in the future, they can hardly do better than to know a little computer science.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Personal

July 03, 2014 2:13 PM

Agile Moments: Conspicuous Progress and Partial Value

Dorian Taylor, in Toward a Theory of Design as Computation:

You can scarcely compress the time it takes to do good design. The best you can do is arrange the process so that progress is conspicuous and the partially-completed result has its own intrinsic value.

Taylor's piece is about an idea much bigger than simply software methodology, but this passage leapt off the page at me. It seems to embody two of the highest goals of the various agile approaches to making software: progress that is conspicuous and partial results that have intrinsic value to the user.

If you like ambition attempts to create a philosophy of design, check out the whole essay. Taylor connects several disparate sources:

  • Edwin Hutchins and Cognition in the Wild,
  • Donald Norman and Things That Make Us Smart, and
  • Douglas Hofstadter and Gödel, Escher, Bach
with the philosophy of Christopher Alexander, in particular Notes on the Synthesis of Form and The Nature of Order. Ambitious it is.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

July 02, 2014 4:31 PM

My Jacket Blurb for "Exercises in Programming Style"

On Monday, my copy of Crista Lopes's new book, Exercises in Programming Style, arrived. After blogging about the book last year, Crista asked me to review some early chapters. After I did that, the publisher graciously offered me a courtesy copy. I'm glad it did! The book goes well beyond Crista's talk at StrangeLoop last fall, with thirty three styles grouped loosely into nine categories. Each chapter includes historical notes and a reading list for going deeper. Readers of this blog know that I often like to go deeper.

I haven't had a chance to study any of the chapters deeply yet, so I don't have a detailed review. For now, let me share the blurb I wrote for the back cover. It gives a sense of why I was so excited by the chapters I reviewed last summer and by Crista's talk last fall:

It is difficult to appreciate a programming style until you see it in action. Cristina's book does something amazing: it shows us dozens of styles in action on the same program. The program itself is simple. The result, though, is a deeper understanding of how thinking differently about a problem gives rise to very different programs. This book not only introduced me to several new styles of thinking; it also taught me something new about the styles I already know well and use every day.

The best way to appreciate a style is to use it yourself. I think Crista's book opens the door for many programmers to do just that with many styles most of us don't use very often.

As for the blurb itself: it sounds a little stilted as I read it now, but I stand by the sentiment. It is very cool to see my blurb and name along side blurbs from James Noble and Grady Booch, two people whose work I respect so much. Very cool. Leave it to James to sum up his thoughts in a sentence!

While you are waiting for your copy of Crista's book to arrive, check out her recent blog entry on the evolution of CS papers in publication over the last 50+ years. It presents a lot of great information, with some nice images of pages from a few classics. It's worth a read.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

June 27, 2014 3:55 PM

Beautiful Words, File Format Edition

In The Great Works of Software, Paul Ford tells us that the Photoshop file format is

a fascinating hellish palimpsest.

"Palimpsest" is one of those words I seem always have to look up whenever I run across it. What a lyrical word.

After working with a student a few summers ago on a translator from Photoshop PSD format to HTML/CSS (mentioned in the first paragraph of this essay, I can second the assertion that PSD is fascinating and hellish. Likewise, however often it has changed over time, it looks in several places as if it is held together with bailing wire.

Ford said it better than I could have, though.

Posted by Eugene Wallingford | Permalink | Categories: Computing

June 25, 2014 2:03 PM

You Shouldn't Need a License to Program

In Generation Liminal, Dorian Taylor recalls how the World Wide Web arrived at the perfect time in his life:

It's difficult to appreciate this tiny window of opportunity unless you were present for it. It was the World-Wild West, and it taught me one essential idea: that I can do things. I don't need a license, and I don't need credentials. I certainly don't need anybody telling me what to do. I just need the operating manual and some time to read it. And with that, I can bring some amazing -- and valuable -- creations to life.

I predate the birth of the web. But when we turned on the computers at my high school, BASIC was there. We could program, and it seemed the natural thing to do. These days, the dominant devices are smart phones and iPads and tablets. Users begin their experience far away from the magic of creating. It is a user experience for consumers.

One day many years ago, my older daughter needed to know how many words she had written for a school assignment. I showed her and wc. She was amazed by its simplicity; it looked like nothing else she'd ever seen. She still uses it occasionally.

I spent several days last week watching middle schoolers -- play. They consumed other people's creations, including some tools my colleagues set up for them. They have creative minds, but for the most part it doesn't occur to them that they can create things, too.

We need to let them know they don't need our permission to start, or credentials defined by anyone else. We need to give them the tools they need, and the time to play with them. And, sometimes, we need to give them a little push to get started.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

June 23, 2014 3:13 PM

The Coder's High Beats The Rest

At least David Auerbach thinks so. One of the reasons is that programming has a self-perpetuating cycle of creation, implementation, repair, and new birth:

"Coding" isn't just sitting down and churning out code. There's a fair amount of that, but it's complemented by large chunks of testing and debugging, where you put your code through its paces and see where it breaks, then chase down the clues to figure out what went wrong. Sometimes you spend a long time in one phase or another of this cycle, but especially as you near completion, the cycle tightens -- and becomes more addictive. You're boosted by the tight feedback cycle of coding, compiling, testing, and debugging, and each stage pretty much demands the next without delay. You write a feature, you want to see if it works. You test it, it breaks. It breaks, you want to fix it. You fix it, you want to build the next piece. And so on, with the tantalizing possibility of -- just maybe! -- a perfect piece of code gesturing at you in the distance.

My experience is similar. I can get lost for hours in code, and come out tired but mentally energized. Writing has never given me that kind of high, but then I've not written a really long piece of prose in a long time. Perhaps writing fiction could give me the sort of high I experience when deep in a program.

What about playing games? Back in my younger days, I experienced incredible flow while playing chess for long stretches. I never approached master level play, but a good game could still take my mind to a different level of consciousness. That high differed from a coder's high, though, in that it left me tired. After a three-round day at a chess tournament, all I wanted to do was sleep.

Getting lost in a computer game gives me a misleading feeling of flow, but it differs from the chess high. When I come out of a session lost in most computer games, I feel destroyed. The experience doesn't give life the way coding does, or the way I imagine meditation does. I just end up feeling tired and used. Maybe that's what drug addiction feels like.

I was thinking about computer games even before reading Auerbach's article. Last week, I was sitting next to one of the more mature kids in our summer camp after he had just spent some time gaming, er, collecting data for our our study of internet traffic. We had an exchange that went something like this:

Student: I love this feeling. I'd like to create a game like this some day.

Eugene: You can!

Student: Really? Where?

Eugene: Here. A group of students in my class last month wrote a computer game next door. And it's way cooler than playing a game.

I was a little surprised to find that this young high schooler had no idea that he could learn computer programming at our university. Or maybe he didn't make the connection between computer games and computer programs.

In any case, this is one of the best reasons for us CS profs to get out of their university labs and classrooms and interact with younger students. Many of them have no way of knowing what computer science is, what they can do with computer science, or what computer science can do for them -- unless we show them!

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

June 20, 2014 1:27 PM

Programming Everywhere, Business Edition

Q: What do you call a company that has staff members with "programmer" or "software developer" in their titles?

A: A company.

Back in 2012, Alex Payne wrote What Is and Is Not A Technology Company to address a variety of issues related to the confounding of companies that sell technology with companies that merely use technology to sell something else. Even then, developing technology in house was a potential source of competitive advantage for many businesses, whether that involved modifying existing software or writing new.

The competitive value in being able to adapt and create software is only larger and more significant in the last two years. Not having someone on staff with "programmer" in the title is almost a red flag even for non-tech companies these days.

Those programmers aren't likely to have been CS majors in college, though. We don't produce enough. So we need to find a way to convince more non-majors to learn a little programming.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

June 19, 2014 2:11 PM

Yet Another Version of Alan Kay's Definition of "Object-Oriented"

In 2003, Stefan Ram asked Alan Kay to explain some of the ideas and history behind the term "object-oriented". Ram posted Kay's responses for all to see. Here is how Kay responded to the specific question, "What does 'object-oriented [programming]' mean to you?":

OOP to me means only messaging, local retention and protection and hiding of state-process, and extreme late-binding of all things.

Messaging and extreme late-binding have been consistent parts of Kay's answer to this question over the years. He has also always emphasized the encapsulated autonomy of objects, with analogy to cells from biology and nodes on the internet. As Kay has said many times, in his conception of the basic unit of computation is a whole computer.

For some reason, I really like the way Kay phrased the encapsulated autonomy clause in this definition: local retention and protection and hiding of state-process. It's not poetry or anything, but it has a rhythm.

Kay's e-mail mentions another of Kay's common themes, that most computer scientists didn't take full advantage of the idea of objects. Instead, we stayed too close to the dominant data-centric perspective. I often encounter this with colleagues who confound object-oriented programming with abstract data types. A system designed around ADTs will not offer the same benefits that Kay envisions for objects defined by their interactions.

In some cases, the words we adopted for OO concepts may have contributed to the remaining bias toward data, even if unintentionally. For example, Kay thinks that the term "polymorphism" hews too closely to the standard concept of a function to convey the somewhat different notion of an object as embodying multiple algebras.

Kay's message also mentions two projects I need to learn more about. I've heard of Robert Balzer's Dataless Programming paper but never read it. I've heard of GEDANKEN, a programming language project by John Reynolds, but never seen any write-up. This time I downloaded GEDANKEN: A Simple Typeless Language Which Permits Functional Data Structures and Coroutines, Reynolds's tech report from Argonne National Lab. Now I am ready to become a little better informed than I was this morning.

The messages posted by Ram are worth a look. They serve as a short precursor to (re-)reading Kay's history of Smalltalk paper. Enjoy!

Posted by Eugene Wallingford | Permalink | Categories: Computing

June 17, 2014 2:38 PM

Cookies, Games, and Websites: A Summer Camp for Kids

Cut the Rope 2 logo

Today is the first day of Cookies, Games, and Websites, a four-day summer camp for middle-school students being offered by our department. A colleague of mine developed the idea for a workshop that would help kids of that age group understand better what goes on when they play games on their phones and tablets. I have been helping, as a sounding board for ideas during the prep phase and now as a chaperone and helper during the camp. A local high school student has been providing much more substantial help, setting up hardware and software and serving as a jack-of-all-trades.

The camp's hook is playing games. To judge from this diverse group of fifteen students from the area, kids this age already know very well how to download, install, and play games. Lots of games. Lots and lots of games. If they had spent as much time learning to program as they seem to have spent playing games, they would be true masters of the internet.

The first-order lesson of the camp is privacy. Kids this age play a lot of games, but they don't have a very good idea how much network traffic a game like Cut the Rope 2 generates, or how much traffic accessing Instagram generates. Many of their apps and social websites allow them to exercise some control over who sees what in their space, but they don't always know what that means. More importantly, they don't realize how important all this all is, because they don't know how much traffic goes on under the hood when they use their mobiles devices -- and even when they don't!

The second-order lesson of the camp, introduced as a means to an end, is computing: the technology that makes communication on the web possible, and some of the tools they can use to look at and make sense of the network traffic. We can use some tools they already know and love, such as Google maps, to visualize the relevant data.

This is a great idea: helping young people understand better the technology they use and why concepts like privacy matter to them when they are using that technology. If the camp is successful, they will be better-informed users of on-line technology, and better prepared to protect their identities and privacy. The camp should be a lot of fun, too, so perhaps one or two of them will be interested diving deeper into computer science after the camp is over.

This morning, the campers learned a little about IP addresses and domain names, mostly through interactive exercises. This afternoon, they are learning a little about watching traffic on the net and then generating traffic by playing some of their favorite games. Tomorrow, we'll look at all the traffic they generated playing, as well as all the traffic generated while their tablets were idle overnight.

We are only three-fourths of the way through Day 1, and I have already learned my first lesson: I really don't want to teach middle school. The Grinch explains why quite succinctly: noise, noise, NOISE! One thing seems to be true of any room full of fifteen middle-school students: several of them are talking at any given time. They are fun people to be around, but they are wearing me out...

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

June 05, 2014 2:45 PM

Choosing the Right Languages for Early CS Instruction is Important

In today's ACM interview, Donald Knuth identifies one of the problems he has with computer science instruction:

Similarly, the most common fault in computer classes is to emphasize the rules of specific programming languages, instead of to emphasize the algorithms that are being expressed in those languages. It's bad to dwell on form over substance.

I agree. The challenges are at least two in number:

  • ... finding the right level of support for the student learning his or her first language. It is harder for students to learn their first language than many people realize until after they've tried to teach them.

  • ... helping students develop the habit and necessary skills to learn new languages on their own with some facility. For many, this involves overcoming the fear they feel until they have done it on their own a time or two.

Choosing the right languages can greatly help in conquering Challenges 1 and 2. Choosing the wrong languages can make overcoming them almost impossible, if only because we lose students before they cross the divide.

I guess that makes choosing the right languages Challenge 3.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

May 29, 2014 2:14 PM

Invention, One Level Down

Brent Simmons wrote a blog entry on his time at UserLand. After describing a few of the ideas that founder Dave Winer created and extending, such as RSS and blogging, Simmons said this about Winer:

The tech was his invention too: he built the thing he needed to be able to build other things.

This is among the highest praise one can bestow on an inventor. It's also one of the things I like about computer science. The hallmark of so many interesting advances in computing is the creation of a technology or language that makes the advance possible. Sometimes the enabling technology turns out to be pretty important in its own right. Sometimes, it's a game changer. But even when it is only a scaffold to something bigger, it needed to be created.

Posted by Eugene Wallingford | Permalink | Categories: Computing

May 28, 2014 4:20 PM

Programming for Everyone, Intro Physics Edition

Rhett Allain asked his intro physics students to write a short bit of Python code to demonstrate some idea from the course, such as the motion of an object with a constant force, or projectile motion with air resistance. Apparently, at least a few complained: "Wait! I'm not a computer scientist." That caused Allain to wonder...

I can just imagine the first time a physics faculty told a class that they needed to draw a free body diagram of the forces on an object for the physics solutions. I wonder if a student complained that this was supposed to be a physics class and not an art class.

As Allain points out, the barriers that used to prevent students from doing numerical calculations in computer programs have begun to disappear. We have more accessible languages now, such as Python, and powerful computers are everywhere, capable of running VPython and displaying beautiful visualizations.

About all that remains is teaching all physics students, even the non-majors, a little programming. The programs they write are simply another medium through which they can explore physical phenomena and perhaps come to understand them better.

Allain is exactly right. You don't have to be an artist to draw simple diagrams or a mathematician to evaluate an integral. All students accept, if grudgingly, that people might reasonably expect them to present an experiment orally in class.

Students don't have to be "writers", either, in order for teachers or employers to reasonably expect them to write an essay about physics or computer science. Even so, you might be surprised how many physics and computer science students complain if you ask them to write an essay. And if you dare expect them to spell words correctly, or to write prose somewhat more organized than Faulkner stream of consciousness -- stand back.

(Rant aside, I have been quite lucky this May term. I've had my students write something for me every night, whether a review of something they've read or a reflection on the practices they are struggling to learn. There's been nary a complaint, and most of their writings have been organized, clear, and enjoyable to read.)

You don't have to be a physicist to like physics. I hope that most educated adults in the 21st century understand how the physical world works and appreciate the basic mechanisms of the universe. I dare to hope that many of them are curious enough to want to learn more.

You also don't have to be a computer programmer, let alone a computer scientist, to write a little code. Programs are simply another medium through which we can create and express ideas from across the spectrum of human thought. Hurray to Allain for being in the vanguard.


Note. Long-time readers of this blog may recognize the ideas underlying Allain's approach to teaching introductory physics. He uses Matter and Interactions, a textbook and set of supporting materials created by Ruth Chabay and Bruce Sherwood. Six years ago, I wrote about some of Chabay's and Sherwood's ideas in an entry on creating a dialogue between science and CS and mentioned the textbook project in an entry on scientists who program. These entries were part of a report on my experiences attending SECANT, a 2007 NSF workshop on the intersection of science, computation, and education.

I'm glad to see that the Matter and Interactions project continued to fruition and has begun to seep into university physics instruction. It sounds like a neat way to learn physics. It's also a nice way to pick up a little "stealth programming" along the way. I can imagine a few students creating VPython simulations and thinking, "Hey, I'd like to learn more about this programming thing..."

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

May 25, 2014 12:03 PM

CS Prof From Iowa Was a 'Heroine of Computing' -- and a Nun

While cleaning up the house recently for a family visit, I came across a stack of newspaper articles I'd saved from last fall. Among them was an article about a September 7, 2013, exhibition at The National Museum of Computing in Bletchley Park, Milton Keynes, England. The exhibition was titled "Celebrating the Heroines of Computing". That alone would have made the article worth clipping, but it had a closer connection to me: it featured a CS professor from the state of Iowa, who was also a Catholic nun.

Sister Mary Kenneth Keller, with Paul Laube, MD, undated

Sister Mary Kenneth Keller was a professed member of the Sisters of Charity of the Blessed Virgin Mary, an order of nuns based in Dubuque, Iowa. If you have had the privilege of working or studying with nuns, you know that they are often amazing people. Sister Mary Kenneth certainly was. She was also a trailblazer who studied computer science before it was a thing and helped to create a CS department:

As the first person to receive a Ph.D. in computer science from the University of Wisconsin-Madison, she was a strong advocate for women entering the field of computer science. For nearly 20 years she served as chair of the newly-created computer science department at Clarke University and was among the first to recognize the future importance of computers in the sciences, libraries and business. Under her leadership at Clarke, a master's degree program in computer applications in education was included.

Claims that some individual was the "first person to receive a Ph.D. in computer science" have been relatively common over the years. The Department of Computer Science at Wisconsin has a page listing Ph.D.'s conferred, 1965-1970, which list Sister Mary Kenneth first, for a dissertation titled "Inductive Inference on Computer Generated Patterns". But that wasn't her only first; this ACM blog piece by Ralph London asserts that Keller is the first woman to receive a Ph.D. in CS anywhere in the US, and one of the first two US CS Ph.D.s overall.

This bit of history is only a small part of Keller's life in academia and computing. She earned a master's degree in math at DePaul University in the early 1950s. In 1958, she worked at the Dartmouth University Computer Center as part of an NSF workshop, during which time she participated in the development of the BASIC programming language. She wrote four books on computing and served as consultant for a group of business and government organizations that included the city of Dubuque and the state of Illinois.

Sister Mary Kenneth spent her career on the faculty of Clarke University, apparently chairing the Department of Computer Science until her retirement. The university's computer center is named the Keller Computer Center and Information Service in her honor, as is a scholarship for students of computing.

I'd been in Iowa twenty years before I first heard this story of an Iowan's role in the history of computing. Her story also adds to the history of women in computing and, for me, creates a whole new area in the history of computing: women religious. A pretty good find for cleaning up the house.


The passage quoted above come from an article by Jody Iler, "BVM to be Featured as One of the 'Heroines of Computing'", which ran some time last fall in The Witness, the newspaper of the Archdiocese of Dubuque. I found substantially the same text on a news archive page on the web site of the Sisters of Charity, BVM. There is, of course, a Wikipedia page for Sister Mary Kenneth that reports many of the same details of her life.

The photo above, which appears both in Iler's article and on the web site, shows Sister Mary Kenneth with Dr. Paul Laube, a flight surgeon from Dubuque who consulted with her on some computing matter. (Laube's obituary indicates he lived an interesting life as well.) In the article, the photo is credited to Clarke University.

Posted by Eugene Wallingford | Permalink | Categories: Computing

May 07, 2014 3:39 PM

Thinking in Types, and Good Design

Several people have recommended Pat Brisbin's Thinking in Types for programmers with experience in dynamically-typed languages who are looking to grok Haskell-style typing. He wrote it after helping one of his colleagues of mine was get unstuck with a program that "seemed conceptually simple but resulted in a type error" in Haskell when implemented in a way similar to a solution in a language such as Python or Ruby.

This topic is of current interest to me at a somewhat higher level. Few of our undergrads have a chance to program in Haskell as a part of their coursework, though a good number of them learn Scala while working at a local financial tech company. However, about two-thirds of undergrads now start with a one or two semesters of Python, and types are something of a mystery to them. This affects their learning of Java and colors how they think about types if they take my course on programming languages.

So I read this paper. I have two comments.

First, let me say that I agree with my friends and colleagues who are recommending this paper. It is a clear, concise, and well-written description of how to use Haskell's types to think about a problem. It uses examples that are concrete enough that even our undergrads could implement with a little help. I may use this as a reading in my languages course next spring.

Second, I think think this paper does more than simply teach people about types in a Haskell-like language. It also gives a great example of how thinking about types can help programmers create better designs for their programs, even if they are working in an object-oriented language! Further, it hits right at the heart of the problem we face these days, with students who are used to working in scripting languages that provide high-level but very generic data structures.

The problem that Brisbin addresses happens after he helps his buddy create type classes and two instance classes, and they reach this code:

    renderAll [ball, playerOne, playerTwo]

renderAll takes a list of values that are Render-able. Unfortunately, in this case, the arguments come from two different classes... and Haskell does not allow heterogeneous lists. We could try to work around this feature of Haskell and "make it fit", but as Brisbin points out, doing so would cause you to lose the advantages of using Haskell in the first place. The compiler wouldn't be able to find errors in the code.

The Haskell way to solve the problem is to replace the generic list of stuff we pass to renderAll with a new type. With a new Game type that composes a ball with two players, we are able to achieve several advantages at once:

  • create a polymorphic render method for Game that passes muster with the type checker
  • allow the type checker to ensure that this element of our program is correct
  • make the program easier to extend in a type-safe way
  • our program is correct
  • and, perhaps most importantly, express the intent of the program more clearly

It's this last win that jumped off the page for me. Creating a Game class would give us a better object-oriented design in his colleague's native language, too!

Students who become accustomed to programming in languages like Python and Ruby often become accustomed to using untyped lists, arrays, hashes, and tuples as their go-to collections. They are oh, so, handy, often the quickest route to a program that works on the small examples at hand. But those very handy data structures promote sloppy design, or at least enable it; they make it easy not to see very basic objects living in the code.

Who needs a Game class when a Python list or Ruby array works out of the box? I'll tell you: you do, as soon as you try to almost anything else in your program. Otherwise, you begin working around the generality of the list or array, writing code to handle special cases really aren't special cases at all. They are simply unbundled objects running wild in the program.

Good design is good design. Most of the features of a good design transcend any particular programming style or language.

So: This paper is a great read! You can use it to learn better how to think like a Haskell programmer. And you can use it to learn even if thinking like a Haskell programmer is not your goal. I'm going to use it, or something like it, to help my students become better OO programmers.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

April 09, 2014 3:26 PM

Programming Everywhere, Vox Edition

In a report on the launch of Vox Media, we learn that line between software developers and journalists at Vox is blurred, as writers and reporters work together "to build the tools they require".

"It is thrilling as a journalist being able to envision a tool and having it become a real thing," Mr. Topolsky said. "And it is rare."

It will be less rare in the future. Programming will become a natural part of more and more people's toolboxes.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

April 04, 2014 12:43 PM

Creative Recombination of Existing Ideas

In a post on his move to California, Jon Udell notes that he may be out of step with the dominant view of the tech industry there:

And I think differently about innovation than Silicon Valley does. I don't think we lack new ideas. I think we lack creative recombination of proven tech, and the execution and follow-through required to surface its latent value.

As he found with the Elm City project, sometimes a good idea doesn't get traction quickly, even with sustained effort. Calendar aggregation seems like such a win even for a university the size of mine, yet a lot of folks don't get it. It's hard to know whether the slowness results from the idea, the technology, or the general resistance of communities to change how they operate.

In any case, Udell is right: there is a lot of latent value in the "creative recombination" of existing ideas. Ours is a remix culture, too. That's why it's so important to study widely in and out of computing, to build the base of tools needed to have a great idea and execute on it.

Posted by Eugene Wallingford | Permalink | Categories: Computing

March 31, 2014 3:21 PM

Programming, Defined and Re-imagined

By Chris Granger of Light Table fame:

Programming is our way of encoding thought such that the computer can help us with it.

Read the whole piece, which recounts Granger's reflection after the Light Table project left him unsatisfied and he sought answers. He concludes that we need to re-center our idea of what programming is and how we can make it accessible to more people. Our current idea of programming doesn't scale because, well,

It turns out masochism is a hard sell.

Every teacher knows this. You can sell masochism to a few proud souls, but not to anyone else.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

March 12, 2014 3:55 PM

Not Content With Content

Last week, the Chronicle of Higher Ed ran an article on a new joint major at Stanford combining computer science and the humanities.

[Students] might compose music or write a short story and translate those works, through code, into something they can share on the web.

"For students it seems perfectly natural to have an interest in coding," [the program's director] said. "In one sense these fields might feel like they're far apart, but they're getting closer and closer."

The program works in both directions, by also engaging CS students in the societal issues created by ubiquitous networks and computing power.

We are doing something similar at my university. A few years ago, several departments began to collaborate on a multidisciplinary program called Interactive Digital Studies which went live in 2012. In the IDS program, students complete a common core of courses from the Communication Studies department and then take "bundles" of coursework involving digital technology from at least two different disciplines. These areas of emphasis enable students to explore the interaction of computing with various topics in media, the humanities, and culture.

Like Stanford's new major, most of the coursework is designed to work at the intersection of disciplines, rather than pursuing disciplines independently, "in parallel".

The initial version of the computation bundle consists of an odd mix of application tools and opportunities to write programs. Now that the program is in place, we are finding that students and faculty alike desire more depth of understanding about programming and development. We are in the process of re-designing the bundle to prepare students to work in a world where so many ideas become web sites or apps, and in which data analytics plays an important role in understanding what people do.

Both our IDS program and Stanford's new major focus on something that we are seeing increasingly at universities these days: the intersections of digital technology and other disciplines, in particular the humanities. Computational tools make it possible for everyone to create more kinds of things, but only if people learn how to use new tools and think about their work in new ways.

Consider this passage by Jim O'Loughlin, a UNI English professor, from a recent position statement on the the "digital turn" of the humanities:

We are increasingly unlikely to find writers who only provide content when the tools for photography, videography and digital design can all be found on our laptops or even on our phones. It is not simply that writers will need to do more. Writers will want to do more, because with a modest amount of effort they can be their own designers, photographers, publishers or even programmers.

Writers don't have to settle for producing "content" and then relying heavily on others to help bring the content to an audience. New tools enable writers to take greater control of putting their ideas before an audience. But...

... only if we [writers] are willing to think seriously not only about our ideas but about what tools we can use to bring our ideas to an audience.

More tools are within the reach of more people now than ever before. Computing makes that possible, not only for writers, but also for musicians and teachers and social scientists.

Going further, computer programming makes it possible to modify existing tools and to create new tools when the old ones are not sufficient. Writers, musicians, teachers, and social scientists may not want to program at that level, but they can participate in the process.

The critical link is preparation. This digital turn empowers only those who are prepared to think in new ways and to wield a new set of tools. Programs like our IDS major and Stanford's new joint major are among the many efforts hoping to spread the opportunities available now to a larger and more diverse set of people.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

March 08, 2014 10:18 AM

Sometimes a Fantasy

This week I saw a link to The Turing School of Software & Design, "a seven-month, full-time program for people who want to become professional developers". It reminded me of Neumont University, a ten-year-old school that offers a B.S. degree program in Computer science that students can complete in two and a half years.

While riding the bike, I occasionally fantasize about doing something like this. With the economics of universities changing so quickly [ 1 | 2 ], there is an opportunity for a new kind of higher education. And there's something appealing about being able to work closely with a cadre of motivated students on the full spectrum of computer science and software development.

This could be an accelerated form of traditional CS instruction, without the distractions of other things, or it could be something different. Traditional university courses are pretty confining. "This course is about algorithms. That one is about programming languages." It would be fun to run a studio in which students serve as apprentices making real stuff, all of us learning as we go along.

A few years ago, one of our ChiliPLoP hot topic groups conducted a greenfield thought experiment to design an undergrad CS program outside of the constraints of any existing university structure. Student advancement was based on demonstrating professional competencies, not completing packaged courses. It was such an appealing idea! Of course, there was a lot of hard work to be done working out the details.

My view of university is still romantic, though. I like the idea of students engaging the great ideas of humanity that lie outside their major. These days, I think it's conceivable to include the humanities and other disciplines in a new kind of CS education. In a recent blog entry, Hollis Robbins floats the idea of Home College for the first year of a liberal arts education. The premise is that there are "thousands of qualified, trained, energetic, and underemployed Ph.D.s [...] struggling to find stable teaching jobs". Hiring a well-rounded tutor could be a lot less expensive than a year at a private college, and more lucrative for the tutor than adjuncting.

Maybe a new educational venture could offer more than targeted professional development in computing or software. Include a couple of humanities profs, maybe some a social scientist, and it could offer a more complete undergraduate education -- one that is economical both in time and money.

But the core of my dream is going broad and deep in CS without the baggage of a university. Sometimes a fantasy is all you need. Other times...

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

March 01, 2014 11:35 AM

A Few Old Passages

I was looking over a couple of files of old notes and found several quotes that I still like, usually from articles I enjoyed as well. They haven't found their way into a blog entry yet, but they deserve to see the light of day.

Evidence, Please!

From a short note on the tendency even among scientists to believe unsubstantiated claims, both in and out of the professional context:

It's hard work, but I suspect the real challenge will lie in persuading working programmers to say "evidence, please" more often.

More programmers and computer scientists are trying to collect and understand data these days, but I'm not sure we've made much headway in getting programmers to ask for evidence.

Sometimes, Code Before Math

From a discussion of The Expectation Maximization Algorithm:

The code is a lot simpler to understand than the math, I think.

I often understand the language of code more quickly than the language of math. Reading, or even writing, a program sometimes helps me understand a new idea better than reading the math. Theory is, however, great for helping me to pin down what I have learned more formally.

Grin, Wave, Nod

From Iteration Inside and Out, a review of the many ways we loop over stuff in programs:

Right now, the Rubyists are grinning, the Smalltalkers are furiously waving their hands in the air to get the teacher's attention, and the Lispers are just nodding smugly in the back row (all as usual).

As a Big Fan of all three languages, I am occasionally conflicted. Grin? Wave? Nod? Look like the court jester by doing all three simultaneously?

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

February 21, 2014 3:35 PM

Sticking with a Good Idea

My algorithms students and I recently considered the classic problem of finding the closest pair of points in a set. Many of them were able to produce a typical brute-force approach, such as:

    minimum ← 
    for i ← 1 to n do
        for j ← i+1 to n do
            distance ← sqrt((x[i] - x[j])² + (y[i] - y[j])²)
            if distance < minimum then
               minimum ← distance
               first   ← i
               second  ← j
    return (first, second)

Alas, this is an O(n²) process, so we considered whether we might do better with a divide-and-conquer approach. It did not look promising, though. Divide-and-conquer doesn't let us solve the sub-problems independently. What if the closest pair straddles two partitions?

This is a common theme in computing, and problem solving more generally. We try a technique only to find that it doesn't quite work. Something doesn't fit, or a feature of the domain violates a requirement of the technique. It's tempting in such cases to give up and settle for something less.

Experienced problem solvers know not to give up too quickly. Many of the great advances in computing came under conditions just like this. Consider Leonard Kleinrock and the theory of packet switching.

In a Computing Conversations podcast published last year, Kleinrock talks about his Ph.D. research. He was working on the problem of how to support a lot of bursty network traffic on a shared connection. (You can read a summary of the podcast in an IEEE Computer column also published last year.)

His wonderful idea: apply the technique of time sharing from multi-user operating systems. The system could break all messages into "packets" of a fixed size, let messages take turns on the shared line, then reassemble each message on the receiving end. This would give every message a chance to move without making short messages wait too long behind large ones.

Thus was born the idea of packet switching. But there was a problem. Kleinrock says:

I set up this mathematical model and found it was analytically intractable. I had two choices: give up and find another problem to work on, or make an assumption that would allow me to move forward. So I introduced a mathematical assumption that cracked the problem wide open.

His "independence assumption" made it possible for him to complete his analysis and optimize the design of a packet-switching network. But an important question remained: Was his simplifying assumption too big a cheat? Did it skew the theoretical results in such a way that his model was no longer a reasonable approximation of how networks would behave in the real world?

Again, Kleinrock didn't give up. He wrote a program instead.

I had to write a program to simulate these networks with and without the assumption. ... I simulated many networks on the TX-2 computer at Lincoln Laboratories. I spent four months writing the simulation program. It was a 2,500-line assembly language program, and I wrote it all before debugging a single line of it. I knew if I didn't get that simulation right, I wouldn't get my dissertation.

High-stakes programming! In the end, Kleinrock was able to demonstrate that his analytical model was sufficiently close to real-world behavior that his design would work. Every one of us reaps the benefit of his persistence every day.

Sometimes, a good idea poses obstacles of its own. We should not let those obstacles beat us without a fight. Often, we just have to find a way to make it work.

This lesson applies quite nicely to using divide-and-conquer on the closest pairs problem. In this case, we don't make a simplifying assumption; we solve the sub-problem created by our approach:

After finding a candidate for the closest pair, we check to see if there is a closer pair straddling our partitions. The distance between the candidate points constrains the area we have to consider quite a bit, which makes the post-processing step feasible. The result is an O(n log n) algorithm that improves significantly on brute force.

This algorithm, like packet switching, comes from sticking with a good idea and finding a way to make it work. This is a lesson every computer science student and novice programmer needs to learn.

There is a complementary lesson to be learned, of course: knowing when to give up on an idea and move on to something else. Experience helps us tell the two situations apart, though never with perfect accuracy. Sometimes, we just have to follow an idea long enough to know when it's time to move on.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

February 19, 2014 4:12 PM

Teaching for the Perplexed and the Traumatized

Teaching for the Perplexed and the Traumatized

On need for empathy when writing about math for the perplexed and the traumatized, Steven Strogatz says:

You have to help them love the questions.

Teachers learn this eventually. If students love the questions, they will do an amazing amount of working searching for answers.

Strogatz is writing about writing, but everything he says applies to teaching as well, especially teaching undergraduates and non-majors. If you teach only grad courses in a specialty area, you may be able to rely on the students to provide their own curiosity and energy. Otherwise having empathy, making connections, and providing Aha! moments are a big part of being successful in the classroom. Stories trump formal notation.

This semester, I've been trying a particular angle on achieving this trifecta of teaching goodness: I try to open every class session with a game or puzzle that the students might care about. From there, we delve into the details of algorithms and theoretical analysis. I plan to write more on this soon.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

February 02, 2014 5:19 PM

Things That Make Me Sigh

In a recent article unrelated to modern technology or the so-called STEM crisis, a journalist writes:

Apart from mathematics, which demands a high IQ, and science, which requires a distinct aptitude, the only thing that normal undergraduate schooling prepares a person for is... more schooling.


On the one hand, this seems to presume that one need neither a high IQ nor any particular aptitude to excel in any number of non-math and science disciplines.

On the other, it seems to say that if one does not have the requisite characteristics, which are limited to a lucky few, one need not bother with computer science, math or science. Best become a writer or go into public service, I guess.

I actually think that the author is being self-deprecating, at least in part, and that I'm probably reading too much into one sentence. It's really intended as a dismissive comment on our education system, the most effective outcome of which often seems to be students who are really good at school.

Unfortunately, the attitude expressed about math and science is far too prevalent, even in our universities. It demeans our non-scientists as well as our scientists and mathematicians. It also makes it even harder to convince younger students that, with a little work and practice, they can achieve satisfying lives and careers in technical disciplines.

Like I said, sigh.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

January 27, 2014 11:39 AM

The Polymath as Intellectual Polygamist

Carl Djerassi, quoted in The Last Days of the Polymath:

Nowadays people [who] are called polymaths are dabblers -- are dabblers in many different areas. I aspire to be an intellectual polygamist. And I deliberately use that metaphor to provoke with its sexual allusion and to point out the real difference to me between polygamy and promiscuity.

On this view, a dilettante is merely promiscuous, making no real commitment to any love interest. A polymath has many great loves, and loves them all deeply, if not equally.

We tend to look down on dilettantes, but they can perform a useful service. Sometimes, making a connection between two ideas at the right time and in the right place can help spur someone else to "go deep" with the idea. Even when that doesn't happen, dabbling can bring great personal joy and provide more substantial entertainment than a lot of pop culture.

Academics are among the people these days with a well-defined social opportunity to be explore at least two areas deeply and seriously: their chosen discipline and teaching. This is perhaps the most compelling reason to desire a life in academia. It even offers a freedom to branch out into new areas later in one's career that is not so easily available to people who work in industry.

These days, it's hard to be a polymath even inside one's own discipline. To know all sub-areas of computer science, say, as well as the experts in those sub-areas is a daunting challenge. I think back to the effort my fellow students and I put in over the years that enabled us to take the Ph.D. qualifying exams in CS. I did quite well across the board, but even then I didn't understand operating systems or programming languages as well as experts in those areas. Many years later, despite continued reading and programming, the gap has only grown.

I share the vague sense of loss, expressed by the author of the article linked to above, of a time when one human could master multiple areas of discourse and make fundamental advances to several. We are certainly better off for collective understanding the world so much much better, but the result is a blow to a certain sort of individual mind and spirit.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

January 26, 2014 3:05 PM

One Reason We Need Computer Programs

Code bridges the gap between theory and data. From A few thoughts on code review of scientific code:

... there is a gulf of unknown size between the theory and the data. Code is what bridges that gap, and specifies how edge cases, weird features of the data, and unknown unknowns are handled or ignored.

I learned this lesson the hard way as a novice programmer. Other activities, such as writing and doing math, exhibit the same characteristic, but it wasn't until I started learning to program that the gap between theory and data really challenged me.

Since learning to program myself, I have observed hundreds of CS students encounter this gap. To their credit, they usually buckle down, work hard, and close the gap. Of course, we have to close the gap for every new problem we try to solve. The challenge doesn't go away; it simply becomes more manageable as we become better programmers.

In the passage above, Titus Brown is talking to his fellow scientists in biology and chemistry. I imagine that they encounter the gap between theory and data in a new and visceral way when they move into computational science. Programming has that power to change how we think.

There is an element of this, too, in how techies and non-techies alike sometimes lose track of how hard it is to create a successful start up. You need an idea, you need a programmer, and you need a lot of hard work to bridge the gap between idea and executed idea.

Whether doing science or starting a company, the code teaches us a lot about out theory. The code makes our theory better.

As Ward Cunningham is fond of saying, it's all talk until the tests run.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

January 23, 2014 4:14 PM

The CS Mindset

Chad Orzel often blogs about the physics mindset, the default orientation that physicists tend to have toward the world, and the way they think about and solve problems. It is fun to read a scientist talking about doing science.

Earlier this week I finally read this article about the popularity of CS50, an intro CS course at Harvard. It's all about how Harvard is "is righting an imbalance in academia" by finally teaching practical skills to its students. When I read:

"CS50 made me look at Harvard with new eyes," Guimaraes said.

That is a sea change from what Harvard represented to the outside world for decades: the guardian of a classic education, where the value of learning is for its own sake.

I sighed audibly, loud enough for the students on the neighboring exercise equipment to hear. A Harvard education used to be about learning only for its own sake, but now students can learn practical stuff, too. Even computer programming!

As I re-read the article now, I see that it's not as blunt as that throughout. Many Harvard students are learning computing because of the important role it plays in their futures, whatever their major, and they understand the value of understanding it better. But there are plenty of references to "practical ends" and Harvard's newfound willingness to teach practical skills it once considered beneath it.

Computer programming is surely one of those topics old Charles William Eliot would deem unworthy of inclusion in Harvard's curriculum.

I'm sensitive to such attitudes because I think computer science is and should be more. If you look deeper, you will see that the creators of CS50 think so, too. On its Should I take CS50? FAQ page, we find:

More than just teach you how to program, this course teaches you how to think more methodically and how to solve problems more effectively. As such, its lessons are applicable well beyond the boundaries of computer science itself.

The next two sentences muddy the water a bit, though:

That the course does teach you how to program, though, is perhaps its most empowering return. With this skill comes the ability to solve real-world problems in ways and at speeds beyond the abilities of most humans.

With this skill comes something else, something even more important: a discipline of thinking and a clarity of thought that are hard to attain when you learn "how to think more methodically and how to solve problems more effectively" in the abstract or while doing almost any other activity.

Later the same day, I was catching up on a discussion taking place on the PLT-EDU mailing list, which is populated by the creators, users, and fans of the Racket programming language and the CS curriculum designed in tandem with it. One poster offered an analogy for talking to HS students about how and why they are learning to program. A common theme in the discussion that ensued was to take the conversation off of the "vocational track". Why encourage students to settle for such a limiting view of what they are doing?

One snippet from Matthias Felleisen (this link works only if you are a member of the list) captured my dissatisfaction with the tone of the Globe article about CS50:

If we require K-12 students to take programming, it cannot be justified (on a moral basis) with 'all of you will become professional programmers.' I want MDs who know the design recipe, I want journalists who write their articles using the design recipe, and so on.

The "design recipe" is a thinking tool students learn in Felleisen "How to Design Programs" curriculum. It is a structured way to think about problems and to create solutions. Two essential ideas stand out for me:

  • Students learn the design recipe in the process of writing programs. This isn't an abstract exercise. Creating a working computer program is tangible evidence that student has understood the problem and created a clear solution.
  • This way of thinking is valuable for everyone. We will all better off if our doctors, lawyers, and journalists are able to think this way.

This is one of my favorite instantiations of the vague term computational thinking so many people use without much thought. It is a way of thinking about problems both abstractly and concretely, that leads to solutions that we have verified with tests.

You might call this the CS mindset. It is present in CS50 independent of any practical ends associated with tech start-ups and massaging spreadsheet data. It is practical on a higher plane. It is also present in the HtDP curriculum and especially in the Racket Way.

It is present in all good CS education, even the CS intro courses that more students should be taking -- even if they are only going to be doctors, lawyers, or journalists.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

January 16, 2014 4:22 PM

Another Take on "Know the Past"

Ted Nelson offers a particularly stark assessment of how well we fulfill our obligation to know the past in his eulogy for Douglas Engelbart:

To quote Joan of Arc, from Shaw's play about her: "When will the world be ready to receive its saints?"

I think we know the answer -- when they are dead, pasteurized and homogenized and simplified into stereotypes, and the true depth and integrity of their ideas and initiatives are forgotten.

Nelson's position is stronger yet, because he laments the way in which Engelbart and his visions of the power of computing were treated throughout his career. How, he wails, could we have let this vision slip through our hands while Engelbart lived among us?

Instead, we worked on another Java IDE or a glue language for object-relational mapping. All the while, as Nelson says, "the urgent and complex problems of mankind have only grown more urgent and more complex."

This teaching is difficult; who can accept it?

Posted by Eugene Wallingford | Permalink | Categories: Computing

January 08, 2014 3:06 PM

"I'm Not a Programmer"

In The Exceptional Beauty of Doom 3's Source Code, Shawn McGrath first says this:

I've never really cared about source code before. I don't really consider myself a 'programmer'.

Then he says this:

Dyad has 193k lines of code, all C++.

193,000 lines of C++? Um, dude, you're a programmer.

Even so, the point is worth thinking about. For most people, programming is a means to an end: a way to create something. Many CS students start with a dream program in mind and only later, like McGrath, come to appreciate code for its own sake. Some of our graduates never really get there, and appreciate programming mostly for what they can do with it.

If the message we send from academic CS is "come to us only if you already care about code for its own sake", then we may want to fix our message.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

January 07, 2014 4:09 PM

Know The Past

In this profile of computational geneticist Jason Moore, the scientist speaks explains how his work draws on work from the 1910s, which may offer computational genetics a better path forward than the work that catapulted genetics forward in the 1990s and 2000s.

Yet despite his use of new media and advanced technology, Moore spends a lot of time thinking about the past. "We have a lot to learn from early geneticists," he says. "They were smart people who were really thinking deeply about the problem."

Today, he argues, genetics students spend too much time learning to use the newest equipment and too little time reading the old genetics literature. Not surprisingly, given his ambivalent attitude toward technology, Moore believes in the importance of history. "Historical context is so important for what we do," he says. "It provides a grounding, a foundation. You have to understand the history in order ... to understand your place in the science."

Anyone familiar with the history of computing knows there is another good reason to know your history: Sometimes, we dream too small these days, and settle for too little. We have a lot to learn from early computer scientists.

I intend to make this a point of emphasis in my algorithms course this spring. I'd like to expose students to important new ideas outside the traditional canon (more on that soon), while at the same time exposing them to some of the classic work that hasn't been topped.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

December 24, 2013 11:35 AM

Inverting the Relationship Between Programs and Literals

This chapter on quasi-literals in the E language tells the story of Scott Kim teaching the author that Apple's HyperCard was "powerful in a way most of had not seen before". This unexpected power led many people to misunderstand its true importance.

Most programs are written as text, sequences of characters. In this model, a literal is a special form of embedded text. When the cursor is inside the quotes of a literal string, we are "effectively no longer in a program editor, but in a nested text editor". Kim calls this a 'pun': Instead of writing text to be evaluated by another program, we are creating output directly.

What if we turn things inside out and embed our program in literal text?

Hypercard is normally conceived of as primarily a visual application builder with an embedded silly programming language (Hypertalk). Think instead of a whole Hypercard stack as a mostly literal program. In addition to numbers and strings, the literals you can directly edit include bitmaps, forms, buttons, and menus. You can literally assemble most things, but where you need dynamic behavior, there's a hole in the literal filled in by a Hypertalk script. The program editor for this script is experienced as being nested inside the direct manipulation user interface editor.

There is a hole in the literal text, where a program goes, instead of a hole in the program, where literal text goes.

HyperTalk must surely have seemed strange to most programmers in 1987. Lisp programmers had long used macros and so knew the power of nesting code to be eval'ed inside of literal text. Of course, the resulting text was then passed on the eval to be treated again as program!

The inside-out idea of HyperCard is alive today in the form of languages such as PHP, which embed code in HTML text:

      echo $_SERVER['HTTP_USER_AGENT'];

This is a different way to think about programming, one perhaps suitable for bringing experts in some domains toward the idea of writing code gradually from documents in their area of expertise.

I sometimes have the students in my compiler course implement a processor for a simple Mustache-like template language as an early warm-up homework assignment. I do not usually require them to go as far as Turing-complete embedded code, but they create a framework that makes it possible. I think I'll look for ways to bring more of this idea into the next offering of our more general course on programming languages.

(HyperCard really was more than many people realized at the time. The people who got it became Big Fans, and the program still has an ardent following. Check out this brief eulogy, which rhapsodizes on "the mystically-enchanting mantra" at the end of the application's About box: "A day of acquaintance, / And then the longer span of custom. / But first -- / The hour of astonishment.")

Posted by Eugene Wallingford | Permalink | Categories: Computing

December 16, 2013 2:20 PM

More Fun with Integer "Assembly Language": Brute-Forcing a Function Minimum

Or: Irrational Exuberance When Programming

My wife and daughter laughed at me yesterday.

A few years ago, I blogged about implementing Farey sequences in Klein, a language for which my students at the time were writing a compiler. Klein was a minimal functional language with few control structures, few data types, and few built-in operations. Computing rational approximations using Farey's algorithm was a challenge in Klein that I likened to "integer assembly programming".

I clearly had a lot of fun with that challenge, especially when I had the chance to watch my program run using my students' compilers.

This semester, I am again teaching the compiler course, and my students are writing a compiler for a new version of Klein.

Last week, while helping my daughter with a little calculus, I ran across a fun new problem to solve in Klein:

the task of optimizing cost across the river

There are two stations on opposite sides of a river. The river is 3 miles wide, and the stations are 5 miles apart along the river. We need to lay pipe between the stations. Pipe laid on land costs $2.00/foot, and pipe laid across the river costs $4.00/foot. What is the minimum cost of the project?

This is the sort of optimization problem one often encounters in calculus textbooks. The student gets to construct a couple of functions, differentiate one, and find a maximum or minimum by setting f' to 0 and solving.

Solving this problem in Klein creates some of challenges. Among them are that ideally it involves real numbers, which Klein doesn't support, and that it requires a square root function, which Klein doesn't have. But these obstacles are surmountable. We already have tools for computing roots using Newton's method in our collection of test programs. Over a 3mi-by-5mi grid, an epsilon of a few feet approximates square roots reasonably well.

My daughter's task was to use the derivative of the cost function but, after talking about the problem with her, I was interested more in "visualizing" the curve to see how the cost drops as one moves in from either end and eventually bottoms out for a particular length of pipe on land.

So I wrote a Klein program that "brute-forces" the minimum. It loops over all possible values in feet for land pipe and compares the cost at each value to the previous value. It's easy to fake such a loop with a recursive function call.

The programmer's challenge in writing this program is that Klein has no local variables other function parameters. So I had to use helper functions to simulate caching temporary variables. This allowed me to give a name to a value, which makes the code more readable, but most importantly it allowed me to avoid having to recompute expensive values in what was already a computationally-expensive program.

This approach creates another, even bigger challenge for my students, the compiler writers. My Klein program is naturally tail recursive, but tail call elimination was left as an optional optimization in our class project. With activation records for all the tail calls stored on the stack, a compiler has to use a lot of space for its run-time memory -- far more than is available on our default target machine.

How many frames do we need? Well, we need to compute the cost at every foot along a (5 miles x 5280 feet/mile) rectangle, for a total of 26,400 data points. There will, of course, be other activation records while computing the last value in the loop.

Will I be able to see the answer generated by my program using my students' compilers? Only if one or more of the teams optimized tail calls away. We'll see soon enough.

So, I spent an hour or so writing Klein code and tinkering with it yesterday afternoon. I was so excited by the time I finished that I ran upstairs to tell my wife and daughter all about it: my excitement at having written the code, and the challenge it sets for my students' compilers, and how we could compute reasonable approximations of square roots of large integers even without real numbers, and how I implemented Newton's method in lieu of a sqrt, and...

That's when my wife and daughter laughed at me.

That's okay. I am programmer. I am still excited, and I'd do it again.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal, Teaching and Learning

December 11, 2013 12:01 PM

"Costs $20" is Functionally Indistinguishable from Gone

In his write-up on the origin of zero-based indexing in computing, Mike Hoye comments on the difficulties he had tracking down original sources:

Part of the problem is access to the historical record, of course. I was in favor of Open Access publication before, but writing this up has cemented it: if you're on the outside edge of academia, $20/paper for any research that doesn't have a business case and a deep-pocketed backer is completely untenable, and speculative or historic research that might require reading dozens of papers to shed some light on longstanding questions is basically impossible. There might have been a time when this was OK and everyone who had access to or cared about computers was already an IEEE/ACM member, but right now the IEEE -- both as a knowledge repository and a social network -- is a single point of a lot of silent failure. "$20 for a forty-year-old research paper" is functionally indistinguishable from "gone", and I'm reduced to emailing retirees to ask them what they remember from a lifetime ago because I can't afford to read the source material.

I'm an academic. When I am on campus, I have access to the ACM Digital Library. When I go home, I do not. I could pay for a personal subscription, but that seems an unnecessary expense when I am on campus so much.

I never have access to IEEE Xplore, Hoy's "single point of silent failure". Our university library chose to drop its institutional subscription a few years ago, and for good reason: it is ridiculously expensive, especially relative to the value we receive from it university-wide. (We don't have an engineering college.) We inquired about sharing a subscription with our sister schools, as we are legally under a single umbrella, but at least at that time, IEEE didn't allow such sharing.

What about non-academics, such as Hoye? We are blessed in computing with innumerable practitioners who study our history, write about, and create new ideas. Some are in industry and may have access to these resources, or an expense account. Many others, though, work on their own as independent contractors and researchers. They need access to materials, and $20 a pop is an acceptable expense.

Their loss if our loss. If Hoye had not written his article on the history of zero-based indexing, most of us wouldn't know the full story.

As time goes by, I hope that open access to research publications continues to grow. We really shouldn't have to badger retired computer scientists with email asking what they remember now about a topic they wrote an authoritative paper on forty years ago.

Posted by Eugene Wallingford | Permalink | Categories: Computing

December 10, 2013 3:33 PM

Your Programming Language is Your Raw Material, Too

Recently someone I know retweeted this familiar sentiment:

If carpenters were hired like programmers:
"Must have at least 5 years experience with the Dewalt 18V 165mm Circular Saw"

This meme travels around the world in various forms all the time, and every so often it shows up in one of my inboxes. And every time I think, "There is more to the story."

In one sense, the meme reflects a real problem in the software world. Job ads often use lists of programming languages and technologies as requirements, when what the company presumably really wants is a competent developer. I may not know the particular technologies on your list, or be expert in them, but if I am an experienced developer I will be able to learn them and become an expert.

Understanding and skill run deeper than a surface list of tools.

But. A programming language is not just a tool. It is a building material, too.

Suppose that a carpenter uses a Dewalt 18V 165mm circular saw to add a room to your house. When he finishes the project and leaves your employ, you won't have any trace of the Dewalt in his work product. You will have a new room.

He might have used another brand of circular saw. He may not have used a power tool at all, preferring the fine craftsmanship of a handsaw. Maybe he used no saw of any kind. (What a magician!) You will still have the same new room regardless, and your life will proceed in the very same way.

Now suppose that a programmer uses the Java programming language to add a software module to your accounting system. When she finishes the project and leaves your employ, you will have the results of running her code, for sure. But you will have a trace of Java in her work product. You will have a new Java program.

If you intend to use the program again, to generate a new report from new input data, you will need an instance of the JVM to run it. If want to modify the program to work differently, then you will also need a Java compiler to create the byte codes that run in the JVM. If you want to extend the program to do more, then you again will need a Java compiler and interpreter.

Programs are themselves tools, and we use programming languages to build them. So, while the language itself is surely a tool at one level, at another level it is the raw material out of which we create other things.

To use a particular language is to introduce a slew of other dependencies to the overall process: compilers, interpreters, libraries, and sometimes even machine architectures. In the general case, to use a particular language is to commit at least some part of the company's future attention to both the language and its attendant tooling.

So, while I am sympathetic to sentiment behind our recurring meme, I think it's important to remember that a programming language is more than just a particular brand of power tool. It is the stuff programs are made of.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

December 04, 2013 3:14 PM

Agile Moments, "Why We Test" Edition

Case 1: Big Programs.

This blog entry tells the sad story of a computational biologist who had to retract six published articles. Why? Their conclusions depended on the output of a computer program, and that program contained a critical error. The writer of the entry, who is not the researcher in question, concludes:

What this should flag is the necessity to aggressively test all the software that you write.

Actually, you should have tests for any program you use to draw important conclusions, whether you wrote it or not. The same blog entry mentions that a grad student in the author's previous lab had found several bugs a molecular dynamics program used by many computational biologists. How many published results were affected before they were found?

Case 2: Small Scripts.

Titus Brown reports finding bugs every time he reused one of his Python scripts. Yet:

Did I start doing any kind of automated testing of my scripts? Hell no! Anyone who wants to write automated tests for all their little scriptlets is, frankly, insane. But this was one of the two catalysts that made me personally own up to the idea that most of my code was probably somewhat wrong.

Most of my code has bugs but, hey, why write tests?

Didn't a famous scientist define insanity as doing the same thing over and over but expecting different results?

I consider myself insane, too, but mostly because I don't write tests often enough for my small scripts. We say to ourselves that we'll never reuse them, so we don't need tests. But we don't throw them away, and then we do reuse them, perhaps with a tweak here or there.

We all face time constraints. When we run a script the first time, we may well pay enough attention to the output that we are confident it is correct. But perhaps we can all agree that the second time we use a script, we should write tests for it if we don't already have them.

There are only three numbers in computing, 0, 1, and many. The second time we use a program is a sign from the universe that we need the added confidence provided by tests.

To be fair, Brown goes on to offer some good advice, such as writing tests for code after you find a bug in it. His article is an interesting read, as is almost everything he writes about computation and science.

Case 3: The Disappointing Trade-Off.

Then there's this classic from Jamie Zawinski, as quoted in Coders at Work:

I hope I don't sound like I'm saying, "Testing is for chumps." It's not. It's a matter of priorities. Are you trying to write good software or are you trying to be done by next week? You can't do both.

Sigh. If you you don't have good software by next week, maybe you aren't done yet.

I understand that the real world imposes constraints on us, and that sometimes worse is better. Good enough is good enough, and we rarely need a perfect program. I also understand that Zawinski was trying to be fair to the idea of testing, and that he was surely producing good enough code before releasing.

Even still, the pervasive attitude that we can either write good programs or get done on time, but not both, makes me sad. I hope that we can do better.

And I'm betting that the computational biologist referred to in Case 1 wishes he had had some tests to catch the simple error that undermined five years worth of research.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 30, 2013 9:45 AM

The Magic at the Heart of AI

This paragraph from The Man Who Would Teach Machines to Think expresses a bit of my uneasiness with the world of AI these days:

As our machines get faster and ingest more data, we allow ourselves to be dumber. Instead of wrestling with our hardest problems in earnest, we can just plug in billions of examples of them. Which is a bit like using a graphing calculator to do your high-school calculus homework -- it works great until you need to actually understand calculus.

I understand the desire to solve real problems and the resulting desire to apply opaque mathematics to large data sets. Like most everyone, I revel in what Google can do for me and watch in awe when Watson defeats the best human Jeopardy! players ever. But for me, artificial intelligence was about more than just getting the job done.

Over the years teaching AI, my students often wanted to study neural networks in much greater detail than my class tended to go. But I was more interested in approaches to AI and learning that worked at a more conceptual level. Often we could find a happy middle ground while studying genetic algorithms, which afforded them the magic of something-for-nothing and afforded me the potential for studying ideas as they evolved over time.

(Maybe my students were simply exhibiting Astrachan's Law.)

When I said goodbye to AAAI a few years ago, I mentioned Hofstadter's work as one of my early inspirations -- Gödel, Escher, Bach and the idea of self-reference, with its "intertwining worlds of music, art, mathematics, and computers". That entry said I was leaving AAAI because my own work had moved in a different direction. But it left unstated a second truth, which The Man Who Would Teach Machines to Think asserts as Hofstadter's own reason for working off the common path: the world of AI had moved in a different direction, too.

For me, as for Hofstadter, AI has always meant more than engineering a solution. It was about understanding scientifically something that seemed magical, something that is both deeply personal and undeniably universal to human experience, about how human consciousness seems to work. My interest in AI will always lie there.


If you enjoy the article about Hofstadter and his work linked above, perhaps you will enjoy a couple of entries I wrote after he visited my university last year:

Posted by Eugene Wallingford | Permalink | Categories: Computing

November 24, 2013 10:54 AM

Teaching Algorithms in 2014

This spring, I will be teaching the undergraduate algorithms course for first time in nine years, since the semester before I became department head. I enjoy this course. It gives both the students and me opportunities to do a little theory, a little design, and a little programming. I also like to have some fun, using what we learn to play games and solve puzzles.

Nine years is a long time in computing, even in an area grounded in well-developed theory. I will need to teach a different sort of course. At the end of this entry, I ask for your help in meeting this challenge.

Algorithms textbooks don't look much different now than they did in the spring of 2005. Long-time readers of this blog know that I face the existential crisis of selecting a textbook nearly every semester. Picking a textbook requires balancing several forces, including the value they give to the instructor, the value they give to the student during and after the course, and the increasing expense to students.

My primary focus in these decisions is always on net value to the students. I like to write my own material anyway. When time permits, I'd rather write as much as I can for students to read than outsource that responsibility (and privilege) to a textbook author. Writing my lecture notes in depth lets me weave a lot of different threads together, including pointers into primary and secondary sources. Students benefit from learning to read non-textbook material, the sort they will encounter as throughout their careers.

My spring class brings a new wrinkle to the decision, though. Nearly fifty students are enrolled, with the prospect a few more to come. This is a much larger group than I usually work with, and large classes carry a different set of risks than smaller courses. In particular, when something goes wrong in a small section, it is easier to recover through one-on-one remediation. That option is not so readily available for a fifty-person course.

There is more risk in writing new lecture material than in using a textbook that has been tested over time. A solid textbook can be a security blanket as much for the instructor as for the student. I'm not too keen on selecting a security blanket for myself, but the predictability of a text is tempting. There is one possible consolation in such a choice: perhaps subordinating my creative impulses to the design of someone's else's textbook will make me more creative as a result.

But textbook selection is a fairly ordinary challenge for me. The real question is: Which algorithms should we teach in this course, circa 2014? Surely the rise of big data, multi-core processors, mobile computing, and social networking require a fresh look at the topics we teach undergrads.

Perhaps we need only adjust the balance of topics that we currently teach. Or maybe we need to add a new algorithm or data structure to the undergraduate canon. If we teach a new algorithm, or a new class of algorithms, which standard material should be de-emphasized, or displaced altogether? (Alas, the semester is still but fifteen weeks long.)

Please send me your suggestions! I will write up a summary of the ideas you share, and I will certainly use your suggestions to design a better algorithms course for my students.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 19, 2013 4:49 PM

First Model, Then Improve

Not long ago, I read Unhappy Truckers and Other Algorithmic Problems, an article by Tom Vanderbilt that looks at efforts to optimize delivery schedules at UPS and similar companies. At the heart of the challenge lies the traveling salesman problem. However, in practice, the challenge brings companies face-to-face with a bevy of human issues, from personal to social, psychological to economic. As a result, solving this TSP is more complex than what we see in the algorithms courses we take in our CS programs.

Yet, in the face of challenges both computational and human, the human planners working at these companies do a pretty good job. How? Over the course of time, researchers figured out that finding optimal routes shouldn't be their main goal:

"Our objective wasn't to get the best solution," says Ted Gifford, a longtime operations research specialist at Schneider. "Our objective was to try to simulate what the real world planners were really doing."

This is a lesson I learned the hard way, too, back in graduate school, when my advisor's lab was trying to build knowledge-based systems for real clients, in chemical engineering, aeronautics, business, and other domains. We were working with real people who were solving hard problems under serious constraints.

At the beginning I was a typically naive programmer, armed with fancy AI techniques and unbounded enthusiasm. I soon learned that, if you walk into a workplace and propose to solve all the peoples' problems with a program, things don't go as smoothly as the programmer might hope.

First of all, this impolitic approach generally creates immediate pushback. These are people, with personal investment in the way things work now. They tend to bristle when a 20-something grad student walks in the door promoting the wonder drug for all their ills. Some might even fear that you are right, and success for your program will mean negative consequences for them personally. We see this dynamic in Vanderbilt's article.

There's a deeper reason that things don't go so smoothly, though, and it's the real lesson of Vanderbilt's piece. Until you implement the existing solution to the problem, you don't really understand the problem yet.

These problems are complex, often with many more constraints than typical theoretical solutions have dealt with. The humans solving the problem often have many years of experience contributing to their approach. They have deep knowledge of the domain, but also repeated exposure to the exceptions and edge cases that sometimes confound theoretical solutions. They use heuristics that are hard to tease apart or articulate.

I learned that it's easy to solve a problem if you are solving the wrong one.

A better way to approach these challenges is: First, model the existing system, including the extant solution. Then, look for ways to improve on the solution.

This approach often gives everyone involved greater confidence that the programmers understand -- and so are solving -- the right problem. It also enables the team to make small, incremental changes to the system, with a correspondingly higher probability of success. Together, these two outcomes greatly increase the chance of human buy-in from the current workers. This makes it easier for the whole team to recognize the need for larger-scale changes to the process, and to support and contribute to an improved solution.

Vanderbilt tells a similarly pragmatic story. He writes:

When I suggest to Gifford that he's trying to understand the real world, mathematically, he concurs, but adds: "The word 'understand' is too strong--we are happy to get positive outcomes."

Positive outcomes are what the company wants. Fortunately for the academics who work on such problems in industry, achieving good outcomes is often an effective way to test theories, encounter their shortcomings, and work on improvements. That, too, is something I learned in grad school. It was a valuable lesson.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 14, 2013 2:55 PM

Toward A New Data Science Culture in Academia

Fernando Perez has a nice write-up, An Ambitious Experiment in Data Science, describing a well-funded new project in which teams at UC Berkeley, the University of Washington, and NYU will collaborate to "change the culture of universities to create a data science culture". A lot of people have been quoting Perez's entry for its colorful assessment of academic incentives and reward structures. I like this piece for the way Perez defines and outlines the problem, in terms of both data science across disciplines and academic culture in general.

For example:

Most scientists are taught to treat computation as an afterthought. Similarly, most methodologists are taught to treat applications as an afterthought.

Methodologists here includes computer scientists, who are often more interested in new data structures, algorithms, and protocols.

This "mirror" disconnect is a problem for a reason many people already understand well:

Computation and data skills are all of a sudden everybody's problem.

(Here are a few past entries of mine that talk about how programming and the nebulous "computational thinking" have spread far and wide: 1 | 2 | 3 | 4.)

Perez rightly points out that the open-source software, while imperfect, often embodies the principles or science and scientific collaboration better than the academy. It will be interesting to see how well this data science project can inject OSS attitudes into big research universities.

He is concerned because, as I have noted before, are, as a whole, a conservative lot. Perez says this in a much more entertaining way:

There are few organizations more proud of their traditions and more resistant to change than universities (churches and armies might be worse, but that's about it).

I think he gives churches and armies more credit than they deserve.

The good news is that experiments of the sort being conducted in the Berkley/UW/NYU project are springing up on a smaller scale around the world. There is some hope for big change in academic culture if a lot of different people at a lot of different institutions experiment, learn, and create small changes that can grow together as they bump into one another.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

October 29, 2013 3:49 PM

My PLoP 2013 Retrospective

wrapper of the Plopp candy bar I received from Rebecca Rikner

PLoP 2013 was as much fun and as invigorating as I had hoped it would be. I hadn't attended in eight years, but it didn't take long to fall back into the rhythm of writers' workshops interspersed among invited talks, focus group sessions, BoFs, mingling, and (yes) games.

I was lead moderator for Workshop 010010, which consisted of pedagogical patterns papers. The focus of all of them was interactivity, whether among students building LEGO Mindstorms robots or among students and instructor on creative projects. The idea of the instructor as an active participant, even "generative" in the sense meant by Christopher Alexander, dominated our discussion. I look forward to seeing published versions of the papers we discussed.

The other featured events included invited talks by Jenny Quillien and Ward Cunningham and a 20-year retrospective panel featuring people who were present at the beginning of PLoP, the Hillside Group, and software patterns.

Quillien spent six years working with Alexander during the years he created The Nature of Order. Her talk shared some of the ways that Alexander was disappointed in the effect of his seminal "A Pattern Language" had on the world, both as a result of people misunderstanding it and as a result of the books inherent faults. Along the way, she tried to give pragmatic advice to people trying to document patterns of software. I may try to write up some of her thoughts, and some of my own in response, in the coming weeks.

Cunningham presented his latest work on federated wiki, the notion of multiple, individual wikis "federated" in relationships that share and present information for a common good. Unlike the original wiki, in which collaboration happened in a common document, federated wiki has a fork button on every page. Anyone can copy, modify, and share pages, which are then visible to everyone and available for merging back into the home wikis.

the favicon for my federated wiki on Ward's server

Ward set me up with a wiki in the federation on his server before I left on Saturday. I want to play with it a bit before I say much more than this: Federated wiki could change how communities share and collaborate in much the same way that wiki did.

I also had the pleasure of participating in one other structured activity while at PLoP. Takashi Iba and his students at Keio University in Japan are making a documentary about the history of the patterns community. Takashi invited me to sit for an interview about pedagogical patterns and their history within the development of software patterns. I was happy to help. It was a fun challenge to explain my understanding of what a pattern language is, and to think about what my colleagues and I struggled with in trying to create small pattern languages to guide instruction. Of course, I strayed off to the topic of elementary patterns as well, and that led to more interesting discussion with Takashi. I look forward to seeing their film in the coming years.

More so than even other conferences, unstructured activity plays a huge role in any PLoP conference. I skipped a few meals so that I could walk the extensive gardens and grounds of Allerton Park (and also so that I would not gain maximum pounds from the plentiful and tasty meals that were served). I caught up with old friends such as Ward, Kyle Brown, Bob Hanmer, Ralph Johnson, and made too many new friends to mention here. All the conversation had my mind swirling with new projects and old... Forefront in my mind is exploring again the idea of design and implementation patterns of functional programming. The time is still right, and I want to help.

Now, to write my last entry or two from StrangeLoop...


Image 1. A photo of the wrapper of a Plopp candy bar, which I received as a gift from Rebecca Rikner. PLoP has a gifting tradition, and I received a box full of cool tools, toys, mementoes, and candy. Plopp is a Swedish candy bar, which made it a natural gift for Rebecca to share from her native land. (It was tasty, too!)

Image 2. The favicon for my federated wiki on Ward's server, I like the color scheme that gave me -- and I'm glad to be early enough an adopter that I could claim my first name as the name of my wiki. The rest of the Eugenes in the world will have to settle for suffix numbers and all the other contortions that come with arriving late to the dance.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

October 19, 2013 7:38 AM

The Proto Interpreter for J

(Update: Josh Grams took my comment about needing a week of work to grok this code as a challenge. He figured it out much more quickly than that and wrote up an annotated version of the program as he went along.

I like finding and reading about early interpreters for programming languages, such as the first Lisp interpreter or Smalltalk-71, which grew out of a one-page proof of concept written by Dan Ingalls on a bet with Alan Kay.

So I was quite happy recently to run across Appendix A from An Implementation of J, from which comes the following code. Arthur Whitney whipped up this one-page interpreter fragment for the AT&T 3B1 one weekend in 1989 to demonstrate his idea for a new APL-like like language. Roger Hui studied this interpreter for a week before writing the first implementation of J.

typedef char C;typedef long I;
typedef struct a{I t,r,d[3],p[2];}*A;
#define P printf
#define R return
#define V1(f) A f(w)A w;
#define V2(f) A f(a,w)A a,w;
#define DO(n,x) {I i=0,_n=(n);for(;i<_n;++i){x;}}
I *ma(n){R(I*)malloc(n*4);}mv(d,s,n)I *d,*s;{DO(n,d[i]=s[i]);}
tr(r,d)I *d;{I z=1;DO(r,z=z*d[i]);R z;}
A ga(t,r,d)I *d;{A z=(A)ma(5+tr(r,d));z->t=t,z->r=r,mv(z->d,d,r);
 R z;}
V1(iota){I n=*w->p;A z=ga(0,1,&n);DO(n,z->p[i]=i);R z;}
V2(plus){I r=w->r,*d=w->d,n=tr(r,d);A z=ga(0,r,d);
 DO(n,z->p[i]=a->p[i]+w->p[i]);R z;}
V2(from){I r=w->r-1,*d=w->d+1,n=tr(r,d);
 A z=ga(w->t,r,d);mv(z->p,w->p+(n**a->p),n);R z;}
V1(box){A z=ga(1,0,0);*z->p=(I)w;R z;}
V2(cat){I an=tr(a->r,a->d),wn=tr(w->r,w->d),n=an+wn;
 A z=ga(w->t,1,&n);mv(z->p,a->p,an);mv(z->p+an,w->p,wn);R z;}
V2(rsh){I r=a->r?*a->d:1,n=tr(r,a->p),wn=tr(w->r,w->d);
 A z=ga(w->t,r,a->p);mv(z->p,w->p,wn=n>wn?wn:n);
 if(n-=wn)mv(z->p+wn,z->p,n);R z;}
V1(sha){A z=ga(0,1,&w->r);mv(z->p,w->d,w->r);R z;}
V1(id){R w;}V1(size){A z=ga(0,0,0);*z->p=w->r?*w->d:1;R z;}
pi(i){P("%d ",i);}nl(){P("\n");}
pr(w)A w;{I r=w->r,*d=w->d,n=tr(r,d);DO(r,pi(d[i]));nl();
 if(w->t)DO(n,P("< ");pr(w->p[i]))else DO(n,pi(w->p[i]));nl();}

C vt[]="+{~<#,"; A(*vd[])()={0,plus,from,find,0,rsh,cat}, (*vm[])()={0,id,size,iota,box,sha,0}; I st[26]; qp(a){R a>='a'&&a<='z';}qv(a){R a<'a';} A ex(e)I *e;{I a=*e; if(qp(a)){if(e[1]=='=')R st[a-'a']=ex(e+2);a= st[ a-'a'];} R qv(a)?(*vm[a])(ex(e+1)):e[1]?(*vd[e[1]])(a,ex(e+2)):(A)a;} noun(c){A z;if(c<'0'||c>'9')R 0;z=ga(0,0,0);*z->p=c-'0';R z;} verb(c){I i=0;for(;vt[i];)if(vt[i++]==c)R i;R 0;} I *wd(s)C *s;{I a,n=strlen(s),*e=ma(n+1);C c; DO(n,e[i]=(a=noun(c=s[i]))?a:(a=verb(c))?a:c);e[n]=0;R e;}

main(){C s[99];while(gets(s))pr(ex(wd(s)));}

I think it will take me a week of hard work to grok this code, too. Whitney's unusually spare APL-like C programming style is an object worthy of study in its own right.

By the way, Hui's Appendix A bears the subtitle Incunabulum, a word that means a work of art or of industry of an early period. So, I not only discovered a new bit of code this week; I also learned a cool new word. That's a good week.

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 16, 2013 11:38 AM

Poetry as a Metaphor for Software

I was reading Roger Hui's Remembering Ken Iverson this morning on the elliptical, and it reminded me of this passage from A Conversation with Arthur Whitney. Whitney is a long-time APL guru and the creator of the A, K, and Q programming languages. The interviewer is Bryan Cantrill.

BC: Software has often been compared with civil engineering, but I'm really sick of people describing software as being like a bridge. What do you think the analog for software is?

AW: Poetry.

BC: Poetry captures the aesthetics, but not the precision.

AW: I don't know, maybe it does.

A poet's use of language is quite precise. It must balance forces in many dimensions, including sound, shape, denotation, and connotation. Whitney seems to understand this. Richard Gabriel must be proud.

Brevity is a value in the APL world. Whitney must have a similar preference for short language names. I don't know the source of his names A, K, and Q, but I like Hui's explanation of where J's name came from:

... on Sunday, August 27, 1989, at about four o'clock in the afternoon, [I] wrote the first line of code that became the implementation described in this document.

The name "J" was chosen a few minutes later, when it became necessary to save the interpreter source file for the first time.

Beautiful. No messing around with branding. Gotta save my file.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

October 11, 2013 1:42 PM

The Tang of Adventure, and a Lively Appreciation

"After you've learned the twelve times table," John Yarnelle asks, "what else is there to do?"

The concepts of modern mathematics give the student something else to do in great abundance and variety at all levels of his development. Not only may he discover unusual types of mathematical structures where, believe it or not, two and two does not equal four, but he may even be privileged to invent a new system virtually on his own. Far from a sense of stagnation, there is the tang of adventure, the challenge of exploration; perhaps also a livelier appreciation of the true nature of mathematical activity and mathematical thought.

Not only the tang of adventure; students might also come to appreciate what math really is. That's an admirable goal for any book or teacher.

This passage comes from Yarnelle's Finite Mathematical Structures, a 1964 paperback that teaches fields, groups, and algebras with the prose of a delighted teacher. I picked this slender, 66-page gem up off a pile of books being discarded by a retired math professor a decade ago. How glad I am that none of the math profs who walked past that pile bothered to claim it before I happened by.

We could use a few CS books like this, too.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

October 07, 2013 12:07 PM

StrangeLoop: Exercises in Programming Style

[My notes on StrangeLoop 2013: Table of Contents]

Crista Lopes

I had been looking forward to Crista Lopes's StrangeLoop talk since May, so I made sure I was in the room well before the scheduled time. I even had a copy of the trigger book in my bag.

Crista opened with something that CS instructors have learned the hard way: Teaching programming style is difficult and takes a lot of time. As a result, it's often not done at all in our courses. But so many of our graduates go into software development for the careers, where they come into contact with many different styles. How can they understand them -- well, quickly, or at all?

To many people, style is merely the appearance of code on the screen or printed. But it's not. It's more, and something entirely different. Style is a constraint. Lopes used images of a few stylistic paintings to illustrate the idea. If an artist limits herself to pointillism or cubism, how can she express important ideas? How does the style limit the message, or enhance it?

But we know this is true of programming as well. The idea has been a theme in my teaching for many years. I occasionally write about the role of constraints in programming here, including Patterns as a Source of Freedom, a few programming challenges, and a polymorphism challenge that I've run as a workshop.

Lopes pointed to a more universal example, though: the canonical The Elements of Programming Style. Drawing on this book and other work in software, she said that programming style ...

  • is a way to express tasks
  • exists at all scales
  • recurs at multiple scales
  • is codified in programming language

For me, the last bullet ties back most directly to idea of style as constraint. A language makes some things easier to express than others. It can also make some things harder to express. There is a spectrum, of course. For example, some OO languages make it easy to create and use objects; others make it hard to do anything else! But the language is an enabler and enforcer of style. It is a proxy for style as a constraint on the programmer.

Back to the talk. Lopes asked, Why is it so important that we understand programming style? First, a style provides the reader with a frame of reference and a vocabulary. Knowing different styles makes us a more effective consumers of code. Second, one style can be more appropriate for a given problem or context than another style. So, knowing different styles makes us a more effective producers of code. (Lopes did not use the producer-consumer distinction in the talk, but it seems to me a nice way to crystallize her idea.)

the cover of Raymond Queneau's Exercises in Style

The, Lopes said, I came across Raymond Queneau's playful little book, "Exercises in Style". Queneau constrains himself in many interesting ways while telling essentially the same story. Hmm... We could apply the same idea to programming! Let's do it.

Lopes picked a well-known problem, the common word problem famously solved in a Programming Pearls column more than twenty-five years. This is a fitting choice, because Jon Bentley included in that column a critique of Knuth's program by Doug McIlroy, who considered both engineering concerns and program style in his critique.

The problem is straightforward: identify and print the k most common terms that occur in a given text document, in decreasing order. For the rest of the talk, Lopes presented several programs that solve the problem, each written in a different style, showing code and highlighting its shape and boundaries.

Python was her language of choice for the examples. She was looking for a language that many readers would be able to follow and understand, and Python has the feel of pseudo-code about it. (I tell my students that it is the Pascal of their time, though I may as well be speaking of hieroglyphics.) Of course, Python has strengths and weaknesses that affect its fit for some styles. This is an unavoidable complication for all communication...

Also, Lopes did not give formal names to the styles she demonstrated. Apparently, at previous versions of this talk, audience members had wanted to argue over the names more than the styles themselves! Vowing not to make that mistake again, she numbered her examples for this talk.

That's what programmers do when they don't have good names.

In lieu of names, she asked the crowd to live-tweet to her what they thought each style is or should be called. She eventually did give each style a fun, informal name. (CS textbooks might be more evocative if we used her names instead of the formal ones.)

I noted eight examples shown by Lopes in the talk, though there may have been more:

  • monolithic procedural code -- "brain dump"
  • a Unix-style pipeline -- "code golf"
  • procedural decomposition with a sequential main -- "cook book"
  • the same, only with functions and composition -- "Willy Wonka"
  • functional decomposition, with a continuation parameter -- "crochet"
  • modules containing multiple functions -- "the kingdom of nouns"
  • relational style -- (didn't catch this one)
  • functional with decomposition and reduction -- "multiplexer"

Lopes said that she hopes to produce solutions using a total of thirty or so styles. She asked the audience for help with one in particular: logic programming. She said that she is not a native speaker of that style, and Python does not come with a logic engine built-in to make writing a solution straightforward.

Someone from the audience suggested she consider yet another style: using a domain-specific language. That would be fun, though perhaps tough to roll from scratch in Python. By that time, my own brain was spinning away, thinking about writing a solution to the problem in Joy, using a concatenative style.

Sometimes, it's surprising just how many programming styles and meaningful variations people have created. The human mind is an amazing thing.

The talk was, I think, a fun one for the audience. Lopes is writing a book based on the idea. I had a chance to review an early draft, and now I'm looking forward to the finished product. I'm sure I'll learn something new from it.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development, Teaching and Learning

October 04, 2013 3:12 PM

StrangeLoop: Rich Hickey on Channels and Program Design

[My notes on StrangeLoop 2013: Table of Contents]

Rich Hickey setting up for his talk

Rich Hickey spoke at one of the previous StrangeLoops I attended, but this was my first time to attend one of his talks in person. I took the shaky photo seen at the right as proof. I must say, he gives a good talk.

The title slide read "Clojure core.async Channels", but Hickey made a disclaimer upfront: this talk would be about what channels are and why Clojure has them, not the details of how they are implemented. Given that there were plenty of good compiler talks elsewhere at the conference, this was a welcome change of pace. It was also a valuable one, because many more people will benefit from what Hickey taught about program design than would have benefited from staring at screens full of Clojure macros. The issues here are important ones, and ones that few programmers understand very well.

The fundamental problem is this: Reactive programs need to be machines, but functions make bad machines. Even sequences of functions.

The typical solution to this problem these days is to decompose the system logic into a set of response handlers. Alas, this leads to callback hell, a modern form of spaghetti code. Why? Even though the logic has been decomposed into pieces, it is still "of a piece", essentially a single logical entity. When this whole is implemented across multiple handlers, we can't see it as a unit, or talk about it easily. We need to, though, because we need to design the state machine that it comprises.

Clojure's solution to the problem, in the form of core.async, is the channel. This is an implementation of Tony Hoare's communicating sequential process. One of the reasons that Hickey likes this approach is that it lets a program work equally well in fully threaded apps and in apps with macro-generated inversion of control.

Hickey then gave some examples of code using channels and talked a bit about the implications of the implementation for system design. For instance, the language provides handy put! and take! operators for integrating channels with code at the edge of non-core.async systems. I don't have much experience with Clojure, so I'll have to study a few examples in detail to really appreciate this.

For me, the most powerful part of the talk was an extended discussion of communication styles in program. Hickey focused on the trade-offs between direct communication via shared state and indirect communication via channels. He highlighted six or seven key distinctions between the two and how these affect the way a system works. I can't do this part of the talk justice, so I suggest you watch the video of the talk. I plan to watch it again myself.

I had always heard that Hickey was eminently quotable, and he did not disappoint. Here are three lines that made me smile:

  • "Friends don't let friends put logic in handlers."
  • "Promises and futures are the one-night stands" of asynchronous architecture.
  • "Unbounded buffers are a recipe for a bad program. 'I don't want to think about this bug yet, so I'll leave the buffer unbounded.'"

That last one captures the indefatigable optimism -- and self-delusion -- that characterizes so many programmers. We can fix that problem later. Or not.

In the end, this talk demonstrates how a good engineer approaches a problem. Clojure and its culture reside firmly in the functional programming camp. However, Hickey recognizes that, for the problem at hand, a sequence of functional calls is not the best solution. So he designs a solution that allows programmers to do FP where it fits best and to do something else where FP doesn't. That's a pragmatic way to approach problems.

Still, this solution is consistent with Clojure's overall design philosophy. The channel is a first-class object in the language. It converts a sequence of functional calls into data, whereas callbacks implement the sequence in code. As code, we see the sequence only at run-time. As data, we see it in our program and can use it in all the ways we can use any data. This consistent focus on making things into data is an attractive part of the Clojure language and the ecosystem that has been cultivated around it.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 28, 2013 12:17 PM

StrangeLoop: This and That, Volume 2

[My notes on StrangeLoop 2013: Table of Contents]

I am at a really good talk and look around the room. So many people are staring at their phones, scrolling away. So many others are staring at their laptops, typing away. The guy next to me: doing both at the same time. Kudos, sir. But you may have missed the point.


Conference talks are a great source of homework problems. Sometimes, the talk presents a good problem directly. Others, watching the talk sets my subconscious mind in motion, and it creates something useful. My students thank you. I thank you.


Jenny Finkel talked about the difference between two kinds of recommenders: explorers, who forage for new content, and exploiters, who want to see what's already popular. The former discovers cool new things occasionally but fails occasionally, too. The latter is satisfied most of the time but rarely surprised. As conference goes, I felt this distinction at play in my own head this year. When selecting the next talk to attend, I have to take a few risks if I ever hope to find something unexpected. But when I fail, a small regret tugs at me.


We heard a lot of confident female voices on the StrangeLoop stages this year. Some of these speakers have advanced academic degrees, or at least experience in grad school.


The best advice I received on Day 1 perhaps came not from a talk but from the building:

The 'Do not Climb on Bears' sign on a Peabody statue

"Please do not climb on bears." That sounds like a good idea most everywhere, most all the time.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

September 27, 2013 4:26 PM

StrangeLoop: Add All These Things

[My notes on StrangeLoop 2013: Table of Contents]

I took a refreshing walk in the rain over the lunch hour on Friday. I managed to return late and, as a result, missed the start of Avi Bryant's talk on algebra and analytics. Only a few minutes, though, which is good. I enjoyed this presentation.

Bryant didn't talk about the algebra we study in eighth or ninth grade, but the mathematical structure math students encounter in a course called "abstract" or "modern" algebra. A big chunk of the talk focused on an even narrower topic: why +, and operators like it, are cool.

One reason is that grouping doesn't matter. You can add 1 to 2, and then add 4 to the result, and have the same answer as if you added 4 to 1, and then added 2 to the result. This is, of course, the associative property.

Another is that order doesn't matter. 1 + 2 is the same as 2 + 1. That's the commutative property.

Yet another is that, if you have nothing to add, you can add nothing and have the same value you started with. 4 + 0 = 4. 0 is the identity element for addition.

Finally, when you add two numbers, you get a number back. This is not quite as true in computers as in math, because an operation can cause an overflow or underflow and create an error. But looked at through fuzzy lenses, this is true in our computers, too. This is the closure property for addition of integers and real numbers.

Addition isn't the only operation on numbers that has these properties. Finding the maximum value in a set of numbers, does, too. The maximum of two numbers is a number. max(x,y) = max(y,x), and if we have three or more numbers, it doesn't how matter how we group them; max will find the maximum among them. The identity value is tricky -- there is no smallest number... -- but in practice we can finesse this by using the smallest number of a given data type, or even allowing max to take "nothing" as a value and return its other argument.

When we see a pattern like this, Bryant said, we should generalize:

  • We have a function f that takes two values from a set and produces another member of the same set.
  • The order of f's arguments doesn't matter.
  • The grouping of f's arguments doesn't matter.
  • There is some identity value, a conceptual "zero", that doesn't matter, in the sense that f(i,zero) for any i is simply i.

There is a name for this pattern. When we have such as set and operation, we have a commutative monoid.

     S ⊕ S → S
     x ⊕ y = y ⊕ x
     x ⊕ (y ⊕ z) = (x ⊕ y) ⊕ z
     x ⊕ 0 = x

I learned about this and other such patterns in grad school when I took an abstract algebra course for kicks. No one told me at the time that I'd being seeing them again as soon as someone created the Internet and it unleashed a torrent of data on everyone.

Just why we are seeing the idea of a commutative monoid again was the heart of Bryant's talk. When we have data coming into our company from multiple network sources, at varying rates of usage and data flow, and we want to extract meaning from the data, it can be incredibly handy if the meaning we hope to extract -- the sum of all the values, or the largest -- can be computed using a commutative monoid. You can run multiple copies of your function at the entry point of each source, and combine the partial results later, in any order.

Bryant showed this much more attractively than that, using cute little pictures with boxes. But then, there should be an advantage to going to the actual talk... With pictures and fairly straightforward examples, he was able to demystify the abstract math and deliver on his talk's abstract:

A mathematician friend of mine tweeted that anyone who doesn't understand abelian groups shouldn't build analytics systems. I'd turn that around and say that anyone who builds analytics systems ends up understanding abelian groups, whether they know it or not.

That's an important point. Just because you haven't studied group theory or abstract algebra doesn't mean you shouldn't do analytics. You just need to be prepared to learn some new math when it's helpful. As programmers, we are all looking for opportunities to capitalize on patterns and to generalize code for use in a wider set of circumstances. When we do, we may re-invent the wheel a bit. That's okay. But also look for opportunities to capitalize on patterns recognized and codified by others already.

Unfortunately, not all data analysis is as simple as summing or maximizing. What if I need to find an average? The average operator doesn't form a commutative monoid with numbers. It falls short in almost every way. But, if you switch from the set of numbers to the set of pairs [n, c], where n is a number and c is a count of how many times you've seen n, then you are back in business. Counting is addition.

So, we save the average operation itself as a post-processing step on a set of number/count pairs. This turns out to be a useful lesson, as finding the average of a set is a lossy operation: it loses track of how many numbers you've seen. Lossy operations are often best saved for presenting data, rather than building them directly into the system's computation.

Likewise, finding the top k values in a set of numbers (a generalized form of maximum) can be handled just fine as long as we work on lists of numbers, rather than numbers themselves.

This is actually one of the Big Ideas of computer science. Sometimes, we can use a tool or technique to solve a problem if only we transform the problem into an equivalent one in a different space. CS theory courses hammer this home, with oodles of exercises in which students are asked to convert every problem under the sun into 3-SAT or the clique problem. I look for chances to introduce my students to this Big Idea when I teach AI or any programming course, but the lesson probably gets lost in the noise of regular classwork. Some students seem to figure it out by the time they graduate, though, and the ones who do are better at solving all kinds of problems (and not by converting them all 3-SAT!).

Sorry for the digression. Bryant didn't talk about 3-SAT, but he did demonstrate several useful problem transformations. His goal was more practical: how can we use this idea of a commutative monoid to extract as many interesting results from the stream of data as possible.

This isn't just an academic exercise, either. When we can frame several problems in this way, we are able to use a common body of code for the processing. He called this body of code an aggregator, comprising three steps:

  • prepare the data by transforming it into the space of a commutative monoid
  • reduce the data to a single value in that space, using the appropriate operator
  • present the result by transforming it back into its original space

In practice, transforming the problem into the space of a monoid presents challenges in the implementation. For example, it is straightforward to compute the number of unique values in a collection of streams by transforming each item into a set of size one and then using set union as the operator. But union requires unbounded space, and this can be inconvenient when dealing with very large data sets.

One approach is to compute an estimated number of uniques using a hash function and some fancy arithmetic. We can make the expected error in estimate smaller and smaller by using more and more hash functions. (I hope to write this up in simple code and blog about it soon.)

Bryant looked at one more problem, computing frequencies, and then closed with a few more terms from group theory: semigroup, group, and abelian group. Knowing these terms -- actually, simply knowing that they exist -- can be useful even for the most practical of practitioners. They let us know that there is more out there, should our problems become harder or our needs become larger.

That's a valuable lesson to learn, too. You can learn all about abelian groups in the trenches, but sometimes it's good to know that there may be some help out there in the form of theory. Reinventing wheels can be cool, but solving the problems you need solved is even cooler.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

September 24, 2013 4:38 PM

StrangeLoop: Compilers, Compilers, Compilers

[My notes on StrangeLoop 2013: Table of Contents]

I went to a lot of talks about compilation. There seemed to be more this year than last, but perhaps I was suffering from a perception bias. I'm teaching compilers this semester and have been reading a bit about V8 and Crankshaft on the elliptical of late.

Many of the talks I saw revolved around a common theme: dynamic run-time systems. Given the prominence these days of Javascript, Python, Ruby, Lua, and their like, it's not surprising that finding better ways to organize dynamic run-times and optimize their performance are receiving a lot of attention.

The problem of optimizing dynamic run-time systems is complicated by the wide range of tasks being performed dynamically: type checking, field access, function selection, and the most common whipping horse of my static-language friends, garbage collection. Throw in eval, which allows the execution of arbitrary code, possibly changing even the definition of core classes, and it's amazing that our dynamic languages can run in human time at all. That's a tribute to the people who have been creating compilers for us over the years.

As I listened to these talks, my ears were tuned to ideas and topics that I need to learn more about. That's what my notes captured best. Here are a few ideas that stood out.

The JavaScript interpreter, interpreted. Martha Girdler gave a quick, jargon-free tour of how Javascript works, using Javascript code to illustrate basic ideas like contexts. This sort of talk can help relatively inexperienced developers understand the common "pain points" of the language, such as variable hoisting.

Fast and Dynamic. Maxime Chevalier-Boisvert went a level deeper, tracing some of the fundamental ideas used to implement run-time systems from their historical roots in Lisp, Smalltalk, and Self up to research prototypes such as Chevalier-Boisvert's own Higgs compiler.

Many of the ideas are familiar to anyone who has had an undergrad compiler course, such as type tagging and microcoded instructions. Others are simple extensions of such ideas, such as inline caching, which is a workhorse in any dynamic compiler. Still others have entered popular discussion only recently. Maps, which are effectively hidden classes, originated in Self and are now being applied and extended in a number of interesting ways.

Two ideas from this talk that I would like to learn more about are hybrid type inference, which Chevalier-Boisvert mentioned in the context of Chrome and Firefox, and basic block versioning, a technique being explored in the Higgs compiler.

In closing, the speaker speculated on the better compilers of the future. Some of the advances will come from smarter CPUs, which might execute potential future paths in parallel, and more principled language design. But many will come from academic research that discovers new techniques and improves exiting ones.

Some of the ideas of the future are probably already available and simply haven't caught on yet. Chevalier-Boisvert offered three candidates: direct manipulation of the AST, pattern matching, and the persistent image. I certainly hear a lot of people talking about the first of these, but I've yet to see a compelling implementation yet.

Ruby Doesn't Have to Be Slow. In this session, Alex Gaynor explained why dynamic languages don't have to be slow. Though Ruby was his working example, everything he said applies to Javascript, Python, Lua, and other dynamic languages. He then talked about how he is putting these ideas to work in Topaz, a fast Ruby interpreter written in RPython. Topaz uses a number of advanced techniques, including a tracing JIT, type-specialized field look-up, maps, quasi-immutable fields, and escape analysis. It supports a subset of Ruby, though much of what is missing now is simply standard library classes and methods.

Two of the more interesting points of this talk for me were about meta-issues. First, he opened with an elaboration of the claim, "Ruby is slow", which he rightfully rejected as too imprecise to be meaningful. What people probably mean is something like, "Code written in Ruby executes CPU-bound tasks slower than other languages." I would add that, for many of my CS colleagues, the implicit benchmark is compiled C.

Further, Ruby users tend to respond to this claim poorly. Rather than refute it, they accept its premise and dance around its edges. Saddest, he says, is when they say, "If it turns out to matter, we can rewrite the program in some serious language." The compiler nerd in him says, "We can do this." Topaz is, in part, an attempt to support that claim.

Second, in response to an audience question, he claimed that people responsible for Java got something right fifteen years: they convinced people to abandon their C extensions. If the Ruby world followed course, and moved away from external dependencies that restrict what the compiler and run-time system can know, then many performance improvements would follow.

Throughout this talk, I kept coming back to JRuby in my mind...

The Art of Finding Shortcuts. Vyacheslav " @mraleph" Egorov's talk was ostensibly about an optimizing compiler for Dart, but like most of the compiler talks this year, it presented ideas of value for handling any dynamic language. Indeed, this talk gave a clear introduction to what an optimizing compiler does, what in-line caching is, and different ways that the compiler might capitalize on them.

According to Egorov, writing an optimizing compiler for language like Dart is the art of finding -- and taking -- shortcuts. The three key issues to address are representation, resolution, and redundancy. You deal with representation when you design your run-time system. The other two fall to the optimizing compiler.

Resolution is fundamentally a two-part question. Given the expression obj.prop,

  • What is obj?
  • Where is prop?

In-line caches eliminate redundancy by memoizing where/what pairs. The goal is to use the same hidden class maps to resolve property access whenever possible. Dart's optimizer uses in-line caching to give type feedback for use in improving the performance of loads and stores.

Egorov was one of the most quotable speakers I heard at StrangeLoop this year. In addition to "the art of finding shortcuts", I noted several other pithy sayings that I'll probably steal at some point, including:

  • "If all you have is an in-line cache, then everything looks like an in-line cache stub."
  • "In-lining is a Catch-22." You can't know if you will benefit from inlining unless you try, but trying (and undoing) is expensive.

Two ideas I plan to read more about after hearing this talk are allocation sinking and load forwarding.


I have a lot of research to do now.

Posted by Eugene Wallingford | Permalink | Categories: Computing

September 23, 2013 4:22 PM

StrangeLoop: This and That, Volume 1

[My notes on StrangeLoop 2013: Table of Contents]

the Peabody Opera House's Broadway series poster

I'm working on a post about the compiler talks I attended, but in the meantime here are a few stray thoughts, mostly from Day 1.

The Peabody Opera House really is a nice place to hold a conference of this size. If StrangeLoop were to get much larger, it might not fit.

I really don't like the word "architected".

The talks were scheduled pretty well. Only once in two days did I find myself really wanting to go to two talks at the same time. And only once did I hear myself thinking, "I don't want to hear any of these...".

My only real regret from Day 1 was missing Scott Vokes's talk on data compression. I enjoyed the talk I went to well enough, but I think I would have enjoyed this one more.

What a glorious time to be a programming language theory weenie. Industry practitioners are going to conferences and attending talks on dependent types, continuations, macros, immutable data structures, and functional reactive programming.

Moon Hooch? Interesting name, interesting sound.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

September 22, 2013 3:51 PM

StrangeLoop: Jenny Finkel on Machine Learning at Prismatic

[My notes on StrangeLoop 2013: Table of Contents]

The conference opened with a talk by Jenny Finkel on the role machine learning play at Prismatic, the customized newsfeed service. It was a good way to start the conference, as it introduced a few themes that would recur throughout, had a little technical detail but not too much, and reported a few lessons from the trenches.

Prismatic is trying to solve the discovery problem: finding content that users would like to read but otherwise would not see. This is more than simply a customized newsfeed from a singular journalistic source, because it draws from many sources, including other reader's links, and because it tries to surprise readers with articles that may not be explicitly indicated by their profiles.

The scale of the problem is large, but different from the scale of the raw data facing Twitter, Facebook, and the like. Finkel said that Prismatic is processing only about one million timely docs at a time, with the set of articles turning over roughly weekly. The company currently uses 5,000 categories to classify the articles, though that number will soon go up to the order of 250,000.

The complexity here comes from the cross product of readers, articles, and categories, along with all of the features used to try to tease out why readers like the things they do and don't like the others. On top of this are machine learning algorithms that are themselves exponentially expensive to run. And with articles turning over roughly weekly, they have to be amassing data, learning from it, and moving on constantly.

The main problem at the heart of a service like this is: What is relevant? Everywhere one turns in AI, one sees this question, or its more general cousin, Is this similar? In many ways, this is the problem at the heart of all intelligence, natural and artificial.

Prismatic's approach is straight from AI, too. They construct a feature vector for each user/article pair and then try to learn weights that, when applied to the values in a given vector, will rank desired articles high and undesired articles low. One of the key challenges when doing this kind of working is to choose the right features to use in the vector. Finkel mentioned a few used by Prismatic, including "Does the user follow this topic?", "How many times has the reader read an article from this publisher?", and "Does the article include a picture?"

With a complex algorithm, lots of data, and a need to constantly re-learn, Prismatic has to make adjustments and take shortcuts wherever possible in order to speed up the process. This is a common theme at a conference where many speakers are from industry. First, learn your theory and foundations; learn the pragmatics and heuristics need to turn basic techniques into the backbone of practical applications.

Finkel shared one pragmatic idea of this sort that Prismatic uses. They look for opportunities to fold user-specific feature weights into user-neutral features. This enables their program to compute many user-specific dot products using a static vector.

She closed the talk with five challenges that Prismatic has faced that other teams might be on the look out for:

Bugs in the data. In one case, one program was updating a data set before another program could take a snapshot of the original. With the old data replaced by the new, they thought their ranker was doing better than it actually was. As Finkel said, this is pretty typical for an error in machine learning. The program doesn't crash; it just gives the wrong answer. Worse, you don't even have reason to suspect something is wrong in the offending code.

Presentation bias. Readers tend to look at more of the articles at the top of a list of suggestions, even if they would have enjoyed something further down the list. This is a feature of the human brain, not of computer programs. Any time we write programs that interact with people, we have to be aware of human psychology and its effects.

Non-representative subsets. When you are creating a program that ranks things, its whole purpose is to skew a set of user/article data points toward the subset of articles that the reader most wants to read. But this subset probably doesn't have the same distribution as the full set, which hampers your ability to use statistical analysis to draw valid conclusions.

Statistical bleeding. Sometimes, one algorithm looks better than it is because it benefits from the performance of the other. Consider two ranking algorithms, one an "explorer" that seeks out new content and one an "exploiter" that recommend articles that have already been found to be popular. If we in comparing their performances, the exploiter will tend to look better than it is because it benefits from the successes of the explorer without being penalized for its failures. It is crucial to recognize that one feature you measure is not dependent on another. (Thanks to Christian Murphy for the prompt!)

Simpson's Paradox. The iPhone and the web have different clickthrough rates. They once found them in a situation where one recommendation algorithm performed worse than another on both platforms, yet better overall. This can really disorient teams who follow up experiments by assessing the results. The issue here is usually a hidden variable that is confounding the results.

(I remember discussing this classic statistical illusion with a student in my early years of teaching, when we encountered a similar illusion in his grade. I am pretty sure that I enjoyed our discussion of the paradox more than he did...)

This part of a talk is of great value to me. Hearing about another team's difficulties rarely helps me avoid the same problems in my own projects, but it often does help me recognize those problems when they occur and begin thinking about ways to work around them. This was a good way for me to start the conference.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

September 22, 2013 10:27 AM

Back from StrangeLoop 2013

the front of my StrangeLoop 2013 badge

I'm back home for StrangeLoop 2013. It was, again, an outstanding conference: a great location, excellent amenities, fun side events, and -- most importantly -- a solid set of talks: diverse topics, strong technical content, and a some very good speakers. Alex Miller and his team put on a good conference.

This year, I went to the talks old school: with a steno notebook and no technology but a camera. As a result, a couple of things are different about how I'm blogging the conference. First, I did not write or post any entries during the event itself. Second, my notes are a bit shorter than usual and will need to be typed up before they become blog entries. I'll write my thoughts up over the next week or so and post the entries as they emerge.

This entry will serve as a table of contents for my StrangeLoop posts, a home base for readers who might stumble onto one post and care to read more. For now, I'll list a few entries I expect to write, but I'll only know what belongs here after I have written them.

Primary entries:

Ancillary entries:

Is it too early to start looking forward to StrangeLoop 2014?

Posted by Eugene Wallingford | Permalink | Categories: Computing

September 10, 2013 3:40 PM

A Laugh at My Own Expense

This morning presented a short cautionary tale for me and my students, from a silly mistake I made in a procmail filter.

Back story: I found out recently that I am still subscribed to a Billy Joel fan discussion list from the 1990s. The list has been inactive for years, or I would have been filtering its messages to a separate mailbox. Someone has apparently hacked the list, as a few days ago it started spewing hundreds of spam messages a day.

I was on the road for a few days after the deluge began and was checking mail through a shell connection to the mail server. Because I was busy with my trip and checking mail infrequently, I just deleted the messages by hand. When I got back, soon learned they were junk and filtered them away for me. But the spam was still hitting my inbox on the mail server, where I read my mail occasionally even on campus.

After a session on the server early this morning, I took a few minutes to procmail them away. Every message from the list has a common pattern in the Subject: line, so I copied it and pasted it into a new procmail recipe to send all list traffic to /dev/null :

    * ^Subject.*[billyjoel]

Do you see the problem? Of course you do.

I didn't at the time. My blindness probably resulted from a combination of the early hour, a rush to get over to the gym, and the tunnel vision that comes from focusing on a single case. It all looked obvious.

This mistake offers programming lessons at several different levels.

The first is at the detailed level of the regular expression. Pay attention to the characters in your regex -- all of them. Those brackets really are in the Subject: line, but by themselves mean something else in the regex. I need to escape them:

    * ^Subject.*\[billyjoel\]

This relates to a more general piece of problem-solving advice. Step back from individual case you are solving and think about the code you are writing more generally. Focused on the annoying messages from the list, the brackets are just characters in a stream. Looked at from the perspective of the file of procmail recipes, they are control characters.

The second is at the level of programming practice. Don't /dev/null something until you know it's junk. Much better to send the offending messages to a junk mbox first:

    * ^Subject.*\[billyjoel\]

Once I see that all and only the messages from the list are being matched by the pattern, I can change that line send list traffic where it belongs. That's a specific example of the sort of defensive programming that we all should practice. Don't commit to solutions too soon.

This, too, relates to more general programming advice about software validation and verification. I should have exercised a few test cases to validate my recipe before turning it loose unsupervised on my live mail stream.

I teach my students this mindset and program that way myself, at least most of the time. Of course, the time you most need test cases will be the time you don't write them.

The day provided a bit of irony to make the story even better. The topic of today's session in my compilers course? Writing regular expressions to describe the tokens in a language. So, after my mail admin colleague and I had a good laugh at my expense, I got to tell the story to my students, and they did, too.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

August 31, 2013 11:32 AM

A Good Language Conserves Programmer Energy

Game programmer Jeff Wofford wrote a nice piece on some of the lessons he learned by programming a game in forty-eight hours. One of the recurring themes of his article is the value of a high-powered scripting language for moving fast. That's not too surprising, but I found his ruminations on this phenomenon to be interesting. In particular:

A programmer's chief resource is the energy of his or her mind. Everything that expends or depletes that energy makes him or her less effective, more tired, and less happy.

A powerful scripting language sitting atop the game engine is one of the best ways to conserve programmer energy. Sometimes, though, a game programmer must work hard to achieve the performance required by users. For this reason, Wofford goes out of his way not to diss C++, the tool of choice for many game programmers. But C++ is an energy drain on the programmer's mind, because the programmer has to be in a constant state of awareness of machine cycles and memory consumption. This is where the trade-off with a scripting language comes in:

When performance is of the essence, this state of alertness is an appropriate price to pay. But when you don't have to pay that price -- and in every game there are systems that have no serious likelihood of bottlenecking -- you will gain mental energy back by essentially ignoring performance. You cannot do this in C++: it requires an awareness of execution and memory costs at every step. This is another argument in favor of never building a game without a good scripting language for the highest-level code.

I think this is true of almost every large system. I sure wish that the massive database systems at the foundation of my university's operations had scripting languages sitting on top. I even want to script against the small databases that are the lingua franca of most businesses these days -- spreadsheets. The languages available inside the tools I use are too clunky or not powerful, so I turn to Ruby.

Unfortunately, most systems don't come with a good scripting language. Maybe the developers aren't allowed to provide one. Too many CS grads don't even think of "create a mini-language" as a possible solution to their own pain.

Fortunately for Wofford, he both has the skills and inclination. One of his to-dos after the forty-eight hour experience is all about language:

Building a SWF importer for my engine could work. Adding script support to my engine and greatly refining my tools would go some of the distance. Gotta do something.

Gotta do something.

I'm teaching our compiler course again this term. I hope that the dozen or so students in the course leave the university knowing that creating a language is often the right next action and having the skills to do it when they feel compelled to do something.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

August 29, 2013 4:31 PM

Asimov Sees 2014, Through Clear Eyes and Foggy

Isaac Asimov, circa 1991

A couple of years ago, I wrote Psychohistory, Economics, and AI, in which I mentioned Isaac Asimov and one way that he had influenced me. I never read Asimov or any other science fiction expecting to find accurate predictions of future. What drew me in was the romance of the stories, dreaming "what if?" for a particular set of conditions. Ultimately, I was more interested in the relationships among people under different technological conditions than I was in the technology itself. Asimov was especially good at creating conditions that generated compelling human questions.

Some of the scenarios I read in Asimov's SF turned out to be wildly wrong. The world today is already more different from the 1950s than the world of the Foundation, set thousands of years in the future. Others seem eerily on the mark. Fortunately, accuracy is not the standard by which most of us judge good science fiction.

But what of speculation about the near future? A colleague recently sent me a link to Visit to the World's Fair of 2014, an article Asimov wrote in 1964 speculating about the world fifty years hence. As I read it, I was struck by just how far off he was in some ways, and by how close he was in others. I'll let you read the story for yourself. Here are a few selected passages that jumped out at me.

General Electric at the 2014 World's Fair will be showing 3-D movies of its "Robot of the Future," neat and streamlined, its cleaning appliances built in and performing all tasks briskly. (There will be a three-hour wait in line to see the film, for some things never change.)

3-D movies are now common. Housecleaning robots are not. And while some crazed fans will stand in line for many hours to see the latest comic-book blockbuster, going to a theater to see a movie has become much less important part of the culture. People stream movies into their homes and into their hands. My daughter teases me for caring about the time any TV show or movie starts. "It's on Hulu, Dad." If it's not on Hulu or Netflix or the open web, does it even exist?

Any number of simultaneous conversations between earth and moon can be handled by modulated laser beams, which are easy to manipulate in space. On earth, however, laser beams will have to be led through plastic pipes, to avoid material and atmospheric interference. Engineers will still be playing with that problem in 2014.

There is no one on the moon with whom to converse. Sigh. The rest of this passage sounds like fiber optics. Our world is rapidly becoming wireless. If your device can't connect to the world wireless web, does it even exist?

In many ways, the details of technology are actually harder to predict correctly than the social, political, economic implications of technological change. Consider:

Not all the world's population will enjoy the gadgety world of the future to the full. A larger portion than today will be deprived and although they may be better off, materially, than today, they will be further behind when compared with the advanced portions of the world. They will have moved backward, relatively.

Spot on.

When my colleague sent me the link, he said, "The last couple of paragraphs are especially relevant." They mention computer programming and a couple of its effects on the world. In this regard, Asimov's predictions meet with only partial success.

The world of A.D. 2014 will have few routine jobs that cannot be done better by some machine than by any human being. Mankind will therefore have become largely a race of machine tenders. Schools will have to be oriented in this direction. ... All the high-school students will be taught the fundamentals of computer technology will become proficient in binary arithmetic and will be trained to perfection in the use of the computer languages that will have developed out of those like the contemporary "Fortran" (from "formula translation").

The first part of this paragraph is becoming truer every day. Many people husband computers and other machines as they do tasks we used to do ourselves. The second part is, um, not true. Relatively few people learn to program at all, let alone master a programming language. And how many people understand this t-shirt without first receiving an impromptu lecture on the street?

Again, though, Asimov is perhaps closer on what technological change means for people than on which particular technological changes occur. In the next paragraph he says:

Even so, mankind will suffer badly from the disease of boredom, a disease spreading more widely each year and growing in intensity. This will have serious mental, emotional and sociological consequences, and I dare say that psychiatry will be far and away the most important medical specialty in 2014. The lucky few who can be involved in creative work of any sort will be the true elite of mankind, for they alone will do more than serve a machine.

This is still speculation, but it is already more true than most of us would prefer. How much truer will it be in a few years?

My daughters will live most of their lives post-2014. That worries the old fogey in me a bit. But it excites me more. I suspect that the next generation will figure the future out better than mine, or the ones before mine, can predict it.


PHOTO. Isaac Asimov, circa 1991. Britannica Online for Kids. Web. 2013 August 29.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

August 22, 2013 2:45 PM

A Book of Margin Notes on a Classic Program?

I recently stumbled across an old How We Will Read interview with Clive Thompson and was intrigued by his idea for a new kind of annotated book:

I've had this idea to write a provocative piece, or hire someone to write it, and print it on-demand it with huge margins, and then send it around to four people with four different pens -- red, blue, green and black. It comes back with four sets of comments all on top of the text. Then I rip it all apart and make it into an e-book.

This is an interesting mash-up of ideas from different eras. People have been writing in the margins of books for hundreds of years. These days, we comment on blog entries and other on-line writing in plain view of everyone. We even comment on other people's comments. Sites such as, home of the Thompson interview, aim to bring this cultural practice to everything digital.

Even so, it would be pretty cool to see the margin notes of three or four insightful, educated people, written independently of one another, overlaid in a single document. Presentation as an e-book offers another dimension of possibilities.

Ever the computer scientist, I immediately began to think of programs. A book such as Beautiful Code gives us essays from master programmers talking about their programs. Reading it, I came to appreciate design decisions that are usually hidden from readers of finished code. I also came to appreciate the code itself as a product of careful thought and many iterations.

My thought is: Why not bring Thompson's mash-up of ideas to code, too? Choose a cool program, perhaps one that changed how we work or think, or one that unified several ideas into a standard solution. Print it out with huge margins, and send it to three of four insightful, thoughtful programmers who read it, again or for the first time, and mark it up with their own thoughts and ideas. It comes back with four sets of comments all on top of the text. Rip it apart and create an e-book that overlays them all in a single document.

Maybe we can skip the paper step. Programming tools and Web 2.0 make it so easy to annotate documents, including code, in ways that replace handwritten comments. That's how most people operate these days. I'm probably showing my age in harboring a fondness for the written page.

In any case, the idea stands apart from the implementation. Wouldn't it be cool to read a book that interleaves and overlays the annotations made by programmers such as Ward Cunningham and Grady Booch as they read John McCarthy's first Lisp interpreter, the first Fortran compiler from John Backus's team, QuickDraw, or Qmail? I'd stand in line for a copy.

Writing this blog entry only makes the idea sound more worth doing. If you agree, I'd love to hear from you -- especially if you'd like to help. (And especially if you are Ward Cunningham and Grady Booch!)

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

July 18, 2013 2:22 PM

AP Computer Science in Iowa High Schools

Mark Guzdial posted a blog entry this morning pointing to a Boston Globe piece, Interest in computer science lags in Massachusetts schools. Among the data supporting this assertion was participation in Advanced Placement:

Of the 85,753 AP exams taken by Massachusetts students last year, only 913 were in computing.

Those numbers are a bit out of context, but they got me to wondering about the data for Iowa. So I tracked down this page on AP Program Participation and Performance Data 2012 and clicked through to the state summary report for Iowa. The numbers are even more dismal than Massachusetts's.

Of the 16,413 AP exams taken by Iowa students in 2012, only sixty-nine were in computer science. The counts for groups generally underrepresented in computing were unsurprisingly small, given that Iowa is less diverse than many US states. Of the sixty-nine, fifty-four self-reported as "white", ten as "Asian", and one as "Mexican-American", with four not indicating a category.

The most depressing number of all: only nine female students took the AP Computer Science exam last year in Iowa.

Now, Massachusetts has roughly 2.2 times as many people as Iowa, but even so Iowa compares unfavorably. Iowans took about one-fifth as many AP exams as many Massachusetts students, and for CS the percentage drops to 7.5%. If AP exams indicate much about the general readiness of a state's students for advanced study in college, then Iowa is at a disadvantage.

I've never been a huge proponent of the AP culture that seems to dominate many high schools these days (see, for instance, this piece), but the low number of AP CS exams taken in Iowa is consistent with what I hear when I talk to HS students from around the state and their parents: Iowa schools are not teaching much computer science at all. The university is the first place most students have an opportunity to take a CS course, and by then the battle for most students' attention has already been lost. For a state with a declared goal of growing its promising IT sector, this is a monumental handicap.

Those of us interested in broadening participation in CS face an even tougher challenge. Iowa's demographics create some natural challenges for attracting minority students to computing. And if the AP data are any indication, we are doing a horrible job of reaching women in our high schools.

There is much work to do.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

July 17, 2013 1:41 PM

And This Gray Spirit Yearning

On my first day as a faculty member at the university, twenty years ago, the department secretary sent me to Public Safety to pick up my office and building keys. "Hi, I'm Eugene Wallingford," I told the person behind the window, "I'm here to pick up my keys." She smiled, welcomed me, and handed them to me -- no questions asked.

Back at the department, I commented to one of my new colleagues that this seemed odd. No one asked to see an ID or any form of authorization. They just handed me keys giving me access to a lot of cool stuff. My colleague shrugged. There has never been a problem here with unauthorized people masquerading as new faculty members and picking up keys. Until there is a problem, isn't it nice living in a place where trust works?

Things have changed. These days, we don't order keys for faculty; we "request building access". This phrase is more accurate than a reference to keys, because it includes activating the faculty ID to open electronically-controlled doors. And we don't simply plug a new computer into an ethernet jack and let faculty start working; to get on the wireless network, we have to wait for the Active Directory server to sync with the HR system, which updates only after electronic approval of a Personnel Authorization Form that set up of the employee's payroll record. I leave that as a run-on phrase, because that's what living it feels like.

The paperwork needed to get a new faculty member up and running these days reminds me just how simple life was when in 1992. Of course, it's not really "paperwork" any more.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

July 15, 2013 2:41 PM

Version Control for Writers and Publishers

Mandy Brown again, this time on on writing tools without memory:

I've written of the web's short-term memory before; what Manguel trips on here is that such forgetting is by design. We designed tools to forget, sometimes intentionally so, but often simply out of carelessness. And we are just as capable of designing systems that remember: the word processor of today may admit no archive, but what of the one we build next?

This is one of those places where the software world has a tool waiting to reach a wider audience: the version control system. Programmers using version control can retrieve previous states of their code all the way back to its creation. The granularity of the versions is limited only by the frequency with which they "commit" the code to the repository.

The widespread adoption of version control and the existence of public histories at place such as GitHub have even given rise to a whole new kind of empirical software engineering, in which we mine a large number of repositories in order to understand better the behavior of developers in actual practice. Before, we had to contrive experiments, with no assurance that devs behaved the same way under artificial conditions.

Word processors these days usually have an auto-backup feature to save work as the writer types text. Version control could be built into such a feature, giving the writer access to many previous versions without the need to commit changes explicitly. But the better solution would be to help writers learn the value of version control and develop the habits of committing changes at meaningful intervals.

Digital version control offers several advantages over the writer's (and programmer's) old-style history of print-outs of previous versions, marked-up copy, and notebooks. An obvious one is space. A more important one is the ability to search and compare old versions more easily. We programmers benefit greatly from a tool as simple as diff, which can tell us the textual differences between two files. I use diff on non-code text all the time and imagine that professional writers could use it to better effect than I.

The use of version control by programmers leads to profound changes in the practice of programming. I suspect that the same would be true for writers and publishers, too.

Most version control systems these days work much better with plain text than with the binary data stored by most word processing programs. As discussed in my previous post, there are already good reasons for writers to move to plain text and explicit mark-up schemes. Version control and text analysis tools such as diff add another layer of benefit. Simple mark-up systems like Markdown don't even impose much burden on the writer, resembling as they do how so many of us used to prepare text in the days of the typewriter.

Some non-programmers are already using version control for their digital research. Check out William Turkel's How To for doing research with digital sources. Others, such The Programming Historian and A Companion to Digital Humanities, don't seem to mention it. But these documents refer mostly to programs for working with text. The next step is to encourage adoption of version control for writers doing their own thing: writing.

Then again, it has taken a long time for version control to gain such widespread acceptance even among programmers, and it's not yet universal. So maybe adoption among writers will take a long time, too.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 12, 2013 3:08 PM

"Either Take Control, or Cede it to the Software"

Mandy Brown tells editors and "content creators" to take control of their work:

It's time content people of all stripes recognized the WYSIWYG editor for what it really is: not a convenient shortcut, but a dangerous obstacle placed between you and the actual content. Because content on the web is going to be marked up one way or another: you either take control of it or you cede it to the software, but you can't avoid it. WYSIWYG editors are fine for amateurs, but if you are an editor, or copywriter, or journalist, or any number of the kinds of people who work with content on the web, you cannot afford to be an amateur.

Pros can sling a little code, too.

Brown's essay reminded me of a blog entry I was discussing with a colleague recently, Andrew Hayes's Why Learn Syntax? Hayes tells statisticians that they, too, should take control of their data, by learning the scripting language of the statistical packages they use. Code is a record of an analysis, which allows it to be re-run and shared with others. Learning to write code also hones one's analytical skills and opens the door to features not available through the GUI.

These articles speak to two very different audiences, but the message is the same. Don't just be a user of someone else's tools and be limited to their vision. Learn to write a little code and take back the power to create.

Posted by Eugene Wallingford | Permalink | Categories: Computing

July 11, 2013 2:57 PM

Talking to the New University President about Computer Science

Our university recently hired a new president. Yesterday, he and the provost came to a meeting of the department heads in humanities, arts, and sciences, so that he could learn a little about the college. The dean asked each head to introduce his or her department in one minute or less.

I came in under a minute, as instructed. Rather than read a litany of numbers that he can read in university reports, I focused on two high-level points:

  • Major enrollment has recovered nicely since the deep trough after the bust and is now steady. We have near-100% placement, but local and state industry could hire far more graduates.
  • For the last few years we have also been working to reach more non-majors, which is a group we under-serve relative to most other schools. This should be an important part of the university's focus on STEM and STEM teacher education.

I closed with a connection to current events:

We think that all university graduates should understand what 'metadata' is and what computer programs can do with it -- enough so that they can understand the current stories about the NSA and be able to make informed decisions as a citizen.

I hoped that this would be provocative and memorable. The statement elicited laughs and head nods all around. The president commented on the Snowden case, asked me where I thought he would land, and made an analogy to The Man Without a Country. I pointed out that everyone wants to talk about Snowden, including the media, but that's not even the most important part of the story. Stories about people are usually of more interest than stories about computer programs and fundamental questions about constitutional rights.

I am not sure how many people believe that computer science is a necessary part of a university education these days, or at least the foundations of computing in the modern world. Some schools have computing or technology requirements, and there is plenty of press for the "learn to code" meme, even beyond the CS world. But I wonder how many US university graduates in 2013 understand enough computing (or math) to understand this clever article and apply that understand to the world they live in right now.

Our new president seemed to understand. That could bode well for our department and university in the coming years.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 08, 2013 1:05 PM

A Random Thought about the Metadata and Government Surveillance

In a recent mischievous mood, I decided it might be fun to see the following.

The next whistleblower with access to all the metadata that the US government is storing on its citizens assembles a broad list of names: Republican and Democrat; legislative, executive, and judicial branches; public official and private citizens. The only qualification for getting on the list is that the person has uttered any variation of the remarkably clueless statement, "If you aren't doing anything wrong, then you have nothing to hide."

The whistleblower thens mine the metadata and, for each person on this list, publishes a brief that demonstrates just how much someone with that data can conclude -- or insinuate -- about a person.

If they haven't done anything wrong, then they don't have anything to worry about. Right?

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 07, 2013 9:32 AM

Interesting Sentences, Personal Weakness Edition

The quest for comeuppance is a misallocation of personal resources. -- Tyler Cowen

Far too often, my reaction to events in the world around me is to focus on other people not following rules, and the unfairness that results. It's usually not my business, and even when it is, it's a foolish waste of mental energy. Cowen expresses this truth nicely in neutral, non-judgmental language. That may help me develop a more productive mental habit.

What we have today is a wonderful bike with training wheels on. Nobody knows they are on, so nobody is trying to take them off. -- Alan Kay, paraphrased from The MIT/Brown Vannevar Bush Symposium

Kay is riffing off Douglas Engelbart's tricycle analogy, mentioned last time. As a computer scientist, and particularly one fortunate enough to have been exposed to the work of Ivan Sutherland, Englebart, Kay and the Xerox PARC team, and so many others, I should be more keenly conscious that we are coasting along with training wheels on. I settle for limited languages and limited tools.

Even sadder, when computer scientists and software developers settle for training wheels, we tend to limit everyone else's experience, too. So my apathy has consequences.

I'll try to allocate my personal resources more wisely.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

July 06, 2013 9:57 AM

Douglas Engelbart Wasn't Just Another Computer Guy

Bret Victor nails it:

Albert Einstein, Discoverer of Photoelectric Effect, Dies at 76

In the last few days, I've been telling family and friends about Engelbart's vision and effect on the world the computing, and thus on their world. He didn't just "invent the mouse".

It's hard to imagine these days just how big Engelbart's vision was for the time. Watching The Mother of All Demos now, it's easy to think "What's the big deal? We have all that stuff now." or even "Man, that video looks prehistoric." First of all, we don't have all that stuff today. Watch again. Second, in a sense, that demo was prehistory. Not only did we not have such technology at the time, almost no one was thinking about it. It's not that people thought such things were impossible; they couldn't think about them at all, because no one had conceived them yet. Engelbart did.

Engelbart didn't just invent a mouse that allows us to point at files and web links. His ideas helped point an entire industry toward the future.

Like so many of our computing pioneers, though, he dreamed of more than what we have now, and expected -- or at least hoped -- that we would build on the industry's advanced to make his vision real. Engelbart understood that skills which make people productive are probably difficult to learn. But they are so valuable that the effort is worth it. I'm reminded of Alan Kay's frequent use of a violin as an example, compared to a a simpler music-making device, or even to a radio. Sure, a violin is difficult to play well. But when you can play -- wow.

Engelbart was apparently fond of another example: the tricycle

Riding a bicycle -- unlike a tricycle -- is a skill that requires a modest degree of practice (and a few spills), but the rider of a bicycle quickly outpaces the rider of a tricycle.

Most of the computing systems we use these days are tricycles. Doug Engelbart saw a better world for us.

Posted by Eugene Wallingford | Permalink | Categories: Computing

July 03, 2013 10:22 AM

Programming for Everyone, Venture Capital Edition

Christina Cacioppo left Union Square Ventures to learn how to program:

Why did I want to do something different? In part, because I wanted something that felt more tangible. But mostly because the story of the internet continues to be the story of our time. I'm pretty sure that if you truly want to follow -- or, better still, bend -- that story's arc, you should know how to write code.

So, rather than settle for her lot as a non-programmer, beyond the accepted school age for learning these things -- technology is a young person's game, you know -- Cacioppo decided to learn how to build web apps. And build one.

When did we decide our time's most important form of creation is off-limits? How many people haven't learned to write software because they didn't attend schools that offered those classes, or the classes were too intimidating, and then they were "too late"? How much better would the world be if those people had been able to build their ideas?

Yes, indeed.

These days, she is enjoying the experience of making stuff: trying ideas out in code, discarding the ones that don't work, and learning new things every day. Sounds like a programmer to me.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

July 01, 2013 11:10 AM

Happy in my App

In a typically salty post, Jamie Zawinski expresses succinctly one of the tenets of my personal code:

I have no interest in reading my feeds through a web site (no more than I would tolerate reading my email that way, like an animal).

Living by this code means that, while many of my friends and readers are gnashing their teeth on this first of July, my life goes on uninterrupted. I remain a happy long-time user of NetNewsWire (currently, v3.1.6).

Keeping my feeds in sync with NetNewsWire has always been a minor issue, as I run the app on at least two different computers. Long ago, I wrote a couple of extremely simple scripts -- long scp commands, really -- that do a pretty good job. They don't give me thought-free syncing, but that's okay.

A lot of people tell me that apps are dead, that the HTML5-powered web is the future. I do know that we're very quickly running out of stuff we can't do in the browser and applaud the people who are making that happen. If I were a habitual smartphone-and-tablet user, I suspect that I would be miffed if web sites made me download an app just to read their web content. All that said, though, I still like what a clean, simple app gives me.

Posted by Eugene Wallingford | Permalink | Categories: Computing

June 26, 2013 2:30 PM

An Opportunity to Learn, Born of Deprivation

Earlier this summer, my daughter was talking about something one of her friends had done with Instagram. As a smug computer weenie, I casually mentioned that she could do that, too.

She replied, "Don't taunt me, Dad."

You see, no one in our family has a cell phone, smart or otherwise, so none of us use Instagram. That's not a big deal for dear old dad, even though (or perhaps because) he's a computer scientist. But she is a teenager growing up in an entirely different world, filled with technology and social interaction, and not having a smart phone must surely seem like a form of child abuse. Occasionally, she reminds us so.

This gave me a chance to explain that Instagram filters are, at their core, relatively simple little programs, and that she could learn to write them. And if she did, she could run them on almost any computer, and make them do things that even Instagram doesn't do.

I had her attention.

So, this summer I am going to help her learn a little Python, using some of the ideas from media computation. At the end of our first pass, I hope that she will be able to manipulate images in a few basic ways: changing colors, replacing colors, copying pixels, and so on. Along the way, we can convert color images to grayscale or sepia tones, posterize images, embed images, and make simple collages.

That will make her happy. Even if she never feels the urge to write code again, she will know that it's possible. And that can be empowering.

I have let my daughter know that we probably will not write code that does as good a job as what she can see in Instagram or Photoshop. Those programs are written by pros, and they have evolved over time. I hope, though, that she will appreciate how simple the core ideas are. As James Hague said in a recent post, then key idea in most apps require relatively few lines of code, with lots and lots of lines wrapped around them to handle edge cases and plumbing. We probably won't write much code for plumbing... unless she wants to.

Desire and boredom often lead to creation. They also lead to the best kind of learning.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

June 14, 2013 2:48 PM

The History of Achievement in AI

... often looks something like this:

  1. X := some task that people do well, but not computers.
  2. "It would really be impressive if a computer could X."
  3. Computer does X.
  4. "That's not intelligent. The computer is only doing search (or number crunching, or ...)."
  5. X := something else.
  6. Go to 2.

A common variation of this pattern is to replace Step 3 with a different dodge:

"That's no big deal. X doesn't really require intelligence."

In either case, the target moves.

Occasionally, the critic must admit, if grudgingly, that the task requires intelligence, whatever that means, and that the computer performs it well. But there is still one last move available to deflect the achievement from the computer:

"This is a human achievement. People had to program the computer."

I suspect that until a computer learns everything it knows from scratch -- whatever that means -- this pattern will repeat. We humans have an image to protect.


Postscript. I wrote this after reading a short interview interview with playwright Matt Charman, who has dramatized Deep Blue's epic 1997 match win over world chess champion Garry Kasparov. Note that Charman does not employ the dodges I list. He simply chose to focus on the human personalities involved in the drama. And those personalities are worthy of exploration, especially the fascinating Kasparov!

Posted by Eugene Wallingford | Permalink | Categories: Computing

June 13, 2013 3:01 PM

It's Okay to Talk About Currying!

James Hague offers some sound advice for writing functional programming tutorials. I agree with most of it, having learned the hard way by trying to teach functional style to university students for many years. But I disagree with one of his suggestions: I think it's okay to talk about currying.

Hague's concern with currying is practical:

Don't get so swept up in that theory that you forget the obvious: in any programming language ever invented, there's already a way to easily define functions of multiple arguments. That you can build this up from more primitive features is not useful or impressive to non-theoreticians.

Of course, my context is a little different. We teach functional programming in a course on programming languages, so a little theory is important. We want students not only to be able to write code in a functional style but also to understand some of the ideas at the foundation of the languages they use. We also want them to understand a bit about how different programming styles relate to one another.

But even in the context of teaching people to think functionally and to write code in that style, I think it's okay to talk about currying. Indeed, it is essential. Currying is not simply a theoretical topic. It is a valuable programming technique.

Here is an example. When we write a language interpreter, we often write a procedure names eval-exp. It takes two arguments: an expression to evaluate, and a list of variable/value bindings.

   (define eval-exp
     (lambda (exp env)

The binding list, sometimes called an environment, is a map of names declared in the local block to their values, along with the bindings from the blocks that contain the local block. Each time the interpreter enters a new block, it pushes a new set of name/value pairs onto the binding list and recurses.

To evaluate a function call for which arguments are passed by value, the interpreter must first evaluate all of the function's arguments. As the arguments are all in the same block, they are evaluated using the same binding list. We could write a new procedure to evaluate the arguments recursively, but this seems like a great time to map a procedure over a list: (map eval-exp args), get a list of the results, and pass them to the code that applies the function to them.

We can't do that, though, because eval-exp is a two-argument procedure, and map works only with a one-argument procedure. But the same binding list is used to evaluate each of the expressions, so that argument to eval-exp is effectively a constant for the purposes of the mapping operation.

So we curry eval-exp:

   (define eval-exp-with
     (lambda (bindings)
       (lambda (exp)
         (eval-exp exp bindings))))

... to create the one-argument evaluator that we need, and we use it to evaluate the arguments with map:

   ; in eval-exp
   (map (eval-exp-with env) arguments)

In most functional languages, we can use a nameless lambda to curry eval-exp "in place" and avoid writing an explicit helper function:

   ; an alternative approach in eval-exp
   (map (lambda (exp)
          (eval-exp exp bindings))

This doesn't look much like currying because we never created the procedure that takes the bindings argument. But we can reach this same piece of code by writing eval-exp-with, calling it in eval-exp, and then using program derivation to substitute the value of the call for the call itself. This is actually a nice connection to be able to make in a course about programming languages!

When I deliver short tutorials on functional style, currying often does not make the cut, because there are so many cool and useful ideas to cover. But it doesn't take long writing functional code before currying becomes useful. As this example shows, currying is a practical tool for the functional programmer to have in his or her pocket. In FP, currying isn't just theory. It's part of the style.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

June 05, 2013 1:52 PM

I Fooled Around and Fell in Love

Cue the Elvin Bishop [ video ]...

I smile whenever I see this kind of statement on a website's About page:

Erika Carlson was studying clinical psychology in 2011, when she wrote her first line of Python code. She fell in love with programming, decided to change paths, and is now a software developer at Pillar Technology.

I fell in love upon writing my first line of code, too.

Not everyone will have the same reaction Erika and I had, but it's good that we give people at least an opportunity to learn how to program. Knowing that someone might react this way focuses my mind on giving novice programmers a good enough experience that they can, if they are so inclined.

My teaching should never get in the way of true love.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

May 31, 2013 1:44 PM

Quotes of the Week, in Four Dimensions


Michael Bernstein, in A Generation Ago, A Thoroughly Modern Sampling:

The AI Memos are an extremely fertile ground for modern research. While it's true that what this group of pioneers thought was impossible then may be possible now, it's even clearer that some things we think are impossible now have been possible all along.

When I was in grad school, we read a lot of new and recent research papers. But the most amazing, most educational, and most inspiring stuff I read was old. That's often true today as well.


Financial Agile tweets:

"If it disagrees with experiment, it's wrong". Classic.

... with a link to The Scientific Method with Feynman, which has a wonderful ten-minute video of the physicist explaining how science works. Among its important points is that guessing is huge part of science. It's just that scientists have a way of telling which guesses are right and which are wrong.


James Boyk, in Six Words:

Like others of superlative gifts, he seemed to think the less gifted could do as well as he, if only they knew a few powerful specifics that could readily be conveyed. Sometimes he was right!

"He" is Leonid Hambro, who played with Victor Borge and P. D. Q. Bach but was also well-known as a teacher and composer. Among my best teachers have been some extraordinarily gifted people. I'm thankful for the time they tried to convey their insights to the likes of me.


Amanda Palmer, in a conference talk:

We can only connect the dots that we collect.

Palmer uses this sentence to explain in part why all art is about the artist, but it means something more general, too. You can build, guess, and teach only with the raw materials that you assemble in your mind and your world. So collect lots of dots. In this more prosaic sense, Palmer's sentence applies to not only to art but also to engineering, science, and teaching.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

May 10, 2013 4:03 PM

Using Language to Understand a Data Set

Today was our twice-annual undergraduate research presentation day. Every B.S. student must do an undergraduate research project and present the results publicly. For the last few years, we have pooled the presentations on the morning of the Friday in finals week, after all the exams are given and everyone has a chunk of time free to present. It also means that more students and professors can attend, which makes for more a more engaging audience and a nice end to everyone's semester.

I worked with one undergraduate research student this spring. As I mentioned while considering the role of parsing in a compilers course, this student was looking for patterns in several years of professional basketball play-by-play data. His ultimate goal was to explore ways of measuring the impact of individual defensive performance in the NBA -- fairly typical MoneyBall stuff applied to an skill that is not well measured or understood.

This project fell into my hands serendipitously. The student had approached a couple of other professors, who upon hearing the word "basketball" immediately pointed him to me. Of course, the project is really a data analytics project that just happens to involve a dataset from basketball, but... Fortunately, I am interested in both the approach and the domain!

As research sometimes does, this problem led the student to a new problem first. In order to analyze data in the way he wanted, he needed data of a particular sort. There is plenty of play-by-play data available publicly on the web, but it's mostly prepared for presentation in HTML. So he first had to collect the data by scraping the web, and then organize it into a data format amenable to analysis.

This student had taken my compiler course the last time around, and his ability to parse several files of similar but just-different-enough data proved to be invaluable. As presented on sites like, the data is no where near ready to be studied.

As the semester wore on, he and I came to realize that his project this semester wouldn't be the data analysis he originally intended to do. It was a substantial project simply to make sense of the data he had found.

As he presented his work today, I realized something further. He was using language to understand a data set.

He started by defining a grammar to model the data he found, so that he could parse it into a database. This involved recognizing categories of expression that were on the surface of the data, such as made and missed field goals, timeouts, and turnovers. When he ran this first version of his parser, he found unhandled entries and extended his grammar.

Then he looked at the semantics of the data and noticed discrepancies deeper in the data. The number of possessions his program observed in a game differed from the expected values, sometimes wildly and with no apparent pattern.

As we looked deeper, we realized that the surface syntax of the data often obscured some events that would extend or terminate a possession. A simple example is a missed FT, which sometimes ends a possession and sometimes not. It depends in part on the next event in the timeline.

To handle these case, the student created new syntactic categories that enabled his parser to resolve such issues by recognized composite events in the data. As he did this, his grammar grew, and his parser became better at building a more accurate semantic model of the game.

This turned out to be a semester-long project in its own right. He's still not done and intends to continue with this research after graduation. We were both a bit surprised at how much effort it took to corral the data, but in retrospect we should not have been too surprised. Data are collected and presented with many different purposes in mind. Having an accurate deep model of the underlying the phenomenon in question isn't always one of them.

I hope the student was pleased with his work and progress this semester. I was. In addition to its practical value toward solving a problem of mutual interest, it reminded me yet again of the value of language in understanding the world around us, and the remarkable value that the computational ideas we study in computer science have to offer. For some reason, it also reminded me, pleasantly, of the Racket Way. As I noted in that blog entry, this is really the essence of computer science.

Of course, if some NBA team were to give my student the data he needs in suitable form, he could dive into the open question of how better to measure individual defensive performance in basketball. He has some good ideas, and the CS and math skills needed to try them out.

Some NBA team should snatch this guy up.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

May 08, 2013 12:11 PM

Not So Random Sentences

I start with a seemingly random set of sentences to blog about and, in the process of writing about them, find that perhaps they aren't so random after all.

An Era of Sharing Our Stuff

Property isn't theft; property is an inefficient distribution of resources.

This assertion comes from an interesting article on "economies of scale as a service", in reaction to a Paul Graham tweet:

Will ownership turn out to be largely a hack people resorted to before they had the infrastructure to manage sharing properly?

Open-source software, the Creative Commons, crowdsourcing. The times they are a-changin'.

An Era of Observing Ourselves

If the last century was marked by the ability to observe the interactions of physical matter -- think of technologies like x-ray and radar -- this century is going to be defined by the ability to observe people through the data they share.

... from The Data Made Me Do It.

I'm not too keen on being "observed" via data by every company in the world, even as understand the value it can brings the company and even me. But I like very much the idea that I can observe myself more easily and more productively. For years, I collected and studied data about my running and used what I learned to train and race better. Programmers are able to do this better now than ever before. You can learn a lot just by watching.

An Era of Thinking Like Scientist

... which leads to this line attributed to John C. Reynolds, an influential computer scientist who passed away recently:

Well, we know less than we did before, but more of what we know is actually true.

It's surprising how easy it is to know stuff when we don't have any evidence at all. Observing the world methodically, building models, and comparing them to what we observe in the future helps to know less of the wrong stuff and more of the right stuff.

Not everyone need be a scientist, but we'd all be better off if more of us thought like a scientist more often.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Personal

April 21, 2013 10:25 AM

Catnip for Programmers

This morning, Maciej Ceglowski of Pinboard introduced me to the Matasano crypto challenges, a set of exercises created by Thomas Ptacek and his team as a tool for teaching programmers a little about cryptography, some of its challenges, and the need for more awareness of how easy it is to do it wrong. With the coming break from the grind of the academic year, I plan on giving them a try.

After having completed the exercises himself, Ceglowski observes:

Crypto is like catnip for programmers. It is hard to keep us away from it, because it's challenging and fun to play with. And programmers respond very badly to the insinuation that they're not clever enough to do something. We see the F-16 just sitting there, keys in the ignition, no one watching, lights blinking, ladder extended. And some infosec nerd is telling us we can't climb in there, even though we just want to taxi around a little and we've totally read the manual.

I've noticed this with a number of topics in computing. In addition to cryptography, data compression and sorting/searching are sirens to the best programmers among our students. "What do you mean we can't do better?"

For many undergrads, the idea of writing a compiler seems a mystery. Heck, I admit to my students that even after years of teaching the course I remain in awe of my language tools and the people who build them. This challenge keeps a steady if relatively small stream of programmers flowing into our "Translation of Programming Languages" project course.

One of the great things about all these challenges is that after we tackle them, we have not only the finished product in hand but also what we learn about the topic -- and ourselves -- along the way. Then we are ready for a bigger challenge, and another program to write.

For CS faculty, catnip topics are invaluable ways to draw more students into the spell of computing, and more deeply. We are always on the lookout.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

April 20, 2013 10:25 AM

Reminiscing about Making My First Computer

Steve Wozniak

A friend put a copy of GameInformer magazine in my box yesterday with a pointer to an interview with the Great and Powerful Woz, Steve Wozniak. It's a short interview, only two pages, but it reminded me just how many cool things Wozniak (and so many others) did in the mid-1970s. It also reminded me of my younger days, coming into contact with the idea of games and machine learning for the first time.

Woz described how, after seeing Pong in a video arcade, he went home and built his own Pong game out of twenty-eight $1 chips. Steve Jobs took the game to Atari, where he encountered Nolan Bushnell, who had an idea for a single-player version of Pong. Thus did Woz design Breakout, a game with an especially apt name. It helped define Apple Computer.

The thought of building a computer game out of chips still amazes me. I was never a hardware guy growing up. I never had access to computer chips or that culture, and I had little inclination to fiddle with electronics, save for a few attempts to take apart radios and put them back together. When I designed things as a kid, they were houses or office buildings. I was going to be an architect. But Woz's story reminded me of one experience that foreshadowed my career as a computer scientist.

One year in school, I won a math contest. First prize was a copy of The Unexpected Hanging and Other Mathematical Diversions, a collection of Martin Gardner's columns from Scientific American. Chapter 8 was called "A Matchbox Game-Learning Machine". It described Hexapawn, a game played on a 3x3 board with chess pawns. The game was no more complex than Tic Tac Toe, but it was new. And I loved board games.

Gardner's article had more in store for me, though, than simply another game to study. He described how to create a "computer" -- a system of matchboxes -- that learns how to play the game! Here's how:

You make one box for each possible board position. In the box, you put different colored marbles corresponding to the moves that can be played in the position. Then you play a bunch of games against the matchbox computer. When it is the computer's turn to move, you pick up the box for that board position, shake it, and see which marble lands in the lower-right corner of the box. That's the computer's move.

When the game is over, the computer gets feedback. If it won the game, then put all the marbles back in their boxes. If it lost, punish it by keeping the marble responsible for its last move; put all the rest back in their boxes. Gardner claimed that by following this strategy, the matchbox computer would learn to play a perfect game in something under fifty moves.

This can't possibly work, can it? So I built it. And it did learn. I was happy, and amazed.

I remember experimenting a bit. Maybe a move wasn't always a loser? So I seeded the computer with more than one marble for each candidate move, so that the computer could overcome bad luck. Hexapawn is so simple that this wasn't necessary -- losing moves are losing moves -- but the computer still learned to play a perfect game, just a bit slower than before.

This is one of the earliest experiences I remember that started me down the road of studying artificial intelligence. Reading copious amounts of science fiction pushed me in that direction, too, but this was different. I had made something, and it learned. I was hooked.

So, I wasn't a hardware kid, but I had a hardware experience. It just wasn't digital hardware. But my inclination was always more toward ideas than gadgets. My interests quickly turned to writing programs, which made it so much easier to tinker with variations and to try brand-new ideas.

(Not so quickly, though, that I turned away from my dream of being an architect. The time I spent in college studying architecture turned out to be valuable in many ways.)

Wozniak was a hardware guy, but he quickly saw the potential of software. "Games were not yet software, and [the rise of the microprocessor] triggered in my mind: microprocessors can actually program games." He called the BASIC interpreter he wrote "Game BASIC". Ever the engineer, he designed the Apple II with integrated hardware and software so that programmers could write cool games.

I don't have a lot in common with Steve Wozniak, but one thing we share is the fun we have playing games. And, in very different ways, we once made computers that changed our lives.


The GameInformer interview is on-line for subscribers only, but there is a cool video of Wozniak playing Tetris -- and talking about George H.W. Bush and Mikhail Gorbachev!

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

April 05, 2013 3:21 PM

The Role of Parsing in a Compilers Course

I teach compilers again this fall. I'm looking forward to summer, when I'll have a chance to write some code and play with some ideas for the course.

This morning I thought a bit about a topic that pops up every time I prep the course. The thoughts were prompted by a tweet from James Coglan, which said "Really wish this Compilers course weren't half about parsing. ... get on with semantics." The ellipsis is mine; James's tweet said something about using lex/yacc to get something up and running fast. Then, presumably, we could get on to the fun of semantics.

This is a challenge for my compilers course, too. I know I don't want to rush through scanning and parsing, yet I also wish I had more time for static analysis, optimization, and code generation. Even though I know the value of parsing, I wish I had equal time for a lot of other cool topics.

Geoff Wozniak's response expressed one of the reasons parsing still has such a large role in my compilers course, and so many others:

Parsing is like assembly language: it seems superfluous at the time, but provides deep understanding later. It's worth it.

That's part of what keeps me from de-emphasizing it in my course. Former students often report back to me that they have used their skill at writing parsers frequently in their careers, whether for parsing DSLs they whip up or for making sense of a mess of data they want to process.

A current student is doing an undergrad research project that involves finding patterns in several years of professional basketball play-by-play data, and his ability to parse several files of similar but just-different-enough data proved invaluable. Of course, he was a bit surprised that corralling the data took as much effort as it did. Kind of like how scanning and parsing are such a big part of a compiler project.

I see now that James has tweeted a retraction:

... ppl are RTing something I said about wishing the Compilers course would get done with parsing ASAP. Don't believe this any more.

I understand the change of opinion. After going writing a compiler for a big language and learning the intricacies that are possible, it's easy to reach Geoff's position: a deep understanding comes from the experience.

That doesn't mean I don't wish my semester were twenty weeks instead of fifteen, so that I could go deeper on some other topics, too. I figure there will always be some tension in the design of the course for just that reason.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

April 01, 2013 3:16 PM

Good Sentences, Programming State Edition

I've read a couple of interesting papers recently that included memorable sentences related to program state.

First, Stuart Sierra in On the Perils of Dynamic Scope:

Global state is the zombie in the closet of every Clojure program.

This essay explains the difference between scope and extent, a distinction that affects how easy it is to some of what happens in a program with closures and first-order functions with free variables. Sierra also shows the tension between variables of different kinds, using examples from Clojure. An informative read.

Next, Rob Pike in Go at Google: Language Design in the Service of Software Engineering, a write-up of his SPLASH 2012 keynote address:

The motto [of the Go language] is, "Don't communicate by sharing memory, share memory by communicating."

Imperative programmers who internalize this simple idea are on their way to understanding and using functional programming style effectively. The inversion of sharing and communication turns a lot of design and programming patterns inside out.

Pike's notes provide a comprehensive example of how a new language can grow out of the needs of a particular set of applications, rather than out of programming language theory. The result can look a little hodgepodge, but using such a language often feels just fine. (This reminds me of a different classification of languages with similar practical implications.)


(These papers weren't published April Fool's Day, so I don't think I've been punked.)

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

March 27, 2013 12:46 PM

Programming Language as Operating System

We are deep in the semester now, using Racket in our programming languages course. I was thinking recently about how little of Racket's goodness we use in this course. We use it primarily as a souped-up R5RS Scheme and handy IDE. Tomorrow we'll see some of Racket's tools for creating new syntax, which will explore one of the rich niches of the system my students haven't seen yet.

I'm thinking about ways to introduce a deeper understanding of The Racket Way, in which domain concepts are programming language constructs and programming languages are extensible and composable. But it goes deeper. Racket isn't just a language, or a set of languages. It is an integrated family of tools to support language creation and use. To provide all these services, Racket acts like an operating system -- and gives you full programmatic access to the system.

(You can watch the video of Flatt's StrangeLoop talk "The Racket Way" at InfoQ -- and you should.)

The idea is bigger than Racket, of course. Dan Ingalls expressed this idea in his 1981 Byte article, Design Principles Behind Smalltalk:

Operating System: An operating system is a collection of things that don't fit into a language. There shouldn't be one.

Alan Kay talks often about this philosophy. The divide between programming language and operating system makes some things more difficult for programmers, and complicates the languages and tools we use. It also creates a divide in the minds of programmers and imposes unnecessary limitations on what programmers think is possible. One of things that appealed to me in Flatt's StrangeLoop talk is that presented a vision of programming without those limits.

There are implications of this philosophy, and costs. Smalltalk isn't just a language, with compilers and tools that you use at your Unix prompt. It's an image, and a virtual machine, and an environment. You don't use Smalltalk; you live inside it.

After you live in Smalltalk for a while, it feels strange to step outside and use other languages. More important, when you live outside Smalltalk and use traditional languages and tools, Smalltalk feels uncomfortable at best and foreboding at worst. You don't learn Smalltalk; you assimilate. -- At least that's what it feels like to many programmers.

But the upside of the "programming language as operating system" mindset you find in Smalltalk and Racket can be huge.

This philosophy generalizes beyond programming languages. emacs is a text editor that subsumes most everything else you do, if you let it. (Before I discovered Smalltalk in grad school, I lived inside emacs for a couple of years.)

You can even take this down to the level of the programs we write. In a blog entry on delimited continuations, Andy Wingo talks about the control this construct gives the programmer over how their programs work, saying:

It's as if you were implementing a shell in your program, as if your program were an operating system for other programs.

When I keep seeing the same idea pop up in different places, with a form that fits the niche, I'm inclined to think I am seeing one of the Big Ideas of computer science.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

March 20, 2013 4:39 PM

Team Rings, Not Turing Awards

Alan Kay

Alan Kay recently wrote this on the Fundamentals of New Computing mailing list:

My own personal thoughts about what was accomplished [with Smalltalk] are completely intertwined with what our entire group was able to do in a few years at PARC. I would give us credit for a very high level combination of "computer science" and "software engineering" and "human centered design" and "commingled software and hardware", etc. The accomplishment was the group's accomplishment. And this whole (to me at least) was a lot more interesting than just a language idea.

I hasten to redirect personal praise to the group accomplishment whenever it happens.

I think this is also true for the larger ARPA-PARC community, and why it was able to accomplish so much at so many levels.

The "awards to individuals" structure beloved of other fields and of journalists completely misses the nature of this process. Any recognition should be like "World Series" rings -- everybody gets one, and that's it.

When Kay spoke at the 2004 OOPSLA Educators' Symposium as part of his Turing Award festivities, he frequently acknowledged the contributions of his team, in particular Dan Ingalls, and the influence that so many other people had on his team's work. Kay must have particularly appreciated receiving the Charles Stark Draper Prize together with Butler Lampson, Robert Taylor, and Charles Thacker, who helped create the conditions in which his team thrived.

In academia, we talk a lot about teamwork, but we tend to isolate individual performance for recognition. I like Kay's analogy to the rings received by teams that win sports championships. In those venues, the winners are unmistakably teams, even when a Michael Jordan or a Tom Brady stands out. That's how academic research tends to work, too. Perhaps we should make that clear more often in the awards we give.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Managing and Leading

March 12, 2013 4:30 PM

Two Views on Why Lisp's Syntax Matters

In year or so, I have seen a few people write to debunk the idea that Lisp is special because its code is written in the primary data structure of the language, list. The one I remember best is Dave Herman's Homoiconicity isn't the point, which points out that the critical technical feature that makes Lisp syntax powerful is that it can be read without being parsed.

This morning I read an old Slashdot post in which Lisp guru Kent Pitman gives a more philosophical answer to the question about what makes Lisp's syntax so special:

I like Lisp's willingness to represent itself. People often explain this as its ability to represent itself, but I think that's wrong. Most languages are capable of representing themselves, but they simply don't have the will to.

That's a nice turn of phrase:  : Lisp is willing to represent itself in data, whereas most languages don't have "the will" to do so. It's not about possibility, but facility.

It's easier to manipulate and generate programs from inside a Lisp or Scheme program than any other language that most of us might see on a daily basis. Rubyists manipulate nested arrays of symbols that encode abstract syntax trees, but this style feels somewhat artificial, and besides Ruby's syntax is so large that it's hard for a Ruby program to process other Ruby programs in this way.

As Pitman says, the fact that Lisp programs are represented by lists is almost besides the point. It might well have been arrays of some other data structure. The key is that it is the program's structure being represented, and not the character-level syntax of the programs. This is the same reason that code can be read without being parsed, and that the macro system can be so powerful.

It's also what makes it so easy to provide powerful support for programmers in their text editors and other tools. These tools don't require a lot of machinery or runtime to navigate and manipulate program code. The structure of the code lies close to its surface.

In the end, I like have two ways to think about Lisp's and Scheme's syntactic advantages: the technical reasons that live in the read procedure and the visceral reasons that embody how programmers feel will they work with a syntax that is willing to help the programmer.

Posted by Eugene Wallingford | Permalink | Categories: Computing

March 11, 2013 4:25 PM

Does Readability Give a False Sense of Understandability?

In Good for Whom?, Daniel Lyons writes about the readability of code. He starts with Dan Ingall's classic Design Principles Behind Smalltalk, which places a high value on a system being comprehensible by a single person, and then riffs on readability in J and Smalltalk.

Early on, Lyons made me smile when he noted that, while J is object-oriented, it's not likely to be used that way by many people:

... [because] to use advanced features of J one must first use J, and there isn't a lot of that going on either.

As a former Smalltalker, I know how he feels.

Ultimately, Lyons is skeptical about claims that readability increases the chances that a language will attract a large audience. For one thing, there are too many counterexamples in both directions. Languages like C, which "combines the power of assembly language with the readability of assembly language" [ link ], are often widely used. Languages such as Smalltalk, Self, and Lisp, which put a premium on features such as purity and factorability, which in turn enhance readability, never seem to grow beyond a niche audience.

Lyons's insight is that readability can mislead. He uses as an example the source code of the J compiler, which is written in C but in a style mimicking J itself:

So looking at the J source code, it's easy for me to hold my nose and say, that's totally unreadable garbage; how can that be maintained? But at the same time, it's not my place to maintain it. Imagine if it were written in the most clean, beautiful C code possible. I might be able to dupe myself into thinking I could maintain it, but it would be a lie! Is it so bad that complex projects like J have complex code? If it were a complex Java program instead, I'd still need substantial time to learn it before I would stand a chance at modifying it. Making it J-like means I am required to understand J to change the source code. Wouldn't I have to understand J to change it anyway?

There is no point in misleading readers who have trouble understanding J-like code into thinking they understand the compiler, because they don't. A veneer of readability cannot change that.

I know how Lyons feels. I sometimes felt the same way as I learned Smalltalk by studying the Smalltalk system itself. I understood how things worked locally, within a method and then within a class, but I didn't the full network of classes that made up the system. And I had the scars -- and trashed images -- to prove it. Fortunately, Smalltalk was able to teach me many things, including object-oriented programming, along the way. Eventually I came to understand better, if not perfectly, how Smalltalk worked down its guts, but that took a lot of time and work. Smalltalk's readability made the code accessible to me early, but understanding still took time.

Lyons's article brought to mind another insight about code's understandability that I blogged about many years ago in an entry on comments in code. This insight came from Brian Marick, himself no stranger to Lisp or Smalltalk:

[C]ode can only ever be self-explanatory with respect to an expected reader.

Sometimes, perhaps it's just as well that a language or a program not pretend to be more understandable than it really is. Maybe a barrier to entry is good, by keeping readers out until they are ready to wield the power it affords.

If nothing else, Lyons's stance can be useful as a counterweight to an almost unthinking admiration of readable syntax and programming style.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

February 21, 2013 3:13 PM

Ray Bradbury Channels Alan Kay

... in a Comic-Con 2010 interview:

Don't think about things, just do them.
Don't predict them, just make them.

This goes a bit farther than Kay's "The best way to predict the future is invent it". In particular, I think he is okay with thinking about things.

Text and audio excerpts of the Bradbury interview are available on-line at Brain Pickings.

Posted by Eugene Wallingford | Permalink | Categories: Computing

February 18, 2013 12:59 PM

Code Duplication as a Hint to Think Differently

Last week, one of my Programming Languages students sent me a note saying that his homework solution worked correctly but that he was bothered by some duplicated code.

I was so happy.

Any student who has me for class for very long hears a lot about the dangers of duplication for maintaining code, and also that duplication is often a sign of poor design. Whenever I teach OOP or functional programming, we learn ways to design code that satisfy the DRY principle and ways to eliminate it via refactoring when it does sneak in.

I sent the student an answer, along with hearty congratulations for recognizing the duplication and wanting to eliminate it. My advice

When I sat down to blog the solution, I had a sense of deja vu... Hadn't I written this up before? Indeed I had, a couple of years ago: Increasing Duplication to Eliminate Duplication. Even in the small world of my own teaching, it seems there is nothing new under the sun.

Still, there was a slightly different feel to the way I talked about this in class later that day. The question had come earlier in the semester this time, so the code involved was even simpler. Instead of processing a vector or a nested list of symbols, we were processing with a flat list of symbols. And, instead of applying an arbitrary test to the list items, we were simply counting occurrences of a particular symbol, s.

The duplication occurred in the recursive case, where the procedure handles a pair:

    (if (eq? s (car los))
        (+ 1 (count s (cdr los)))      ; <---
        (count s (cdr los)))           ; <---

Then we make the two sub-cases more parallel:

    (if (eq? s (car los))
        (+ 1 (count s (cdr los)))      ; <---
        (+ 0 (count s (cdr los))))     ; <---

And then use distributivity to push the choice down a level:

    (+ (if (eq? s (car los)) 1 0)
       (count s (cdr los)))            ; <--- just once!

This time, I made a point of showing the students that not only does this solution eliminate the duplication, it more closely follows the command to follow the shape of the data:

When defining a program to process an inductively-defined data type, the structure of the program should follow the structure of the data.

This guideline helps many programmers begin to write recursive programs in a functional style, rather than an imperative style.

Note that in the first code snippet above, the if expression is choosing among two different solutions, depending on whether we see the symbol s in the first part of the pair or not. That's imperative thinking.

But look at the list-of-symbols data type:

    <list-of-symbols> ::= ()
                        | (<symbol> . <list-of-symbols>)

How many occurrences of s are in a pair? Obviously, the number of s's found in the car of the list plus the number of s's found in the cdr of the list. If we design our solution to match the code to the data type, then the addition operation should be at the top to begin:

    (+ ; number of s's found in the car
       ; number of s's found in the cdr )

If we define the answer for the problem in terms of the data type, we never create the duplication-by-if in the first place. We think about solving the subproblems for the car and the cdr, fill in the blanks, and arrive immediately at the refactored code snippet above.

I have been trying to help my students begin to "think functionally" sooner this semester. There is a lot or room for improvement yet in my approach. I'm glad this student asked his question so early in the semester, as it gave me another chance to model "follow the data" thinking. In any case, his thinking was on the right track.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development, Teaching and Learning

February 12, 2013 2:53 PM

Student Wisdom on Monad Tutorials

After class today, a few of us were discussing the market for functional programmers. Talk turned to Clojure and Scala. A student who claims to understand monads said:

To understand monad tutorials, you really have to understand monads first.

Priceless. The topic of today's class was mutual recursion. I think we are missing a base case here.

I don't know whether this is a problem with monads, a problem with the writers of monad tutorials, or a problem with the rest of us. If it is true, then it seems a lot of people are unclear on the purpose of a tutorial.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Teaching and Learning

February 07, 2013 5:01 PM

Quotes of the Day

Computational Thinking Division. From Jon Udell, another lesson that programming and computing teach us which can be useful out in the world:

Focus on understanding why the program is doing what it's doing, rather than why it's not doing what you wanted it to.

This isn't the default approach of everyone. Most of my students have to learn this lesson as a part of learning how to program. But it can be helpful outside of programming, in particular by influencing how we interact with people. As Udell says, it can be helpful to focus on understanding why one's spouse or child or friend is doing what she is doing, rather than on why she isn't doing what you want.

Motivational Division. From the Portland Ballet, of all places, several truths about being a professional dancer that generalize beyond the studio, including:

There's a lot you don't know.
There may not be a tomorrow.
There's a lot you can't control.
You will never feel 100% ready.

So get to work, even if it means reading the book and writing the code for the fourth time. That is where the fun and happiness are. All you can affect, you affect by the work you do.

Mac Chauvinism Division. From Matt Gemmell, this advice on a particular piece of software:

There's even a Windows version, so you can also use it before you've had sufficient success to afford a decent computer.

But with enough work and a little luck, you can afford better next time.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Managing and Leading

January 26, 2013 5:52 PM

Computing Everywhere: Indirection

Alice: The hardest word you'll ever be asked to spell is "ichdericious".

Bob: Yikes. Which word?

A few of us have had fun with the quotations in English and Scheme over the last few days, but this idea is bigger than symbols as data values in programs or even words and strings in natural language. They are examples of a key element of computational thinking, indirection, which occurs in real life all the time.

A few years ago, my city built a new water park. To account for the influx of young children in the area, the city dropped the speed limit in the vicinity of the pool from 35 MPH to 25 MPH. The speed limit in that area has been 35 MPH for a long time, and many drivers had a hard time adjusting to the change. So the city put up a new traffic sign a hundred yards up the road, to warn drivers of the coming change. It looks like this one:

traffic sign: 40 MPH speed limit ahead

The white image in the middle of this sign is a quoted version of what drivers see down the road, the usual:

traffic sign: 40 MPH speed limit

Now, many people slow down to the new speed limit well in advance, often before reaching even the warning sign. Maybe they are being safe. Then again, maybe they are confusing a sign about a speed limit sign with the speed limit sign itself.

If so, they have missed a level of indirection.

I won't claim that computer scientists are great drivers, but I will say that we get used to dealing with indirection as a matter of course. A variable holds a value. A pointer holds the address of a location, which holds a value. A URL refers to a web page. The list goes on.

Indirection is a fundamental element in the fabric of computation. As computation becomes an integral part of nearly everyone's daily life, there is a lot to be gained by more people understanding the idea of indirection and recognizing opportunities to put it to work to mutual benefit.

Over the last few years, Jon Udell has been making a valiant attempt to bring this issue to the attention of computer scientists and non-computer scientists alike. He often starts with the idea of a hyperlink in a web page, or the URL to which it is tied, as a form of computing indirection that everyone already groks. But his goal is to capitalize on this understanding to sneak the communication strategy of pass by reference into people's mental models.

As Udell says, most people use hyperlinks every day but don't use them as well as they might, because the distinction between "pass by value" and "pass by reference" is not a part of their usual mental machinery:

The real problem, I think, is that if you're a newspaper editor, or a city official, or a citizen, pass-by-reference just isn't part of your mental toolkit. We teach the principle of indirection to programmers. But until recently there was no obvious need to teach it to everybody else, so we don't.

He has made the community calendar his working example of pass by reference, and his crusade:

In the case of calendar events, you're passing by value when you send copies of your data to event sites in email, or when you log into an events site and recopy data that you've already written down for yourself and published on your own site.

You're passing by reference when you publish the URL of your calendar feed and invite people and services to subscribe to your feed at that URL.

"Pass by reference rather than by value" is one of Udell's seven ways to think like the web, his take on how to describe computational thinking in a world of distributed, network media. That essay is a good start on an essential module in any course that wants to prepare people to live in a digital world. Without these skills, how can we hope to make the best use of technology when it involves two levels of indirection, as shared citations and marginalia do?

Quotation in Scheme and pass-by-reference are different issue, but they are related in a fundamental way to the concept of indirection. We need to arm more people with this concept than just CS students learning how programming languages work.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

January 25, 2013 4:47 PM

More on Real-World Examples of Quotation

My rumination on real-world examples of quotation to use with my students learning Scheme sparked the imaginations of several readers. Not too surprisingly, they came up with better examples than my own... For example, musician and software developer Chuck Hoffman suggested:

A song, he sang.
"A song", he sang.

The meaning of these is clearly different depending on whether we treat a song as a variable or as a literal.

My favorite example came from long-time friend Joe Bergin:

"Lincoln" has seven letters.
Lincoln has seven letters.

Very nice. Joe beat me with my own example!

As Chuck wrote, song titles create an interesting challenge, whether someone is singing a certain song or singing in a way defined by the words that happen to also be the song's title. I have certainly found it hard to find words both that are part of a title or a reference and that flow seamlessly in a sentence.

This turns out to be a fun form of word play, independent of its use as a teaching example. Feel free to send me your favorites.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

January 24, 2013 4:32 PM

Real-Life Examples of Quotation in Scheme

The new semester is fully underway, and I'm already enjoying Programming Languages. My Tuesday session this week felt like a hodgepodge of topics, including Scheme definitions and conditionals, and didn't inspire my students much. Today's session on pairs and lists seemed to go much more smoothly, at least from my side of the classroom.

One thing that has been different the first two weeks this time around has been several questions about the quote character in Scheme, which is shorthand for the special form quote.

The purpose of the quote is to tell the interpreter to take its argument literally. When the argument is a list, say, '(* 2 3), quotation prevents the interpreter from evaluating the list as a Scheme procedure call. When the argument is a symbol, say, 'a, the quote lets the interpreter know not to treat the a as an identifier, looking up the value bound to that name in the current environment. Instead, it is treated as the literal symbol a. Most of our students have not yet worked in languages where symbols are first-class data values, so this idea takes some getting used to.

In the course of talking about quotation with them, I decided to relate this idea to an example of quotation from real life. The first thing that came to mind at that instant was the distinction between these two sentences:

Lincoln was disappointing.
"Lincoln" was disappointing.

In the former, Lincoln is a name to be evaluated. Depending on the context, it could refer to the 16th president of the United States, the capital of Nebraska, or some other object in the world. (The sentence doesn't have to be true, of course!)

In the latter, quoting Lincoln makes it a title. I intended for this "literal" reference to the word Lincoln to evoke the current feature film of that name.

Almost immediately I began to second-guess my example. The quoted Lincoln is still a name for something -- a film, or a boo, or some such -- and so still needs to be "dereferenced" to retrieve the object signified. It's just that we treat titles differently than other names.

So it's close to what I wanted to convey, but it could mislead students in a dangerous way.

The canonical real-world example of quotation is to quote a word so that we treat the utterance as the word itself. Consider:

Creativity is overused.
"Creativity" is overused.

In the former, creativity is a name to be evaluated. It signifies an abstract concept, a bundle of ideas revolving around creation, originality, art, and ingenuity. We might say creativity is overused in a context where people should be following the rules but are instead blazing their own trails.

In the latter, the quoted creativity signifies the word itself, taken literally. We might say "creativity" is overused to suggest an author improve a piece of writing by choosing a near-synonym such as "cleverness" or "originality", or by rephrasing a sentence so that the abstract concept is recast as the verb in an active statement.

This example stays more faithful to the use of quote in Scheme, where an expression is taken literally, with no evaluation of of any kind needed.

I like giving examples of how programming concepts exist in other parts of our lives and world. Even when they are not perfect matches, they can sometimes help a student's mind click on the idea as it works in a programming language or style.

I like it better when I use better examples!

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

January 18, 2013 2:42 PM

Alive with Infinite Possibilities

the PARC 5-key Chord Keyboard, courtesy the Buxton collection

Engelbart's Violin tells the interesting story of Douglas Engelbart's chorded keyboard, or "chorder", an input device intended as a supplement to the traditional keyboard. Engelbart was part of a generation that saw computing as a universe of unlimited possibilities, and more than many others he showed us glimpses of what it could be.

I grew up in an age when an unadorned BASIC interpreter was standard equipment on any computer, and with so little software available to us, we all wrote programs to make the machine do our bidding. In a narrower way, we felt the sense of unlimited possibilities that drove Engelbart, Sutherland, and the generations that came before us. If only we all had vision as deep.

Unfortunately, not many teenagers get to have that kind of experience anymore. BASIC became VB.Net, a corporate language for a corporate world. The good news is that languages like Python and even JavaScript make programming accessible to more people again, but the ethos of anyone learning to program on his or her own at home seems to have died off.

Engelbart's Violin uses strong language to judge the current state of computing, with some of its strongest lamenting the "cruel discrepancy" between the experience of a creative child learning to program and the world of professional programming:

When you are a teenager, alone with a (programmable) computer, the universe is alive with infinite possibilities. You are a god. Master of all you survey. Then you go to school, major in "Computer Science", graduate -- and off to the salt mines with you, where you will stitch silk purses out of sow's ears in some braindead language, building on the braindead systems created by your predecessors, for the rest of your working life. There will be little room for serious, deep creativity. You will be constrained by the will of your master (whether the proverbial "pointy-haired boss", or lemming-hordes of fickle startup customers) and by the limitations of the many poorly-designed systems you will use once you no longer have an unconstrained choice of task and medium.

Ouch. We who teach CS at the university find ourselves trapped between the needs of a world that employs most of our graduates and the beauty that computing offers. Alas, what Alan Kay said about Engelbart applies more broadly: "Engelbart, for better or for worse, was trying to make a violin.... [M]ost people don't want to learn the violin." I'm heartened to see so many people, including my own colleagues, working so hard to bring the ethos and joy of programming back to children, using Scratch, media computation, and web programming.

This week, I began a journey with thirty or so undergraduate CS students, who over the next four months will learn Scheme and -- I hope -- get a glimpse of the infinite possibilities that extend beyond their first jobs, or even their last. At the very least, I hope I don't shut any more doors on them.


PHOTO. The PARC 5-key Chord Keyboard, from the Buxton collection at Microsoft Research.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

January 10, 2013 3:59 PM

The Pleasure of Elegant Composition in a Programming Language

At the 1978 APL Conference, Alan Perlis gave a talk called Almost Perfect Artifacts Improve only in Small Ways, in which he said:

What attracted me, then, to APL was a feeling that perhaps through APL one might begin to acquire some of the dimensions in programming that we revere in natural language -- some of the pleasures of composition; of saying things elegantly; of being brief, poetic, artistic, that makes our natural languages so precious to us. That aspect of programming was one that I've long been interested in but have never found any lever for coming close to in my experience with languages of the FORTRAN, ALGOL, PL/I school.

I learned APL as as an undergrad and knew immediately that thinking in it was unlike thinking in any other language I had learned, even Lisp. These languages, though, shared a Wow! factor. They enabled programs that did amazing things, or ordinary things in amazing ways. By contrast, BASIC and FORTRAN and PL/I seemed so prosaic.

As an undergrad, I never developed the sort of fluency in that would allow me to, say, write an assembler in 20 lines of APL or Lisp. I did develop a fondness for functional programming that stayed with me into graduate school, where I came into deeper contact with Lisp. I also learned Smalltalk, which I came to admire in a way similar to Perlis's feeling for APL.

I must admit, the beauty and expressiveness of array languages and functional languages have always felt less natural to me than natural language. Their more mathematical orientation felt foreign to me, less like writing in a natural language than solving a puzzle. This wasn't a matter of me not liking math; I took advanced math throughout school and always enjoyed. But it felt different to me. This is, I see now, a personal preference and likely an indicator of why I was drawn more intimately into computer science than into more study of math.

The language I use these days that makes me feel the way Perlis feels about APL is Ruby. It occupies a similar space as Python, which we teach our students and use in several courses. I like Python a lot, more than I like most languages, but it feels plain to me in the sense once explained by John Cook. It is simple, and I get things done when I program in it, but when I use it, I feel like I am programming.

Ruby has this goofy, complex syntax that makes it possible to write some hideous stuff. But Ruby also makes it possible to write code that is brief and elegant, even artistic.

I first saw this at PLoP many years ago, when looking over Gerard Meszaros's shoulder at code he had written to support the writing and publishing of his XUnit Test Patterns book. His code read like the book he was writing. Then I began to see DSLs embedded in Ruby, tools like Rake and Treetop, that made me forget about the language they were implemented in. When you use those tools and others like them, you were writing in a new language, one that fit the thoughts in your head. Yet you were still unmistakably writing Ruby.

Perhaps if I were more an engineer at heart, I would feel differently about simple, sturdy languages that let me get things done. I like them, but they don't make me feel like I am "under the influence", as Perlis writes. They are just attractive tools. Perhaps if I were more a mathematician at heart, I would feel even more at home with the elegance that Haskell and APL give me.

Whatever the reasons, Smalltalk and Ruby grabbed in ways that no other languages have. I think that is due at least in part to the way they connected to my personal understanding and love for natural language. It's interesting how we can all feel this way about different programming languages. I think it says something important about the nature of computing and programming.

Posted by Eugene Wallingford | Permalink | Categories: Computing

December 31, 2012 8:22 AM

Building Things and Breaking Things Down

As I look toward 2013, I've been thinking about Alan Kay's view of CS as science [ link ]:

I believe that the only kind of science computing can be is like the science of bridge building. Somebody has to build the bridges and other people have to tear them down and make better theories, and you have to keep on building bridges.

In 2013, what will I build? What will I break down, understand, and help others to understand better?

One building project I have in mind is an interactive text. One analysis project in mind involves functional design patterns.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Patterns

December 28, 2012 10:03 AM

Translating Code Gibberish to Human-Speak

Following an old link to Ridiculous Fish's Unix shell fish, I recently stumbled upon the delightful cdecl, a service that translates C declarations, however inscrutable, into plain English (and vice versa). As this introductory post says,

Every C declaration will be as an open book to you! Your coworkers' scruffy beards and suspenders will be nigh useless!

The site even provides permalinks so that you can share translations of your thorniest C casts with friends and family.

These pages are more than three years old, so I'm surely telling you something you already know. How did I just find this?

I don't program in C much these days, so cdecl itself is of use to me only as humorous diversion. But it occurs to me that simple tools like this could be useful in a pedagogical setting. Next semester, my students will be learning Scheme and functional programming style. The language doesn't have much syntax, but it does have all those parentheses. Whatever I say or do, they disorient many of my students for a while. Some them will look at even simple code such as

     (let ((x (square 4))
           (y 7))
       (+ x y))

... and feel lost. We spend time in class learning how to read code, and talk about the semantics of such expressions, which helps. But in a pinch, wouldn't it be nice for a student to hit a button and have that code translated into something more immediately comprehensible? Perhaps:

Let x be the square of 4 and y be 7 in the sum of x and y.

This might be a nice learning tool for students as they struggle with a language that seems to them -- at least early on -- to be gibberish on a par with char (*(*(* const x[3])())[5])(int).

Some Scheme masters might well say, "But the syntax and semantics of a let are straightforward. You don't really need this tool." At one level, this is true. Unfortunately, it ignores the cognitive and psychological challenges that most people face when they learn something that is sufficiently unfamiliar to them.

Actually, I think we can use the straightforwardness of the translation as a vehicle to help students learn more than just how a let expression works. I have a deeper motive.

Learning Scheme and functional programming are only a part of the course. Its main purpose is to help students understand programming languages more generally, and how they are processed by interpreters and compilers.

When we look at the let expression above, we can see that translating it into the English expression is not only straightforward, it is 100% mechanical. If it's a mechanical process, then we can write a program to do it for us! Following a BNF description of the expression's syntax, we can write an interpreter that exposes the semantics of the expression.

In many ways, that is the essence of this course.

At this point, this is only a brainstorm, perhaps fueled by holiday cooking and several days away from the office. I don't know yet how much I will do with this in class next term, but there is some promise here.

Of course, we can imagine using a cdecl-like tool to help beginners learn other languages, too. Perhaps there are elements of writing OO code in Java that confuse students enough to make a simple translator useful. Surely public static void main( String[] args) deserves some special treatment! Ruby is complex enough that it might require dozens of little translators to do it justice. Unfortunately, it might take Matz's inside knowledge to write them.

(The idea of translating inscrutable code into language understandable by humans is not limited to computer code, of course. There is a popular movement, to write laws and other legal code in Plain English. This movement is occasionally championed by legislators -- especially in election years. The U.S. Securities and Exchange Commission has its own Plain English Initiative and Plain English Handbook. At seventy-seven pages, the SEC handbook is roughly the same size as R6RS description of Scheme.)

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

December 17, 2012 3:39 PM

The Web is More Than The Latest App or Walled Garden

Anil Dash, on the web we lost:

... today's social networks, they've brought in hundreds of millions of new participants to these networks, and they've certainly made a small number of people rich.

But they haven't shown the web itself the respect and care it deserves, as a medium which has enabled them to succeed. And they've now narrowed the possibilities of the web for an entire generation of users who don't realize how much more innovative and meaningful their experience could be.

I've never warmed to Facebook, for much this reason. I enjoy Twitter, but I treat it as a source of ephemera. Anything that I want to last gets cached in a file of links, shared with colleagues or friends by e-mail, or -- best of all -- blogged about.

I sometimes wonder if blog readers will weary of finding links to things they've already seen via Twitter, or if Twitter has trained too many of us not to want to read someone's comments on such articles in blog entries. But this seems one of the great and lasting values of a blog, one that will remain even after Facebook and Twitter have gone the way of Usenet and GeoCities. The social web is more, and I want to remain a part of it.

Posted by Eugene Wallingford | Permalink | Categories: Computing

December 12, 2012 4:18 PM

Be a Driver, Not a Passenger

Some people say that programming isn't for everyone, just as knowing how to tinker under the hood of one's car isn't for everyone. Some people design and build cars; other people fix them; and the rest of us use them as high-level tools.

Douglas Rushkoff explains why this analogy is wrong:

Programming a computer is not like being the mechanic of an automobile. We're not looking at the difference between a mechanic and a driver, but between a driver and a passenger. If you don't know how to drive the car, you are forever dependent on your driver to take you where you want to go. You're even dependent on that driver to tell you when a place exists.

This is CS Education week, "a highly distributed celebration of the impact of computing and the need for computer science education". As a part of the festivities, Rushkoff was scheduled to address members of Congress and their staffers today about "the value of digital literacy". The passage quoted above is one of ten points he planned to make in his address.

As good as the other nine points are -- and several are very good -- I think the distinction between driver and passenger is the key, the essential idea for folks to understand about computing. If you can't program, you are not a driver; you are a passenger on someone else's trip. They get to decide where you go. You may want to invent a new place entirely, but you don't have the tools of invention. Worse yet, you may not even have the tools you need to imagine the new place. The world is as it is presented to you.

Don't just go along for the ride. Drive.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

December 09, 2012 5:12 PM

Just Build Things

The advantage of knowing how to program is that you can. The danger of knowing how to program is that you will want to.

From Paul Graham's How to Get Startup Ideas:

Knowing how to hack also means that when you have ideas, you'll be able to implement them. That's not absolutely necessary..., but it's an advantage. It's a big advantage, when you're considering an idea ..., if instead of merely thinking, "That's an interesting idea," you can think instead, "That's an interesting idea. I'll try building an initial version tonight."

Writing programs, like any sort of fleshing out of big ideas, is hard work. But what's the alternative? Not being able to program, and then you'll just need a programmer.

If you can program, what should you do?

[D]on't take any extra classes, and just build things. ... But don't feel like you have to build things that will become startups. That's premature optimization. Just build things.

Even the professor in me has to admit this is true. You will learn a lot of valuable theory, tools, and practices in class. But when a big idea comes to mind, you need to build it.

As Graham says, perhaps the best way that universities can help students start startups is to find ways to "leave them alone in the right way".

Of course, programming skills are not all you need. You'll probably need to be able to understand and learn from users:

When you find an unmet need that isn't your own, it may be somewhat blurry at first. The person who needs something may not know exactly what they need. In that case I often recommend that founders act like consultants -- that they do what they'd do if they'd been retained to solve the problems of this one user.

That's when those social science courses can come in handy.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

December 07, 2012 11:17 AM

Agglutination and Crystallization

Alan Kay talks about programming languages quite a bit in this wide-ranging interview. (Aren't all interviews with Kay wide-ranging?) I liked this fuzzy bifurcation of the language world:

... a lot of them are either the agglutination of features or ... a crystallization of style.

My initial reaction was that I'm a crystallization-of-style guy. I have always had a deep fondness for style languages, with Smalltalk at the head of the list and Joy and Scheme not far behind.

But I'm not a purist when it comes to neat and scruffy. As an undergrad, I really liked programming in PL/I. Java never bothered me as much as it bothered some of my purist friends, and I admit unashamedly that I enjoy programming in it.

These days, I like Ruby as much as I like any language. It is a language that lies in the fuzz between Kay's categories. It has an "everything is an object" ethos but, man alive, is it an amalgamation of syntactic and semantic desiderata.

I attribute linguistic split personality to this: I prefer languages with a "real center", but I don't mind imposing a stylistic filter on an agglutinated language. PL/I always felt comfortable because I programmed with a pure structured programming vibe. When I program in Java or Ruby now, somewhere in the center of my mind is a Smalltalk programmer seeing the language through a Smalltalk lens. I have to make a few pragmatic concessions to the realities of my tool, and everything seems to work out fine.

This semester, I have been teaching with Java. Next semester, I will be teaching with Scheme. I guess I can turn off the filter.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 30, 2012 3:49 PM

Passing Out the Final Exam on Day One

I recently ran across an old blog posting called Students learn what they need, not what is assigned, in which a Ted Dunning described a different sort of "flipped course" than is usually meant: He gave his students the final exam on Day 1, passed out the raw materials for the course, and told them to "get to work". They decided what they needed to learn, and when, and asked for instruction and guidance on their own schedule.

Dunning was happy with the results and concluded that...

... these students could learn vastly more than was expected of them if they just wanted to.

When students like what they are doing, they can surprise most everyone with what they will do to learn. Doing something cool like building a robot (as Dunning's students did) can be all the motivation some students need.

I'm sometimes surprised by just what catches my students' fancy. A few weeks ago, I asked my sophomore- and junior-level OOP class to build the infrastructure for a Twitter-like app. It engaged them like only graphical apps usually do. They've really dug into the specs to figure out what they mean. Many of them don't use Twitter, which has been good, because it frees them of too many preconceived limitations on where they can take their program.

They are asking good questions, too, about design: Should this object talk to that one? The way I divided up the task led to code that feels fragile; is there a better way? It's so nice not to still be answering Java questions. I suspect that some are still encountering problems at the language level, but they are solving them on their own and spending more time thinking about the program at a higher level.

I made this a multi-part project. They submitted Iteration 1 last weekend, will submit Iteration 2 tomorrow, and will work on Iteration 3 next week. That's a crucial element, I think, in getting students to begin taking their designs more seriously. It matters how hard it easy to change the code, because they have to change it now -- and tomorrow!

The point of Dunning's blog is that students have to discover the need to know something before they are really interesting in learning it. This is especially true if the learning process is difficult or tedious. You can apply this idea to a lot of software development, and even more broadly to CS.

I'm not sure when I'll try the give-the-final-exam-first strategy. My compiler course already sort of works that way, since we assign the term project upfront and then go about learning what we need to build the compiler. But I don't make my students request lectures; I still lay the course out in advance and take only occasional detours.

I think I will go at least that far next semester in my programming languages course, too: show them a language on day one and explain that our goal for the semester is to build an interpreter for it by the end of the semester, along with a few variations that explore the range of possibilities that programming languages offer. That may create a different focus in my mind as I go through the semester. I'm curious to see.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 28, 2012 6:34 PM

Converting Lecture Notes into an Active Website

... in which the author seeks pointers to interactive Scheme materials on-line.

Last summer, I fiddled around a bit with Scribble, a program for writing documentation in (and for) Racket. I considered using it to write the lecture notes and website for my fall OOP course, but for a variety of reasons set it aside.

the icon for Slideshow

In the spring I'll be teaching Programming Languages again, and using Racket with my students. This seems like the perfect time to dive in and use Scribble and Slideshow to create all my course materials. This will create a synergy between what I do in class and how I prep, which will be good for me. Using Racket tools will also set a good example for my students.

After seeing The Racket Way, Matthew Flatt's talk at StrangeLoop, I am inspired to do more than simply use Racket tools to create text and slides and web pages. I'd like to re-immerse myself in a world where everything is a program, or nearly so. This would set an even more important example for my students, and perhaps help them to see more clearly that they don't ever to settle for the programs, the tools, or the languages that people give them. That is the Computer Science way as well as the Racket way.

I've also been inspired recently by the idea of an interactive textbook a lá Miller and Ranum. I have a pretty good set of lecture notes for Programming Languages, but the class website should be more than a 21st-century rendition of a 19th-century presentation. I think that using Scribble and Slideshow are a step in the right direction.

So, a request: I am looking for examples of people using the Racket presentation tools to create web pages that have embedded Scheme REPLs, perhaps even a code stepper of the sort Miller and Ranum use for Python. Any pointers you might have are welcome.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 18, 2012 9:13 AM

Programming Languages Quote of the Day

... comes from Gilad Bracha:

I firmly believe that a time traveling debugger is worth more than a boatload of language features[.]

This passage comes as part of a discussion of what it would take to make Bret Victor's vision of programming a reality. Victor demonstrates powerful ideas using "hand crafted illustrations of how such a tool might behave". Bracha, whose work on Smalltalk and Newspeak have long inspired me -- reflects on what it would take to offer Victor's powerful ideas in a general purpose programming environment.

Smalltalk as a language and environment works at a level where we conceive of providing the support Victor and Bracha envision, but most of the language tools people use today are too far removed from the dynamic behavior of the programs being written. The debugger is the most notable example.

Bracha suggests that we free the debugger from the constraints of time and make it a tool for guiding the evolution of the program. He acknowledges that he is not the first person to propose such an idea, pointing specifically to Bill Lewis's proposal for an omniscient debugger. What remains is the hard work needed to take the idea farther and provide programmers more transparent support for understanding dynamic behavior while still writing the code.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 15, 2012 4:04 PM

Teaching Students to Read and Study in a New Way

Mark Guzdial's How students use an electronic book, reports on the research paper "Performance and Use Evaluation of an Electronic Book for Introductory Python Programming" [ pdf ]. In this paper, Alvarado et al. evaluate how students used the interactive textbook How to Think Like a Computer Scientist by Ranum and Miller in an intro CS course. The textbook integrates traditional text with embedded video, "active" examples using an embedded Python interpreter, and empirical examples using a code stepper a lá a debugger.

The researchers were surprised to find how little some students used the book's interactive features:

One possible explanation for the less-than-anticipated use of the unique features may be student study skills. The survey results tend to suggest that students "study" by "reading". Few students mention coding or tracing programs as a way of "studying" computer science.

I am not using an interactive textbook in my course this semester, but I have encountered the implicit connection in many students' minds between studying and reading. It caught me off-guard, too.

After lengthy searching and some thought, I decided to teach my sophomore-level OOP course without a required text. I gave students links to two on-line books they could use as Python references, but neither covers the programming principles and techniques that are at the heart of the course. In lieu of a traditional text, I have been giving my students notes for each session, written up carefully in a style that resembles a textbook, and source code -- lots and lots of source code.

Realizing that this would be an unusual way for students to study for a CS class, at least compared to their first-year courses, I have been pretty consistent in encouraging them to work this way. Daily I suggest that they unpack the code, read it, compile it, and tinker with it. The session notes often include little exercises they can do to test or extend their understanding of a topic we have covered in class. In later sessions, I often refer back to an example or use it as the basis for something new.

I figured that, without a textbook to bog them down, they would use my session notes as a map and spend most of their time in the code spelunking, learning to read and write code, and seeing the ideas we encounter in class alive in the code.

a snapshot of Pousse cells in two dimensions

Like the results reported in the Alvarado paper, my experiences have been mixed, and in many ways not what I expected. Some students read very little, and many of those who do read the lecture notes spend relatively little time playing with the code. They will spend plenty of time on our homework assignments, but little or no time on code for the purposes of studying. My data is anecdotal, based on conversations with the subset of students who visit office hours and e-mail exchanges with students who ask questions late at night. But performance on the midterm exam and some of the programming assignments are consistent with my inference.

OO programs are the literature of this course. Textbooks are like commentaries and (really long) Cliff Notes. If indeed the goal is to get students to read and write code, how should we proceed? I have been imagining an even more extreme approach:

  • no textbook, only a language reference
  • no detailed lecture notes, only cursory summaries of what we did in class
  • code as a reading assignment before each session
  • every day in class, students do tasks related to the assigned reading -- engaging, fun tasks, but tasks they can't or wouldn't want to do without having studied the assigned code

A decade or so ago, I taught a course that mixed topics in user interfaces and professional ethics using a similar approach. It didn't provide magic results, but I did notice that once students got used to the unusual rhythm of the course they generally bought in to the approach. The new element here is the emphasis on code as the primary literature to read and study.

Teaching a course in a way that subverts student expectations and experience creates a new pedagogical need: teaching new study skills and helping students develop new work habits. Alvarado et al. recognize that this applies to using a radically different sort of textbook, too:

Might students have learned more if we encouraged them to use codelens more? We may need to teach students new study skills to take advantage of new learning resources and opportunities.


Another interesting step would be to add some meta-instruction. Can we teach students new study skills, to take advantage of the unique resources of the book? New media may demand a change in how students use the media.

I think those of us who teach at the university level underestimate how important meta-level instruction of this sort is to most of students. We tend to assume that students will figure it out on their own. That's a dangerous assumption to make, at least for a discipline that tends to lose too many good students on the way to graduation.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 03, 2012 11:17 AM

When "What" Questions Presuppose "How"

John Cook wrote about times in mathematics when maybe you don't need to do what you were asked to do. As one example, he used remainder from division. In many cases, you don't need to do division, because you can find the answer using a different, often simpler, method.

We see a variation of John's theme in programming, too. Sometimes, a client will ask for a result in a way that presupposes the method that will be used to produce it. For example, "Use a stack to evaluate these nested expressions." We professors do this to students a lot, because they want the students to learn the particular technique specified. But you see subtle versions of this kind of request more often than you might expect outside the classroom.

An important part of learning to design software is learning to tease apart the subtle conflation of interface and implementation in the code we write. Students who learn OO programming after a traditional data structures course usually "get" the idea of data abstraction, yet still approach large problems in ways that let implementations leak out of their abstractions in the form of method names and return values. Kent Beck talked about how this problem afflicts even experienced programmers in his blog entry Naming From the Outside In.

Primitive Obsession is another symptom of conflating what we need with how we produce it. For beginners, it's natural to use base types to implement almost any behavior. Hey, the extreme programming principle You Ain't Gonna Need It encourages even us more experienced developers not to create abstractions too soon, until we know we need them and in what form. The convenience offered by hashes, featured so prominently in the scripting languages that many of us use these days, makes it easy to program for a long time without having to code a collection of any sort.

But learning to model domain objects as objects -- interfaces that do not presuppose implementation -- is one of the powerful stepping stones on the way to writing supple code, extendible and adaptable in the face of reasonable changes in the spec.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

October 24, 2012 11:38 AM

"Don't Break the Phone; Fix the Computer"

Rob Pike in The Set-Up:

Twenty years ago, you expected a phone to be provided everywhere you went, and that phone worked the same everywhere. At a friend's house, or a restaurant, or a hotel, or a pay phone, you could pick up the receiver and make a call. You didn't carry a phone around with you; phones were part of the infrastructure. Computers, well, that was a different story. As laptops came in, people started carrying computers around with them everywhere. The reason was to have the state stored on the computer, not the computer itself. You carry around a computer so you can access its disk.

In summary, it used to be that phones worked without you having to carry them around, but computers only worked if you did carry one around with you. The solution to this inconsistency was to break the way phones worked rather than fix the way computers work.

Ah, the memories of grad school, WYSE terminals, and VT-100 emulation.

The advent of ubiquitous networking is making it possible for us to return to the days of dumb terminals. Is that where we want to live?

Pike's vision notwithstanding: I still carry a computer, both for state and processor. I access networked computers frequently. I do not yet carry a phone. I remain happy.

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 19, 2012 3:08 PM

Computer Programming, Education Reform, and Changing Our Schools

Seymour Papert

You almost can't go wrong by revisiting Seymour Papert's work every so often. This morning I read Why School Reform Is Impossible, which reminds us that reform and change are different things. When people try to "reform" education by injecting a new idea from outside, schools seem to assimilate the reform into its own structure, which from the perspective of the reformer blunts or rejects the intended reform. Yet schools and our education system do change over time, evolving as the students, culture, and other environmental factors change.

As people such as Papert and Alan Kay have long argued, a big part of the problem in school reform involving computers is that we misunderstand what a computer is:

If you ask, "Which is not like the other two?" in the list "educational movie, textbook, computer", it is pretty obvious from my perspective that the answer must be "computer."

... not "textbook", which is how most people answer, including many people who want to introduce more computers into the classroom. Textbooks and movies are devices for receiving content that someone else made. Computers are for creating content. It just so happens that we can use them to communicate ideas in new ways, too.

This misunderstanding leads people to push computers for the wrong reasons, or at least for reasons that miss their game-changing power. We sometimes here that "programming is the new Latin". Papert reminds us that the reasons we used to teach Latin in schools changed over time:

In recent times, Latin was taught in schools because it was supposed to be good for the development of general cognitive skills. Further back, it was taught because it was the language in which all scholarly knowledge was expressed, and I have suggested that computational language could come to play a similar role in relation to quite extensive areas of knowledge.

If programming is the new Latin, it's not Latin class, circa 1960, in which Latin taught us to be rigorous students. It's Latin class, circa 1860 or 1760 or 1560, in which Latin was the language of scholarly activity. As we watch computing become a central part of the language of science, communication, and even the arts and humanities, we will realize that students need to learn to read and write code because -- without that skill -- they are left out of the future.

No child left behind, indeed.

In this essay, Paper gives a short version of his discussion in Mindstorms of why we teach the quadratic equation of the parabola to every school child. He argues that its inclusion in the curriculum has more to do with its suitability to the medium of the say -- pencil and paper -- than to intrinsic importance. I'm not too sure that's true; knowing how parabolas and ellipses work is pretty important for understanding the physical world. But it is certainly true that how and when we introduce parabolas to students can change when we have a computer and a programming language at hand.

Even at the university we encounter this collision of old and new. Every student here must take a course in "quantitative reasoning" before graduating. For years, that was considered to be "a math course" by students and advisors alike. A few years ago, the CS department introduced a new course into the area, in which students can explores a lot of the same quantitative issues using computation rather than pencil and paper. With software tools for modeling and simulation, many students can approach and even begin to solve complex problems much more quickly than they could working by hand. And it's a lot more fun, too.

To make this work, of course, students have to learn a new programming language and practice using it in meaningful ways. Papert likens it to learning a natural language like French. You need to speak it and read it. He says we would need the programming analog of "the analog of a diverse collection of books written in French and access to French-speaking people".

the Scratch logo cat

The Scratch community is taking at shot at this. The Scratch website offers not only a way to download the Scratch environment and a way to view tutorials on creating with Scratch. It also offers -- front and center, the entire page, really -- links to shared projects and galleries. This gives students a chance first to be inspired by other kids and then to download and read the actual Scratch programs that enticed them. It's a great model.

The key is to help everyone see that computers are not like textbooks and televisions and movie projectors. As Mitch Resnick has said:

Computers for most people are black boxes. I believe kids should understand objects are "smart" not because they're just smart, but because someone programmed them to be smart.

What's most important ... is that young children start to develop a relationship with the computer where they feel they're in control. We don't want kids to see the computer as something where they just browse and click. We want them to see digital technologies as something they can use to express themselves.

Don't just play with other people's products. Make your own.

Changes in the world's use of computing may do more to cause schools to evolve in a new direction than anyone's educational reforms ever could. Teaching children that they can be creators and not simply consumers is a subversive first step.


IMAGE 1: Seymour Papert at the OLPC offices in Cambridge, Massachusetts, in 2006. Source: Wikimedia Commons License: Creative Commons Attribution-Share Alike 2.0.

IMAGE 2: The Scratch logo. Source: Wikimedia Commons License: Creative Commons Attribution-Share Alike 2.0.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

October 12, 2012 10:37 AM

Make Let, Not Var

I don't blog images for their own sake often, but this mash-up makes me happy:

a parody of Lennon and Ono's 'Make Love, Not War' image

Even as I enjoy teaching OO programming this semester, this reminds me that I'll enjoy teaching functional programming in the spring.

This came to me via a tweet. If you know the source, I'd love to hear from you.

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 07, 2012 2:50 PM

Equality Check Patterns for Recursive Structures

I first encountered this trio of programming patterns when writing Smalltalk programs to manipulate graphs back in the late 1980s. These days I see them most often when comparing recursive data types in language processors. Andrew Black wrote about these patterns in the context of Smalltalk on the Squeak mailing list in the mid-2000s.


Recursive Equality Check


You are working with a recursive data structure. In the simplest form, you might have a list or a record that can contain lists or records as parts. In a language application, you might have a structured type that can have the same type as one of its parts. For instance, a function type might allow a function as an argument or a function as a return value.

You have two instances of the structure and need to compare them, say, for equality or for dominance. In the language app, for example, you will need to verify that the type of an argument matches the type of the formal parameter on a procedure.


Standard structural recursion works. Walk the two structures in parallel, checking to see that they have the same values in the same positions. When one of the positions holds values the same structure, make a recursive call to compare them.

But what if ...


Recursive Equality Check with Identity Check


An instance of the recursive structure can contain a reference to itself as a value, either directly or through mutual recursion. In the simplest form, this might be a dictionary that contains itself as a value in a key/vaue pair, as in Smalltalk, where the global variable Smalltalk is the dictionary of all global variables, including Smalltalk.

Comparing two instances now raises concerns. Two instances may be identical, or contain identical components. In such cases, the standard recursive comparison will never terminate.


Check for identity first. Recurse only if the two values are distinct.

But what if...


Recursive Equality Check with Cache


You have two structures that do not share any elements, but they are structurally isomorphic. For example, this can occur in the simplest of structures, two one-element maps:

    a = { :self => a }
    b = { :self => b }

Now, even with an identity test up front, the recursive comparison will never terminate.


Maintain a cache of compared pairs. Before you begin to compare two objects, check to see if the pair is in the cache. If yes, return true. Otherwise, add the pair to the cache and proceed.

This approach works even though the function has not finished comparing the two objects yet. If there turns out to be a difference between the two, the check currently in progress will find it elsewhere and answer false. There is no need to enter a recursive check.


A variation of this caching technique can also be used in other situations, such as computing a hash value for a recursive structure. If in the course of computing the hash you encounter the same structure again, assume that the value is a suitable constant, such as 0 or 1. Hashes are only approximations anyway, so making only one pass over the structure is usually enough. If you really need a fixpoint, then you can't take this shortcut.

Ruby hashes handle all three of these problems correctly:

    a = { :first  => :int,
          :second => { :first  => :int, :second => :int } }
    b = { :second => { :second => :int, :first  => :int },
          :first  => :int, }

a == b # --> true

c = { :first => c } a = { :first => :int, :second => { :first => c, :second => :int } } b = { :first => :int, :second => { :first => c, :second => :int } }

a == b # --> true

a = { :self => a } b = { :self => b }

a == b # --> true

I don't know if either MRI Ruby or JRuby uses these patterns to implement their solutions, or if they use some other technique.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

September 30, 2012 12:45 PM

StrangeLoop 8: Reactions to Brett Victor's Visible Programming

The last talk I attended at StrangeLoop 2012 was Bret Victor's Visible Programming. He has since posted an extended version of his presentation, as a multimedia essay titled Learnable Programming. You really should read his essay and play the video in which he demonstrates the implementation of his ideas. It is quite impressive, and worthy of the discussion his ideas have engendered over the last few months.

In this entry, I give only a high-level summary of the idea, react to only one of his claims, and discuss only one of his design principles in ay detail. This entry grew much longer than I originally intended. If you would like to skip most of my reaction, jump to the mini-essay that is the heart of this entry, Programing By Reacting, in the REPL.


Programmers often discuss their productivity as at least a partial result of the programming environments they use. Victor thinks this is dangerously wrong. It implies, he says, that the difficulty with programming is that we aren't doing it fast enough.

But speed is not the problem. The problem is that our programming environments don't help us to think. We do all of our programming in our minds, then we dump our ideas into code via the editor.

Our environments should do more. They should be our external imagination. They should help us see how our programs work as we are writing them.

This is an attractive guiding principle for designing tools to help programmers. Victor elaborates this principle into a set of five design principles for an environment:

  • read the vocabulary -- what do these words mean?
  • follow the flow -- what happens when?
  • see the state -- what is the computer thinking?
  • create by reacting -- start somewhere, then sculpt
  • create by abstracting -- start concrete, then generalize

Victor's talk then discussed each design principle in detail and showed how one might implement the idea using JavaScript and Processing.js in a web browser. The demo was cool enough that the StrangeLoop crowd broke into applause at leas twice during the talk. Read the essay.


As I watched the talk, I found myself reacting in a way I had not expected. So many people have spoken so highly of this work. The crowd was applauding! Why was I not as enamored? I was impressed, for sure, and I was thinking about ways to use these ideas to improve my teaching. But I wasn't falling head over heels in love.

A Strong Claim

First, I was taken aback by a particular claim that Victor made at the beginning of his talk as one of the justifications for this work:

If a programmer cannot see what a program is doing, she can't understand it.

Unless he means this metaphorically, seeing "in the mind's eye", then it is simply wrong. We do understand things we don't see in physical form. We learn many things without seeing them in physical form. During my doctoral study, I took several courses in philosophy, and only rarely did we have recourse to images of the ideas we were studying. We held ideas in our head, expressed in words, and manipulated them there.

We did externalize ideas, both as a way to learn them and think about them. But we tended to use stories, not pictures. By speaking an idea, or writing it down, and sharing it with others, we could work with them.

So, my discomfort with one of Victor's axioms accounted for some of my unexpected reaction. Professional programmers can and do manipulate ideas abstractly. Visualization can help, but when is it necessary, or even most helpful?

Learning, Versus Doing

This leads to a second element of my concern. I think I had a misconception about Victor's work. His talk and its title, "Visible Programming", led me to think his ideas are aimed primarily at working programmers, that we need to make programs visible for all programmers.

The title of his essay, "Learnable Programming", puts his claims into a different context. We need to make programs visible for people who are learning to program. This seems a much more reasonable position on its face. It also lets me see the axiom that bothered me so much in a more sympathetic light: If a novice programmer cannot see what a program is doing, then she may not be able to understand it.

Seeing how a program works is a big part of learning to program. A few years ago, I wrote about "biction" and the power of drawing a picture of what code does. I often find that if I require a student to draw a picture of what his code is doing before he can ask me for debugging help, he will answer his own question before getting to me.

The first time a student experiences this can be a powerful experience. Many students begin to think of programming in a different way when they realize the power of thinking about their programs using tools other than code. Visible programming environments can play a role in helping students think about their programs, outside their code and outside their heads.

I am left puzzling over two thoughts:

  • How much of the value my students see in pictures comes from not from seeing the program work but from drawing the picture themselves -- the act of reflecting about the program? If our tools visualizes the code for them, will we see the same learning effect that we see in drawing their own pictures?

  • Certainly Victor's visible programming tools can help learners. How much will they help programmers once they become experts? Ben Shneiderman's Designing the User Interface taught me that novices and experts have different needs, and that it's often difficult to know what works well for experts until we run experiments.

Mark Guzdial has written a more detailed analysis of Victor's essay from the perspective of a computer science educator. As always, Mark's ideas are worth reading.

Programming By Reacting, in the REPL

My favorite parts of this talk were the sections on creating by reacting and abstracting. Programmers, Victor says, don't work like other creators. Painters don't stare at a blank canvas, think hard, create a painting in their minds, and then start painting the picture they know they want to create. Sculptors don't stare at a block of stone, envision in their mind's eye the statue they intend to make, and then reproduce that vision in stone. They start creating, and react, both to the work of art they are creating and to the materials they are using.

Programmers, Victor says, should be able to do the same thing -- if only our programming environments helped us.

As a teacher, I think this is an area ripe for improvement in how we help students learn to program. Students open up their text editor or IDE, stare at that blank screen, and are terrified. What do I do now? A lot of my work over the last fifteen to twenty years has been in trying to find ways to help students get started, to help them to overcome the fear of the blank screen.

My approaches haven't been through visualization, but through other ways to think about programs and how we grow them. Elementary patterns can give students tools for thinking about problems and growing their code at a scale larger than characters or language keywords. An agile approach can help them start small, add one feature at a time, proceed in confidence with working tests, and refactor to make their code better as they go along. Adding Victor-style environment support for the code students write in CS1 and CS2 would surely help as well.

However, as I listened to Victor describe support for creating by reacting, and then abstracting variables and functions out of concrete examples, I realized something. Programmers don't typically write code in an environment with data visualizations of the sort Victor proposes, but we do program in the style that such visualizations enable.

We do it in the REPL!

A simple, interactive computer programming environment enables programmers to create by reacting.

  • They write short snippets of code that describe how a new feature will work.
  • They test the code immediately, seeing concrete results from concrete examples.
  • They react to the results, shaping their code in response to what the code and its output tell them.
  • They then abstract working behaviors into functions that can be used to implement another level of functionality.

Programmers from the Lisp and Smalltalk communities, and from the rest of the dynamic programming world, will recognize this style of programming. It's what we do, a form of creating by reacting, from concrete examples in the interaction pane to code in the definitions pane.

In the agile software development world, test-first development encourages a similar style of programming, from concrete examples in the test case to minimal code in the application class. Test-driven design stimulates an even more consciously reactive style of programming, in which the programmer reacts both to the evolving program and to the programmer's evolving understanding of it.

The result is something similar to Victor's goal for programmers as they create abstractions:

The learner always gets the experience of interactively controlling the lower-level details, understanding them, developing trust in them, before handing off that control to an abstraction and moving to a higher level of control.

It seems that Victor would like to perform even more support for novices than these tools can provide, down to visualizing what the program does as they type each line of code. IDEs with autocomplete is perhaps the closest analog in our current arsenal. Perhaps we can do more, not only for novices but also professionals.


I love the idea that our environments could do more for us, to be our external imaginations.

Like many programmers, though, as I watched this talk, I occasionally wondered, "Sure, this works great if you creating art in Processing. What about when I'm writing a compiler? What should my editor do then?"

Victor anticipated this question and pre-emptively answered it. Rather than asking, How does this scale to what I do?, we should turn the question inside out and ask, These are the design requirements for a good environment. How do we change programming to fit?

I doubt such a dogmatic turn will convince skeptics with serious doubts about this approach.

I do think, though, that we can reformulate the original question in a way that focuses on helping "real" programmers. What does a non-graphical programmer need in an external imagination? What kind of feedback -- frequent, even in-the-moment -- would be most helpful to, say, a compiler writer? How could our REPLs provide even more support for creating, reacting, and abstracting?

These questions are worth asking, whatever one thinks of Victor's particular proposal. Programmers should be grateful for his causing us to ask them.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development, Teaching and Learning

September 29, 2012 4:04 PM

StrangeLoop 7: The Racket Way

I have been using Racket since before it was Racket, back when it was "just another implementation of Scheme". Even then, though, it wasn't just another implementation of Scheme, because it had such great libraries, a devoted educational community around it, and an increasingly powerful facility for creating and packaging languages. I've never been a deep user of Racket, though, so I was eager to see this talk by one of its creators and learn from him.

Depending on your perspective, Racket is either a programming language (that looks a lot like Scheme), a language plus a set of libraries, or a platform for creating programs. This talk set out to show us that Racket is more.

Flatt opened with a cute animated fairy tale, about three princesses who come upon a wishing well. The first asks for stuff. The second asks for more wishes. The third asks for a kingdom full of wishing wells. Smart girl, that third one. Why settle for stuff when you can have the source of all stuff?

This is, Flatt said, something like computer science. There is a similar progression of power from:

  • a document, to
  • a language for documents, to
  • a language for languages.

Computer scientists wish for a way to write programs that do... whatever.

This is the Racket way:

  1. Everything is a program.
  2. Concepts are programming language constructs.
  3. Programming languages are extensible and composable.

The rest of the talk was a series of impressive mini-demos that illustrated each part of the Racket way.

To show what it means to say that everything is a program, Flatt demoed Scribble, a language for producing documents -- even the one he was using to give his talk. Scribble allows writers to abstract over every action.

To show what it means to say that concepts are programming language constructs, Flatt talked about the implementation of Dr. Racket, the flexible IDE that comes with the system. Dr. Racket needs to be able to create, control, and terminate processes. Relying on the OS to do this for it means deferring to what that OS offers. In the end, that means no control.

Dr. Racket needs to control everything, so the language provides constructs for these concepts. Flatt showed as examples threads and custodians. He then showed this idea at work in an incisive way: he wrote a mini-Dr. Racket, called Racket, Esq. -- live using Racket. To illustrate its completeness, he then ran his talk inside racket-esq. Talk about a strange loop. Very nice.

To show what it means to say that programming languages are extensible and composable, Flatt showed a graph of the full panoply of Racket's built-in languages and demoed several languages. He then used some of the basic language-building tools in Racket -- #lang, require, define-syntax, syntax-rules, and define-syntax-rule -- to build the old text-based game Adventure, which needs a natural language-like scripting language for defining worlds. Again, very nice -- so much power in so many tools.

This kind of power comes from taking seriously a particular way of thinking about the world. It starts with "Everything is a program." That is the Racket way.

Flatt is a relaxed and confident presenter. As a result, this was a deceptively impressive talk. It reinforced its own message by the medium in which it was delivered: using documents -- programs -- written and processed in Racket. I am not sure how anyone could see a slideshow with "hot" code, a console for output, and a REPL within reach, all written in the environment being demoed, and not be moved to rethink how they write programs. And everything else they create.

As Flatt intimated briefly early on, The Racket Way of thinking is not -- or should not be -- limited to Racket. It is, at its core, the essence of of computer science. The duality of code and data makes what we do so much more powerful than most people realize, and makes what we can do so much more powerful than most us actually do with the tools we accept. I hope that Flatt's talk inspires a few more of us not to settle for less than we have to.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 29, 2012 3:40 PM

StrangeLoop 6: Y Y

I don't know if it was coincidence or by design of the conference organizers, but Wednesday morning was a topical repeat of Tuesday morning for me: two highly engaging talks on functional programming. I had originally intended to write them up in a single entry, but that write-up grew so long that I decided to give them their own entries.

Y Not?

Watching talks and reading papers about the Y combinator are something of a spectator code kata for me. I love to see new treatments, and enjoy seeing even standard treatments every now and then. Jim Weirich presented it at StrangeLoop with a twist I hadn't seen before.

Weirich opened, as speakers often do, with him. This is a motivational talk, so it should be...

  • non-technical. But it's not. It is highly technical.
  • relevant. But it's not. It is extremely pointless.
  • good code. But it's not. It shows the worst Clojure code ever.

But it will be, he promises, fun!

Before diving in, he had one more joke, or at least the first half of one. He asked for audience participation, then asked his volunteer to calculate cos(n) for some value of n I missed. Then he asked the person to keep hitting the cosine button repeatedly until he told him to stop.

At the dawn of computing, to different approaches were taken in an effort to answer the question, What is effectively computable?

Alan Turing devised what we now call a universal Turing machine to embody the idea. Weirich showed a video demonstration of a physical Turing machine to give his audience a sense of what a TM is like.

(If you'd like to read more about Turing and the implication of his universal machine, check out this reflection I wrote earlier this year after a visit by Doug Hofstadter to my campus. Let's just say that the universal TM means more than just an answer to what functions are effectively computable.)

A bit ahead of Turing, Alonzo Church devised an answer to the same question in the form of the lambda calculus, a formal logical system. As with the universal TM, the lambda calculus can be used to compute everything, for a particular value of eveything. These days, nearly every programming language has lambdas of some form

... now came the second half of the joke running in the background. Weirich asked his audience collaborator what was in his calculator's display. The assistant called out some number, 0.7... Then Weirich showed his next slide -- the same number, taken out many more digits. How was he able to do this? There is a number n such that cos(n) = n. By repeatedly pressing his cosine button, Weirich's assistant eventually reached it. That number n is called the fixed point of the cosine function. Other functions have fixed points to, and they can be a source of great fun.

Then Weirich opened up his letter and wrote some code from the ground up to teach some important concepts of functional programming, using the innocuous function 3(n+1). With this short demo, Weirich demonstrated the idea of a higher-order function, including function factories, a set of useful functional refactorings that included

  • Introduce Binding
    -- where the new binding is unused in the body
  • Inline Definition
    -- where a call to a function is replaced by the function body, suitably parameterized
  • Wrap Function
    -- where an expression is replaced by a function call that computes the expression
  • Tennent Correspondence Principle
    -- where an expression is turned into a think

At the end of his exercise, Weirich had created a big function call that contained no named function definitions yet computed the same answer.

He asks the crowd for applause, then demurs. This is 80-year-old technology. Now you know, he says, what a "chief scientist" at New Context does. (Looks a lot like what an academic might do...)

Weirich began a second coding exercise, the point behind all his exposition to this point: He wrote the factorial function, and began to factor and refactor it just as he had the simpler 3(n+1). But now inlining the function breaks the code! There is a recursive call, and the name is now out of scope. What to do?

He refactors, and refactors some more, until the body of factorial is an argument to a big melange of lambdas and applications of lambdas. The result is a function that computes the fixed point of any function passed it.

That is Y. The Y combinator.

Weirich talked a bit about Y and related ideas, and why it matters. He closed with a quote from Wittgenstein, from Philosophical Investigations:

The aspects of things that are most important for us are hidden because of their simplicity and familiarity. (One is unable to notice something -- because it is always before one's eyes.) The real foundations of his enquiry do not strike a man at all. Unless that fact has at some time struck him. -- And this means: we fail to be struck by what, once seen, is most striking and most powerful.

The thing that sets Weirich's presentation of Y apart from the many others I've seen is its explicit use of refactoring to derive Y. He created Y from a sequence of working pieces of code, each the result of a refactoring we can all understand. I love to do this sort of thing when teaching programming ideas, and I was pleased to see it used to such good effect on such a challenging idea.

The title of this talk -- Y Not? -- plays on Y's interrogative homonym. Another classic in this genre echos the homonym in its title, then goes on to explain Y in four pages of English and Scheme. I suggest that you study @rpg's essay while waiting for Weirich's talk to hit InfoQ. Then watch Weirich's talk. You'll like it.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

September 28, 2012 3:59 PM

StrangeLoop 5: Miscellany -- At All Levels

Most of the Tuesday afternoon talks engaged me less deeply than the ones that came before. Part of that was the content, part was the style of delivery, and part was surely that my brain was swimming in so many percolating ideas that there wasn't room for much more.

Lazy Guesses

Oleg Kiselyov, a co-author of the work behind yesterday's talk on miniKanren, gave a talk on how to implement guessing in computer code. That may sound silly, for a couple of reasons. But it's not.

First, why would we want to guess at all? Don't we want to follow principles that guarantee we find the right answer? Certainly, but those principles aren't always available, and even when they are the algorithms that implement them may be computationally intractable. So we choose to implement solutions that restrict the search space, for which we pay a price along some other dimension, often expressiveness.

Kiselyov mentioned scheduling tasks early in his talk, and any student of AI can list many other problems for which "generate and test" is a surprisingly viable strategy. Later in the talk, he mentioned parsing, which is also a useful example. Most interesting grammars have nondeterministic choices in them. Rather than allow our parsers to make choices and fail, we usually adopt rules that make the process predictable. The result is an efficient parser, but a loss in what we can reasonably say in the language.

So, perhaps the ability to make good guesses is valuable. What is so hard about implementing them? The real problem is that there are so many bad guesses. We'd like to use knowledge to guide the process of guessing again, to favor some guesses over others.

The abstract for the talk promises a general principle on which to build guessing systems. I must admit that I did not see it. Kiselyov moved fast at times through his code, and I lost sight of the big picture. I did see discussions of forking a process at the OS level, a fair amount of OCaml code, parser combinators, and lazy evaluation. Perhaps my attention drifted elsewhere at a key moment.

The speaker closed his talk by showing a dense slide and saying, "Here is a list of buzzwords, some of which I said in my talk and some of which I didn't say in my talk." That made me laugh: a summary of a talk he may or may not have given. That seemed like a great way to end a talk about guessing.


I don't know much about the details of Akka. Many of my Scala-hacking former students talk about it every so often, so I figured I'd listen to this quick tour and pick up a little more. The underlying idea, of course, is Hewitt's Actor model. This is something I'm familiar with from my days in AI and my interest in Smalltalk.

The presenter, Akka creator Jonas Boner, reminded the audience that Actors were a strong influence on the original Smalltalk. In many ways, it is truer to Kay's vision of OOP than the languages we use today.

This talk was a decent introduction to Hewitt's idea and its implementation in Akka. My two favorite things from the talk weren't technical details, but word play:

  • The name "Akka" has many inspirations, including a mountain in northern Sweden, a goddess of the indigenous people of northern Scandinavia, and a palindrome of Actor Kernel / Kernel Actor.

  • Out of context, this quote made the talk for me:
    We have made some optimizations to random.
    Ah, aren't we all looking for those?

Expressiveness and Abstraction

This talk by Ola Bini was a personal meditation on the expressiveness of language. Bini, whose first slide listed him as a "computational metalinguist", started from the idea that, informally, the expressiveness of a language is inversely proportional to the distance between our thoughts and the code we have to write in that language.

In the middle part of the talk, he considered a number of aspects of expressiveness and abstraction. In the latter part, he listed ideas from natural language and wondered aloud what their equivalents would be in programming languages, among them similes, metaphors, repetition, elaboration, and multiple equivalent expressions with different connotations.

During this part of the talk, my mind wandered, too, to a blog entry I wrote about parts of speech in programming languages back in 2003, and a talk by Crista Lopes at OOPSLA that year. Nouns, verbs, pronouns, adjectives, and adverbs -- these are terms I use metaphorically when teaching students about new languages. Then I thought about different kinds of sentence -- declarative, interrogative, imperative, and exclamatory -- and began to think about their metaphorical appearances in our programming languages.

Another fitting way for a talk to end: my mind wondering at the end of a wondering talk.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 27, 2012 5:33 PM

StrangeLoop 4: Computing Like The Brain

Tuesday morning kicked off with a keynote address by Jeff Hawkins entitled "Computing Like The Brain". Hawkins is currently with Numenta, a company he co-founded in 2005, after having founding the Redwood Neuroscience Institute and two companies most technophiles will recognize: Palm and Handspring.

Hawkins said he has devoted his professional life to understanding machine science. He recalls reading an article by Francis Crick in Scientific American as a youth and being inspired to study neuroscience. It was a data-rich, theory-poor discipline, one crying out for abstractions to unify our understanding of how the brain works from the mass of data we were collecting. He says he dedicated life then to discovering principles of how the brain works, especially the neocortex, and to build computer systems that implement these principles.

The talk began with a primer on the neocortex, which can be thought of as a predictive modeling system to controls human intelligence. If we take into account all the components of what we think of as our five senses, the brain has millions of sensors that constantly stream data to the neocortex. Its job is to build an on-line model from this streaming data. It constantly predicts what he expects to receive next, detects anomalies, updates itself, and produces actions. When the neocortex updates, we learn.

On this view, the brain doesn't "compute". It is a memory system. (I immediately thought of Roger Schank, his views on AI, and case-based reasoning...) The brain is really one memory algorithm operating over all of our sensory inputs. The key elements of this memory system are:

  • a hierarchy of regions,
  • sequence memory, and
  • sparse distributed representation.

Hawkins spoke briefly about hierarchy and sequence memory, but he quickly moved into the idea of sparse distributed representation (SDR). This can be contrasted to the dense, localized memory of traditional computer systems. For example, ASCII code consists of seven bits, all combinations of which we use to represent a single character. Capital 'A' is 65, or 1000001; the digit '5' is 55, or 0110111. The coincidence of '5' and 55 notwithstanding, the individual bits of an ASCII code don't mean anything. Change one bit, and you get a different character, sometimes a very different one.

An SDR uses a large number of bits, with only a few set to 1. Hawkins said that typically only ~ 2% of the bits are "on". Each bit in an SDR has specific meaning, one that has been learned through memory updating, not assigned. He then demonstrated several properties of an SDR, such as how it can be used to detect similarities, how it can do "store-and-compare" using only indices, and how it can perform remarkably well using on a sampling of the indices. Associative look-up in the brain's SDR produces surprisingly few errors, and those tend to be related to the probe, corresponding to similar situations encountered previously.

The first takeaway point of the talk was this: Intelligent systems of the future will be built using sparse distributed representation.

At this point, my note-taking slowed. I am not a biologist, so most of what Hawkins was describing lies far outside my area of expertise. So I made a master note -- gotta get this guy's book! -- and settled into more focused listening.

(It turns out that a former student recommended Hawkins's book, On Intelligence, to me a year or two ago. I should have listened to Allyn then and read it!)

One phrase that made me smile later in the talk was the semantic meaning of the wrongness. Knowing why something is wrong, or how, is a huge step up on "just" being wrong. Hawkins referred to this in particular as part of the subtlety of making predictions.

To close, Hawkins offered some conjectures. He thinks that the future of machine intelligence will depend on us developing more and better theory to explain how the brain works, especially in the areas of hierarchy and attention. The most compelling implementation will be an embodied intelligence, with embedded agents distributed across billions of sensors. We need better hardware in order to create faster systems. recall that the brain is more a memory systems than a computation device, so better memory is as or more important than better processors. Finally, we need to find a way to increase the level connectivity among components. Neurons have tens or hundreds of connections to other neurons, and these can be grown or strengthened dynamically. Currently, our computer chips are not good at this.

Where will breakthrough applications come from? He's not sure. In the past, breakthrough applications of technologies have not always been where we expected them.

I gotta read more. As a student of AI, I was never been all that interested in neurobiology or even its implications for my discipline. The cognitive level has always excited me more. But Hawkins makes an interesting case that the underlying technologies we need to reach the cognitive level will look more like our brains than today's computers.

Posted by Eugene Wallingford | Permalink | Categories: Computing

September 25, 2012 9:35 PM

StrangeLoop 2: The Future of Databases is In Memory

The conference opened with a keynote address by Michael Stonebraker, who built Ingres, Postgres, and several other influential database systems. Given all the hubbub about NoSQL the last few years, including at StrangeLoop 2010, this talk brought some historical perspective to a conversation that has been dominated in recent years by youngsters. Stonebraker told the audience that the future is indeed here, but from the outside it will look a lot like the past

The problem, of course, is "big data". It's big because of volume, velocity, and variety. Stonebraker framed his opening comments in terms of volume. In the traditional database setting back in the 1980s, we all bought airplane tickets through a travel agent who acted, for all meaningful purposes, in the role of professional terminal operator. We were doing business "at the speed of the intermediary". The ACID properties were inviolable: "Don't you dare lose my data."

Then came change. The internet disintermediated access to database, cutting intermediaries out of the equation. Volume shot through the roof. PDAs further disintermediated access, removing limitations on the locations from which we accessed our data. Volume shot up even further. Suddenly, databases came to be part of the solution to a much broader class of problems: massively online games, ad placement, new forms of commerce. We all know what that meant for volume.

Stonebraker then offered two reality checks to frame the solution to our big data problems. The first involved the cost of computer memory. One terabyte is a really big database for transaction processing, yet it 1TB of memory now costs $25-50K. Furthermore, the price is dropping faster than transaction volume is rising. So: the big data problem is really now a problem for main memory.

The second reality check involved database performance. Well under 10% of the time spent by a typical database is spent doing useful work. Over 90% is overhead: managing a buffer pool, latching, locking, and recovery. We can't make faster databases by creating better DB data structures or algorithms; a better B-tree can affect only 4% of application runtime. If we could eliminate the buffer pool, we can gain up to 25% in performance. We must focus on overhead.

Where to start? We can forget about all the traditional database vendors. They have code lines that are thirty years old and older. They have to manage backward compatibility for a huge installed customer base. They are, from the perspective of the future, bloatware. They can't improve.

How about the trend toward NoSQL? We can just use raw data storage and write our own low-level code, optimized to the task. Well, the first thing to realize is that the compiler already translates SQL into lower-level operations. In the world of databases as almost everywhere else, it is really hard to beat the compiler at its game. High-level languages are good, and our compilers do an awesome job generating near-optimal code. Moving down an abstraction layer is, Stonebraker says, a fundamental mistake: "Always move code to the data, never data to the code."

Second, we must realize that the ACID properties really are a good thing. More important, they are nearly impossible to retrofit into a system that doesn't already provide them. "Eventually consistent" doesn't really mean eventually consistent if it's possible to sell your last piece of inventory. In any situation where there exists a pair of non-commutative transactions, "eventually consistent" is a recipe for corruption.

So, SQL and ACID are good. Let's keep them. Stonebraker says that instead of NoSQL databases, we should build "NewSQL" databases that improve performance through innovative architectures. Putting the database in main memory is one way to start. He addressed several common objections to this idea ("But what if the power fails??") by focusing on speed and replication. Recovery may be slow, but performance is wildly better. We should optimize for the most common case and treat exceptional cases for what they are: rare.

He mentioned briefly several other components of a new database architecture, such horizontally scaling across a cluster of nodes, automatic sharding, and optimization via stored procedures targeted at the most common activities. The result is not a general purpose solution, but then why does it need to be?

I have a lot of gray hair, Stonebraker said, but that means he has seen these wars before. It's better to stick with what we know to be valuable and seek better performance where our technology has taken us.

Posted by Eugene Wallingford | Permalink | Categories: Computing

September 25, 2012 8:35 PM

StrangeLoop 1: A Miscellany of Ideas

For my lunch break, I walked a bit outside, to see the sun and bend my knee a bit. I came back for a set of talks without an obvious common thread. After seeing the talks, I saw a theme: ideas for writing programs more conveniently or more concisely.


David Nolen talked about ClojureScript, a Clojure-like language that compiles to Javascript. As he noted, there is a lot of work in this space, both older and newer. The goal of all that work is to write Javascript more conveniently, or generate it from something else. The goal of ClojureScript is to bring the expressibility and flexible programming style of the Lisp world to JS world. Nolen's talk gave us some insights into the work being done to make the compiler produce efficient Javascript, as well as into why you might use ClojureScript in the first place.

Data Structures and Hidden Code

The message of this talk by Scott Vokes is that your choice in data structures plays a big role in determining how much code you have to write. You can make a lot of code disappear by using more powerful data structures. We can, of course, generalize this claim from data structures to data. This is the theme of functional and object-oriented programming, too. This talk highlights how often we forget the lowly data structure when we think of writing less code.

As Vokes said, your choice in data structures sets the "path of least resistance" for what your program will do and also for the code you will write. When you start writing code, you often don't know what the best data structure for your application is. As long as you don't paint yourself into a corner, you should be able to swap a new structure in for the old. The key to this is something novice programmers learn early, writing code not in terms of a data structure but in terms of higher-level behaviors. Primitive obsession can become implementation obsession if you aren't careful.

The meat of this talk was a quick review of four data structures that most programmers don't learn in school: skip lists, difference list, rolling hashes, and jumpropes, a structure Vokes claims to invented.

This talk was a source of several good quote, including

  • "A data structure is just a stupid programming language." -- Bill Gosper
  • "A data structure is just a virtual machine." -- Vokes himself, responding to Gosper
  • "The cheapest, fastest, and most reliable components are those that aren't there." -- Gordon Bell

The first two quotes there would make nice mottos for a debate between functional and OO programming. They also are two sides of the same coin, which destroys the premise of the debate.


As a Scheme programmer and a teacher of programming languages, I have developed great respect and fondness for the work of Dan Friedman over the last fifteen years. As a computer programmer who began his studies deeply interested in AI, I have long had a fondness for Prolog. How could I not go to the talk on miniKanren? This is a small implementation (~600 lines written in a subset of Scheme) of Kanren, a declarative logic programming system described in The Reasoned Schemer.

This talk was like a tag-team vaudeville act featuring Friedman and co-author William Byrd. I can't so this talk justice in a blog entry. Friedman and Byrd interleaved code demo with exposition as they

  • showed miniKanren at its simplest, built from three operators (fresh, conde, and run)
  • extended the language with a few convenient operators for specifying constraints, types, and exclusions, and
  • illustrated how to program in miniKanren by building a language interpreter, EOPL style.

The cool endpoint of using logic programming to build the interpreter is that, by using variables in a specification, the interpreter produces legal programs that meet a given specification. It generates code via constraint resolution.

If that weren't enough, they also demo'ed how their system can, given a language grammar, produce quines -- programs p such that

    (equal p (eval p))
-- and twines, pairs of programs p and q such that
    (and (equal p (eval q))
         (equal q (eval p)))

Then they live-coded an implementation of typed lambda calculus.

Yes, all in fifty minutes. Like I said, you really need to watch the talk at InfoQ as soon as it's posted.

In the course of giving the talk, Friedman stated a rule that my students can use:

Will's law: If your function has a recursion, do the recursion last.

Will followed up with cautionary advice:

Will's second law: If your function has two recursions, call Will.

We'll see how serious he was when I put a link to his e-mail address in my Programming Languages class notes next spring.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 25, 2012 7:31 PM

Blogging from StrangeLoop

StrangeLoop logo

This week I have the pleasure of spending a couple of days expanding my mind at StrangeLoop 2012. I like StrangeLoop because it's a conference for programmers. The program is filled with hot programming topics and languages, plus a few keynotes to punctuate our mental equilibria. The 2010 conference gave me plenty to think about, but I had to skip 2011 while teaching and recovering. This year was a must-see.

I'll be posting the following entries from the conference as time permits me to write them.

You can find links to other write-ups of the conference, as well as slides from some talks and other material, at the StrangeLoop 2012 github site.

Now that the conference has ended, I can say with confidence that StrangeLoop 2012 was even better than StrangeLoop 2010.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 20, 2012 8:09 PM

Computer Science is a Liberal Art

Over the summer, I gave a talk as part of a one-day conference on the STEM disciplines for area K-12, community college, and university advisors. They were interested in, among other things, the kind of classes that CS students take at the university and the kind of jobs they get when they graduate.

In the course of talking about how some of the courses our students take (say, algorithms and the theory of computing) seem rather disconnected from many of the jobs they get (say, web programmer and business analyst), I claimed that the more abstract courses prepare students to understand the parts of the computing world that never change, and the ones that do. The specific programming languages or development stack they use after they graduate to build financial reporting software may change occasionally, but the foundation they get as a CS major prepares them to understand what comes next and to adapt quickly.

In this respect, I said, a university CS education is not job training. Computer Science is a liberal art.

This is certainly true when you compare university CS education with what students get at a community college. Students who come out of a community college networking program often possess specific marketable skills at a level we are hard-pressed to meet in a university program. We bank our program's value on how well it prepares students for a career, in which networking infrastructure changes multiple times and our grads are asked to work at the intersection of networks and other areas of computing, some of which may not exist yet.

It is also true relative to the industries they enter after graduation. A CS education provides a set of basic skills and, more important, several ways to think about problems and formulate solutions. Again, students who come out of a targeted industry or 2-year college training program in, say, web dev, often have "shovel ready" skills that are valuable in industry and thus highly marketable. We bank our program's value on how well it prepares students for a career in which ASP turns to JSP turns PHP turns to JavaScript. Our students should be prepared to ramp up quickly and have a shovel in the hands producing value soon.

And, yes, students in a CS program must learn to write code. That's a basic skill. I often hear people comment that computer science programs do not prepare students well for careers in software development. I'm not sure that's true, at least at schools like mine. We can't get away with teaching all theory and abstraction; our students have to get jobs. We don't try to teach them everything they need to know to be good software developers, or even many particular somethings. That should and will come on the job. I want my students to be prepared for whatever they encounter. If their company decides to go deep with Scala, I'd like my former students to be ready to go with them.

In a comment on John Cook's timely blog entry How long will there be computer science departments?, Daniel Lemire suggests that we emulate the model of medical education, in which doctors serve several years in residency, working closely with experienced doctors and learning the profession deeply. I agree. Remember, though, that aspiring doctors go to school for many years before they start residency. In school, they study biology, chemistry, anatomy, and physiology -- the basic science at the foundation of their profession. That study prepares them to understand medicine at a much deeper level than they otherwise might. That's the role CS should play for software developers.

(Lemire also smartly points out that programmers have the ability to do residency almost any time they like, by joining an open source project. I love to read about how Dave Humphrey and people like him bring open-source apprenticeship directly into the undergrad CS experience and wonder how we might do something similar here.)

So, my claim that Computer Science is a liberal arts program for software developers may be crazy, but it's not entirely crazy. I am willing to go even further. I think it's reasonable to consider Computer Science as part of the liberal arts for everyone.

I'm certainly not the first person to say this. In 2010, Doug Baldwin and Alyce Brady wrote a guest editors' introduction to a special issue of the ACM Transactions on Computing Education called Computer Science in the Liberal Arts. In it, they say:

In late Roman and early medieval times, seven fields of study, rooted in classical Greek learning, became canonized as the "artes liberales" [Wagner 1983], a phrase denoting the knowledge and intellectual skills appropriate for citizens free from the need to labor at the behest of others. Such citizens had ample leisure time in which to pursue their own interests, but were also (ideally) civic, economic, or moral leaders of society.


[Today] people ... are increasingly thinking in terms of the processes by which things happen and the information that describes those processes and their results -- as a computer scientist would put it, in terms of algorithms and data. This transformation is evident in the explosion of activity in computational branches of the natural and social sciences, in recent attention to "business processes," in emerging interest in "digital humanities," etc. As the transformation proceeds, an adequate education for any aspect of life demands some acquaintance with such fundamental computer science concepts as algorithms, information, and the capabilities and limitations of both.

The real value in a traditional Liberal Arts education is in helping us find better ways to live, to expose us to the best thoughts of men and women in hopes that we choose a way to live, rather than have history or accident choose a way to live for us. Computer science, like mathematics, can play a valuable role in helping students connect with their best aspirations. In this sense, I am comfortable at least entertaining the idea that CS is one of the modern liberal arts.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

September 05, 2012 5:24 PM

Living with the Masters

I sometimes feel guilty that most of what I write here describes connections between teaching or software development and what I see in other parts of the world. These connections are valuable to me, though, and writing them down is valuable in another way.

I'm certainly not alone. In Why Read, Mark Edmondson argues for the value of reading great literature and trying on the authors' view of the world. Doing so enables us to better understand our own view of the world, It also gives us the raw material out of which to change our worldview, or build a new one, when we encounter better ideas. In the chapter "Discipline", Edmondson writes:

The kind of reading that I have been describing here -- the individual quest for what truth a work reveals -- is fit for virtually all significant forms of creation. We can seek vital options in any number of places. They may be found for this or that individual in painting, in music, in sculpture, in the arts of furniture making or gardening. Thoreau felt he could derive a substantial wisdom by tending his bean field. He aspired to "know beans". He hoed for sustenance, as he tells us, but he also hoed in search of tropes, comparisons between what happened in the garden and what happened elsewhere in the world. In his bean field, Thoreau sought ways to turn language -- and life -- away from old stabilities.

I hope that some of my tropes are valuable to you.

The way Edmondson writes of literature and the liberal arts applies to the world of software in a much more direct ways too. First, there is the research literature of computing and software development. One can seek truth in the work of Alan Kay, David Ungar, Ward Cunningham, or Kent Beck. One can find vital options in the life's work of Robert Floyd, Peter Landin, or Alan Turing; Herbert Simon, Marvin Minsky, or John McCarthy. I spent much of my time in grad school immersed in the writings and work of B. Chandrasekaran, which affected my view of intelligence in both humans and machines.

Each of these people offers a particular view into a particular part of the computing world. Trying out their worldviews can help us articulate our own worldviews better, and in the process of living their truths we sometimes find important new truths for ourselves.

We in computing need not limit ourselves to the study of research papers and books. As Edmondson says the individual quest for the truth revealed in a work "is fit for virtually all significant forms of creation". Software is a significant form of creation, one not available to our ancestors even sixty years ago. Live inside any non-trivial piece of software for a while, especially one that has withstood the buffets of human desire over a period of time, and you will encounter truth -- truths you find there, and truths you create for yourself. A few months trying on Smalltalk and its peculiar view of the world taught me OOP and a whole lot more.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development, Teaching and Learning

August 08, 2012 1:50 PM

Examples First, Names Last

Earlier this week, I reviewed a draft chapter from a book a friend is writing, which included a short section on aspect-oriented programming. The section used common AOP jargon: "cross cutting", "advice", and "point cut". I know enough about AOP to follow his text, but I figured that many of his readers -- young software developers from a variety of backgrounds -- would not. On his use of "cross cutting", I commented:

Your ... example helps to make this section concrete, but I bet you could come up with a way of explaining the idea behind AOP in a few sentences that would be (1) clear to readers and (2) not use "cross cutting". Then you could introduce the term as the name of something they already understand.

This may remind you of the famous passage from Richard Feynman about learning names and understanding things. (It is also available on a popular video clip.) Given that I was reviewing a chapter for a book of software patterns, it also brought back memories of advice that Ralph Johnson gave many years ago on the patterns discussion list. Most people, he said, learn best from concrete examples. As a result, we should write software patterns in such a way that we lead with a good example or two and only then talk about the general case. In pattern style, he called this idea "Concrete Before Abstract".

I try to follow this advice in my teaching, though I am not dogmatic about it. There is a lot of value in mixing up how we organize class sessions and lectures. First, different students connect better with some approaches than others, so variety increases the chances that of connecting with everyone a few times each semester. Second, variety helps to keep students in interested, and being interested is a key ingredient in learning.

Still, I have a preference for approaches that get students thinking about real code as early as possible. Starting off by talking about polymorphism and its theoretical forms is a lot less effective at getting the idea across to undergrads than showing students a well-chosen example or two of how plugging a new object into an application makes it easier to extend and modify programs.

So, right now, I have "Concrete Before Abstract" firmly in mind as I prepare to teaching object-oriented programming to our sophomores this fall.

Classes start in twelve days. I figured I'd be blogging more by now about my preparations, but I have been rethinking nearly everything about the way I teach the course. That has left my mind more muddled that settled for long stretches. Still, my blog is my outboard brain, so I should be rethinking more in writing.

I did have one crazy idea last night. My wife learned Scratch at a workshop this summer and was talking about her plans to use it as a teaching tool in class this fall. It occurred to me that implementing Scratch would be a fun exercise for my class. We'll be learning Java and a little graphics programming as a part of the course, and conceptually Scratch is not too many steps from the pinball game construction kit in Budd's Understanding Object-Oriented Programming with Java, the textbook I have used many times in the course. I'm guessing that Budd's example was inspired by Bill Budge's game for Electronic Arts, Pinball Construction Set. (Unfortunately, Budd's text is now almost as out of date as that 1983 game.)

Here is an image of a game constructed using the pinball kit and Java's AWT graphics framework:

a pinball game constructed using a simple game kit

The graphical ideas needed to implement Scratch are a bit more complex, including at least:

  • The items on the canvas must be clickable and respond to messages.
  • Items must be able to "snap" together to create units of program. This could happen when a container item such as a choice or loop comes into contact with an item it is to contain.

The latter is an extension of collision-detecting behavior that students would be familiar with from earlier "ball world" examples. The former is something we occasionally do in class anyway; it's awfully handy to be able to reconfigure the playing field after seeing how the game behaves with the ball in play. The biggest change would be that the game items are little pieces of program that know how to "interpret" themselves.

As always, the utility of a possible teaching idea lies in the details of implementing it. I'll give it a quick go over the next week to see if it's something I think students would be able to handle, either as a programming assignment or as an example we build and discuss in class.

I'm pretty excited by the prospect, though. If this works out, it will give me a nice way to sneak basic language processing into the course in a fun way. CS students should see and think about languages and how programs are processed throughout their undergrad years, not only in theory courses and programming languages courses.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Teaching and Learning

August 03, 2012 3:23 PM

How Should We Teach Algebra in 2012?

Earlier this week, Dan Meyer took to task a New York Times opinion piece from the weekend, Is Algebra Necessary?:

The interesting question isn't, "Should every high school graduate in the US have to take Algebra?" Our world is increasingly automated and programmed and if you want any kind of active participation in that world, you're going to need to understand variable representation and manipulation. That's Algebra. Without it, you'll still be able to clothe and feed yourself, but that's a pretty low bar for an education. The more interesting question is, "How should we define Algebra in 2012 and how should we teach it?" Those questions don't even seem to be on Hacker's radar.

"Variable representation and manipulation" is a big part of programming, too. The connection between algebra and programming isn't accidental. Matthias Felleisen won the ACM's Outstanding Educator Award in 2010 for his long-term TeachScheme! project, which has now evolved into Program by Design. In his SIGCSE 2011 keynote address, Felleisen talked about the importance of a smooth progression of teaching languages. Another thing he said in that talk stuck with me. While talking about the programming that students learned, he argued that this material could be taught in high school right now, without displacing as much material as most people think. Why? Because "This is algebra."

Algebra in 2012 still rests fundamentally on variable representation and manipulation. How should we teach it? I agree with Felleisen. Programming.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

July 20, 2012 3:39 PM

A Philosopher of Imitation

Ian Bogost, in The Great Pretender: Turing as a Philosopher of Imitation, writes:

Intelligence -- whatever it is, the thing that goes on inside a human or a machine -- is less interesting and productive a topic of conversation than the effects of such a process, the experience it creates in observers and interlocutors.

This is a very nice one-sentence summary of Turing's thesis in Computing Machinery and Intelligence. I wrote a bit about Turing's ideas on machine intelligence a few months back, but the key idea in Bogost's essay relates more closely to my discussion in Turing's ideas on representation and universal machines.

In this centennial year of his birth, we can hardly go wrong in considering again and again the depth of Turing's contributions. Bogost uses a lovely turn of phrase in his title: a philosopher of imitation. What may sound like a slight or a trifle is, in fact, the highest of compliments. Turing made that thinkable.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 18, 2012 2:31 PM

Names, Values, and The Battle of Bull Run

the cover of 'Encyclopedia Brown Finds the Clues'

Author Donald Sobol died Monday. I know him best from his long-running series, Encyclopedia Brown. Like many kids of my day, I loved these stories. I couldn't get enough. Each book consisted of ten or so short mysteries solved by Encyclopedia or Sally Kimball, his de facto partner in the Brown Detective Agency. I wanted to be Encyclopedia.

The stories were brain teasers. Solving them required knowledge and, more important, careful observation and logical deduction. I learned to pay close attention while reading Encyclopedia Brown, otherwise I had no hope of solving the crime before Encyclopedia revealed the solution. In many ways, these stories prepared me for a career in math and science. They certainly were a lot of fun.

One of the stories I remember best after all these years is "The Case of the Civil War Sword", from the very first Encyclopedia Brown book. I'm not the only person who found it memorable; Rob Bricken ranks it #9 among the ten most difficult Encyclopedia Brown mysteries. The solution to this case turned on the fact that one battle had two different names. Northerners often named battles for nearby bodies of water or prominent natural features, while Southerners named them for the nearest town or prominent man-made features. So, the First Battle of Bull Run and the First Battle of Manassas were the same event.

This case taught me a bit of historical trivia and opened my mind to the idea that naming things from the Civil War was not trivial at all.

This story taught me more than history, though. As a young boy, it stood out as an example of something I surely already knew: names aren't unique. The same value can have different names. In a way, Encyclopedia Brown taught me one of my first lessons about computer science.


IMAGE: the cover of Encyclopedia Brown Finds the Clues, 1966. Source: Topless Robot.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 14, 2012 11:01 AM

"Most Happiness Comes From Friction"

Last time, I mentioned again the value in having students learn broadly across the sciences and humanities, including computer science. This is a challenge going in both directions. Most students like to concentrate on one area, for a lot of different reasons. Computer science looks intimidating to students in other majors, perhaps especially to the humanities-inclined.

There is hope. Earlier this year, the Harvard Magazine ran The Frisson of Friction, an essay by Sarah Zhang, a non-CS student who decided to take CS 50, Harvard's intro to computer science. Zhang tells the story of finding a thorny, semicolon-induced bug in a program (an extension for Google's Chrome browser) on the eve of her 21st birthday. Eventually, she succeeded. In retrospect, she writes:

Plenty of people could have coded the same extension more elegantly and in less time. I will never be as good a programmer as -- to set the standard absurdly high -- Mark Zuckerberg. But accomplishments can be measured in terms relative to ourselves, rather than to others. Rather than sticking to what we're already good at as the surest path to résumé-worthy achievements, we should see the value in novel challenges. How else will we discover possibilities that lie just beyond the visible horizon?

... Even the best birthday cake is no substitute for the deep satisfaction of accomplishing what we had previously deemed impossible -- whether it's writing a program or writing a play.

The essay addresses some of the issues that keep students from seeking out novel challenges, such as fear of low grades and fear of looking foolish. At places like Harvard, students who are used to succeeding find themselves boxed in by their friends' expectations, and their own, but those feelings are familiar to students at any school. Then you have advisors who subtly discourage venturing too far from the comfortable, out of their own unfamiliarity and fear. This is a social issue as big as any pedagogical challenge we face in trying to make introductory computer science more accessible to more people.

With work, we can help students feel the deep satisfaction that Zhang experienced. Overcoming challenges often leads to that feeling. She quotes a passage about programmers in Silicon Valley, who thrive on such challenges: "Most happiness probably comes from friction." Much satisfaction and happiness come out of the friction inherent in making things. Writing prose and writing programs share this characteristic.

Sharing the deep satisfaction of computer science is a problem with many facets. Those of us who know the satisfaction know it's a problem worth solving.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

July 13, 2012 12:02 PM

How Science -- and Computing -- Are Changing History

While reading a recent Harvard Magazine article about Eric Mazur's peer instruction technique in physics teaching, I ran across a link to an older paper that fascinated me even more! Who Killed the Men of England? tells several stories of research at the intersection of history, archaeology, genomics, evolution, demography, and simulation, such as the conquest of Roman England by the Anglo Saxons.

Not only in this instance, but across entire fields of inquiry, the traditional boundaries between history and prehistory have been melting away as the study of the human past based on the written record increasingly incorporates the material record of the natural and physical sciences. Recognizing this shift, and seeking to establish fruitful collaborations, a group of Harvard and MIT scholars have begun working together as part of a new initiative for the study of the human past. Organized by [professor of medieval history Michael] McCormick, who studies the fall of the Roman empire, the aim is to bring together researchers from the physical, life, and computer sciences and the humanities to explore the kinds of new data that will advance our understanding of human history.

... The study of the human past, in other words, has entered a new phase in which science has begun to tell stories that were once the sole domain of humanists.

I love history as much as computing and was mesmerized by these stories of how scientists reading the "material record" of the world are adding to our knowledge of the human past.

However, this is more than simply a one-way path of information flowing from scientists to humanists. The scientific data and models themselves are underconstrained. The historians, cultural anthropologists, and demographers are able to provide context to the data and models and so extract even more meaning from them. This is a true collaboration. Very cool.

The rise of science is erasing boundaries between the disciplines that we all studied in school. Scholars are able to define new disciplines, such as "the study of the human past", mentioned in the passage above. These disciplines are organized with a greater focus on what is being studied than on how we are studying it.

We are also blurring the line between history and pre-history. It used to be that history required a written record, but that is no longer a hard limit. Science can read nature's record. Computer scientists can build models using genomic data and migration data that suggest possible paths of change when the written and scientific record are incomplete. These ideas become part of the raw material that humanists use to construct a coherent story of the past.

This change in how we are able to study the world highlights the importance of a broad education, something I've written about a few times recently [ 1 | 2 | 3 ] and not so recently. This sort of scholarship is best done by people who are good at several things, or at least curious and interested enough in several things to get to know them intimately. As I wrote in Failure and the Liberal Arts, it's important both not to be too narrowly trained and not to be too narrowly "liberally educated".

Even at a place like Harvard, this can leave scholars in a quandary:

McCormick is fired with enthusiasm for the future of his discipline. "It is exciting. I jump up every morning. But it is also challenging. Division and department boundaries are real. Even with a generally supportive attitude, it is difficult [to raise funds, to admit students who are excellent in more than one discipline, and so on]. ..."

So I will continue to tell computer science students to take courses from all over the university, not just from CS and math. This is one point of influence I have as a professor, advisor, and department head. And I will continue to look for ways to encourage non-CS students to take CS courses and students outside the sciences to study science, including CS. As that paragraph ends:

"... This is a whole new way of studying the past. It is a unique intellectual opportunity and practically all the pieces are in place. This should happen here--it will happen, whether we are part of it or not."

"Here" doesn't have to be Harvard. There is a lot of work to be done.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 11, 2012 2:45 PM

A Few Comments on the Alan Kay Interview, and Especially Patterns

Alan Kay

Many of my friends and colleagues on Twitter today are discussing the Interview with Alan Kay posted by Dr. Dobb's yesterday. I read the piece this morning while riding the exercise bike and could not contain my desire to underline passages, star paragraphs, and mark it up with my own comments. That's hard to do while riding hard, hurting a little, and perspiring a lot. My desire propelled me forward in the face of all these obstacles.

Kay is always provocative, and in this interview he leaves no oxen ungored. Like most people do when whenever they read outrageous and provocative claims, I cheered when Kay said something I agreed with and hissed -- or blushed -- when he said something that gored me or one of my pet oxen. Twitter is a natural place to share one's cheers and boos for an artyicle with or by Alan Kay, given the amazing density of soundbites one finds in his comments about the world of computing.

(One might say the same thing about Brian Foote, the source of both soundbites in that paragraph.)

I won't air all my cheers and hisses here. Read the article, if you haven't already, and enjoy your own. I will comment on one paragraph that didn't quite make me blush:

The most disastrous thing about programming -- to pick one of the 10 most disastrous things about programming -- there's a very popular movement based on pattern languages. When Christopher Alexander first did that in architecture, he was looking at 2,000 years of ways that humans have made themselves comfortable. So there was actually something to it, because he was dealing with a genome that hasn't changed that much. I think he got a few hundred valuable patterns out of it. But the bug in trying to do that in computing is the assumption that we know anything at all about programming. So extracting patterns from today's programming practices ennobles them in a way they don't deserve. It actually gives them more cachet.

Long-time Knowing and Doing readers know that patterns are one of my pet oxen, so it would have been natural for me to react somewhat as Keith Ray did and chide Kay for what appears to be a typical "Hey, kids, get off my lawn!" attitude. But that's not my style, and I'm such a big fan of Kay's larger vision for computing that my first reaction was to feel a little sheepish. Have I been wasting my time on a bad idea, distracting myself from something more important? I puzzled over this all morning, and especially as I read other people's reactions to the interview.

Ultimately, I think that Kay is too pessimistic when he says we hardly know anything at all about programming. We may well be closer to the level of the Egyptians who built the pyramids than we are to the engineers who built the Empire State Building. But I simply don't believe that people such as Ward Cunningham, Ralph Johnson, and Martin Fowler don't have a lot to teach most of us about how to make better software.

Wherever we are, I think it's useful to identify, describe, and catalog the patterns we see in our software. Doing so enables us to talk about our code at a level higher than parentheses and semicolons. It helps us bring other programmers up to speed more quickly, so that we don't all have to struggle through all the same detours and tar pits our forebears struggled through. It also makes it possible for us to talk about the strengths and weaknesses of our current patterns and to seek out better ideas and to adopt -- or design -- more powerful languages. These are themes Kay himself expresses in this very same interview: the importance of knowing our history, of making more powerful languages, and of education.

Kay says something about education in this interview that is relevant to the conversation on patterns:

Education is a double-edged sword. You have to start where people are, but if you stay there, you're not educating.

The real bug in what he says about patterns lies at one edge of the sword. We may not know very much about how to make software yet, but if we want to remedy that, we need to start where people are. Most software patterns are an effort to reach programmers who work in the trenches, to teach them a little of what we do know about how to make software. I can yammer on all I want about functional programming. If a Java practitioner doesn't appreciate the idea of a Value Object yet, then my words are likely wasted.

Ward Cunningham

Ironically, many argue that the biggest disappointment of the software patterns effort lies at the other edge of education's sword: an inability to move the programming world quickly enough from where it was in the mid-1990s to a better place. In his own Dr. Dobb's interview, Ward Cunningham observed with a hint of sadness that an unexpected effect of the Gang of Four Design Patterns book was to extend the life of C++ by a decade, rather than reinvigorating Smalltalk (or turning people on to Lisp). Changing the mindset of a large community takes time. Many in the software patterns community tried to move people past a static view of OO design embodied in the GoF book, but the vocabulary calcified more quickly than they could respond.

Perhaps that is all Kay meant by his criticism that patterns "ennoble" practices in a way they don't deserve. But if so, it hardly qualifies in my mind as "one of the 10 most disastrous things about programming". I can think of a lot worse.

Kurt Vonnegut

To all this, I can only echo the Bokononists in Kurt Vonnegut's novel Cat's Cradle: "Busy, busy, busy." The machinery of life is usually more complicated and unpredictable than we expect or prefer. As a result, reasonable efforts don't always turn out as we intend them to. So it goes. I don't think that means we should stop trying.

Don't let my hissing about one paragraph in the interview dissuade you from reading the Dr. Dobb's interview. As usual, Kay stimulates our minds and encourages us to do better.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

June 28, 2012 4:13 PM

"Doing research is therefore writing software."

The lede from RA Manual: Notes on Writing Code, by Gentzkow and Shapiro:

Every step of every research project we do is written in code, from raw data to final paper. Doing research is therefore writing software.

The authors are economists at the University of Chicago. I have only skimmed the beginning of the paper, but I like what little I've seen. They take seriously the writing of computer programs.

  • "This document lays out some broad principles we should all follow."
  • "We encourage you to invest in reading more broadly about software craftsmanship, looking critically at your own code and that of your colleagues, and suggesting improvements or additions to the principles below."
  • "Apply these principles to every piece of code you check in without exception."
  • "You should also take the time to improve code you are modifying or extending even if you did not write the code yourself."

...every piece of code you check in... Source code management and version control? They are a couple of steps up on many CS professors and students.

Thanks to Tyler Cowen for the link.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

June 19, 2012 3:04 PM

Basic Arithmetic, APL-Style, and Confident Problem Solvers

After writing last week about a cool array manipulation idiom, motivated by APL, I ran across another reference to "APL style" computation yesterday while catching up with weekend traffic on the Fundamentals of New Computing mailing list. And it was cool, too.

Consider the sort of multi-digit addition problem that we all spend a lot of time practicing as children:

     +  366

The technique requires converting two-digit sums, such as 6 + 5 = 11 in the rightmost column, into a units digit and carrying the tens digit into the next column to the left. The process is straightforward but creates problems for many students. That's not too surprising, because there is a lot going on in a small space.

David Leibs described a technique, which he says he learned from something Kenneth Iverson wrote, that approaches the task of carrying somewhat differently. It takes advantage of the fact that a multi-digit number is a vector of digits times another vector of powers.

First, we "spread the digits out" and add them, with no concern for overflow:

        3   6   5
     +  3   6   6
        6  12  11

Then we normalize the result by shifting carries from right to left, "in fine APL style".

        6  12  11
        6  13   1
        7   3   1

According to Leibs, Iverson believed that this two-step approach was easier for people to get right. I don't know if he had any empirical evidence for the claim, but I can imagine why it might be true. The two-step approach separates into independent operations the tasks of addition and carrying, which are conflated in the conventional approach. Programmers call this separation of concerns, and it makes software easier to get right, too.

Multiplication can be handled in a conceptually similar way. First, we compute an outer product by building a digit-by-digit times table for the digits:

     |   |  3  6  6|
     | 3 |  9 18 18|
     | 6 | 18 36 36|
     | 5 | 15 30 30|

This is straightforward, simply an application of the basic facts that students memorize when they first learn multiplication.

Then we sum the diagonals running southwest to northeast, again with no concern for carrying:

     (9) (18+18) (18+36+15) (36+30) (30)
      9      36         69      66   30

In the traditional column-based approach, we do this implicitly when we add staggered columns of digits, only we have to worry about the carries at the same time -- and now the carry digit may be something other than one!

Finally, we normalize the resulting vector right to left, just as we did for addition:

         9  36  69  66  30
         9  36  69  69   0
         9  36  75   9   0
         9  43   5   9   0
        13   3   5   9   0
     1   3   3   5   9   0

Again, the three components of the solution are separated into independent tasks, enabling the student to focus on one task at a time, using for each a single, relatively straightforward operator.

(Does this approach remind some of you of Cannon's algorithm for matrix multiplication in a two-dimensional mesh architecture?)

Of course, Iverson's APL was designed around vector operations such as these, so it includes operators that make implementing such algorithms as straightforward as the calculate-by-hand technique. Three or four Greek symbols and, voilá, you have a working program. If you are Dave Ungar, you are well on your way to a compiler!

the cover of High-Speed Math Self-Taught, by Lester Meyers

I have a great fondness for alternative ways to do arithmetic. One of the favorite things I ever got from my dad was a worn copy of Lester Meyers's High-Speed Math Self-Taught. I don't know how many hours I spent studying that book, practicing its techniques, and developing my own shortcuts. Many of these techniques have the same feel as the vector-based approaches to addition and multiplication: they seem to involve more steps, but the steps are simpler and easier to get right.

A good example of this I remember learning from High-Speed Math Self-Taught is a shortcut for finding 12.5% of a number: first multiply by 100, then divide by 8. How can a multiplication and a division be faster than a single multiplication? Well, multiplying by 100 is trivial: just add two zeros to the number, or shift the decimal point two places to the right. The division that remains involves a single-digit divisor, which is much easier than multiplying by a three-digit number in the conventional way. The three-digit number even has its own decimal point, which complicates matters further!

To this day, I use shortcuts that Meyers taught me whenever I'm making updating the balance in my checkbook register, calculating a tip in a restaurant, or doing any arithmetic that comes my way. Many people avoid such problems, but I seek them out, because I have fun playing with the numbers.

I am able to have fun in part because I don't have to worry too much about getting a wrong answer. The alternative technique allows me to work not only faster but also more accurately. Being able to work quickly and accurately is a great source of confidence. That's one reason I like the idea of teaching students alternative techniques that separate concerns and thus offer hope for making fewer mistakes. Confident students tend to learn and work faster, and they tend to enjoy learning more than students who are handcuffed by fear.

I don't know if anyone was tried teaching Iverson's APL-style basic arithmetic to children to see if it helps them learn faster or solve problems more accurately. Even if not, it is both a great demonstration of separation of concerns and a solid example of how thinking about a problem differently opens the door to a new kind of solution. That's a useful thing for programmers to learn.


Postscript. If anyone has a pointer to a paper or book in which Iverson talks about this approach to arithmetic, I would love to hear from you.

IMAGE: the cover of Meyers's High-Speed Math Self-Taught, 1960. Source:

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

May 29, 2012 2:44 PM

Some Final Thoughts and Links from JRubyConf

You are probably tired of hearing me go on about JRubyConf, so I'll try to wrap up with one more post. After the first few talks, the main result of the rest of conference was to introduced me to several cool projects and a few interesting quotes.

Sarah Allen speaking on agile business development

Sarah Allen gave a talk on agile business development. Wow, she has been part of creating several influential pieces of software, including AfterEffects, ShockWave, and FlashPlayer. She talked a bit about her recent work to increase diversity among programmers and reminded us that diversity is about more than the categories we usually define:

I may be female and a minority here, but I'm way more like everybody in here than everybody out there.

Increasing diversity means making programming accessible to people who wouldn't otherwise program.

Regarding agile development, Sarah reminded us that agile's preference for working code over documentation is about more than just code:

Working code means not only "passes the tests" but also "works for the customer".

... which is more about being the software they need than simply getting right answers to some tests written in JUnit.

Nate Schutta opened day two with a talk on leading technical change. Like Venkat Subramaniam on day one, Schutta suggested that tech leaders consider management's point of view when trying to introduce new technology, in particular the risks that managers face. If you can tie new technology to the organization's strategic goals and plans, then managers can integrate it better into other actions. Schutta attributed his best line to David Hussman:

Change must happen with people, not to them.

The award for the conference's most entertaining session goes to Randall Thomas and Tammer Saleh for "RUBY Y U NO GFX?", their tag-team exegesis of the history of computer graphics and where Ruby fits into that picture today. They echoed several other speakers in saying that JRuby is the bridge to the rest of the programming world that Ruby programmers need, because the Java community offers so many tools. For example, it had never occurred to me to use JRuby to connect my Ruby code to Processing, the wonderful open-source language for programming images and animations. (I first mentioned Processing here over four years ago in its original Java form, and most recently was thinking of its JavaScript implementation.)

Finally, a few quickies:

  • Jim Remsik suggested Simon Sinek's TED talk, How great leaders inspire action, with the teaser line, It's not what you do; it's why you do it.

  • Yoko Harada introduced me to Nokogiri, a parser for HTML, XML, and the like.

  • Andreas Ronge gave a talk on graph databases as a kind of NoSQL database and specifically about Neo4j.rb, his Ruby wrapper on the Java library Neo4J.

  • I learned about Square, which operates in the #fintech space being explored by the Cedar Valley's own T8 Webware and by Iowa start-up darling Dwolla.

  • rapper Jay Z
    I mentioned David Wood in yesterday's entry. He also told a great story involving rapper Jay-Z, illegal music downloads, multi-million-listener audiences, Coca Cola, logos, and video releases that encapsulated in a nutshell the new media world in which we live. It also gives a very nice example of why Jay-Z will soon be a billionaire, if he isn't already. He gets it.

  • The last talk I attended before hitting the road was by Tony Arcieri, on concurrent programming in Ruby, and in particular his concurrency framework Celluloid. It is based on the Actor model of concurrency, much like Erlang and Scala's Akka framework. Regarding these two, Arcieri said that Celluloid stays truer the original model's roots than Akka by having objects at its core and that he currently views any differences in behavior between Celluloid and Erlang as bugs in Celluloid.

One overarching theme for me of my time at JRubyConf: There is a lot of stuff I don't know. I won't run out of things to read and learn and do for a long, long, time.


IMAGE 1: my photo of Sarah Allen during her talk on agile business development. License: Creative Commons Attribution-ShareAlike 3.0 Unported.

IMAGE 2: Jay-Z, 2011. Source: Wikimedia Commons.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

May 28, 2012 10:58 AM

The Spirit of Ruby... and of JRuby

JRubyConf was my first Ruby-specific conference, and one of the things I most enjoyed was seeing how the spirit of the language permeates the projects created by its community of users. It's one thing to read books, papers, and blog posts. It's another to see the eyes and mannerisms of the people using the language to make things they care about. Being a variant, JRuby has its own spirit. Usually it is in sync with Ruby's, but occasionally it diverges.

the letter thnad, created

The first talk after lunch was by Ian Dees, talking about his toy programming language project Thnad. (He took the name from one of the new letters of the alphabet in Dr. Seuss's On Beyond Zebra.) Thnad looks a lot like Klein, a language I created for my compiler course a few years ago, a sort of functional integer assembly language.

The Thnad project is a great example of how easy it is to roll little DSLs using Ruby and other DSLs created in it. To implement Thnad, Dees uses Parslet, a small library for generating scanners and parsers PEG-style, and BiteScript, a Ruby DSL for generating Java bytecode and classes. This talk demonstrated the process of porting Thnad from JRuby to Rubinius, a Ruby implementation written in Ruby. (One of the cool things I learned about the Rubinius compiler is that it can produce s-expressions as output, using the switch -S.)

Two other talks exposed basic tenets of the Ruby philosophy and the ways in which implementations such as JRuby and Rubinius create new value in the ecosystem.

On Wednesday afternoon, David Wood described how his company, the Jun Group, used JRuby to achieve the level of performance its on-line application requires. He told some neat stories about the evolution of on-line media over the last 15-20 years and how our technical understanding for implementing such systems has evolved in tandem. Perhaps his most effective line was this lesson learned along the way, which recalled an idea from the keynote address the previous morning:

Languages don't scale. Architectures do. But language and platform affect architecture.

In particular, after years of chafing, he had finally reached peace with one of the overarching themes of Ruby: optimize for developer ease and enjoyment, rather than performance or scalability. This is true of the language and of most of the tools built around, such as Rails. As a result, Ruby makes it easy to write many apps quickly. Wood stopped fighting the lack of emphasis on performance and scalability when he realized that most apps don't succeed anyway. If one does, you have to rewrite it anyway, so suck it up and do it. You will have benefited from Ruby's speed of delivery.

This is the story Twitter, apparently, and what Wood's team did. They spent three person-months to port their app from MRI to JRuby, and are now quite happy.

Where does some of that performance bump come from? Concurrency. Joe Kutner gave a talk after Thnad on Tuesday afternoon about using JRuby to deploy efficient Ruby web apps on the JVM, in which he also exposed a strand of Ruby philosophy and place where JRuby diverges.

The canonical implementations of Ruby and Python use a Global Interpreter Lock to ensure that non-thread-safe code does not interfere with the code in other threads. In effect, the interpreter maps all threads onto a single thread in the kernel. This may seem like an unnecessary limitation, but it is consistent with Matz's philosophy for Ruby: Programming should be fun and easy. Concurrency is hard, so don't do allow it to interfere with the programmer's experience.

Again, this works just fine for many applications, so it's a reasonable default position for the language. But it does not work so well for web apps, which can't scale if they can't spawn new, independent threads. This is a place where JRuby offers a big win by running atop the JVM, with its support for multithreading. It's also a reason why the Kilim fibers GSoC project mentioned by Charles Nutter in the State of JRuby session is so valuable.

In this talk, I learned about three different approaches to delivering Ruby apps on the JVM:

  • Warbler, a light and simple tool for packaging .war files,
  • Trinidad, which is a JRuby wrapper for a Tomcat server, and
  • TorqueBox, an all-in-one app server that appears to be the hot new thing.

Links, links, and more links!

Talks such as these reminded me of the feeling of ease and power that Ruby gives developers, and the power that language implementors have to shape the landscape in which programmers work. They also gave me a much better appreciation for why projects like Rubinius and JRuby are essential to the Ruby world because -- not despite -- deviating from a core principle of the language.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

May 25, 2012 4:07 PM

JRubyConf, Day 1: The State of JRuby

Immediately after the keynote address, the conference really began for me. As a newcomer to JRuby, this was my first chance to hear lead developers Charles Nutter and Tom Enebo talk about the language and community. The program listed this session as "JRuby: Full of Surprises", and Nutter opened with a slide titled "Something About JRuby", but I just thought of the session as a "state of the language" report.

Nutter opened with some news. First, JRuby 1.7.0.preview1 is available. The most important part of this for me is that Ruby 1.9.3 is now the default language mode for the interpreter. I still run Ruby 1.8.7 on my Macs, because I have never really needed more and that kept my installations simple. It will be nice to have a 1.9.3 interpreter running, too, for cases where I want to try out some of the new goodness that 1.9 offers.

Second, JRuby has been awarded eight Google Summer of Code placements for 2012. This was noteworthy because there were no Ruby projects at all in 2010 or 2011, for different reasons. Several of the 2012 projects are worth paying attention to:

  • creating a code generator for Dalvik byte code, which will give native support for JRuby on Android
  • more work on Ruboto, the current way to run Ruby on Android, via Java
  • implementing JRuby fibers using Kilim fibers, for lighterweight and faster concurrency than Java threads can provide
  • work on krypt, "SSL done right" for Ruby, which will eliminate the existing dependence on OpenSSL
  • filling in some of the gaps in the graphics framework Shoes, both Swing and SWT versions

Charles Nutter discussing details of the JRuby compiler

Enebo then described several projects going on with JRuby. Some are smaller, including closing gaps in the API for embedding Ruby code in Java, and Noridoc, a tool for generating integrated Ruby documentation for Ruby and Java APIs that work together. Clever -- "No ri doc".

One project is major: work on the JRuby compiler itself. This includes improving to the intermediate representation used by JRuby, separating more cleanly the stages of the compiler, and writing a new, better run-time system. I didn't realize until this talk just how much overlap there is in the existing scanner, parser, analyzer, and code generator of JRuby. I plan to study the JRuby compiler in some detail this summer, so this talk whet my appetite. One of the big challenges facing the JRuby team is to identify execution hot spots that will allow the compiler to do a better job of caching, inlining, and optimizing byte codes.

This led naturally into Nutter's discussion of the other major project going on: JRuby support for the JVM's new invokedynamic instruction. He hailed invokedynamic as "the most important change to the JVM -- ever". Without it, JRuby's method invocation logic is opaque to the JVM optimizer, including caching and inlining. With it, the JRuby compiler will be able not only to generate optimizable function calls but also more efficient treatment of instance variables and constants.

Charles Nutter donning his new RedHat

Nutter showed some performance data comparing JRuby to MRI Ruby 1.9.3 on some standard benchmarks. Running on Java 6, JRuby is between 1.3 and 1.9 times faster than the C-based compiler on the benchmark suites. When they run it on Java 7, performance jumps to speed-ups of between 2.6 and 4.3. That kind of speed is enough to make JRuby attractive for many compute-intensive applications.

Just as Nutter opened with news, he closed with news. He and Enebo are moving to RedHat. They will work with various RedHat and JBoss teams, including TorqueBox, which I'll mention in an upcoming JRubyConf post. Nutter and Enebo have been at EngineYard for three years, following a three-year stint at Sun. It is good to know that, as the corporate climate around Java and Ruby evolves, there is usually a company willing to support open-source JRuby development.


IMAGE 1: my photo of Charles Nutter talking about some details of the JRuby compiler. License: Creative Commons Attribution-ShareAlike 3.0 Unported.

IMAGE 2: my photo of Charles Nutter and Tom Enebo announcing their move to RedHat. License: Creative Commons Attribution-ShareAlike 3.0 Unported.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

May 24, 2012 3:05 PM

JRubyConf 2012: Keynote Address on Polyglot Programming

JRubyConf opened with a keynote address on polyglot programming by Venkat Subramaniam. JRuby is itself a polyglot platform, serving as a nexus between a highly dynamic scripting language and a popular enterprise language. Java is not simply a language but an ecosphere consisting of language, virtual machine, libraries, and tools. For many programmers, the Java language is the weakest link in its own ecosphere, which is one reason we see so many people porting their favorite languages run on the JVM, or creating new languages with the JVM as a native backend.

Subramaniam began his talk by extolling the overarching benefits of being able to program in many languages. Knowing multiple programming languages changes how we design software in any language. It changes how we think about solutions. Most important, it changes how we perceive the world. This is something that monolingual programmers often do not appreciate. When we know several languages well, we see problems -- and solutions -- differently.

Why learn a new language now, even if you don't need to? So that you can learn a new language more quickly later, when you do need to. Subramaniam claimed that the amount of time required to learn a new language is inversely proportional to the number of languages a person has learned in last ten years. I'm not sure whether there is any empirical evidence to support this claim, but I agree with the sentiment. I'd offer one small refinement: The greatest benefits come from learning different kinds of language. A new language that doesn't stretch your mind won't stretch your mind.

Not everything is heaven for the polyglot programmer. Subramaniam also offered some advice for dealing with the inevitable downsides. Most notable among these was the need to "contend with the enterprise".

Many companies like to standardize on a familiar and well-established technology stack. Introducing a new language into the mix raises questions and creates resistance. Subramaniam suggested that we back up one step before trying to convince our managers to support a polyglot environment and make sure that we have convinced ourselves. If you were really convinced of a language's value, you would find a way to do it. Then, when it comes time to convince your manager, be sure to think about the issue from her perspective. Make sure that your argument speaks to management's concerns. Identify the problem. Explain the proposed solution. Outline the costs of the solution. Outline its benefits. Show how the move can be a net win for the organization.

The nouveau JVM languages begin with a head start over other new technologies because of their interoperability with the rest of the Java ecosphere. They enable you to write programs in a new language or style without having to move anyone else in the organization. You can experiment with new technology while continuing to use the rest of the organization's toolset. If the experiments succeed, managers can have hard evidence about what works well and what doesn't before making larger changes to the development environment.

I can see why Subramaniam is a popular conference speaker. He uses fun language and coins fun terms. When talking about people who are skeptical of unit testing, he referred to some processes as Jesus-driven development. He admonished programmers who are trying to do concurrency in JRuby without knowing the Java memory model, because "If you don't understand the Java memory model, you can't get concurrency right." But he followed that immediately with, Of course, even if you do know the Java memory model, you can't get concurrency right. Finally, my favorite: At one point, he talked about how some Java developers are convinced that they can do anything they need to do in Java, with no other languages. He smiled and said, I admire Java programmers. They share an unrelenting hope.

There were times, though, when I found myself wanting to defend Java. That happens to me a lot when I hear talks like this one, because so many complaints about it are really about OOP practiced poorly; Java is almost an innocent bystander. For example, the speaker chided Java programmers for suffering from primitive obsession. This made me laugh, because most Ruby programmers seem to have an unhealthy predilection for strings, hashes, and integers.

In other cases, Subramaniam demonstrated the virtues of Ruby by showing a program that required a gem and then solved a thorny problem with three lines of code. Um, I could do that in Java, too, if I used the right library. And Ruby programmers probably don't want to draw to much attention to gems and the problems many Ruby devs have with dependency management.

But those are small issues. Over the next two days, I repeatedly heard echoes of Subramaniam's address in the conference's other talks. This is the sign of a successful keynote.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

May 22, 2012 7:53 PM

A Few Days at JRubyConf

It's been fourteen months since I last attended a conference. I decided to celebrate the end of the year, the end of my compiler course, and the prospect of writing a little code this summer by attending JRubyConf 2012. I've programmed a fair amount in Ruby but have only recently begun to play with JRuby, an implementation of Ruby in Java which runs atop the JVM. There are some nice advantages to this, including the ability to use Java graphics with Ruby models and the ability to do real concurrency. It also offers me a nice combination for the summer. I will be teaching our sophomore-level intermediate computing course this fall, which focuses in large part on OO design and Java implementation, as JRuby will let me program in Ruby while doing a little class prep at the same time.

the Stone Arch Bridge in Minneapolis

Conference organizer Nick Sieger opened the event with the obligatory welcome remarks. He said that he thinks the overriding theme of JRubyConf is being a bridge. This is perhaps a natural effect of Minneapolis, a city of many bridges, as the hometown of JRuby, its lead devs, and the conference. The image above is of the Stone Arch Bridge, as seen from the ninth level of the famed Guthrie Center, the conference venue. (The yellow tint is from the window itself.)

The goal for the conference is to be a bridge connecting people to technologies. But it also aims to be a bridge among people, promoting what Sieger called "a more sensitive way of doing business". Emblematic of this goal were its Sunday workshop, a Kids CodeCamp, and its Monday workshop, Railsbridge. This is my first open-source conference, and when I look around I see the issue that so many people talk about. Of 150 or so attendees, there must be fewer than one dozen women and fewer than five African-Americans. The computing world certainly has room to make more and better connections into the world.

My next few entries will cover some of the things I learn at the conference. I start with a smile on my face, because the conference organizers gave me a cookie when I checked in this morning:

the sugar cookie JRubyConf gave me at check-in

That seems like a nice way to say 'hello' to a newcomer.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

May 14, 2012 3:26 PM

Lessons Learned from Another Iteration of the Compiler Course

I am putting the wrap on spring semester, so that I can get down to summer duties and prep for fall teaching. Here are a few lessons I learned this spring.

•  A while back, I wrote briefly about re-learning the value of a small source language for the course. If I want to add a construct or concept, then I need also to subtract a corresponding load from language. In order to include imperative features, I need to eliminate recursive functions, or perhaps eliminate functions altogether. Eliminating recursion may be sufficient, as branching to a function is not much more complex than branching in loops. It is the stack of activation records that seems to slow down most students.

•  Using a free on-line textbook worked out okay. The main problem was that this particular book contained less implementation detail than books we have used in the past, such as Louden, and that hurt the students. We used Louden's TM assembly language and simulator, and the support I gave them for that stage of the compiler in particular was insufficient. The VM and assembly language themselves are simple enough, but students wanted more detailed examples of a code generator than I gave them.

•  I need to teach code generation better. I felt that way as the end of the course approached, and then several students suggested as much in our final session review. This is the most salient lesson I take from this iteration of the course.

I'm not sure at this moment if I need only to do a better job of explaining the process or if I need a different approach to the task more generally. That's something I'll need to think about between now and next time. I do think that I need to show them how to implement function calls in a bit more detail. Perhaps we could spend more time in class with statically-allocated activation records, and then let the students extend those ideas for a run-time stack and recursion.

•  For the first time ever, a few students suggested that I require something simpler than a table-driven parser. Of course, I can address several issues with parsing and code generation by using scaffolding: parser generators, code-generation frameworks and the like. But I still prefer that students write a compiler from scratch, even if only a modest one. There is something powerful in making something from scratch. A table-driven parser is a nice blend of simplicity (in algorithm) and complexity (in data) for learning how compilers really work.

I realize that I have to draw the abstraction line somewhere, and even after several offerings of the course I'm ready to draw it there. To make that work as well as possible, I may have to improve parts of the course to make it work better.

•  Another student suggestion that seems spot-on is that, as we learn each stage of the compiler, we take some time to focus on specific design decisions that the teams will have to make. This will alway them, as they said in their write-ups, "to make informed decisions". I do try to introduce key implementation decisions that they face and offer advice on how to proceed. Clearly I can do better. One way, I think, is to connect more directly with the programming styles they are working in.


As usual, the students recognized some of the same shortcomings of the course that I noticed and suggested a couple more that had not occurred to me. I'm always glad I ask for their feedback, both open and anonymous. They are an indispensable source of information about the course.

Writing your first compiler is a big challenge. I can't help but recall something writer Samuel Delany said when asked if he "if it was fun" to write a set of novellas on the order of Eliot's The Waste Land, Pound's The Cantos, and Joyce's Ulysses:

No, not at all. ... But at least now, when somebody asks, "I wonder if Joyce could have done all the things he's supposed to have done in Ulysses," I can answer, "Yes, he could have. I know, because I tried it myself. It's possible."

Whatever other virtues there are in learning to write a compiler, it is valuable for computer science students to take on big challenges and know that it is possible to meet the challenge, because they have tried it themselves.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

May 01, 2012 8:56 AM

The Pleasure in a Good Name

Guile is a Scheme. It began life as GEL, which stood for GNU Extension Language. This brief history of Guile explains the change:

Due to a naming conflict with another programming language, Jim Blandy suggested a new name for GEL: "Guile". Besides being a recursive acronym, "Guile" craftily follows the naming of its ancestors, "Planner", "Conniver", and "Schemer". (The latter was truncated to "Scheme" due to a 6-character file name limit on an old operating system.) Finally, "Guile" suggests "guy-ell", or "Guy L. Steele", who, together with Gerald Sussman, originally discovered Scheme.

That is how you name a language, making it significant on at least three levels. Recursive acronyms are a staple of the GNU world, beginning with GNU itself. Guile recurses as the GNU Ubiquitous Intelligent Language for Extensions. Synonymny with Planner and Conniver keeps alive the historical connection to artificial intelligence, and is reinforced by the use of intelligent in the acronym. Finally, the homophonic connection to Steele is pure genius.

I bow to you, Mr. Blandy.

(While we are talking about words, I must say that I love the author's use of discovered in the passage quoted above. Most people say that Steele and Sussman "created" Scheme, or "designed" it, or "invented" it. But if you read Steele's and Gabriel's The Evolution of Lisp, you will see that discovery is a better label for what happened. Scheme was lurking in the implementation of Lisp's apply primitive and Carl Hewitt's theory of actors. Hewitt, of course, created Planner, which is another connection back to Guile.)

Posted by Eugene Wallingford | Permalink | Categories: Computing

April 24, 2012 4:55 PM

Recursive Discussions about Recursion

The SIGCSE listserv has erupted today with its seemingly annual discussion of teaching recursion. I wrote about one of the previous discussions a couple of years ago. This year's conversation has actually included a couple of nice ideas, so it was worth following along.

Along the way, one prof commented on an approach he has seen used to introduce students to recursion, often in a data structures course. First you cover factorial, gcd, and the Fibonacci sequence. Then you cover the Towers of Hanoi and binary search. Unfortunately, such an approach is all too common. The poster's wistful analysis:

Of the five problems, only one (binary search) is a problem students might actually want to solve. Only two (Fibonacci and Hanoi) are substantially clearer in recursive than iterative form, and both of them take exponential time. In other words, recursion is a confusing way to solve problems you don't care about, extremely slowly.

Which, frankly, I think is the lesson [some CS profs] want to convey.

And this on a day when I talked with my compiler students about how a compiler can transform many recursive programs into iterative ones, and even eliminate the cost of a non-recursive function call when it is in a tail position.

The quoted passage contains my favorite line of the week thus far: In other words, recursion is a confusing way to solve problems you don't care about, extremely slowly. If that's not the message you want to convey to your students, then please don't introduce them to recursion in this way. If that is the message you want to send to your students, then I am sad for you, but mostly for your students.

I sometimes wonder about the experiences that some computer scientists bring to the classroom. It only takes a little language processing to grok the value of recursion. And a data structures course is a perfectly good place for students to see and do a little language processing. Syntax, abstract or not, is a great computer science example of trees. And students get to learn a little more CS to boot!

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

April 15, 2012 8:10 PM

Learning From the Wheel of Reincarnation

Last week I was pointed in the direction of a forty-five old CACM paper. It had an innocuous title, On the Design of Display Processors, and was outside my areas of primary interest, so I might well have passed it by. But it was co-written by Ivan Sutherland, whose work Alan Kay has praised often, and it was recommended by someone on Kay's the Fundamentals of New Computing mailing list, where a lot of neat ideas are discussed. So I printed it out for my daily exercise bike ride. I'm glad I did.

Myer and Sutherland tell the story of needing a display system for a research computer. That was a bigger deal in 1967 than it is today, so they did some research of their own....

Finally we decided to design the processor ourselves, because only in this way, we thought, could we obtain a truly complete display processor. We approached the task by starting with a simple scheme and adding commands and features that we felt would enhance the power of the machine. Gradually the processor became more complex. We were not disturbed by this because computer graphics, after all, are complex. Finally the display processor came to resemble a full-fledged computer with some special graphics features. And then a strange thing happened. We felt compelled to add to the processor a second, subsidiary processor, which, itself, began to grow in complexity.

It was then that we discovered a disturbing truth. Designing a display processor can become a never-ending cyclical process. In fact, we found the process so frustrating that we have come to call it the "wheel of reincarnation." We spent a long time trapped on that wheel before we finally broke free. In the remainder of this paper we describe our experiences. We have written it in the hope that it may speed others on toward "Nirvana."

A mantra from the paper characterizes the authors' time on the wheel: "For just a little more money...". I'll bet that sounds familiar to a lot of researchers, not to mention all of us who buy computing equipment for labs and end users.

I was really happy to read this paper. It's an experience report in which the authors share honestly the mistakes they made. But they paid attention, recognized a pattern, and learned from it. Even better, they wrote what they learned, in hopes of teaching the rest of us.

The wheel of reincarnation is not limited to display design or hardware design. It occurs any place where we encounter complexity. We try to tame it, first by specializing and then by generalizing. The design of programming languages is just as prone to dizzying cycle.

(In software design, we have a related phenomenon, captured in Greenspun's Tenth Rule.)

In language design, we almost have to look for a fixed point at which we stabilize the pendulum between general and specialized. What we most often need as users is the ability to grow systems gracefully over time. This speaks to the value of a good macro system and good compiler design.

Reading this paper reminded me of a couple of lessons I've learned over the years:

  1. I should read and watch everything I can get my hands on from Sutherland, Robert Floyd, and other computer scientists of their generation. They were solving hard problems long ago, in the face of resource limitations few of us can imagine today.
  2. As someone tweeted recently, @fogus, I think: reading these old papers makes me think that I will never have an original idea in your life. But then they also teach me a lot and prepare me to have more and sometimes better ideas.
  3. I need to do my best to hang around with smart, curious people. It's old advice, I know, but it requires action and, in some environments, constant vigilance. Simply eavesdropping on the FONC mailing list raises the level of my brain's activity by a few degrees.

These papers also remind us of a valuable role that academics can play in the software and computing worlds, which are also heavily influenced by industry practitioners. We need to keep papers like these alive, so that the smart and curious people in our classes and in industry will read them. We never know when two ideas will crash into each other and lead to something new and better.

Posted by Eugene Wallingford | Permalink | Categories: Computing

April 11, 2012 4:06 PM

What Penn and Teller Have in Common With a Compilers Course

Early this morning (and I mean early!), Alfred Thompson posted What Do Magic and Computer Science Have in Common?, relaying that Alex Suter of Industrial Light & Magic will give the closing keynote at this summer's Computer Science and Information Technology conference. That sounds pretty cool. The title of his entry conjured up other thoughts for me, though, especially in light of something I said in class yesterday.

Recently, I used a superhero reference in a blog entry. That is how many people feel when they use a program to accomplish something meaningful -- like a superhero. I feel that way, too, sometimes. However, like many other people, I am more prone to magical imagery. To someone who has not learned to code, a program is like an incantation, capable of making the computer do something mystical.

There is a long tradition of magical imagery in computer science. The Hacker's Dictionary tells us that a wizard is someone who knows how a complex piece of software or hardware works (that is, who groks it). A wizard can do things that hackers and mere mortals cannot. The entry for "wizard" has links to other magical jargon, such as heavy wizardry incantation>/A> and magic itself.

Structure and Interpretation of Computer Programs

I tell my Programming Languages students that this is a course in which the "high priests" of computer science reveal their secrets, that after the course students will understand the magic embodied in the interpreters and compilers that process their programs. I should probably refer to wizards, rather high priests, given that so many of the course's ideas are covered masterfully in SICP.

Lots of CS courses reveal magic, or propose it. A course in compliers finishes the job of Programming Languages, driving program translation all the way down to hardware. Artificial Intelligence describes our best ideas for how to make a computer do human-like magic: reasoning, recognizing patterns, learning, and playing Jeopardy!.

In my compliers class yesterday, I was showing my students a technique for generating three-address code from an abstract syntax tree, based in large part on ideas found in the Dragon book (surely not a coincidence). I wrote on the board this template for a grammar rule denoting addition:

    E → E1 + E2 := makeNewTemporaryIdentifier() E.code := [ E1.code ] [ E2.code ] emitCode( " := " " + " )

When I finished, I stepped back, looked it over, and realized again just how un-magical that seems. Indeed, when in written in black on white, it looks pretty pedestrian.

the magician due of Penn and Teller

That made me think of another connection between magic and computer science, one that applies to practitioners and outsiders alike. Taking an AI course or a compilers course is like having Penn and Teller explain to you how they made a person disappear or a ball levitate in thin air. For some people, that kills any joy that might have in watching the act. They don't want to know. They want to be amazed. And, knowing that something is implemented -- often in a way that doesn't seem especially artful, performed with an obvious misdirection -- prevents them from being amazed.

That can happen in CS, too. My friends and I came in to our AI course wide-eyed and wanting to be amazed -- and to build amazing things. We studied search and logical resolution, Bayes' Theorem and decision tree induction. And it all looked so... pedestrian. Many of my friends lost their sense of wonder then and there. Without the magic of AI, they were just as interested in operating systems or databases. More interested, really, because AI had let them down. It all looked like parlor tricks.

But there is a second kind of person in the world. Lots of people love to watch Penn and Teller explain a trick. They want to know how it works. They want to watch again, knowing how it works, to see if they can notice the deception. If they don't, they are amazed again. If they do, though, they still feel wonder -- at the skill of the magicians, at the design of the illusion, and even at the way their mind wants to trick them at the moment of execution.

I am of this second kind, for magic and especially for computer science. Yes, I know that compilers and AI programs are, at their cores, implemented using techniques that don't always look all that impressive in the light of day. Sometimes, those techniques look pretty boring indeed.

Yet I am still amazed when a C compiler takes in my humble instructions and creates machine code that compresses a musical file or fills out my tax return. I am amazed when Watson crunches through gazillions of bytes of data in a second or two and beats two most worthy human opponents to the buzzer. I like to hear my friends and colleagues de-mystify their current projects with nitty-gritty details. I still feel wonder -- at their skill, at the cleverness of their designs, and even at the moment the program runs and makes something out of what seems like nothing.

That's one thing magic and computer science have in common.


IMAGE 1: a photo of the cover of Structure and Interpretation of Computer Programs. Source: the book's website.

IMAGE 2: a publicity still of magicians Penn and Teller. Source:

Posted by Eugene Wallingford | Permalink | Categories: Computing

April 06, 2012 4:29 PM

A Reflection on Alan Turing, Representation, and Universal Machines

Douglas Hofstadter speaking at UNI

The day after Douglas Hofstadter spoke here on assertions, proof's and Gödel's theorem, he gave a second public lecture hosted by the philosophy department. Ahead of time, we knew only that Hofstadter would reflect on Turing during his centennial. I went in expecting more on the Turing test, or perhaps a popular talk on Turing's proof of The Halting Problem. Instead, he riffed on Chapter 17 from I Am a Strange Loop.

In the end, we are self-perceiving, self-inventing, locked-in mirages that are little miracles of self-reference.

Turing, he said, is another peak in the landscape occupied by Tarski and Gödel, whose work he had discussed the night before. (As a computer scientist, I wanted to add to this set contemporaries such as Alonzo Church and Claude Shannon.) Hofstadter mentioned Turing's seminal paper about the Entscheidungsproblem but wanted to focus instead on the model of computation for which he is known, usually referred to by the name "Turing machine". In particular, he asked us to consider a key distinction that Turing made when talking about his model: that between dedicated and universal machines.

A dedicated machine performs one task. Human history is replete with dedicated machines, whether simple, like the wheel, or complex, such as a typewriter. We can use these tools with different ends in mind, but the basic work is fixed in their substance and structure.

The 21st-century cell phone is, in contrast, a universal machine. It can take pictures, record audio, and -- yes -- even be used as a phone. But it can also do other things for us, if we but go to the app store and download another program.

Hofstadter shared a few of his early personal experiences with programs enabling line printers to perform tasks for which they had not been specifically designed. He recalled seeing a two-dimensional graph plotted by "printing" mostly blank lines that contained a single *. Text had been turned into graphics. Taking the idea further, someone used the computer to print a large number of cards which, when given to members of the crowd at a football game, could be used to create a massive two-dimensional message visible from afar. Even further, someone used a very specific layout of the characters available on the line printer to produce a print-out that appeared from the other side of the room to be a black-and-white photograph of Raquel Welch. Text had been turned into image.

People saw each of these displays as images by virtue of our eyes and mind interpreting a specific configuration of characters in a certain way. We can take that idea down a level into the computer itself. Consider this transformation of bits:

0000 0000 0110 1011 → 0110 1011 0000 0000

A computer engineer might see this as a "left shift" of 8 bits. A computer programmer might see it as multiplying the number on the left by 256. A graphic designer might see us moving color from one pixel to another. A typesetter may see one letter being changed into another. What one sees depends on how one interprets what the data represent and what the process means.

Alan Turing was the first to express clearly the idea that a machine can do them all.

"Aren't those really binary numbers?", someone asked. "Isn't that real, and everything else interpretation?" Hofstadter said that this is a tempting perspective, but we need to keep in mind that they aren't numbers at all. They are, in most computers, pulses of electricity, or the states of electronic components, that we interpret as 0s and 1s.

After we have settled on interpreting those pulses or states as 0s and 1s, we then interpret configurations of 0s and 1s to mean something else, such as decimal numbers, colors, or characters. This second level of interpretation exposes the flaw in popular claims that computers can do "only" process 0s and 1s. Computers can deal with numbers, colors, or characters -- anything that can be represented in any way -- when we interpret not only what the data mean but also what the process means.

(In the course of talking representations, he threw in a cool numeric example: Given an integer N, factor it as 2^a * 3^b * 5^c *7^d ... and use [a.b.c.d. ...] to stand for N. I see a programming assignment or two lying in wait.)

The dual ideas of representation and interpretation take us into a new dimension. The Principia Mathematica describes a set of axioms and formal rules for reasoning about numeric structures. Gödel saw that it could be viewed at a higher level, as a system in its own right -- as a structure of integers. Thus the Principia can talk about itself. It is, in a sense, universal.

This is the launching point for Turing's greatest insight. In I Am a Strange Loop, Hofstadter writes:

Inspired by Gödel's mapping of PM into itself, Alan Turing realized that the critical threshold for this kind of computational universality comes exactly at the point where a machine is flexible enough to read and correctly interpret a set of data that describes its own structure. At this crucial juncture, a machine can, in principle, explicitly watch how it does any particular task, step by step. Turing realized that a machine that has this critical level of flexibility can imitate any other machine, no matter how complex the latter is. In other words, there is nothing more flexible than a universal machine. Universality is as far as you can go!

Alan Turing

Thus was Turing first person to recognize the idea of a universal machine, circa 1935-1936: that a Turing machine can be given, as input, data that encodes its own instructions. This is the beginning of perhaps the biggest of the Big Ideas of computer science: the duality of data and program.

We should all be glad he didn't patent this idea.

Turing didn't stop there, of course, as I wrote in my recent entry on the Turing test. He recognized that humans are remarkably capable and efficient representational machines.

Hofstadter illustrates this with the idea of "hub", a three-letter word that embodies an enormous amount of experience and knowledge, chunked in numerous ways and accreted slowly over time. The concept is assembled in our minds out of our experiences. It is a representation. Bound up in that representation is an understanding of ourselves as actors in certain kinds of interactions, such as booking a flight on an airplane.

It is this facility with representations that distinguishes us humans from dogs and other animals. They don't seem capable of seeing themselves or others as representations. Human beings, though, naturally take other people's representations into their own. This results in a range of familiarities and verisimilitude. We "absorb" some people so well that we feel we know them intimately. This is what we mean when we say that someone is "in our soul". We use the word 'soul' not in a religious sense; we are referring to our essence.

Viewed this way, we are all distributed beings. We are "out there", in other people, as well as "in here", in ourselves. We've all had dreams of the sort Hofstadter used as example, a dream in which his deceased father appeared, seemingly as real as he ever had been while alive. I myself recently dreamt that I was running, and the experience of myself was as real as anything I feel when I'm awake. Because we are universal machines, we are able to process the representations we hold of ourselves and of others and create sensations that feel just like the ones we have when we interact in the world.

It is this sense that we are self-representation machines that gives rise to the title of his book, "I am a strange loop". In Hofstadter's view, our identity is a representation of self that we construct, like any other representation.

This idea underlies the importance of the Turing test. It takes more than "just syntax" to pass the test. Indeed, syntax is itself more than "just" syntax! We quickly recurse into the dimension of representation, of models, and a need for self-reference that makes our syntactic rules more than "just" rules.

Indeed, as self-representation machines, we are able to have a sense of our own smallness within the larger system. This can be scary, but also good. It makes life seem precious, so we feel a need to contribute to the world, to matter somehow.

Whenever I teach our AI course, I encounter students who are, for religious or philosophical reasons, deeply averse to the idea of an intelligent machine, or even of scientific explanations of who we are. When I think about identity in terms of self-representation, I can't help but feel that, at an important level, it does not matter. God or not, I am in awe of who we are and how we got to here.

So, we owe Alan Turing a great debt. Building on the work of philosophers, mathematicians, and logicians, Turing gave us the essential insight of the universal machine, on which modern computing is built. He also gave us a new vocabulary with which to think about our identity and how we understand the world. I hope you can appreciate why celebrating his centennial is worthwhile.


IMAGE 1: a photo of Douglas Hofstadter speaking at UNI, March 7, 2012. Source: Kevin C. O'Kane.

IMAGE 2: the Alan Turing centenary celebration. Source: 2012 The Alan Turing Year.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

April 04, 2012 4:39 PM

Computational Search Answers an Important Question

Update: Well, this is embarrassing. Apparently, Mat and I were the victims of a prank by the folks at ChessBase. You'd think that, after more than twenty-five years on the internet, I would be more circumspect at this time of year. Rather than delete the post, I will leave it here for the sake of posterity. If nothing else, my students can get a chuckle from their professor getting caught red-faced.

I stand behind my discussion of solving games, my recommendation of Rybka, and my praise for My 60 Memorable Games (my favorite chess book of all time. I also still marvel at the chess mind of Bobby Fischer.


Thanks to reader Mat Roberts for pointing me to this interview with programmer Vasik Rajlich, which describes a recent computational result of his: one of the most famous openings in chess, the King's Gambit, is a forced draw.

Games are, of course, a fertile testbed for computing research, including AI and parallel computation. Many researchers make one of their goals to "solve" a game, that is, to show that, with best play by both players, a game has a particular outcome. Games with long histories and large communities of players naturally attract a lot of interest, and solving one of them is usually considered a valuable achievement.

For us in CS, interest grows as with the complexity of the game. Solving Connect Four was cool, but solving Othello on a full-sized board would be cooler. Almost five years ago, I blogged about what I still consider the most impressive result in this domain: the solving of checkers by Jonathan Schaeffer and his team at the University of Alberta.

the King's Gambit

The chess result is more limited. Rajlich, an International Master of chess and the programmer of the chess engine Rybka, has shown results only for games that begin 1.e4 e5 2.f4 exf4. If White plays 3.Nf3 -- the most common next move -- then Black can win with 3... d6. 3.Bc4 also loses. Only one move for White can force a draw, the uncommon 3.Be2. Keep in mind that these results all assume best play by both players from there on out. White can win, lose, or draw in all variations if either player plays a sub-optimal move.

I say "only" when describing this result because it leaves a lot of chess unsolved, all games starting with some other sequence of moves. Yet the accomplishment is still quite impressive! The King's Gambit is one of the oldest and most storied opening sequences in all of chess, and it remains popular to this day among players at every level of skill.

Besides, consider the computational resources that Rajlich had to use to solve even the King's Gambit:

... a cluster of computers, currently around 300 cores [created by Lukas Cimiotti, hooked up to] a massively parallel cluster of IBM POWER 7 Servers provided by David Slate, senior manager of IBM's Semantic Analysis and Integration department -- 2,880 cores at 4.25 GHz, 16 terabytes of RAM, very similar to the hardware used by IBM's Watson in winning the TV show "Jeopardy". The IBM servers ran a port of the latest version of Rybka, and computation was split across the two clusters, with the Cimiotti cluster distributing the search to the IBM hardware.

Oh, and this set up had to run for over four months to solve the opening. I call that impressive. If you want something less computationally intensive yet still able to beat you me and everybody we know at chess, you can by Rybka, a chess engine available commercially. (An older version is available for free!)

What effect will this result have on human play? Not much, practically speaking. Our brains aren't big enough or fast enough to compute all the possible paths, so human players will continue to play the opening, create new ideas, and explore the action in real time over the board. Maybe players with the Black pieces will be more likely to play one of the known winning moves now, but results will remain uneven between White and Black. The opening leads to complicated positions.

the cover of Bobby Fischer's 'My 60 Memorable Games'

If, like some people, you worry that results such as this one somehow diminish us as human beings, take a look again at the computational resources that were required to solve this sliver of one game, the merest sliver of human life, and then consider: This is not the first time that someone claimed the King's Gambit busted. In 1961, an eighteen-year-old U.S. chess champion named Bobby Fischer published an article claiming that 1.e4 e5 2.f4 exf4 3.Nf3 was a forced loss. His prescription? 3... d6. Now we know for sure. Like so many advances in AI, this one leaves me marveling at the power of the human mind.

Well, at least Bobby Fischer's mind.


IMAGE 1: The King's Gambit. Source: Wikimedia Commons.

IMAGE 2: a photograph of the cover of my copy of My 60 Memorable Games by Bobby Fischer. Bobby analyzes a King's Gambit or two in this classic collection of games.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

April 03, 2012 4:00 PM

Intermediate Representations and Life Beyond the Compiler

In the simplest cases, a compiler can generate target code directly from the abstract syntax tree:

generating target code directly from the abstract syntax tree

In many cases, though, there are good reasons why we don't want to generate code for the target machine immediately. One is modularity. A big part of code generation for a particular target machine is machine-dependent. If we write a monolithic code generator, then we will have to reimplement the machine-independent parts every time we want to target a new machine.

Even if we stick with one back-end architecture, modularity helps us. Not all of the machine-dependent elements of code generation depend in the same way on the machine. If we write a monolithic code generator, then any small change in the target machine -- perhaps even a small upgrade in the processor -- can cause changes throughout our program. If instead we write a modular code generator, with modules that reflect particular shear layers in the generation process, a lá How Buildings Learn, then we may be able to contain changes in target machine specification to an easily identified subset of modules.

So, more generally we think of code generation in two parts:

  • one or more machine-independent transformations from an abstract syntax tree to intermediate representations of the program, followed by

  • one or more machine-dependent transformations from the final intermediate representation to machine code.

generating target code directly from the abstract syntax tree

Intermediate representations between the abstract syntax tree and assembly code have other advantages, too. In particular, they enable us to optimize code in machine-independent ways, without having to manipulate a complex target language.

In practice, an intermediate representation sometimes outlives the compiler for which it was created. Chris Clark described an example of this phenomenon in Build a Tree--Save a Parse:

Sometimes the intermediate language (IL) takes on a life of its own. Several systems that have multiple parsers, sophisticated optimizers, and multiple code generators have been developed and marketed commercially. Each of these systems has its own common virtual assembly language used by the various parsers and code generators. These intermediate languages all began connecting just one parser to one code generator.

P-code is an example IL that took on a life of its own. It was invented by Nicklaus Wirth as the IL for the ETH Pascal compiler. Many variants of that compiler arose [Ne179], including the USCD Pascal compiler that was used at Stanford to define an optimizer [Cho83]. Chow's compiler evolved into the MIPS compiler suite, which was the basis for one of the DEC C compilers -- acc. That compiler did not parse the same language nor use any code from the ETH compiler, but the IL survived.

Good language design usually pays off, sometimes in unexpected ways.

(If you like creating languages and writing language processors, Clark's paper is worth a read!)

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

March 30, 2012 5:22 PM

A Reflection on Alan Turing, the Turing Test, and Machine Intelligence

Alan Turing

In 1950, Alan Turing published a paper that launched the discipline of artificial intelligence, Computing Machinery and Intelligence. If you have not read this paper, go and do so. Now. 2012 is the centennial of Turing's birth, and you owe yourself a read of this seminal paper as part of the celebration. It is a wonderful work from a wonderful mind.

This paper gave us the Imitation Game, an attempt to replace the question of whether a computer could be intelligent by withn something more concrete: a probing dialogue. The Imitation became the Turing Test, now a staple of modern culture and the inspiration for contests and analogies and speculation. After reading the paper, you will understand something that many people do not: Turing is not describing a way for us to tell the difference between human intelligence and machine intelligence. He is telling us that the distinction is not as important as we seem to think. Indeed, I think he is telling us that there is no distinction at all.

I mentioned in an entry a few years ago that I always have my undergrad AI students read Turing's paper and discuss the implications of what we now call the Turing Test. Students would often get hung up on religious objections or, as noted in that entry, a deep and a-rational belief in "gut instinct". A few ended up putting their heads in the sand, as Turing knew they might, because they simply didn't want to confront the implication of intelligences other than our own. And yet they were in an AI course, learning techniques that enable us to write "intelligent" programs. Even students with the most diehard objections wanted to write programs that could learn from experience.

Douglas Hofstadter, who visited campus this month, has encountered another response to the Turing Test that surprised him. On his second day here, in honor of the Turing centenary, Hofstadter offered a seminar on some ideas related to the Turing Test. He quoted two snippets of hypothetical man-machoine dialogue from Turing's seminal paper in his classic Gödel, Escher, Bach. Over the the years, he occasionally runs into philosophers who think the Turing Test is shallow, trivial to pass with trickery and "mere syntax". Some are concerned that it explores "only behavior". Is behavior all there is? they ask.

As a computer programmer, the idea that the Turing test explores only behavior never bothered me. Certainly, a computer program is a static construct and, however complex it is, we can read and understand it. (Students who take my programming languages course learn that even another program can read and process programs in a helpful way.) This was not a problem for Hofstadter either, growing up as he did in a physicist's household. Indeed, he found Turing's formulation of the Imitation Game to be deep and brilliant. Many of us who are drawn to AI feel the same. "If I could write a program capable of playing the Imitation Game," we think, "I will have done something remarkable."

One of Hofstadter's primary goals in writing GEB was to make a compelling case form Turing's vision.

Douglas Hofstadter

Those of us who attended the Turing seminar read a section from Chapter 13 of Le Ton beau de Marot, a more recent book by Hofstadter in which he explores many of the same ideas about words, concepts, meaning, and machine intelligence as GEB, in the context of translating text from one language to another. Hofstadter said the focus in this book is on the subtlety of words and the ideas they embody, and what that means for translation. Of course, these are the some of the issues that underlie Turing's use of dialogue as sufficient for us to understand what it means to be intelligent.

In the seminar, he shared with us some of his efforts to translate a modern French poem into faithful English. His source poem had itself been translated from older French into modern French by a French poet friend of his. I enjoyed hearing him talk about "the forces" that pushed him toward and away from particular words and phrases. Le Ton beau de Marot uses creative dialogues of the sort seen in GEB, this time between the Ace Mechanical Translator (his fictional computer program) and a Dull Rigid Human. Notice the initials of his raconteurs! They are an homage to Turing. The human translator, Douglas R. Hofstadter himself, is cast in the role of AMT, which shares its initials with Alan M. Turing, the man who started this conversation over sixty years ago.

Like Hofstadter, I have often encountered people who object to the Turing test. Many of my AI colleagues are comfortable with a behavioral test for intelligence but dislike that Turing considers only linguistic behavior. I am comfortable with linguistic behavior because it captures what is for me the most important feature of intelligence: the ability to express and discuss ideas.

Others object that it sets too low a bar for AI, because it is agnostic on method. What if a program "passes the test", and when we look inside the box we don't understand what we see? Or worse, we do understand what we see and are unimpressed? I think that this is beside the point. Not to say that we shouldn't want to understand. If we found such I program, I think that we would make it an overriding goal to figure out how it works. But how an entity manages to be "intelligent" is a different question from whether it is intelligent. That is precisely Turing's point!

I agree with Brian Christian, who won the prize for being "The Most Human Human" in a competition based on Turing's now-famous test. In an interview with The Paris Review, he said,

Some see the history of AI as a dehumanizing narrative; I see it as much the reverse.

Turing does not diminish what it is to be human when he suggests that a computer might be able to carry on a rich conversation about something meaningful. Neither do AI researchers or teenagers like me, who dreamed of figuring just what it is that makes it possible for humans to do what we do. We ask the question precisely because we are amazed. Christian again:

We build these things in our own image, leveraging all the understanding of ourselves we have, and then we get to see where they fall short. That gap always has something new to teach us about who we are.

As in science itself, every time we push back the curtain, we find another layer of amazement -- and more questions.

I agree with Hofstadter. If a computer could do what it does in Turing's dialogues, then no one could rightly say that it wasn't "intelligent", whatever that might mean. Turing was right.


PHOTOGRAPH 1: the Alan Turing centenary celebration. Source: 2012 The Alan Turing Year.

PHOTOGRAPH 2: Douglas Hofstadter in Bologna, Italy, 2002. Source: Wikimedia Commons.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

March 09, 2012 3:33 PM

This and That from Douglas Hofstadter's Visit

Update: In the original, I conflated two quotes in
"Food and Hygiene". I have un-conflated them.

In addition to his lecture on Gödel's incompleteness theorem, Douglas Hofstadter spent a second day on campus, leading a seminar and giving another public talk. I'll blog on those soon. In the meantime, here are a few random stories I heard and impressions I formed over the two days.

The Value of Good Names.   Hofstadter told a story about his "favorite chapter on Galois theory" (don't we all have one?), from a classic book that all the mathematicians in the room recognized. The only thing Hofstadter didn't like about this chapter was that it referred to theorems by number, and he could never remember which theorem was which. That made an otherwise good text harder to follow than it needed to be.

In contrast, he said, was a book by Galyan that gave each theorem a name, a short phrase evocative of what the theorem meant. So much better for reader! So he gave his students one semester an exercise to make his favorite chapter better: they were to give each of the numbered theorems in the chapter an evocative name.

This story made me think of my favorite AI textbook, Patrick Henry Winston's Artificial Intelligence. Winston's book stands out from the other AI books as quirky. He uses his own vocabulary and teaches topics very much in the MIT AI fashion. But he also gives evocative names to many of the big ideas he wants us to learn, among them the representation principle, the principle of least commitment, the diversity principle, and the eponymous "Winston's principle of parallel evolution". My favorite of all is the convergent intelligence principle:

The world manifests constraints and regularities. If a computer is to exhibit intelligence, it must exploit those constraints and regularities, no matter of what the computer happens to be made.

To me, that is AI.

Food and Hygiene.   The propensity of mathematicians to make their work harder for other people to understand, even other mathematicians, reminded Doug Shaw of two passages, from famed mathematicians Gian-Carlo Rota and André Weil. Rota said that we must guard ... against confusing the presentation of mathematics with the content of mathematics. More colorfully, Weil cautioned [If] logic is the hygiene of the mathematician, it is not his source of food. Theorems, proofs, and Greek symbols are mathematical hygiene. Pictures, problems, and understanding are food.

A Good Gig, If You Can Get It.   Hofstadter holds a university-level appointment at Indiana, and his research on human thought and the fluidity of concepts is wide enough to include everything under the sun. Last semester, he taught a course on The Catcher in the Rye. He and his students read the book aloud and discussed what makes it great. Very cool.

If You Need a Research Project...   At some time in the past, Hofstadter read, in a book or article about translating natural language into formal logic, that 'but' is simply a trivial alternative to 'and' and so can be represented as such. "Nonsense", he said! 'but' embodies all the complexity of human thought. "If we could write a program that could use 'but' correctly, we would have accomplished something impressive."

Dissatisfied.   Hofstadter uses that word a lot in conversation, or words like it, such as 'unsatisfying'. He does not express the sentiment in a whiny way. He says it in a curious way. His tone always indicates a desire to understand something better, to go deeper to the core of the question. That's a sign of a good researcher and a deep thinker.


Let's just say that this was a great treat. Thanks to Dr. Hofstadter for sharing so much time with us here.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Teaching and Learning

March 07, 2012 5:35 PM

Douglas Hofstadter on Questions, Proofs, and Passion

In the spring of my sophomore year in college, I was chatting with the head of the Honors College at my alma mater. His son, a fellow CS major, had recently read what he considered a must-read book for every thinking computer scientist. I went over to library and checked it out in hardcopy. I thumbed through it, and it was love at first sight. So I bought the paperback and spent my summer studying it, line by line.

the cover of Godel, Escher, Bach

Gödel, Escher, Bach seemed to embody everything that excited me about computer science and artificial intelligence. It made, used, and deconstructed analogies. It talked about programming languages, and computer programs as models. Though I allowed myself to be seduced in grad school by other kinds of AI, I never felt completely satisfied. My mind and heart have never really left go of the feeling I had that summer.

Last night, I had the pleasure of seeing Douglas Hofstadter give the annual Hari Shankar Memorial Lecture here. This lecture series celebrates the beauty of mathematics and its accessibility to everyone. Hofstadter said that he was happy to honored to be asked to give such a public lecture, speaking primarily to non-mathematicians. Math is real; it is in the world. It's important, he said, to talk about it in ways that are accessible to all. His lecture would share the beauty of Gödel's Incompleteness Theorem. Rather than give a dry lecture, he told the story as his story, putting the important issues and questions into the context of his own life in math.

As a 14-year-old, he discovered a paperback copy of Gödel's Proof in a used bookstore. His father mentioned that one of the authors, Ernest Nagel, was one of his teachers and friends. Douglas was immediately fascinated. Gödel used a tool (mathematics) to study itself. It was "a strange loop".

As a child, he figured out that two twos is four. The natural next question is, "What is three threes?" But this left him dissatisfied, because two was still lurking in the question. What is "three three threes"? It wasn't even clear to him what that might mean.

But he was asking questions about patterns and and seeking answers. He was on his way to being a mathematician. How did he find answers? He understood science to be about experiments, so he looked for answers by examining a whole bunch of cases, until he had seen enough to convince himself that a claim was true.

He did not know yet what a proof was. There are, of course, many different senses of proof, including informal arguments and geometric demonstrations. Mathematicians use these, but they are not what they mean by 'proof'.

Douglas Hofstadter

So he explored problems and tried to find answers, and eventually he tried to prove his answers right. He became passionate about math. He was excited by every new discovery. (Pi!) In retrospect, his excitement does not surprise him. It took mathematicians hundreds of years to create and discover these new ideas. When he learned about them after the fact, they look like magic.

Hofstadter played with numbers. Squares. Triangular numbers. Primes. He noticed that 2^3 and 3^2 adjacent to one another and wondered if any other powers were adjacent.

Mathematicians have faith that there is an answer to questions like that. It may be 'yes', it may be 'no', but there's an answer. He said this belief is so integral to the mindset that he calls this the Mathematician's Credo:

If something is true, it has a proof, and if something has a proof, then it is true.

As an example, he wrote the beginning of the Fibonacci series on the chalk board: 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233. The list contains some powers of integers: 1, 8, and 144. Are there more squares? Are there more powers? Are there infinitely many? How often do they appear? It turns out that someone recently discovered that there are no more integer powers in the list. A mathematician may be surprised that this is true, but she would not be surprised that, if it is true, there is a proof of it.

Then he gave an open question as an example. Consider this variation of a familiar problem:

  • Rule 1: n → 2n
  • Rule 2: 3n+1 → n
  • Start with 1.

Rule 1 takes us from 1 to 2. Rule 1 takes us to 4. Rule 2 takes us to 1. We've already been there, so that's not very interesting. But Rule 1 takes us to 8.... And so on.

Hofstadter called the numbers we visited C-numbers. He then asked a question: Where can we go using these rules? Can we visit every integer? Which numbers are C-numbers? Are all integers C-numbers?

The answer is, we don't know. People have used computers to test all the integers up to a very large number (20 x 2^58) and found that we can reach every one of them from 1. So many people conjecture strongly that all integers are C-numbers. But we don't have a proof, so the purist mathematician will say only, "We don't know".

At this point in the talk, my mind wanders.... (Wonders?) It would be fun to write a program to answer, "Is n a C-number?" in Flair, the language for which my students this semester are writing a compiler. That would make a nice test program. Flair is a subset of Pascal without any data structures, so there is an added challenge... A danger of teaching a compilers course -- any course, really -- is that I would rather write programs than do almost anything else in the world.

One could ask the same question of the Fibonacci series: Is every number a Fibonacci number? It is relatively easy to answer this question with 'no'. The sequence grows larger in each new entry, so once you skip any number, you know it's not in the list. C-numbers are tougher. They grow and shrink. For any given number n, we can search the tree of values until we find it. But there is no proof for all n.

As one last bit of preparation, Hofstadter gave an informal proof of the statement, "There are an infinite number of prime numbers." The key is that his argument used one assumption (there are a finite number of primes) to destroy a necessary consequence of the same (p1*p2*...*pk+1 is prime).

From there, Hofstadter told a compact, relatively simple version of Tarski's undefinability theorem and, at the end, made the bridge to Gödel's theorem. I won't tell that story here, for a couple of reasons. First, this entry is already quite long. Second, Hofstadter himself has already told this story better than I ever could, in Gödel, Escher, Bach. You really should read it there.

This story gave him a way to tell us about the importance of the proof: it drives a wedge between truth and provability. This undermines the Mathematician's Credo. It also allowed him to demonstrate his fascination with Gödel's Proof so many years ago: it uses mathematical logic to say something interesting, powerful, and surprising about mathematical logic itself.

Hofstadter opened the floor to questions. An emeritus CS professor asked his opinion of computer proofs, such as the famous 1976 proof of the four color theorem. That proof depends on a large number of special cases and requires hundreds of pages of analysis. At first, Hofstadter said he doesn't have much of an opinion. Of course, such proofs require new elements of trust, such as trust that the program is correct and trust that the computer is functioning correctly. He is okay with that. But then he said that he finds such proofs to be unsatisfying. Invariably, they are a form of brute force, and that violates the spirit of mathematics that excites him. In the end, these proofs do not help him to understand why something isn true, and that is the whole point of exploring: to understand why.

This answer struck a chord in me. There are whole swaths of artificial intelligence that make me feel the same way. For example, many of my students are fascinated by neural networks. Sure, it's exciting any time you can build a system that solves a problem you care about. (Look ma, no hands!) But these programs are unsatisfying because they don't give me any insight into the nature of the problem, or into how humans solve the problem. If I ask a neural network, "Why did you produce this output for this input?", I can't expect an answer at a conceptual level. A vector of weights leaves me cold.

To close the evening, Hofstadter responded to a final question about the incompleteness theorem. He summarized Gödel's result in this way: Every interesting formal system says true things, but it does not say all true things. He also said that Tarski's result is surprising, but in a way comforting. If an oracle for T-numbers existed, then mathematics would be over. And that would be depressing.

As expected, I enjoyed the evening greatly. Having read GEB and taken plenty of CS theory courses, I already knew the proofs themselves, so the technical details weren't a big deal. What really highlighted the talk for me was hearing Hofstadter talk about his passions: where they came from, how he has pursued them, and how these questions and answers continue to excite him as they do. Listening to an accomplished person tell stories that make connections to their lives always makes me happy.

We in computer science need to do more of what people like Hofstadter do: talk about the beautiful ideas of our discipline to as many people as we can, in way that is accessible to all. We need a Sagan or a Hofstadter to share the beauty.


PHOTOGRAPH 1: a photograph of the cover of my copy of Gödel, Escher, Bach.

PHOTOGRAPH 2: Douglas Hofstadter in Bologna, Italy, 2002. Source: Wikimedia Commons.

Posted by Eugene Wallingford | Permalink | Categories: Computing

February 14, 2012 3:51 PM

Beautiful Sentences, Programming Languages Division

From The Heroes of Java: Ward Cunningham (emphasis added):

Making a new language is the ultimate power move. There are lots of ways to do it. Get to know them all. A switch statement inside a for loop makes an interpreter. When you write that, ask yourself: what language am I interpreting?

Those last two sentences distill a few weeks of a course on programming languages into an idea that spans all of computer science. Beautiful.

As in most conversations with Ward, this one is full of good sentences, including:

The shortest path to exceeding expectations rarely goes through meeting expectations.


When [pair programming] you can assume both people bring something to the collaboration. I'd rather find my colleague's strength or passion and learn from them as we go forward together.

In my experience, Ward brings the same mindset to all of his professional interactions. I am still a fan, though I have never asked for his autograph.

Posted by Eugene Wallingford | Permalink | Categories: Computing

January 10, 2012 4:05 PM

Looking Forward: Preparing for Compilers

Spring semester is underway. My compilers course met for the first time today. After all these years, I still get excited at the prospect of writing a compiler. On top of that, we get to talk about programming languages and programming all semester.

I've been preparing for the course since last semester, during the programming languages course I debriefed recently. I've written blog entries as I planned previous offerings of the compiler course, on topics such as short iterations and teaching by example, fifteen compilers in fifteen weeks and teaching the course backwards. I haven't written anything yet this time for one of the same reasons I haven't been writing about my knee rehab: I haven't had much to say. Actually, I have two small things.

First, on textbooks. I found that the textbook I've used for the last few offering of the course now costs students over $140, even at Amazon. That's no $274.70, but sheesh. I looked at several other popular undergrad compiler texts and found them all to be well over $100. The books my students might want to keep for their professional careers are not suitable for an undergrad course, and the ones that are suitable are expensive. I understand the reasons why yet can't stomach hitting my students with such a large bill. The Dragon book is the standard, of course, but I'm not convinced it's a good book for my audience -- too few examples, and so much material. (At least it's relatively inexpensive, at closer to $105.)

I found a few compiler textbooks available free on-line, including that $275 book I like. Ultimately I settled on Torben Mogensen's Basics of Compiler Design. It covers the basic material without too much fluff, though it lacks a running example with full implementation. I'll augment it with my own material and web readings. The price is certainly attractive. I'll let you know how it works out.

Second, as I was filing a pile of papers over break, I ran across the student assessments from last offering of the course. Perfect timing! I re-read them and am able to take into account student feedback. The last group was pretty pleased with the course and offered two broad suggestions for improvement: more low-level details and more code examples. I concur in both. It's easy when covering so many new ideas to stay at an abstract level, and the compiler course is no exception. Code examples help students connect the ideas we discuss with the reality of their own projects.

These are time consuming improvements to make, and time will be at a premium with a new textbook for the course. This new text makes them even more important, though, because it has few code examples. My goal is to add one new code example to each week of the course. I'll be happy if I manage one really good example every other week.

And we are off.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

December 01, 2011 3:06 PM

Programming for Everyone: Journalists

Jacob Harris begins his recent In Praise of Impractical Programming with a short discussion of how programming is becoming an integral part of the newsroom:

For the past few years, I've been working as a software developer in the newsroom, where perceptions of my kind have changed from novelty to a necessity. Recognizing this, some journalism schools now even require programming courses to teach students practical skills with databases or web frameworks. It's thrilling to contemplate a generation of web-hacking journalists -- but I wish we could somehow squeeze a little magic into their course load.

This seems like a natural evolutionary path that many industries will follow in the coming years or decades. At first it will be enough to use other people's tools. Then, practitioners will want to be able to write code in a constrained environment, such as a web framework or a database application. Eventually, I suspect that at least a few of the programming practitioners will tire of the constraints, step outside of the box, and write the code -- and maybe even the tools -- they want and need. If historians can do it, so can journalists.

Posted by Eugene Wallingford | Permalink | Categories: Computing

November 30, 2011 7:07 PM

A Definition of Design from Charles Eames

Paul Rand's logo for NeXT

Our council of department heads meets in the dean's conference room, in the same building that houses the Departments of Theater and Art, among others. Before this morning's meeting, I noticed an Edward Tufte poster on the wall and went out to take a look. It turns out that the graphic design students were exhibiting posters they had made in one of their classes, while studying accomplished designers such as Tufte and Paul Rand, the creator of the NeXT logo for Steve Jobs.

As I browsed the gallery, I came across a couple of posters on the work of Charles and Ray Eames. One of them prominently featured this quote from Charles:

Design is a plan for arranging elements in such a way as best to accomplish a particular purpose.

This definition works just as well for software design as it does for graphic design. It is good to be reminded occasionally how universal the idea of design is to the human condition.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 12, 2011 10:40 AM

Tools, Software Development, and Teaching

Last week, Bret Victor published a provocative essay on the future of interaction design that reminds us we should be more ambitious in our vision of human-computer interaction. I think it also reminds us that we can and should be more ambitious in our vision of most of our pursuits.

I couldn't help but think of how Victor's particular argument applies to software development. First he defines "tool":

Before we think about how we should interact with our Tools Of The Future, let's consider what a tool is in the first place.

I like this definition: A tool addresses human needs by amplifying human capabilities.

a tool addresses human needs by amplifying human capabilities

That is, a tool converts what we can do into what we want to do. A great tool is designed to fit both sides.

The key point of the essay is that our hands have much more consequential capabilities than our current interfaces use. They feel. They participate with our brains in numerous tactile assessments of the objects we hold and manipulate: "texture, pliability, temperature; their distribution of weight; their edges, curves, and ridges; how they respond in your hand as you use them". Indeed, this tactile sense is more powerful than the touch-and-slide interfaces we have now and, in many ways, is more powerful than even sight. These tactile senses are real, not metaphorical.

As I read the essay, I thought of the software tools we use, from language to text editors to development processes. When I am working on a program, especially a big one, I feel much more than I see. At various times, I experience discomfort, dread, relief, and joy.

Some of my colleagues tell me that these "feelings" are metaphorical, but I don't think so. A big part of my affinity for so-called agile approaches is how these sensations come into play. When I am afraid to change the code, it often means that I need to write more or better unit tests. When I am reluctant to add a new feature, it often means that I need to refactor the code to be more hospitable. When I come across a "code smell", I need to clean up, even if I only have time for a small fix. YAGNI and doing the simplest thing that can possibly work are ways that I feel my way along the path to a more complete program, staying in tune with the code as I go. Pair programming is a social practice that engages more of my mind than programming alone.

Victor closes with some inspiration for inspiration:

In 1968 -- three years before the invention of the microprocessor -- Alan Kay stumbled across Don Bitzer's early flat-panel display. Its resolution was 16 pixels by 16 pixels -- an impressive improvement over their earlier 4 pixel by 4 pixel display.

Alan saw those 256 glowing orange squares, and he went home, and he picked up a pen, and he drew a picture of a goddamn iPad.

We can think bigger about so much of what we do. The challenge I take from Victor's essay is to think about the tools I to teach: what needs do they fulfill, and how well do they amplify my own capabilities? Just as important are the tools we give our students as they learn: what needs do they fulfill, and how well do they amplify our students' capabilities?

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

November 09, 2011 4:29 PM

Sentences of the Day: Sheer Fun

From Averia, The Average Font:

Having found a simple process to use, I was ready to start. And after about a month of part-time slaving away (sheer fun! Better than any computer game) -- in the process of which I learned lots about bezier curves and font metrics -- I had a result.

Programmers love to slave away in their free time on projects that put fires in their bellies, with no slave driver other than their own passion.

The story of Averia is a worthy one to read, even if you are not particularly a font person. It's really about how the seed of an idea can grow as our initial efforts pull us deeper into the beautiful intricacies of a problem. It also reminds us how programs make nice testbeds for our experiments.

Posted by Eugene Wallingford | Permalink | Categories: Computing

November 05, 2011 10:09 AM

Is Computing Too Hard, Too Foreign, or Too Disconnected?

A lot of people are discussing a piece published in the New York Times piece yesterday, Why Science Majors Change Their Minds (It's Just So Darn Hard). It considers many factors that may be contributing to the phenomenon, such as low grades and insufficient work habits.

Grades are typically much lower in STEM departments, and students aren't used to getting that kind of marks. Ben Deaton argues that this sort of rigor is essential, quoting his undergrad steel design prof: "If you earn an A in this course, I am giving you a license to kill." Still, many students think that a low grade -- even a B! -- is a sign that they are not suited for the major, or for the specialty area. (I've had students drop their specialty in AI after getting a B in the foundations course.)

Most of the students who drop under such circumstances are more than capable of succeeding. Unfortunately, they have not usually developed the disciplined work habits they need to succeed in such challenging majors. It's a lot easier to switch to a different major where their current skills suffice.

I think there are two more important factors at play. On the first, the Times article paraphrases Peter Kilpatrick, Notre Dame's Dean of Engineering:

... it's inevitable that students will be lost. Some new students do not have a good feel for how deeply technical engineering is.

In computer science, our challenge is even bigger: most HS students don't have any clue at all what computer science is. My university is nearing the end of its fall visit days for prospective students, who are in the process of choosing a college and a major. The most common question I am asked is, "What is computer science?", or its cousin, "What do computer scientists do?". This question comes from even the brightest students, ones already considering math or physics. Even more students walk by the CS table with their parents with blank looks on their faces. I'm sure some are thinking, "Why consider a major I have no clue about?"

This issue also plagues students who decide to major in CS and then change their minds, which is the topic of the Times article. Students begin the major not really knowing what CS is, they find out that they don't like it as much as they thought they might, and they change. Given what they know coming into the university, it really is inevitable that a lot of students will start and leave CS before finishing.

On the second factor I think most important, here is the money paragraph from the Times piece:

But as Mr. Moniz (a student exceedingly well prepared to study engineering) sat in his mechanics class in 2009, he realized he had already had enough. "I was trying to memorize equations, and engineering's all about the application, which they really didn't teach too well," he says. "It was just like, 'Do these practice problems, then you're on your own.'" And as he looked ahead at the curriculum, he did not see much relief on the horizon.

I have written many times here about the importance of building instructions around problems, beginning with Problems Are The Thing. Students like to work on problems, especially problems that matter to someone in the world. Taken to the next level, as many engineering schools are trying to do, courses should -- whenever possible -- be built around projects. Projects ground theory and motivate students, who will put in a surprising effort on a project they care about or think matters in the world. Projects are also often the best way to help students understand why they are working so hard to memorize and practice tough material.

In closing, I can take heart that schools like mine are doing a better job retaining majors:

But if you take two students who have the same high school grade-point average and SAT scores, and you put one in a highly selective school like Berkeley and the other in a school with lower average scores like Cal State, that Berkeley student is at least 13 percent less likely than the one at Cal State to finish a STEM degree.

Schools tend to teach less abstractly than our research-focused sister schools. We tend to provide more opportunities early in the curriculum to work on projects and to do real research with professors. I think the other public universities in my state do a good job, but if a student is considering an undergrad STEM major, they will be much better served at my university.

There is one more reason for the better retention rate at the "less selective" schools: pressure. The students at the more selective schools are likely to be more competitive about grades and success than the students at the less selective schools. This creates an environment more conducive to learning for most students. In my department, we try not to "treat the freshman year as a 'sink or swim' experience and accept attrition as inevitable" for reasons of Darwinian competition. As the Times article says, this is both unfair to students and wasteful of resources.

By changing our curricula and focusing more on student learning than on how we want to teach, universities can address the problem of motivation and relevance. But that will leave us with the problem of students not really knowing what CS or engineering are, or just how technical and rigorous they need to be. This is an education problem of another sort, one situated in the broader population and in our HS students. We need to find ways to both share the thrill and help more people see just what the STEM disciplines are and what they entail.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 02, 2011 7:39 AM

Programming for All:

While reading a bit about the recent flap over racism in the tech start-up world, I found this passage in a piece by Michael Arrington: was proposing an ambitious new idea to help get inner city youth (mostly minorities) to begin to see superstar entrepreneurs as the new role models, instead of NBA stars. He believes that we can effect real societal change by getting young people to learn how to program, and realize that they can start businesses that will change the world.

Cool. is a pop star who has had the ear of a lot of kids over the last few years. I hope they listen to this message from him as much as they do to his music.

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 31, 2011 3:47 PM

"A Pretty Good Lisp"

I occasionally read or hear someone say, "X is a pretty good Lisp", where X is a programming language. Usually, it's a newish language that is more powerful than the languages many of us learned in school. For a good example, see Why Ruby is an acceptable LISP. A more recent article, Ruby is beautiful (but I'm moving to Python) doesn't go quite that far. It says only "almost":

Ruby does not revel in structures or minutiae. It is flexible. And powerful. It really almost is a Lisp.

First, let me say that I like both of these posts. They tell us about how we can do functional programming in Ruby, especially through its support for higher-order functions. As a result, I have found both posts to be useful reading for students. And, of course, I love Ruby, and like Python well enough.

But that's not all there is to Lisp. It's probably not even the most important thing.

Kenny Tilton tells a story about John McCarthy's one-question rebuttal to such claims at the very end of his testimonial on adopting Lisp, Ooh! Ooh! My turn! Why Lisp?:

... [McCarthy] simply asked if Python could gracefully manipulate Python code as data.

"No, John, it can't," said Peter [Norvig] and nothing more...

That's the key: data == program. It really is the Big Idea that sets Lisp apart from the other programming languages we use. I've never been a 100% full-time Lisper, and as a result I don't think I fully appreciate the full power to which Lisp programmers put this language feature. But I've programmed enough with and without macros to be able to glimpse what they see in the ability to gracefully manipulate their code -- all code -- as data.

In the "acceptable Lisp" article linked above, Kidd does address this shortcoming and says that "Ruby gives you about 80% of what you want from macros". Ruby's rather diverse syntax lets us create readable DSLs such as Treetop and Rake, which is one of the big wins that Lisp and Scheme macros give us. In this sense, Ruby code can feel generative, much as macros do.

Unfortunately, Ruby, Python, and other "pretty good Lisps" miss out on the other side of the code-as-data equation, the side McCarthy drew out in his question: manipulation. Ruby syntax is too irregular to generate "by hand" or to read and manipulate gracefully. We can fake it, of course, but to a Lisp programmer it always feels fake.

I think what most people mean when they say a language is a pretty good Lisp is that it can be used as a pretty good functional programming language. But Lisp is not only an FP language. Many would claim it is not even primarily a functional programming language.

I love Ruby. But it's not a pretty good Lisp. It is a fine programming languages, perhaps my favorite these days, with strengths that take it beyond the system programming languages that most of us cut our teeth on. Among those strengths are excellent support for a functional programming style. It also has its weaknesses, like every other programming language.

Neither is Python a pretty good Lisp. Nor is most anything else, for that matter. That's okay.

All I ask is this: When you are reading articles like the ones linked above, don't dismiss every comment you see that says, "No, it's not, and here's why" as the ranting of a smug Lisp weenie. It may be a rant, and it may be written by a smug Lisp weenie. But it may instead be written by a perfectly sane programmer who is trying to teach you that there is more to Lisp than higher-order functions, and that the more you've missed is a whole lot more. We can learn from some of those comments, and think about how to make our programming languages even better.

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 25, 2011 3:53 PM

On the Passing of John McCarthy

John McCarthy tribute -- 'you are doing it wrong'

It's been a tough couple of weeks for the computer science community. First we lost Steve Jobs, then Dennis Ritchie. Now word comes that John McCarthy, the creator of Lisp, died late Sunday night at the age of 84. I'm teaching Programming Languages this semester based on the idea of implementing small language interpreters, and we are using Scheme. McCarthy's ideas and language are at the heart of what my students and I are doing every day.

Scheme is a Lisp, so McCarthy is its grandfather. Lisp is different from just about every other programming language. It's not just the parentheses, which are only syntax. In Lisp and Scheme, programs and data are the same. To be more specific, the representation of a Lisp program is the the same representation used to represent Lisp data. The equivalence of data and program is one of the truly Big Ideas of computer science, one which I wrote about in Basic Concepts: The Unity of Data and Program. This idea is crucial to many areas of computer science, even ones in which programmers do not take direct advantage of it through their programming language.

We also owe McCarthy for the idea that we can write a language interpreter in the language being interpreted. Actually, McCarthy did more: he stated the features of Lisp in terms of the language features themselves. Such a program defines the language in which the program is written. This is the idea of meta-circular interpreter, in which two procedures:

  • a procedure that evaluates an expression, and
  • a procedure that applies a procedure to its arguments
recurse mutually to evaluate a program. This is one of the most beautiful ideas in computing, as well as serving as the mechanism and inspiration for modern-day interpreters and compilers.

Last week, the CS world lost Dennis Ritchie, the creator of the C programming language. By all accounts I've read and heard, McCarthy and Ritchie were very different kinds of people. Ritchie was an engineer through and through, while McCarthy was an academic's academic. So, too, are the languages they created very different. Yet they are without question the two most influential programming languages ever created. One taught us about simplicity and made programming across multiple platforms practical and efficient; the other taught us about simplicity made programming a matter of expressiveness and concision.

Though McCarthy created Lisp, he did not implement the first Lisp interpreter. As Paul Graham relates in Revenge of the Nerds, McCarthy first developed Lisp as a theoretical exercise, an attempt to create an alternative to the Turing Machine. Steve Russell, one of McCarthy's grad students, suggested that he could implement the theory in an IBM 704 machine language program. McCarthy laughed and told him, "You're confusing theory with practice..." Russell did it any way. (Thanks to Russell and the IBM 704, we also have car and cdr!) McCarthy and Russell soon discovered that Lisp was more powerful than the language they had planned to build after their theoretical exercise, and the history of computing was forever changed.

If you'd like, take a look at my Scheme implementation of John McCarthy's Lisp written in Lisp. It is remarkable how much can be built out of so little. Alan Kay has often compared this interpreter to Maxwell's equations in physics. To me, its parts usually feel like the basic particles out of which all matter is built. Out of these few primitives, all programs are built.

I first learned of McCarthy not from Lisp but from my first love, AI. McCarthy coined the term "Artificial Intelligence" when organizing (along with Minsky, Rochester, and Shannon) the 1956 Dartmouth conference that gave birth to the field. I studied McCarthy's work in AI using the language he had created. To me, he was a giant of AI long before I recognized that he was giant of programming languages, too. Like many pioneers of our field, he laid the groundwork in many subdisciplines. They had no choice; they had to build their work out of ideas using only the rawest materials. McCarthy is even credited with the first public descriptions of time-sharing systems and what we now call cloud computing. (For McCarthy's 1970-era predictions about home computers and the cloud, see his The Home Information Terminal, reprinted in 2000.)

Our discipline has lost a giant.

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 24, 2011 7:38 PM

Simple/Complex Versus Easy/Hard

A few years ago, I heard a deacon give a rather compelling talk to a group of college students on campus. When confronted with a recommended way to live or act, students will often say that living or acting that way is hard. These same students are frustrated with the people who recommend that way of living or acting, because the recommenders -- often their parents or teachers -- act as if it is easy to live or act that way. The deacon told the students that their parents and teachers don't think it is easy, but they might well think it is simple.

How can this be? The students were confounding "simple" and "easy". A lot of times, life is simple, because we know what we should do. But that does not make life easy, because doing a simple thing may be quite difficult.

This made an impression on me, because I recognized that conflict in my own life. Often, I know just what to do. That part is simple. Yet I don't really want to do it. To do it requires sacrifice or pain, at least in the short term. To do it means not doing something else, and I am not ready or willing to forego that something. That part is difficult.

Switch the verb from "do" to "be", and the conflict becomes even harder to reconcile. I may know what I want to be. However, the gap between who I am and who I want to be may be quite large. Do I really want to do what it takes to get there? There may be a lot of steps to take which individually are difficult. The knowing is simple, but the doing is hard.

This gap surely faces college students, too, whether it means wanting to get better grades, wanting to live a healthier life, or wanting to reach a specific ambitious goal.

When I heard the deacon's story, I immediately thought of some of my friends, who like very much the idea of being a "writer" or a "programmer", but they don't really want to do the hard work that is writing or programming. Too much work, too much disappointment. I thought of myself, too. We all face this conflict in all aspects of life, not just as it relates to personal choices and values. I see it in my teaching and learning. I see it in building software.

I thought of this old story today when I watched Rich Hickey's talk from StrangeLoop 2011, Simple Made Easy. I had put off watching this for a few days, after tiring of a big fuss that blew up a few weeks ago over Hickey's purported views about agile software development techniques. I knew, though, that the dust-up was about more than just Hickey's talk, and several of my friends recommended it strongly. So today I watched. I'm glad I did; it is a good talk. I recommend it to you!

Based only on what I heard in this talk, I would guess that Hickey misunderstands the key ideas behind XP's practices of test-driven development and refactoring. But this could well be a product of how some agilistas talk about them. Proponents of agile and XP need to be careful not to imply that tests and refactoring make change or any other part of software development easy. They don't. The programmer still has to understand the domain and be able to think deeply about the code.

Fortunately, I don't base what I think about XP practices on what other people think, even if they are people I admire for other reasons. And if you can skip or ignore any references Hickey makes to "tests as guard rails" or to statements that imply refactoring is debugging, I think you will find this really is a very good talk.

Hickey's important point is that simple/complex and easy/hard are different dimensions. Simplicity should be our goal when writing code, not complexity. Doing something that is hard should be our goal when it makes us better, especially when it makes us better able to create simplicity.

Simplicity and complexity are about the interconnectedness of a system. In this dimension, we can imagine objective measures. Ease and difficulty are about what is most readily at hand, what is most familiar. Defined as they are in terms of a person's experience or environment, this dimension is almost entirely subjective.

And that is good because, as Hickey says a couple of times in the talk, "You can solve the familiarity problem for yourself." We are not limited to our previous experience or our current environment; we can take on a difficult challenge and grow.

a Marin mountain bike

Alan Kay often talks about how it is worth learning to play a musical instrument, even though playing is difficult, at least at the start. Without that skill, we are limited in our ability to "make music" to turning on the radio or firing up YouTube. With it, you are able make music. Likewise riding a bicycle versus walking, or learning to fly an airplane versus learning to drive a car. None of these skills is necessarily difficult once we learn them, and they enable new kinds of behaviors that can be simple or complex in their own right.

One of the things I try to help my students see is the value in learning a new, seemingly more difficult language: it empowers us to think new and different thoughts. Likewise making the move from imperative procedural style to OOP or to functional programming. Doing so stretches us. We think and program differently afterward. A bonus is that something that seemed difficult before is now less daunting. We are able to work more effectively in a bigger world.

In retrospect, what Hickey says about simplicity and complexity is actually quite compatible with the key principles of XP and other agile methods. Writing tests is a part of how we create systems that are as simple as we can in the local neighborhood of a new feature. Tests can also help us to recognize complexity as it seeps into our program, though they are not enough by themselves to help us see complexity. Refactoring is an essential part of how we eliminate complexity by improving design globally. Refactoring in the presence of unit tests does not make programming easy. It doesn't replace thinking about design; indeed, it is thinking about design. Unit tests and refactoring do help us to grapple with complexity in our code.

Also in retrospect, I gotta make sure I get down to St. Louis for StrangeLoop 2012. I missed the energy this year.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

October 19, 2011 1:37 PM

A Programming Digression: Benford's Law and Factorials

leading digits of factorials up to 500

This morning, John Cook posted a blog entry on the leading digits of factorials and how, despite what might be our intuition, they follow Benford's Law. He whipped up some Python code and showed the results of his run for factorials up to 500. I have linked to his graphic at the right.

As I am , I decided to whip up a quick Scheme version of Cook's experiment. He mentioned some implementation issues involving the sizes of integers and floating-point numbers in Python, and I wondered how well Scheme would fare.

For my first attempt, I did the simplest thing that would possibly work. I already had a tail-recursive factorial function and so wrote a procedure that would call it n times and record the first digit of each:

(define benford-factorials
  (lambda (n)
    (let ((counts (make-vector 10 0)))
      (letrec ((foreach
                 (lambda (n)
                   (if (zero? n)
                       (let ((lead-digit (first-digit (factorial n))))
                         (vector-set! counts lead-digit
                                      (+ 1 (vector-ref counts lead-digit)))
                         (foreach (- n 1)))))))
        (foreach n)))))

This gets the answers for us:

     > (benford-factorials 500)
     #(0 148 93 67 38 34 43 24 28 25)

Of course, it is wildly inefficient. My naive implementation computes and acts on each factorial independently, which means that it recomputes (n-1)!, (n-2)!, ... for each value less than n. As a result, benford-factorials becomes unnecessarily sluggish for even relatively small values of n. How can I do better?

I decided to create a new factorial function, one that caches the smaller factorials it creates on the way to n!. I call it all-factorials-up-to:

(define all-factorials-up-to
  (lambda (n)
    (letrec ((aps (lambda (i acc)
                    (if (> i n)
                        (aps (+ i 1)
                             (cons (* i (car acc)) acc))))))
      (aps 2 '(1)))))

Now, benford-factorials can use a more functional approach: map first-digit over the list of factorials, and then map a count incrementer over the list of first digits.

(define benford-factorials
  (lambda (n)
    (let ((counts (make-vector 10 0))
          (first-digits (map first-digit
                             (all-factorials-up-to n))))
      (map (lambda (digit)
             (vector-set! counts digit
                          (+ 1 (vector-ref counts digit))))

(We can, of course, do without the temporary variable first-digit by dropping the first map right into the second. I often create an explaining temporary variable such as this one to make my code easier for me to write and read.)

How does this one perform? It gets the right answers and runs more comfortably on larger n:

     > (benford-factorials 500)
     #(0 148 93 67 38 34 43 24 28 25)
     > (benford-factorials 1000)
     #(0 293 176 124 102 69 87 51 51 47)
     > (benford-factorials 2000)
     #(0 591 335 250 204 161 156 107 102 94)
     > (benford-factorials 3000)
     #(0 901 515 361 301 244 233 163 147 135)
     > (benford-factorials 4000)
     #(0 1192 707 482 389 311 316 227 201 175)
     > (benford-factorials 5000)
     #(0 1491 892 605 477 396 387 282 255 215)

This procedure begins to be sluggish for n ≥ 3000 on my iMac.

Cook's graph shows how closely the predictions of Benford's Law fit for factorials up to 500. How well do the actual counts match the predicted values for the larger sets of factorials? Here is a comparison for n = 3000, 4000, and 5000:

     n = 3000
       digit        1   2   3   4   5   6   7   8   9
       actual     901 515 361 301 244 233 163 147 135
       predicted  903 528 375 291 238 201 174 153 137

n = 4000 digit 1 2 3 4 5 6 7 8 9 actual 1192 707 482 389 311 316 227 201 175 predicted 1204 704 500 388 317 268 232 205 183

n = 5000 digit 1 2 3 4 5 6 7 8 9 actual 1491 892 605 477 396 387 282 255 215 predicted 1505 880 625 485 396 335 290 256 229

That looks pretty close to the naked eye. I've always found Benford's Law to be almost magic, even though mathematicians can give a reasonable account of why it holds. Seeing it work so well with something seemingly as arbitrary as factorials only reinforces my sense of wonder.

If you would like play with these ideas, feel free to start with my Scheme code. It has everything you need to replicate my results above. If you improve on my code or take it farther, please let me know!

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 17, 2011 4:46 PM

Computational Thinking Everywhere: Experiments in Education

I recently ran across Why Education Startups Do Not Succeed, based on the author's experience working as an entrepreneur in the education sector. He admits upfront that he isn't offering objective data to support his conclusions, so we should take them with a grain of salt. Still, I found his ideas interesting. Here is the take-home point in sentences:

Most entrepreneurs in education build the wrong type of business, because entrepreneurs think of education as a quality problem. The average person thinks of it as a cost problem.

That disconnect creates a disconnect between the expectations of sellers and buyers, which ends up hurting, even killing, most education start-ups.

The old AI guy in me latched on to this paragraph:

Interestingly, in the US, the people who are most willing to try new things are the poor and uneducated because they have a similar incentive structure to a person in rural India. Their default state is "screwed." If a poor person doesn't do something dramatic, they are going to stay screwed. Many parents and teachers in these communities understand this. So the communities are often willing to try new, experimental things -- online education, charter schools, longer school days, no summer vacation, co-op programs -- even if they may not work. Why? Because their students default state is "screwed", and they need something dramatically better. Doing something significantly higher quality is the only way to overcome the inertia of already being screwed. The affordable, but poor quality approaches just aren't good enough. These communities are on the hunt for dramatically better approaches and willing to try new things.

Local and global maxima in hill-climbing

I've seen other discussions of the economic behavior of people in the lowest socioeconomic categories that fit this model. Among them were the consumption of lottery tickets in lieu of saving, and more generally the trade-off between savings and consumption. If a small improvement won't help a people much, then it seems they are more likely willing to gamble on big improvements or to simply enjoy short-term rewards of spending.

This mindset immediately brought to mind the AI search technique known as hill climbing. When you know you are on a local maximum that is significantly lower than the global maximum, you are willing to take big steps in search of a better hill to climb, even if that weakens your position in the short-term. Baby steps won't get you there.

This is a small example of unexpected computational thinking in the real world. Psychologically, it seems, that we are often hill climbers.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

October 13, 2011 3:10 PM

Learning and New Kinds of Problems

I recently passed this classic by Reg Braithwaite to a grad student who is reading in the areas of functional programming and Ruby. I love how Braithwaite prefaces the technical content of the entry with an exhortation to learners:

... to obtain the deepest benefit from learning a new language, you must learn to think in the new language, not just learn to translate your favourite programming language syntax and idioms into it.

The more different the thing you are learning from what you already know, the more important this advice. You are already good at solving the problems your current languages solve well!

And worse, when a new tool is applied to a problem you think you know well, you will probably dismiss the things the new tool does well. Look at how many people dismiss brevity of code. Note that all of the people ignore the statistics about the constant ratio between bugs and lines of code use verbose languages. Look at how many people dismiss continuation-based servers as a design approach. Note that all of them use programming languages bereft of control flow abstractions.

Real programmers know Y.

This is great advice for people trying to learn functional programming, which is all the rage these days. Many people come to a language like Scheme, find it lacking for the problems they have been solving in Python, C, and Java, and assume something is wrong with Scheme, or with functional programming more generally. It's easy to forget that the languages you know and the problems you solve are usually connected in a variety of ways, not the least of which for university students is that we teach them to solve problems most easily solved by the languages we teach them!

If you keep working on the problems your current language solves well, then you miss out on the strengths of something different. You need to stretch not only your skill set but also your imagination.

If you buy this argument, schedule some time to work through Braithwaite's derivation of the Y combinator in Ruby. It will, as my daughter likes to say, make your brain hurt. That's a good thing. Just like with physical exercise, sometimes we need to stretch our minds, and make them hurt a bit, on the way to making them stronger.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

October 12, 2011 12:31 PM

Programming for Everyone -- Really?

TL;DR version: Yes.

Yesterday, I retweeted a message that is a common theme here:

Teaching students how to operate software, but not produce software, is like teaching kids to read & not write. (via @KevlinHenney)

It got a lot more action than my usual fare, both retweets and replies. Who knew? One of the common responses questioned the analogy by making another, usually of this sort:

Yeah, that would be like teaching kids how to drive a car, but not build a car. Oh, wait...

This is a sounds like a reasonable comparison. A car is a tool. A computer is a tool. We use tools to perform tasks we value. We do not always want to make our own tools.

But this analogy misses out on the most important feature of computation. People don't make many things with their cars. People make things with a computer.

When people speak of "using a computer", they usually mean using software that runs on a computer: a web browser, a word processor, a spreadsheet program. And people use many of these tools to make things.

As soon as we move into the realm of creation, we start to bump into limits. What if the tool we are given doesn't allow us to say or do what we want? Consider the spreadsheet, a general data management tool. Some people use it simply as a formatted data entry tool, but it is more. Every spreadsheet program gives us a formula language for going beyond what the creators of Excel or Numbers imagined.

But what about the rest of our tools? Must we limit what we say to what our tool affords us -- to what our tool builders afford us?

A computer is not just a tool. It is also a medium of expression, and an increasingly important one.

If you think of programming as C or Java, then the idea of teaching everyone to program may seem silly. Even I am not willing to make that case here. But there are different kinds of programming. Even professional programmers write code at many levels of abstraction, from assembly language to the highest high-level language. Non-programmers such as physicists and economists use scripting languages like Python. Kids of all ages are learning to program Scratch.

Scratch is a good example of what I was thinking when I retweeted. Scratch is programming. But Scratch is really a way to tell stories. Just like writing and speaking.

Alfred Thompson summed up this viewpoint succinctly:

[S]tudents need to be creators and not just consumers.

Kids today understand this without question. They want to make video mash-ups and interactive web pages and cutting-edge presentations. They need to know that they can do more than just use the tools we deign to give them.

One respondent wrote:

As society evolves there is an increasing gap between those that use technology and those that can create technology. Whilst this is a concern, it's not the lowest common denominator for communication: speaking, reading and writing.

The first sentence is certainly true. The question for me is: on which side of this technology divide does computing live? If you think of computation as "just" technology, then the second sentence seems perfectly reasonable. People use Office to do their jobs. It's "just a tool".

It could, however, be a better tool. Many scientists and business people write small scripts or programs to support their work. Many others could, too, if they had the skills. What about teachers? Many routine tasks could be automated in order to give them more time to do what they do best, teach. We can write software packages for them, but then we limit them to being consumers of what we provide. They could create, too.

Is computing "just tech", or more? Most of the world acts like it is the former. The result is, indeed, an ever increasing gap between the haves and the have nots. Actually, the gap is between the can dos and the cannots.

I, and many others, think computation is more than simply a tool. In the wake of Steve Jobs's death last week, many people posted his famous quote that computing is a liberal art. Alan Kay, one of my inspirations, has long preached that computing is a new medium on the order of reading and writing. The list of people in the trenches working to make this happen is too numerous to include.

More practically, software and computer technology are the basis of much innovation these days. If we teach the new medium to only a few, the "5 percent of the population over in the corner" to whom Jobs refers, we exclude the other 95% from participating fully in the economy. That restricts economic growth and hurts everyone. It is also not humane, because it restricts people's personal growth. Everyone has a right to the keys to the kingdom.

I stand in solidarity with the original tweeter and retweeter. Teaching students how to operate software, but not produce software, is like teaching kids to read but not to write. We can do better.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

October 04, 2011 4:43 PM

Programming in Context: Digital History

Last April I mentioned The Programming Historian, a textbook aimed at a specific set of non-programmers who want or need to learn how to program in order to do their job in the digital age. I was browsing through the textbook today and came across a paragraph that applies to more than just historians or so-called applied programmers:

Many books about programming fall into one of two categories: (1) books about particular programming languages, and (2) books about computer science that demonstrate abstract ideas using a particular programming language. When you're first getting started, it's easy to lose patience with both of these kinds of books. On the one hand, a systematic tour of the features of a given language and the style(s) of programming that it supports can seem rather remote from the tasks that you'd like to accomplish. On the other hand, you may find it hard to see how the abstractions of computer science are related to your specific application.

I don't think this feeling is limited to people with a specific job to do, like historians or economists. Students who come to the university intending to major in Computer Science lose patience with many of our CS1 textbooks and CS1 courses for the very same reasons. Focusing too much on all the features of a language is overkill when you are just trying to make something work. The abstractions we throw at them don't have a home in their understanding of programming or CS yet and so seem, well, too abstract.

Writing for the aspiring applied programmer has an advantage over writing for CS1: your readers have something specific they want to do, and they know just what it is. Turkel and MacEachern can teach a subset of several tools, including Python and Javascript, focused on what historians want to be able to do. Greg Wilson and his colleagues can teach what scientists want and need to know, even if the book is pitched more broadly.

In CS1, your students don't have a specific task in mind and do eventually need to take a systematic tour of a language's features and to learn a programming style or three. They do, eventually, need to learn a set of abstractions and make sense of them in the context of several languages. But when they start, they are much like any other person learning to program: they would like to do something that matters. The problems we ask them to solve matter.

Guzdial, Ericson, and their colleagues have used media computation as context in which to learn how to program, with the idea that many students, CS majors and non-majors alike, can be enticed to manipulate images, sounds, and video, the raw materials out of which students' digital lives are now constructed. It's not quite the same -- students still need to be enticed, rather than starting with their own motivation -- but it's a shorter leap to caring than the run-off-the-mill CS textbook has to make.

Some faculty argue that we need a CS0 course that all students take, in which they can learn basic programming skills in a selected context before moving onto the major's first course. The context can be general enough, say, media manipulation or simple text processing on the web, that the tools students learn will be useful after the course whether they continue on or not. Students who elect to major in CS move on to take a systematic tour of a language's features, to learn about OO or FP style, and to begin learning the abstractions of the discipline.

My university used to follow this approach, back in the early and mid 1990s. Students had to take a one-year HS programming course or a one-semester programming course at the university before taking CS1. We dropped this requirement when faculty began asking, why shouldn't we put the same care into teaching low-level programming skills in CS1 as we do into teaching CS0? The new approach hasn't always been as successful as we hoped, due to the difficulty of finding contexts that motivate students as well as we want, but I think the approach is fundamentally sound. It means that CS1 may not teach all the things that it did when the course had a prerequisite.

That said, students who take one of our non-majors programming courses, C and Visual Basic, and then move decide to major in CS perform better on average in CS1 than students who come in fresh. We have work to do.

Finally, one sentence from The Programming Historian made me smile. It embodies the "programming for all" theme that permeates this blog:

Programming is for digital historians what sketching is for artists or architects: a mode of creative expression and a means of exploration.

I once said that being able to program is like having superhuman strength. But it is both more mundane and more magical than that. For digital historians, being able to program means being able to do the mundane, everyday tasks of manipulating text. It also gives digital historians a way to express themselves creatively and to explore ideas in ways hard to imagine otherwise.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

October 03, 2011 7:20 AM

Softmax, Recursion, and Higher-Order Procedures

Update: This entry originally appeared on September 28. I bungled my blog directory and lost two posts, and the simplest way to get the content back on-line is to repost.

John Cook recently reported that he has bundled up some of his earlier writings about the soft maximum as a tech report. The soft maximum is "a smooth approximation to the maximum of two real variables":

    softmax(x, y) = log(exp(x) + exp(y))

When John posted his first blog entry about the softmax, I grabbed the idea and made it a homework problem for my students, who were writing their first Scheme procedures. I gave them a link to John's page, so they had access to this basic formula as well as a Python implementation of it. That was fine with me, because I was simply trying to help students become more comfortable using Scheme's unusual syntax:

    (define softmax
      (lambda (x y)
        (log (+ (exp x)
                (exp y)))))

On the next assignment, I asked students to generalize the definition of softmax to more than two variables. This gave them an opportunity to write a variable arity procedure in Scheme. At that point, they had seen only a couple simple examples of variable arity, such as this implementation of addition using a binary + operator:

    (define plus              ;; notice: no parentheses around
      (lambda args            ;; the args parameter in lambda
        (if (null? args)
            (+ (car args) (apply plus (cdr args))) )))

Many students followed this pattern directly for softmax:

    (define softmax-var
      (lambda args
        (if (null? (cdr args))
            (car args)
            (softmax (car args)
                     (apply softmax-var (cdr args))))))

Some of their friends tried a different approach. They saw that they could use higher-order procedures to solve the problem -- without explicitly using recursion:

    (define softmax-var
      (lambda args
        (log (apply + (map exp args)))))

When students saw each other's solutions, they wondered -- as students often do -- which one is correct?

John's original blog post on the softmax tells us that the function generalizes as we might expect:

    softmax(x1, x2, ..., xn) = log(exp(x1) + exp(x2) + ... + exp(xn))

Not many students had looked back for that formula, I think, but we can see that it matches the higher-order softmax almost perfectly. (map exp args) constructs a list of the exp(xi) values. (apply + ...) adds them up. (log ...) produces the final answer.

What about the recursive solution? If we look at how its recursive calls unfold, we see that this procedure computes:

    softmax(x1, softmax(x2, ..., softmax(xn-1, xn)...))

This is an interesting take on the idea of a soft maximum, but it is not what John's generalized definition says, nor is it particularly faithful to the original 2-argument function.

How might we roll our own recursive solution that computes the generalized function faithfully? The key is to realize that the function needs to iterate not over the maximizing behavior but the summing behavior. So we might write:

    (define softmax-var
      (lambda args
        (log (accumulate-exps args))))

(define accumulate-exps (lambda (args) (if (null? args) 0 (+ (exp (car args)) (accumulate-exps (cdr args))))))

This solution turns softmax-var into interface procedure and then uses structural recursion over a flat list of arguments. One advantage of using an interface procedure is that the recursive procedure accumulate-exps no longer has to deal with variable arity, as it receives a list of arguments.

It was remarkable to me and some of my students just how close the answers produced by the two student implementations of softmax were, given how different the underlying behaviors are. Often, the answers were identical. When different, they differed only in the 12th or 15th decimal digit. As several blog readers pointed out, softmax is associative, so the two solutions are identical mathematically. The differences in the values of the functions result from the vagaries of floating-point precision.

The programmer in me left the exercise impressed by the smoothness of the soft maximum. The idea is resilient across multiple implementations, which makes it seem all the more useful to me.

More important, though, this programming exercise led to several interesting discussions with students about programming techniques, higher-order procedures, and the importance of implementing solutions that are faithful to the problem domain. The teacher in me left the exercise pleased.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

October 02, 2011 5:16 PM

Computing Everywhere: Economist Turns Programmer

This About Me page, on Ana Nelson's web site, is a great example of how computing can sneak up on people:

Having fallen in love with programming while studying for my Ph.D. in economics, I now develop open source software to explore and explain data. I am the creator of Dexy, a new tool for reproducible research and document automation.

A lot of disciplines explore and explain data, from particular domains and within particular models. I'm not surprised when I encounter someone in one of those disciplines who finds she likes exploring and explaining data more than the specific domain or model. Programming is a tool that lets rise above disciplinary silos and consider data, patterns, and ideas across the intellectual landscape.

Posted by Eugene Wallingford | Permalink | Categories: Computing

September 26, 2011 9:02 PM

Taking Computer Science Off the Beaten Track

I love this idea: a workshop at POPL 2012, the premier programming languages research conference, called Off the Beaten Track. Here is the gist.

Programming language researchers have the principles, tools, algorithms and abstractions to solve all kinds of problems, in all areas of computer science. However, identifying and evaluating new problems, particularly those that lie outside the typical core PL problems we all know and love, can be a significant challenge. Hence, the goal of this workshop is to identify and discuss problems that do not often show up in our top conferences, but where programming language researchers can make a substantial impact. The hope is that by holding such a forum and associating it directly with a top conference like POPL, we can slowly start to increase the diversity of problems that are studied by PL researchers and that by doing so we will increase the impact that our community has on the world.

I remember when I first started running across papers written by physicists on networks in social science and open-source software. Why were physicists writing these papers? They are out of their league. In fact, though they were something else: curious, and equipped with good tools for studying the problems. Good for them -- and good fro the rest of us. too, as they contributed to our understanding of how the world works.

Computer science has even better tools and methods for studying all manners of problems and systems, especially the more dynamic systems. Our ability to reify the language of any domain and then write programs to implement the semantics of the domain is one step up from the models that most mathematicians and physicists bring to the table.

Sometimes we forget the power we have in language. As Tim Ottinger tweeted today, "I think people have lost the idea that OO builds the language you implement in, as well as the product." We forget to use our own ability to create and use language even in an approach built on the premise! And of course we can go farther when we build architectures and virtual machines for domain-specific languages, rather than living inside a relatively restrictive model like Java.

The organizers of Off the Beaten Track remind us to think about the wealth of "principles, tools, algorithms, and abstractions" we possess and can bring to bear on problems far beyond the narrow technical area of programming languages research, from the natural sciences to art and music, from economics and the law to linguistics and education. They even acknowledge that we don't always appreciate the diversity in our own research field and so encourage submissions on "unusual compilers" and "underrepresented programming languages".

The last sentence in the passage above expresses an important ultimate goal: to increase the impact the programming languages community has on the world. I heartily support this goal and suggest that it is an important one not only for programming languages researchers. It is essential that many more of us in computer science look off the beaten track for ways to apply what we have learned to problems far beyond our own borders. If we start focusing on problems that matter to other people, problems that matter, we might just solve them.

My favorite line in the Off the Beaten Track home page is the last item in its bullet list of potential topic areas: Surprise us.. Indeed. Surprise us.

Posted by Eugene Wallingford | Permalink | Categories: Computing

September 23, 2011 3:52 PM

Grading and Learning in the Age of Social Media

Yesterday morning, I was grading the first quiz from my programming languages course and was so surprised by the responses to the first short-answer question that I tweeted in faux despair:

Wow. We discussed a particular something every day in class for three weeks. Student quiz answers give no evidence of this.

Colleagues around the globe consoled me and commiserated. But I forgot that I am also followed by several of my student, and their reaction was more like... panic. Even though I soon followed up with a tweet saying that their Scheme code made me happy, they were alarmed about that first tweet.

It's a new world. I never used to grade with my students watching or to think out loud while I was grading. Twitter changes that, unless I change me and stop using Twitter. On balance, I think I'm still better off. When I got to class, students all had smiles on their faces, some more nervous than others. We chatted. I did my best to calm them. We made a good start on the day with them all focused on the course.

We have reached the end of Week 5, one-third of the way through the course. Despite the alarm I set off in students' minds, they have performed on par with students in recent offerings over the first three homeworks and the first quiz. At this point, I am more concerned with my performance than theirs. After class yesterday, I was keenly aware of the pace of the session being out of sync with the learning curve of material. The places where I've been slowing down aren't always the best places to slow down, and the places where I've been speeding up (whether intentional or unintentional) aren't always the best places to speed up. A chat with one student that afternoon cemented my impression.

Even with years of experience, teaching is hard to get right. One shortcoming of teaching a course only every third semester is that the turnaround time on improvements is so long. What I need to do is use my realization to improve the rest of this offering, first of all this unit on recursive programming.

I spent some time early this week digging into Peter Norvig's Prescient but Not Perfect, a reconsideration of Christopher Strachey's 1966 Sci Am article and in particular Strachey's CPL program to play checkers. Norvig did usual wonderful job with the code. It's hard to find a CPL compiler these days, and has been since about 1980, so he wrote a CPL-to-Python translator, encoded and debugged Strachey's original program, and published the checkers program and a literate program essay that explains his work.

This is, of course, a great topic for a programming languages course. Norvig exemplifies the attitude I encourage to my students on Day 1: if you need a language processor, write one. It's just another program. I am not sure yet when I will bring this topic into my course; perhaps when we first talk in detail about interpreters, or perhaps when we talk about parsing and parser generators. (Norvig uses YAPPS, a Python parser generator, to convert a representation of CPL's grammar into a CPL parser written in Python.)

There are some days when I had designed all of my course sessions to be 60 minutes instead of 75 minutes, so that we had more lüft for opportunistic topics like this one. Or that I could design a free day into the course every 2-3 weeks for the same purpose. Alas, the CS curriculum depends on this course to expose students to a number of important ideas and practices, and the learning curve for some of the material is non-trivial. I'll do my best to provide at least a cursory coverage of Norvig's article and program. I hope that a few students will turn out to his approach to the world -- the computer scientist's mindset.

If nothing else, working through his paper and code excite me, and that will leak over into the rest of my work.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

September 16, 2011 4:13 PM

The Real Technological Revolution in the Classroom Hasn't Happened Yet

Earlier this month, the New York Times ran a long article exploring the topic of technology in K-12 classrooms, in particular the lack of evidence that the mad rush to create the "classroom of future" is having any effect on student learning. Standardized test scores are stagnant in most places, and in schools showing improvements, research has not been able to separate the effect of using technology from the effect of extra teacher training.

We should not be surprised. It is unlikely that simply using a new technology will have any effect on student learning. If we teach the same way and have students do the same things, then we should expect student learning to be about the same whether they are writing on paper, using a typewriter, or typing on a computer keyboard. There are certainly some very cool things one can do with, say, Keynote, and I think having those features available can augment a student's experience. But I doubt that those features can have a substantial effect on learning. Technology is like a small booster rocket for students who are already learning a lot and like training wheels for those who are not.

As I read that article, one fact screamed out at me. Computers are being used in classrooms everywhere for almost everything. Everything, that is, except the one thing that makes them useful at all: computer programming.

After reading the Times piece, Clive Thompson pulled the critical question out of it and asked, What can computers teach that textbooks and paper can't? Mark Guzdial has written thoughtfully on this topic in the past as well. Thompson offers two answers: teaching complexity and seeing patterns, (His third answer is a meta-answer, more effective communication between teacher and student.) We can improve both teaching complexity and seeing patterns by using the right software, but -- thankfully! -- Thompson points out that we can do even better if we teach kids even a little computer programming.

Writing a little code is a great vehicle for exploring a complex problem and trying to create and communicate understanding. Using or writing a program to visualize data and relationships among them is, too.

Of course, I don't have hard evidence for these claims, either. But it is human nature to want to tinker, to hack, to create. If I am going to go out on a limb without data, I'd rather do it with students creating tools that help them understand their world than with students mostly consuming media using canned tools. And I'm convinced that we can expand our students' minds more effectively by showing them how to program than by teaching most of what we teach in K-12. Programming can be tedious, like many learning tasks, and students need to learn how to work through tedium to deeper understanding. But programming offers rewards in a way and on a scale that, say, the odd problems at the end of a chapter in an algebra textbook can never do by themselves.

Mark Surman wrote a very nice blog entry this week, Mozilla as teacher, expressing a corporate vision for educating the web-using public that puts technology in context:

... people who make stuff on the internet are better creators and better online citizens if they know at least a little bit about the web's basic building blocks.

As I've written before, we do future teachers, journalists, artists, filmmakers, scientists, citizens, and curious kids a disservice if we do not teach them a little bit of code. Without this knowledge, they face unnecessary limits on their ability to write, create, and think. They deserve the ability to tinker, to hack, to trick out their digital worlds. The rest of us often benefit when they do, because some of things they create make all of our lives better.

(And increasingly, the digital world intersects with the physical world in surprising ways!)

I will go a step further than Surman's claim. I think that people who create and who are engaged with the world they inhabit have an opportunity to better citizens, period. They will be more able, more willing participants in civic life when they understand more clearly their connections to and dependence on the wider society around them. By giving them the tools they need to think more deeply and to create more broadly, education can enable them to participate in the economy of the world and improve all our lots.

I don't know if K-12 standardized test scores would get better if we taught more students programming, but I do think there would be benefits.

As I often am, I am drawn back to the vision Alan Kay has for education. We can use technology -- computers -- to teach different content in different ways, but ultimately it all comes back to new and better ways to think in a new medium. Until we make the choice to cross over into that new land, we can spend all the money we want on technology in the classroom and not change much at all.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

September 11, 2011 11:27 AM

Using Plain Text to Scrub Spreadsheet Files

For a few years, I have been saving many .xls as plain text. This is handy when I want to program against the data more often than I want to use spreadsheet tools. It also makes the data more accessible across apps and platforms, which is good now and in the future. While doing this, I have came across this technique that I find useful more generally. Maybe you will, too.

People use spreadsheets used for two purposes, structuring data and presenting data. Both Excel and Apple's Numbers offer as much or more functionality for presenting information as they do for manipulating it. For me, the presentation stuff often gets in the way of organizing and manipulating, both by cluttering the UI with commands I don't need to know about and by adding formatting information to my data. The result is a UI more complicated than I need and data files much larger than I need them to be.

When I run into one of those bloated files, I sometimes take a round trip:

  • export the file as .csv or .tsv
  • import that text file back into Numbers or OpenOffice as a spreadsheet file

The result is a clean data file, with the data and its basic structure, but nothing more. No text formatting, no variably spaced rows or columns, and no presentation widgets. When I do work with the data in the spreadsheet app, it's unadorned, just as I like it.

Posted by Eugene Wallingford | Permalink | Categories: Computing

September 09, 2011 2:19 PM

The Future of Your Data

In Liberating and future-proofing your research data, J.B. Deaton describes a recent tedious effort "to liberate several gigabytes of data from a directory of SigmaPlot files". Deaton works in Python, so he had to go through the laborious process of installing an evaluation copy of SigmaPlot, stepping through each SigmaPlot file workbook by workbook, exporting each to Excel, and then converting the files from Excel to CSV or some other format he could process in Python.

Of course, this was time spent not doing research.

All of this would have been a moot point if the data had been stored as CSV or plain text. I can open and process data stored in CSV on any operating system with a large number of tools, for free. And I am confident in 10 years time, I will be able to do the same.

This is a problem we face when we need to work with old data, as Deaton is doing. It's a problem we face when working with current data, too. I wrote recently about how doing a basic task of my job, such as scheduling courses each semester, in a spreadsheet gets in the way getting the job done as well as I might.

Had Deaton not taken the time to liberate his data, things could have been worse in the long run. Not only would the data have been unavailable to his current project, but it may well have fallen into disuse forever and eventually disappeared.

Kari Kraus wrote this week about the problem of data disappearing. One problem is the evolution of media:

When you saved that unpublished manuscript on [a 5-1/4" floppy disk], you figured it would be accessible forever. But when was the last time you saw a floppy drive?

Well, not a 5-1/4". I do have a 3-1/2" USB floppy drive at home and another in my office. But Kraus has a point. Most of the people creating data aren't CS professionals or techno-geeks. And while I do have a floppy drive, I never use them for my own data. Over the years, I've been careful to migrate my data, from floppy drives to zip drives, from CDs to large, replicated hard drives. Eventually it may live somewhere in the cloud, and I will certainly have to move it to the next new thing in hardware sometime in the future.

Deaton's problem wasn't hardware, though, and Kraus points out the bigger problem: custom encoded-data from application software:

If you don't have a copy of WordPerfect 2 around, you're out of luck.

The professional data that I have lost over the years hasn't been "lost" lost. The problem has always been with software. I, too, have occasionally wished I had a legacy of copy of WordPerfect lying around. My wife and I created a lot of files in pre-Windows WordPerfect back in the late 1980s, and I continued to use Mac versions of WP through the 1990s. As I moved to newer apps, I converted most of the files over, but every once in a while I still run across an old file in .wp format. At this point, it is rarely anything important enough to devote the time Deaton spent on his conversion experience. I choose to let that data die.

Fortunately, not all of my data from that era was encoded. I wrote most of grad school papers in nroff. That's also how I created our wedding invitations.

This is a risk we run as more of our world moves from paper to digital, even when it's just entertainment. Fortunately, for the last 5 years or so, I've been storing more and more of my data in plain text or, short of that, rich text. Like Deaton, I am pretty confident that I will be able to read and process that data 10 years hence. And, whenever possible, I have used an open file formats only policy with my colleagues.

Rather than having to liberate data in the future, it is wiser to let it live free from birth. That reduces friction now and later. Deaton offers a set of preferences that can help you keep your data as free as possible:

  • Open source beats closed source.
  • Ubiquitous beats niche software.
  • Automation/scripting beats manual processes.
  • Plain text beats binaries.
  • READMEs in every project directory.

That third bullet is good advice even if you are not a computer scientist. Deaton isn't. But you don't have to be a computer scientist to reap the benefits of a little programming!

Posted by Eugene Wallingford | Permalink | Categories: Computing

August 31, 2011 8:52 PM

Learning to Learn to Automate Work

Jon Udell recently wrote about the real problem with automating work: most of us don't know how. The problem isn't with using particular tools, which come and go, but with recognizing the power of information networks and putting data into a form from which it can be leveraged.

I want to apply his advice more often than I do. I have found that for me it is not enough simply to learn the principles.

The first challenge is fighting an uphill battle against institutional inertia. The university provides too much of its data in dead-tree form, and what data comes to us in digital form comes unstructured. Despite a desire to increase efficiency and decrease costs, a university is a big, slow organization. It takes a long time for its many bureaucracies to re-make themselves. It also takes a persistent, pervasive effort to change on the parts of many people. Too many administrators and faculty thrive in a papered society, which makes change even harder. This is the broad base of people who need to learn Udell's core principles of information networks.

The second challenge is my own habits. I may not be of my students' generation, but I've been in computer science for a long time, and I think I get the value of data. Even still, it's easy -- especially as department head -- to be sucked into institutional habits. My secretary and I are combating this by trying to convert as much data entering our office as possible into live, structured data. In the process, I am trying to teach her, a non-computer scientist, a bit about the principles of data and structured representation. We aren't worrying yet about networks and pub/sub, simply getting data into a named, structured form that supports computational processing.

Yet I need to change some of my own habits, too. When under time pressure, it's easy for me to, say, whip up assignments of graduate assistants to tasks and lab hours on a legal pad. Once the assignments are made, I can communicate a subset of the information in a couple of e-mail messages. The result is a lot of information and not a byte of structured data. Oh, and a lot of lost opportunities for using code to check consistency, make changes, publish the assignments in multiple forms, or reuse the data in adapted form next semester.

My next big opportunity to practice better what I preach is scheduling courses for spring semester. Instead of using spreadsheets as we have in the past, perhaps I should open up Dr. Racket and use it to record all the data we collect and create about the schedule. Scheme's reliance on the simple list as its primary data structure usually puts me in the mindset of grammars and syntax-driven programming. Sometimes, the best way to break a bad old habit is to create a good new one.

So, yes, we need to teach the principles of data networks in a systematic way to information technologists and everyone else. We also need to practice applying them and look for ways to help individuals and institutions alike change their habits.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Managing and Leading

August 18, 2011 3:52 PM

Some Thoughts on "Perlis Languages"

Alan Perlis

Fogus recently wrote a blog entry, Perlis Languages, that has traveled quickly through parts of software world. He bases his title on one of Alan Perlis's epigrams: "A language that doesn't affect the way you think about programming is not worth knowing." Long-time Knowing and Doing readers may remember this quote from my entry, Keeping Up Versus Settling Down. If you are a programmer, you should read Fogus's article, which lists a few languages he thinks might change how you think about programming.

There can be no single list of Perlis languages that works for everyone. Perlis says that a language is worth knowing if it affects how you think about programming. That depends on you: your background, your current stage of development as a programmer, and the kind of problems you work on every day. As an example, in the Java world, the rise of Scala and Clojure offered great opportunities for programmers to expand their thinking about programming. To Haskell and Scheme programmers, the opportunity was much smaller, perhaps non-existent.

The key to this epigram is that each programmer should be thinking about her knowledge and on the look out for languages that can expand her mind. For most of us, there is plenty of room for growth. We tend to work in one or two styles on a daily basis. Languages that go deep in a different style or make a new idea their basic building block can change us.

That said, some languages will show up lots of peoples' Perlis lists, if only because they are so different from the languages most people know and use on a daily basis. Lisp is one of the languages that used to be a near universal in this regard. It has a strangely small and consistent syntax, with symbols as first-order objects, multiple ways to work with functions, and macros for extending the language in a seamless way. With the appearance of Clojure, more and more people are being exposed to the wonders of Lisp, so perhaps won't be on everyone's Perlis list in 10 years. Fogus mentions Clojure only in passing; he has written one of the better early Clojure books, and he doesn't want to make a self-serving suggestion.

I won't offer my own Perlis list here. This blog often talks about languages that interest me, so readers have plenty of chances to hear my thoughts. I will add my thoughts about two of the languages Fogus mentions in his article.

Joy. *Love* it! It's one of my favorite little languages, and one that remains very different from what most programmers know. Scripting languages have put a lot of OOP and functional programming concepts before mainstream programmers across the board, but the idea of concatenative programming is still "out there" for most.

Fogus suggests the Forth programming language in this space. I cannot argue too strongly against this and have explained my own fascination with it in a previous entry. Forth is very cool. Still, I prefer Joy as a first step into the world of concatenative programming. It is clean, simple, and easy to learn. It is also easy to write a Joy interpreter in your favorite language, which I think is one of the best ways to grok a language in a deep way. As I mentioned in the Forth entry linked above, I spent a few months playing with Joy and writing an interpreter for it while on sabbatical a decade ago.

If you play with Joy and like it, you may find yourself wanting more than Joy offers. Then pick up Forth. It will not disappoint you.

APL. Fogus says, "I will be honest. I have never used APL and as a result find it impenetrable." Many things are incomprehensible before we try them. (A student or two will be telling me that Scheme is incomprehensible in the next few weeks...) I was fortunate to write a few programs in APL back in my undergrad programming languages course. I'm sure if I wrote a lot of APL it would become more understandable, but every time I return to the language, it is incomprehensible again to me for a while.

David Ungar told one of my favorite APL stories at OOPSLA 2003, which I mentioned in my report on his keynote address. The punchline of that story fits very well with the theme of so-called Perlis languages: "They could have done the same thing [I] did in APL -- but they didn't think of it!"

There are modern descendants of APL, but I still think there is something special about the language's unique character set. I miss the one-liners consisting or five or twenty Greek symbols, punctuation, and numbers, which accomplished unfathomable tasks such as implementing a set of accounting books.

I do second Fogus's reason for recommending APL despite never having programmed in it: creator Kenneth Iverson's classic text, A Programming Language. It is an unusually lucid account of the design of a programming language -- a new language, not an adaptation of a language we already know. Read it. I had the wonderful opportunity to meet Iverson when he spoke at Michigan State in the 1980s, as described in my entry on Iverson's passing.

... So, I encourage you to follow the spirit of Fogus's article, if not its letter. Find the languages that can change how you think, and learn them. I begin helping a new set of students on this path next week, when we begin our study of Scheme, functional programming, and the key ideas of programming languages and styles.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

August 03, 2011 7:55 PM

Psychohistory, Economics, and AI

Or, The Best Foresight Comes from a Good Model

Hari Seldon from the novel Foundation

In my previous entry, I mentioned re-reading Asimov's Foundation Trilogy and made a passing joke about psychohistory being a great computational challenge. I've never heard a computer scientist mention psychohistory as a primary reason for getting involved with computers and programming. Most of us were lucky to see so many wonderful and more approachable problems to solve with a program that we didn't need to be motivated by fiction, however motivating it might be.

I have, though, heard and read several economists mention that they were inspired to study economics by the ideas of psychohistory. The usual reason for the connection is that econ is the closest thing to psychohistory in modern academia. Trying to model the behavior of large groups of people, and reaping the advantages of grouping for predictability, is a big part of what macroeconomics does. (Asimov himself was most likely motivated in creating psychohistory by physics, which excels at predicting the behavior of masses of atoms over predicting the behavior of individual atoms.)

As you can tell from recent history, economists are no where near the ability to do what Hari Seldon did in Foundation, but then Seldon did his work more than 10,000 years in the future. Maybe 10,00 years from now economists will succeed as much and as well. Like my economist friends, I too am intrigued by economics, which also shares some important features in common with computer science, in particular a concern with the trade-offs among limited resources and the limits of rational behavior.

The preface to the third book in Asimov's trilogy, Second Foundation, includes a passage that caught my eye on this reading:

He foresaw (or he solved his [system's] equations and interpreted its symbols, which amounts to the same thing)...

I could not help but be struck by how this one sentence captured so well the way science empowers us and changes the intellectual world in which we live. Before the rapid growth of science and broadening of science education, the notion of foresight was limited to personal experience and human beings' limited ability to process that experience and generalize accurately. When someone had an insight, the primary way to convince others was to tell a good story. Foresight could be feigned and sold through stories that sounded good. With science, we have a more reliable way to assess the stories we are told, and a higher standard to which we can hold the stories we are told.

(We don't always do well enough in using science to make us better listeners, or better judges of purported foresights. Almost all of us can do better, both in professional settings and personal life.)

As a young student, I was drawn to artificial intelligence as the big problem to solve. Like economics, it runs directly into problems of limited resources and limited rationality. Like Asimov's quote above, it runs directly into the relationship between foresight and accurate models of the world. During my first few years teaching AI, I was often surprised by how fiercely my students defended the idea of "intuition", a seemingly magical attribute of men and women forever unattainable by computer programs. It did me little good to try to persuade them that their belief in intuition and "gut instinct" were outside the province of scientific study. Not only didn't they care; that was an integral part of their belief. The best thing I could do was introduce them to some of the techniques used to write AI programs and to show them such programs behaving in a seemingly intelligent manner in a situation that piqued my students' interest -- and maybe opened their minds a bit.

Over the course of teaching those early AI courses, I was eventually able to see one of the fundamental attractions I had to the field. When I wrote an AI program, I was building a model of intelligent behavior, much as Seldon's psychohistory involved building a model of collective human behavior. My inspiration did not come from Asimov, but it was similar in spirit to the inspiration my economist friends' drew from Asimov. I have never been discouraged or deterred by any arguments against the prospect of artificial intelligence, whether my students' faith-based reasons or by purportedly rational arguments such as John Searle's Chinese room argument. I call Searle's argument "purportedly rational" because, as it is usually presented, ultimately it rests on the notion that human wetware -- as a physical medium -- is capable of representing symbols in a way that silicon or other digital means cannot.

I always believed that, given enough time and enough computational power, we could build a model that approximated human intelligence as closely as we desired. I still believe this and enjoy watching (and occasionally participating in) efforts that create more and more intelligent programs. Unlike many, I am undeterred by the slow progress of AI. We are only sixty years into an enterprise that may take a few thousand years. Asimov taught me that much.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

June 06, 2011 9:12 PM

Magazines, PDF Files, and a Lifetime of Memories

I remember the day I received my first issue of Chess Life & Review magazine. It was the summer of 1979, in late June or early July. I had won a membership in the U.S. Chess Federation as part of a local goodwill tournament, by virtue of beating my good buddy and only competition for the junior prize. My victory entitled me to the membership as $20 of loot, which includes a portable set I use to this day and a notation book that records my games over a period of five or ten years.

Play it again, Sam.

That first issue arrived while I was at summer school. The cover heralded the upcoming U.S. Open championship, but inside the story of Montreal 1979, a super-GM tournament, captivated me with the games of familiar names (Karpov, Tal, Larsen, and Spassky) and new heroes (Portisch, Huuml;bner, Hort, and Ljubojevic). A feature article reviewed films of the 1940s that featured chess and introduced me to Humphrey Bogart's love of and skill at the game I loved to play. Bogart: the man's man, the tough-guy leading man at whose feet women swooned. Bogart! The authors of the magazine's regular columns became my fast friends, and for years thereafter I looked forward monthly to Andy Soltis's fun little stories, which always seemed to teach me something, and Larry Evans's Q-n-A column, which always seemed to entertain.

I was smitten, as perhaps only a young bookish kid can be.

Though I haven't played tournament chess regularly in three decades, I have remained an occasional player, a passionate programmer, and a lovestruck fan. And I've maintained my membership in the USCF, which entitles me to a monthly issue of Chess Life. Though life as husband, father, and professor leave me little time for the game I once played so much, every month I anticipate the arrival of my new issue, replete with new names and new games, tournament reports and feature articles, and regular columns that include Andy Soltis's "Chess to Enjoy". Hurray!

... which is all prelude to my current dilemma, a psychological condition that reveals me a man of my time and not a man of the future, or even the present. It's time to renew my USCF membership, and I am torn: do I opt for the membership that provides on-line access only to Chess Life?

For the last few years, ever since we moved into a new house and I cam face to face with just how much stuff I have, I've been in the process of cutting back. Even before then, I have made some society membership choices based in part on how little I need more piles of paper taking up space in my house and attention in my mind. This is the 21st century, right? I am a computer scientist, who deals daily in digital materials, who has disk space beyond his wildest dreams, whose students have effortlessly grown into a digital world that makes magazines seem like quaint compendia of the past. Right?

Yet I waffle. I can save roughly $7 a year by going paperless, which is a trifle, I know, but a prudent choice nonetheless. Right?

Undoubtedly, my CL&R-turned-CL collection takes up space. If I stem the tide of incoming issues, I can circumscribe the space needed to store my archive and devote future space to more worthy application. Perhaps I could even convert some of the archive into digital form and recoup space already spent?

This move would space, but if I am honest it does not free up all my attention. My magazines will join my music collection in the river of bits flowing into my future, being copied along from storage device to storage device, from medium to medium, and from software application to software application. I've lived through several generations of storage media, beginning in earnest with 5-1/4" floppies, and I'm sure I'll live through several more.

And what of changing formats? The text files that have followed me from college remain readable, for the most part, but not everything survives. For every few files I've converted from WordPerfect for DOS I have surely lost a file or two. Occasionally I run across one and ask myself, is it worth my time to try to open it and convert it to something more modern? I am sad to say that too often the answer is, well, no. This never happens to my books and magazines and pamphlets from that time. I choose to keep or to discard, and if I have it, I can read it. Where will PDF be in 50 years?

the cover of Bobby Fischer's 'My 60 Memorable Games'

I am also just old enough that I somewhat cherish having a life that is separate from my digital existence. When I have the chance to play chess these days, I still prefer to pull out a board and set up the pieces. The feel of the ivory or plastic or wood in my hands is part of the experience -- not essential to the experience, I suppose, in a cosmic sense, but a huge ingredient to my personal experience. I have been playing chess on computers since 1980 or so, which isn't much later than I began playing the game in earnest as in grade school, so I know that feeling, too. But feeling the pieces in my hand, poring over My 60 Memorable Games (another lifelong treasure from the booty that first brought me Chess Life) line by line in search of Bobby Fischer's magic... these are a part of the game for me.

Ultimately, that's where my renewal dilemma lies, too. My memories of checking the mailbox every day at that time of the month, eager to find the next issue of the magazine. The smell of the ink as I thumbed through the pages, peeking ahead at the delights that awaited me. The feel of the pages as I turned to the next column or article or advertisement. The joy of picking out an old issue, grabbing that magnetic portable set from 30-odd years ago, and settling into a comfortable chair for an evening of reminiscence and future-making. All are a part of what chess has been for me. A cache of PDF files, $22 over three years, and a little closet space hardly seem sufficient consideration.

Alas, we are all creatures of our own times, I no less than any man. Even though I know better, I find myself pulled backward in time just as much as Kurt Vonnegut, who occasionally waxed poetic about the future of printed book. Both Vonnegut and I realize that the future may well exceed our imaginations, but our presents retain the gifts of days past.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

May 02, 2011 3:52 PM

Thinking and Doing in the Digital Age

Last week, someone I follow tweeted this link in order to share this passage:

You will be newbie forever. Get good at the beginner mode, learning new programs, asking dumb questions, making stupid mistakes, soliciting help, and helping others with what you learn (the best way to learn yourself).

That blog entry is about inexorable change of technology in then modern world and how, if we want to succeed in this world, we need a mindset that accommodates change. We might even argue that we need a mindset that welcomes or seeks out change. To me, this is one of the more compelling reasons for us to broaden the common definition of the liberal arts to include computing and other digital forms of communication.

As much as I like the quoted passage, I liked a couple of others as much or more. Consider:

Understanding how a technology works is not necessary to use it well. We don't understand how biology works, but we still use wood well.

As we introduce computing and other digital media to more people, we need to balance teaching how to use new ideas and techniques and teaching underlying implementations. Some tools change how we work without us knowing how they work, or needing to know. It's easy for people like me to get so excited about, say, programming that we exaggerate its importance. Not everyone needs to program all the time.

Then again, consider this:

The proper response to a stupid technology is to make a better one yourself, just as the proper response to a stupid idea is not to outlaw it but to replace it with a better idea.

In the digital world as in the physical world, we are not limited by our tools. We can change how our tools work, through configuration files and scripts. We can make our own tools.

Finally, an aphorism that captures differences between how today's youth think about technology and how people my age often think (emphasis added):

Nobody has any idea of what a new invention will really be good for. To evaluate, don't think; try.

This has always been true of inventions. I doubt many people appreciated just how different the world would be after the creation of the automobile or the transistor. But with digital tools, the cost of trying things out has been driven so low, relative to the cost of trying things in the physical world, that the cost is effectively zero. In so many situations now, the net value of trying things exceeds the net value of thinking.

I know that sounds strange, and I certainly don't mean to say that we should all just stop thinking. That's the sort of misinterpretation too many people made of the tenets of extreme programming. But the simple fact is, thinking too much means waiting too long. While you are thinking -- waiting to start -- someone else is trying, learning faster, and doing things that matter.

I love this quote from Elisabeth Hendrickson, who reminded herself of the wisdom of "try; don't think" when creating her latest product:

... empirical evidence trumps speculation. Every. Single. Time.

The scientific method has been teaching us the value of empiricism over pure thought for a long time. In the digital world, the value is even more pronounced.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

April 19, 2011 6:04 PM

A New Blog on Patterns of Functional Programming

(... or, as my brother likes to say about re-runs, "Hey, it's new to me.")

I was excited this week to find, via my Twitter feed, a new blog on functional programming patterns by Jeremy Gibbons, especially an entry on recursion patterns. I've written about recursion patterns, too, though in a different context and for a different audience. Still, the two pieces are about a common phenomenon that occurs in functional programs.

I poked around the blog a bit and soon ran across articles such as Lenses are the Coalgebras for the Costate Comonad. I began to fear that the patterns on this blog would not be able to help the world come to functional programming in the way that the Gang of Four book helped the world come to object-oriented programming. As difficult as the GoF book was for every-day programmers to grok, it eventually taught them much about OO design and helped to make OO programming mainstream. Articles about coalgebras and the costate comonad are certainly of value, but I suspect they will be most valuable to an audience that is already savvy about functional programming. They aren't likely to reach every-day programmers in a deep way or help them learn The Functional Way.

But then I stumbled across an article that explains OO design patterns as higher-order datatype-generic programs. Gibbons didn't stop with the formalism. He writes:

Of course, I admit that "capturing the code parts of a pattern" is not the same as capturing the pattern itself. There is more to the pattern than just the code; the "prose, pictures, and prototypes" form an important part of the story, and are not captured in a HODGP representation of the pattern. So the HODGP isn't a replacement for the pattern.

This is one of the few times that I've seen an FP expert speak favorably about the idea that a design pattern is more than just the code that can be abstracted away via a macro or a type class. My hope rebounds!

There is work to be done in the space of design patterns of functional programming. I look forward to reading Gibbons's blog as he reports on his work in that space.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

April 14, 2011 10:20 PM

Al Aho, Teaching Compiler Construction, and Computational Thinking

Last year I blogged about Al Aho's talk at SIGCSE 2010. Today he gave a major annual address sponsored by the CS department at Iowa State University, one of our sister schools. When former student and current ISU lecturer Chris Johnson encouraged me to attend, I decided to drive over for the day to hear the lecture and to visit with Chris.

Aho delivered a lecture substantially the same as his SIGCSE talk. One major difference was that he repackaged it in the context of computational thinking. First, he defined computational thinking as the thought processes involved in formulating problems so that their solutions can be expressed as algorithms and computational steps. Then he suggested that designing and implementing a programming language is a good way to learn computational thinking.

With the talk so similar to the one I heard last year, I listened most closely for additions and changes. Here are some of the points that stood out for me this time around, including some repeated points:

  • One of the key elements for students when designing a domain-specific language is to exploit domain regularities in a way that delivers expressiveness and performance.
  • Aho estimates that humans today rely on somewhere between 0.5 and 1.0 trillion lines of software. If we assume that the total cost associated with producing each line is $100, then we are talking about a most serious investment. I'm not sure where he found the $100/LOC number, but...
  • Awk contains a fast, efficient regular expression matcher. He showed a figure from the widely read Regular Expression Matching Can Be Simple And Fast, with a curve showing Awk's performance -- quite close to Thompson NFA curve from the paper. Algorithms and theory do matter.
  • It is so easy to generate compiler front ends these days using good tools in nearly every implementation language. This frees up time in his course for language design and documentation. This is a choice I struggle with every time I teach compilers. Our students don't have as strong a theory background as Aho's do when they take the course, and I think they benefit from rolling their own lexers and parsers by hand. But I'm tempted by what we could with the extra time, including processing a more compelling source language and better coverage of optimization and code generation.
  • An automated build system and a complete regression test suite are essential tools for compiler teams. As Aho emphasized in both talks, building a compiler is a serious exercise in software engineering. I still think it's one of the best SE exercises that undergrads can do.
  • The language for quantum looks cool, but I still don't understand it.

After the talk, someone asked Aho why he thought functional programming languages were becoming so popular. Aho's answer revealed that he, like any other person, has biases that cloud his views. Rather than answering the question, he talked about why most people don't use functional languages. Some brains are wired to understand FP, but most of us are wired for, and so prefer, imperative languages. I got the impression that he isn't a fan of FP and that he's glad to see it lose out in the social darwinian competition among languages.

If you'd like to see an answer to the question that was asked, you might start with Guy Steel's StrangeLoop 2010 talk. Soon after that talk, I speculated that documenting functional design patterns would help ease FPs into the mainstream.

I'm glad I took most of my day for this visit. The ISU CS department and chair Dr. Carl Chang graciously invited me to attend a dinner this evening in honor of Dr. Aho and the department's external advisory board. This gave me a chance to meet many ISU CS profs and to talk shop with a different group of colleagues. A nice treat.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

April 06, 2011 7:10 PM

Programming for All: The Programming Historian

Knowing how to program is crucial for doing advanced research with digital sources.

This is the opening line of William Turkel's how-to for digital researchers in the humanities, A Workflow for Digital Research Using Off-the-Shelf Tools. This manual helps such researchers "Go Digital", which is, he says, the future in disciplines such as history. Turkel captures one of the key changes in mindset that faces humanities scholars as they make the move to the digital world:

In traditional scholarship, scarcity was the problem: travel to archives was expensive, access to elite libraries was gated, resources were difficult to find, and so on. In digital scholarship, abundance is the problem. What is worth your attention or your trust?

Turkel suggests that programs -- and programming -- are the only way to master the data. Once you do master the data, you can be an even more productive researcher than you were in the paper-only world.

I haven't read any of Turkel's code yet, but his how-to shows a level of sophistication as a developer. I especially like that his first step for the newly-digital is:

Start with a backup and versioning strategy.

There are far too many CS grads who need to learn the version control habit, and a certain CS professor has been bitten badly by a lack of back-up strategy. Turkel wisely makes this Job #1 for digital researchers.

The how-to manual does not currently have a chapter on programming itself, he does talk about using RSS feeds to do work for you and about and about measuring and refactoring constantly -- though at this point he is talking about one's workflow, not one's programs. Still, it's a start.

As soon as I have some time, I'm going to dig into Turkel's The Programming Historian, "an open-access introduction to programming in Python, aimed at working historians". I think there is a nice market for many books like this.

This how-to pointed me toward a couple of tools I might add to my own workflow. One is Feed43, a web-based tool to create RSS feeds for any web page (an issue I've discussed here before). On first glance, Feed43 looks a little complex for beginners, but it may be worth learning. The manual also reminded me of Mendeley an on-line reference manager. I've been looking for a new tool to manage bibliographies, so I'll give it another look.

But the real win here is a path for historians into the digital world and then into programming -- because it makes historians more powerful at what they do. Superhuman strength for everyone!

Posted by Eugene Wallingford | Permalink | Categories: Computing

March 24, 2011 10:23 PM

Teachers and Programming Languages as Permission Givers

Over spring break, I read another of William Zinsser's essays at The American Scholar, called Permission Givers. Zinsser talks about importance of people who give others permission to do, to grow, and to explore, especially in a world that offers so many freedoms but is populated with people and systems that erect barriers at every turn.

My first reaction to the paper was as a father. I have recognized our elementary and high schools as permission-denying places in a way I didn't experience them as a student myself, and I've watched running the gauntlet of college admissions cause a bright, eager, curious child to wonder whether she is good enough after all. But my rawest emotions were fear and hope -- fear that I had denied my children permission too often, and hope that on the whole I had given them permission to do what they wanted to do and become who they can be. I'm not talking about basic rules; some of those are an essential part of learning discipline and even cultivating creativity. I mean encouraging the sense of curiosity and eagerness that happy, productive people carry through life.

The best teachers are permission givers. They show students some of what is possible and then create conditions in which students can run with ideas, put them together and take them apart, and explore the boundaries of their knowledge and their selves. I marvel when I see students creating things of beauty and imagination; often, there is a good teacher to be found there as well. I'm sad whenever I see teachers who care deeply about students and learning but who sabotage their students' experience by creating "a long trail of don'ts and can'ts and shouldn'ts", by putting subtle roadblocks along the path of advancement.

I don't think that by nature I am permission giver, but over my career as a teacher I think I've gotten better. At least now I am more often aware of when I'm saying 'no' in subtle and damaging ways, so that I can change my behavior, and I am more often aware of the moments when the right words can help a student create something that matters to them.

In the time since I read the essay, another strange connection formed in my mind: Some programming languages are permission givers. Some are not.

Python is a permission giver. It doesn't erect many barriers that get in the way of the novice, or even the expert, as she explores ideas. Ruby is a permission giver, too, but not to the extent that Python is. It's enough more complex syntactically and semantically that things don't always work the way one first suspects. As a programmer, I prefer Ruby for the expressiveness it affords me, but I think that Python is the more empowering language for novices.

Simplicity and consistency seem to be important features of permission-giving languages, but they are probably not sufficient. Another of my favorite languages, Scheme, is simple and offers a consistent model of programming and computation, but I don't think of it as a permission giver. Likewise Haskell.

I don't think that the tired argument between static typing and dynamic typing is at play here. Pascal had types but it was a permission giver. Its descendant Ada, not so much.

I know many aficionados of other languages often feel differently. Haskell programmers will tell me that their language makes them so productive. Ada programmers will tell me how their language helps them build reliable software. I'm sure they are right, but it seems to me there is a longer learning curve before some languages feel like permission givers to most people.

I'm not talking about type safety, power, or even productivity. I'm talking about the feeling people have when they are deep in the flow of programming and reach out for something they want but can't quite name... and there it is. I admit, too, that I also have beginners in mind. Students who are learning to program, more than experts, need to be given permission to experiment and persevere.

I also admit that this idea is still new in mind and is almost surely colored heavily by my own personal experiences. Still, I can't shake the feeling that there is something valuable in this notion of language as permission giver.


If nothing else, Zinsser's essay pointed me toward a book I'd not heard of, Michelle Feynman's Reasonable Deviations from the Beaten Track, a collection of the personal and professional letters written by her Nobel Prize-winning father. Even in the most mundane personal correspondence, Richard Feynman tells stories that entertain and illuminate. I've only begun reading and am already enjoying it.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

March 22, 2011 4:45 PM

Encounters with Large Numbers and the Limits of Programs

Yesterday, this xkcd illustration of radiation doses made the round of Twitter. My first thought was computational: this graph is a great way to help students see what "order of magnitude" means and how the idea matters to our understanding of a real-world phenomenon.

Late yesterday afternoon, one of my colleagues stopped by to describe a Facebook conversation he had been having with a few of our students, and in particular one of out better students. This student announced that he was going to write a program to generate all possible brackets for the men's NCAA basketball tournament. My colleague said, "Um, there are 2 to the 67th power brackets", to which the student responded, "Yeah, I know, that's why I'm going to write a program. There are too many to do by hand." From this followed a discussion of just how big 2**67 is and how long it would take a program to generate all the brackets. Even using a few heuristics to trim the problem down, such as always picking a 1-seed to beat a 16-seed, the number is astronomically large. (Or, as Richard Feynman suggests, "economically large".)

Sometimes, even good students can gain a better understanding of a concept by encountering it in the wild. This is perhaps even more often true when the idea is unintuitive or beyond our usual experience.

Posted by Eugene Wallingford | Permalink | Categories: Computing

March 10, 2011 9:21 PM

SIGCSE Day 2 -- Limited Exposure

For a variety of reasons, I am scheduled for only two days at SIGCSE this year. I did not realize just how little time that is until I arrived and started trying to work in all the things I wanted to do: visit the exhibits, attend a few sessions and learn a new thing or two, and -- most important -- catch up with several good friends.

It turns out that's hard to do in a little more than a day. Throw in a bout of laryngitis in the aftermath of a flu-riddled week, and the day passed even more quickly. Here are a few ideas that stood out from sessions on either end of the day.

Opening Keynote Address

Last March I blogged about Matthias Felleisen winning ACM's Outstanding Educator Award. This morning, Felleisen gave the opening address for the conference, tracing the evolution of his team's work over the last fifteen years in a smooth, well-designed talk. One two-part idea stood out for me: design a smooth progression of teaching languages that are neither subset nor superset of any particular industrial-strength language, then implement them, so that your tools can support student learning as well as possible.

Matthias's emphasis on the smooth progression reminds me of Alan Kay's frequent references to the fact that English-speaking children learn the same language used by Shakespeare to write our greatest literature, growing into it over time. One of his goals for Smalltalk, or whatever replaces it, is a language that allows children to learn programming and grow smoothly into more powerful modes of expression as their experience and cognitive skills grow.

Two Stories from Scratch

At the end of the day, I listened in on a birds-of-a-feather session about Scratch, mostly in K-12 classrooms. One HS teacher described how his students learn to program in Scratch and then move onto a "real language". As they learn concepts and vocabulary in the new language, he connects the new terms back to their concrete experiences in Scratch. This reminded me of a story in one of Richard Feynman's books, in which he outlines his father's method of teaching young Richard science. He didn't put much stock in learning the proper names of things at first, instead helping his son to learn about how things work and how they relate to one another. The names come later, after understanding. One of the advantages of a clean language such as Scratch (or one of Felleisen's teaching languages) is that it enables students to learn powerful ideas by using them, not by memorizing their names in some taxonomy.

Later in the session, Brian Harvey told the story of a Logo project conducted back in the 1970s, in which each 5th-grader in a class was asked to write a Logo program to teach a 3rd-grader something about fractions. An assignment so wide open gave every student a chance to do something interesting, whatever they themselves knew about fractions. I need to pull this trick out of my teaching toolbox a little more often.

(If you know of a paper about this project, please send me a pointer. Thanks.)


There is one unexpected benefit of a short stay: I am not likely to leave any dynamite blog posts sitting in the queue to be written, unlike last year and 2008. Limited exposure also limits the source of triggers!

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

March 09, 2011 11:31 PM

SIGCSE Day 1 -- Innovative Approaches for Introducing CS

SIGCSE 2011 in Dallas, Texas

I'm in Dallas for a couple of days for SIGCSE 2011. I owe my presence to Jeff Forbes and Owen Astrachan, who organized a pre-conference workshop on innovative approaches for introducing computer science and provided support for its participants, courtesy of their NSF projects.

The Sheraton Dallas is a big place, and I managed to get lost on the way to the workshop this morning. As I entered the room fifteen minutes late, Owen was just finishing up talking about something called the Jinghui Rule. I still don't know what it is, but I assume it had something to do with us not being able to use our laptops during much of the day. This saves you from reading a super-long breakdown of the day, which is just as well. The group will produce a report soon, and I'm sure Jeff and Owen will do more complete job than I might -- not least of which because we all produced summaries of our discussion throughout the day, presented them to the group as a whole, and submitted them to our leaders for their use.

The topics we discussed were familiar ones, including problems, interdisciplinary approaches, integrative approaches, motivating students, and pedagogical issues. Even still, the discussions were often fresh, as most everyone in the room wrestles with these topics in the trenches and is constantly trying new things.

I did take a few notes the old-fashioned way about some things that stood out to me:

  • Owen captured the distinction between "interdisciplinary" and "integrative" well; here is my take. Interdisciplinary approaches pull ideas from other areas of study into our CS courses as a way to illustrate or motivate ideas. Integrative approaches push CS techniques out into courses in other areas of study where they become a native part of how people in those disciplines work.
  • Several times during the day people mentioned the need to "document best practices" of various sorts. Joe Bergin was surely weeping gently somewhere. We need more than disconnected best practices; we need a pattern language or two for designing certain kinds of courses and learning experiences.
  • Several times during the day talk turned to what one participant termed student-driven discovery learning. Alan Kay's dream of an Exploratorium never strays far from my mind, especially when we talk about problem-driven learning. We seem to know what we need to do!
  • A group of us discussed problems and big data in a "blue sky" session, but the talk was decidedly down-to-earth: the need to format, sanitize, and package data sets for use in the classroom.
  • One of the biggest challenges we face is the invisibility of computing today. Most everyone at the workshop today views computing's ubiquity as a great opportunity, and I often feel the same way. But I fear the reality is that, for most everyone else, computing has disappeared into the background noise of life. Convincing them that it is cool to understand how, say, Facebook works may be a tougher task than we realize.

Finally, Ge Wang demoed some of the cool things you can do with an iPhone using apps like those from Smule. Wow. That was cool.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

February 24, 2011 4:17 PM

What are Java, C++, and Prolog?

That's the correct question for the clue, "The languages used to write Watson", courtesy of the IBM Watson team answering questions in the usual way on Reddit. I'm looking forward to learning more about how Watson operates at the software level, especially how it handles the uncertainty of multiple knowledge sources working in parallel on imprecise data.

I meet with prospective students and their parents frequently as part of on- and off-campus recruiting events. Last week and this are Up Close Days on campus, when students who have been admitted to the university but not yet accepted the offer visit in hopes of making their decision. I usually try to use part of our time together to help them see what computer science is and what graduates of our program can do, because so few of either the students or parents have any idea. With the match on my mind for teaching and research, it occurred to me that the timing of the match offered a perfect opportunity.

Here was a story that everyone had heard about, the kind that captures the attention of even the least technically minded among us: a battle of man versus machine in an arena that has historical been ours alone. Watson is not only a triumph of computer science; it covers the full spectrum of what we do, from racks of servers and an open-source operating system, through algorithms and data structures and programs that embody them, to the dreams of artificial intelligence and the promise of applications that help people all over the world. So I built a short presentation around the match, showing short clips from the show, offering commentary at key moments, taking wide-eyed questions at any time, and tying what the program was doing to what computer science is and what computer science graduates do.

Last week's first session went very well. If nothing else, all of our visitors could see my excitement at what we do and what remains to be done. Sharing the thrill, indeed!

Yesterday, Lance Fortnow tweeted:

Asked: What is Computer Science? Answered: Everything that happens after you ask a question to Google until you get a result.

I don't know what all the future implications of Watson's performance will be for AI, CS, or the world. I do know that, for now, the match has given us a compelling way to talk to others about computer science ripped right from the headlines and our TV screens. When we can connect our story to something like Google, we have a chance of helping others to get what we do. When the story connects to something that has already excited our audience, our job is perhaps a little bit easier.

Posted by Eugene Wallingford | Permalink | Categories: Computing

February 18, 2011 4:51 PM

The Dawn of a New Age

Jeopardy! champ Ken Jennings passes the baton to Watson

My world this week has been agog with the Watson match on Jeopardy!. The victory by a computer over the game's two greatest champions may well signal a new era, in much the same way as the rise of the web and the advent of search. I'd like to collect my thoughts before writing anything detailed about the match itself.

In anticipation of match, last week my Intelligent Systems students and I began to talk about some of the techniques that were likely being used by Watson under the hood. When I realized how little they had learned in their AI course about reasoning in the face of uncertainty, I dusted off an old lecture from my AI course, circa 2001. With a few tweaks and additions, it held up well. Among its topics were probability and Bayes' Law. In many ways, this material is more timely today than it was then.

Early in the week, as I began to think about what this match would mean for computing and for the world, I was reminded by Peter Norvig's The Machine Age that, in so many ways, the Jeopardy! extravaganza heralds a change already in progress. If you haven't read it yet, you should.

The shift from classical AI to the data-driven AI that underlies the advances Norvig lists happened while I was in graduate school. I saw glimpses of it at conferences on expert systems in finance and accounting, where the idea of mining reams of data seemed to promise new avenues for decision makers in business. The data might be audited financial statements of public corporations or, more tantalizing, collected from grocery store scanners. But I was embedded pretty deeply in a particular way of thinking about AI, and I missed the paradigm shift.

What is most ironic for me is that my own work, which involved writing programs that could construct legal arguments using both functional knowledge of a domain and case law, was most limited by the problem that we see addressed in systems like Google and Watson: the ability to work with the staggering volume of text that makes up our case law. Today's statistical techniques for processing language make extending my work and seeing how well it works in large domains possible in a way I could only dream of. And I never even dreamed of doing it myself, such was my interest in classical AI. (I just needed a programmer, right?)

There is no question that data-driven, statistical AI has proven to be our most successful way to build intelligent systems, particularly those of a certain scale. As an engineering approach, it has won. It may well turn out to be the best scientific approach to understanding intelligence as well, but... For an old classicist like me, there is something missing. Google is more idiot savant than bon vivant; more Rain Man than Man on the Street.

There was a telling scene in one of the many short documentary films about Watson in which the lead scientist on the project said something like, "People ask me if I know why Watson gave the answer it did. I don't how it got the answer right. I don't know how it got the answer wrong." His point was that Watson is so complex and so data-rich it can surprise us with its answers. This is true, of course, and an important revelation to many people who think that, because a computer can only do what we program it to do, it can never surprise or create.

But immediately I was thinking about another sense of that response. When Watson asks "What is Toronto? in a Final Jeopardy! on "U.S. Cities", I want to ask it, "What in the world were you thinking?" If it were a human player, it might be able to tell about its reasoning process. I'd be able to learn from what it did right or wrong. But if I ask most computer programs based on statistical computations over large data sets "Why?" I can't get much more than the ranked lists of candidates we saw at the bottom of the screen on Jeopardy!

There is still a romantic part of me that wants to understand what it means to think and reason at a conscious level. Perhaps my introspection misleads me with explanations constructed post hoc, but it sure seems like I am able to think at a level above my neurons firing. That feeling is especially strong when I perform more complex tasks, such as writing a program or struggling to understand a new idea.

So, when it comes to the science of AI, I still hope for more. Maybe our statistical systems will become complex and data-rich enough that they will be able to explain their reasoning in a meaningful way. It's also possible that my desire is nothing more than a form of chauvinism for my own species, that the desire to sit around and talk about stuff, including how and why we think the way we do, is a quaint feature peculiar to humans. I don't know the answer to this question, but I can't shake the deep belief in an architecture of thought and intelligent behavior that accounts for metacognition.

In any case, it was fun to watch Watson put on such an impressive show!


If you are interested in such things, you might want to (re-)read Newell and Simon's 1975 Turing Award lecture, Computer Science as Empirical Inquiry: Symbols and Search. It's a classic of the golden era of AI.

Posted by Eugene Wallingford | Permalink | Categories: Computing

February 11, 2011 4:15 PM

DIY Empirical Analysis Via Scripting

In my post on learning in a code base, I cited Michael Feathers's entry on measuring the closure of code. Michael's entry closes with a postscript:

It's relatively easy to make these diagrams yourself. grep the log of your VCS for lines that depict adds and modifications. Strip everything except the file names, sort it, run it through 'uniq -c', sort it again, and then plot the first column.

Ah, the Unix shell. A few years ago I taught a one-unit course on bash scripting, and I used problems like this as examples in class. Many students are surprised to learn just how much you can do with a short pipeline of Unix commands, operating on plain text data pulled from any source.

You can also do this sort of thing almost as easily in a more full-featured scripting language, such as Python or Ruby. That is one reason languages like them are so attractive to me for teaching programming in context.

Of course, using a powerful, fun language in CS1 creates a new set of problems for us. A while back, a CS educator on the SIGCSE mailing list pointed out one:

Starting in Python postpones the discovery that "CS is not for me".

After years of languages such as C++, Java, and Ada in CS1, which hastened the exit of many a potential CS major, it's ironic that our new problem might be students succeeding too long for their own good. When they do discover that CS isn't for them, they will be stuck with the ability to write scripts and analyze data.

With all due concern for not wasting students' time, this is a problem we in CS should willingly accept.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

February 10, 2011 4:04 PM

This and That: Problems, Data, and Programs

Several articles caught my eye this week which are worth commenting on, but at this point none has triggered a full entry of its own. Some of my favorite bloggers do what they call "tab sweeps", but I don't store cool articles in browser tabs. I cache URLs and short notes to myself. So I'll sweep up three of my notes as a single entry, related to programming.

Programmer as Craftsman

Seth Godin writes about:

... the craftsperson, someone who takes real care and produces work for the ages. Everyone else might be a hack, or a factory guy or a suit or a drone, but a craftsperson was someone we could respect.

There's a lot of talk in the software development world these days about craftsmanship. All the conversation and all the hand-waving boil down to this. A craftsman is the programmer we all respect and the programmer we all want to be.

Real Problems...

Dan Meyer is an erstwhile K-12 math teacher who rails against the phony problems we give kids when we ask them to learn math. Textbooks do so in the name of "context". Meyer calls it "pseudocontext". He gives an example in his entry Connect These Two Dots, and then explains concisely what is wrong with pseudocontext:

Pseudocontext sends two signals to our students, both false:
  • Math is only interesting in its applications to the world, and
  • By the way, we don't have any of those.

Are we really surprised that students aren't motivated to practice and develop their craft on such nonsense? Then we do the same things to CS students in our programming courses...

... Are Everywhere These Days

Finally, Greg Wilson summarizes what he thinks "computational science" means in one of his Software Carpentry lessons. It mostly comes down to data and how we understand it:

It's all just data.

Data doesn't mean anything on its own -- it has to be interpreted.

Programming is about creating and composing abstractions.


The tool shapes the hand.

We drown in data now. We collect faster than we can understand it. There is room for more programmers, better programmers, across the disciplines and in CS.

We certainly shouldn't be making our students write Fahrenheit-to-Celsius converters or processing phony data files.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

February 07, 2011 9:03 PM

Teaching and Learning in a Code Base

In a pair of tweets today, Brian Marick offered an interesting idea for designing instruction for programmers:

A useful educational service: examine a person's codebase. Devise a new feature request that would be hard, given existing code and skill...

... Keep repeating as the codebase and skill improve. Would accelerate a programmer's skill at dealing with normal unexpected change.

This could also be a great way to help each programmer develop competencies that are missing from his or her skill set. I like how this technique would create an individualized learning for each student. The cost, of course, is in the work needed by the instructor to study the codebases and devise the feature requests. With a common set of problems to work on, over time an instructor might be able to develop a checklist of (codebase characteristic, feature request) pairs that covered a lot of the instructional space. This idea definitely deserves some more thought!

Of course, we can sometimes analyze valuable features of a codebase with relatively simple programs. Last month, Michael Feathers blogged about measuring the closure of code, in which he showed how we can examine the Open/Closed Principle in a codebase by extracting and plotting the per-file commit frequencies of source files in a project's version control repository. Feathers discussed how developers could use this information intentionally to improve the quality of their code. I think this sort of analysis could be used to great effect in the classroom. Students could see the OCP graphically for a number of projects and, combined with their programming knowledge of the projects, begin to appreciate what the OCP means to a programmer.

A serendipitous side effect would be for students to experience CS as an empirical discipline. This would help us prepare developers more readily in sync with Feathers's use of analytical data in their practice and CS grads who understand the ways in which CS can and should be an empirical endeavor.

I actually blogged a bit about studying program repositories last semester, for the purpose of understanding how to design better programming languages. That work used program repositories for research purposes. What I like about Marick's and Feathers's recent ideas is that they bring to mind how studying a program repository can aid instruction, too. This didn't occur to me so much back when one of my grad students studied relationships among open-source software packages with automated analysis of a large codebase. I'm glad to have received a push in that direction now.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

February 03, 2011 3:30 PM

Science and Engineering in CS

A long discussion on the SIGCSE members listserv about math requirements for CS degrees has drifted, as most curricular discussions seem to do, to "What is computer science?" Somewhere along the way, someone said, "Computer Science *is* a science, by name, and should therefore be one by definition". Brian Harvey responded:

The first thing I tell my intro CS students is "Computer Science isn't a science, and it isn't about computers." (It should be called "information engineering.")

I think that this assertion is wrong, at least without a couple of "only"s thrown in, but it is a great way to start a conversation with students.

I've been seeing the dichotomy between CS as science and CS as system-building again this semester in my Intelligent course. The textbook my students used in their AI course last semester is, like nearly every undergrad AI text, primarily an introduction to the science of AI: a taxonomy of concepts, results of research that help to define and delimit the important ideas. It contains essentially no pragmatic results for building intelligent systems. Sure, students learn about state-space search, logic as a knowledge representation, planning, and learning, along with algorithms for the basic methods of the field. But they are not prepared for the fact that, when they try to implement search or logical inference for a given problem, they still have a huge amount of work to do, with little guidance from the text.

In class today, we discussed this gap in two contexts: the gap one sees between low-level programming and high-level programming languages, and the difference between general-purpose languages and domain-specific languages.

My students seemed to understand my point of view, but I am not sure they really grok it. That happens best after they gain experience writing code and feel the gap while making real systems run. This is one of the reasons I'm such a believer in projects, real problems, and writing code. We don't always understand ideas until we see them in a concrete.

I don't imagine that intro CS students have any of the experience they need to understand the subtleties academics debate about what computer science is or what computer scientists. We are almost surely better off asking them to do something that matters them, whether a small problem or a larger project. In these problems and projects, students can learn from us and from their own work what CS is and how computer scientists think.

Eventually, I hope that the students writing large-ish AI programs in my course this semester learn just how much more there is to writing an intelligent system than just implementing a general-purpose algorithm from their text. The teams that are using pre-existing packages as part of they system might even learn that integrating software systems is "more like performing a heart transplant than snapping together LEGO blocks". (Thanks to John Cook for that analogy.)

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

January 31, 2011 4:12 PM

Side Thoughts on Tasks, Methods, and Intelligent Systems

I've been enjoying teaching Intelligent Systems again this semester. A big part of the fun is re-reading old papers. I had a lot of fun re-reading and thinking about the ideas in William Clancey's classic paper, Heuristic Classification. Long-time readers may recall me mentioning this paper in a long-ago post about one of computing's basic concepts, the unity of data and program.

My students didn't read this paper, but they did read a chapter from Peter Jackson's Introduction to Expert Systems that discusses it in some detail. The chapter is called "Heuristic Classification", too, but it's really about much more: the distinction between tasks and methods, the distinction between analytic tasks and synthesis tasks, and Clancey's heuristic classification method itself.

This one paper set me off on a mini-lecture that could have been much longer. To me, it felt like a technical reminiscence, a story of ideas that set the stage for my own work on design patterns of knowledge-based systems. Eventually, I had to cut my story short, both in the interest of having time to do other things in class that day and in the interest of not boring students with my own deep fascination with these topics. I hope I said enough to give them a sense of how important these ideas are and to whet the appetite of any student who might have his or her own fascination with the ideas.

Our reading and our conversation that day brought to my mind a rich set of ideas for students of intelligent systems to think about, such as:

  • The distinction between task (what) and method (how) is, at one level, no big deal. Students begin learning about this in their intro course, see its idea more clearly in their data structures course, and study the lifecycle issues involved with both in Software Engineering. But understanding the specification for a software system is often a challenge; it's easy to focus on the wrong task. The problem is made more difficult in many AI software projects because we are working in domains we understand less well, on problems at the edge of our intellectual abilities.
  • Many problems in the world are really composites of other, more primitive tasks. For example, a tutoring system must diagnose "bugs" in a student's thinking, develop a plan to fix the bug, and monitor student performance while solving problems. When we decompose a task in this way, we are in effect talking about a method for performing the task, but at a level that commits only to the exchange of information among high-level modules.
  • Where do learning systems fit into our classification of tasks. In one sense, a learning system "goes meta", because it creates a system capable of performing one of the tasks in the taxonomy. In another, though, it, too, is performing a task. But which one?

Many of these go beyond my current interests precisely because they are at the beginning of their studies, focused on building a particular system. They may find their interest in these ideas piqued after they build one of these systems, or three, and realize just how many degrees of freedom they face when writing a program that behaves "intelligently".

Posted by Eugene Wallingford | Permalink | Categories: Computing

December 15, 2010 4:30 PM

You May Be A Programmer If ... #1

... you get to the code generation part of the compiler course you are teaching. You realize that you have forgotten the toy assembly language from the textbook for its toy virtual machine, so you have to relearn it. You think, "It's gonna be fun to write programs in this language again."

Assembly language? A toy assembly language? Really?

I just like to program. Besides, after programming in Klein this semester, a language I termed as an integer assembly language, moving to a very RISC assembly doesn't seem like that big a step down. Assembly language can be fun, though I don't think I'd want to program in it for a living!

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

December 10, 2010 3:43 PM

Mean Time to Competence and Mean Time to Mastery

I'm on a mailing list with a bunch of sports fans, many of whom are also CS academics and techies. Yesterday, one of the sysadmins posted a detailed question about a problem he was having migrating a Sun Solaris server to Ubuntu, due to a conflict between default /etc/group values on the two operating systems. For example, the staff group has a value of 50 on Ubuntu and a value of 10 on Solaris. On Ubuntu, the value 10 identifies the uucp group.

Another of the sysadmins on the list wrote an even more detailed answer, explaining how group numbers are supposed to work in Unix, describing an ideal solution, and outlining a couple of practical approaches involving files such as /etc/nsswitch.conf.

After I read the question and the answer, I sent my response to the group:

Thank goodness I'm a CS professor and don't have to know how all this works.

I was joking, of course. Some people love to talk about how CS profs are disconnected from computing in the real world, and this is the sort of real-world minutia that CS profs might not know, or even have to know if they teach courses on algorithms, intro programming, or software engineering. After seeing my friends' exchange and seeing all that Unix guru-speak, I was happy to play to the stereotype.

Of course, the numbers used to implement Unix group numbers really are minutia and something only practicing sysadmins would need to know. The professor who teaches our systems courses is deeply versed in these details, as are the prof or two who manage servers for their courses and research. There certainly are CS academics divorced from reality, but you can say that of any group of people. Most know what they need to know, and a bit more.

Later in the day, I became curious about the problem my friends had discussed, so I dug in, studied a bit, and came to understand the problem and candidate solutions. Fun stuff.

Our department has recently been discussing our curriculum and in particular the goals of our B.A. and B.S. programs: What are the desired outcomes of each program? That is jargon for the simpler, "When students graduate, what would we like for them to be able to do?" For departments like ours, that means skills such as being able to write programs of a certain size, design a database, and choose appropriate data structures for solving a specific problem.

I was thinking about the Unix exchange on my mailing list in this context. Let's be honest... There is a lot of computer science, especially CS as it is applied in specific technology, that I don't know. Should I have known to fix my friend's problem? Should our students? What can we reasonably expect of our students or of ourselves as faculty?

Obviously, we professors can't know everything, and neither can our students. That is true in any discipline and especially in one like CS, which changes and grows so fast. This is one of the reasons it is so important for us to define clearly what we expect our programs to achieve. The space of computing knowledge and skills is large and growing. Without a pretty good idea of what we are hoping to achieve with our courses and curricula, our students could wander around in the space aimlessly and not having anything coherent to show for the time or effort. Or the money they spent paying tuition.

So, when I first read my friends' messages about Unix groups, I didn't know how to solve the problem. And that's okay, because I can't know everything about CS, let alone every arcane detail of every Unix distro out in the world. But I do have a CS degree and lots of experience. What difference should that make when I approach problems like this? If one of our graduates confronts this situation or one like it, how will they be difference from the average person on the street, or even the average college graduate?

Whatever specific skills our graduates have, I think that they should be able to come up to speed on computing problems relatively quickly. They should have enough experience with a broad set of CS domains and enough theoretical background to be able to make sense of unfamiliar problems and understand candidate solutions. They should be able to propose solutions that make sense to a computer scientist, even if the solutions lack a detailed knowledge of the domain.

That is, CS graduates should have a relatively low mean time to competence in most sub-areas of computing, even the ones they have not studied in detail yet.

For a smaller set of sub-areas, our students should also have a relatively low mean time to mastery. These are the areas they have studied in some detail, either in class or through project work, but which they have not yet mastered. A CS degree should put them in a position to master them more quickly than most educated non-computer scientists.

Mean time to competence (MTTC) and mean time to mastery (MTTM) are actually a big part of how I distinguish a university education from a community college education when I speak to prospective students and their parents, though I have never used those terms before. They always wonder about the value of technical certifications, which community college programs often stress, and why my department does not make study for certification exams an explicit goal for students.

We hope, I tell them, to put the student in a position of being ready to prepare for any certification exam in relatively short order, rather than spending a couple of years preparing them to take a specific exam. We also hope that the knowledge and experience they gain will prepare them for the inevitable developments in our discipline that will eventually make any particular certification obsolete.

I am not certain if mean time to competence and mastery are student learning outcomes in the traditional educational jargon sense of the word, or whether they are abstractions of several more concrete outcomes. In any case, I am left thinking about how we can help to create these outcomes in students and how we can know whether we are successful or not. (The agile developer in me says, if we can't figure out how to test our program, we need to break the feature down into smaller, more concrete steps.)

Whatever the practical challenges of curriculum design and outcomes, I think MTTC and MTTM are essential results for a CS program to generate. Indeed, they are the hallmark of a university education in any discipline and of education in general.

Now, to figure out how to do that.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 30, 2010 5:08 PM

No More Bugs

Maurice Wilkes, one of computer science's pioneers, passed away yesterday at the grand age of 97. Wilkes spent most of his career working on computer systems, so I never studied his work in the detail that I studied Backus's or Floyd's. He is perhaps best known as the creator of the EDSAC, the first practical (if slow!) stored program computer, in May 1949. Wilkes also had an undeniable impact on programming as the primary developer of the idea of microprogramming, whereby the CPU is controlled by a program stored in ROM.

Finally, every programmer must hold a special place in his or her heart for this, Wilkes's most quoted aphorism:

... the realization came over me that a good part of my life was going to be spent finding errors in my own programs.

All programmers have their Wilkes Moment sooner or later.

Posted by Eugene Wallingford | Permalink | Categories: Computing

November 19, 2010 4:45 PM

Debugging the Law

Debugging the Law

Recently, Kevin Carey's Decoding the Value of Computer Science got a lot of play among CS faculty I know. Carey talks about how taking a couple of computer programming courses way back at the beginning of his academic career has served him well all these years, though he ended up majoring in the humanities and working in the public affairs sector. Some of my colleagues suggested that this article gives great testimony about the value of computational thinking. But note that Carey didn't study abstractions about computation or theory or design or abstraction. He studied BASIC and Pascal. He learned computer programming.

Indeed, programming plays a central role in the key story within the story. In his first job out of grad school, Carey encountered a convoluted school financing law in my home state of Indiana. He wrote code to simulate the law in SAS and, between improving his program and studying the law, he came to understand the convolution so well that he felt confident writing a simpler formula "from first principles". His formula became the basis of an improved state law.

That's right. His code was so complicated and hard to maintain, he through the system away and wrote a new one. Every programmer has lived this experience with computer code. Carey tried to debug a legal code and found its architecture to be so bad that he was better off creating a new one.

CS professors should use this story every time they try to sell the idea of universal computer programming experience to the rest of the university!

The idea of refactoring legal code via a program that implements it is not really new. When I studied logic programming and Prolog in grad school, I read about the idea of expressing law as a Prolog programming and using the program to explore its implications. Later, I read examples where Prolog was used to do just that. The AI and law community still works on problems of this sort. I should dig into some of the recent work to see what progress, if any, has been made since I moved away from that kind of work.

My doctoral work involved modeling and reasoning about legal arguments, which are very much like computer programs. I managed to think in terms of argumentation patterns, based on the work of Stephen Toulmin (whose work I have mentioned here before). I wish I had been smart or insightful enough to make the deeper connection from argumentation to software development ideas such as architecture and refactoring. It seems like there is room for some interesting cross-fertilization.

(As always, if you know about work in this domain, please let me know!)

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 18, 2010 3:43 PM

The Will to Run, or Do Anything Else

In "How Do You Do It?", an article in the latest issue of Running Times about how to develop the intrinsic motivation to do crazy things like run every morning at 5:00 AM, ultrarunner Eric Grossman writes:

The will to run emerges gradually where we cultivate it. It requires humility -- we can't just decide spontaneously and make it happen. Yet we must hold ourselves accountable for anything about which we can say, "I could have done differently."

Cultivation, humility, patience, commitment, accountability -- all features of developing the habits I need to run on days I'd rather stay in bed. After a while, you do it, because that's what you do.

I think this paragraph is true of whatever habit of thinking an doing that you are trying to develop, whether it's object-oriented programming, playing piano, or test-driven design.


Eugene speaking at Tech Talk Cedar Valley, 2010/11/17

Or functional programming. Last night I gave a talk at Tech Talk Cedar Valley, a monthly meet-up of tech and software folks in the region. Many of these developers are coming to grips with a move from Java to Scala and are peddling fast to add functional programming style to their repertoires. I was asked to talk about some of the basic ideas of functional programming. My talk was called "Don't Drive on the Railroad Tracks", referring to Bill Murray's iconic character in the movie Groundhog Day. After hundreds or thousands of days reliving February 2 from the same starting point, Phil Connors finally comes to understand the great power of living in a world without side effects. I hope that my talk can help software developers in the Cedar Valley reach that state of mind sooner than a few years from now.

If you are interested, check out the slides of the talk (also available on SlideShare) and the code. both Ruby and Scheme, that I used to illustrate some of the ideas.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General

November 13, 2010 9:28 AM

Shape, or Be Shaped

Playwright Arthur Miller is often quoted as saying:

Man must shape his tools lest they shape him.

I read this again yesterday, in the online book Focus, while listening to faculty at a highly-ranked local school talk about the value of a liberal arts education. The quote reminds me about one of the reasons I so like being a computer scientist. I can shape my tools. If I need a new tool, or even a new kind of tool, I can make it.

Our languages are tools, too. We can shape them, grow them, change them. We can create new ones. (Thanks, Matz.)

Via the power of the Internet I am continuously surrounded by colleagues smarter and more motivated than I doing the same. I've been enjoying watching Brian Marick tweet about his thoughts and decision making as he implements Midje in Clojure. His ongoing dialog reminds me that I do not have to settle.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

November 03, 2010 2:12 PM

Ideas from Readers on Recent Posts

A few recent entries have given rise to interesting responses from readers. Here are two.

Fat Arrows

Relationships, Not Characters talked about how the most important part of design often lies in the space between the modules we create, whether objects or functions, not the modules themselves. After reading this, John Cook reminded me about an article by Thomas Guest, Distorted Software. Near the end of that piece, which talks about design diagrams, Guest suggests that the arrows in application diagrams should be larger, so that they would be proportional to the time their components take to develop. Cook says:

We typically draw big boxes and little arrows in software diagrams. But most of the work is in the arrows! We should draw fat arrows and little boxes.

I'm not sure that would make our OO class diagrams better, but it might help us to think more accurately!

My Kid Could Do That

Ideas, Execution, and Technical Achievement wistfully admitted that knowing how to build Facebook or Twitter isn't enough to become a billionaire. You have to think to do it. David Schmüdde mentioned this entry in his recent My Kid Could Do That, which starts:

One of my favorite artists is Mark Rothko. Many reject his work thinking that they're missing some genius, or offended that others see something in his work that they don't. I don't look for genius because genuine genius is a rare commodity that is only understood in hindsight and reflection. The beauty of Rothko's work is, of course, its simplicity.

That paragraph connects with one of the key points of my entry: Genius is rare, and in most ways irrelevant to what really matters. Many people have ideas; many people have skills. Great things happen when someone brings these ingredients together and does something.

Later, he writes:

The real story with Rothko is not the painting. It's what happens with the painting when it is placed in a museum, in front of people at a specific place in the world, at a specific time.

In a comment on this post, I thanked Dave, and not just because he discusses my personal reminiscence. I love art but am a novice when it comes to understanding much of it. My family and I saw an elaborate Rothko exhibit at the Smithsonian this summer. It was my first trip to the Smithsonian complex -- a wonderful two days -- and my first extended exposure to Rothko's work. I didn't reject his art, but I did leave the exhibit puzzled. What's the big deal?, I wondered. Now I have a new context in which to think about that question and Rothko's art. I didn't expect the new context to come from a connection a reader made to my post on tech start-up ideas that change the world!

I am glad to know that thinkers like Schmüdde are able to make connections like these. I should note that he is a professional artist (both visual and aural), a teacher, and a recovering computer scientist -- and a former student of mine. Opportunities to make connections arise when worlds collide.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Personal, Software Development

October 28, 2010 4:27 PM

Ideas, Execution, and Technical Achievement

Four or five years ago, my best buddy on campus and I were having lunch at our favorite Chinese buffet. He looked up between bites of General Tsao's and asked, "Why didn't you and I sit down five years ago and write Facebook?"

You see, he is an awesome programmer and has worked with me enough to know that I do all right myself. At various times, both of us have implemented bits and pieces of the technology that makes up Facebook. It doesn't look like all that big a deal.

I answered, "Because we didn't think of it."

The technical details may or may not have been a big deal. Once implemented, they look straightforward. In any case, though, the real reason was that it never occurred to us to write Facebook. We were techies who got along nicely with the tools available to us in 1999 or 2000, such as e-mail, wiki, and the web. If we needed to improve our experience, we did so by improving our tools. Driven by one of his own itches, Troy had done his M.S. research with me as his advisor, writing a Bayesian filter to detect spam. But neither of us thought about supplanting e-mail with a new medium.

We had the technical skills we needed to write Facebook. We just didn't have the idea of Facebook. Turns out, that matters.

That lunch conversation comes into my mind every so often. It came back yesterday when I read Philip Greenspun's blog entry on The Social Network. Greenspun wrote one of my favorite books, Philip and Alex's Guide to Web Publishing, which appeared in 1998 and which describes in great detail (and with beautiful photos) how to implement web community software. When his students ask how he feels about Zuckerberg getting rich without creating anything "new", Greenspun gives a wonderfully respectful and dispassionate answer: "I didn't envision every element of Facebook." Then he explains what he means.

Technically, Greenspun was positioned as well or better than my buddy and I to write Facebook. But he didn't the idea, either, at least not to the same level as Zuckerberg. Having the core of an idea is one thing. Developing it to the point that it becomes a platform that changes the world in which it lives is another. Turns out, that matters, too.

I like Lawrence Lessig's most succinct summation of what makes Zuckerberg writing Facebook a notable achievement: He did it. He didn't just have an idea, or talk about it, or dream about it. He implemented it.

That's what great hackers do.

Watch this short video to hear Zuckerberg himself say why he built. His answer is also central to the hacker ethic: Because he wanted to.

(Also read through to the end of Lessig's article for a key point that many people miss when they think about the success and achievement of things like Facebook and Twitter and Napster: The real story is not the invention.

Zuckerberg may or may not be a genius? I don't know or care. That is a word that modern culture throws around far too carelessly these days. I will say this. I don't think that creating Facebook is in itself sufficient evidence for concluding so. A lot of people have cool ideas. A more select group of people write the code to make their ideas come alive. Those people are hackers. Zuckerberg is clearly a great hacker.

I'm not a big Facebook user, but it has been on my mind more than usual the last couple of days. Yesterday was my birthday, and I was overwhelmed by all the messages I received from Facebook friends wishing me a happy day. They came from all corners of the country; from old grade-school friends I haven't seen in over thirty years; from high school and college friends; from professional friends and acquaintances. These people all took the time to type a few words of encouragement to someone hundreds of miles away in the middle of the Midwest. I felt privileged and a little humbled.

Clearly, this tool has made the world a different place. The power of the social network lies in the people. The technology merely enables the communication. That's a significant accomplishment, even if most of the effects are beyond what the creator imagined. That's the power of a good idea.


All those years ago, my buddy and I talked about how little technical innovation there was in Facebook. Greenspun's answer reminds us that there was some. I think there is another element to consider, something that was a driving force at StrangeLoop: big data. The most impressive technical achievement of Facebook and smaller web platforms such as Twitter is the scale at which they operate. They've long ago outgrown naive implementations and have had to try to offer uninterrupted service in the face of massive numbers of users and exponential growth. Solving the problems associated with operating at such a scale is an ongoing technical challenge and a laudable achievement in its own right.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

October 25, 2010 4:50 PM

A Programming Digression: Farey Sequences

Farey sequences as Ford circles

Last Wednesday, I saw John Cook's article on rational approximations using Farey's algorithm, and it led me on a programming journey. You might find it entertaining.

First, some background. The students in my compilers course are writing a compiler for a simple numerical language I call Klein. Klein is a minimal functional language inspired by Doug Baldwin's MinimL, designed to give compiler students experience with all the important issues in the course while not duplicating very much. The goal is that students be able to write a complete and correct compiler for the language from scratch in a semester. Among Klein's limitations are:

  • Repetition is done with recursion. There are no loops.
  • There are only two data types, integers and booleans.
  • All binding of values to names is done via function calls. There are no local variables, no assignments statements, and no sequences.
  • There is only one primitive function, print, that can be called only at the top of a function definition.

As you can imagine, it is difficult to find good, large problems for which we can write Klein programs. So I'm always on the look-out for numerical algorithms that can be implemented in Klein.

When I saw Cook's article and Python program, I got to thinking... Wouldn't this be a cool application for Klein?

Of course, I faced a few problems. Cook's program takes as arguments a real number in [0..1), r, and an integer N. It returns a fraction that is the rational number closest in value to r with a denominator that no bigger than N. Klein does not have floating-point values, so I would have to fake even the real value being approximated! Without assignment statements and sequences of statements, let alone Python's multiple assignments and multiple returns, I would have to convert the algorithm's loop to a set of functions for computing the individual components of the converging bounds. Finally, without floats again, I would also have to compare the bounding fractions to the faked real number as fractions.

Those are just the sort of challenges that intrigue a programmer. I spent a couple of hours on Thursday slowly refactoring Cook's code to a Klein-like subset of Python. Instead of passing r to the main function, I fake it by passing in two arguments: the digits to the right of the decimal point as an integer, and a power of 10 to indicate its magnitude. These serve as numerator and denominator of a rational approximation, albeit with a very denominator. For example, r = 0.763548745, I pass in 7635487 and 10,000,000. The third argument to the function is N. Here is an example:

     >>> farey( 127, 1000, 74)

8/63 is the best rational approximation for 0.127 with a denominator ≤ 74. My code prints the 8 and returns the 63, because that is best way for an equivalent Klein program to work.

Here is Cook's example of 1/e with a denominator bounded by 100:

     >>> farey( 367879, 1000000, 100 )

Take a look at my Python code if you are easily amused. In one part of the program, I duplicated code with only small changes to the resulting functions, farey_num() andfarey_den. In another section, I avoided creating duplicate functions by adding a selector argument that I allowed me to use one bit of code for two different calculations. while_loop_for returns the next value for one of a, b, c, and d, depending on the value of its first argument. For some reason, I am fond of the code that replaces Cook's mediant variable.

At this point, the port to Klein was straightforward.

In the end, there is so much code to do what is basically a simple task: to compute row N of this tree on an as-needed basis, in the service of a binary search:

Farey sequences as a lattice

Programming in Klein feels a lot like programming in an integer assembly language. That can be an interesting experience, but the real point of the language is to exercise students as they write a compiler. Still, I find myself now wanting to implement refactoring support for Klein, in order to make such digressions flow faster in the future!

I enjoyed this little diversion so much that I've been buttonholing every colleague who walks through the door. I decided I'd do the same to you.

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 21, 2010 4:45 PM

On Kazoos and Violins

Coming out of Strange Loop, I have been reading more about Scala and Clojure, both of which bring a lot of cool functional programming ideas to the JVM world. A lot of people have seem to have made deeper inroads into FP via Scala than via Clojure. That is certainly true among the companies in my area. That is not too surprising. Scala looks more like what they are used to than Clojure. It feels more comfortable.

This context is probably one of the reasons that I so liked this quote I read yesterday from Brenton Ashworth, a serial contributor to the Clojure community:

I'm now convinced that choosing a language based on how easy it is to read or write is a very bad idea. What if we chose musical instruments based on how easy they are to play? Everyone would playing kazoos. You can't think deeply about music if all you know is a kazoo.

This reminds me of a story Alan Kay often tells about violins. I have alluded to it at least once before:

And we always need to keep in mind the difference between essential and gratuitous complexity. I am often reminded of Alan Kay's story about learning to play violin. Of course it's hard to learn. But the payoff is huge.

New tools often challenge us, because knowing them well will change us. And such change is a part of progress. Imagine if no one had wanted to learn to fly an airplane because it was too different from driving a car or a horse.

Don't get me wrong: I am not comparing Scala to a kazoo and Clojure to a violin! I don't know either language well enough to make grand pronouncements about them, but my small bit of study tells me that both are powerful, valuable languages, languages worth knowing. I'm simply concerned that too many people opt out of learning Clojure well because of how it looks. As Ashworth says in the sentence immediately following the passage quoted above, "Skill at reading and writing code is learned."

You can do it! Don't settle on something only because something else looks unfamiliar. By taking the easier path in this moment, you may be making your path harder in the long run.

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 21, 2010 8:50 AM

Strange Loop Redux

StrangeLoop 2010 logo

I am back home from St. Louis and Des Moines, up to my next in regular life. I recorded some of my thoughts and experiences from Strange Loop in a set of entries here:

Unlike most of the academic conferences I attend, Strange Loop was not held in a convention center or in a massive conference hotel. The primary venue for the conference was the Pageant Theater, a concert nightclub in the Delmar Loop:

The Pageant Theater

This setting gave the conference's keynotes something of an edgy feel. The main conference lodging was the boutique Moonrise Hotel a couple of doors down:

The Pageant Theater

Conference session were also held in the Moonrise and in the Regional Arts Commission building across the street. The meeting rooms in the Moonrise and the RAC were ordinary, but I liked being in human-scale buildings that had some life to them. It was a refreshing change from my usual conference venues.

It's hard to summarize the conference in only a few words, other than perhaps to say, "Two thumbs up!" I do think, though, that one of the subliminal messages in Guy Steele's keynote is also a subliminal message of the conference. Steele talked for half an hour about a couple of his old programs and all of his machinations twenty-five or forty years to make them run in the limited computing environments of those days. As he went to all the effort to reconstruct the laborious effort that went into those programs in the first place, the viewer can't help but feel that the joke's on him. He was programming in the Stone Age!

But then he gets to the meat of his talk and shows us that how we program now is the relic of a passing age. For all the advances we have made, we still write code that transitions from state to state to state, one command at a time, just like our cave-dwelling ancestors in the 1950s.

It turns out that the joke is on us.

The talks and conversations at Strange Loop were evidence that one relatively small group of programmers in the midwestern US are ready to move into the future.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Software Development

October 16, 2010 7:59 PM

Strange Loop This and That

Some miscellaneous thoughts after a couple of days in the mix...

Pertaining to Knowing and Doing

** Within the recurring theme of big data, I still have a few things to study: MongoDB, CouchDB, and FlockDB. I also learned about Pig, a language I'd never heard of before. I think I need to learn it.

** I need to be sure that my students learn about the idea of multimethods. Clojure has brought them back into mainstream discussion.

** Kevin Weil, who spoke about NoSQL at Twitter, told us that his background is math and physics. Not CS. Yet another big-time developer who came to programming from another domain because they had real problems to solve.

Pertaining to the Conference

** The conference served soda all day long, from 8:00 in the morning to the end of the day. Hurray! My only suggestion: add Diet Mountain Dew to the selections.

** The conference venues consist of two rooms in a hotel, two rooms in a small arts building, and the auditorium of the Pageant Theater. The restrooms are all small. During breaks, the line for the men's room was, um, long. The women in attendance came and went without concern. This is exactly opposite of what one typically sees out in public. The women of Strange Loop have their revenge!

** This is the first time I have ever attended a conference with two laptop batteries. And I saw that it was good. Now, I just have to find out why every couple of weeks my keyboard and trackpad freeze up and effectively lock me out. Please, let it not be a failing mother board...

Pertaining to Nothing But Me

** Like every conference, Strange Loop fills the silence between sessions with a music loop. The music the last two days has been aimed at its audience, which is mostly younger and mostly hipper than I am. I really enjoyed it. I even found a song that will enter my own rotation, "Hey, Julie" by Fountains of Wayne. You can, of course, listen to it on YouTube. I'll have to check out more Fountains of Wayne later.

** On Twitter, I follow a relatively small number of people, mostly professional colleagues who share interesting ideas and links. I also follow a few current and former students. Rounding out the set are a couple connections I made with techies through others, back when Twitter was small. I find that I enjoy their tweets even though I don't know them, or perhaps because I don't.

On Thursday, it occurred to me: Maybe it would be fun to follow some arbitrary incredibly popular person. During one of the sessions, we learned that Lady Gaga has about 6,500,000 followers, surpassing Ashton Kutcher's six million. I wonder it would be like to have their tweets flowing in a stream with those of Brian Marick and Kevlin Henney, Kent Beck and Michael Feathers.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

October 16, 2010 9:09 AM

Strange Loop, Day 2 Afternoon

After eating some dandy deep-dish BBQ chicken at Pi Pizzeria with the guys from T8 Webware (thanks, Wade!), I returned for a last big run of sessions. I'll save the first session for last, because my report of it is the longest.

Android Squared

I went to this session because so many of my students want me to get down in the trenches with phone programming. I saw a few cool tools, especially RetroFit, a new open-source framework for Android. There are not enough hours in a day for me to explore every tool out there. Maybe I can have a student do an Android project.

Java Puzzlers

And I went to this session because I am weak. I am a sucker for silly programming puzzles, especially ones that take advantage of the dark corners of our programming languages. This session did not disappoint in this regard. Oh, the tortured code they showed us! I draw from this experience a two-part moral:

  1. Bad programmers can write really bad code, especially in a complex language.
  2. A language that is too complex makes bad programmers of us all.

Brian Marick on Outside-In TDD

Marick demoed a top-down | outside-in style of TDD in Clojure using Midje, his homebrew test package. This package and style make heavy use of mock objects. Though I've dinked around a bit in Scala, I've done almost nothing in Clojure, so I'll have to try this out. The best quote of the session echoed a truth about all programming: You should have no words in your test that are not specifically about the test.

Douglas Crockford, Open-Source Heretic

Maybe my brain was fried by the end of the two days, or perhaps I'm simply not clever enough. While I able to chuckle several times through this closing keynote, I never saw the big picture or the point of the talk. There were plenty of interesting, if disconnected, stories and anecdotes. I enjoyed Crockford's coverage of several historical mark-up languages, including Runoff and Scribe. (Runoff was the forebear of troff, a Unix utility I used throughout grad school -- I even produced my wedding invitations using it! Fans of Scribe should take a look at Scribble, a mark-up tool built-on top of Racket.) He also told an absolutely wonderful story about Grace Murray Hopper's A-0, the first compiler-like tool and likely the first open-source software project.

Panel on the Future of Languages

Panels like this often don't have a coherent summary. About all I can do is provide a couple of one-liners and short answers to a couple of particularly salient questions.

Joshua Bloch: Today's patterns are tomorrow's language features. Today's bugs are tomorrow's type system features.

Douglas Crockford: Javascript has become what Java was meant to be, the language that runs everywhere, the assembly language of the web.

Someone in the audience asked, "Are changes in programming languages driven by academic discovery or by practitioner pain?" Guy Steele gave the best answer: The evolution of ideas is driven by academics. Uptake is driven by practitioner needs.

So, what is the next big thing in programming languages? Some panelists gave answers grounded in today's problems: concurrency, a language that could provide and end-to-end solution for the web, and security. One panelists offered laziness. I think that laziness will change how many programmers think -- but only after functional programming has blanketed the mainstream. Collectively, several panelists offered variations of sloppy programming, citing as early examples Erlang's approach to error recovery, NoSQL's not-quite-data consistency, and Martin Rinard's work on acceptability-oriented computing.

The last question from the audience elicited some suggestions you might be able to use. What language, obscure or otherwise, should people learn in order to learn the language you really want them to learn? For this one, I'll give you a complete list of the answers:

  • Io. (Bruce Tate)
  • Rebol. (Douglas Crockford)
  • Forth. Factor. (Alex Payne)
  • Scheme. Assembly. (Josh Bloch)
  • Clojure. Haskell. (Guy Steele)

I second all of these suggestions. I also second Steele's more complete answer: Learn any three languages you do not know. The comparisons and contrasts among them will teach you more than any one language can.

Panel moderator Ted Neward closed the session with a follow-up question: "But what should the Perl guys learn while they are waiting for Perl 6?" We are still waiting for the answer.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

October 15, 2010 11:46 PM

Strange Loop, Day 2 Morning

Perhaps my brain is becoming overloaded, or I have been less disciplined in picking talks to attend, or the slate of sessions for Day 2 is less coherent than Day 1's. But today has felt scattered, and so far less satisfying than yesterday. Still, I have had some interesting thoughts.

Billy Newport on Enterprise NoSQL

This was yet another NoSQL talk, but not, because it was different than the preceding ones at the conference. This talk was not about any particular technologies. It was about mindsets.

Newport explained that NoSQL means not only SQL. These two general approaches to data storage offer complementary strengths and weaknesses. This means that they are best used in different contexts.

I don't do enough programming for big data apps to appreciate all the details of this talk. Actually, I understood most of the basic concepts, but they soon starting blurring in my mind, because I don't have personal experience on which to hang them. A few critical points stood out:

  • In the SQL world, the database is the "system of record" for all data, so consistency is a given. In the NoSQL world, having multiple systems of record is normal. In order to ensure consistency, the application uses business rules to bring data back into sync. This requires a big mind shift for SQL guys.

  • In the SQL world, the row is a bottleneck. In the NoSQL world, any node can handle the request. So there is not a bottleneck, which means the NoSQL approach scales transparently. J But see the first bullet.

These two issues are enough to see one of Newport's key points. The differences between the two worlds is not only technical but also cultural. SQL and NoSQL programmers use different vocabulary and have different goals. Consider that "in NoSQL, 'query' is a dirty word". NoSQL programmers do everything they can to turn queries into look-ups. For the SQL programmer, the query is a fundamental concept.

The really big idea I took away from this talk is that SQL and NoSQL solve different problems. The latter optimizes for one dominant question, while the former seeks to support an infinite number of questions. Most of the database challenges facing NoSQL shops boil down to this: "What happens if you ask a different question?"

Dean Wampler on Scala

The slot in which this tutorial ran was longer than the other sessions at the conference. This allowed Wampler to cover a lot of details about Scala. I didn't realize how much of an "all but the kitchen sink" language Scala is. It seems to include just about every programming language feature I know about, drawn from just about every programming language I know about.

I left the talk a bit sad. Scala contains so much. It performs so much syntactic magic, with so many implicit conversions and so many shortcuts. On the one hand, I fear that large Scala programs will overload programmers' minds the way C++ does. On the other, I worry that its emphasis on functional style will overload programmers' minds the way Haskell does.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

October 15, 2010 12:34 PM

Guy Steele's Own Strange Loop

The last talk of the afternoon of Day 1 was a keynote by Guy Steele. My notes for his talk are not all that long, or at least weren't when I started writing. However, as I expected, Steele presented a powerful talk, and I want to be able to link directly to it later.

Steele opened with a story about a program he wrote forty years ago, which he called the ugliest program he ever wrote. It fit on a single punch card. To help us understand this program, he described in some detail the IBM hardware on which it ran. One problem he faced as a programmer is that the dumps were undifferentiated streams of bytes. Steele wanted line breaks, so he wrote an assembly language program to do that -- his ugliest program.

Forty years later, all he has is the punch card -- no source. Steele's story then turned into CSI: Mainframe. He painstakingly reverse-engineered his code from punches on the card. We learned about instruction format, data words, register codes... everything we needed to know how this program managed to dump memory with newlines and fit on a single card. The number of hacks he used, playing on puns between op codes and data and addresses, was stunning. That he could resurrect these memories forty years later was just as impressive.

I am just old enough to have programmed assembly for a couple of terms on punch cards. This talk brought back memories, even how you can recognize data tables on a card by the unused rows where there are no op codes. What a wonderful forensics story.

The young guys in the room liked the story, but I think some were ready for the meet of the talk. But Steele told another, about a program for computing sin 3x on a PDP-11. To write this program, Steele took advantage of changes in the assembly languages between the IBM mainframe and the PDP-11 to create more readable code. Still, he had to use several idioms to make it run quickly in the space available.

These stories are all about automating resource management, from octal code to assemblers on up to virtual memory and garbage collection. These techniques let the programmer release concerns about managing memory resources to tools. Steele's two stories demonstrate the kind of thinking that programmers had to do back in the days when managing memory was the programmer's job. It turns out that the best way to think about memory management is not to think about it at all.

At this point, Steele closed his own strange loop back to the title of his talk. His thesis is this: the best way to think about parallel programming is not to have to.

If we program using a new set of idioms, then parallelism can be automated in our tools. The idioms aren't about parallelism; they are more like functional programming patterns that commit the program less to underlying implementation.

There are several implications of Steele's thesis. Here are two:

  • Accumulators are bad. Divide and conquer is good.

  • Certain algebraic properties of our code are important. Programmers need to know and preserve them in then code they write.

Steele illustrated both of these implications by solving an example problem that would fit nicely in a CS1 course: finding all the words in a string. With such a simple problem, everyone in the room has an immediate intuition about how to solve it. And nearly everyone's intuition produces a program using accumulators that violates several important algebraic properties that our code might have.

One thing I love about Steele's talks: he grounds ideas in real code. He developed a complete solution to the problem in Fortress, the language Steele and his team have been creating at Sun/Oracle for the last few years. I won't try to reproduce the program or the process. I will say this much. One, the process demonstrated a wonderful interplay between functions and objects. Two, in the end, I felt like we had just used a process very similar to the one I use when teaching students to create this functional merge sort function:

    (define mergesort
       (lambda (lst)
          (merge-all (map list lst))))

Steele closed his talk with the big ideas that his programs and stories embody. Among the important algebraic properties that programs should have whenever possible are ones we all learned in grade school, explicitly or implicitly. Though they may still sound scary, they all have intuitive common meanings:

  • associative -- grouping don't matter
  • commutative -- order doesn't matter
  • idempotent -- duplicates don't matter
  • identity -- this value doesn't matter
  • zero -- other values don't matter

Steele said that "wiggle room" was the key buzzword to take away from his talk. Preserving invariants of these algebraic properties give the compiler wiggle room to choose among alternative ways to implement the solution. In particulars, associativity and commutativity give the compiler wiggle room to parallelize the implementation.

(Note that the merge-all operation in my mergesort program satisfies all five properties.)

One way to convert an imperative loop to a parallel solution is to think in terms of grouping and functions:

  1. Bunch mutable state together as a state "value".
  2. Look at the loop as an application of one or more state transformation functions.
  3. Look for an efficient way to compose these transformation functions into a single function.

The first two steps are relatively straightforward. The third step is the part that requires ingenuity!

In this style of programming, associative combining operators are a big deal. Creating new, more diverse associative combining operators is the future of programming. Creating new idioms -- the patterns of programs written in this style -- is one of our challenges. Good programming languages of the future will provide, encourage, and enforce invariants that give compilers wiggle room.

In closing, Steele summarized our task as this: We need to do for processor allocation what garbage collection did for memory allocation. This is essential in a world in which we have parallel computers of wildly different sizes. (Multicore processors, anyone?)

I told some of the guys at the conference that I go to hear Guy Steele irrespective of his topic. I've been fortunate enough to be in a small OOPSLA workshop on creativity with Steele, gang-writing poetry and Sudoku generators, and I have seen him speak a few times over the years. Like his past talks, this talk makes me think differently about programs. It also crystallizes several existing ideas in a way that clarifies important questions.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

October 14, 2010 11:17 PM

Strange Loop 2010, Day 1 Afternoon

I came back from a lunchtime nap ready for more ideas.

You Already Use Closures in Ruby

This is one of the talks I chose for a specific personal reason. I was looking for ideas I might use in a Cedar Valley Tech Talk on functional programming later this month, and more generally in my programming languages course. I found one, a simple graphical notation for talking about closures as a combination code and environment. Something the speaker said about functions sharing state also gave me an idea for how to use the old koan on the dual "a closure is poor man's object / an object is poor man's closure".

NoSQL at Twitter

Speaker Kevin Weil started by decrying his own title. Twitter uses MySQL and relational databases everywhere. They use distributed data stores for applications that need specific performance attributes.

Weil traced Twitter's evolution toward different distributed data solutions. They started with Syslog for logging applications. It served there early needs but didn't scale well. They then moved to Scribe, which was created by the Facebook team to solve the same problem and then open-sourced.

This move led to a realization. Scribe solved Twitter's problem and opened new vistas. It made logging data so easy, that they started logging more. Having more data gave them better insight into the behavior of their users, behaviors they didn't even know to look for before. Now, data analytics is one of Twitter's most interesting areas.

The Scribe solution works, scaling with no changes to the architecture as data throughput doubles; they just add more servers. But this data creates a 'write' problem: giving today's takes technology, it takes something like 42 hours to write 12 TB to a single hard drive. This led Twitter to add Hadoop to its toolset. Hadoop is both scalable and powerful. Weil mentioned an a 4,000-node cluster at Yahoo! that had sorted one terabyte of integers in 62 seconds.

The rest of Weil's talk focused on data analytics. The key point underlying all he said was this: It is easy to answer questions. It is hard to ask the right questions. This makes experimental programming valuable, and by extension a powerful scripting language and short turnaround times. They need time to ask a lot of questions, looking for good ones and refining promising questions into more useful ones. Hadoop is a Java platform, which doesn't fit those needs.

So, Twitter added Pig, a high-level language that sits atop Hadoop. Programs written in Pig are easy to read and almost English-like. Equivalent SQL programs would probably be shorter, but Pig compiles to MapReduce jobs that run directly on Hadoop. Pig extracts a performance penalty, but the Twitter team doesn't mind. Weil captured why in another pithy sentence: I don't mind if a 10-minute job runs in 12 minutes if it took me 10 minutes to write the script instead of an hour.

Twitter works on several kinds of data-analytic problems. A couple stood out:

  • correlating big data. How do different kinds of user behave -- mobile, web, 3rd-party clients? What features hook users? How do user cohorts work? What technical details go wrong at the same time, leading to site problems?

  • research on big data. What can we learn from a users' tweets, the tweets of those they follow, or the tweets of those who follow them? What can we learn from asymmetric follow relationships about social and personal interests?

As much as Weil had already described, there was more! HBase, Cassandra, FlockDB, .... Big data means big problems and big opportunities, which lead to hybrid solutions that optimize competing forces. Interesting stuff.

Solutions to the Expression Problem

This talk was about Clojure, which interests me for obvious reasons, but the real reason I chose this talk was that I wanted to know what is the expression problem! Like many in the room, I had experienced the expression problem without knowing it by this name:

The Expression Problem is a new name for an old problem. The goal is to define a datatype by cases, where one can add new cases to the datatype and new functions over the datatype, without recompiling existing code, and while retaining static type safety (e.g., no casts).

Speaker Chris Houser used an example in which we need to add a behavior to an existing class that is hard or impossible to modify. He then stepped through four possible solutions: the adapter pattern and monkey patching, which are available in languages like Java and Ruby, and multimethods and protocols, which are available in Clojure.

I liked two things about this talk. First, he taught his a "failure-driven" way: pose a problem, solve it using a known technique, expose a weakness, and move on to a more effective solution. I often use this technique when teaching design patterns. Second, the breadth of the problem and its potential solutions encouraged language geeks to talk ideas in language design. The conversation included not only Java and Clojure but also Javascript, C#, and macros.

Guy Steele

My notes on this talk aren't that long, but it was important enough to have its own entry. Besides, this entry is already long enough!

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

October 14, 2010 10:02 PM

Strange Loop 2010, Day 1 Morning

I arrived in the University City district this morning ready for a different kind of conference. The program is an interesting mix of hot topics in technology, programming languages, and computing. My heuristic for selecting talks to attend was this: Of the talks in each time slot, which discusses the biggest idea I need to know more about? The only exceptions will be talks that fill a specific personal need, in teaching, research, or personal connections.

A common theme among the talks I attended today was big data. Here are some numbers I jotted down. They may be out of date by now, or I may have made an error in some of the details, but the orders of magnitude are impressive:

  • Hilary Mason reported that processes 10s of millions URLs per day, 100s of millions of events per day, and 1,000s of millions data points each day.
  • Google processes 24 petabytes a day, which amounts to 8 exabytes a year.
  • Digg has a 3 terabyte Cassandra database. This is dwarfed by Facebook's 150-node Cassandra cluster that comprises 150 terabytes.
  • Twitter users generate 12 terabytes of data each day -- the equivalent of 17,000 CDs -- and more than 4 petabytes each year. Also astounding: the data rate is doubling multiple times a year.

All of these numbers speak to the need for more CS programs to introduce students to data-intensive computing.

Hilary Mason on Machine Learning

Mason is a data scientist at, a company that faces "wicked hard problems" at scale. These problems arise from the combination of algorithms, on-demand computing, and "data, data, data". Mason gave an introduction to machine learning as applied to some of these problems. She had a few quotable lines:

  • AI was "founded on a remarkable conceit". This reminded me of a recent entry.
  • "Academics like to create new terms, because if they catch on the academics get prizes."
  • "Only a Bayesian can tell you why if there's a 50-50 chance that something will happen, then 90% of the time it will."

She also sprinkled her top with programming tips. One of the coolest was that you can append a "+" to the end of any URL to get analytic metrics for it.


I had planned to go to a talk on Flickr's architecture, but I talked so long in the break that the room full before I got there. So I stopped in on a talk by Ken Sipe on Gradle, a scriptable Java build system built on top of Groovy. He had one very nice quote as well: "I can't think of a real use of this idea. I just like showing it off." The teacher and hacker in me smiled.

Eben Hewitt on Adopting Cassandra

Even if you are not Google, you may have to process a lot of data. Hewitt talked about some of the efforts made to scale relational databases to handle big data (query tuning, indexes, vertical scaling, shards, and denormalization). This sequence of fixes made me think of epicycles, fixes applied to orbits of heavenly bodies in an effort to match observed data. At some point, you start looking for a theory that fits better up front.

That's what happened in the computing world, too. Soon there were a number of data systems defined mostly by what they are not: "NoSQL". That idea is not new. Lotus Notes used a document-oriented storage system; Smalltalk images store universes of objects. Now, as the problem of big data becomes more prevalent, new and different implementations have been proposed.

Hewitt talked about Cassandra and what distinguishes it from other approaches. He called Cassandra the love child of the Google BigTable paper (2006) and the Amazon Dynamo paper (2007). He also pointed out some of the limitations that circumscribe its applicability. In the NoSQL approaches, there is "no free lunch": you give up many relational DB advantages, such as transactions, joins, and ad hoc queries.

He did advocates one idea I'll need to read more about: that we should shift our attention from the CAP theorem of distributed data systems, which is useful but misses some important dynamic distinctions, with Abadi's PACELC model: If the data store is Partitioned, then you trade off between Availability and Consistency; Else you trade off between Latency and Consistency.

Posted by Eugene Wallingford | Permalink | Categories: Computing

October 06, 2010 11:24 AM

Studying Program Repositories

Hackage, through Cabal

Last week, Garrett Morris presented an experience report at the 2010 Haskell Symposium entitled Using Hackage to Inform Language Design, which is also available at Morris's website. Hackage is an online repository for the Haskell programming community. Morris describes how he and his team studied programs in the Hackage repository to see how Haskell programmers work with a type-system concept known as overlapping instances. The details of this idea aren't essential to this entry, but if you'd like to learn more, check out Section 6.2.3 in this user guide.)

In Morris's search, he sought answers to three questions:

  • What proportion of the total code on Hackage uses overlapping instances?
  • In code that uses overlapping instances, how many instances overlap each other?
  • Are there common patterns among the uses of overlapping instances?

Morris and his team used what they learned from this study to design the class system for their new language. They found that their language did not need to provide the full power of overlapping instances in order to support what programmers were really doing. Instead, they developed a new idea they call "instance chains" that was sufficient to support a large majority of the uses of overlapping instances. They are confident that their design can satisfy programmers because the design decision reflects actual uses of the concept in an extensive corpus of programs.

I love this kind of empirical research. One of the greatest assets the web gives us is access to large corpora of code: SourceForge and GitHub are examples large public repositories, and there are an amazing number of smaller domain-specific repositories such as Hackage and RubyGems. Why design languages or programming tools blindly when we can see how real programmers work through the code they write?

The more general notion of designing languages via observing behavior, forming theories, and trying them out is not new. In particular, I recommend the classic 1981 paper Design Principles Behind Smalltalk. Dan Ingalls describes the method by which the Smalltalk team grew its language and system as explicitly paralleling the scientific method. Morris's paper talks about a similar method, only with the observation phase grounded in an online corpus of code.

Not everyone in computer science -- or outside CS -- thinks of this method as science. Just this weekend Dirk Riehle blogged about the need to broaden the idea of science in CS. In particular, he encourages us to include exploration as a part of our scientific method, as it provides a valuable way for us to form the theories that we will test using the sort of experiments that everyone recognizes as science.

Dirk Riehle's cycle of theory formation and validation

Unfortunately, too many people in computer science do not take this broad view. Note that Morris published his paper as an experience report at a symposium. He would have a hard time trying to get an academic conference program committee to take such a paper in its research track, without first dressing it up in the garb of "real" research.

As I mentioned in an earlier blog entry, one of my grad students, Nate Labelle, did an M.S. thesis a few years ago based on information gathered from the study of a large corpus of programs. Nate was interested in dependencies among open-source software packages, so he mapped the network of relationships within several different versions of Linux and within a few smaller software packages. This was the raw data he used to analyze the mathematical properties of the dependencies.

In that project, we trolled repositories looking for structural information about the code they contained. Morris's work studied Hackage to learn about the semantic content of the programs. While on sabbatical several years ago, I started a much smaller project of this sort to look for design patterns in functional programs. That project was sidetracked by some other questions I ran across, but I've always wanted to get back to it. I think there is a lot we could learn about functional programming in this way that would help us to teach a new generation of programmers in industry.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

September 29, 2010 9:27 PM

The Manifest Destiny of Computer Science?

The June 2010 issue of Communications of the ACM included An Interview with Ed Feigenbaum, who is sometimes called the father of expert systems. Feigenbaum was always an ardent promoter of AI, and time doesn't seem to have made him less brash. The interview closes with the question, "Why is AI important?" The father of expert systems pulls no punches:

There are certain major mysteries that are magnificent open questions of the greatest import. Some of the things computer scientists study are not. If you're studying the structure of databases -- well, sorry to say, that's not one of the big magnificent questions.

I agree, though occasionally I find installing and configuring Rails and MySQL on my MacBook Pro to be one of the great mysteries of life. Feigenbaum is thinking about the questions that gave rise to the field of artificial intelligence more than fifty years ago:

I'm talking about mysteries like the initiation and development of life. Equally mysterious is the emergence of intelligence. Stephen Hawking once asked, "Why does the universe even bother to exist?" You can ask the same question about intelligence. Why does intelligence even bother to exist?

That is the sort of question that captivates a high school student with an imagination bigger than his own understanding of the world. Some of those young people are motivated by a desire to create an "ultra-intelligent computer:, as Feigenbaum puts it. Others are motivated more by the second prize on which AI founders set their eyes:

... a very complete model of how the human mind works. I don't mean the human brain, I mean the mind: the symbolic processing system.

That's the goal that drew the starry-eyed HS student who became the author of this blog into computer science.

Feigenbaum closes his answer with one of the more bodacious claims you'll find in any issue of Communications:

In my view the science that we call AI, maybe better called computational intelligence, is the manifest destiny of computer science.

There are, of course, many areas of computer science worthy of devoting one's professional life to. Over the years I have become deeply interested in questions related to language, expressiveness, and even the science or literacy that is programming. But it is hard for me to shake the feeling, still deep in my bones, that the larger question of understanding what we mean by "mind" is the ultimate goal of all that we do.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

September 26, 2010 7:12 PM

Competition, Imitation, and Nothingness on a Sunday Morning

Joseph Brodsky said:

Another poet who really changed not only my idea of poetry, but also my perception of the world -- which is what it's all about, ya? -- is Tsvetayeva. I personally feel closer to Tsvetayeva -- to her poetics, to her techniques, which I was never capable of. This is an extremely immodest thing to say, but, I always thought, "Can I do the Mandelstam thing?" I thought on several occasions that I succeeded at a kind of pastiche.

But Tsvetayeva. I don't think I ever managed to approximate her voice. She was the only poet -- and if you're a professional that's what's going on in your mind -- with whom I decided not to compete.

Tsvetayeva was one of Brodsky's closest friends in Russia. I should probably read some of her work, though I wonder how well the poems translate into English.


absolutely nothing: next 22 miles

When I am out running 22 miles, I have lots of time to think. This morning, I spent a few minutes thinking about Brodsky's quote, in the world of programmers. Every so often I have encountered a master programmer whose work changes my perception of world. I remember coming across several programs by Ward Cunningham's, including his wiki, and being captivated by its combination of simplicity and depth. Years before that, the ParcPlace Smalltalk image held my attention for months as I learned what object-oriented programming really was. That collection of code seemed anonymous at first, but I later learned its history and and became a fan of Ingalls, Maloney, and the team. I am sure this happens to other programmers, too.

Brodsky also talks about his sense of competition with other professional poets. From the article, it's clear that he means not a self-centered or destructive competition. He liked Tsvetayeva deeply, both professionally and personally. The competition he felt is more a call to greatness, an aspiration. He was following the thought, "That is beautiful" with "I can do that" -- or "Can I do that?"

I think programmers feel this all the time, whether they are pros or amateurs. Like artists, many programmers learn by imitating the code they see. These days, the open-source software world gives us so many options! See great code; imitate great code. Find a programmer whose work you admire consistently; imitate the techniques, the style, and, yes, the voice. The key in software, as in art, is finding the right examples to imitate.

Do programmers ever choose not to compete in Brodsky's sense? Maybe, maybe not. There are certainly people whose deep grasp of computer science ideas usually feels beyond my reach. Guy Steele comes to mind. But I think for programmers it's mostly a matter of time. We have to make trade-offs between learning one thing well or another.


22 miles is a long run. I usually do only one to two runs that long during my training for a given marathon. Some days I start with the sense of foreboding implied by the image above, but more often the run is just there. Twenty-two miles. Run.

This time the morning was chill, 40 degrees with a bright sun. The temperature had fallen so quickly overnight that the previous day's rain had condensed in the leaves of every tree and bush, ready to fall like a new downpour at the slightest breeze.

This is my last long run before taking on Des Moines in three weeks. It felt neutral and good at the same time. It wasn't a great run, like my 20-miler two weeks ago, but it did what it needed to do: stress my legs and mind to run for about as long as the marathon will be. And I had plenty of time to think through the nothingness.

Now begins my taper, an annual ritual leading to a race. The 52 miles I logged this week will seep into my body for the next ten days or so as it acclimates to the stress. Now, I will pare back my mileage and devote a few more short and medium-sized runs to converting strength into the speed.


The quote that opens this entry comes from Joseph Brodsky, The Art of Poetry No. 28, an interview in The Paris Review by Sven Birkerts in December 1979. I like very much to hear writers talk about how they write, about other writers, and about the culture of writing. This long interview repaid me several times for the time I spent reading.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Running, Teaching and Learning

September 20, 2010 4:28 PM

Alan Kay on "Real" Object-Oriented Programming

In Alan Kay's latest comment-turned-blog entry, he offers thoughts in response to Moti Ben-Ari's CACM piece about object-oriented programming. Throughout, he distinguishes "real oop" from what passes under the term these days. His original idea of object-oriented software is really quite simple to express: encapsulated modules all the way down, with pure messaging. This statement boils down to its essence many of the things he has written and talked about over the years, such as in his OOPSLA 2004 talks.

This is a way to think about and design systems, not a set of language features grafted onto existing languages or onto existing ways of programming. He laments what he calls "simulated data structure programming", which is, sadly, the dominant style one sees in OOP books these days. I see this style in nearly every OOP textbook -- especially those aimed at beginners, because those books generally start with "the fundamentals" of the old style. I see it in courses at my university, and even dribbling into my own courses.

One of the best examples of an object-oriented system is one most people don't think of as a system at all: the Internet:

It has billions of completely encapsulated objects (the computers themselves) and uses a pure messaging system of "requests, not commands", etc.

Distributed client-server apps make good examples and problems in courses on OO design precisely because they separate control of the client and the server. When we write OO software in which we control both sides of the message, it's often too tempting to take advantage of how we control both objects and to violate encapsulation. These violations can be quite subtle, even when we take into account idioms such as access methods and the Law of Demeter. To what extent does one component depend on how another component does its job? The larger the answer, the more coupled the components.

Encapsulation isn't an end unto itself, of course. Nor are other features of our implementation:

The key to safety lies in the encapsulation. The key to scalability lies in how messaging is actually done (e.g., maybe it is better to only receive messages via "postings of needs"). The key to abstraction and compactness lies in a felicitous combination of design and mathematics.

I'd love to hear Kay elaborate on this "felicitous combination of design and mathematics"... I'm not sure just what he means!

As an old AI guy, I am happy to see Kay make reference to the Actor model proposed by Carl Hewitt back in the 1970s. Hewitt's ideas drew some of their motivation from the earliest Smalltalk and gave rise not only to Hewitt's later work on concurrent programming but also Scheme. Kay even says that many of Hewitt's ideas "were more in the spirit of OOP than the subsequent Smalltalks."

Another old AI idea that came to my mind as I read the article was blackboard architecture. Kay doesn't mention blackboards explicitly but does talk about how messaging might better be if instead of an object sent messages to specific targets they might "post their needs". In a blackboard system, objects capable of satisfying needs monitor the blackboard and offer to respond to a request as they are able. The blackboard metaphor maintains some currency in the software world, especially in the distributed computing world; it even shows up as an architectural pattern in Pattern-Oriented Software Architecture. This is a rich metaphor with much room for exploration as a mechanism for OOP.

Finally, as a CS educator, I could help but notice Kay repeating a common theme of his from the last decade, if not longer:

The key to resolving many of these issues lies in carrying out education in computing in a vastly different way than is done today.

That is a tall order all its own, much harder in some ways than carrying out software development in a vastly different way than is done today.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 10, 2010 2:22 PM

Recursion, Trampolines, and Software Development Process

Yesterday I read an old article on tail calls and trampolining in Scala by Rich Dougherty, which summarizes nicely the problem of recursive programming on the JVM. Scala is a functional language, which lends itself to recursive and mutually recursive programs. Compiling those programs to run on the JVM presents problems because (1) the JVM's control stack is shallow and (2) the JVM doesn't support tail-call optimization. Fortunately, Scala supports first-class functions, which enables the programmer to implement a "trampoline" that avoids the growing the stack. The resulting code is harder to understand and so to maintain, but it runs without growing the control stack. This is a nice little essay.

Dougherty's conclusion about trampoline code being harder to understand reminded me of a response by reader Edward Coffin to my July entry on CS faculty sending bad signals about recursion. He agreed that recursion usually is not a problem from a technical standpoint but pointed out a social problem (paraphrased):

I have one comment about the use of recursion in safety-critical code, though: it is potentially brittle with respect to changes made by someone not familiar with that piece of code, and brittle in a way that makes breaking the code difficult to detect. I'm thinking of two cases here: (1) a maintainer unwittingly modifies the code in a way that prevents the compiler from making the formerly possible tail-call optimization and (2) the organization moves to a compiler that doesn't support tail-call optimization from one that did.

Edward then explained how hard it is to warn the programmers that they have just made changes to the code that invalidate essential preconditions. This seems like a good place to comment the code, but we can't rely on programmers paying attention to such comments, even that the comments will accompany the code forever. The compiler may not warn us, and it may be hard to write test cases that reliably fail when the optimization is missed. Scala's @tailrec annotation is a great tool to have in this situation!

"Ideally," he writes, "these problems would be things a good organization could deal with." Unfortunately, I'm guessing that most enterprise computing shops are probably not well equipped to handle them gracefully, either by personnel or process. Coffin closes with a pragmatic insight (again, paraphrased):

... it is quite possible that [such organizations] are making the right judgement by forbidding it, considering their own skill levels. However, they may be using the wrong rationale -- "We won't do this because it is generally a bad idea." -- instead of the more accurate "We won't do this because we aren't capable of doing it properly."

Good point. I don't suppose it's reasonable for me or anyone to expect people in software shops to say that. Maybe the rise of languages such and Scala and Clojure will help both industry and academia improve the level of developers' ability to work with functional programming issues. That might allow more organizations to use a good technical solution when it is suitable.

That's one of the reasons I still believe that CS educators should take care to give students a balanced view of recursive programming. Industry is beginning to demand it. Besides, you never know when a young person will get excited about a problem whose solution feels so right as a recursion and set off to write a program to grow his excitement. We also want our graduates to be able to create solutions to hard problems that leverage the power of recursion. We need for students to grok the Big Idea of recursion as a means for decomposing problems and composing systems. The founding of Google offers an instructive example of using inductive definition recursion, as discussed in this Scientific American article on web science:

[Page and Brin's] big insight was that the importance of a page -- how relevant it is -- was best understood in terms of the number and importance of the pages linking to it. The difficulty was that part of this definition is recursive: the importance of a page is determined by the importance of the pages linking to it, whose importance is determined by the importance of the pages linking to them. [They] figured out an elegant mathematical way to represent that property and developed an algorithm they called PageRank to exploit the recursiveness, thus returning pages ranked from most relevant to least.

Much like my Elo ratings program that used successive approximation, PageRank may be implemented in some other way, but it began as a recursive idea. Students aren't likely to have big recursive ideas if we spend years giving them the impression it is an esoteric technique best reserved for their theory courses.

So, yea! for Scala, Clojure, and all the other languages that are making recursion respectable in practice.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 07, 2010 4:53 PM

Quick Thoughts on "Computational Thinking"

When Mark Guzdial posted his article Go to the Data, offering "two stories of (really) computational thinking", I thought for sure that I'd write something on the topic soon. Over four and a half weeks have passed, and I have not written yet. That's because, despite thinking about it on and off ever since, I still don't have anything deep or detailed to say. I'm as confused as Mark says he and Barb are, I guess.

Still, I can't seem to shake these two simple thoughts:

  • Whatever else else it may be, "computational thinking" has to involve, well, um, computation. As several commenters on his piece point out, lots of scientists collect and analyze data. Computer science connects data and algorithm in a way that other disciplines don't.

  • If a bunch of smart, well-informed people can discuss a name like "computational thinking" for several years and still be confused, maybe the problem is with the name. Maybe it really is just another name for a kind of computer science, generalized so that we can speak meaningfully of non-CS types and non-programmers doing it. I believe strongly that computing offers a new medium for expressing and creating ideas but, as for the term computational thinking, maybe there's no there there.

I do agree with Mark that we runners can learn a lot by looking back at our running logs! An occasional short program can often expose truths that elude the naked eye.

Posted by Eugene Wallingford | Permalink | Categories: Computing

September 01, 2010 9:12 PM

The Beginning of a New Project

"Tell me you're not playing chess," my colleague said quizzically.

But I was. My newest grad student and I were sitting in my office playing a quick couple of games of progressive chess, in which I've long been interested. In progressive chess, white makes one move, then black makes two; white makes three moves, then black makes four. The game proceeds in this fashion until one of the players delivers checkmate or until the game ends in any other traditional way.

This may seem like a relatively simple change to the rules of the game, but the result is something that almost doesn't feel like chess. The values of the pieces changes radically, as does the value of space and the meaning of protection. That's why we needed to play a couple of games: to acquaint my student with how different it is from the classical chess I know and love and which has played since a child.

For his master's project, the grad student wanted to do something in the general area of game-playing and AI, and we both wanted to work on a problem that is relatively untouched, where a few cool discoveries are still accessible to mortals. Chess, the fruit fly of AI from the 1950s into the 1970s, long ago left the realm where newcomers could make much of a contribution. Chess isn't solved in the technical sense, as checkers is, but the best programs now outplay even the best humans. To improve on the state of the art requires specialty hardware or exquisitely honed software.

Progressive chess, on the other hand, has a funky feel to it and looks wide open. We are not yet aware of much work that has been done on it, either in game theory or automation. My student is just beginning his search of the literature and will know soon how much has been done and what problems have been solved, if any.

That is why we were playing chess in my office on a Wednesday afternoon, so that we could discuss some of the ways in which we will have to think differently about this problem as we explore solutions. Static evaluation of positions is most assuredly different from what works in classical chess, and I suspect that the best ways to search the state space will be quite different, too. After playing only a few games, my student proposed a new way to do search to capitalize on progressive chess's increasingly long sequences of moves by one player. I'm looking forward to exploring it further, giving it a try in code, and finding other new ideas!

I may not be an AI researcher first any more, but this project excites me. You never know what you will discover until you wander away from known territory, and this problem offers us a lot of unknowns.

And I'll get to say, "Yes, we are playing chess," every once in a while, too.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

August 31, 2010 7:18 PM

Notes from a Master Designer

Someone tweeted recently about a recent interview with Fred Brooks in Wired magazine. Brooks is one of the giants of our field, so I went straight to the page. I knew that I wanted to write something about the interview as soon as I saw this exchange, which followed up questions about how a 1940s North Carolina schoolboy ended up working with computers:

Wired: When you finally got your hands on a computer in the 1950s, what did you do with it?

Brooks: In our first year of graduate school, a friend and I wrote a program to compose tunes. The only large sample of tunes we had access to was hymns, so we started generating common-meter hymns. They were good enough that we could have palmed them off to any choir.

It never surprises me when I learn that programmers and computer scientists are first drawn to software by a desire to implement creative and intelligent tasks. Brooks was first drawn to computers by a desire to automatic data retrieval, which at the time must have seemed almost as fantastic as composing music. In an Communications of the ACM interview printed sometime last year, Ed Feigenbaum called AI the "manifest destiny" of computer science. I often think he is right. (I hope to write about that interview soon, too.)

But that's not the only great passage in Brooks's short interview. Consider:

Wired: You say that the Job Control Language you developed for the IBM 360 OS was "the worst computer programming language ever devised by anybody, anywhere." Have you always been so frank with yourself?

Brooks: You can learn more from failure than success. In failure you're forced to find out what part did not work. But in success you can believe everything you did was great, when in fact some parts may not have worked at all. Failure forces you to face reality.

As an undergrad, I took a two-course sequence in assembly language programming and JCL on an old IBM 370 system. I don't know how much the JCL on that machine had advanced beyond Brooks's worst computer programming language ever devised, if it had at all. But I do know that the JCL course gave me a negative-split learning experience unlike any I had ever had before or have had since. As difficult as that was, I will be forever grateful for Dr. William Brown, a veteran of the IBM 360/370 world, and what he taught me that year.

There are at least two more quotables from Brooks that are worth hanging on my door some day:

Great design does not come from great processes; it comes from great designers.

Hey to Steve Jobs.

The insight most likely to improve my own work came next:

The critical thing about the design process is to identify your scarcest resource.

This one line will keep me on my toes for many projects to come.

If great design comes from great designers, then how can the rest of us work toward the goal of becoming a great designer, or at least a better one?

Design, design, and design; and seek knowledgeable criticism.

Practice, practice, practice. But that probably won't be enough. Seek out criticism from thoughtful programmers, designers, and users. Listen to what they have to say, and use it to improve your practice.

A good start might be to read this interview and Brooks's books.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

August 24, 2010 4:30 PM

Dreaming, Doing, Perl, and Language Translation

Today, I quoted Larry Wall's 2000 Atlanta Linux Showcase Talk in the first day of my compilers course. In that talk, he gives a great example of using a decompiler to port code -- in this case, from Perl 5 to Perl 6. While re-reading the talk, I remembered something that struck me as wrong when I read it the first time:

["If you can dream it, you can do it"--Walt Disney]

"If you can dream it, you can do it"--Walt Disney. Now this is actually false (massive laughter). I think Walt was confused between necessary and sufficient conditions. If you *don't* dream it, you can't do it; that is certainly accurate.

I don't think so. I think this is false, too. (Laugh now.)

It is possible to do things you don't dream of doing first. You certainly have to be open to doing things. Sometimes we dream something, set out to do it, and end up doing something else. The history of science and engineering are full of accidents and incidental results.

I once was tempted to say, "If you don't start it, you can't do it; that is certainly accurate." But I'm not sure that's true either, because of the first "it". These days, I'm more inclined to say that if you don't start doing something, you probably won't do anything.

Back to Day 1 of the compilers: I do love this course. The Perl quote in my lecture notes is but one element in a campaign to convince my students that this isn't just a compilers course. The value in the course material and in the project itself go far beyond the creation of an old-style source language-to-machine language translator. Decompilers, refactoring browsers, cross-compilers, preprocessors, interpreters, and translators for all sorts of domain-specific languages -- a compilers course will help you learn about all of these tools, both how they work and how to build them. Besides, there aren't many better ways to consolidate your understanding of the breadth of computer science than to build a compiler.

The official title of my department's course is "Translation of Programming Languages". Back in 1994, before the rebirth of mainstream language experimentation and the growth of interest in scripting languages and domain-specific languages, this seemed like a daring step. These days, the title seems much more fitting than "Compiler Construction". Perhaps my friend and former colleague Mahmoud Pegah and I had a rare moment of foresight. More likely, Mahmoud had the insight, and I was simply wise enough to follow.

Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

August 20, 2010 3:30 PM

Simplicity, Complexity, and Good Enough Chess Ratings

Andrew Gelman