April 09, 2014 3:26 PM

Programming Everywhere, Vox Edition

In a report on the launch of Vox Media, we learn that line between software developers and journalists at Vox is blurred, as writers and reporters work together "to build the tools they require".

"It is thrilling as a journalist being able to envision a tool and having it become a real thing," Mr. Topolsky said. "And it is rare."

It will be less rare in the future. Programming will become a natural part of more and more people's toolboxes.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

April 04, 2014 12:43 PM

Creative Recombination of Existing Ideas

In a post on his move to California, Jon Udell notes that he may be out of step with the dominant view of the tech industry there:

And I think differently about innovation than Silicon Valley does. I don't think we lack new ideas. I think we lack creative recombination of proven tech, and the execution and follow-through required to surface its latent value.

As he found with the Elm City project, sometimes a good idea doesn't get traction quickly, even with sustained effort. Calendar aggregation seems like such a win even for a university the size of mine, yet a lot of folks don't get it. It's hard to know whether the slowness results from the idea, the technology, or the general resistance of communities to change how they operate.

In any case, Udell is right: there is a lot of latent value in the "creative recombination" of existing ideas. Ours is a remix culture, too. That's why it's so important to study widely in and out of computing, to build the base of tools needed to have a great idea and execute on it.


Posted by Eugene Wallingford | Permalink | Categories: Computing

March 31, 2014 3:21 PM

Programming, Defined and Re-imagined

By Chris Granger of Light Table fame:

Programming is our way of encoding thought such that the computer can help us with it.

Read the whole piece, which recounts Granger's reflection after the Light Table project left him unsatisfied and he sought answers. He concludes that we need to re-center our idea of what programming is and how we can make it accessible to more people. Our current idea of programming doesn't scale because, well,

It turns out masochism is a hard sell.

Every teacher knows this. You can sell masochism to a few proud souls, but not to anyone else.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

March 12, 2014 3:55 PM

Not Content With Content

Last week, the Chronicle of Higher Ed ran an article on a new joint major at Stanford combining computer science and the humanities.

[Students] might compose music or write a short story and translate those works, through code, into something they can share on the web.

"For students it seems perfectly natural to have an interest in coding," [the program's director] said. "In one sense these fields might feel like they're far apart, but they're getting closer and closer."

The program works in both directions, by also engaging CS students in the societal issues created by ubiquitous networks and computing power.

We are doing something similar at my university. A few years ago, several departments began to collaborate on a multidisciplinary program called Interactive Digital Studies which went live in 2012. In the IDS program, students complete a common core of courses from the Communication Studies department and then take "bundles" of coursework involving digital technology from at least two different disciplines. These areas of emphasis enable students to explore the interaction of computing with various topics in media, the humanities, and culture.

Like Stanford's new major, most of the coursework is designed to work at the intersection of disciplines, rather than pursuing disciplines independently, "in parallel".

The initial version of the computation bundle consists of an odd mix of application tools and opportunities to write programs. Now that the program is in place, we are finding that students and faculty alike desire more depth of understanding about programming and development. We are in the process of re-designing the bundle to prepare students to work in a world where so many ideas become web sites or apps, and in which data analytics plays an important role in understanding what people do.

Both our IDS program and Stanford's new major focus on something that we are seeing increasingly at universities these days: the intersections of digital technology and other disciplines, in particular the humanities. Computational tools make it possible for everyone to create more kinds of things, but only if people learn how to use new tools and think about their work in new ways.

Consider this passage by Jim O'Loughlin, a UNI English professor, from a recent position statement on the the "digital turn" of the humanities:

We are increasingly unlikely to find writers who only provide content when the tools for photography, videography and digital design can all be found on our laptops or even on our phones. It is not simply that writers will need to do more. Writers will want to do more, because with a modest amount of effort they can be their own designers, photographers, publishers or even programmers.

Writers don't have to settle for producing "content" and then relying heavily on others to help bring the content to an audience. New tools enable writers to take greater control of putting their ideas before an audience. But...

... only if we [writers] are willing to think seriously not only about our ideas but about what tools we can use to bring our ideas to an audience.

More tools are within the reach of more people now than ever before. Computing makes that possible, not only for writers, but also for musicians and teachers and social scientists.

Going further, computer programming makes it possible to modify existing tools and to create new tools when the old ones are not sufficient. Writers, musicians, teachers, and social scientists may not want to program at that level, but they can participate in the process.

The critical link is preparation. This digital turn empowers only those who are prepared to think in new ways and to wield a new set of tools. Programs like our IDS major and Stanford's new joint major are among the many efforts hoping to spread the opportunities available now to a larger and more diverse set of people.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

March 08, 2014 10:18 AM

Sometimes a Fantasy

This week I saw a link to The Turing School of Software & Design, "a seven-month, full-time program for people who want to become professional developers". It reminded me of Neumont University, a ten-year-old school that offers a B.S. degree program in Computer science that students can complete in two and a half years.

While riding the bike, I occasionally fantasize about doing something like this. With the economics of universities changing so quickly [ 1 | 2 ], there is an opportunity for a new kind of higher education. And there's something appealing about being able to work closely with a cadre of motivated students on the full spectrum of computer science and software development.

This could be an accelerated form of traditional CS instruction, without the distractions of other things, or it could be something different. Traditional university courses are pretty confining. "This course is about algorithms. That one is about programming languages." It would be fun to run a studio in which students serve as apprentices making real stuff, all of us learning as we go along.

A few years ago, one of our ChiliPLoP hot topic groups conducted a greenfield thought experiment to design an undergrad CS program outside of the constraints of any existing university structure. Student advancement was based on demonstrating professional competencies, not completing packaged courses. It was such an appealing idea! Of course, there was a lot of hard work to be done working out the details.

My view of university is still romantic, though. I like the idea of students engaging the great ideas of humanity that lie outside their major. These days, I think it's conceivable to include the humanities and other disciplines in a new kind of CS education. In a recent blog entry, Hollis Robbins floats the idea of Home College for the first year of a liberal arts education. The premise is that there are "thousands of qualified, trained, energetic, and underemployed Ph.D.s [...] struggling to find stable teaching jobs". Hiring a well-rounded tutor could be a lot less expensive than a year at a private college, and more lucrative for the tutor than adjuncting.

Maybe a new educational venture could offer more than targeted professional development in computing or software. Include a couple of humanities profs, maybe some a social scientist, and it could offer a more complete undergraduate education -- one that is economical both in time and money.

But the core of my dream is going broad and deep in CS without the baggage of a university. Sometimes a fantasy is all you need. Other times...


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

March 01, 2014 11:35 AM

A Few Old Passages

I was looking over a couple of files of old notes and found several quotes that I still like, usually from articles I enjoyed as well. They haven't found their way into a blog entry yet, but they deserve to see the light of day.

Evidence, Please!

From a short note on the tendency even among scientists to believe unsubstantiated claims, both in and out of the professional context:

It's hard work, but I suspect the real challenge will lie in persuading working programmers to say "evidence, please" more often.

More programmers and computer scientists are trying to collect and understand data these days, but I'm not sure we've made much headway in getting programmers to ask for evidence.

Sometimes, Code Before Math

From a discussion of The Expectation Maximization Algorithm:

The code is a lot simpler to understand than the math, I think.

I often understand the language of code more quickly than the language of math. Reading, or even writing, a program sometimes helps me understand a new idea better than reading the math. Theory is, however, great for helping me to pin down what I have learned more formally.

Grin, Wave, Nod

From Iteration Inside and Out, a review of the many ways we loop over stuff in programs:

Right now, the Rubyists are grinning, the Smalltalkers are furiously waving their hands in the air to get the teacher's attention, and the Lispers are just nodding smugly in the back row (all as usual).

As a Big Fan of all three languages, I am occasionally conflicted. Grin? Wave? Nod? Look like the court jester by doing all three simultaneously?


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

February 21, 2014 3:35 PM

Sticking with a Good Idea

My algorithms students and I recently considered the classic problem of finding the closest pair of points in a set. Many of them were able to produce a typical brute-force approach, such as:

    minimum ← 
    for i ← 1 to n do
        for j ← i+1 to n do
            distance ← sqrt((x[i] - x[j])² + (y[i] - y[j])²)
            if distance < minimum then
               minimum ← distance
               first   ← i
               second  ← j
    return (first, second)

Alas, this is an O(n²) process, so we considered whether we might do better with a divide-and-conquer approach. It did not look promising, though. Divide-and-conquer doesn't let us solve the sub-problems independently. What if the closest pair straddles two partitions?

This is a common theme in computing, and problem solving more generally. We try a technique only to find that it doesn't quite work. Something doesn't fit, or a feature of the domain violates a requirement of the technique. It's tempting in such cases to give up and settle for something less.

Experienced problem solvers know not to give up too quickly. Many of the great advances in computing came under conditions just like this. Consider Leonard Kleinrock and the theory of packet switching.

In a Computing Conversations podcast published last year, Kleinrock talks about his Ph.D. research. He was working on the problem of how to support a lot of bursty network traffic on a shared connection. (You can read a summary of the podcast in an IEEE Computer column also published last year.)

His wonderful idea: apply the technique of time sharing from multi-user operating systems. The system could break all messages into "packets" of a fixed size, let messages take turns on the shared line, then reassemble each message on the receiving end. This would give every message a chance to move without making short messages wait too long behind large ones.

Thus was born the idea of packet switching. But there was a problem. Kleinrock says:

I set up this mathematical model and found it was analytically intractable. I had two choices: give up and find another problem to work on, or make an assumption that would allow me to move forward. So I introduced a mathematical assumption that cracked the problem wide open.

His "independence assumption" made it possible for him to complete his analysis and optimize the design of a packet-switching network. But an important question remained: Was his simplifying assumption too big a cheat? Did it skew the theoretical results in such a way that his model was no longer a reasonable approximation of how networks would behave in the real world?

Again, Kleinrock didn't give up. He wrote a program instead.

I had to write a program to simulate these networks with and without the assumption. ... I simulated many networks on the TX-2 computer at Lincoln Laboratories. I spent four months writing the simulation program. It was a 2,500-line assembly language program, and I wrote it all before debugging a single line of it. I knew if I didn't get that simulation right, I wouldn't get my dissertation.

High-stakes programming! In the end, Kleinrock was able to demonstrate that his analytical model was sufficiently close to real-world behavior that his design would work. Every one of us reaps the benefit of his persistence every day.

Sometimes, a good idea poses obstacles of its own. We should not let those obstacles beat us without a fight. Often, we just have to find a way to make it work.

This lesson applies quite nicely to using divide-and-conquer on the closest pairs problem. In this case, we don't make a simplifying assumption; we solve the sub-problem created by our approach:

After finding a candidate for the closest pair, we check to see if there is a closer pair straddling our partitions. The distance between the candidate points constrains the area we have to consider quite a bit, which makes the post-processing step feasible. The result is an O(n log n) algorithm that improves significantly on brute force.

This algorithm, like packet switching, comes from sticking with a good idea and finding a way to make it work. This is a lesson every computer science student and novice programmer needs to learn.

There is a complementary lesson to be learned, of course: knowing when to give up on an idea and move on to something else. Experience helps us tell the two situations apart, though never with perfect accuracy. Sometimes, we just have to follow an idea long enough to know when it's time to move on.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

February 19, 2014 4:12 PM

Teaching for the Perplexed and the Traumatized

Teaching for the Perplexed and the Traumatized

On need for empathy when writing about math for the perplexed and the traumatized, Steven Strogatz says:

You have to help them love the questions.

Teachers learn this eventually. If students love the questions, they will do an amazing amount of working searching for answers.

Strogatz is writing about writing, but everything he says applies to teaching as well, especially teaching undergraduates and non-majors. If you teach only grad courses in a specialty area, you may be able to rely on the students to provide their own curiosity and energy. Otherwise having empathy, making connections, and providing Aha! moments are a big part of being successful in the classroom. Stories trump formal notation.

This semester, I've been trying a particular angle on achieving this trifecta of teaching goodness: I try to open every class session with a game or puzzle that the students might care about. From there, we delve into the details of algorithms and theoretical analysis. I plan to write more on this soon.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

February 02, 2014 5:19 PM

Things That Make Me Sigh

In a recent article unrelated to modern technology or the so-called STEM crisis, a journalist writes:

Apart from mathematics, which demands a high IQ, and science, which requires a distinct aptitude, the only thing that normal undergraduate schooling prepares a person for is... more schooling.

Sigh.

On the one hand, this seems to presume that one need neither a high IQ nor any particular aptitude to excel in any number of non-math and science disciplines.

On the other, it seems to say that if one does not have the requisite characteristics, which are limited to a lucky few, one need not bother with computer science, math or science. Best become a writer or go into public service, I guess.

I actually think that the author is being self-deprecating, at least in part, and that I'm probably reading too much into one sentence. It's really intended as a dismissive comment on our education system, the most effective outcome of which often seems to be students who are really good at school.

Unfortunately, the attitude expressed about math and science is far too prevalent, even in our universities. It demeans our non-scientists as well as our scientists and mathematicians. It also makes it even harder to convince younger students that, with a little work and practice, they can achieve satisfying lives and careers in technical disciplines.

Like I said, sigh.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

January 27, 2014 11:39 AM

The Polymath as Intellectual Polygamist

Carl Djerassi, quoted in The Last Days of the Polymath:

Nowadays people [who] are called polymaths are dabblers -- are dabblers in many different areas. I aspire to be an intellectual polygamist. And I deliberately use that metaphor to provoke with its sexual allusion and to point out the real difference to me between polygamy and promiscuity.

On this view, a dilettante is merely promiscuous, making no real commitment to any love interest. A polymath has many great loves, and loves them all deeply, if not equally.

We tend to look down on dilettantes, but they can perform a useful service. Sometimes, making a connection between two ideas at the right time and in the right place can help spur someone else to "go deep" with the idea. Even when that doesn't happen, dabbling can bring great personal joy and provide more substantial entertainment than a lot of pop culture.

Academics are among the people these days with a well-defined social opportunity to be explore at least two areas deeply and seriously: their chosen discipline and teaching. This is perhaps the most compelling reason to desire a life in academia. It even offers a freedom to branch out into new areas later in one's career that is not so easily available to people who work in industry.

These days, it's hard to be a polymath even inside one's own discipline. To know all sub-areas of computer science, say, as well as the experts in those sub-areas is a daunting challenge. I think back to the effort my fellow students and I put in over the years that enabled us to take the Ph.D. qualifying exams in CS. I did quite well across the board, but even then I didn't understand operating systems or programming languages as well as experts in those areas. Many years later, despite continued reading and programming, the gap has only grown.

I share the vague sense of loss, expressed by the author of the article linked to above, of a time when one human could master multiple areas of discourse and make fundamental advances to several. We are certainly better off for collective understanding the world so much much better, but the result is a blow to a certain sort of individual mind and spirit.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

January 26, 2014 3:05 PM

One Reason We Need Computer Programs

Code bridges the gap between theory and data. From A few thoughts on code review of scientific code:

... there is a gulf of unknown size between the theory and the data. Code is what bridges that gap, and specifies how edge cases, weird features of the data, and unknown unknowns are handled or ignored.

I learned this lesson the hard way as a novice programmer. Other activities, such as writing and doing math, exhibit the same characteristic, but it wasn't until I started learning to program that the gap between theory and data really challenged me.

Since learning to program myself, I have observed hundreds of CS students encounter this gap. To their credit, they usually buckle down, work hard, and close the gap. Of course, we have to close the gap for every new problem we try to solve. The challenge doesn't go away; it simply becomes more manageable as we become better programmers.

In the passage above, Titus Brown is talking to his fellow scientists in biology and chemistry. I imagine that they encounter the gap between theory and data in a new and visceral way when they move into computational science. Programming has that power to change how we think.

There is an element of this, too, in how techies and non-techies alike sometimes lose track of how hard it is to create a successful start up. You need an idea, you need a programmer, and you need a lot of hard work to bridge the gap between idea and executed idea.

Whether doing science or starting a company, the code teaches us a lot about out theory. The code makes our theory better.

As Ward Cunningham is fond of saying, it's all talk until the tests run.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

January 23, 2014 4:14 PM

The CS Mindset

Chad Orzel often blogs about the physics mindset, the default orientation that physicists tend to have toward the world, and the way they think about and solve problems. It is fun to read a scientist talking about doing science.

Earlier this week I finally read this article about the popularity of CS50, an intro CS course at Harvard. It's all about how Harvard is "is righting an imbalance in academia" by finally teaching practical skills to its students. When I read:

"CS50 made me look at Harvard with new eyes," Guimaraes said.

That is a sea change from what Harvard represented to the outside world for decades: the guardian of a classic education, where the value of learning is for its own sake.

I sighed audibly, loud enough for the students on the neighboring exercise equipment to hear. A Harvard education used to be about learning only for its own sake, but now students can learn practical stuff, too. Even computer programming!

As I re-read the article now, I see that it's not as blunt as that throughout. Many Harvard students are learning computing because of the important role it plays in their futures, whatever their major, and they understand the value of understanding it better. But there are plenty of references to "practical ends" and Harvard's newfound willingness to teach practical skills it once considered beneath it.

Computer programming is surely one of those topics old Charles William Eliot would deem unworthy of inclusion in Harvard's curriculum.

I'm sensitive to such attitudes because I think computer science is and should be more. If you look deeper, you will see that the creators of CS50 think so, too. On its Should I take CS50? FAQ page, we find:

More than just teach you how to program, this course teaches you how to think more methodically and how to solve problems more effectively. As such, its lessons are applicable well beyond the boundaries of computer science itself.

The next two sentences muddy the water a bit, though:

That the course does teach you how to program, though, is perhaps its most empowering return. With this skill comes the ability to solve real-world problems in ways and at speeds beyond the abilities of most humans.

With this skill comes something else, something even more important: a discipline of thinking and a clarity of thought that are hard to attain when you learn "how to think more methodically and how to solve problems more effectively" in the abstract or while doing almost any other activity.

Later the same day, I was catching up on a discussion taking place on the PLT-EDU mailing list, which is populated by the creators, users, and fans of the Racket programming language and the CS curriculum designed in tandem with it. One poster offered an analogy for talking to HS students about how and why they are learning to program. A common theme in the discussion that ensued was to take the conversation off of the "vocational track". Why encourage students to settle for such a limiting view of what they are doing?

One snippet from Matthias Felleisen (this link works only if you are a member of the list) captured my dissatisfaction with the tone of the Globe article about CS50:

If we require K-12 students to take programming, it cannot be justified (on a moral basis) with 'all of you will become professional programmers.' I want MDs who know the design recipe, I want journalists who write their articles using the design recipe, and so on.

The "design recipe" is a thinking tool students learn in Felleisen "How to Design Programs" curriculum. It is a structured way to think about problems and to create solutions. Two essential ideas stand out for me:

  • Students learn the design recipe in the process of writing programs. This isn't an abstract exercise. Creating a working computer program is tangible evidence that student has understood the problem and created a clear solution.
  • This way of thinking is valuable for everyone. We will all better off if our doctors, lawyers, and journalists are able to think this way.

This is one of my favorite instantiations of the vague term computational thinking so many people use without much thought. It is a way of thinking about problems both abstractly and concretely, that leads to solutions that we have verified with tests.

You might call this the CS mindset. It is present in CS50 independent of any practical ends associated with tech start-ups and massaging spreadsheet data. It is practical on a higher plane. It is also present in the HtDP curriculum and especially in the Racket Way.

It is present in all good CS education, even the CS intro courses that more students should be taking -- even if they are only going to be doctors, lawyers, or journalists.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

January 16, 2014 4:22 PM

Another Take on "Know the Past"

Ted Nelson offers a particularly stark assessment of how well we fulfill our obligation to know the past in his eulogy for Douglas Engelbart:

To quote Joan of Arc, from Shaw's play about her: "When will the world be ready to receive its saints?"

I think we know the answer -- when they are dead, pasteurized and homogenized and simplified into stereotypes, and the true depth and integrity of their ideas and initiatives are forgotten.

Nelson's position is stronger yet, because he laments the way in which Engelbart and his visions of the power of computing were treated throughout his career. How, he wails, could we have let this vision slip through our hands while Engelbart lived among us?

Instead, we worked on another Java IDE or a glue language for object-relational mapping. All the while, as Nelson says, "the urgent and complex problems of mankind have only grown more urgent and more complex."

This teaching is difficult; who can accept it?


Posted by Eugene Wallingford | Permalink | Categories: Computing

January 08, 2014 3:06 PM

"I'm Not a Programmer"

In The Exceptional Beauty of Doom 3's Source Code, Shawn McGrath first says this:

I've never really cared about source code before. I don't really consider myself a 'programmer'.

Then he says this:

Dyad has 193k lines of code, all C++.

193,000 lines of C++? Um, dude, you're a programmer.

Even so, the point is worth thinking about. For most people, programming is a means to an end: a way to create something. Many CS students start with a dream program in mind and only later, like McGrath, come to appreciate code for its own sake. Some of our graduates never really get there, and appreciate programming mostly for what they can do with it.

If the message we send from academic CS is "come to us only if you already care about code for its own sake", then we may want to fix our message.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

January 07, 2014 4:09 PM

Know The Past

In this profile of computational geneticist Jason Moore, the scientist speaks explains how his work draws on work from the 1910s, which may offer computational genetics a better path forward than the work that catapulted genetics forward in the 1990s and 2000s.

Yet despite his use of new media and advanced technology, Moore spends a lot of time thinking about the past. "We have a lot to learn from early geneticists," he says. "They were smart people who were really thinking deeply about the problem."

Today, he argues, genetics students spend too much time learning to use the newest equipment and too little time reading the old genetics literature. Not surprisingly, given his ambivalent attitude toward technology, Moore believes in the importance of history. "Historical context is so important for what we do," he says. "It provides a grounding, a foundation. You have to understand the history in order ... to understand your place in the science."

Anyone familiar with the history of computing knows there is another good reason to know your history: Sometimes, we dream too small these days, and settle for too little. We have a lot to learn from early computer scientists.

I intend to make this a point of emphasis in my algorithms course this spring. I'd like to expose students to important new ideas outside the traditional canon (more on that soon), while at the same time exposing them to some of the classic work that hasn't been topped.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

December 24, 2013 11:35 AM

Inverting the Relationship Between Programs and Literals

This chapter on quasi-literals in the E language tells the story of Scott Kim teaching the author that Apple's HyperCard was "powerful in a way most of had not seen before". This unexpected power led many people to misunderstand its true importance.

Most programs are written as text, sequences of characters. In this model, a literal is a special form of embedded text. When the cursor is inside the quotes of a literal string, we are "effectively no longer in a program editor, but in a nested text editor". Kim calls this a 'pun': Instead of writing text to be evaluated by another program, we are creating output directly.

What if we turn things inside out and embed our program in literal text?

Hypercard is normally conceived of as primarily a visual application builder with an embedded silly programming language (Hypertalk). Think instead of a whole Hypercard stack as a mostly literal program. In addition to numbers and strings, the literals you can directly edit include bitmaps, forms, buttons, and menus. You can literally assemble most things, but where you need dynamic behavior, there's a hole in the literal filled in by a Hypertalk script. The program editor for this script is experienced as being nested inside the direct manipulation user interface editor.

There is a hole in the literal text, where a program goes, instead of a hole in the program, where literal text goes.

HyperTalk must surely have seemed strange to most programmers in 1987. Lisp programmers had long used macros and so knew the power of nesting code to be eval'ed inside of literal text. Of course, the resulting text was then passed on the eval to be treated again as program!

The inside-out idea of HyperCard is alive today in the form of languages such as PHP, which embed code in HTML text:

    <body>
    <?php
      echo $_SERVER['HTTP_USER_AGENT'];
    ?>
    </body>

This is a different way to think about programming, one perhaps suitable for bringing experts in some domains toward the idea of writing code gradually from documents in their area of expertise.

I sometimes have the students in my compiler course implement a processor for a simple Mustache-like template language as an early warm-up homework assignment. I do not usually require them to go as far as Turing-complete embedded code, but they create a framework that makes it possible. I think I'll look for ways to bring more of this idea into the next offering of our more general course on programming languages.

(HyperCard really was more than many people realized at the time. The people who got it became Big Fans, and the program still has an ardent following. Check out this brief eulogy, which rhapsodizes on "the mystically-enchanting mantra" at the end of the application's About box: "A day of acquaintance, / And then the longer span of custom. / But first -- / The hour of astonishment.")


Posted by Eugene Wallingford | Permalink | Categories: Computing

December 16, 2013 2:20 PM

More Fun with Integer "Assembly Language": Brute-Forcing a Function Minimum

Or: Irrational Exuberance When Programming

My wife and daughter laughed at me yesterday.

A few years ago, I blogged about implementing Farey sequences in Klein, a language for which my students at the time were writing a compiler. Klein was a minimal functional language with few control structures, few data types, and few built-in operations. Computing rational approximations using Farey's algorithm was a challenge in Klein that I likened to "integer assembly programming".

I clearly had a lot of fun with that challenge, especially when I had the chance to watch my program run using my students' compilers.

This semester, I am again teaching the compiler course, and my students are writing a compiler for a new version of Klein.

Last week, while helping my daughter with a little calculus, I ran across a fun new problem to solve in Klein:

the task of optimizing cost across the river

There are two stations on opposite sides of a river. The river is 3 miles wide, and the stations are 5 miles apart along the river. We need to lay pipe between the stations. Pipe laid on land costs $2.00/foot, and pipe laid across the river costs $4.00/foot. What is the minimum cost of the project?

This is the sort of optimization problem one often encounters in calculus textbooks. The student gets to construct a couple of functions, differentiate one, and find a maximum or minimum by setting f' to 0 and solving.

Solving this problem in Klein creates some of challenges. Among them are that ideally it involves real numbers, which Klein doesn't support, and that it requires a square root function, which Klein doesn't have. But these obstacles are surmountable. We already have tools for computing roots using Newton's method in our collection of test programs. Over a 3mi-by-5mi grid, an epsilon of a few feet approximates square roots reasonably well.

My daughter's task was to use the derivative of the cost function but, after talking about the problem with her, I was interested more in "visualizing" the curve to see how the cost drops as one moves in from either end and eventually bottoms out for a particular length of pipe on land.

So I wrote a Klein program that "brute-forces" the minimum. It loops over all possible values in feet for land pipe and compares the cost at each value to the previous value. It's easy to fake such a loop with a recursive function call.

The programmer's challenge in writing this program is that Klein has no local variables other function parameters. So I had to use helper functions to simulate caching temporary variables. This allowed me to give a name to a value, which makes the code more readable, but most importantly it allowed me to avoid having to recompute expensive values in what was already a computationally-expensive program.

This approach creates another, even bigger challenge for my students, the compiler writers. My Klein program is naturally tail recursive, but tail call elimination was left as an optional optimization in our class project. With activation records for all the tail calls stored on the stack, a compiler has to use a lot of space for its run-time memory -- far more than is available on our default target machine.

How many frames do we need? Well, we need to compute the cost at every foot along a (5 miles x 5280 feet/mile) rectangle, for a total of 26,400 data points. There will, of course, be other activation records while computing the last value in the loop.

Will I be able to see the answer generated by my program using my students' compilers? Only if one or more of the teams optimized tail calls away. We'll see soon enough.

So, I spent an hour or so writing Klein code and tinkering with it yesterday afternoon. I was so excited by the time I finished that I ran upstairs to tell my wife and daughter all about it: my excitement at having written the code, and the challenge it sets for my students' compilers, and how we could compute reasonable approximations of square roots of large integers even without real numbers, and how I implemented Newton's method in lieu of a sqrt, and...

That's when my wife and daughter laughed at me.

That's okay. I am programmer. I am still excited, and I'd do it again.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal, Teaching and Learning

December 11, 2013 12:01 PM

"Costs $20" is Functionally Indistinguishable from Gone

In his write-up on the origin of zero-based indexing in computing, Mike Hoye comments on the difficulties he had tracking down original sources:

Part of the problem is access to the historical record, of course. I was in favor of Open Access publication before, but writing this up has cemented it: if you're on the outside edge of academia, $20/paper for any research that doesn't have a business case and a deep-pocketed backer is completely untenable, and speculative or historic research that might require reading dozens of papers to shed some light on longstanding questions is basically impossible. There might have been a time when this was OK and everyone who had access to or cared about computers was already an IEEE/ACM member, but right now the IEEE -- both as a knowledge repository and a social network -- is a single point of a lot of silent failure. "$20 for a forty-year-old research paper" is functionally indistinguishable from "gone", and I'm reduced to emailing retirees to ask them what they remember from a lifetime ago because I can't afford to read the source material.

I'm an academic. When I am on campus, I have access to the ACM Digital Library. When I go home, I do not. I could pay for a personal subscription, but that seems an unnecessary expense when I am on campus so much.

I never have access to IEEE Xplore, Hoy's "single point of silent failure". Our university library chose to drop its institutional subscription a few years ago, and for good reason: it is ridiculously expensive, especially relative to the value we receive from it university-wide. (We don't have an engineering college.) We inquired about sharing a subscription with our sister schools, as we are legally under a single umbrella, but at least at that time, IEEE didn't allow such sharing.

What about non-academics, such as Hoye? We are blessed in computing with innumerable practitioners who study our history, write about, and create new ideas. Some are in industry and may have access to these resources, or an expense account. Many others, though, work on their own as independent contractors and researchers. They need access to materials, and $20 a pop is an acceptable expense.

Their loss if our loss. If Hoye had not written his article on the history of zero-based indexing, most of us wouldn't know the full story.

As time goes by, I hope that open access to research publications continues to grow. We really shouldn't have to badger retired computer scientists with email asking what they remember now about a topic they wrote an authoritative paper on forty years ago.


Posted by Eugene Wallingford | Permalink | Categories: Computing

December 10, 2013 3:33 PM

Your Programming Language is Your Raw Material, Too

Recently someone I know retweeted this familiar sentiment:

If carpenters were hired like programmers:
"Must have at least 5 years experience with the Dewalt 18V 165mm Circular Saw"

This meme travels around the world in various forms all the time, and every so often it shows up in one of my inboxes. And every time I think, "There is more to the story."

In one sense, the meme reflects a real problem in the software world. Job ads often use lists of programming languages and technologies as requirements, when what the company presumably really wants is a competent developer. I may not know the particular technologies on your list, or be expert in them, but if I am an experienced developer I will be able to learn them and become an expert.

Understanding and skill run deeper than a surface list of tools.

But. A programming language is not just a tool. It is a building material, too.

Suppose that a carpenter uses a Dewalt 18V 165mm circular saw to add a room to your house. When he finishes the project and leaves your employ, you won't have any trace of the Dewalt in his work product. You will have a new room.

He might have used another brand of circular saw. He may not have used a power tool at all, preferring the fine craftsmanship of a handsaw. Maybe he used no saw of any kind. (What a magician!) You will still have the same new room regardless, and your life will proceed in the very same way.

Now suppose that a programmer uses the Java programming language to add a software module to your accounting system. When she finishes the project and leaves your employ, you will have the results of running her code, for sure. But you will have a trace of Java in her work product. You will have a new Java program.

If you intend to use the program again, to generate a new report from new input data, you will need an instance of the JVM to run it. If want to modify the program to work differently, then you will also need a Java compiler to create the byte codes that run in the JVM. If you want to extend the program to do more, then you again will need a Java compiler and interpreter.

Programs are themselves tools, and we use programming languages to build them. So, while the language itself is surely a tool at one level, at another level it is the raw material out of which we create other things.

To use a particular language is to introduce a slew of other dependencies to the overall process: compilers, interpreters, libraries, and sometimes even machine architectures. In the general case, to use a particular language is to commit at least some part of the company's future attention to both the language and its attendant tooling.

So, while I am sympathetic to sentiment behind our recurring meme, I think it's important to remember that a programming language is more than just a particular brand of power tool. It is the stuff programs are made of.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

December 04, 2013 3:14 PM

Agile Moments, "Why We Test" Edition

Case 1: Big Programs.

This blog entry tells the sad story of a computational biologist who had to retract six published articles. Why? Their conclusions depended on the output of a computer program, and that program contained a critical error. The writer of the entry, who is not the researcher in question, concludes:

What this should flag is the necessity to aggressively test all the software that you write.

Actually, you should have tests for any program you use to draw important conclusions, whether you wrote it or not. The same blog entry mentions that a grad student in the author's previous lab had found several bugs a molecular dynamics program used by many computational biologists. How many published results were affected before they were found?

Case 2: Small Scripts.

Titus Brown reports finding bugs every time he reused one of his Python scripts. Yet:

Did I start doing any kind of automated testing of my scripts? Hell no! Anyone who wants to write automated tests for all their little scriptlets is, frankly, insane. But this was one of the two catalysts that made me personally own up to the idea that most of my code was probably somewhat wrong.

Most of my code has bugs but, hey, why write tests?

Didn't a famous scientist define insanity as doing the same thing over and over but expecting different results?

I consider myself insane, too, but mostly because I don't write tests often enough for my small scripts. We say to ourselves that we'll never reuse them, so we don't need tests. But we don't throw them away, and then we do reuse them, perhaps with a tweak here or there.

We all face time constraints. When we run a script the first time, we may well pay enough attention to the output that we are confident it is correct. But perhaps we can all agree that the second time we use a script, we should write tests for it if we don't already have them.

There are only three numbers in computing, 0, 1, and many. The second time we use a program is a sign from the universe that we need the added confidence provided by tests.

To be fair, Brown goes on to offer some good advice, such as writing tests for code after you find a bug in it. His article is an interesting read, as is almost everything he writes about computation and science.

Case 3: The Disappointing Trade-Off.

Then there's this classic from Jamie Zawinski, as quoted in Coders at Work:

I hope I don't sound like I'm saying, "Testing is for chumps." It's not. It's a matter of priorities. Are you trying to write good software or are you trying to be done by next week? You can't do both.

Sigh. If you you don't have good software by next week, maybe you aren't done yet.

I understand that the real world imposes constraints on us, and that sometimes worse is better. Good enough is good enough, and we rarely need a perfect program. I also understand that Zawinski was trying to be fair to the idea of testing, and that he was surely producing good enough code before releasing.

Even still, the pervasive attitude that we can either write good programs or get done on time, but not both, makes me sad. I hope that we can do better.

And I'm betting that the computational biologist referred to in Case 1 wishes he had had some tests to catch the simple error that undermined five years worth of research.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 30, 2013 9:45 AM

The Magic at the Heart of AI

This paragraph from The Man Who Would Teach Machines to Think expresses a bit of my uneasiness with the world of AI these days:

As our machines get faster and ingest more data, we allow ourselves to be dumber. Instead of wrestling with our hardest problems in earnest, we can just plug in billions of examples of them. Which is a bit like using a graphing calculator to do your high-school calculus homework -- it works great until you need to actually understand calculus.

I understand the desire to solve real problems and the resulting desire to apply opaque mathematics to large data sets. Like most everyone, I revel in what Google can do for me and watch in awe when Watson defeats the best human Jeopardy! players ever. But for me, artificial intelligence was about more than just getting the job done.

Over the years teaching AI, my students often wanted to study neural networks in much greater detail than my class tended to go. But I was more interested in approaches to AI and learning that worked at a more conceptual level. Often we could find a happy middle ground while studying genetic algorithms, which afforded them the magic of something-for-nothing and afforded me the potential for studying ideas as they evolved over time.

(Maybe my students were simply exhibiting Astrachan's Law.)

When I said goodbye to AAAI a few years ago, I mentioned Hofstadter's work as one of my early inspirations -- Gödel, Escher, Bach and the idea of self-reference, with its "intertwining worlds of music, art, mathematics, and computers". That entry said I was leaving AAAI because my own work had moved in a different direction. But it left unstated a second truth, which The Man Who Would Teach Machines to Think asserts as Hofstadter's own reason for working off the common path: the world of AI had moved in a different direction, too.

For me, as for Hofstadter, AI has always meant more than engineering a solution. It was about understanding scientifically something that seemed magical, something that is both deeply personal and undeniably universal to human experience, about how human consciousness seems to work. My interest in AI will always lie there.

~~~~~

If you enjoy the article about Hofstadter and his work linked above, perhaps you will enjoy a couple of entries I wrote after he visited my university last year:


Posted by Eugene Wallingford | Permalink | Categories: Computing

November 24, 2013 10:54 AM

Teaching Algorithms in 2014

This spring, I will be teaching the undergraduate algorithms course for first time in nine years, since the semester before I became department head. I enjoy this course. It gives both the students and me opportunities to do a little theory, a little design, and a little programming. I also like to have some fun, using what we learn to play games and solve puzzles.

Nine years is a long time in computing, even in an area grounded in well-developed theory. I will need to teach a different sort of course. At the end of this entry, I ask for your help in meeting this challenge.

Algorithms textbooks don't look much different now than they did in the spring of 2005. Long-time readers of this blog know that I face the existential crisis of selecting a textbook nearly every semester. Picking a textbook requires balancing several forces, including the value they give to the instructor, the value they give to the student during and after the course, and the increasing expense to students.

My primary focus in these decisions is always on net value to the students. I like to write my own material anyway. When time permits, I'd rather write as much as I can for students to read than outsource that responsibility (and privilege) to a textbook author. Writing my lecture notes in depth lets me weave a lot of different threads together, including pointers into primary and secondary sources. Students benefit from learning to read non-textbook material, the sort they will encounter as throughout their careers.

My spring class brings a new wrinkle to the decision, though. Nearly fifty students are enrolled, with the prospect a few more to come. This is a much larger group than I usually work with, and large classes carry a different set of risks than smaller courses. In particular, when something goes wrong in a small section, it is easier to recover through one-on-one remediation. That option is not so readily available for a fifty-person course.

There is more risk in writing new lecture material than in using a textbook that has been tested over time. A solid textbook can be a security blanket as much for the instructor as for the student. I'm not too keen on selecting a security blanket for myself, but the predictability of a text is tempting. There is one possible consolation in such a choice: perhaps subordinating my creative impulses to the design of someone's else's textbook will make me more creative as a result.

But textbook selection is a fairly ordinary challenge for me. The real question is: Which algorithms should we teach in this course, circa 2014? Surely the rise of big data, multi-core processors, mobile computing, and social networking require a fresh look at the topics we teach undergrads.

Perhaps we need only adjust the balance of topics that we currently teach. Or maybe we need to add a new algorithm or data structure to the undergraduate canon. If we teach a new algorithm, or a new class of algorithms, which standard material should be de-emphasized, or displaced altogether? (Alas, the semester is still but fifteen weeks long.)

Please send me your suggestions! I will write up a summary of the ideas you share, and I will certainly use your suggestions to design a better algorithms course for my students.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 19, 2013 4:49 PM

First Model, Then Improve

Not long ago, I read Unhappy Truckers and Other Algorithmic Problems, an article by Tom Vanderbilt that looks at efforts to optimize delivery schedules at UPS and similar companies. At the heart of the challenge lies the traveling salesman problem. However, in practice, the challenge brings companies face-to-face with a bevy of human issues, from personal to social, psychological to economic. As a result, solving this TSP is more complex than what we see in the algorithms courses we take in our CS programs.

Yet, in the face of challenges both computational and human, the human planners working at these companies do a pretty good job. How? Over the course of time, researchers figured out that finding optimal routes shouldn't be their main goal:

"Our objective wasn't to get the best solution," says Ted Gifford, a longtime operations research specialist at Schneider. "Our objective was to try to simulate what the real world planners were really doing."

This is a lesson I learned the hard way, too, back in graduate school, when my advisor's lab was trying to build knowledge-based systems for real clients, in chemical engineering, aeronautics, business, and other domains. We were working with real people who were solving hard problems under serious constraints.

At the beginning I was a typically naive programmer, armed with fancy AI techniques and unbounded enthusiasm. I soon learned that, if you walk into a workplace and propose to solve all the peoples' problems with a program, things don't go as smoothly as the programmer might hope.

First of all, this impolitic approach generally creates immediate pushback. These are people, with personal investment in the way things work now. They tend to bristle when a 20-something grad student walks in the door promoting the wonder drug for all their ills. Some might even fear that you are right, and success for your program will mean negative consequences for them personally. We see this dynamic in Vanderbilt's article.

There's a deeper reason that things don't go so smoothly, though, and it's the real lesson of Vanderbilt's piece. Until you implement the existing solution to the problem, you don't really understand the problem yet.

These problems are complex, often with many more constraints than typical theoretical solutions have dealt with. The humans solving the problem often have many years of experience contributing to their approach. They have deep knowledge of the domain, but also repeated exposure to the exceptions and edge cases that sometimes confound theoretical solutions. They use heuristics that are hard to tease apart or articulate.

I learned that it's easy to solve a problem if you are solving the wrong one.

A better way to approach these challenges is: First, model the existing system, including the extant solution. Then, look for ways to improve on the solution.

This approach often gives everyone involved greater confidence that the programmers understand -- and so are solving -- the right problem. It also enables the team to make small, incremental changes to the system, with a correspondingly higher probability of success. Together, these two outcomes greatly increase the chance of human buy-in from the current workers. This makes it easier for the whole team to recognize the need for larger-scale changes to the process, and to support and contribute to an improved solution.

Vanderbilt tells a similarly pragmatic story. He writes:

When I suggest to Gifford that he's trying to understand the real world, mathematically, he concurs, but adds: "The word 'understand' is too strong--we are happy to get positive outcomes."

Positive outcomes are what the company wants. Fortunately for the academics who work on such problems in industry, achieving good outcomes is often an effective way to test theories, encounter their shortcomings, and work on improvements. That, too, is something I learned in grad school. It was a valuable lesson.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 14, 2013 2:55 PM

Toward A New Data Science Culture in Academia

Fernando Perez has a nice write-up, An Ambitious Experiment in Data Science, describing a well-funded new project in which teams at UC Berkeley, the University of Washington, and NYU will collaborate to "change the culture of universities to create a data science culture". A lot of people have been quoting Perez's entry for its colorful assessment of academic incentives and reward structures. I like this piece for the way Perez defines and outlines the problem, in terms of both data science across disciplines and academic culture in general.

For example:

Most scientists are taught to treat computation as an afterthought. Similarly, most methodologists are taught to treat applications as an afterthought.

Methodologists here includes computer scientists, who are often more interested in new data structures, algorithms, and protocols.

This "mirror" disconnect is a problem for a reason many people already understand well:

Computation and data skills are all of a sudden everybody's problem.

(Here are a few past entries of mine that talk about how programming and the nebulous "computational thinking" have spread far and wide: 1 | 2 | 3 | 4.)

Perez rightly points out that the open-source software, while imperfect, often embodies the principles or science and scientific collaboration better than the academy. It will be interesting to see how well this data science project can inject OSS attitudes into big research universities.

He is concerned because, as I have noted before, are, as a whole, a conservative lot. Perez says this in a much more entertaining way:

There are few organizations more proud of their traditions and more resistant to change than universities (churches and armies might be worse, but that's about it).

I think he gives churches and armies more credit than they deserve.

The good news is that experiments of the sort being conducted in the Berkley/UW/NYU project are springing up on a smaller scale around the world. There is some hope for big change in academic culture if a lot of different people at a lot of different institutions experiment, learn, and create small changes that can grow together as they bump into one another.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

October 29, 2013 3:49 PM

My PLoP 2013 Retrospective

wrapper of the Plopp candy bar I received from Rebecca Rikner

PLoP 2013 was as much fun and as invigorating as I had hoped it would be. I hadn't attended in eight years, but it didn't take long to fall back into the rhythm of writers' workshops interspersed among invited talks, focus group sessions, BoFs, mingling, and (yes) games.

I was lead moderator for Workshop 010010, which consisted of pedagogical patterns papers. The focus of all of them was interactivity, whether among students building LEGO Mindstorms robots or among students and instructor on creative projects. The idea of the instructor as an active participant, even "generative" in the sense meant by Christopher Alexander, dominated our discussion. I look forward to seeing published versions of the papers we discussed.

The other featured events included invited talks by Jenny Quillien and Ward Cunningham and a 20-year retrospective panel featuring people who were present at the beginning of PLoP, the Hillside Group, and software patterns.

Quillien spent six years working with Alexander during the years he created The Nature of Order. Her talk shared some of the ways that Alexander was disappointed in the effect of his seminal "A Pattern Language" had on the world, both as a result of people misunderstanding it and as a result of the books inherent faults. Along the way, she tried to give pragmatic advice to people trying to document patterns of software. I may try to write up some of her thoughts, and some of my own in response, in the coming weeks.

Cunningham presented his latest work on federated wiki, the notion of multiple, individual wikis "federated" in relationships that share and present information for a common good. Unlike the original wiki, in which collaboration happened in a common document, federated wiki has a fork button on every page. Anyone can copy, modify, and share pages, which are then visible to everyone and available for merging back into the home wikis.

the favicon for my federated wiki on Ward's server

Ward set me up with a wiki in the federation on his server before I left on Saturday. I want to play with it a bit before I say much more than this: Federated wiki could change how communities share and collaborate in much the same way that wiki did.

I also had the pleasure of participating in one other structured activity while at PLoP. Takashi Iba and his students at Keio University in Japan are making a documentary about the history of the patterns community. Takashi invited me to sit for an interview about pedagogical patterns and their history within the development of software patterns. I was happy to help. It was a fun challenge to explain my understanding of what a pattern language is, and to think about what my colleagues and I struggled with in trying to create small pattern languages to guide instruction. Of course, I strayed off to the topic of elementary patterns as well, and that led to more interesting discussion with Takashi. I look forward to seeing their film in the coming years.

More so than even other conferences, unstructured activity plays a huge role in any PLoP conference. I skipped a few meals so that I could walk the extensive gardens and grounds of Allerton Park (and also so that I would not gain maximum pounds from the plentiful and tasty meals that were served). I caught up with old friends such as Ward, Kyle Brown, Bob Hanmer, Ralph Johnson, and made too many new friends to mention here. All the conversation had my mind swirling with new projects and old... Forefront in my mind is exploring again the idea of design and implementation patterns of functional programming. The time is still right, and I want to help.

Now, to write my last entry or two from StrangeLoop...

~~~~

Image 1. A photo of the wrapper of a Plopp candy bar, which I received as a gift from Rebecca Rikner. PLoP has a gifting tradition, and I received a box full of cool tools, toys, mementoes, and candy. Plopp is a Swedish candy bar, which made it a natural gift for Rebecca to share from her native land. (It was tasty, too!)

Image 2. The favicon for my federated wiki on Ward's server, eugene.fed.wiki.org. I like the color scheme that fed.wiki.org gave me -- and I'm glad to be early enough an adopter that I could claim my first name as the name of my wiki. The rest of the Eugenes in the world will have to settle for suffix numbers and all the other contortions that come with arriving late to the dance.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

October 19, 2013 7:38 AM

The Proto Interpreter for J

(Update: Josh Grams took my comment about needing a week of work to grok this code as a challenge. He figured it out much more quickly than that and wrote up an annotated version of the program as he went along.

I like finding and reading about early interpreters for programming languages, such as the first Lisp interpreter or Smalltalk-71, which grew out of a one-page proof of concept written by Dan Ingalls on a bet with Alan Kay.

So I was quite happy recently to run across Appendix A from An Implementation of J, from which comes the following code. Arthur Whitney whipped up this one-page interpreter fragment for the AT&T 3B1 one weekend in 1989 to demonstrate his idea for a new APL-like like language. Roger Hui studied this interpreter for a week before writing the first implementation of J.

typedef char C;typedef long I;
typedef struct a{I t,r,d[3],p[2];}*A;
#define P printf
#define R return
#define V1(f) A f(w)A w;
#define V2(f) A f(a,w)A a,w;
#define DO(n,x) {I i=0,_n=(n);for(;i<_n;++i){x;}}
I *ma(n){R(I*)malloc(n*4);}mv(d,s,n)I *d,*s;{DO(n,d[i]=s[i]);}
tr(r,d)I *d;{I z=1;DO(r,z=z*d[i]);R z;}
A ga(t,r,d)I *d;{A z=(A)ma(5+tr(r,d));z->t=t,z->r=r,mv(z->d,d,r);
 R z;}
V1(iota){I n=*w->p;A z=ga(0,1,&n);DO(n,z->p[i]=i);R z;}
V2(plus){I r=w->r,*d=w->d,n=tr(r,d);A z=ga(0,r,d);
 DO(n,z->p[i]=a->p[i]+w->p[i]);R z;}
V2(from){I r=w->r-1,*d=w->d+1,n=tr(r,d);
 A z=ga(w->t,r,d);mv(z->p,w->p+(n**a->p),n);R z;}
V1(box){A z=ga(1,0,0);*z->p=(I)w;R z;}
V2(cat){I an=tr(a->r,a->d),wn=tr(w->r,w->d),n=an+wn;
 A z=ga(w->t,1,&n);mv(z->p,a->p,an);mv(z->p+an,w->p,wn);R z;}
V2(find){}
V2(rsh){I r=a->r?*a->d:1,n=tr(r,a->p),wn=tr(w->r,w->d);
 A z=ga(w->t,r,a->p);mv(z->p,w->p,wn=n>wn?wn:n);
 if(n-=wn)mv(z->p+wn,z->p,n);R z;}
V1(sha){A z=ga(0,1,&w->r);mv(z->p,w->d,w->r);R z;}
V1(id){R w;}V1(size){A z=ga(0,0,0);*z->p=w->r?*w->d:1;R z;}
pi(i){P("%d ",i);}nl(){P("\n");}
pr(w)A w;{I r=w->r,*d=w->d,n=tr(r,d);DO(r,pi(d[i]));nl();
 if(w->t)DO(n,P("< ");pr(w->p[i]))else DO(n,pi(w->p[i]));nl();}

C vt[]="+{~<#,"; A(*vd[])()={0,plus,from,find,0,rsh,cat}, (*vm[])()={0,id,size,iota,box,sha,0}; I st[26]; qp(a){R a>='a'&&a<='z';}qv(a){R a<'a';} A ex(e)I *e;{I a=*e; if(qp(a)){if(e[1]=='=')R st[a-'a']=ex(e+2);a= st[ a-'a'];} R qv(a)?(*vm[a])(ex(e+1)):e[1]?(*vd[e[1]])(a,ex(e+2)):(A)a;} noun(c){A z;if(c<'0'||c>'9')R 0;z=ga(0,0,0);*z->p=c-'0';R z;} verb(c){I i=0;for(;vt[i];)if(vt[i++]==c)R i;R 0;} I *wd(s)C *s;{I a,n=strlen(s),*e=ma(n+1);C c; DO(n,e[i]=(a=noun(c=s[i]))?a:(a=verb(c))?a:c);e[n]=0;R e;}

main(){C s[99];while(gets(s))pr(ex(wd(s)));}

I think it will take me a week of hard work to grok this code, too. Whitney's unusually spare APL-like C programming style is an object worthy of study in its own right.

By the way, Hui's Appendix A bears the subtitle Incunabulum, a word that means a work of art or of industry of an early period. So, I not only discovered a new bit of code this week; I also learned a cool new word. That's a good week.


Posted by Eugene Wallingford | Permalink | Categories: Computing

October 16, 2013 11:38 AM

Poetry as a Metaphor for Software

I was reading Roger Hui's Remembering Ken Iverson this morning on the elliptical, and it reminded me of this passage from A Conversation with Arthur Whitney. Whitney is a long-time APL guru and the creator of the A, K, and Q programming languages. The interviewer is Bryan Cantrill.

BC: Software has often been compared with civil engineering, but I'm really sick of people describing software as being like a bridge. What do you think the analog for software is?

AW: Poetry.

BC: Poetry captures the aesthetics, but not the precision.

AW: I don't know, maybe it does.

A poet's use of language is quite precise. It must balance forces in many dimensions, including sound, shape, denotation, and connotation. Whitney seems to understand this. Richard Gabriel must be proud.

Brevity is a value in the APL world. Whitney must have a similar preference for short language names. I don't know the source of his names A, K, and Q, but I like Hui's explanation of where J's name came from:

... on Sunday, August 27, 1989, at about four o'clock in the afternoon, [I] wrote the first line of code that became the implementation described in this document.

The name "J" was chosen a few minutes later, when it became necessary to save the interpreter source file for the first time.

Beautiful. No messing around with branding. Gotta save my file.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

October 11, 2013 1:42 PM

The Tang of Adventure, and a Lively Appreciation

"After you've learned the twelve times table," John Yarnelle asks, "what else is there to do?"

The concepts of modern mathematics give the student something else to do in great abundance and variety at all levels of his development. Not only may he discover unusual types of mathematical structures where, believe it or not, two and two does not equal four, but he may even be privileged to invent a new system virtually on his own. Far from a sense of stagnation, there is the tang of adventure, the challenge of exploration; perhaps also a livelier appreciation of the true nature of mathematical activity and mathematical thought.

Not only the tang of adventure; students might also come to appreciate what math really is. That's an admirable goal for any book or teacher.

This passage comes from Yarnelle's Finite Mathematical Structures, a 1964 paperback that teaches fields, groups, and algebras with the prose of a delighted teacher. I picked this slender, 66-page gem up off a pile of books being discarded by a retired math professor a decade ago. How glad I am that none of the math profs who walked past that pile bothered to claim it before I happened by.

We could use a few CS books like this, too.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

October 07, 2013 12:07 PM

StrangeLoop: Exercises in Programming Style

[My notes on StrangeLoop 2013: Table of Contents]

Crista Lopes

I had been looking forward to Crista Lopes's StrangeLoop talk since May, so I made sure I was in the room well before the scheduled time. I even had a copy of the trigger book in my bag.

Crista opened with something that CS instructors have learned the hard way: Teaching programming style is difficult and takes a lot of time. As a result, it's often not done at all in our courses. But so many of our graduates go into software development for the careers, where they come into contact with many different styles. How can they understand them -- well, quickly, or at all?

To many people, style is merely the appearance of code on the screen or printed. But it's not. It's more, and something entirely different. Style is a constraint. Lopes used images of a few stylistic paintings to illustrate the idea. If an artist limits herself to pointillism or cubism, how can she express important ideas? How does the style limit the message, or enhance it?

But we know this is true of programming as well. The idea has been a theme in my teaching for many years. I occasionally write about the role of constraints in programming here, including Patterns as a Source of Freedom, a few programming challenges, and a polymorphism challenge that I've run as a workshop.

Lopes pointed to a more universal example, though: the canonical The Elements of Programming Style. Drawing on this book and other work in software, she said that programming style ...

  • is a way to express tasks
  • exists at all scales
  • recurs at multiple scales
  • is codified in programming language

For me, the last bullet ties back most directly to idea of style as constraint. A language makes some things easier to express than others. It can also make some things harder to express. There is a spectrum, of course. For example, some OO languages make it easy to create and use objects; others make it hard to do anything else! But the language is an enabler and enforcer of style. It is a proxy for style as a constraint on the programmer.

Back to the talk. Lopes asked, Why is it so important that we understand programming style? First, a style provides the reader with a frame of reference and a vocabulary. Knowing different styles makes us a more effective consumers of code. Second, one style can be more appropriate for a given problem or context than another style. So, knowing different styles makes us a more effective producers of code. (Lopes did not use the producer-consumer distinction in the talk, but it seems to me a nice way to crystallize her idea.)

the cover of Raymond Queneau's Exercises in Style

The, Lopes said, I came across Raymond Queneau's playful little book, "Exercises in Style". Queneau constrains himself in many interesting ways while telling essentially the same story. Hmm... We could apply the same idea to programming! Let's do it.

Lopes picked a well-known problem, the common word problem famously solved in a Programming Pearls column more than twenty-five years. This is a fitting choice, because Jon Bentley included in that column a critique of Knuth's program by Doug McIlroy, who considered both engineering concerns and program style in his critique.

The problem is straightforward: identify and print the k most common terms that occur in a given text document, in decreasing order. For the rest of the talk, Lopes presented several programs that solve the problem, each written in a different style, showing code and highlighting its shape and boundaries.

Python was her language of choice for the examples. She was looking for a language that many readers would be able to follow and understand, and Python has the feel of pseudo-code about it. (I tell my students that it is the Pascal of their time, though I may as well be speaking of hieroglyphics.) Of course, Python has strengths and weaknesses that affect its fit for some styles. This is an unavoidable complication for all communication...

Also, Lopes did not give formal names to the styles she demonstrated. Apparently, at previous versions of this talk, audience members had wanted to argue over the names more than the styles themselves! Vowing not to make that mistake again, she numbered her examples for this talk.

That's what programmers do when they don't have good names.

In lieu of names, she asked the crowd to live-tweet to her what they thought each style is or should be called. She eventually did give each style a fun, informal name. (CS textbooks might be more evocative if we used her names instead of the formal ones.)

I noted eight examples shown by Lopes in the talk, though there may have been more:

  • monolithic procedural code -- "brain dump"
  • a Unix-style pipeline -- "code golf"
  • procedural decomposition with a sequential main -- "cook book"
  • the same, only with functions and composition -- "Willy Wonka"
  • functional decomposition, with a continuation parameter -- "crochet"
  • modules containing multiple functions -- "the kingdom of nouns"
  • relational style -- (didn't catch this one)
  • functional with decomposition and reduction -- "multiplexer"

Lopes said that she hopes to produce solutions using a total of thirty or so styles. She asked the audience for help with one in particular: logic programming. She said that she is not a native speaker of that style, and Python does not come with a logic engine built-in to make writing a solution straightforward.

Someone from the audience suggested she consider yet another style: using a domain-specific language. That would be fun, though perhaps tough to roll from scratch in Python. By that time, my own brain was spinning away, thinking about writing a solution to the problem in Joy, using a concatenative style.

Sometimes, it's surprising just how many programming styles and meaningful variations people have created. The human mind is an amazing thing.

The talk was, I think, a fun one for the audience. Lopes is writing a book based on the idea. I had a chance to review an early draft, and now I'm looking forward to the finished product. I'm sure I'll learn something new from it.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development, Teaching and Learning

October 04, 2013 3:12 PM

StrangeLoop: Rich Hickey on Channels and Program Design

[My notes on StrangeLoop 2013: Table of Contents]

Rich Hickey setting up for his talk

Rich Hickey spoke at one of the previous StrangeLoops I attended, but this was my first time to attend one of his talks in person. I took the shaky photo seen at the right as proof. I must say, he gives a good talk.

The title slide read "Clojure core.async Channels", but Hickey made a disclaimer upfront: this talk would be about what channels are and why Clojure has them, not the details of how they are implemented. Given that there were plenty of good compiler talks elsewhere at the conference, this was a welcome change of pace. It was also a valuable one, because many more people will benefit from what Hickey taught about program design than would have benefited from staring at screens full of Clojure macros. The issues here are important ones, and ones that few programmers understand very well.

The fundamental problem is this: Reactive programs need to be machines, but functions make bad machines. Even sequences of functions.

The typical solution to this problem these days is to decompose the system logic into a set of response handlers. Alas, this leads to callback hell, a modern form of spaghetti code. Why? Even though the logic has been decomposed into pieces, it is still "of a piece", essentially a single logical entity. When this whole is implemented across multiple handlers, we can't see it as a unit, or talk about it easily. We need to, though, because we need to design the state machine that it comprises.

Clojure's solution to the problem, in the form of core.async, is the channel. This is an implementation of Tony Hoare's communicating sequential process. One of the reasons that Hickey likes this approach is that it lets a program work equally well in fully threaded apps and in apps with macro-generated inversion of control.

Hickey then gave some examples of code using channels and talked a bit about the implications of the implementation for system design. For instance, the language provides handy put! and take! operators for integrating channels with code at the edge of non-core.async systems. I don't have much experience with Clojure, so I'll have to study a few examples in detail to really appreciate this.

For me, the most powerful part of the talk was an extended discussion of communication styles in program. Hickey focused on the trade-offs between direct communication via shared state and indirect communication via channels. He highlighted six or seven key distinctions between the two and how these affect the way a system works. I can't do this part of the talk justice, so I suggest you watch the video of the talk. I plan to watch it again myself.

I had always heard that Hickey was eminently quotable, and he did not disappoint. Here are three lines that made me smile:

  • "Friends don't let friends put logic in handlers."
  • "Promises and futures are the one-night stands" of asynchronous architecture.
  • "Unbounded buffers are a recipe for a bad program. 'I don't want to think about this bug yet, so I'll leave the buffer unbounded.'"

That last one captures the indefatigable optimism -- and self-delusion -- that characterizes so many programmers. We can fix that problem later. Or not.

In the end, this talk demonstrates how a good engineer approaches a problem. Clojure and its culture reside firmly in the functional programming camp. However, Hickey recognizes that, for the problem at hand, a sequence of functional calls is not the best solution. So he designs a solution that allows programmers to do FP where it fits best and to do something else where FP doesn't. That's a pragmatic way to approach problems.

Still, this solution is consistent with Clojure's overall design philosophy. The channel is a first-class object in the language. It converts a sequence of functional calls into data, whereas callbacks implement the sequence in code. As code, we see the sequence only at run-time. As data, we see it in our program and can use it in all the ways we can use any data. This consistent focus on making things into data is an attractive part of the Clojure language and the ecosystem that has been cultivated around it.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 28, 2013 12:17 PM

StrangeLoop: This and That, Volume 2

[My notes on StrangeLoop 2013: Table of Contents]

I am at a really good talk and look around the room. So many people are staring at their phones, scrolling away. So many others are staring at their laptops, typing away. The guy next to me: doing both at the same time. Kudos, sir. But you may have missed the point.

~~~~

Conference talks are a great source of homework problems. Sometimes, the talk presents a good problem directly. Others, watching the talk sets my subconscious mind in motion, and it creates something useful. My students thank you. I thank you.

~~~~

Jenny Finkel talked about the difference between two kinds of recommenders: explorers, who forage for new content, and exploiters, who want to see what's already popular. The former discovers cool new things occasionally but fails occasionally, too. The latter is satisfied most of the time but rarely surprised. As conference goes, I felt this distinction at play in my own head this year. When selecting the next talk to attend, I have to take a few risks if I ever hope to find something unexpected. But when I fail, a small regret tugs at me.

~~~~

We heard a lot of confident female voices on the StrangeLoop stages this year. Some of these speakers have advanced academic degrees, or at least experience in grad school.

~~~~

The best advice I received on Day 1 perhaps came not from a talk but from the building:

The 'Do not Climb on Bears' sign on a Peabody statue

"Please do not climb on bears." That sounds like a good idea most everywhere, most all the time.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

September 27, 2013 4:26 PM

StrangeLoop: Add All These Things

[My notes on StrangeLoop 2013: Table of Contents]

I took a refreshing walk in the rain over the lunch hour on Friday. I managed to return late and, as a result, missed the start of Avi Bryant's talk on algebra and analytics. Only a few minutes, though, which is good. I enjoyed this presentation.

Bryant didn't talk about the algebra we study in eighth or ninth grade, but the mathematical structure math students encounter in a course called "abstract" or "modern" algebra. A big chunk of the talk focused on an even narrower topic: why +, and operators like it, are cool.

One reason is that grouping doesn't matter. You can add 1 to 2, and then add 4 to the result, and have the same answer as if you added 4 to 1, and then added 2 to the result. This is, of course, the associative property.

Another is that order doesn't matter. 1 + 2 is the same as 2 + 1. That's the commutative property.

Yet another is that, if you have nothing to add, you can add nothing and have the same value you started with. 4 + 0 = 4. 0 is the identity element for addition.

Finally, when you add two numbers, you get a number back. This is not quite as true in computers as in math, because an operation can cause an overflow or underflow and create an error. But looked at through fuzzy lenses, this is true in our computers, too. This is the closure property for addition of integers and real numbers.

Addition isn't the only operation on numbers that has these properties. Finding the maximum value in a set of numbers, does, too. The maximum of two numbers is a number. max(x,y) = max(y,x), and if we have three or more numbers, it doesn't how matter how we group them; max will find the maximum among them. The identity value is tricky -- there is no smallest number... -- but in practice we can finesse this by using the smallest number of a given data type, or even allowing max to take "nothing" as a value and return its other argument.

When we see a pattern like this, Bryant said, we should generalize:

  • We have a function f that takes two values from a set and produces another member of the same set.
  • The order of f's arguments doesn't matter.
  • The grouping of f's arguments doesn't matter.
  • There is some identity value, a conceptual "zero", that doesn't matter, in the sense that f(i,zero) for any i is simply i.

There is a name for this pattern. When we have such as set and operation, we have a commutative monoid.

     S ⊕ S → S
     x ⊕ y = y ⊕ x
     x ⊕ (y ⊕ z) = (x ⊕ y) ⊕ z
     x ⊕ 0 = x

I learned about this and other such patterns in grad school when I took an abstract algebra course for kicks. No one told me at the time that I'd being seeing them again as soon as someone created the Internet and it unleashed a torrent of data on everyone.

Just why we are seeing the idea of a commutative monoid again was the heart of Bryant's talk. When we have data coming into our company from multiple network sources, at varying rates of usage and data flow, and we want to extract meaning from the data, it can be incredibly handy if the meaning we hope to extract -- the sum of all the values, or the largest -- can be computed using a commutative monoid. You can run multiple copies of your function at the entry point of each source, and combine the partial results later, in any order.

Bryant showed this much more attractively than that, using cute little pictures with boxes. But then, there should be an advantage to going to the actual talk... With pictures and fairly straightforward examples, he was able to demystify the abstract math and deliver on his talk's abstract:

A mathematician friend of mine tweeted that anyone who doesn't understand abelian groups shouldn't build analytics systems. I'd turn that around and say that anyone who builds analytics systems ends up understanding abelian groups, whether they know it or not.

That's an important point. Just because you haven't studied group theory or abstract algebra doesn't mean you shouldn't do analytics. You just need to be prepared to learn some new math when it's helpful. As programmers, we are all looking for opportunities to capitalize on patterns and to generalize code for use in a wider set of circumstances. When we do, we may re-invent the wheel a bit. That's okay. But also look for opportunities to capitalize on patterns recognized and codified by others already.

Unfortunately, not all data analysis is as simple as summing or maximizing. What if I need to find an average? The average operator doesn't form a commutative monoid with numbers. It falls short in almost every way. But, if you switch from the set of numbers to the set of pairs [n, c], where n is a number and c is a count of how many times you've seen n, then you are back in business. Counting is addition.

So, we save the average operation itself as a post-processing step on a set of number/count pairs. This turns out to be a useful lesson, as finding the average of a set is a lossy operation: it loses track of how many numbers you've seen. Lossy operations are often best saved for presenting data, rather than building them directly into the system's computation.

Likewise, finding the top k values in a set of numbers (a generalized form of maximum) can be handled just fine as long as we work on lists of numbers, rather than numbers themselves.

This is actually one of the Big Ideas of computer science. Sometimes, we can use a tool or technique to solve a problem if only we transform the problem into an equivalent one in a different space. CS theory courses hammer this home, with oodles of exercises in which students are asked to convert every problem under the sun into 3-SAT or the clique problem. I look for chances to introduce my students to this Big Idea when I teach AI or any programming course, but the lesson probably gets lost in the noise of regular classwork. Some students seem to figure it out by the time they graduate, though, and the ones who do are better at solving all kinds of problems (and not by converting them all 3-SAT!).

Sorry for the digression. Bryant didn't talk about 3-SAT, but he did demonstrate several useful problem transformations. His goal was more practical: how can we use this idea of a commutative monoid to extract as many interesting results from the stream of data as possible.

This isn't just an academic exercise, either. When we can frame several problems in this way, we are able to use a common body of code for the processing. He called this body of code an aggregator, comprising three steps:

  • prepare the data by transforming it into the space of a commutative monoid
  • reduce the data to a single value in that space, using the appropriate operator
  • present the result by transforming it back into its original space

In practice, transforming the problem into the space of a monoid presents challenges in the implementation. For example, it is straightforward to compute the number of unique values in a collection of streams by transforming each item into a set of size one and then using set union as the operator. But union requires unbounded space, and this can be inconvenient when dealing with very large data sets.

One approach is to compute an estimated number of uniques using a hash function and some fancy arithmetic. We can make the expected error in estimate smaller and smaller by using more and more hash functions. (I hope to write this up in simple code and blog about it soon.)

Bryant looked at one more problem, computing frequencies, and then closed with a few more terms from group theory: semigroup, group, and abelian group. Knowing these terms -- actually, simply knowing that they exist -- can be useful even for the most practical of practitioners. They let us know that there is more out there, should our problems become harder or our needs become larger.

That's a valuable lesson to learn, too. You can learn all about abelian groups in the trenches, but sometimes it's good to know that there may be some help out there in the form of theory. Reinventing wheels can be cool, but solving the problems you need solved is even cooler.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

September 24, 2013 4:38 PM

StrangeLoop: Compilers, Compilers, Compilers

[My notes on StrangeLoop 2013: Table of Contents]

I went to a lot of talks about compilation. There seemed to be more this year than last, but perhaps I was suffering from a perception bias. I'm teaching compilers this semester and have been reading a bit about V8 and Crankshaft on the elliptical of late.

Many of the talks I saw revolved around a common theme: dynamic run-time systems. Given the prominence these days of Javascript, Python, Ruby, Lua, and their like, it's not surprising that finding better ways to organize dynamic run-times and optimize their performance are receiving a lot of attention.

The problem of optimizing dynamic run-time systems is complicated by the wide range of tasks being performed dynamically: type checking, field access, function selection, and the most common whipping horse of my static-language friends, garbage collection. Throw in eval, which allows the execution of arbitrary code, possibly changing even the definition of core classes, and it's amazing that our dynamic languages can run in human time at all. That's a tribute to the people who have been creating compilers for us over the years.

As I listened to these talks, my ears were tuned to ideas and topics that I need to learn more about. That's what my notes captured best. Here are a few ideas that stood out.

The JavaScript interpreter, interpreted. Martha Girdler gave a quick, jargon-free tour of how Javascript works, using Javascript code to illustrate basic ideas like contexts. This sort of talk can help relatively inexperienced developers understand the common "pain points" of the language, such as variable hoisting.

Fast and Dynamic. Maxime Chevalier-Boisvert went a level deeper, tracing some of the fundamental ideas used to implement run-time systems from their historical roots in Lisp, Smalltalk, and Self up to research prototypes such as Chevalier-Boisvert's own Higgs compiler.

Many of the ideas are familiar to anyone who has had an undergrad compiler course, such as type tagging and microcoded instructions. Others are simple extensions of such ideas, such as inline caching, which is a workhorse in any dynamic compiler. Still others have entered popular discussion only recently. Maps, which are effectively hidden classes, originated in Self and are now being applied and extended in a number of interesting ways.

Two ideas from this talk that I would like to learn more about are hybrid type inference, which Chevalier-Boisvert mentioned in the context of Chrome and Firefox, and basic block versioning, a technique being explored in the Higgs compiler.

In closing, the speaker speculated on the better compilers of the future. Some of the advances will come from smarter CPUs, which might execute potential future paths in parallel, and more principled language design. But many will come from academic research that discovers new techniques and improves exiting ones.

Some of the ideas of the future are probably already available and simply haven't caught on yet. Chevalier-Boisvert offered three candidates: direct manipulation of the AST, pattern matching, and the persistent image. I certainly hear a lot of people talking about the first of these, but I've yet to see a compelling implementation yet.

Ruby Doesn't Have to Be Slow. In this session, Alex Gaynor explained why dynamic languages don't have to be slow. Though Ruby was his working example, everything he said applies to Javascript, Python, Lua, and other dynamic languages. He then talked about how he is putting these ideas to work in Topaz, a fast Ruby interpreter written in RPython. Topaz uses a number of advanced techniques, including a tracing JIT, type-specialized field look-up, maps, quasi-immutable fields, and escape analysis. It supports a subset of Ruby, though much of what is missing now is simply standard library classes and methods.

Two of the more interesting points of this talk for me were about meta-issues. First, he opened with an elaboration of the claim, "Ruby is slow", which he rightfully rejected as too imprecise to be meaningful. What people probably mean is something like, "Code written in Ruby executes CPU-bound tasks slower than other languages." I would add that, for many of my CS colleagues, the implicit benchmark is compiled C.

Further, Ruby users tend to respond to this claim poorly. Rather than refute it, they accept its premise and dance around its edges. Saddest, he says, is when they say, "If it turns out to matter, we can rewrite the program in some serious language." The compiler nerd in him says, "We can do this." Topaz is, in part, an attempt to support that claim.

Second, in response to an audience question, he claimed that people responsible for Java got something right fifteen years: they convinced people to abandon their C extensions. If the Ruby world followed course, and moved away from external dependencies that restrict what the compiler and run-time system can know, then many performance improvements would follow.

Throughout this talk, I kept coming back to JRuby in my mind...

The Art of Finding Shortcuts. Vyacheslav " @mraleph" Egorov's talk was ostensibly about an optimizing compiler for Dart, but like most of the compiler talks this year, it presented ideas of value for handling any dynamic language. Indeed, this talk gave a clear introduction to what an optimizing compiler does, what in-line caching is, and different ways that the compiler might capitalize on them.

According to Egorov, writing an optimizing compiler for language like Dart is the art of finding -- and taking -- shortcuts. The three key issues to address are representation, resolution, and redundancy. You deal with representation when you design your run-time system. The other two fall to the optimizing compiler.

Resolution is fundamentally a two-part question. Given the expression obj.prop,

  • What is obj?
  • Where is prop?

In-line caches eliminate redundancy by memoizing where/what pairs. The goal is to use the same hidden class maps to resolve property access whenever possible. Dart's optimizer uses in-line caching to give type feedback for use in improving the performance of loads and stores.

Egorov was one of the most quotable speakers I heard at StrangeLoop this year. In addition to "the art of finding shortcuts", I noted several other pithy sayings that I'll probably steal at some point, including:

  • "If all you have is an in-line cache, then everything looks like an in-line cache stub."
  • "In-lining is a Catch-22." You can't know if you will benefit from inlining unless you try, but trying (and undoing) is expensive.

Two ideas I plan to read more about after hearing this talk are allocation sinking and load forwarding.

~~~~

I have a lot of research to do now.


Posted by Eugene Wallingford | Permalink | Categories: Computing

September 23, 2013 4:22 PM

StrangeLoop: This and That, Volume 1

[My notes on StrangeLoop 2013: Table of Contents]

the Peabody Opera House's Broadway series poster

I'm working on a post about the compiler talks I attended, but in the meantime here are a few stray thoughts, mostly from Day 1.

The Peabody Opera House really is a nice place to hold a conference of this size. If StrangeLoop were to get much larger, it might not fit.

I really don't like the word "architected".

The talks were scheduled pretty well. Only once in two days did I find myself really wanting to go to two talks at the same time. And only once did I hear myself thinking, "I don't want to hear any of these...".

My only real regret from Day 1 was missing Scott Vokes's talk on data compression. I enjoyed the talk I went to well enough, but I think I would have enjoyed this one more.

What a glorious time to be a programming language theory weenie. Industry practitioners are going to conferences and attending talks on dependent types, continuations, macros, immutable data structures, and functional reactive programming.

Moon Hooch? Interesting name, interesting sound.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

September 22, 2013 3:51 PM

StrangeLoop: Jenny Finkel on Machine Learning at Prismatic

[My notes on StrangeLoop 2013: Table of Contents]

The conference opened with a talk by Jenny Finkel on the role machine learning play at Prismatic, the customized newsfeed service. It was a good way to start the conference, as it introduced a few themes that would recur throughout, had a little technical detail but not too much, and reported a few lessons from the trenches.

Prismatic is trying to solve the discovery problem: finding content that users would like to read but otherwise would not see. This is more than simply a customized newsfeed from a singular journalistic source, because it draws from many sources, including other reader's links, and because it tries to surprise readers with articles that may not be explicitly indicated by their profiles.

The scale of the problem is large, but different from the scale of the raw data facing Twitter, Facebook, and the like. Finkel said that Prismatic is processing only about one million timely docs at a time, with the set of articles turning over roughly weekly. The company currently uses 5,000 categories to classify the articles, though that number will soon go up to the order of 250,000.

The complexity here comes from the cross product of readers, articles, and categories, along with all of the features used to try to tease out why readers like the things they do and don't like the others. On top of this are machine learning algorithms that are themselves exponentially expensive to run. And with articles turning over roughly weekly, they have to be amassing data, learning from it, and moving on constantly.

The main problem at the heart of a service like this is: What is relevant? Everywhere one turns in AI, one sees this question, or its more general cousin, Is this similar? In many ways, this is the problem at the heart of all intelligence, natural and artificial.

Prismatic's approach is straight from AI, too. They construct a feature vector for each user/article pair and then try to learn weights that, when applied to the values in a given vector, will rank desired articles high and undesired articles low. One of the key challenges when doing this kind of working is to choose the right features to use in the vector. Finkel mentioned a few used by Prismatic, including "Does the user follow this topic?", "How many times has the reader read an article from this publisher?", and "Does the article include a picture?"

With a complex algorithm, lots of data, and a need to constantly re-learn, Prismatic has to make adjustments and take shortcuts wherever possible in order to speed up the process. This is a common theme at a conference where many speakers are from industry. First, learn your theory and foundations; learn the pragmatics and heuristics need to turn basic techniques into the backbone of practical applications.

Finkel shared one pragmatic idea of this sort that Prismatic uses. They look for opportunities to fold user-specific feature weights into user-neutral features. This enables their program to compute many user-specific dot products using a static vector.

She closed the talk with five challenges that Prismatic has faced that other teams might be on the look out for:

Bugs in the data. In one case, one program was updating a data set before another program could take a snapshot of the original. With the old data replaced by the new, they thought their ranker was doing better than it actually was. As Finkel said, this is pretty typical for an error in machine learning. The program doesn't crash; it just gives the wrong answer. Worse, you don't even have reason to suspect something is wrong in the offending code.

Presentation bias. Readers tend to look at more of the articles at the top of a list of suggestions, even if they would have enjoyed something further down the list. This is a feature of the human brain, not of computer programs. Any time we write programs that interact with people, we have to be aware of human psychology and its effects.

Non-representative subsets. When you are creating a program that ranks things, its whole purpose is to skew a set of user/article data points toward the subset of articles that the reader most wants to read. But this subset probably doesn't have the same distribution as the full set, which hampers your ability to use statistical analysis to draw valid conclusions.

Statistical bleeding. Sometimes, one algorithm looks better than it is because it benefits from the performance of the other. Consider two ranking algorithms, one an "explorer" that seeks out new content and one an "exploiter" that recommend articles that have already been found to be popular. If we in comparing their performances, the exploiter will tend to look better than it is because it benefits from the successes of the explorer without being penalized for its failures. It is crucial to recognize that one feature you measure is not dependent on another. (Thanks to Christian Murphy for the prompt!)

Simpson's Paradox. The iPhone and the web have different clickthrough rates. They once found them in a situation where one recommendation algorithm performed worse than another on both platforms, yet better overall. This can really disorient teams who follow up experiments by assessing the results. The issue here is usually a hidden variable that is confounding the results.

(I remember discussing this classic statistical illusion with a student in my early years of teaching, when we encountered a similar illusion in his grade. I am pretty sure that I enjoyed our discussion of the paradox more than he did...)

This part of a talk is of great value to me. Hearing about another team's difficulties rarely helps me avoid the same problems in my own projects, but it often does help me recognize those problems when they occur and begin thinking about ways to work around them. This was a good way for me to start the conference.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

September 22, 2013 10:27 AM

Back from StrangeLoop 2013

the front of my StrangeLoop 2013 badge

I'm back home for StrangeLoop 2013. It was, again, an outstanding conference: a great location, excellent amenities, fun side events, and -- most importantly -- a solid set of talks: diverse topics, strong technical content, and a some very good speakers. Alex Miller and his team put on a good conference.

This year, I went to the talks old school: with a steno notebook and no technology but a camera. As a result, a couple of things are different about how I'm blogging the conference. First, I did not write or post any entries during the event itself. Second, my notes are a bit shorter than usual and will need to be typed up before they become blog entries. I'll write my thoughts up over the next week or so and post the entries as they emerge.

This entry will serve as a table of contents for my StrangeLoop posts, a home base for readers who might stumble onto one post and care to read more. For now, I'll list a few entries I expect to write, but I'll only know what belongs here after I have written them.

Primary entries:

Ancillary entries:

Is it too early to start looking forward to StrangeLoop 2014?


Posted by Eugene Wallingford | Permalink | Categories: Computing

September 10, 2013 3:40 PM

A Laugh at My Own Expense

This morning presented a short cautionary tale for me and my students, from a silly mistake I made in a procmail filter.

Back story: I found out recently that I am still subscribed to a Billy Joel fan discussion list from the 1990s. The list has been inactive for years, or I would have been filtering its messages to a separate mailbox. Someone has apparently hacked the list, as a few days ago it started spewing hundreds of spam messages a day.

I was on the road for a few days after the deluge began and was checking mail through a shell connection to the mail server. Because I was busy with my trip and checking mail infrequently, I just deleted the messages by hand. When I got back, Mail.app soon learned they were junk and filtered them away for me. But the spam was still hitting my inbox on the mail server, where I read my mail occasionally even on campus.

After a session on the server early this morning, I took a few minutes to procmail them away. Every message from the list has a common pattern in the Subject: line, so I copied it and pasted it into a new procmail recipe to send all list traffic to /dev/null :

    :0
    * ^Subject.*[billyjoel]
    /dev/null

Do you see the problem? Of course you do.

I didn't at the time. My blindness probably resulted from a combination of the early hour, a rush to get over to the gym, and the tunnel vision that comes from focusing on a single case. It all looked obvious.

This mistake offers programming lessons at several different levels.

The first is at the detailed level of the regular expression. Pay attention to the characters in your regex -- all of them. Those brackets really are in the Subject: line, but by themselves mean something else in the regex. I need to escape them:

    * ^Subject.*\[billyjoel\]

This relates to a more general piece of problem-solving advice. Step back from individual case you are solving and think about the code you are writing more generally. Focused on the annoying messages from the list, the brackets are just characters in a stream. Looked at from the perspective of the file of procmail recipes, they are control characters.

The second is at the level of programming practice. Don't /dev/null something until you know it's junk. Much better to send the offending messages to a junk mbox first:

    * ^Subject.*\[billyjoel\]
    in.tmp.junk

Once I see that all and only the messages from the list are being matched by the pattern, I can change that line send list traffic where it belongs. That's a specific example of the sort of defensive programming that we all should practice. Don't commit to solutions too soon.

This, too, relates to more general programming advice about software validation and verification. I should have exercised a few test cases to validate my recipe before turning it loose unsupervised on my live mail stream.

I teach my students this mindset and program that way myself, at least most of the time. Of course, the time you most need test cases will be the time you don't write them.

The day provided a bit of irony to make the story even better. The topic of today's session in my compilers course? Writing regular expressions to describe the tokens in a language. So, after my mail admin colleague and I had a good laugh at my expense, I got to tell the story to my students, and they did, too.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

August 31, 2013 11:32 AM

A Good Language Conserves Programmer Energy

Game programmer Jeff Wofford wrote a nice piece on some of the lessons he learned by programming a game in forty-eight hours. One of the recurring themes of his article is the value of a high-powered scripting language for moving fast. That's not too surprising, but I found his ruminations on this phenomenon to be interesting. In particular:

A programmer's chief resource is the energy of his or her mind. Everything that expends or depletes that energy makes him or her less effective, more tired, and less happy.

A powerful scripting language sitting atop the game engine is one of the best ways to conserve programmer energy. Sometimes, though, a game programmer must work hard to achieve the performance required by users. For this reason, Wofford goes out of his way not to diss C++, the tool of choice for many game programmers. But C++ is an energy drain on the programmer's mind, because the programmer has to be in a constant state of awareness of machine cycles and memory consumption. This is where the trade-off with a scripting language comes in:

When performance is of the essence, this state of alertness is an appropriate price to pay. But when you don't have to pay that price -- and in every game there are systems that have no serious likelihood of bottlenecking -- you will gain mental energy back by essentially ignoring performance. You cannot do this in C++: it requires an awareness of execution and memory costs at every step. This is another argument in favor of never building a game without a good scripting language for the highest-level code.

I think this is true of almost every large system. I sure wish that the massive database systems at the foundation of my university's operations had scripting languages sitting on top. I even want to script against the small databases that are the lingua franca of most businesses these days -- spreadsheets. The languages available inside the tools I use are too clunky or not powerful, so I turn to Ruby.

Unfortunately, most systems don't come with a good scripting language. Maybe the developers aren't allowed to provide one. Too many CS grads don't even think of "create a mini-language" as a possible solution to their own pain.

Fortunately for Wofford, he both has the skills and inclination. One of his to-dos after the forty-eight hour experience is all about language:

Building a SWF importer for my engine could work. Adding script support to my engine and greatly refining my tools would go some of the distance. Gotta do something.

Gotta do something.

I'm teaching our compiler course again this term. I hope that the dozen or so students in the course leave the university knowing that creating a language is often the right next action and having the skills to do it when they feel compelled to do something.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

August 29, 2013 4:31 PM

Asimov Sees 2014, Through Clear Eyes and Foggy

Isaac Asimov, circa 1991

A couple of years ago, I wrote Psychohistory, Economics, and AI, in which I mentioned Isaac Asimov and one way that he had influenced me. I never read Asimov or any other science fiction expecting to find accurate predictions of future. What drew me in was the romance of the stories, dreaming "what if?" for a particular set of conditions. Ultimately, I was more interested in the relationships among people under different technological conditions than I was in the technology itself. Asimov was especially good at creating conditions that generated compelling human questions.

Some of the scenarios I read in Asimov's SF turned out to be wildly wrong. The world today is already more different from the 1950s than the world of the Foundation, set thousands of years in the future. Others seem eerily on the mark. Fortunately, accuracy is not the standard by which most of us judge good science fiction.

But what of speculation about the near future? A colleague recently sent me a link to Visit to the World's Fair of 2014, an article Asimov wrote in 1964 speculating about the world fifty years hence. As I read it, I was struck by just how far off he was in some ways, and by how close he was in others. I'll let you read the story for yourself. Here are a few selected passages that jumped out at me.

General Electric at the 2014 World's Fair will be showing 3-D movies of its "Robot of the Future," neat and streamlined, its cleaning appliances built in and performing all tasks briskly. (There will be a three-hour wait in line to see the film, for some things never change.)

3-D movies are now common. Housecleaning robots are not. And while some crazed fans will stand in line for many hours to see the latest comic-book blockbuster, going to a theater to see a movie has become much less important part of the culture. People stream movies into their homes and into their hands. My daughter teases me for caring about the time any TV show or movie starts. "It's on Hulu, Dad." If it's not on Hulu or Netflix or the open web, does it even exist?

Any number of simultaneous conversations between earth and moon can be handled by modulated laser beams, which are easy to manipulate in space. On earth, however, laser beams will have to be led through plastic pipes, to avoid material and atmospheric interference. Engineers will still be playing with that problem in 2014.

There is no one on the moon with whom to converse. Sigh. The rest of this passage sounds like fiber optics. Our world is rapidly becoming wireless. If your device can't connect to the world wireless web, does it even exist?

In many ways, the details of technology are actually harder to predict correctly than the social, political, economic implications of technological change. Consider:

Not all the world's population will enjoy the gadgety world of the future to the full. A larger portion than today will be deprived and although they may be better off, materially, than today, they will be further behind when compared with the advanced portions of the world. They will have moved backward, relatively.

Spot on.

When my colleague sent me the link, he said, "The last couple of paragraphs are especially relevant." They mention computer programming and a couple of its effects on the world. In this regard, Asimov's predictions meet with only partial success.

The world of A.D. 2014 will have few routine jobs that cannot be done better by some machine than by any human being. Mankind will therefore have become largely a race of machine tenders. Schools will have to be oriented in this direction. ... All the high-school students will be taught the fundamentals of computer technology will become proficient in binary arithmetic and will be trained to perfection in the use of the computer languages that will have developed out of those like the contemporary "Fortran" (from "formula translation").

The first part of this paragraph is becoming truer every day. Many people husband computers and other machines as they do tasks we used to do ourselves. The second part is, um, not true. Relatively few people learn to program at all, let alone master a programming language. And how many people understand this t-shirt without first receiving an impromptu lecture on the street?

Again, though, Asimov is perhaps closer on what technological change means for people than on which particular technological changes occur. In the next paragraph he says:

Even so, mankind will suffer badly from the disease of boredom, a disease spreading more widely each year and growing in intensity. This will have serious mental, emotional and sociological consequences, and I dare say that psychiatry will be far and away the most important medical specialty in 2014. The lucky few who can be involved in creative work of any sort will be the true elite of mankind, for they alone will do more than serve a machine.

This is still speculation, but it is already more true than most of us would prefer. How much truer will it be in a few years?

My daughters will live most of their lives post-2014. That worries the old fogey in me a bit. But it excites me more. I suspect that the next generation will figure the future out better than mine, or the ones before mine, can predict it.

~~~~

PHOTO. Isaac Asimov, circa 1991. Britannica Online for Kids. Web. 2013 August 29. http://kids.britannica.com/comptons/art-136777.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

August 22, 2013 2:45 PM

A Book of Margin Notes on a Classic Program?

I recently stumbled across an old How We Will Read interview with Clive Thompson and was intrigued by his idea for a new kind of annotated book:

I've had this idea to write a provocative piece, or hire someone to write it, and print it on-demand it with huge margins, and then send it around to four people with four different pens -- red, blue, green and black. It comes back with four sets of comments all on top of the text. Then I rip it all apart and make it into an e-book.

This is an interesting mash-up of ideas from different eras. People have been writing in the margins of books for hundreds of years. These days, we comment on blog entries and other on-line writing in plain view of everyone. We even comment on other people's comments. Sites such as Findings.com, home of the Thompson interview, aim to bring this cultural practice to everything digital.

Even so, it would be pretty cool to see the margin notes of three or four insightful, educated people, written independently of one another, overlaid in a single document. Presentation as an e-book offers another dimension of possibilities.

Ever the computer scientist, I immediately began to think of programs. A book such as Beautiful Code gives us essays from master programmers talking about their programs. Reading it, I came to appreciate design decisions that are usually hidden from readers of finished code. I also came to appreciate the code itself as a product of careful thought and many iterations.

My thought is: Why not bring Thompson's mash-up of ideas to code, too? Choose a cool program, perhaps one that changed how we work or think, or one that unified several ideas into a standard solution. Print it out with huge margins, and send it to three of four insightful, thoughtful programmers who read it, again or for the first time, and mark it up with their own thoughts and ideas. It comes back with four sets of comments all on top of the text. Rip it apart and create an e-book that overlays them all in a single document.

Maybe we can skip the paper step. Programming tools and Web 2.0 make it so easy to annotate documents, including code, in ways that replace handwritten comments. That's how most people operate these days. I'm probably showing my age in harboring a fondness for the written page.

In any case, the idea stands apart from the implementation. Wouldn't it be cool to read a book that interleaves and overlays the annotations made by programmers such as Ward Cunningham and Grady Booch as they read John McCarthy's first Lisp interpreter, the first Fortran compiler from John Backus's team, QuickDraw, or Qmail? I'd stand in line for a copy.

Writing this blog entry only makes the idea sound more worth doing. If you agree, I'd love to hear from you -- especially if you'd like to help. (And especially if you are Ward Cunningham and Grady Booch!)


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

July 18, 2013 2:22 PM

AP Computer Science in Iowa High Schools

Mark Guzdial posted a blog entry this morning pointing to a Boston Globe piece, Interest in computer science lags in Massachusetts schools. Among the data supporting this assertion was participation in Advanced Placement:

Of the 85,753 AP exams taken by Massachusetts students last year, only 913 were in computing.

Those numbers are a bit out of context, but they got me to wondering about the data for Iowa. So I tracked down this page on AP Program Participation and Performance Data 2012 and clicked through to the state summary report for Iowa. The numbers are even more dismal than Massachusetts's.

Of the 16,413 AP exams taken by Iowa students in 2012, only sixty-nine were in computer science. The counts for groups generally underrepresented in computing were unsurprisingly small, given that Iowa is less diverse than many US states. Of the sixty-nine, fifty-four self-reported as "white", ten as "Asian", and one as "Mexican-American", with four not indicating a category.

The most depressing number of all: only nine female students took the AP Computer Science exam last year in Iowa.

Now, Massachusetts has roughly 2.2 times as many people as Iowa, but even so Iowa compares unfavorably. Iowans took about one-fifth as many AP exams as many Massachusetts students, and for CS the percentage drops to 7.5%. If AP exams indicate much about the general readiness of a state's students for advanced study in college, then Iowa is at a disadvantage.

I've never been a huge proponent of the AP culture that seems to dominate many high schools these days (see, for instance, this piece), but the low number of AP CS exams taken in Iowa is consistent with what I hear when I talk to HS students from around the state and their parents: Iowa schools are not teaching much computer science at all. The university is the first place most students have an opportunity to take a CS course, and by then the battle for most students' attention has already been lost. For a state with a declared goal of growing its promising IT sector, this is a monumental handicap.

Those of us interested in broadening participation in CS face an even tougher challenge. Iowa's demographics create some natural challenges for attracting minority students to computing. And if the AP data are any indication, we are doing a horrible job of reaching women in our high schools.

There is much work to do.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

July 17, 2013 1:41 PM

And This Gray Spirit Yearning

On my first day as a faculty member at the university, twenty years ago, the department secretary sent me to Public Safety to pick up my office and building keys. "Hi, I'm Eugene Wallingford," I told the person behind the window, "I'm here to pick up my keys." She smiled, welcomed me, and handed them to me -- no questions asked.

Back at the department, I commented to one of my new colleagues that this seemed odd. No one asked to see an ID or any form of authorization. They just handed me keys giving me access to a lot of cool stuff. My colleague shrugged. There has never been a problem here with unauthorized people masquerading as new faculty members and picking up keys. Until there is a problem, isn't it nice living in a place where trust works?

Things have changed. These days, we don't order keys for faculty; we "request building access". This phrase is more accurate than a reference to keys, because it includes activating the faculty ID to open electronically-controlled doors. And we don't simply plug a new computer into an ethernet jack and let faculty start working; to get on the wireless network, we have to wait for the Active Directory server to sync with the HR system, which updates only after electronic approval of a Personnel Authorization Form that set up of the employee's payroll record. I leave that as a run-on phrase, because that's what living it feels like.

The paperwork needed to get a new faculty member up and running these days reminds me just how simple life was when in 1992. Of course, it's not really "paperwork" any more.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

July 15, 2013 2:41 PM

Version Control for Writers and Publishers

Mandy Brown again, this time on on writing tools without memory:

I've written of the web's short-term memory before; what Manguel trips on here is that such forgetting is by design. We designed tools to forget, sometimes intentionally so, but often simply out of carelessness. And we are just as capable of designing systems that remember: the word processor of today may admit no archive, but what of the one we build next?

This is one of those places where the software world has a tool waiting to reach a wider audience: the version control system. Programmers using version control can retrieve previous states of their code all the way back to its creation. The granularity of the versions is limited only by the frequency with which they "commit" the code to the repository.

The widespread adoption of version control and the existence of public histories at place such as GitHub have even given rise to a whole new kind of empirical software engineering, in which we mine a large number of repositories in order to understand better the behavior of developers in actual practice. Before, we had to contrive experiments, with no assurance that devs behaved the same way under artificial conditions.

Word processors these days usually have an auto-backup feature to save work as the writer types text. Version control could be built into such a feature, giving the writer access to many previous versions without the need to commit changes explicitly. But the better solution would be to help writers learn the value of version control and develop the habits of committing changes at meaningful intervals.

Digital version control offers several advantages over the writer's (and programmer's) old-style history of print-outs of previous versions, marked-up copy, and notebooks. An obvious one is space. A more important one is the ability to search and compare old versions more easily. We programmers benefit greatly from a tool as simple as diff, which can tell us the textual differences between two files. I use diff on non-code text all the time and imagine that professional writers could use it to better effect than I.

The use of version control by programmers leads to profound changes in the practice of programming. I suspect that the same would be true for writers and publishers, too.

Most version control systems these days work much better with plain text than with the binary data stored by most word processing programs. As discussed in my previous post, there are already good reasons for writers to move to plain text and explicit mark-up schemes. Version control and text analysis tools such as diff add another layer of benefit. Simple mark-up systems like Markdown don't even impose much burden on the writer, resembling as they do how so many of us used to prepare text in the days of the typewriter.

Some non-programmers are already using version control for their digital research. Check out William Turkel's How To for doing research with digital sources. Others, such The Programming Historian and A Companion to Digital Humanities, don't seem to mention it. But these documents refer mostly to programs for working with text. The next step is to encourage adoption of version control for writers doing their own thing: writing.

Then again, it has taken a long time for version control to gain such widespread acceptance even among programmers, and it's not yet universal. So maybe adoption among writers will take a long time, too.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 12, 2013 3:08 PM

"Either Take Control, or Cede it to the Software"

Mandy Brown tells editors and "content creators" to take control of their work:

It's time content people of all stripes recognized the WYSIWYG editor for what it really is: not a convenient shortcut, but a dangerous obstacle placed between you and the actual content. Because content on the web is going to be marked up one way or another: you either take control of it or you cede it to the software, but you can't avoid it. WYSIWYG editors are fine for amateurs, but if you are an editor, or copywriter, or journalist, or any number of the kinds of people who work with content on the web, you cannot afford to be an amateur.

Pros can sling a little code, too.

Brown's essay reminded me of a blog entry I was discussing with a colleague recently, Andrew Hayes's Why Learn Syntax? Hayes tells statisticians that they, too, should take control of their data, by learning the scripting language of the statistical packages they use. Code is a record of an analysis, which allows it to be re-run and shared with others. Learning to write code also hones one's analytical skills and opens the door to features not available through the GUI.

These articles speak to two very different audiences, but the message is the same. Don't just be a user of someone else's tools and be limited to their vision. Learn to write a little code and take back the power to create.


Posted by Eugene Wallingford | Permalink | Categories: Computing

July 11, 2013 2:57 PM

Talking to the New University President about Computer Science

Our university recently hired a new president. Yesterday, he and the provost came to a meeting of the department heads in humanities, arts, and sciences, so that he could learn a little about the college. The dean asked each head to introduce his or her department in one minute or less.

I came in under a minute, as instructed. Rather than read a litany of numbers that he can read in university reports, I focused on two high-level points:

  • Major enrollment has recovered nicely since the deep trough after the dot.com bust and is now steady. We have near-100% placement, but local and state industry could hire far more graduates.
  • For the last few years we have also been working to reach more non-majors, which is a group we under-serve relative to most other schools. This should be an important part of the university's focus on STEM and STEM teacher education.

I closed with a connection to current events:

We think that all university graduates should understand what 'metadata' is and what computer programs can do with it -- enough so that they can understand the current stories about the NSA and be able to make informed decisions as a citizen.

I hoped that this would be provocative and memorable. The statement elicited laughs and head nods all around. The president commented on the Snowden case, asked me where I thought he would land, and made an analogy to The Man Without a Country. I pointed out that everyone wants to talk about Snowden, including the media, but that's not even the most important part of the story. Stories about people are usually of more interest than stories about computer programs and fundamental questions about constitutional rights.

I am not sure how many people believe that computer science is a necessary part of a university education these days, or at least the foundations of computing in the modern world. Some schools have computing or technology requirements, and there is plenty of press for the "learn to code" meme, even beyond the CS world. But I wonder how many US university graduates in 2013 understand enough computing (or math) to understand this clever article and apply that understand to the world they live in right now.

Our new president seemed to understand. That could bode well for our department and university in the coming years.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 08, 2013 1:05 PM

A Random Thought about the Metadata and Government Surveillance

In a recent mischievous mood, I decided it might be fun to see the following.

The next whistleblower with access to all the metadata that the US government is storing on its citizens assembles a broad list of names: Republican and Democrat; legislative, executive, and judicial branches; public official and private citizens. The only qualification for getting on the list is that the person has uttered any variation of the remarkably clueless statement, "If you aren't doing anything wrong, then you have nothing to hide."

The whistleblower thens mine the metadata and, for each person on this list, publishes a brief that demonstrates just how much someone with that data can conclude -- or insinuate -- about a person.

If they haven't done anything wrong, then they don't have anything to worry about. Right?


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 07, 2013 9:32 AM

Interesting Sentences, Personal Weakness Edition

The quest for comeuppance is a misallocation of personal resources. -- Tyler Cowen

Far too often, my reaction to events in the world around me is to focus on other people not following rules, and the unfairness that results. It's usually not my business, and even when it is, it's a foolish waste of mental energy. Cowen expresses this truth nicely in neutral, non-judgmental language. That may help me develop a more productive mental habit.

What we have today is a wonderful bike with training wheels on. Nobody knows they are on, so nobody is trying to take them off. -- Alan Kay, paraphrased from The MIT/Brown Vannevar Bush Symposium

Kay is riffing off Douglas Engelbart's tricycle analogy, mentioned last time. As a computer scientist, and particularly one fortunate enough to have been exposed to the work of Ivan Sutherland, Englebart, Kay and the Xerox PARC team, and so many others, I should be more keenly conscious that we are coasting along with training wheels on. I settle for limited languages and limited tools.

Even sadder, when computer scientists and software developers settle for training wheels, we tend to limit everyone else's experience, too. So my apathy has consequences.

I'll try to allocate my personal resources more wisely.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

July 06, 2013 9:57 AM

Douglas Engelbart Wasn't Just Another Computer Guy

Bret Victor nails it:

Albert Einstein, Discoverer of Photoelectric Effect, Dies at 76

In the last few days, I've been telling family and friends about Engelbart's vision and effect on the world the computing, and thus on their world. He didn't just "invent the mouse".

It's hard to imagine these days just how big Engelbart's vision was for the time. Watching The Mother of All Demos now, it's easy to think "What's the big deal? We have all that stuff now." or even "Man, that video looks prehistoric." First of all, we don't have all that stuff today. Watch again. Second, in a sense, that demo was prehistory. Not only did we not have such technology at the time, almost no one was thinking about it. It's not that people thought such things were impossible; they couldn't think about them at all, because no one had conceived them yet. Engelbart did.

Engelbart didn't just invent a mouse that allows us to point at files and web links. His ideas helped point an entire industry toward the future.

Like so many of our computing pioneers, though, he dreamed of more than what we have now, and expected -- or at least hoped -- that we would build on the industry's advanced to make his vision real. Engelbart understood that skills which make people productive are probably difficult to learn. But they are so valuable that the effort is worth it. I'm reminded of Alan Kay's frequent use of a violin as an example, compared to a a simpler music-making device, or even to a radio. Sure, a violin is difficult to play well. But when you can play -- wow.

Engelbart was apparently fond of another example: the tricycle

Riding a bicycle -- unlike a tricycle -- is a skill that requires a modest degree of practice (and a few spills), but the rider of a bicycle quickly outpaces the rider of a tricycle.

Most of the computing systems we use these days are tricycles. Doug Engelbart saw a better world for us.


Posted by Eugene Wallingford | Permalink | Categories: Computing

July 03, 2013 10:22 AM

Programming for Everyone, Venture Capital Edition

Christina Cacioppo left Union Square Ventures to learn how to program:

Why did I want to do something different? In part, because I wanted something that felt more tangible. But mostly because the story of the internet continues to be the story of our time. I'm pretty sure that if you truly want to follow -- or, better still, bend -- that story's arc, you should know how to write code.

So, rather than settle for her lot as a non-programmer, beyond the accepted school age for learning these things -- technology is a young person's game, you know -- Cacioppo decided to learn how to build web apps. And build one.

When did we decide our time's most important form of creation is off-limits? How many people haven't learned to write software because they didn't attend schools that offered those classes, or the classes were too intimidating, and then they were "too late"? How much better would the world be if those people had been able to build their ideas?

Yes, indeed.

These days, she is enjoying the experience of making stuff: trying ideas out in code, discarding the ones that don't work, and learning new things every day. Sounds like a programmer to me.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

July 01, 2013 11:10 AM

Happy in my App

In a typically salty post, Jamie Zawinski expresses succinctly one of the tenets of my personal code:

I have no interest in reading my feeds through a web site (no more than I would tolerate reading my email that way, like an animal).

Living by this code means that, while many of my friends and readers are gnashing their teeth on this first of July, my life goes on uninterrupted. I remain a happy long-time user of NetNewsWire (currently, v3.1.6).

Keeping my feeds in sync with NetNewsWire has always been a minor issue, as I run the app on at least two different computers. Long ago, I wrote a couple of extremely simple scripts -- long scp commands, really -- that do a pretty good job. They don't give me thought-free syncing, but that's okay.

A lot of people tell me that apps are dead, that the HTML5-powered web is the future. I do know that we're very quickly running out of stuff we can't do in the browser and applaud the people who are making that happen. If I were a habitual smartphone-and-tablet user, I suspect that I would be miffed if web sites made me download an app just to read their web content. All that said, though, I still like what a clean, simple app gives me.


Posted by Eugene Wallingford | Permalink | Categories: Computing

June 26, 2013 2:30 PM

An Opportunity to Learn, Born of Deprivation

Earlier this summer, my daughter was talking about something one of her friends had done with Instagram. As a smug computer weenie, I casually mentioned that she could do that, too.

She replied, "Don't taunt me, Dad."

You see, no one in our family has a cell phone, smart or otherwise, so none of us use Instagram. That's not a big deal for dear old dad, even though (or perhaps because) he's a computer scientist. But she is a teenager growing up in an entirely different world, filled with technology and social interaction, and not having a smart phone must surely seem like a form of child abuse. Occasionally, she reminds us so.

This gave me a chance to explain that Instagram filters are, at their core, relatively simple little programs, and that she could learn to write them. And if she did, she could run them on almost any computer, and make them do things that even Instagram doesn't do.

I had her attention.

So, this summer I am going to help her learn a little Python, using some of the ideas from media computation. At the end of our first pass, I hope that she will be able to manipulate images in a few basic ways: changing colors, replacing colors, copying pixels, and so on. Along the way, we can convert color images to grayscale or sepia tones, posterize images, embed images, and make simple collages.

That will make her happy. Even if she never feels the urge to write code again, she will know that it's possible. And that can be empowering.

I have let my daughter know that we probably will not write code that does as good a job as what she can see in Instagram or Photoshop. Those programs are written by pros, and they have evolved over time. I hope, though, that she will appreciate how simple the core ideas are. As James Hague said in a recent post, then key idea in most apps require relatively few lines of code, with lots and lots of lines wrapped around them to handle edge cases and plumbing. We probably won't write much code for plumbing... unless she wants to.

Desire and boredom often lead to creation. They also lead to the best kind of learning.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

June 14, 2013 2:48 PM

The History of Achievement in AI

... often looks something like this:

  1. X := some task that people do well, but not computers.
  2. "It would really be impressive if a computer could X."
  3. Computer does X.
  4. "That's not intelligent. The computer is only doing search (or number crunching, or ...)."
  5. X := something else.
  6. Go to 2.

A common variation of this pattern is to replace Step 3 with a different dodge:

"That's no big deal. X doesn't really require intelligence."

In either case, the target moves.

Occasionally, the critic must admit, if grudgingly, that the task requires intelligence, whatever that means, and that the computer performs it well. But there is still one last move available to deflect the achievement from the computer:

"This is a human achievement. People had to program the computer."

I suspect that until a computer learns everything it knows from scratch -- whatever that means -- this pattern will repeat. We humans have an image to protect.

~~~~

Postscript. I wrote this after reading a short interview interview with playwright Matt Charman, who has dramatized Deep Blue's epic 1997 match win over world chess champion Garry Kasparov. Note that Charman does not employ the dodges I list. He simply chose to focus on the human personalities involved in the drama. And those personalities are worthy of exploration, especially the fascinating Kasparov!


Posted by Eugene Wallingford | Permalink | Categories: Computing

June 13, 2013 3:01 PM

It's Okay to Talk About Currying!

James Hague offers some sound advice for writing functional programming tutorials. I agree with most of it, having learned the hard way by trying to teach functional style to university students for many years. But I disagree with one of his suggestions: I think it's okay to talk about currying.

Hague's concern with currying is practical:

Don't get so swept up in that theory that you forget the obvious: in any programming language ever invented, there's already a way to easily define functions of multiple arguments. That you can build this up from more primitive features is not useful or impressive to non-theoreticians.

Of course, my context is a little different. We teach functional programming in a course on programming languages, so a little theory is important. We want students not only to be able to write code in a functional style but also to understand some of the ideas at the foundation of the languages they use. We also want them to understand a bit about how different programming styles relate to one another.

But even in the context of teaching people to think functionally and to write code in that style, I think it's okay to talk about currying. Indeed, it is essential. Currying is not simply a theoretical topic. It is a valuable programming technique.

Here is an example. When we write a language interpreter, we often write a procedure names eval-exp. It takes two arguments: an expression to evaluate, and a list of variable/value bindings.

   (define eval-exp
     (lambda (exp env)
       ...))

The binding list, sometimes called an environment, is a map of names declared in the local block to their values, along with the bindings from the blocks that contain the local block. Each time the interpreter enters a new block, it pushes a new set of name/value pairs onto the binding list and recurses.

To evaluate a function call for which arguments are passed by value, the interpreter must first evaluate all of the function's arguments. As the arguments are all in the same block, they are evaluated using the same binding list. We could write a new procedure to evaluate the arguments recursively, but this seems like a great time to map a procedure over a list: (map eval-exp args), get a list of the results, and pass them to the code that applies the function to them.

We can't do that, though, because eval-exp is a two-argument procedure, and map works only with a one-argument procedure. But the same binding list is used to evaluate each of the expressions, so that argument to eval-exp is effectively a constant for the purposes of the mapping operation.

So we curry eval-exp:

   (define eval-exp-with
     (lambda (bindings)
       (lambda (exp)
         (eval-exp exp bindings))))

... to create the one-argument evaluator that we need, and we use it to evaluate the arguments with map:

   ; in eval-exp
   (map (eval-exp-with env) arguments)

In most functional languages, we can use a nameless lambda to curry eval-exp "in place" and avoid writing an explicit helper function:

   ; an alternative approach in eval-exp
   (map (lambda (exp)
          (eval-exp exp bindings))
        arguments)

This doesn't look much like currying because we never created the procedure that takes the bindings argument. But we can reach this same piece of code by writing eval-exp-with, calling it in eval-exp, and then using program derivation to substitute the value of the call for the call itself. This is actually a nice connection to be able to make in a course about programming languages!

When I deliver short tutorials on functional style, currying often does not make the cut, because there are so many cool and useful ideas to cover. But it doesn't take long writing functional code before currying becomes useful. As this example shows, currying is a practical tool for the functional programmer to have in his or her pocket. In FP, currying isn't just theory. It's part of the style.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

June 05, 2013 1:52 PM

I Fooled Around and Fell in Love

Cue the Elvin Bishop [ video ]...

I smile whenever I see this kind of statement on a website's About page:

Erika Carlson was studying clinical psychology in 2011, when she wrote her first line of Python code. She fell in love with programming, decided to change paths, and is now a software developer at Pillar Technology.

I fell in love upon writing my first line of code, too.

Not everyone will have the same reaction Erika and I had, but it's good that we give people at least an opportunity to learn how to program. Knowing that someone might react this way focuses my mind on giving novice programmers a good enough experience that they can, if they are so inclined.

My teaching should never get in the way of true love.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

May 31, 2013 1:44 PM

Quotes of the Week, in Four Dimensions

Engineering.

Michael Bernstein, in A Generation Ago, A Thoroughly Modern Sampling:

The AI Memos are an extremely fertile ground for modern research. While it's true that what this group of pioneers thought was impossible then may be possible now, it's even clearer that some things we think are impossible now have been possible all along.

When I was in grad school, we read a lot of new and recent research papers. But the most amazing, most educational, and most inspiring stuff I read was old. That's often true today as well.

Science.

Financial Agile tweets:

"If it disagrees with experiment, it's wrong". Classic.

... with a link to The Scientific Method with Feynman, which has a wonderful ten-minute video of the physicist explaining how science works. Among its important points is that guessing is huge part of science. It's just that scientists have a way of telling which guesses are right and which are wrong.

Teaching.

James Boyk, in Six Words:

Like others of superlative gifts, he seemed to think the less gifted could do as well as he, if only they knew a few powerful specifics that could readily be conveyed. Sometimes he was right!

"He" is Leonid Hambro, who played with Victor Borge and P. D. Q. Bach but was also well-known as a teacher and composer. Among my best teachers have been some extraordinarily gifted people. I'm thankful for the time they tried to convey their insights to the likes of me.

Art.

Amanda Palmer, in a conference talk:

We can only connect the dots that we collect.

Palmer uses this sentence to explain in part why all art is about the artist, but it means something more general, too. You can build, guess, and teach only with the raw materials that you assemble in your mind and your world. So collect lots of dots. In this more prosaic sense, Palmer's sentence applies to not only to art but also to engineering, science, and teaching.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

May 10, 2013 4:03 PM

Using Language to Understand a Data Set

Today was our twice-annual undergraduate research presentation day. Every B.S. student must do an undergraduate research project and present the results publicly. For the last few years, we have pooled the presentations on the morning of the Friday in finals week, after all the exams are given and everyone has a chunk of time free to present. It also means that more students and professors can attend, which makes for more a more engaging audience and a nice end to everyone's semester.

I worked with one undergraduate research student this spring. As I mentioned while considering the role of parsing in a compilers course, this student was looking for patterns in several years of professional basketball play-by-play data. His ultimate goal was to explore ways of measuring the impact of individual defensive performance in the NBA -- fairly typical MoneyBall stuff applied to an skill that is not well measured or understood.

This project fell into my hands serendipitously. The student had approached a couple of other professors, who upon hearing the word "basketball" immediately pointed him to me. Of course, the project is really a data analytics project that just happens to involve a dataset from basketball, but... Fortunately, I am interested in both the approach and the domain!

As research sometimes does, this problem led the student to a new problem first. In order to analyze data in the way he wanted, he needed data of a particular sort. There is plenty of play-by-play data available publicly on the web, but it's mostly prepared for presentation in HTML. So he first had to collect the data by scraping the web, and then organize it into a data format amenable to analysis.

This student had taken my compiler course the last time around, and his ability to parse several files of similar but just-different-enough data proved to be invaluable. As presented on sites like nba.com, the data is no where near ready to be studied.

As the semester wore on, he and I came to realize that his project this semester wouldn't be the data analysis he originally intended to do. It was a substantial project simply to make sense of the data he had found.

As he presented his work today, I realized something further. He was using language to understand a data set.

He started by defining a grammar to model the data he found, so that he could parse it into a database. This involved recognizing categories of expression that were on the surface of the data, such as made and missed field goals, timeouts, and turnovers. When he ran this first version of his parser, he found unhandled entries and extended his grammar.

Then he looked at the semantics of the data and noticed discrepancies deeper in the data. The number of possessions his program observed in a game differed from the expected values, sometimes wildly and with no apparent pattern.

As we looked deeper, we realized that the surface syntax of the data often obscured some events that would extend or terminate a possession. A simple example is a missed FT, which sometimes ends a possession and sometimes not. It depends in part on the next event in the timeline.

To handle these case, the student created new syntactic categories that enabled his parser to resolve such issues by recognized composite events in the data. As he did this, his grammar grew, and his parser became better at building a more accurate semantic model of the game.

This turned out to be a semester-long project in its own right. He's still not done and intends to continue with this research after graduation. We were both a bit surprised at how much effort it took to corral the data, but in retrospect we should not have been too surprised. Data are collected and presented with many different purposes in mind. Having an accurate deep model of the underlying the phenomenon in question isn't always one of them.

I hope the student was pleased with his work and progress this semester. I was. In addition to its practical value toward solving a problem of mutual interest, it reminded me yet again of the value of language in understanding the world around us, and the remarkable value that the computational ideas we study in computer science have to offer. For some reason, it also reminded me, pleasantly, of the Racket Way. As I noted in that blog entry, this is really the essence of computer science.

Of course, if some NBA team were to give my student the data he needs in suitable form, he could dive into the open question of how better to measure individual defensive performance in basketball. He has some good ideas, and the CS and math skills needed to try them out.

Some NBA team should snatch this guy up.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

May 08, 2013 12:11 PM

Not So Random Sentences

I start with a seemingly random set of sentences to blog about and, in the process of writing about them, find that perhaps they aren't so random after all.

An Era of Sharing Our Stuff

Property isn't theft; property is an inefficient distribution of resources.

This assertion comes from an interesting article on "economies of scale as a service", in reaction to a Paul Graham tweet:

Will ownership turn out to be largely a hack people resorted to before they had the infrastructure to manage sharing properly?

Open-source software, the Creative Commons, crowdsourcing. The times they are a-changin'.

An Era of Observing Ourselves

If the last century was marked by the ability to observe the interactions of physical matter -- think of technologies like x-ray and radar -- this century is going to be defined by the ability to observe people through the data they share.

... from The Data Made Me Do It.

I'm not too keen on being "observed" via data by every company in the world, even as understand the value it can brings the company and even me. But I like very much the idea that I can observe myself more easily and more productively. For years, I collected and studied data about my running and used what I learned to train and race better. Programmers are able to do this better now than ever before. You can learn a lot just by watching.

An Era of Thinking Like Scientist

... which leads to this line attributed to John C. Reynolds, an influential computer scientist who passed away recently:

Well, we know less than we did before, but more of what we know is actually true.

It's surprising how easy it is to know stuff when we don't have any evidence at all. Observing the world methodically, building models, and comparing them to what we observe in the future helps to know less of the wrong stuff and more of the right stuff.

Not everyone need be a scientist, but we'd all be better off if more of us thought like a scientist more often.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Personal

April 21, 2013 10:25 AM

Catnip for Programmers

This morning, Maciej Ceglowski of Pinboard introduced me to the Matasano crypto challenges, a set of exercises created by Thomas Ptacek and his team as a tool for teaching programmers a little about cryptography, some of its challenges, and the need for more awareness of how easy it is to do it wrong. With the coming break from the grind of the academic year, I plan on giving them a try.

After having completed the exercises himself, Ceglowski observes:

Crypto is like catnip for programmers. It is hard to keep us away from it, because it's challenging and fun to play with. And programmers respond very badly to the insinuation that they're not clever enough to do something. We see the F-16 just sitting there, keys in the ignition, no one watching, lights blinking, ladder extended. And some infosec nerd is telling us we can't climb in there, even though we just want to taxi around a little and we've totally read the manual.

I've noticed this with a number of topics in computing. In addition to cryptography, data compression and sorting/searching are sirens to the best programmers among our students. "What do you mean we can't do better?"

For many undergrads, the idea of writing a compiler seems a mystery. Heck, I admit to my students that even after years of teaching the course I remain in awe of my language tools and the people who build them. This challenge keeps a steady if relatively small stream of programmers flowing into our "Translation of Programming Languages" project course.

One of the great things about all these challenges is that after we tackle them, we have not only the finished product in hand but also what we learn about the topic -- and ourselves -- along the way. Then we are ready for a bigger challenge, and another program to write.

For CS faculty, catnip topics are invaluable ways to draw more students into the spell of computing, and more deeply. We are always on the lookout.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

April 20, 2013 10:25 AM

Reminiscing about Making My First Computer

Steve Wozniak

A friend put a copy of GameInformer magazine in my box yesterday with a pointer to an interview with the Great and Powerful Woz, Steve Wozniak. It's a short interview, only two pages, but it reminded me just how many cool things Wozniak (and so many others) did in the mid-1970s. It also reminded me of my younger days, coming into contact with the idea of games and machine learning for the first time.

Woz described how, after seeing Pong in a video arcade, he went home and built his own Pong game out of twenty-eight $1 chips. Steve Jobs took the game to Atari, where he encountered Nolan Bushnell, who had an idea for a single-player version of Pong. Thus did Woz design Breakout, a game with an especially apt name. It helped define Apple Computer.

The thought of building a computer game out of chips still amazes me. I was never a hardware guy growing up. I never had access to computer chips or that culture, and I had little inclination to fiddle with electronics, save for a few attempts to take apart radios and put them back together. When I designed things as a kid, they were houses or office buildings. I was going to be an architect. But Woz's story reminded me of one experience that foreshadowed my career as a computer scientist.

One year in school, I won a math contest. First prize was a copy of The Unexpected Hanging and Other Mathematical Diversions, a collection of Martin Gardner's columns from Scientific American. Chapter 8 was called "A Matchbox Game-Learning Machine". It described Hexapawn, a game played on a 3x3 board with chess pawns. The game was no more complex than Tic Tac Toe, but it was new. And I loved board games.

Gardner's article had more in store for me, though, than simply another game to study. He described how to create a "computer" -- a system of matchboxes -- that learns how to play the game! Here's how:

You make one box for each possible board position. In the box, you put different colored marbles corresponding to the moves that can be played in the position. Then you play a bunch of games against the matchbox computer. When it is the computer's turn to move, you pick up the box for that board position, shake it, and see which marble lands in the lower-right corner of the box. That's the computer's move.

When the game is over, the computer gets feedback. If it won the game, then put all the marbles back in their boxes. If it lost, punish it by keeping the marble responsible for its last move; put all the rest back in their boxes. Gardner claimed that by following this strategy, the matchbox computer would learn to play a perfect game in something under fifty moves.

This can't possibly work, can it? So I built it. And it did learn. I was happy, and amazed.

I remember experimenting a bit. Maybe a move wasn't always a loser? So I seeded the computer with more than one marble for each candidate move, so that the computer could overcome bad luck. Hexapawn is so simple that this wasn't necessary -- losing moves are losing moves -- but the computer still learned to play a perfect game, just a bit slower than before.

This is one of the earliest experiences I remember that started me down the road of studying artificial intelligence. Reading copious amounts of science fiction pushed me in that direction, too, but this was different. I had made something, and it learned. I was hooked.

So, I wasn't a hardware kid, but I had a hardware experience. It just wasn't digital hardware. But my inclination was always more toward ideas than gadgets. My interests quickly turned to writing programs, which made it so much easier to tinker with variations and to try brand-new ideas.

(Not so quickly, though, that I turned away from my dream of being an architect. The time I spent in college studying architecture turned out to be valuable in many ways.)

Wozniak was a hardware guy, but he quickly saw the potential of software. "Games were not yet software, and [the rise of the microprocessor] triggered in my mind: microprocessors can actually program games." He called the BASIC interpreter he wrote "Game BASIC". Ever the engineer, he designed the Apple II with integrated hardware and software so that programmers could write cool games.

I don't have a lot in common with Steve Wozniak, but one thing we share is the fun we have playing games. And, in very different ways, we once made computers that changed our lives.

~~~~

The GameInformer interview is on-line for subscribers only, but there is a cool video of Wozniak playing Tetris -- and talking about George H.W. Bush and Mikhail Gorbachev!


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

April 05, 2013 3:21 PM

The Role of Parsing in a Compilers Course

I teach compilers again this fall. I'm looking forward to summer, when I'll have a chance to write some code and play with some ideas for the course.

This morning I thought a bit about a topic that pops up every time I prep the course. The thoughts were prompted by a tweet from James Coglan, which said "Really wish this Compilers course weren't half about parsing. ... get on with semantics." The ellipsis is mine; James's tweet said something about using lex/yacc to get something up and running fast. Then, presumably, we could get on to the fun of semantics.

This is a challenge for my compilers course, too. I know I don't want to rush through scanning and parsing, yet I also wish I had more time for static analysis, optimization, and code generation. Even though I know the value of parsing, I wish I had equal time for a lot of other cool topics.

Geoff Wozniak's response expressed one of the reasons parsing still has such a large role in my compilers course, and so many others:

Parsing is like assembly language: it seems superfluous at the time, but provides deep understanding later. It's worth it.

That's part of what keeps me from de-emphasizing it in my course. Former students often report back to me that they have used their skill at writing parsers frequently in their careers, whether for parsing DSLs they whip up or for making sense of a mess of data they want to process.

A current student is doing an undergrad research project that involves finding patterns in several years of professional basketball play-by-play data, and his ability to parse several files of similar but just-different-enough data proved invaluable. Of course, he was a bit surprised that corralling the data took as much effort as it did. Kind of like how scanning and parsing are such a big part of a compiler project.

I see now that James has tweeted a retraction:

... ppl are RTing something I said about wishing the Compilers course would get done with parsing ASAP. Don't believe this any more.

I understand the change of opinion. After going writing a compiler for a big language and learning the intricacies that are possible, it's easy to reach Geoff's position: a deep understanding comes from the experience.

That doesn't mean I don't wish my semester were twenty weeks instead of fifteen, so that I could go deeper on some other topics, too. I figure there will always be some tension in the design of the course for just that reason.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

April 01, 2013 3:16 PM

Good Sentences, Programming State Edition

I've read a couple of interesting papers recently that included memorable sentences related to program state.

First, Stuart Sierra in On the Perils of Dynamic Scope:

Global state is the zombie in the closet of every Clojure program.

This essay explains the difference between scope and extent, a distinction that affects how easy it is to some of what happens in a program with closures and first-order functions with free variables. Sierra also shows the tension between variables of different kinds, using examples from Clojure. An informative read.

Next, Rob Pike in Go at Google: Language Design in the Service of Software Engineering, a write-up of his SPLASH 2012 keynote address:

The motto [of the Go language] is, "Don't communicate by sharing memory, share memory by communicating."

Imperative programmers who internalize this simple idea are on their way to understanding and using functional programming style effectively. The inversion of sharing and communication turns a lot of design and programming patterns inside out.

Pike's notes provide a comprehensive example of how a new language can grow out of the needs of a particular set of applications, rather than out of programming language theory. The result can look a little hodgepodge, but using such a language often feels just fine. (This reminds me of a different classification of languages with similar practical implications.)

~~~~

(These papers weren't published April Fool's Day, so I don't think I've been punked.)


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

March 27, 2013 12:46 PM

Programming Language as Operating System

We are deep in the semester now, using Racket in our programming languages course. I was thinking recently about how little of Racket's goodness we use in this course. We use it primarily as a souped-up R5RS Scheme and handy IDE. Tomorrow we'll see some of Racket's tools for creating new syntax, which will explore one of the rich niches of the system my students haven't seen yet.

I'm thinking about ways to introduce a deeper understanding of The Racket Way, in which domain concepts are programming language constructs and programming languages are extensible and composable. But it goes deeper. Racket isn't just a language, or a set of languages. It is an integrated family of tools to support language creation and use. To provide all these services, Racket acts like an operating system -- and gives you full programmatic access to the system.

(You can watch the video of Flatt's StrangeLoop talk "The Racket Way" at InfoQ -- and you should.)

The idea is bigger than Racket, of course. Dan Ingalls expressed this idea in his 1981 Byte article, Design Principles Behind Smalltalk:

Operating System: An operating system is a collection of things that don't fit into a language. There shouldn't be one.

Alan Kay talks often about this philosophy. The divide between programming language and operating system makes some things more difficult for programmers, and complicates the languages and tools we use. It also creates a divide in the minds of programmers and imposes unnecessary limitations on what programmers think is possible. One of things that appealed to me in Flatt's StrangeLoop talk is that presented a vision of programming without those limits.

There are implications of this philosophy, and costs. Smalltalk isn't just a language, with compilers and tools that you use at your Unix prompt. It's an image, and a virtual machine, and an environment. You don't use Smalltalk; you live inside it.

After you live in Smalltalk for a while, it feels strange to step outside and use other languages. More important, when you live outside Smalltalk and use traditional languages and tools, Smalltalk feels uncomfortable at best and foreboding at worst. You don't learn Smalltalk; you assimilate. -- At least that's what it feels like to many programmers.

But the upside of the "programming language as operating system" mindset you find in Smalltalk and Racket can be huge.

This philosophy generalizes beyond programming languages. emacs is a text editor that subsumes most everything else you do, if you let it. (Before I discovered Smalltalk in grad school, I lived inside emacs for a couple of years.)

You can even take this down to the level of the programs we write. In a blog entry on delimited continuations, Andy Wingo talks about the control this construct gives the programmer over how their programs work, saying:

It's as if you were implementing a shell in your program, as if your program were an operating system for other programs.

When I keep seeing the same idea pop up in different places, with a form that fits the niche, I'm inclined to think I am seeing one of the Big Ideas of computer science.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

March 20, 2013 4:39 PM

Team Rings, Not Turing Awards

Alan Kay

Alan Kay recently wrote this on the Fundamentals of New Computing mailing list:

My own personal thoughts about what was accomplished [with Smalltalk] are completely intertwined with what our entire group was able to do in a few years at PARC. I would give us credit for a very high level combination of "computer science" and "software engineering" and "human centered design" and "commingled software and hardware", etc. The accomplishment was the group's accomplishment. And this whole (to me at least) was a lot more interesting than just a language idea.

I hasten to redirect personal praise to the group accomplishment whenever it happens.

I think this is also true for the larger ARPA-PARC community, and why it was able to accomplish so much at so many levels.

The "awards to individuals" structure beloved of other fields and of journalists completely misses the nature of this process. Any recognition should be like "World Series" rings -- everybody gets one, and that's it.

When Kay spoke at the 2004 OOPSLA Educators' Symposium as part of his Turing Award festivities, he frequently acknowledged the contributions of his team, in particular Dan Ingalls, and the influence that so many other people had on his team's work. Kay must have particularly appreciated receiving the Charles Stark Draper Prize together with Butler Lampson, Robert Taylor, and Charles Thacker, who helped create the conditions in which his team thrived.

In academia, we talk a lot about teamwork, but we tend to isolate individual performance for recognition. I like Kay's analogy to the rings received by teams that win sports championships. In those venues, the winners are unmistakably teams, even when a Michael Jordan or a Tom Brady stands out. That's how academic research tends to work, too. Perhaps we should make that clear more often in the awards we give.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Managing and Leading

March 12, 2013 4:30 PM

Two Views on Why Lisp's Syntax Matters

In year or so, I have seen a few people write to debunk the idea that Lisp is special because its code is written in the primary data structure of the language, list. The one I remember best is Dave Herman's Homoiconicity isn't the point, which points out that the critical technical feature that makes Lisp syntax powerful is that it can be read without being parsed.

This morning I read an old Slashdot post in which Lisp guru Kent Pitman gives a more philosophical answer to the question about what makes Lisp's syntax so special:

I like Lisp's willingness to represent itself. People often explain this as its ability to represent itself, but I think that's wrong. Most languages are capable of representing themselves, but they simply don't have the will to.

That's a nice turn of phrase:  : Lisp is willing to represent itself in data, whereas most languages don't have "the will" to do so. It's not about possibility, but facility.

It's easier to manipulate and generate programs from inside a Lisp or Scheme program than any other language that most of us might see on a daily basis. Rubyists manipulate nested arrays of symbols that encode abstract syntax trees, but this style feels somewhat artificial, and besides Ruby's syntax is so large that it's hard for a Ruby program to process other Ruby programs in this way.

As Pitman says, the fact that Lisp programs are represented by lists is almost besides the point. It might well have been arrays of some other data structure. The key is that it is the program's structure being represented, and not the character-level syntax of the programs. This is the same reason that code can be read without being parsed, and that the macro system can be so powerful.

It's also what makes it so easy to provide powerful support for programmers in their text editors and other tools. These tools don't require a lot of machinery or runtime to navigate and manipulate program code. The structure of the code lies close to its surface.

In the end, I like have two ways to think about Lisp's and Scheme's syntactic advantages: the technical reasons that live in the read procedure and the visceral reasons that embody how programmers feel will they work with a syntax that is willing to help the programmer.


Posted by Eugene Wallingford | Permalink | Categories: Computing

March 11, 2013 4:25 PM

Does Readability Give a False Sense of Understandability?

In Good for Whom?, Daniel Lyons writes about the readability of code. He starts with Dan Ingall's classic Design Principles Behind Smalltalk, which places a high value on a system being comprehensible by a single person, and then riffs on readability in J and Smalltalk.

Early on, Lyons made me smile when he noted that, while J is object-oriented, it's not likely to be used that way by many people:

... [because] to use advanced features of J one must first use J, and there isn't a lot of that going on either.

As a former Smalltalker, I know how he feels.

Ultimately, Lyons is skeptical about claims that readability increases the chances that a language will attract a large audience. For one thing, there are too many counterexamples in both directions. Languages like C, which "combines the power of assembly language with the readability of assembly language" [ link ], are often widely used. Languages such as Smalltalk, Self, and Lisp, which put a premium on features such as purity and factorability, which in turn enhance readability, never seem to grow beyond a niche audience.

Lyons's insight is that readability can mislead. He uses as an example the source code of the J compiler, which is written in C but in a style mimicking J itself:

So looking at the J source code, it's easy for me to hold my nose and say, that's totally unreadable garbage; how can that be maintained? But at the same time, it's not my place to maintain it. Imagine if it were written in the most clean, beautiful C code possible. I might be able to dupe myself into thinking I could maintain it, but it would be a lie! Is it so bad that complex projects like J have complex code? If it were a complex Java program instead, I'd still need substantial time to learn it before I would stand a chance at modifying it. Making it J-like means I am required to understand J to change the source code. Wouldn't I have to understand J to change it anyway?

There is no point in misleading readers who have trouble understanding J-like code into thinking they understand the compiler, because they don't. A veneer of readability cannot change that.

I know how Lyons feels. I sometimes felt the same way as I learned Smalltalk by studying the Smalltalk system itself. I understood how things worked locally, within a method and then within a class, but I didn't the full network of classes that made up the system. And I had the scars -- and trashed images -- to prove it. Fortunately, Smalltalk was able to teach me many things, including object-oriented programming, along the way. Eventually I came to understand better, if not perfectly, how Smalltalk worked down its guts, but that took a lot of time and work. Smalltalk's readability made the code accessible to me early, but understanding still took time.

Lyons's article brought to mind another insight about code's understandability that I blogged about many years ago in an entry on comments in code. This insight came from Brian Marick, himself no stranger to Lisp or Smalltalk:

[C]ode can only ever be self-explanatory with respect to an expected reader.

Sometimes, perhaps it's just as well that a language or a program not pretend to be more understandable than it really is. Maybe a barrier to entry is good, by keeping readers out until they are ready to wield the power it affords.

If nothing else, Lyons's stance can be useful as a counterweight to an almost unthinking admiration of readable syntax and programming style.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

February 21, 2013 3:13 PM

Ray Bradbury Channels Alan Kay

... in a Comic-Con 2010 interview:

Don't think about things, just do them.
Don't predict them, just make them.

This goes a bit farther than Kay's "The best way to predict the future is invent it". In particular, I think he is okay with thinking about things.

Text and audio excerpts of the Bradbury interview are available on-line at Brain Pickings.


Posted by Eugene Wallingford | Permalink | Categories: Computing

February 18, 2013 12:59 PM

Code Duplication as a Hint to Think Differently

Last week, one of my Programming Languages students sent me a note saying that his homework solution worked correctly but that he was bothered by some duplicated code.

I was so happy.

Any student who has me for class for very long hears a lot about the dangers of duplication for maintaining code, and also that duplication is often a sign of poor design. Whenever I teach OOP or functional programming, we learn ways to design code that satisfy the DRY principle and ways to eliminate it via refactoring when it does sneak in.

I sent the student an answer, along with hearty congratulations for recognizing the duplication and wanting to eliminate it. My advice

When I sat down to blog the solution, I had a sense of deja vu... Hadn't I written this up before? Indeed I had, a couple of years ago: Increasing Duplication to Eliminate Duplication. Even in the small world of my own teaching, it seems there is nothing new under the sun.

Still, there was a slightly different feel to the way I talked about this in class later that day. The question had come earlier in the semester this time, so the code involved was even simpler. Instead of processing a vector or a nested list of symbols, we were processing with a flat list of symbols. And, instead of applying an arbitrary test to the list items, we were simply counting occurrences of a particular symbol, s.

The duplication occurred in the recursive case, where the procedure handles a pair:

    (if (eq? s (car los))
        (+ 1 (count s (cdr los)))      ; <---
        (count s (cdr los)))           ; <---

Then we make the two sub-cases more parallel:

    (if (eq? s (car los))
        (+ 1 (count s (cdr los)))      ; <---
        (+ 0 (count s (cdr los))))     ; <---

And then use distributivity to push the choice down a level:

    (+ (if (eq? s (car los)) 1 0)
       (count s (cdr los)))            ; <--- just once!

This time, I made a point of showing the students that not only does this solution eliminate the duplication, it more closely follows the command to follow the shape of the data:

When defining a program to process an inductively-defined data type, the structure of the program should follow the structure of the data.

This guideline helps many programmers begin to write recursive programs in a functional style, rather than an imperative style.

Note that in the first code snippet above, the if expression is choosing among two different solutions, depending on whether we see the symbol s in the first part of the pair or not. That's imperative thinking.

But look at the list-of-symbols data type:

    <list-of-symbols> ::= ()
                        | (<symbol> . <list-of-symbols>)

How many occurrences of s are in a pair? Obviously, the number of s's found in the car of the list plus the number of s's found in the cdr of the list. If we design our solution to match the code to the data type, then the addition operation should be at the top to begin:

    (+ ; number of s's found in the car
       ; number of s's found in the cdr )

If we define the answer for the problem in terms of the data type, we never create the duplication-by-if in the first place. We think about solving the subproblems for the car and the cdr, fill in the blanks, and arrive immediately at the refactored code snippet above.

I have been trying to help my students begin to "think functionally" sooner this semester. There is a lot or room for improvement yet in my approach. I'm glad this student asked his question so early in the semester, as it gave me another chance to model "follow the data" thinking. In any case, his thinking was on the right track.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development, Teaching and Learning

February 12, 2013 2:53 PM

Student Wisdom on Monad Tutorials

After class today, a few of us were discussing the market for functional programmers. Talk turned to Clojure and Scala. A student who claims to understand monads said:

To understand monad tutorials, you really have to understand monads first.

Priceless. The topic of today's class was mutual recursion. I think we are missing a base case here.

I don't know whether this is a problem with monads, a problem with the writers of monad tutorials, or a problem with the rest of us. If it is true, then it seems a lot of people are unclear on the purpose of a tutorial.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Teaching and Learning

February 07, 2013 5:01 PM

Quotes of the Day

Computational Thinking Division. From Jon Udell, another lesson that programming and computing teach us which can be useful out in the world:

Focus on understanding why the program is doing what it's doing, rather than why it's not doing what you wanted it to.

This isn't the default approach of everyone. Most of my students have to learn this lesson as a part of learning how to program. But it can be helpful outside of programming, in particular by influencing how we interact with people. As Udell says, it can be helpful to focus on understanding why one's spouse or child or friend is doing what she is doing, rather than on why she isn't doing what you want.

Motivational Division. From the Portland Ballet, of all places, several truths about being a professional dancer that generalize beyond the studio, including:

There's a lot you don't know.
There may not be a tomorrow.
There's a lot you can't control.
You will never feel 100% ready.

So get to work, even if it means reading the book and writing the code for the fourth time. That is where the fun and happiness are. All you can affect, you affect by the work you do.

Mac Chauvinism Division. From Matt Gemmell, this advice on a particular piece of software:

There's even a Windows version, so you can also use it before you've had sufficient success to afford a decent computer.

But with enough work and a little luck, you can afford better next time.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Managing and Leading

January 26, 2013 5:52 PM

Computing Everywhere: Indirection

Alice: The hardest word you'll ever be asked to spell is "ichdericious".

Bob: Yikes. Which word?

A few of us have had fun with the quotations in English and Scheme over the last few days, but this idea is bigger than symbols as data values in programs or even words and strings in natural language. They are examples of a key element of computational thinking, indirection, which occurs in real life all the time.

A few years ago, my city built a new water park. To account for the influx of young children in the area, the city dropped the speed limit in the vicinity of the pool from 35 MPH to 25 MPH. The speed limit in that area has been 35 MPH for a long time, and many drivers had a hard time adjusting to the change. So the city put up a new traffic sign a hundred yards up the road, to warn drivers of the coming change. It looks like this one:

traffic sign: 40 MPH speed limit ahead

The white image in the middle of this sign is a quoted version of what drivers see down the road, the usual:

traffic sign: 40 MPH speed limit

Now, many people slow down to the new speed limit well in advance, often before reaching even the warning sign. Maybe they are being safe. Then again, maybe they are confusing a sign about a speed limit sign with the speed limit sign itself.

If so, they have missed a level of indirection.

I won't claim that computer scientists are great drivers, but I will say that we get used to dealing with indirection as a matter of course. A variable holds a value. A pointer holds the address of a location, which holds a value. A URL refers to a web page. The list goes on.

Indirection is a fundamental element in the fabric of computation. As computation becomes an integral part of nearly everyone's daily life, there is a lot to be gained by more people understanding the idea of indirection and recognizing opportunities to put it to work to mutual benefit.

Over the last few years, Jon Udell has been making a valiant attempt to bring this issue to the attention of computer scientists and non-computer scientists alike. He often starts with the idea of a hyperlink in a web page, or the URL to which it is tied, as a form of computing indirection that everyone already groks. But his goal is to capitalize on this understanding to sneak the communication strategy of pass by reference into people's mental models.

As Udell says, most people use hyperlinks every day but don't use them as well as they might, because the distinction between "pass by value" and "pass by reference" is not a part of their usual mental machinery:

The real problem, I think, is that if you're a newspaper editor, or a city official, or a citizen, pass-by-reference just isn't part of your mental toolkit. We teach the principle of indirection to programmers. But until recently there was no obvious need to teach it to everybody else, so we don't.

He has made the community calendar his working example of pass by reference, and his crusade:

In the case of calendar events, you're passing by value when you send copies of your data to event sites in email, or when you log into an events site and recopy data that you've already written down for yourself and published on your own site.

You're passing by reference when you publish the URL of your calendar feed and invite people and services to subscribe to your feed at that URL.

"Pass by reference rather than by value" is one of Udell's seven ways to think like the web, his take on how to describe computational thinking in a world of distributed, network media. That essay is a good start on an essential module in any course that wants to prepare people to live in a digital world. Without these skills, how can we hope to make the best use of technology when it involves two levels of indirection, as shared citations and marginalia do?

Quotation in Scheme and pass-by-reference are different issue, but they are related in a fundamental way to the concept of indirection. We need to arm more people with this concept than just CS students learning how programming languages work.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

January 25, 2013 4:47 PM

More on Real-World Examples of Quotation

My rumination on real-world examples of quotation to use with my students learning Scheme sparked the imaginations of several readers. Not too surprisingly, they came up with better examples than my own... For example, musician and software developer Chuck Hoffman suggested:

A song, he sang.
"A song", he sang.

The meaning of these is clearly different depending on whether we treat a song as a variable or as a literal.

My favorite example came from long-time friend Joe Bergin:

"Lincoln" has seven letters.
Lincoln has seven letters.

Very nice. Joe beat me with my own example!

As Chuck wrote, song titles create an interesting challenge, whether someone is singing a certain song or singing in a way defined by the words that happen to also be the song's title. I have certainly found it hard to find words both that are part of a title or a reference and that flow seamlessly in a sentence.

This turns out to be a fun form of word play, independent of its use as a teaching example. Feel free to send me your favorites.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

January 24, 2013 4:32 PM

Real-Life Examples of Quotation in Scheme

The new semester is fully underway, and I'm already enjoying Programming Languages. My Tuesday session this week felt like a hodgepodge of topics, including Scheme definitions and conditionals, and didn't inspire my students much. Today's session on pairs and lists seemed to go much more smoothly, at least from my side of the classroom.

One thing that has been different the first two weeks this time around has been several questions about the quote character in Scheme, which is shorthand for the special form quote.

The purpose of the quote is to tell the interpreter to take its argument literally. When the argument is a list, say, '(* 2 3), quotation prevents the interpreter from evaluating the list as a Scheme procedure call. When the argument is a symbol, say, 'a, the quote lets the interpreter know not to treat the a as an identifier, looking up the value bound to that name in the current environment. Instead, it is treated as the literal symbol a. Most of our students have not yet worked in languages where symbols are first-class data values, so this idea takes some getting used to.

In the course of talking about quotation with them, I decided to relate this idea to an example of quotation from real life. The first thing that came to mind at that instant was the distinction between these two sentences:

Lincoln was disappointing.
"Lincoln" was disappointing.

In the former, Lincoln is a name to be evaluated. Depending on the context, it could refer to the 16th president of the United States, the capital of Nebraska, or some other object in the world. (The sentence doesn't have to be true, of course!)

In the latter, quoting Lincoln makes it a title. I intended for this "literal" reference to the word Lincoln to evoke the current feature film of that name.

Almost immediately I began to second-guess my example. The quoted Lincoln is still a name for something -- a film, or a boo, or some such -- and so still needs to be "dereferenced" to retrieve the object signified. It's just that we treat titles differently than other names.

So it's close to what I wanted to convey, but it could mislead students in a dangerous way.

The canonical real-world example of quotation is to quote a word so that we treat the utterance as the word itself. Consider:

Creativity is overused.
"Creativity" is overused.

In the former, creativity is a name to be evaluated. It signifies an abstract concept, a bundle of ideas revolving around creation, originality, art, and ingenuity. We might say creativity is overused in a context where people should be following the rules but are instead blazing their own trails.

In the latter, the quoted creativity signifies the word itself, taken literally. We might say "creativity" is overused to suggest an author improve a piece of writing by choosing a near-synonym such as "cleverness" or "originality", or by rephrasing a sentence so that the abstract concept is recast as the verb in an active statement.

This example stays more faithful to the use of quote in Scheme, where an expression is taken literally, with no evaluation of of any kind needed.

I like giving examples of how programming concepts exist in other parts of our lives and world. Even when they are not perfect matches, they can sometimes help a student's mind click on the idea as it works in a programming language or style.

I like it better when I use better examples!


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

January 18, 2013 2:42 PM

Alive with Infinite Possibilities

the PARC 5-key Chord Keyboard, courtesy the Buxton collection

Engelbart's Violin tells the interesting story of Douglas Engelbart's chorded keyboard, or "chorder", an input device intended as a supplement to the traditional keyboard. Engelbart was part of a generation that saw computing as a universe of unlimited possibilities, and more than many others he showed us glimpses of what it could be.

I grew up in an age when an unadorned BASIC interpreter was standard equipment on any computer, and with so little software available to us, we all wrote programs to make the machine do our bidding. In a narrower way, we felt the sense of unlimited possibilities that drove Engelbart, Sutherland, and the generations that came before us. If only we all had vision as deep.

Unfortunately, not many teenagers get to have that kind of experience anymore. BASIC became VB.Net, a corporate language for a corporate world. The good news is that languages like Python and even JavaScript make programming accessible to more people again, but the ethos of anyone learning to program on his or her own at home seems to have died off.

Engelbart's Violin uses strong language to judge the current state of computing, with some of its strongest lamenting the "cruel discrepancy" between the experience of a creative child learning to program and the world of professional programming:

When you are a teenager, alone with a (programmable) computer, the universe is alive with infinite possibilities. You are a god. Master of all you survey. Then you go to school, major in "Computer Science", graduate -- and off to the salt mines with you, where you will stitch silk purses out of sow's ears in some braindead language, building on the braindead systems created by your predecessors, for the rest of your working life. There will be little room for serious, deep creativity. You will be constrained by the will of your master (whether the proverbial "pointy-haired boss", or lemming-hordes of fickle startup customers) and by the limitations of the many poorly-designed systems you will use once you no longer have an unconstrained choice of task and medium.

Ouch. We who teach CS at the university find ourselves trapped between the needs of a world that employs most of our graduates and the beauty that computing offers. Alas, what Alan Kay said about Engelbart applies more broadly: "Engelbart, for better or for worse, was trying to make a violin.... [M]ost people don't want to learn the violin." I'm heartened to see so many people, including my own colleagues, working so hard to bring the ethos and joy of programming back to children, using Scratch, media computation, and web programming.

This week, I began a journey with thirty or so undergraduate CS students, who over the next four months will learn Scheme and -- I hope -- get a glimpse of the infinite possibilities that extend beyond their first jobs, or even their last. At the very least, I hope I don't shut any more doors on them.

~~~~

PHOTO. The PARC 5-key Chord Keyboard, from the Buxton collection at Microsoft Research.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

January 10, 2013 3:59 PM

The Pleasure of Elegant Composition in a Programming Language

At the 1978 APL Conference, Alan Perlis gave a talk called Almost Perfect Artifacts Improve only in Small Ways, in which he said:

What attracted me, then, to APL was a feeling that perhaps through APL one might begin to acquire some of the dimensions in programming that we revere in natural language -- some of the pleasures of composition; of saying things elegantly; of being brief, poetic, artistic, that makes our natural languages so precious to us. That aspect of programming was one that I've long been interested in but have never found any lever for coming close to in my experience with languages of the FORTRAN, ALGOL, PL/I school.

I learned APL as as an undergrad and knew immediately that thinking in it was unlike thinking in any other language I had learned, even Lisp. These languages, though, shared a Wow! factor. They enabled programs that did amazing things, or ordinary things in amazing ways. By contrast, BASIC and FORTRAN and PL/I seemed so prosaic.

As an undergrad, I never developed the sort of fluency in that would allow me to, say, write an assembler in 20 lines of APL or Lisp. I did develop a fondness for functional programming that stayed with me into graduate school, where I came into deeper contact with Lisp. I also learned Smalltalk, which I came to admire in a way similar to Perlis's feeling for APL.

I must admit, the beauty and expressiveness of array languages and functional languages have always felt less natural to me than natural language. Their more mathematical orientation felt foreign to me, less like writing in a natural language than solving a puzzle. This wasn't a matter of me not liking math; I took advanced math throughout school and always enjoyed. But it felt different to me. This is, I see now, a personal preference and likely an indicator of why I was drawn more intimately into computer science than into more study of math.

The language I use these days that makes me feel the way Perlis feels about APL is Ruby. It occupies a similar space as Python, which we teach our students and use in several courses. I like Python a lot, more than I like most languages, but it feels plain to me in the sense once explained by John Cook. It is simple, and I get things done when I program in it, but when I use it, I feel like I am programming.

Ruby has this goofy, complex syntax that makes it possible to write some hideous stuff. But Ruby also makes it possible to write code that is brief and elegant, even artistic.

I first saw this at PLoP many years ago, when looking over Gerard Meszaros's shoulder at code he had written to support the writing and publishing of his XUnit Test Patterns book. His code read like the book he was writing. Then I began to see DSLs embedded in Ruby, tools like Rake and Treetop, that made me forget about the language they were implemented in. When you use those tools and others like them, you were writing in a new language, one that fit the thoughts in your head. Yet you were still unmistakably writing Ruby.

Perhaps if I were more an engineer at heart, I would feel differently about simple, sturdy languages that let me get things done. I like them, but they don't make me feel like I am "under the influence", as Perlis writes. They are just attractive tools. Perhaps if I were more a mathematician at heart, I would feel even more at home with the elegance that Haskell and APL give me.

Whatever the reasons, Smalltalk and Ruby grabbed in ways that no other languages have. I think that is due at least in part to the way they connected to my personal understanding and love for natural language. It's interesting how we can all feel this way about different programming languages. I think it says something important about the nature of computing and programming.


Posted by Eugene Wallingford | Permalink | Categories: Computing

December 31, 2012 8:22 AM

Building Things and Breaking Things Down

As I look toward 2013, I've been thinking about Alan Kay's view of CS as science [ link ]:

I believe that the only kind of science computing can be is like the science of bridge building. Somebody has to build the bridges and other people have to tear them down and make better theories, and you have to keep on building bridges.

In 2013, what will I build? What will I break down, understand, and help others to understand better?

One building project I have in mind is an interactive text. One analysis project in mind involves functional design patterns.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Patterns

December 28, 2012 10:03 AM

Translating Code Gibberish to Human-Speak

Following an old link to Ridiculous Fish's Unix shell fish, I recently stumbled upon the delightful cdecl, a service that translates C declarations, however inscrutable, into plain English (and vice versa). As this introductory post says,

Every C declaration will be as an open book to you! Your coworkers' scruffy beards and suspenders will be nigh useless!

The site even provides permalinks so that you can share translations of your thorniest C casts with friends and family.

These pages are more than three years old, so I'm surely telling you something you already know. How did I just find this?

I don't program in C much these days, so cdecl itself is of use to me only as humorous diversion. But it occurs to me that simple tools like this could be useful in a pedagogical setting. Next semester, my students will be learning Scheme and functional programming style. The language doesn't have much syntax, but it does have all those parentheses. Whatever I say or do, they disorient many of my students for a while. Some them will look at even simple code such as

     (let ((x (square 4))
           (y 7))
       (+ x y))

... and feel lost. We spend time in class learning how to read code, and talk about the semantics of such expressions, which helps. But in a pinch, wouldn't it be nice for a student to hit a button and have that code translated into something more immediately comprehensible? Perhaps:

Let x be the square of 4 and y be 7 in the sum of x and y.

This might be a nice learning tool for students as they struggle with a language that seems to them -- at least early on -- to be gibberish on a par with char (*(*(* const x[3])())[5])(int).

Some Scheme masters might well say, "But the syntax and semantics of a let are straightforward. You don't really need this tool." At one level, this is true. Unfortunately, it ignores the cognitive and psychological challenges that most people face when they learn something that is sufficiently unfamiliar to them.

Actually, I think we can use the straightforwardness of the translation as a vehicle to help students learn more than just how a let expression works. I have a deeper motive.

Learning Scheme and functional programming are only a part of the course. Its main purpose is to help students understand programming languages more generally, and how they are processed by interpreters and compilers.

When we look at the let expression above, we can see that translating it into the English expression is not only straightforward, it is 100% mechanical. If it's a mechanical process, then we can write a program to do it for us! Following a BNF description of the expression's syntax, we can write an interpreter that exposes the semantics of the expression.

In many ways, that is the essence of this course.

At this point, this is only a brainstorm, perhaps fueled by holiday cooking and several days away from the office. I don't know yet how much I will do with this in class next term, but there is some promise here.

Of course, we can imagine using a cdecl-like tool to help beginners learn other languages, too. Perhaps there are elements of writing OO code in Java that confuse students enough to make a simple translator useful. Surely public static void main( String[] args) deserves some special treatment! Ruby is complex enough that it might require dozens of little translators to do it justice. Unfortunately, it might take Matz's inside knowledge to write them.

(The idea of translating inscrutable code into language understandable by humans is not limited to computer code, of course. There is a popular movement, to write laws and other legal code in Plain English. This movement is occasionally championed by legislators -- especially in election years. The U.S. Securities and Exchange Commission has its own Plain English Initiative and Plain English Handbook. At seventy-seven pages, the SEC handbook is roughly the same size as R6RS description of Scheme.)


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

December 17, 2012 3:39 PM

The Web is More Than The Latest App or Walled Garden

Anil Dash, on the web we lost:

... today's social networks, they've brought in hundreds of millions of new participants to these networks, and they've certainly made a small number of people rich.

But they haven't shown the web itself the respect and care it deserves, as a medium which has enabled them to succeed. And they've now narrowed the possibilities of the web for an entire generation of users who don't realize how much more innovative and meaningful their experience could be.

I've never warmed to Facebook, for much this reason. I enjoy Twitter, but I treat it as a source of ephemera. Anything that I want to last gets cached in a file of links, shared with colleagues or friends by e-mail, or -- best of all -- blogged about.

I sometimes wonder if blog readers will weary of finding links to things they've already seen via Twitter, or if Twitter has trained too many of us not to want to read someone's comments on such articles in blog entries. But this seems one of the great and lasting values of a blog, one that will remain even after Facebook and Twitter have gone the way of Usenet and GeoCities. The social web is more, and I want to remain a part of it.


Posted by Eugene Wallingford | Permalink | Categories: Computing

December 12, 2012 4:18 PM

Be a Driver, Not a Passenger

Some people say that programming isn't for everyone, just as knowing how to tinker under the hood of one's car isn't for everyone. Some people design and build cars; other people fix them; and the rest of us use them as high-level tools.

Douglas Rushkoff explains why this analogy is wrong:

Programming a computer is not like being the mechanic of an automobile. We're not looking at the difference between a mechanic and a driver, but between a driver and a passenger. If you don't know how to drive the car, you are forever dependent on your driver to take you where you want to go. You're even dependent on that driver to tell you when a place exists.

This is CS Education week, "a highly distributed celebration of the impact of computing and the need for computer science education". As a part of the festivities, Rushkoff was scheduled to address members of Congress and their staffers today about "the value of digital literacy". The passage quoted above is one of ten points he planned to make in his address.

As good as the other nine points are -- and several are very good -- I think the distinction between driver and passenger is the key, the essential idea for folks to understand about computing. If you can't program, you are not a driver; you are a passenger on someone else's trip. They get to decide where you go. You may want to invent a new place entirely, but you don't have the tools of invention. Worse yet, you may not even have the tools you need to imagine the new place. The world is as it is presented to you.

Don't just go along for the ride. Drive.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

December 09, 2012 5:12 PM

Just Build Things

The advantage of knowing how to program is that you can. The danger of knowing how to program is that you will want to.

From Paul Graham's How to Get Startup Ideas:

Knowing how to hack also means that when you have ideas, you'll be able to implement them. That's not absolutely necessary..., but it's an advantage. It's a big advantage, when you're considering an idea ..., if instead of merely thinking, "That's an interesting idea," you can think instead, "That's an interesting idea. I'll try building an initial version tonight."

Writing programs, like any sort of fleshing out of big ideas, is hard work. But what's the alternative? Not being able to program, and then you'll just need a programmer.

If you can program, what should you do?

[D]on't take any extra classes, and just build things. ... But don't feel like you have to build things that will become startups. That's premature optimization. Just build things.

Even the professor in me has to admit this is true. You will learn a lot of valuable theory, tools, and practices in class. But when a big idea comes to mind, you need to build it.

As Graham says, perhaps the best way that universities can help students start startups is to find ways to "leave them alone in the right way".

Of course, programming skills are not all you need. You'll probably need to be able to understand and learn from users:

When you find an unmet need that isn't your own, it may be somewhat blurry at first. The person who needs something may not know exactly what they need. In that case I often recommend that founders act like consultants -- that they do what they'd do if they'd been retained to solve the problems of this one user.

That's when those social science courses can come in handy.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

December 07, 2012 11:17 AM

Agglutination and Crystallization

Alan Kay talks about programming languages quite a bit in this wide-ranging interview. (Aren't all interviews with Kay wide-ranging?) I liked this fuzzy bifurcation of the language world:

... a lot of them are either the agglutination of features or ... a crystallization of style.

My initial reaction was that I'm a crystallization-of-style guy. I have always had a deep fondness for style languages, with Smalltalk at the head of the list and Joy and Scheme not far behind.

But I'm not a purist when it comes to neat and scruffy. As an undergrad, I really liked programming in PL/I. Java never bothered me as much as it bothered some of my purist friends, and I admit unashamedly that I enjoy programming in it.

These days, I like Ruby as much as I like any language. It is a language that lies in the fuzz between Kay's categories. It has an "everything is an object" ethos but, man alive, is it an amalgamation of syntactic and semantic desiderata.

I attribute linguistic split personality to this: I prefer languages with a "real center", but I don't mind imposing a stylistic filter on an agglutinated language. PL/I always felt comfortable because I programmed with a pure structured programming vibe. When I program in Java or Ruby now, somewhere in the center of my mind is a Smalltalk programmer seeing the language through a Smalltalk lens. I have to make a few pragmatic concessions to the realities of my tool, and everything seems to work out fine.

This semester, I have been teaching with Java. Next semester, I will be teaching with Scheme. I guess I can turn off the filter.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 30, 2012 3:49 PM

Passing Out the Final Exam on Day One

I recently ran across an old blog posting called Students learn what they need, not what is assigned, in which a Ted Dunning described a different sort of "flipped course" than is usually meant: He gave his students the final exam on Day 1, passed out the raw materials for the course, and told them to "get to work". They decided what they needed to learn, and when, and asked for instruction and guidance on their own schedule.

Dunning was happy with the results and concluded that...

... these students could learn vastly more than was expected of them if they just wanted to.

When students like what they are doing, they can surprise most everyone with what they will do to learn. Doing something cool like building a robot (as Dunning's students did) can be all the motivation some students need.

I'm sometimes surprised by just what catches my students' fancy. A few weeks ago, I asked my sophomore- and junior-level OOP class to build the infrastructure for a Twitter-like app. It engaged them like only graphical apps usually do. They've really dug into the specs to figure out what they mean. Many of them don't use Twitter, which has been good, because it frees them of too many preconceived limitations on where they can take their program.

They are asking good questions, too, about design: Should this object talk to that one? The way I divided up the task led to code that feels fragile; is there a better way? It's so nice not to still be answering Java questions. I suspect that some are still encountering problems at the language level, but they are solving them on their own and spending more time thinking about the program at a higher level.

I made this a multi-part project. They submitted Iteration 1 last weekend, will submit Iteration 2 tomorrow, and will work on Iteration 3 next week. That's a crucial element, I think, in getting students to begin taking their designs more seriously. It matters how hard it easy to change the code, because they have to change it now -- and tomorrow!

The point of Dunning's blog is that students have to discover the need to know something before they are really interesting in learning it. This is especially true if the learning process is difficult or tedious. You can apply this idea to a lot of software development, and even more broadly to CS.

I'm not sure when I'll try the give-the-final-exam-first strategy. My compiler course already sort of works that way, since we assign the term project upfront and then go about learning what we need to build the compiler. But I don't make my students request lectures; I still lay the course out in advance and take only occasional detours.

I think I will go at least that far next semester in my programming languages course, too: show them a language on day one and explain that our goal for the semester is to build an interpreter for it by the end of the semester, along with a few variations that explore the range of possibilities that programming languages offer. That may create a different focus in my mind as I go through the semester. I'm curious to see.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 28, 2012 6:34 PM

Converting Lecture Notes into an Active Website

... in which the author seeks pointers to interactive Scheme materials on-line.

Last summer, I fiddled around a bit with Scribble, a program for writing documentation in (and for) Racket. I considered using it to write the lecture notes and website for my fall OOP course, but for a variety of reasons set it aside.

the icon for Slideshow

In the spring I'll be teaching Programming Languages again, and using Racket with my students. This seems like the perfect time to dive in and use Scribble and Slideshow to create all my course materials. This will create a synergy between what I do in class and how I prep, which will be good for me. Using Racket tools will also set a good example for my students.

After seeing The Racket Way, Matthew Flatt's talk at StrangeLoop, I am inspired to do more than simply use Racket tools to create text and slides and web pages. I'd like to re-immerse myself in a world where everything is a program, or nearly so. This would set an even more important example for my students, and perhaps help them to see more clearly that they don't ever to settle for the programs, the tools, or the languages that people give them. That is the Computer Science way as well as the Racket way.

I've also been inspired recently by the idea of an interactive textbook a lá Miller and Ranum. I have a pretty good set of lecture notes for Programming Languages, but the class website should be more than a 21st-century rendition of a 19th-century presentation. I think that using Scribble and Slideshow are a step in the right direction.

So, a request: I am looking for examples of people using the Racket presentation tools to create web pages that have embedded Scheme REPLs, perhaps even a code stepper of the sort Miller and Ranum use for Python. Any pointers you might have are welcome.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 18, 2012 9:13 AM

Programming Languages Quote of the Day

... comes from Gilad Bracha:

I firmly believe that a time traveling debugger is worth more than a boatload of language features[.]

This passage comes as part of a discussion of what it would take to make Bret Victor's vision of programming a reality. Victor demonstrates powerful ideas using "hand crafted illustrations of how such a tool might behave". Bracha, whose work on Smalltalk and Newspeak have long inspired me -- reflects on what it would take to offer Victor's powerful ideas in a general purpose programming environment.

Smalltalk as a language and environment works at a level where we conceive of providing the support Victor and Bracha envision, but most of the language tools people use today are too far removed from the dynamic behavior of the programs being written. The debugger is the most notable example.

Bracha suggests that we free the debugger from the constraints of time and make it a tool for guiding the evolution of the program. He acknowledges that he is not the first person to propose such an idea, pointing specifically to Bill Lewis's proposal for an omniscient debugger. What remains is the hard work needed to take the idea farther and provide programmers more transparent support for understanding dynamic behavior while still writing the code.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 15, 2012 4:04 PM

Teaching Students to Read and Study in a New Way

Mark Guzdial's How students use an electronic book, reports on the research paper "Performance and Use Evaluation of an Electronic Book for Introductory Python Programming" [ pdf ]. In this paper, Alvarado et al. evaluate how students used the interactive textbook How to Think Like a Computer Scientist by Ranum and Miller in an intro CS course. The textbook integrates traditional text with embedded video, "active" examples using an embedded Python interpreter, and empirical examples using a code stepper a lá a debugger.

The researchers were surprised to find how little some students used the book's interactive features:

One possible explanation for the less-than-anticipated use of the unique features may be student study skills. The survey results tend to suggest that students "study" by "reading". Few students mention coding or tracing programs as a way of "studying" computer science.

I am not using an interactive textbook in my course this semester, but I have encountered the implicit connection in many students' minds between studying and reading. It caught me off-guard, too.

After lengthy searching and some thought, I decided to teach my sophomore-level OOP course without a required text. I gave students links to two on-line books they could use as Python references, but neither covers the programming principles and techniques that are at the heart of the course. In lieu of a traditional text, I have been giving my students notes for each session, written up carefully in a style that resembles a textbook, and source code -- lots and lots of source code.

Realizing that this would be an unusual way for students to study for a CS class, at least compared to their first-year courses, I have been pretty consistent in encouraging them to work this way. Daily I suggest that they unpack the code, read it, compile it, and tinker with it. The session notes often include little exercises they can do to test or extend their understanding of a topic we have covered in class. In later sessions, I often refer back to an example or use it as the basis for something new.

I figured that, without a textbook to bog them down, they would use my session notes as a map and spend most of their time in the code spelunking, learning to read and write code, and seeing the ideas we encounter in class alive in the code.

a snapshot of Pousse cells in two dimensions

Like the results reported in the Alvarado paper, my experiences have been mixed, and in many ways not what I expected. Some students read very little, and many of those who do read the lecture notes spend relatively little time playing with the code. They will spend plenty of time on our homework assignments, but little or no time on code for the purposes of studying. My data is anecdotal, based on conversations with the subset of students who visit office hours and e-mail exchanges with students who ask questions late at night. But performance on the midterm exam and some of the programming assignments are consistent with my inference.

OO programs are the literature of this course. Textbooks are like commentaries and (really long) Cliff Notes. If indeed the goal is to get students to read and write code, how should we proceed? I have been imagining an even more extreme approach:

  • no textbook, only a language reference
  • no detailed lecture notes, only cursory summaries of what we did in class
  • code as a reading assignment before each session
  • every day in class, students do tasks related to the assigned reading -- engaging, fun tasks, but tasks they can't or wouldn't want to do without having studied the assigned code

A decade or so ago, I taught a course that mixed topics in user interfaces and professional ethics using a similar approach. It didn't provide magic results, but I did notice that once students got used to the unusual rhythm of the course they generally bought in to the approach. The new element here is the emphasis on code as the primary literature to read and study.

Teaching a course in a way that subverts student expectations and experience creates a new pedagogical need: teaching new study skills and helping students develop new work habits. Alvarado et al. recognize that this applies to using a radically different sort of textbook, too:

Might students have learned more if we encouraged them to use codelens more? We may need to teach students new study skills to take advantage of new learning resources and opportunities.

...

Another interesting step would be to add some meta-instruction. Can we teach students new study skills, to take advantage of the unique resources of the book? New media may demand a change in how students use the media.

I think those of us who teach at the university level underestimate how important meta-level instruction of this sort is to most of students. We tend to assume that students will figure it out on their own. That's a dangerous assumption to make, at least for a discipline that tends to lose too many good students on the way to graduation.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 03, 2012 11:17 AM

When "What" Questions Presuppose "How"

John Cook wrote about times in mathematics when maybe you don't need to do what you were asked to do. As one example, he used remainder from division. In many cases, you don't need to do division, because you can find the answer using a different, often simpler, method.

We see a variation of John's theme in programming, too. Sometimes, a client will ask for a result in a way that presupposes the method that will be used to produce it. For example, "Use a stack to evaluate these nested expressions." We professors do this to students a lot, because they want the students to learn the particular technique specified. But you see subtle versions of this kind of request more often than you might expect outside the classroom.

An important part of learning to design software is learning to tease apart the subtle conflation of interface and implementation in the code we write. Students who learn OO programming after a traditional data structures course usually "get" the idea of data abstraction, yet still approach large problems in ways that let implementations leak out of their abstractions in the form of method names and return values. Kent Beck talked about how this problem afflicts even experienced programmers in his blog entry Naming From the Outside In.

Primitive Obsession is another symptom of conflating what we need with how we produce it. For beginners, it's natural to use base types to implement almost any behavior. Hey, the extreme programming principle You Ain't Gonna Need It encourages even us more experienced developers not to create abstractions too soon, until we know we need them and in what form. The convenience offered by hashes, featured so prominently in the scripting languages that many of us use these days, makes it easy to program for a long time without having to code a collection of any sort.

But learning to model domain objects as objects -- interfaces that do not presuppose implementation -- is one of the powerful stepping stones on the way to writing supple code, extendible and adaptable in the face of reasonable changes in the spec.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

October 24, 2012 11:38 AM

"Don't Break the Phone; Fix the Computer"

Rob Pike in The Set-Up:

Twenty years ago, you expected a phone to be provided everywhere you went, and that phone worked the same everywhere. At a friend's house, or a restaurant, or a hotel, or a pay phone, you could pick up the receiver and make a call. You didn't carry a phone around with you; phones were part of the infrastructure. Computers, well, that was a different story. As laptops came in, people started carrying computers around with them everywhere. The reason was to have the state stored on the computer, not the computer itself. You carry around a computer so you can access its disk.

In summary, it used to be that phones worked without you having to carry them around, but computers only worked if you did carry one around with you. The solution to this inconsistency was to break the way phones worked rather than fix the way computers work.

Ah, the memories of grad school, WYSE terminals, and VT-100 emulation.

The advent of ubiquitous networking is making it possible for us to return to the days of dumb terminals. Is that where we want to live?

Pike's vision notwithstanding: I still carry a computer, both for state and processor. I access networked computers frequently. I do not yet carry a phone. I remain happy.


Posted by Eugene Wallingford | Permalink | Categories: Computing

October 19, 2012 3:08 PM

Computer Programming, Education Reform, and Changing Our Schools

Seymour Papert

You almost can't go wrong by revisiting Seymour Papert's work every so often. This morning I read Why School Reform Is Impossible, which reminds us that reform and change are different things. When people try to "reform" education by injecting a new idea from outside, schools seem to assimilate the reform into its own structure, which from the perspective of the reformer blunts or rejects the intended reform. Yet schools and our education system do change over time, evolving as the students, culture, and other environmental factors change.

As people such as Papert and Alan Kay have long argued, a big part of the problem in school reform involving computers is that we misunderstand what a computer is:

If you ask, "Which is not like the other two?" in the list "educational movie, textbook, computer", it is pretty obvious from my perspective that the answer must be "computer."

... not "textbook", which is how most people answer, including many people who want to introduce more computers into the classroom. Textbooks and movies are devices for receiving content that someone else made. Computers are for creating content. It just so happens that we can use them to communicate ideas in new ways, too.

This misunderstanding leads people to push computers for the wrong reasons, or at least for reasons that miss their game-changing power. We sometimes here that "programming is the new Latin". Papert reminds us that the reasons we used to teach Latin in schools changed over time:

In recent times, Latin was taught in schools because it was supposed to be good for the development of general cognitive skills. Further back, it was taught because it was the language in which all scholarly knowledge was expressed, and I have suggested that computational language could come to play a similar role in relation to quite extensive areas of knowledge.

If programming is the new Latin, it's not Latin class, circa 1960, in which Latin taught us to be rigorous students. It's Latin class, circa 1860 or 1760 or 1560, in which Latin was the language of scholarly activity. As we watch computing become a central part of the language of science, communication, and even the arts and humanities, we will realize that students need to learn to read and write code because -- without that skill -- they are left out of the future.

No child left behind, indeed.

In this essay, Paper gives a short version of his discussion in Mindstorms of why we teach the quadratic equation of the parabola to every school child. He argues that its inclusion in the curriculum has more to do with its suitability to the medium of the say -- pencil and paper -- than to intrinsic importance. I'm not too sure that's true; knowing how parabolas and ellipses work is pretty important for understanding the physical world. But it is certainly true that how and when we introduce parabolas to students can change when we have a computer and a programming language at hand.

Even at the university we encounter this collision of old and new. Every student here must take a course in "quantitative reasoning" before graduating. For years, that was considered to be "a math course" by students and advisors alike. A few years ago, the CS department introduced a new course into the area, in which students can explores a lot of the same quantitative issues using computation rather than pencil and paper. With software tools for modeling and simulation, many students can approach and even begin to solve complex problems much more quickly than they could working by hand. And it's a lot more fun, too.

To make this work, of course, students have to learn a new programming language and practice using it in meaningful ways. Papert likens it to learning a natural language like French. You need to speak it and read it. He says we would need the programming analog of "the analog of a diverse collection of books written in French and access to French-speaking people".

the Scratch logo cat

The Scratch community is taking at shot at this. The Scratch website offers not only a way to download the Scratch environment and a way to view tutorials on creating with Scratch. It also offers -- front and center, the entire page, really -- links to shared projects and galleries. This gives students a chance first to be inspired by other kids and then to download and read the actual Scratch programs that enticed them. It's a great model.

The key is to help everyone see that computers are not like textbooks and televisions and movie projectors. As Mitch Resnick has said:

Computers for most people are black boxes. I believe kids should understand objects are "smart" not because they're just smart, but because someone programmed them to be smart.

What's most important ... is that young children start to develop a relationship with the computer where they feel they're in control. We don't want kids to see the computer as something where they just browse and click. We want them to see digital technologies as something they can use to express themselves.

Don't just play with other people's products. Make your own.

Changes in the world's use of computing may do more to cause schools to evolve in a new direction than anyone's educational reforms ever could. Teaching children that they can be creators and not simply consumers is a subversive first step.

~~~~

IMAGE 1: Seymour Papert at the OLPC offices in Cambridge, Massachusetts, in 2006. Source: Wikimedia Commons License: Creative Commons Attribution-Share Alike 2.0.

IMAGE 2: The Scratch logo. Source: Wikimedia Commons License: Creative Commons Attribution-Share Alike 2.0.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

October 12, 2012 10:37 AM

Make Let, Not Var

I don't blog images for their own sake often, but this mash-up makes me happy:

a parody of Lennon and Ono's 'Make Love, Not War' image

Even as I enjoy teaching OO programming this semester, this reminds me that I'll enjoy teaching functional programming in the spring.

This came to me via a tweet. If you know the source, I'd love to hear from you.


Posted by Eugene Wallingford | Permalink | Categories: Computing

October 07, 2012 2:50 PM

Equality Check Patterns for Recursive Structures

I first encountered this trio of programming patterns when writing Smalltalk programs to manipulate graphs back in the late 1980s. These days I see them most often when comparing recursive data types in language processors. Andrew Black wrote about these patterns in the context of Smalltalk on the Squeak mailing list in the mid-2000s.

~~~~

Recursive Equality Check

Problem

You are working with a recursive data structure. In the simplest form, you might have a list or a record that can contain lists or records as parts. In a language application, you might have a structured type that can have the same type as one of its parts. For instance, a function type might allow a function as an argument or a function as a return value.

You have two instances of the structure and need to compare them, say, for equality or for dominance. In the language app, for example, you will need to verify that the type of an argument matches the type of the formal parameter on a procedure.

Solution

Standard structural recursion works. Walk the two structures in parallel, checking to see that they have the same values in the same positions. When one of the positions holds values the same structure, make a recursive call to compare them.

But what if ...

~~~~

Recursive Equality Check with Identity Check

Problem

An instance of the recursive structure can contain a reference to itself as a value, either directly or through mutual recursion. In the simplest form, this might be a dictionary that contains itself as a value in a key/vaue pair, as in Smalltalk, where the global variable Smalltalk is the dictionary of all global variables, including Smalltalk.

Comparing two instances now raises concerns. Two instances may be identical, or contain identical components. In such cases, the standard recursive comparison will never terminate.

Solution

Check for identity first. Recurse only if the two values are distinct.

But what if...

~~~~

Recursive Equality Check with Cache

Problem

You have two structures that do not share any elements, but they are structurally isomorphic. For example, this can occur in the simplest of structures, two one-element maps:

    a = { :self => a }
    b = { :self => b }

Now, even with an identity test up front, the recursive comparison will never terminate.

Solution

Maintain a cache of compared pairs. Before you begin to compare two objects, check to see if the pair is in the cache. If yes, return true. Otherwise, add the pair to the cache and proceed.

This approach works even though the function has not finished comparing the two objects yet. If there turns out to be a difference between the two, the check currently in progress will find it elsewhere and answer false. There is no need to enter a recursive check.

~~~~

A variation of this caching technique can also be used in other situations, such as computing a hash value for a recursive structure. If in the course of computing the hash you encounter the same structure again, assume that the value is a suitable constant, such as 0 or 1. Hashes are only approximations anyway, so making only one pass over the structure is usually enough. If you really need a fixpoint, then you can't take this shortcut.

Ruby hashes handle all three of these problems correctly:

    a = { :first  => :int,
          :second => { :first  => :int, :second => :int } }
    b = { :second => { :second => :int, :first  => :int },
          :first  => :int, }

a == b # --> true

c = { :first => c } a = { :first => :int, :second => { :first => c, :second => :int } } b = { :first => :int, :second => { :first => c, :second => :int } }

a == b # --> true

a = { :self => a } b = { :self => b }

a == b # --> true

I don't know if either MRI Ruby or JRuby uses these patterns to implement their solutions, or if they use some other technique.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

September 30, 2012 12:45 PM

StrangeLoop 8: Reactions to Brett Victor's Visible Programming

The last talk I attended at StrangeLoop 2012 was Bret Victor's Visible Programming. He has since posted an extended version of his presentation, as a multimedia essay titled Learnable Programming. You really should read his essay and play the video in which he demonstrates the implementation of his ideas. It is quite impressive, and worthy of the discussion his ideas have engendered over the last few months.

In this entry, I give only a high-level summary of the idea, react to only one of his claims, and discuss only one of his design principles in ay detail. This entry grew much longer than I originally intended. If you would like to skip most of my reaction, jump to the mini-essay that is the heart of this entry, Programing By Reacting, in the REPL.

~~~~

Programmers often discuss their productivity as at least a partial result of the programming environments they use. Victor thinks this is dangerously wrong. It implies, he says, that the difficulty with programming is that we aren't doing it fast enough.

But speed is not the problem. The problem is that our programming environments don't help us to think. We do all of our programming in our minds, then we dump our ideas into code via the editor.

Our environments should do more. They should be our external imagination. They should help us see how our programs work as we are writing them.

This is an attractive guiding principle for designing tools to help programmers. Victor elaborates this principle into a set of five design principles for an environment:

  • read the vocabulary -- what do these words mean?
  • follow the flow -- what happens when?
  • see the state -- what is the computer thinking?
  • create by reacting -- start somewhere, then sculpt
  • create by abstracting -- start concrete, then generalize

Victor's talk then discussed each design principle in detail and showed how one might implement the idea using JavaScript and Processing.js in a web browser. The demo was cool enough that the StrangeLoop crowd broke into applause at leas twice during the talk. Read the essay.

~~~~

As I watched the talk, I found myself reacting in a way I had not expected. So many people have spoken so highly of this work. The crowd was applauding! Why was I not as enamored? I was impressed, for sure, and I was thinking about ways to use these ideas to improve my teaching. But I wasn't falling head over heels in love.

A Strong Claim

First, I was taken aback by a particular claim that Victor made at the beginning of his talk as one of the justifications for this work:

If a programmer cannot see what a program is doing, she can't understand it.

Unless he means this metaphorically, seeing "in the mind's eye", then it is simply wrong. We do understand things we don't see in physical form. We learn many things without seeing them in physical form. During my doctoral study, I took several courses in philosophy, and only rarely did we have recourse to images of the ideas we were studying. We held ideas in our head, expressed in words, and manipulated them there.

We did externalize ideas, both as a way to learn them and think about them. But we tended to use stories, not pictures. By speaking an idea, or writing it down, and sharing it with others, we could work with them.

So, my discomfort with one of Victor's axioms accounted for some of my unexpected reaction. Professional programmers can and do manipulate ideas abstractly. Visualization can help, but when is it necessary, or even most helpful?

Learning, Versus Doing

This leads to a second element of my concern. I think I had a misconception about Victor's work. His talk and its title, "Visible Programming", led me to think his ideas are aimed primarily at working programmers, that we need to make programs visible for all programmers.

The title of his essay, "Learnable Programming", puts his claims into a different context. We need to make programs visible for people who are learning to program. This seems a much more reasonable position on its face. It also lets me see the axiom that bothered me so much in a more sympathetic light: If a novice programmer cannot see what a program is doing, then she may not be able to understand it.

Seeing how a program works is a big part of learning to program. A few years ago, I wrote about "biction" and the power of drawing a picture of what code does. I often find that if I require a student to draw a picture of what his code is doing before he can ask me for debugging help, he will answer his own question before getting to me.

The first time a student experiences this can be a powerful experience. Many students begin to think of programming in a different way when they realize the power of thinking about their programs using tools other than code. Visible programming environments can play a role in helping students think about their programs, outside their code and outside their heads.

I am left puzzling over two thoughts:

  • How much of the value my students see in pictures comes from not from seeing the program work but from drawing the picture themselves -- the act of reflecting about the program? If our tools visualizes the code for them, will we see the same learning effect that we see in drawing their own pictures?

  • Certainly Victor's visible programming tools can help learners. How much will they help programmers once they become experts? Ben Shneiderman's Designing the User Interface taught me that novices and experts have different needs, and that it's often difficult to know what works well for experts until we run experiments.

Mark Guzdial has written a more detailed analysis of Victor's essay from the perspective of a computer science educator. As always, Mark's ideas are worth reading.

Programming By Reacting, in the REPL

My favorite parts of this talk were the sections on creating by reacting and abstracting. Programmers, Victor says, don't work like other creators. Painters don't stare at a blank canvas, think hard, create a painting in their minds, and then start painting the picture they know they want to create. Sculptors don't stare at a block of stone, envision in their mind's eye the statue they intend to make, and then reproduce that vision in stone. They start creating, and react, both to the work of art they are creating and to the materials they are using.

Programmers, Victor says, should be able to do the same thing -- if only our programming environments helped us.

As a teacher, I think this is an area ripe for improvement in how we help students learn to program. Students open up their text editor or IDE, stare at that blank screen, and are terrified. What do I do now? A lot of my work over the last fifteen to twenty years has been in trying to find ways to help students get started, to help them to overcome the fear of the blank screen.

My approaches haven't been through visualization, but through other ways to think about programs and how we grow them. Elementary patterns can give students tools for thinking about problems and growing their code at a scale larger than characters or language keywords. An agile approach can help them start small, add one feature at a time, proceed in confidence with working tests, and refactor to make their code better as they go along. Adding Victor-style environment support for the code students write in CS1 and CS2 would surely help as well.

However, as I listened to Victor describe support for creating by reacting, and then abstracting variables and functions out of concrete examples, I realized something. Programmers don't typically write code in an environment with data visualizations of the sort Victor proposes, but we do program in the style that such visualizations enable.

We do it in the REPL!

A simple, interactive computer programming environment enables programmers to create by reacting.

  • They write short snippets of code that describe how a new feature will work.
  • They test the code immediately, seeing concrete results from concrete examples.
  • They react to the results, shaping their code in response to what the code and its output tell them.
  • They then abstract working behaviors into functions that can be used to implement another level of functionality.

Programmers from the Lisp and Smalltalk communities, and from the rest of the dynamic programming world, will recognize this style of programming. It's what we do, a form of creating by reacting, from concrete examples in the interaction pane to code in the definitions pane.

In the agile software development world, test-first development encourages a similar style of programming, from concrete examples in the test case to minimal code in the application class. Test-driven design stimulates an even more consciously reactive style of programming, in which the programmer reacts both to the evolving program and to the programmer's evolving understanding of it.

The result is something similar to Victor's goal for programmers as they create abstractions:

The learner always gets the experience of interactively controlling the lower-level details, understanding them, developing trust in them, before handing off that control to an abstraction and moving to a higher level of control.

It seems that Victor would like to perform even more support for novices than these tools can provide, down to visualizing what the program does as they type each line of code. IDEs with autocomplete is perhaps the closest analog in our current arsenal. Perhaps we can do more, not only for novices but also professionals.

~~~~

I love the idea that our environments could do more for us, to be our external imaginations.

Like many programmers, though, as I watched this talk, I occasionally wondered, "Sure, this works great if you creating art in Processing. What about when I'm writing a compiler? What should my editor do then?"

Victor anticipated this question and pre-emptively answered it. Rather than asking, How does this scale to what I do?, we should turn the question inside out and ask, These are the design requirements for a good environment. How do we change programming to fit?

I doubt such a dogmatic turn will convince skeptics with serious doubts about this approach.

I do think, though, that we can reformulate the original question in a way that focuses on helping "real" programmers. What does a non-graphical programmer need in an external imagination? What kind of feedback -- frequent, even in-the-moment -- would be most helpful to, say, a compiler writer? How could our REPLs provide even more support for creating, reacting, and abstracting?

These questions are worth asking, whatever one thinks of Victor's particular proposal. Programmers should be grateful for his causing us to ask them.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development, Teaching and Learning

September 29, 2012 4:04 PM

StrangeLoop 7: The Racket Way

I have been using Racket since before it was Racket, back when it was "just another implementation of Scheme". Even then, though, it wasn't just another implementation of Scheme, because it had such great libraries, a devoted educational community around it, and an increasingly powerful facility for creating and packaging languages. I've never been a deep user of Racket, though, so I was eager to see this talk by one of its creators and learn from him.

Depending on your perspective, Racket is either a programming language (that looks a lot like Scheme), a language plus a set of libraries, or a platform for creating programs. This talk set out to show us that Racket is more.

Flatt opened with a cute animated fairy tale, about three princesses who come upon a wishing well. The first asks for stuff. The second asks for more wishes. The third asks for a kingdom full of wishing wells. Smart girl, that third one. Why settle for stuff when you can have the source of all stuff?

This is, Flatt said, something like computer science. There is a similar progression of power from:

  • a document, to
  • a language for documents, to
  • a language for languages.

Computer scientists wish for a way to write programs that do... whatever.

This is the Racket way:

  1. Everything is a program.
  2. Concepts are programming language constructs.
  3. Programming languages are extensible and composable.

The rest of the talk was a series of impressive mini-demos that illustrated each part of the Racket way.

To show what it means to say that everything is a program, Flatt demoed Scribble, a language for producing documents -- even the one he was using to give his talk. Scribble allows writers to abstract over every action.

To show what it means to say that concepts are programming language constructs, Flatt talked about the implementation of Dr. Racket, the flexible IDE that comes with the system. Dr. Racket needs to be able to create, control, and terminate processes. Relying on the OS to do this for it means deferring to what that OS offers. In the end, that means no control.

Dr. Racket needs to control everything, so the language provides constructs for these concepts. Flatt showed as examples threads and custodians. He then showed this idea at work in an incisive way: he wrote a mini-Dr. Racket, called Racket, Esq. -- live using Racket. To illustrate its completeness, he then ran his talk inside racket-esq. Talk about a strange loop. Very nice.

To show what it means to say that programming languages are extensible and composable, Flatt showed a graph of the full panoply of Racket's built-in languages and demoed several languages. He then used some of the basic language-building tools in Racket -- #lang, require, define-syntax, syntax-rules, and define-syntax-rule -- to build the old text-based game Adventure, which needs a natural language-like scripting language for defining worlds. Again, very nice -- so much power in so many tools.

This kind of power comes from taking seriously a particular way of thinking about the world. It starts with "Everything is a program." That is the Racket way.

Flatt is a relaxed and confident presenter. As a result, this was a deceptively impressive talk. It reinforced its own message by the medium in which it was delivered: using documents -- programs -- written and processed in Racket. I am not sure how anyone could see a slideshow with "hot" code, a console for output, and a REPL within reach, all written in the environment being demoed, and not be moved to rethink how they write programs. And everything else they create.

As Flatt intimated briefly early on, The Racket Way of thinking is not -- or should not be -- limited to Racket. It is, at its core, the essence of of computer science. The duality of code and data makes what we do so much more powerful than most people realize, and makes what we can do so much more powerful than most us actually do with the tools we accept. I hope that Flatt's talk inspires a few more of us not to settle for less than we have to.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 29, 2012 3:40 PM

StrangeLoop 6: Y Y

I don't know if it was coincidence or by design of the conference organizers, but Wednesday morning was a topical repeat of Tuesday morning for me: two highly engaging talks on functional programming. I had originally intended to write them up in a single entry, but that write-up grew so long that I decided to give them their own entries.

Y Not?

Watching talks and reading papers about the Y combinator are something of a spectator code kata for me. I love to see new treatments, and enjoy seeing even standard treatments every now and then. Jim Weirich presented it at StrangeLoop with a twist I hadn't seen before.

Weirich opened, as speakers often do, with him. This is a motivational talk, so it should be...

  • non-technical. But it's not. It is highly technical.
  • relevant. But it's not. It is extremely pointless.
  • good code. But it's not. It shows the worst Clojure code ever.

But it will be, he promises, fun!

Before diving in, he had one more joke, or at least the first half of one. He asked for audience participation, then asked his volunteer to calculate cos(n) for some value of n I missed. Then he asked the person to keep hitting the cosine button repeatedly until he told him to stop.

At the dawn of computing, to different approaches were taken in an effort to answer the question, What is effectively computable?

Alan Turing devised what we now call a universal Turing machine to embody the idea. Weirich showed a video demonstration of a physical Turing machine to give his audience a sense of what a TM is like.

(If you'd like to read more about Turing and the implication of his universal machine, check out this reflection I wrote earlier this year after a visit by Doug Hofstadter to my campus. Let's just say that the universal TM means more than just an answer to what functions are effectively computable.)

A bit ahead of Turing, Alonzo Church devised an answer to the same question in the form of the lambda calculus, a formal logical system. As with the universal TM, the lambda calculus can be used to compute everything, for a particular value of eveything. These days, nearly every programming language has lambdas of some form

... now came the second half of the joke running in the background. Weirich asked his audience collaborator what was in his calculator's display. The assistant called out some number, 0.7... Then Weirich showed his next slide -- the same number, taken out many more digits. How was he able to do this? There is a number n such that cos(n) = n. By repeatedly pressing his cosine button, Weirich's assistant eventually reached it. That number n is called the fixed point of the cosine function. Other functions have fixed points to, and they can be a source of great fun.

Then Weirich opened up his letter and wrote some code from the ground up to teach some important concepts of functional programming, using the innocuous function 3(n+1). With this short demo, Weirich demonstrated the idea of a higher-order function, including function factories, a set of useful functional refactorings that included

  • Introduce Binding
    -- where the new binding is unused in the body
  • Inline Definition
    -- where a call to a function is replaced by the function body, suitably parameterized
  • Wrap Function
    -- where an expression is replaced by a function call that computes the expression
  • Tennent Correspondence Principle
    -- where an expression is turned into a think

At the end of his exercise, Weirich had created a big function call that contained no named function definitions yet computed the same answer.

He asks the crowd for applause, then demurs. This is 80-year-old technology. Now you know, he says, what a "chief scientist" at New Context does. (Looks a lot like what an academic might do...)

Weirich began a second coding exercise, the point behind all his exposition to this point: He wrote the factorial function, and began to factor and refactor it just as he had the simpler 3(n+1). But now inlining the function breaks the code! There is a recursive call, and the name is now out of scope. What to do?

He refactors, and refactors some more, until the body of factorial is an argument to a big melange of lambdas and applications of lambdas. The result is a function that computes the fixed point of any function passed it.

That is Y. The Y combinator.

Weirich talked a bit about Y and related ideas, and why it matters. He closed with a quote from Wittgenstein, from Philosophical Investigations:

The aspects of things that are most important for us are hidden because of their simplicity and familiarity. (One is unable to notice something -- because it is always before one's eyes.) The real foundations of his enquiry do not strike a man at all. Unless that fact has at some time struck him. -- And this means: we fail to be struck by what, once seen, is most striking and most powerful.

The thing that sets Weirich's presentation of Y apart from the many others I've seen is its explicit use of refactoring to derive Y. He created Y from a sequence of working pieces of code, each the result of a refactoring we can all understand. I love to do this sort of thing when teaching programming ideas, and I was pleased to see it used to such good effect on such a challenging idea.

The title of this talk -- Y Not? -- plays on Y's interrogative homonym. Another classic in this genre echos the homonym in its title, then goes on to explain Y in four pages of English and Scheme. I suggest that you study @rpg's essay while waiting for Weirich's talk to hit InfoQ. Then watch Weirich's talk. You'll like it.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

September 28, 2012 3:59 PM

StrangeLoop 5: Miscellany -- At All Levels

Most of the Tuesday afternoon talks engaged me less deeply than the ones that came before. Part of that was the content, part was the style of delivery, and part was surely that my brain was swimming in so many percolating ideas that there wasn't room for much more.

Lazy Guesses

Oleg Kiselyov, a co-author of the work behind yesterday's talk on miniKanren, gave a talk on how to implement guessing in computer code. That may sound silly, for a couple of reasons. But it's not.

First, why would we want to guess at all? Don't we want to follow principles that guarantee we find the right answer? Certainly, but those principles aren't always available, and even when they are the algorithms that implement them may be computationally intractable. So we choose to implement solutions that restrict the search space, for which we pay a price along some other dimension, often expressiveness.

Kiselyov mentioned scheduling tasks early in his talk, and any student of AI can list many other problems for which "generate and test" is a surprisingly viable strategy. Later in the talk, he mentioned parsing, which is also a useful example. Most interesting grammars have nondeterministic choices in them. Rather than allow our parsers to make choices and fail, we usually adopt rules that make the process predictable. The result is an efficient parser, but a loss in what we can reasonably say in the language.

So, perhaps the ability to make good guesses is valuable. What is so hard about implementing them? The real problem is that there are so many bad guesses. We'd like to use knowledge to guide the process of guessing again, to favor some guesses over others.

The abstract for the talk promises a general principle on which to build guessing systems. I must admit that I did not see it. Kiselyov moved fast at times through his code, and I lost sight of the big picture. I did see discussions of forking a process at the OS level, a fair amount of OCaml code, parser combinators, and lazy evaluation. Perhaps my attention drifted elsewhere at a key moment.

The speaker closed his talk by showing a dense slide and saying, "Here is a list of buzzwords, some of which I said in my talk and some of which I didn't say in my talk." That made me laugh: a summary of a talk he may or may not have given. That seemed like a great way to end a talk about guessing.

Akka

I don't know much about the details of Akka. Many of my Scala-hacking former students talk about it every so often, so I figured I'd listen to this quick tour and pick up a little more. The underlying idea, of course, is Hewitt's Actor model. This is something I'm familiar with from my days in AI and my interest in Smalltalk.

The presenter, Akka creator Jonas Boner, reminded the audience that Actors were a strong influence on the original Smalltalk. In many ways, it is truer to Kay's vision of OOP than the languages we use today.

This talk was a decent introduction to Hewitt's idea and its implementation in Akka. My two favorite things from the talk weren't technical details, but word play:

  • The name "Akka" has many inspirations, including a mountain in northern Sweden, a goddess of the indigenous people of northern Scandinavia, and a palindrome of Actor Kernel / Kernel Actor.

  • Out of context, this quote made the talk for me:
    We have made some optimizations to random.
    Ah, aren't we all looking for those?

Expressiveness and Abstraction

This talk by Ola Bini was a personal meditation on the expressiveness of language. Bini, whose first slide listed him as a "computational metalinguist", started from the idea that, informally, the expressiveness of a language is inversely proportional to the distance between our thoughts and the code we have to write in that language.

In the middle part of the talk, he considered a number of aspects of expressiveness and abstraction. In the latter part, he listed ideas from natural language and wondered aloud what their equivalents would be in programming languages, among them similes, metaphors, repetition, elaboration, and multiple equivalent expressions with different connotations.

During this part of the talk, my mind wandered, too, to a blog entry I wrote about parts of speech in programming languages back in 2003, and a talk by Crista Lopes at OOPSLA that year. Nouns, verbs, pronouns, adjectives, and adverbs -- these are terms I use metaphorically when teaching students about new languages. Then I thought about different kinds of sentence -- declarative, interrogative, imperative, and exclamatory -- and began to think about their metaphorical appearances in our programming languages.

Another fitting way for a talk to end: my mind wondering at the end of a wondering talk.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 27, 2012 5:33 PM

StrangeLoop 4: Computing Like The Brain

Tuesday morning kicked off with a keynote address by Jeff Hawkins entitled "Computing Like The Brain". Hawkins is currently with Numenta, a company he co-founded in 2005, after having founding the Redwood Neuroscience Institute and two companies most technophiles will recognize: Palm and Handspring.

Hawkins said he has devoted his professional life to understanding machine science. He recalls reading an article by Francis Crick in Scientific American as a youth and being inspired to study neuroscience. It was a data-rich, theory-poor discipline, one crying out for abstractions to unify our understanding of how the brain works from the mass of data we were collecting. He says he dedicated life then to discovering principles of how the brain works, especially the neocortex, and to build computer systems that implement these principles.

The talk began with a primer on the neocortex, which can be thought of as a predictive modeling system to controls human intelligence. If we take into account all the components of what we think of as our five senses, the brain has millions of sensors that constantly stream data to the neocortex. Its job is to build an on-line model from this streaming data. It constantly predicts what he expects to receive next, detects anomalies, updates itself, and produces actions. When the neocortex updates, we learn.

On this view, the brain doesn't "compute". It is a memory system. (I immediately thought of Roger Schank, his views on AI, and case-based reasoning...) The brain is really one memory algorithm operating over all of our sensory inputs. The key elements of this memory system are:

  • a hierarchy of regions,
  • sequence memory, and
  • sparse distributed representation.

Hawkins spoke briefly about hierarchy and sequence memory, but he quickly moved into the idea of sparse distributed representation (SDR). This can be contrasted to the dense, localized memory of traditional computer systems. For example, ASCII code consists of seven bits, all combinations of which we use to represent a single character. Capital 'A' is 65, or 1000001; the digit '5' is 55, or 0110111. The coincidence of '5' and 55 notwithstanding, the individual bits of an ASCII code don't mean anything. Change one bit, and you get a different character, sometimes a very different one.

An SDR uses a large number of bits, with only a few set to 1. Hawkins said that typically only ~ 2% of the bits are "on". Each bit in an SDR has specific meaning, one that has been learned through memory updating, not assigned. He then demonstrated several properties of an SDR, such as how it can be used to detect similarities, how it can do "store-and-compare" using only indices, and how it can perform remarkably well using on a sampling of the indices. Associative look-up in the brain's SDR produces surprisingly few errors, and those tend to be related to the probe, corresponding to similar situations encountered previously.

The first takeaway point of the talk was this: Intelligent systems of the future will be built using sparse distributed representation.

At this point, my note-taking slowed. I am not a biologist, so most of what Hawkins was describing lies far outside my area of expertise. So I made a master note -- gotta get this guy's book! -- and settled into more focused listening.

(It turns out that a former student recommended Hawkins's book, On Intelligence, to me a year or two ago. I should have listened to Allyn then and read it!)

One phrase that made me smile later in the talk was the semantic meaning of the wrongness. Knowing why something is wrong, or how, is a huge step up on "just" being wrong. Hawkins referred to this in particular as part of the subtlety of making predictions.

To close, Hawkins offered some conjectures. He thinks that the future of machine intelligence will depend on us developing more and better theory to explain how the brain works, especially in the areas of hierarchy and attention. The most compelling implementation will be an embodied intelligence, with embedded agents distributed across billions of sensors. We need better hardware in order to create faster systems. recall that the brain is more a memory systems than a computation device, so better memory is as or more important than better processors. Finally, we need to find a way to increase the level connectivity among components. Neurons have tens or hundreds of connections to other neurons, and these can be grown or strengthened dynamically. Currently, our computer chips are not good at this.

Where will breakthrough applications come from? He's not sure. In the past, breakthrough applications of technologies have not always been where we expected them.

I gotta read more. As a student of AI, I was never been all that interested in neurobiology or even its implications for my discipline. The cognitive level has always excited me more. But Hawkins makes an interesting case that the underlying technologies we need to reach the cognitive level will look more like our brains than today's computers.


Posted by Eugene Wallingford | Permalink | Categories: Computing

September 25, 2012 9:35 PM

StrangeLoop 2: The Future of Databases is In Memory

The conference opened with a keynote address by Michael Stonebraker, who built Ingres, Postgres, and several other influential database systems. Given all the hubbub about NoSQL the last few years, including at StrangeLoop 2010, this talk brought some historical perspective to a conversation that has been dominated in recent years by youngsters. Stonebraker told the audience that the future is indeed here, but from the outside it will look a lot like the past

The problem, of course, is "big data". It's big because of volume, velocity, and variety. Stonebraker framed his opening comments in terms of volume. In the traditional database setting back in the 1980s, we all bought airplane tickets through a travel agent who acted, for all meaningful purposes, in the role of professional terminal operator. We were doing business "at the speed of the intermediary". The ACID properties were inviolable: "Don't you dare lose my data."

Then came change. The internet disintermediated access to database, cutting intermediaries out of the equation. Volume shot through the roof. PDAs further disintermediated access, removing limitations on the locations from which we accessed our data. Volume shot up even further. Suddenly, databases came to be part of the solution to a much broader class of problems: massively online games, ad placement, new forms of commerce. We all know what that meant for volume.

Stonebraker then offered two reality checks to frame the solution to our big data problems. The first involved the cost of computer memory. One terabyte is a really big database for transaction processing, yet it 1TB of memory now costs $25-50K. Furthermore, the price is dropping faster than transaction volume is rising. So: the big data problem is really now a problem for main memory.

The second reality check involved database performance. Well under 10% of the time spent by a typical database is spent doing useful work. Over 90% is overhead: managing a buffer pool, latching, locking, and recovery. We can't make faster databases by creating better DB data structures or algorithms; a better B-tree can affect only 4% of application runtime. If we could eliminate the buffer pool, we can gain up to 25% in performance. We must focus on overhead.

Where to start? We can forget about all the traditional database vendors. They have code lines that are thirty years old and older. They have to manage backward compatibility for a huge installed customer base. They are, from the perspective of the future, bloatware. They can't improve.

How about the trend toward NoSQL? We can just use raw data storage and write our own low-level code, optimized to the task. Well, the first thing to realize is that the compiler already translates SQL into lower-level operations. In the world of databases as almost everywhere else, it is really hard to beat the compiler at its game. High-level languages are good, and our compilers do an awesome job generating near-optimal code. Moving down an abstraction layer is, Stonebraker says, a fundamental mistake: "Always move code to the data, never data to the code."

Second, we must realize that the ACID properties really are a good thing. More important, they are nearly impossible to retrofit into a system that doesn't already provide them. "Eventually consistent" doesn't really mean eventually consistent if it's possible to sell your last piece of inventory. In any situation where there exists a pair of non-commutative transactions, "eventually consistent" is a recipe for corruption.

So, SQL and ACID are good. Let's keep them. Stonebraker says that instead of NoSQL databases, we should build "NewSQL" databases that improve performance through innovative architectures. Putting the database in main memory is one way to start. He addressed several common objections to this idea ("But what if the power fails??") by focusing on speed and replication. Recovery may be slow, but performance is wildly better. We should optimize for the most common case and treat exceptional cases for what they are: rare.

He mentioned briefly several other components of a new database architecture, such horizontally scaling across a cluster of nodes, automatic sharding, and optimization via stored procedures targeted at the most common activities. The result is not a general purpose solution, but then why does it need to be?

I have a lot of gray hair, Stonebraker said, but that means he has seen these wars before. It's better to stick with what we know to be valuable and seek better performance where our technology has taken us.


Posted by Eugene Wallingford | Permalink | Categories: Computing

September 25, 2012 8:35 PM

StrangeLoop 1: A Miscellany of Ideas

For my lunch break, I walked a bit outside, to see the sun and bend my knee a bit. I came back for a set of talks without an obvious common thread. After seeing the talks, I saw a theme: ideas for writing programs more conveniently or more concisely.

ClojureScript

David Nolen talked about ClojureScript, a Clojure-like language that compiles to Javascript. As he noted, there is a lot of work in this space, both older and newer. The goal of all that work is to write Javascript more conveniently, or generate it from something else. The goal of ClojureScript is to bring the expressibility and flexible programming style of the Lisp world to JS world. Nolen's talk gave us some insights into the work being done to make the compiler produce efficient Javascript, as well as into why you might use ClojureScript in the first place.

Data Structures and Hidden Code

The message of this talk by Scott Vokes is that your choice in data structures plays a big role in determining how much code you have to write. You can make a lot of code disappear by using more powerful data structures. We can, of course, generalize this claim from data structures to data. This is the theme of functional and object-oriented programming, too. This talk highlights how often we forget the lowly data structure when we think of writing less code.

As Vokes said, your choice in data structures sets the "path of least resistance" for what your program will do and also for the code you will write. When you start writing code, you often don't know what the best data structure for your application is. As long as you don't paint yourself into a corner, you should be able to swap a new structure in for the old. The key to this is something novice programmers learn early, writing code not in terms of a data structure but in terms of higher-level behaviors. Primitive obsession can become implementation obsession if you aren't careful.

The meat of this talk was a quick review of four data structures that most programmers don't learn in school: skip lists, difference list, rolling hashes, and jumpropes, a structure Vokes claims to invented.

This talk was a source of several good quote, including

  • "A data structure is just a stupid programming language." -- Bill Gosper
  • "A data structure is just a virtual machine." -- Vokes himself, responding to Gosper
  • "The cheapest, fastest, and most reliable components are those that aren't there." -- Gordon Bell

The first two quotes there would make nice mottos for a debate between functional and OO programming. They also are two sides of the same coin, which destroys the premise of the debate.

miniKanren

As a Scheme programmer and a teacher of programming languages, I have developed great respect and fondness for the work of Dan Friedman over the last fifteen years. As a computer programmer who began his studies deeply interested in AI, I have long had a fondness for Prolog. How could I not go to the talk on miniKanren? This is a small implementation (~600 lines written in a subset of Scheme) of Kanren, a declarative logic programming system described in The Reasoned Schemer.

This talk was like a tag-team vaudeville act featuring Friedman and co-author William Byrd. I can't so this talk justice in a blog entry. Friedman and Byrd interleaved code demo with exposition as they

  • showed miniKanren at its simplest, built from three operators (fresh, conde, and run)
  • extended the language with a few convenient operators for specifying constraints, types, and exclusions, and
  • illustrated how to program in miniKanren by building a language interpreter, EOPL style.

The cool endpoint of using logic programming to build the interpreter is that, by using variables in a specification, the interpreter produces legal programs that meet a given specification. It generates code via constraint resolution.

If that weren't enough, they also demo'ed how their system can, given a language grammar, produce quines -- programs p such that

    (equal p (eval p))
-- and twines, pairs of programs p and q such that
    (and (equal p (eval q))
         (equal q (eval p)))

Then they live-coded an implementation of typed lambda calculus.

Yes, all in fifty minutes. Like I said, you really need to watch the talk at InfoQ as soon as it's posted.

In the course of giving the talk, Friedman stated a rule that my students can use:

Will's law: If your function has a recursion, do the recursion last.

Will followed up with cautionary advice:

Will's second law: If your function has two recursions, call Will.

We'll see how serious he was when I put a link to his e-mail address in my Programming Languages class notes next spring.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 25, 2012 7:31 PM

Blogging from StrangeLoop

StrangeLoop logo

This week I have the pleasure of spending a couple of days expanding my mind at StrangeLoop 2012. I like StrangeLoop because it's a conference for programmers. The program is filled with hot programming topics and languages, plus a few keynotes to punctuate our mental equilibria. The 2010 conference gave me plenty to think about, but I had to skip 2011 while teaching and recovering. This year was a must-see.

I'll be posting the following entries from the conference as time permits me to write them.

You can find links to other write-ups of the conference, as well as slides from some talks and other material, at the StrangeLoop 2012 github site.

Now that the conference has ended, I can say with confidence that StrangeLoop 2012 was even better than StrangeLoop 2010.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 20, 2012 8:09 PM

Computer Science is a Liberal Art

Over the summer, I gave a talk as part of a one-day conference on the STEM disciplines for area K-12, community college, and university advisors. They were interested in, among other things, the kind of classes that CS students take at the university and the kind of jobs they get when they graduate.

In the course of talking about how some of the courses our students take (say, algorithms and the theory of computing) seem rather disconnected from many of the jobs they get (say, web programmer and business analyst), I claimed that the more abstract courses prepare students to understand the parts of the computing world that never change, and the ones that do. The specific programming languages or development stack they use after they graduate to build financial reporting software may change occasionally, but the foundation they get as a CS major prepares them to understand what comes next and to adapt quickly.

In this respect, I said, a university CS education is not job training. Computer Science is a liberal art.

This is certainly true when you compare university CS education with what students get at a community college. Students who come out of a community college networking program often possess specific marketable skills at a level we are hard-pressed to meet in a university program. We bank our program's value on how well it prepares students for a career, in which networking infrastructure changes multiple times and our grads are asked to work at the intersection of networks and other areas of computing, some of which may not exist yet.

It is also true relative to the industries they enter after graduation. A CS education provides a set of basic skills and, more important, several ways to think about problems and formulate solutions. Again, students who come out of a targeted industry or 2-year college training program in, say, web dev, often have "shovel ready" skills that are valuable in industry and thus highly marketable. We bank our program's value on how well it prepares students for a career in which ASP turns to JSP turns PHP turns to JavaScript. Our students should be prepared to ramp up quickly and have a shovel in the hands producing value soon.

And, yes, students in a CS program must learn to write code. That's a basic skill. I often hear people comment that computer science programs do not prepare students well for careers in software development. I'm not sure that's true, at least at schools like mine. We can't get away with teaching all theory and abstraction; our students have to get jobs. We don't try to teach them everything they need to know to be good software developers, or even many particular somethings. That should and will come on the job. I want my students to be prepared for whatever they encounter. If their company decides to go deep with Scala, I'd like my former students to be ready to go with them.

In a comment on John Cook's timely blog entry How long will there be computer science departments?, Daniel Lemire suggests that we emulate the model of medical education, in which doctors serve several years in residency, working closely with experienced doctors and learning the profession deeply. I agree. Remember, though, that aspiring doctors go to school for many years before they start residency. In school, they study biology, chemistry, anatomy, and physiology -- the basic science at the foundation of their profession. That study prepares them to understand medicine at a much deeper level than they otherwise might. That's the role CS should play for software developers.

(Lemire also smartly points out that programmers have the ability to do residency almost any time they like, by joining an open source project. I love to read about how Dave Humphrey and people like him bring open-source apprenticeship directly into the undergrad CS experience and wonder how we might do something similar here.)

So, my claim that Computer Science is a liberal arts program for software developers may be crazy, but it's not entirely crazy. I am willing to go even further. I think it's reasonable to consider Computer Science as part of the liberal arts for everyone.

I'm certainly not the first person to say this. In 2010, Doug Baldwin and Alyce Brady wrote a guest editors' introduction to a special issue of the ACM Transactions on Computing Education called Computer Science in the Liberal Arts. In it, they say:

In late Roman and early medieval times, seven fields of study, rooted in classical Greek learning, became canonized as the "artes liberales" [Wagner 1983], a phrase denoting the knowledge and intellectual skills appropriate for citizens free from the need to labor at the behest of others. Such citizens had ample leisure time in which to pursue their own interests, but were also (ideally) civic, economic, or moral leaders of society.

...

[Today] people ... are increasingly thinking in terms of the processes by which things happen and the information that describes those processes and their results -- as a computer scientist would put it, in terms of algorithms and data. This transformation is evident in the explosion of activity in computational branches of the natural and social sciences, in recent attention to "business processes," in emerging interest in "digital humanities," etc. As the transformation proceeds, an adequate education for any aspect of life demands some acquaintance with such fundamental computer science concepts as algorithms, information, and the capabilities and limitations of both.

The real value in a traditional Liberal Arts education is in helping us find better ways to live, to expose us to the best thoughts of men and women in hopes that we choose a way to live, rather than have history or accident choose a way to live for us. Computer science, like mathematics, can play a valuable role in helping students connect with their best aspirations. In this sense, I am comfortable at least entertaining the idea that CS is one of the modern liberal arts.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

September 05, 2012 5:24 PM

Living with the Masters

I sometimes feel guilty that most of what I write here describes connections between teaching or software development and what I see in other parts of the world. These connections are valuable to me, though, and writing them down is valuable in another way.

I'm certainly not alone. In Why Read, Mark Edmondson argues for the value of reading great literature and trying on the authors' view of the world. Doing so enables us to better understand our own view of the world, It also gives us the raw material out of which to change our worldview, or build a new one, when we encounter better ideas. In the chapter "Discipline", Edmondson writes:

The kind of reading that I have been describing here -- the individual quest for what truth a work reveals -- is fit for virtually all significant forms of creation. We can seek vital options in any number of places. They may be found for this or that individual in painting, in music, in sculpture, in the arts of furniture making or gardening. Thoreau felt he could derive a substantial wisdom by tending his bean field. He aspired to "know beans". He hoed for sustenance, as he tells us, but he also hoed in search of tropes, comparisons between what happened in the garden and what happened elsewhere in the world. In his bean field, Thoreau sought ways to turn language -- and life -- away from old stabilities.

I hope that some of my tropes are valuable to you.

The way Edmondson writes of literature and the liberal arts applies to the world of software in a much more direct ways too. First, there is the research literature of computing and software development. One can seek truth in the work of Alan Kay, David Ungar, Ward Cunningham, or Kent Beck. One can find vital options in the life's work of Robert Floyd, Peter Landin, or Alan Turing; Herbert Simon, Marvin Minsky, or John McCarthy. I spent much of my time in grad school immersed in the writings and work of B. Chandrasekaran, which affected my view of intelligence in both humans and machines.

Each of these people offers a particular view into a particular part of the computing world. Trying out their worldviews can help us articulate our own worldviews better, and in the process of living their truths we sometimes find important new truths for ourselves.

We in computing need not limit ourselves to the study of research papers and books. As Edmondson says the individual quest for the truth revealed in a work "is fit for virtually all significant forms of creation". Software is a significant form of creation, one not available to our ancestors even sixty years ago. Live inside any non-trivial piece of software for a while, especially one that has withstood the buffets of human desire over a period of time, and you will encounter truth -- truths you find there, and truths you create for yourself. A few months trying on Smalltalk and its peculiar view of the world taught me OOP and a whole lot more.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development, Teaching and Learning

August 08, 2012 1:50 PM

Examples First, Names Last

Earlier this week, I reviewed a draft chapter from a book a friend is writing, which included a short section on aspect-oriented programming. The section used common AOP jargon: "cross cutting", "advice", and "point cut". I know enough about AOP to follow his text, but I figured that many of his readers -- young software developers from a variety of backgrounds -- would not. On his use of "cross cutting", I commented:

Your ... example helps to make this section concrete, but I bet you could come up with a way of explaining the idea behind AOP in a few sentences that would be (1) clear to readers and (2) not use "cross cutting". Then you could introduce the term as the name of something they already understand.

This may remind you of the famous passage from Richard Feynman about learning names and understanding things. (It is also available on a popular video clip.) Given that I was reviewing a chapter for a book of software patterns, it also brought back memories of advice that Ralph Johnson gave many years ago on the patterns discussion list. Most people, he said, learn best from concrete examples. As a result, we should write software patterns in such a way that we lead with a good example or two and only then talk about the general case. In pattern style, he called this idea "Concrete Before Abstract".

I try to follow this advice in my teaching, though I am not dogmatic about it. There is a lot of value in mixing up how we organize class sessions and lectures. First, different students connect better with some approaches than others, so variety increases the chances that of connecting with everyone a few times each semester. Second, variety helps to keep students in interested, and being interested is a key ingredient in learning.

Still, I have a preference for approaches that get students thinking about real code as early as possible. Starting off by talking about polymorphism and its theoretical forms is a lot less effective at getting the idea across to undergrads than showing students a well-chosen example or two of how plugging a new object into an application makes it easier to extend and modify programs.

So, right now, I have "Concrete Before Abstract" firmly in mind as I prepare to teaching object-oriented programming to our sophomores this fall.

Classes start in twelve days. I figured I'd be blogging more by now about my preparations, but I have been rethinking nearly everything about the way I teach the course. That has left my mind more muddled that settled for long stretches. Still, my blog is my outboard brain, so I should be rethinking more in writing.

I did have one crazy idea last night. My wife learned Scratch at a workshop this summer and was talking about her plans to use it as a teaching tool in class this fall. It occurred to me that implementing Scratch would be a fun exercise for my class. We'll be learning Java and a little graphics programming as a part of the course, and conceptually Scratch is not too many steps from the pinball game construction kit in Budd's Understanding Object-Oriented Programming with Java, the textbook I have used many times in the course. I'm guessing that Budd's example was inspired by Bill Budge's game for Electronic Arts, Pinball Construction Set. (Unfortunately, Budd's text is now almost as out of date as that 1983 game.)

Here is an image of a game constructed using the pinball kit and Java's AWT graphics framework:

a pinball game constructed using a simple game kit

The graphical ideas needed to implement Scratch are a bit more complex, including at least:

  • The items on the canvas must be clickable and respond to messages.
  • Items must be able to "snap" together to create units of program. This could happen when a container item such as a choice or loop comes into contact with an item it is to contain.

The latter is an extension of collision-detecting behavior that students would be familiar with from earlier "ball world" examples. The former is something we occasionally do in class anyway; it's awfully handy to be able to reconfigure the playing field after seeing how the game behaves with the ball in play. The biggest change would be that the game items are little pieces of program that know how to "interpret" themselves.

As always, the utility of a possible teaching idea lies in the details of implementing it. I'll give it a quick go over the next week to see if it's something I think students would be able to handle, either as a programming assignment or as an example we build and discuss in class.

I'm pretty excited by the prospect, though. If this works out, it will give me a nice way to sneak basic language processing into the course in a fun way. CS students should see and think about languages and how programs are processed throughout their undergrad years, not only in theory courses and programming languages courses.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Teaching and Learning

August 03, 2012 3:23 PM

How Should We Teach Algebra in 2012?

Earlier this week, Dan Meyer took to task a New York Times opinion piece from the weekend, Is Algebra Necessary?:

The interesting question isn't, "Should every high school graduate in the US have to take Algebra?" Our world is increasingly automated and programmed and if you want any kind of active participation in that world, you're going to need to understand variable representation and manipulation. That's Algebra. Without it, you'll still be able to clothe and feed yourself, but that's a pretty low bar for an education. The more interesting question is, "How should we define Algebra in 2012 and how should we teach it?" Those questions don't even seem to be on Hacker's radar.

"Variable representation and manipulation" is a big part of programming, too. The connection between algebra and programming isn't accidental. Matthias Felleisen won the ACM's Outstanding Educator Award in 2010 for his long-term TeachScheme! project, which has now evolved into Program by Design. In his SIGCSE 2011 keynote address, Felleisen talked about the importance of a smooth progression of teaching languages. Another thing he said in that talk stuck with me. While talking about the programming that students learned, he argued that this material could be taught in high school right now, without displacing as much material as most people think. Why? Because "This is algebra."

Algebra in 2012 still rests fundamentally on variable representation and manipulation. How should we teach it? I agree with Felleisen. Programming.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

July 20, 2012 3:39 PM

A Philosopher of Imitation

Ian Bogost, in The Great Pretender: Turing as a Philosopher of Imitation, writes:

Intelligence -- whatever it is, the thing that goes on inside a human or a machine -- is less interesting and productive a topic of conversation than the effects of such a process, the experience it creates in observers and interlocutors.

This is a very nice one-sentence summary of Turing's thesis in Computing Machinery and Intelligence. I wrote a bit about Turing's ideas on machine intelligence a few months back, but the key idea in Bogost's essay relates more closely to my discussion in Turing's ideas on representation and universal machines.

In this centennial year of his birth, we can hardly go wrong in considering again and again the depth of Turing's contributions. Bogost uses a lovely turn of phrase in his title: a philosopher of imitation. What may sound like a slight or a trifle is, in fact, the highest of compliments. Turing made that thinkable.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 18, 2012 2:31 PM

Names, Values, and The Battle of Bull Run

the cover of 'Encyclopedia Brown Finds the Clues'

Author Donald Sobol died Monday. I know him best from his long-running series, Encyclopedia Brown. Like many kids of my day, I loved these stories. I couldn't get enough. Each book consisted of ten or so short mysteries solved by Encyclopedia or Sally Kimball, his de facto partner in the Brown Detective Agency. I wanted to be Encyclopedia.

The stories were brain teasers. Solving them required knowledge and, more important, careful observation and logical deduction. I learned to pay close attention while reading Encyclopedia Brown, otherwise I had no hope of solving the crime before Encyclopedia revealed the solution. In many ways, these stories prepared me for a career in math and science. They certainly were a lot of fun.

One of the stories I remember best after all these years is "The Case of the Civil War Sword", from the very first Encyclopedia Brown book. I'm not the only person who found it memorable; Rob Bricken ranks it #9 among the ten most difficult Encyclopedia Brown mysteries. The solution to this case turned on the fact that one battle had two different names. Northerners often named battles for nearby bodies of water or prominent natural features, while Southerners named them for the nearest town or prominent man-made features. So, the First Battle of Bull Run and the First Battle of Manassas were the same event.

This case taught me a bit of historical trivia and opened my mind to the idea that naming things from the Civil War was not trivial at all.

This story taught me more than history, though. As a young boy, it stood out as an example of something I surely already knew: names aren't unique. The same value can have different names. In a way, Encyclopedia Brown taught me one of my first lessons about computer science.

~~~~

IMAGE: the cover of Encyclopedia Brown Finds the Clues, 1966. Source: Topless Robot.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 14, 2012 11:01 AM

"Most Happiness Comes From Friction"

Last time, I mentioned again the value in having students learn broadly across the sciences and humanities, including computer science. This is a challenge going in both directions. Most students like to concentrate on one area, for a lot of different reasons. Computer science looks intimidating to students in other majors, perhaps especially to the humanities-inclined.

There is hope. Earlier this year, the Harvard Magazine ran The Frisson of Friction, an essay by Sarah Zhang, a non-CS student who decided to take CS 50, Harvard's intro to computer science. Zhang tells the story of finding a thorny, semicolon-induced bug in a program (an extension for Google's Chrome browser) on the eve of her 21st birthday. Eventually, she succeeded. In retrospect, she writes:

Plenty of people could have coded the same extension more elegantly and in less time. I will never be as good a programmer as -- to set the standard absurdly high -- Mark Zuckerberg. But accomplishments can be measured in terms relative to ourselves, rather than to others. Rather than sticking to what we're already good at as the surest path to résumé-worthy achievements, we should see the value in novel challenges. How else will we discover possibilities that lie just beyond the visible horizon?

... Even the best birthday cake is no substitute for the deep satisfaction of accomplishing what we had previously deemed impossible -- whether it's writing a program or writing a play.

The essay addresses some of the issues that keep students from seeking out novel challenges, such as fear of low grades and fear of looking foolish. At places like Harvard, students who are used to succeeding find themselves boxed in by their friends' expectations, and their own, but those feelings are familiar to students at any school. Then you have advisors who subtly discourage venturing too far from the comfortable, out of their own unfamiliarity and fear. This is a social issue as big as any pedagogical challenge we face in trying to make introductory computer science more accessible to more people.

With work, we can help students feel the deep satisfaction that Zhang experienced. Overcoming challenges often leads to that feeling. She quotes a passage about programmers in Silicon Valley, who thrive on such challenges: "Most happiness probably comes from friction." Much satisfaction and happiness come out of the friction inherent in making things. Writing prose and writing programs share this characteristic.

Sharing the deep satisfaction of computer science is a problem with many facets. Those of us who know the satisfaction know it's a problem worth solving.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

July 13, 2012 12:02 PM

How Science -- and Computing -- Are Changing History

While reading a recent Harvard Magazine article about Eric Mazur's peer instruction technique in physics teaching, I ran across a link to an older paper that fascinated me even more! Who Killed the Men of England? tells several stories of research at the intersection of history, archaeology, genomics, evolution, demography, and simulation, such as the conquest of Roman England by the Anglo Saxons.

Not only in this instance, but across entire fields of inquiry, the traditional boundaries between history and prehistory have been melting away as the study of the human past based on the written record increasingly incorporates the material record of the natural and physical sciences. Recognizing this shift, and seeking to establish fruitful collaborations, a group of Harvard and MIT scholars have begun working together as part of a new initiative for the study of the human past. Organized by [professor of medieval history Michael] McCormick, who studies the fall of the Roman empire, the aim is to bring together researchers from the physical, life, and computer sciences and the humanities to explore the kinds of new data that will advance our understanding of human history.

... The study of the human past, in other words, has entered a new phase in which science has begun to tell stories that were once the sole domain of humanists.

I love history as much as computing and was mesmerized by these stories of how scientists reading the "material record" of the world are adding to our knowledge of the human past.

However, this is more than simply a one-way path of information flowing from scientists to humanists. The scientific data and models themselves are underconstrained. The historians, cultural anthropologists, and demographers are able to provide context to the data and models and so extract even more meaning from them. This is a true collaboration. Very cool.

The rise of science is erasing boundaries between the disciplines that we all studied in school. Scholars are able to define new disciplines, such as "the study of the human past", mentioned in the passage above. These disciplines are organized with a greater focus on what is being studied than on how we are studying it.

We are also blurring the line between history and pre-history. It used to be that history required a written record, but that is no longer a hard limit. Science can read nature's record. Computer scientists can build models using genomic data and migration data that suggest possible paths of change when the written and scientific record are incomplete. These ideas become part of the raw material that humanists use to construct a coherent story of the past.

This change in how we are able to study the world highlights the importance of a broad education, something I've written about a few times recently [ 1 | 2 | 3 ] and not so recently. This sort of scholarship is best done by people who are good at several things, or at least curious and interested enough in several things to get to know them intimately. As I wrote in Failure and the Liberal Arts, it's important both not to be too narrowly trained and not to be too narrowly "liberally educated".

Even at a place like Harvard, this can leave scholars in a quandary:

McCormick is fired with enthusiasm for the future of his discipline. "It is exciting. I jump up every morning. But it is also challenging. Division and department boundaries are real. Even with a generally supportive attitude, it is difficult [to raise funds, to admit students who are excellent in more than one discipline, and so on]. ..."

So I will continue to tell computer science students to take courses from all over the university, not just from CS and math. This is one point of influence I have as a professor, advisor, and department head. And I will continue to look for ways to encourage non-CS students to take CS courses and students outside the sciences to study science, including CS. As that paragraph ends:

"... This is a whole new way of studying the past. It is a unique intellectual opportunity and practically all the pieces are in place. This should happen here--it will happen, whether we are part of it or not."

"Here" doesn't have to be Harvard. There is a lot of work to be done.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

July 11, 2012 2:45 PM

A Few Comments on the Alan Kay Interview, and Especially Patterns

Alan Kay

Many of my friends and colleagues on Twitter today are discussing the Interview with Alan Kay posted by Dr. Dobb's yesterday. I read the piece this morning while riding the exercise bike and could not contain my desire to underline passages, star paragraphs, and mark it up with my own comments. That's hard to do while riding hard, hurting a little, and perspiring a lot. My desire propelled me forward in the face of all these obstacles.

Kay is always provocative, and in this interview he leaves no oxen ungored. Like most people do when whenever they read outrageous and provocative claims, I cheered when Kay said something I agreed with and hissed -- or blushed -- when he said something that gored me or one of my pet oxen. Twitter is a natural place to share one's cheers and boos for an artyicle with or by Alan Kay, given the amazing density of soundbites one finds in his comments about the world of computing.

(One might say the same thing about Brian Foote, the source of both soundbites in that paragraph.)

I won't air all my cheers and hisses here. Read the article, if you haven't already, and enjoy your own. I will comment on one paragraph that didn't quite make me blush:

The most disastrous thing about programming -- to pick one of the 10 most disastrous things about programming -- there's a very popular movement based on pattern languages. When Christopher Alexander first did that in architecture, he was looking at 2,000 years of ways that humans have made themselves comfortable. So there was actually something to it, because he was dealing with a genome that hasn't changed that much. I think he got a few hundred valuable patterns out of it. But the bug in trying to do that in computing is the assumption that we know anything at all about programming. So extracting patterns from today's programming practices ennobles them in a way they don't deserve. It actually gives them more cachet.

Long-time Knowing and Doing readers know that patterns are one of my pet oxen, so it would have been natural for me to react somewhat as Keith Ray did and chide Kay for what appears to be a typical "Hey, kids, get off my lawn!" attitude. But that's not my style, and I'm such a big fan of Kay's larger vision for computing that my first reaction was to feel a little sheepish. Have I been wasting my time on a bad idea, distracting myself from something more important? I puzzled over this all morning, and especially as I read other people's reactions to the interview.

Ultimately, I think that Kay is too pessimistic when he says we hardly know anything at all about programming. We may well be closer to the level of the Egyptians who built the pyramids than we are to the engineers who built the Empire State Building. But I simply don't believe that people such as Ward Cunningham, Ralph Johnson, and Martin Fowler don't have a lot to teach most of us about how to make better software.

Wherever we are, I think it's useful to identify, describe, and catalog the patterns we see in our software. Doing so enables us to talk about our code at a level higher than parentheses and semicolons. It helps us bring other programmers up to speed more quickly, so that we don't all have to struggle through all the same detours and tar pits our forebears struggled through. It also makes it possible for us to talk about the strengths and weaknesses of our current patterns and to seek out better ideas and to adopt -- or design -- more powerful languages. These are themes Kay himself expresses in this very same interview: the importance of knowing our history, of making more powerful languages, and of education.

Kay says something about education in this interview that is relevant to the conversation on patterns:

Education is a double-edged sword. You have to start where people are, but if you stay there, you're not educating.

The real bug in what he says about patterns lies at one edge of the sword. We may not know very much about how to make software yet, but if we want to remedy that, we need to start where people are. Most software patterns are an effort to reach programmers who work in the trenches, to teach them a little of what we do know about how to make software. I can yammer on all I want about functional programming. If a Java practitioner doesn't appreciate the idea of a Value Object yet, then my words are likely wasted.

Ward Cunningham

Ironically, many argue that the biggest disappointment of the software patterns effort lies at the other edge of education's sword: an inability to move the programming world quickly enough from where it was in the mid-1990s to a better place. In his own Dr. Dobb's interview, Ward Cunningham observed with a hint of sadness that an unexpected effect of the Gang of Four Design Patterns book was to extend the life of C++ by a decade, rather than reinvigorating Smalltalk (or turning people on to Lisp). Changing the mindset of a large community takes time. Many in the software patterns community tried to move people past a static view of OO design embodied in the GoF book, but the vocabulary calcified more quickly than they could respond.

Perhaps that is all Kay meant by his criticism that patterns "ennoble" practices in a way they don't deserve. But if so, it hardly qualifies in my mind as "one of the 10 most disastrous things about programming". I can think of a lot worse.

Kurt Vonnegut

To all this, I can only echo the Bokononists in Kurt Vonnegut's novel Cat's Cradle: "Busy, busy, busy." The machinery of life is usually more complicated and unpredictable than we expect or prefer. As a result, reasonable efforts don't always turn out as we intend them to. So it goes. I don't think that means we should stop trying.

Don't let my hissing about one paragraph in the interview dissuade you from reading the Dr. Dobb's interview. As usual, Kay stimulates our minds and encourages us to do better.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

June 28, 2012 4:13 PM

"Doing research is therefore writing software."

The lede from RA Manual: Notes on Writing Code, by Gentzkow and Shapiro:

Every step of every research project we do is written in code, from raw data to final paper. Doing research is therefore writing software.

The authors are economists at the University of Chicago. I have only skimmed the beginning of the paper, but I like what little I've seen. They take seriously the writing of computer programs.

  • "This document lays out some broad principles we should all follow."
  • "We encourage you to invest in reading more broadly about software craftsmanship, looking critically at your own code and that of your colleagues, and suggesting improvements or additions to the principles below."
  • "Apply these principles to every piece of code you check in without exception."
  • "You should also take the time to improve code you are modifying or extending even if you did not write the code yourself."

...every piece of code you check in... Source code management and version control? They are a couple of steps up on many CS professors and students.

Thanks to Tyler Cowen for the link.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

June 19, 2012 3:04 PM

Basic Arithmetic, APL-Style, and Confident Problem Solvers

After writing last week about a cool array manipulation idiom, motivated by APL, I ran across another reference to "APL style" computation yesterday while catching up with weekend traffic on the Fundamentals of New Computing mailing list. And it was cool, too.

Consider the sort of multi-digit addition problem that we all spend a lot of time practicing as children:

        365
     +  366
     ------

The technique requires converting two-digit sums, such as 6 + 5 = 11 in the rightmost column, into a units digit and carrying the tens digit into the next column to the left. The process is straightforward but creates problems for many students. That's not too surprising, because there is a lot going on in a small space.

David Leibs described a technique, which he says he learned from something Kenneth Iverson wrote, that approaches the task of carrying somewhat differently. It takes advantage of the fact that a multi-digit number is a vector of digits times another vector of powers.

First, we "spread the digits out" and add them, with no concern for overflow:

        3   6   5
     +  3   6   6
     ------------
        6  12  11

Then we normalize the result by shifting carries from right to left, "in fine APL style".

        6  12  11
        6  13   1
        7   3   1

According to Leibs, Iverson believed that this two-step approach was easier for people to get right. I don't know if he had any empirical evidence for the claim, but I can imagine why it might be true. The two-step approach separates into independent operations the tasks of addition and carrying, which are conflated in the conventional approach. Programmers call this separation of concerns, and it makes software easier to get right, too.

Multiplication can be handled in a conceptually similar way. First, we compute an outer product by building a digit-by-digit times table for the digits:

     +---+---------+
     |   |  3  6  6|
     +---+---------+
     | 3 |  9 18 18|
     | 6 | 18 36 36|
     | 5 | 15 30 30|
     +---+---------+

This is straightforward, simply an application of the basic facts that students memorize when they first learn multiplication.

Then we sum the diagonals running southwest to northeast, again with no concern for carrying:

     (9) (18+18) (18+36+15) (36+30) (30)
      9      36         69      66   30

In the traditional column-based approach, we do this implicitly when we add staggered columns of digits, only we have to worry about the carries at the same time -- and now the carry digit may be something other than one!

Finally, we normalize the resulting vector right to left, just as we did for addition:

         9  36  69  66  30
         9  36  69  69   0
         9  36  75   9   0
         9  43   5   9   0
        13   3   5   9   0
     1   3   3   5   9   0

Again, the three components of the solution are separated into independent tasks, enabling the student to focus on one task at a time, using for each a single, relatively straightforward operator.

(Does this approach remind some of you of Cannon's algorithm for matrix multiplication in a two-dimensional mesh architecture?)

Of course, Iverson's APL was designed around vector operations such as these, so it includes operators that make implementing such algorithms as straightforward as the calculate-by-hand technique. Three or four Greek symbols and, voilá, you have a working program. If you are Dave Ungar, you are well on your way to a compiler!

the cover of High-Speed Math Self-Taught, by Lester Meyers

I have a great fondness for alternative ways to do arithmetic. One of the favorite things I ever got from my dad was a worn copy of Lester Meyers's High-Speed Math Self-Taught. I don't know how many hours I spent studying that book, practicing its techniques, and developing my own shortcuts. Many of these techniques have the same feel as the vector-based approaches to addition and multiplication: they seem to involve more steps, but the steps are simpler and easier to get right.

A good example of this I remember learning from High-Speed Math Self-Taught is a shortcut for finding 12.5% of a number: first multiply by 100, then divide by 8. How can a multiplication and a division be faster than a single multiplication? Well, multiplying by 100 is trivial: just add two zeros to the number, or shift the decimal point two places to the right. The division that remains involves a single-digit divisor, which is much easier than multiplying by a three-digit number in the conventional way. The three-digit number even has its own decimal point, which complicates matters further!

To this day, I use shortcuts that Meyers taught me whenever I'm making updating the balance in my checkbook register, calculating a tip in a restaurant, or doing any arithmetic that comes my way. Many people avoid such problems, but I seek them out, because I have fun playing with the numbers.

I am able to have fun in part because I don't have to worry too much about getting a wrong answer. The alternative technique allows me to work not only faster but also more accurately. Being able to work quickly and accurately is a great source of confidence. That's one reason I like the idea of teaching students alternative techniques that separate concerns and thus offer hope for making fewer mistakes. Confident students tend to learn and work faster, and they tend to enjoy learning more than students who are handcuffed by fear.

I don't know if anyone was tried teaching Iverson's APL-style basic arithmetic to children to see if it helps them learn faster or solve problems more accurately. Even if not, it is both a great demonstration of separation of concerns and a solid example of how thinking about a problem differently opens the door to a new kind of solution. That's a useful thing for programmers to learn.

~~~~

Postscript. If anyone has a pointer to a paper or book in which Iverson talks about this approach to arithmetic, I would love to hear from you.

IMAGE: the cover of Meyers's High-Speed Math Self-Taught, 1960. Source: OpenLibrary.org.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

May 29, 2012 2:44 PM

Some Final Thoughts and Links from JRubyConf

You are probably tired of hearing me go on about JRubyConf, so I'll try to wrap up with one more post. After the first few talks, the main result of the rest of conference was to introduced me to several cool projects and a few interesting quotes.

Sarah Allen speaking on agile business development

Sarah Allen gave a talk on agile business development. Wow, she has been part of creating several influential pieces of software, including AfterEffects, ShockWave, and FlashPlayer. She talked a bit about her recent work to increase diversity among programmers and reminded us that diversity is about more than the categories we usually define:

I may be female and a minority here, but I'm way more like everybody in here than everybody out there.

Increasing diversity means making programming accessible to people who wouldn't otherwise program.

Regarding agile development, Sarah reminded us that agile's preference for working code over documentation is about more than just code:

Working code means not only "passes the tests" but also "works for the customer".

... which is more about being the software they need than simply getting right answers to some tests written in JUnit.

Nate Schutta opened day two with a talk on leading technical change. Like Venkat Subramaniam on day one, Schutta suggested that tech leaders consider management's point of view when trying to introduce new technology, in particular the risks that managers face. If you can tie new technology to the organization's strategic goals and plans, then managers can integrate it better into other actions. Schutta attributed his best line to David Hussman:

Change must happen with people, not to them.

The award for the conference's most entertaining session goes to Randall Thomas and Tammer Saleh for "RUBY Y U NO GFX?", their tag-team exegesis of the history of computer graphics and where Ruby fits into that picture today. They echoed several other speakers in saying that JRuby is the bridge to the rest of the programming world that Ruby programmers need, because the Java community offers so many tools. For example, it had never occurred to me to use JRuby to connect my Ruby code to Processing, the wonderful open-source language for programming images and animations. (I first mentioned Processing here over four years ago in its original Java form, and most recently was thinking of its JavaScript implementation.)

Finally, a few quickies:

  • Jim Remsik suggested Simon Sinek's TED talk, How great leaders inspire action, with the teaser line, It's not what you do; it's why you do it.

  • Yoko Harada introduced me to Nokogiri, a parser for HTML, XML, and the like.

  • Andreas Ronge gave a talk on graph databases as a kind of NoSQL database and specifically about Neo4j.rb, his Ruby wrapper on the Java library Neo4J.

  • I learned about Square, which operates in the #fintech space being explored by the Cedar Valley's own T8 Webware and by Iowa start-up darling Dwolla.

  • rapper Jay Z
    I mentioned David Wood in yesterday's entry. He also told a great story involving rapper Jay-Z, illegal music downloads, multi-million-listener audiences, Coca Cola, logos, and video releases that encapsulated in a nutshell the new media world in which we live. It also gives a very nice example of why Jay-Z will soon be a billionaire, if he isn't already. He gets it.

  • The last talk I attended before hitting the road was by Tony Arcieri, on concurrent programming in Ruby, and in particular his concurrency framework Celluloid. It is based on the Actor model of concurrency, much like Erlang and Scala's Akka framework. Regarding these two, Arcieri said that Celluloid stays truer the original model's roots than Akka by having objects at its core and that he currently views any differences in behavior between Celluloid and Erlang as bugs in Celluloid.

One overarching theme for me of my time at JRubyConf: There is a lot of stuff I don't know. I won't run out of things to read and learn and do for a long, long, time.

~~~~

IMAGE 1: my photo of Sarah Allen during her talk on agile business development. License: Creative Commons Attribution-ShareAlike 3.0 Unported.

IMAGE 2: Jay-Z, 2011. Source: Wikimedia Commons.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

May 28, 2012 10:58 AM

The Spirit of Ruby... and of JRuby

JRubyConf was my first Ruby-specific conference, and one of the things I most enjoyed was seeing how the spirit of the language permeates the projects created by its community of users. It's one thing to read books, papers, and blog posts. It's another to see the eyes and mannerisms of the people using the language to make things they care about. Being a variant, JRuby has its own spirit. Usually it is in sync with Ruby's, but occasionally it diverges.

the letter thnad, created

The first talk after lunch was by Ian Dees, talking about his toy programming language project Thnad. (He took the name from one of the new letters of the alphabet in Dr. Seuss's On Beyond Zebra.) Thnad looks a lot like Klein, a language I created for my compiler course a few years ago, a sort of functional integer assembly language.

The Thnad project is a great example of how easy it is to roll little DSLs using Ruby and other DSLs created in it. To implement Thnad, Dees uses Parslet, a small library for generating scanners and parsers PEG-style, and BiteScript, a Ruby DSL for generating Java bytecode and classes. This talk demonstrated the process of porting Thnad from JRuby to Rubinius, a Ruby implementation written in Ruby. (One of the cool things I learned about the Rubinius compiler is that it can produce s-expressions as output, using the switch -S.)

Two other talks exposed basic tenets of the Ruby philosophy and the ways in which implementations such as JRuby and Rubinius create new value in the ecosystem.

On Wednesday afternoon, David Wood described how his company, the Jun Group, used JRuby to achieve the level of performance its on-line application requires. He told some neat stories about the evolution of on-line media over the last 15-20 years and how our technical understanding for implementing such systems has evolved in tandem. Perhaps his most effective line was this lesson learned along the way, which recalled an idea from the keynote address the previous morning:

Languages don't scale. Architectures do. But language and platform affect architecture.

In particular, after years of chafing, he had finally reached peace with one of the overarching themes of Ruby: optimize for developer ease and enjoyment, rather than performance or scalability. This is true of the language and of most of the tools built around, such as Rails. As a result, Ruby makes it easy to write many apps quickly. Wood stopped fighting the lack of emphasis on performance and scalability when he realized that most apps don't succeed anyway. If one does, you have to rewrite it anyway, so suck it up and do it. You will have benefited from Ruby's speed of delivery.

This is the story Twitter, apparently, and what Wood's team did. They spent three person-months to port their app from MRI to JRuby, and are now quite happy.

Where does some of that performance bump come from? Concurrency. Joe Kutner gave a talk after Thnad on Tuesday afternoon about using JRuby to deploy efficient Ruby web apps on the JVM, in which he also exposed a strand of Ruby philosophy and place where JRuby diverges.

The canonical implementations of Ruby and Python use a Global Interpreter Lock to ensure that non-thread-safe code does not interfere with the code in other threads. In effect, the interpreter maps all threads onto a single thread in the kernel. This may seem like an unnecessary limitation, but it is consistent with Matz's philosophy for Ruby: Programming should be fun and easy. Concurrency is hard, so don't do allow it to interfere with the programmer's experience.

Again, this works just fine for many applications, so it's a reasonable default position for the language. But it does not work so well for web apps, which can't scale if they can't spawn new, independent threads. This is a place where JRuby offers a big win by running atop the JVM, with its support for multithreading. It's also a reason why the Kilim fibers GSoC project mentioned by Charles Nutter in the State of JRuby session is so valuable.

In this talk, I learned about three different approaches to delivering Ruby apps on the JVM:

  • Warbler, a light and simple tool for packaging .war files,
  • Trinidad, which is a JRuby wrapper for a Tomcat server, and
  • TorqueBox, an all-in-one app server that appears to be the hot new thing.

Links, links, and more links!

Talks such as these reminded me of the feeling of ease and power that Ruby gives developers, and the power that language implementors have to shape the landscape in which programmers work. They also gave me a much better appreciation for why projects like Rubinius and JRuby are essential to the Ruby world because -- not despite -- deviating from a core principle of the language.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

May 25, 2012 4:07 PM

JRubyConf, Day 1: The State of JRuby

Immediately after the keynote address, the conference really began for me. As a newcomer to JRuby, this was my first chance to hear lead developers Charles Nutter and Tom Enebo talk about the language and community. The program listed this session as "JRuby: Full of Surprises", and Nutter opened with a slide titled "Something About JRuby", but I just thought of the session as a "state of the language" report.

Nutter opened with some news. First, JRuby 1.7.0.preview1 is available. The most important part of this for me is that Ruby 1.9.3 is now the default language mode for the interpreter. I still run Ruby 1.8.7 on my Macs, because I have never really needed more and that kept my installations simple. It will be nice to have a 1.9.3 interpreter running, too, for cases where I want to try out some of the new goodness that 1.9 offers.

Second, JRuby has been awarded eight Google Summer of Code placements for 2012. This was noteworthy because there were no Ruby projects at all in 2010 or 2011, for different reasons. Several of the 2012 projects are worth paying attention to:

  • creating a code generator for Dalvik byte code, which will give native support for JRuby on Android
  • more work on Ruboto, the current way to run Ruby on Android, via Java
  • implementing JRuby fibers using Kilim fibers, for lighterweight and faster concurrency than Java threads can provide
  • work on krypt, "SSL done right" for Ruby, which will eliminate the existing dependence on OpenSSL
  • filling in some of the gaps in the graphics framework Shoes, both Swing and SWT versions

Charles Nutter discussing details of the JRuby compiler

Enebo then described several projects going on with JRuby. Some are smaller, including closing gaps in the API for embedding Ruby code in Java, and Noridoc, a tool for generating integrated Ruby documentation for Ruby and Java APIs that work together. Clever -- "No ri doc".

One project is major: work on the JRuby compiler itself. This includes improving to the intermediate representation used by JRuby, separating more cleanly the stages of the compiler, and writing a new, better run-time system. I didn't realize until this talk just how much overlap there is in the existing scanner, parser, analyzer, and code generator of JRuby. I plan to study the JRuby compiler in some detail this summer, so this talk whet my appetite. One of the big challenges facing the JRuby team is to identify execution hot spots that will allow the compiler to do a better job of caching, inlining, and optimizing byte codes.

This led naturally into Nutter's discussion of the other major project going on: JRuby support for the JVM's new invokedynamic instruction. He hailed invokedynamic as "the most important change to the JVM -- ever". Without it, JRuby's method invocation logic is opaque to the JVM optimizer, including caching and inlining. With it, the JRuby compiler will be able not only to generate optimizable function calls but also more efficient treatment of instance variables and constants.

Charles Nutter donning his new RedHat

Nutter showed some performance data comparing JRuby to MRI Ruby 1.9.3 on some standard benchmarks. Running on Java 6, JRuby is between 1.3 and 1.9 times faster than the C-based compiler on the benchmark suites. When they run it on Java 7, performance jumps to speed-ups of between 2.6 and 4.3. That kind of speed is enough to make JRuby attractive for many compute-intensive applications.

Just as Nutter opened with news, he closed with news. He and Enebo are moving to RedHat. They will work with various RedHat and JBoss teams, including TorqueBox, which I'll mention in an upcoming JRubyConf post. Nutter and Enebo have been at EngineYard for three years, following a three-year stint at Sun. It is good to know that, as the corporate climate around Java and Ruby evolves, there is usually a company willing to support open-source JRuby development.

~~~~

IMAGE 1: my photo of Charles Nutter talking about some details of the JRuby compiler. License: Creative Commons Attribution-ShareAlike 3.0 Unported.

IMAGE 2: my photo of Charles Nutter and Tom Enebo announcing their move to RedHat. License: Creative Commons Attribution-ShareAlike 3.0 Unported.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

May 24, 2012 3:05 PM

JRubyConf 2012: Keynote Address on Polyglot Programming

JRubyConf opened with a keynote address on polyglot programming by Venkat Subramaniam. JRuby is itself a polyglot platform, serving as a nexus between a highly dynamic scripting language and a popular enterprise language. Java is not simply a language but an ecosphere consisting of language, virtual machine, libraries, and tools. For many programmers, the Java language is the weakest link in its own ecosphere, which is one reason we see so many people porting their favorite languages run on the JVM, or creating new languages with the JVM as a native backend.

Subramaniam began his talk by extolling the overarching benefits of being able to program in many languages. Knowing multiple programming languages changes how we design software in any language. It changes how we think about solutions. Most important, it changes how we perceive the world. This is something that monolingual programmers often do not appreciate. When we know several languages well, we see problems -- and solutions -- differently.

Why learn a new language now, even if you don't need to? So that you can learn a new language more quickly later, when you do need to. Subramaniam claimed that the amount of time required to learn a new language is inversely proportional to the number of languages a person has learned in last ten years. I'm not sure whether there is any empirical evidence to support this claim, but I agree with the sentiment. I'd offer one small refinement: The greatest benefits come from learning different kinds of language. A new language that doesn't stretch your mind won't stretch your mind.

Not everything is heaven for the polyglot programmer. Subramaniam also offered some advice for dealing with the inevitable downsides. Most notable among these was the need to "contend with the enterprise".

Many companies like to standardize on a familiar and well-established technology stack. Introducing a new language into the mix raises questions and creates resistance. Subramaniam suggested that we back up one step before trying to convince our managers to support a polyglot environment and make sure that we have convinced ourselves. If you were really convinced of a language's value, you would find a way to do it. Then, when it comes time to convince your manager, be sure to think about the issue from her perspective. Make sure that your argument speaks to management's concerns. Identify the problem. Explain the proposed solution. Outline the costs of the solution. Outline its benefits. Show how the move can be a net win for the organization.

The nouveau JVM languages begin with a head start over other new technologies because of their interoperability with the rest of the Java ecosphere. They enable you to write programs in a new language or style without having to move anyone else in the organization. You can experiment with new technology while continuing to use the rest of the organization's toolset. If the experiments succeed, managers can have hard evidence about what works well and what doesn't before making larger changes to the development environment.

I can see why Subramaniam is a popular conference speaker. He uses fun language and coins fun terms. When talking about people who are skeptical of unit testing, he referred to some processes as Jesus-driven development. He admonished programmers who are trying to do concurrency in JRuby without knowing the Java memory model, because "If you don't understand the Java memory model, you can't get concurrency right." But he followed that immediately with, Of course, even if you do know the Java memory model, you can't get concurrency right. Finally, my favorite: At one point, he talked about how some Java developers are convinced that they can do anything they need to do in Java, with no other languages. He smiled and said, I admire Java programmers. They share an unrelenting hope.

There were times, though, when I found myself wanting to defend Java. That happens to me a lot when I hear talks like this one, because so many complaints about it are really about OOP practiced poorly; Java is almost an innocent bystander. For example, the speaker chided Java programmers for suffering from primitive obsession. This made me laugh, because most Ruby programmers seem to have an unhealthy predilection for strings, hashes, and integers.

In other cases, Subramaniam demonstrated the virtues of Ruby by showing a program that required a gem and then solved a thorny problem with three lines of code. Um, I could do that in Java, too, if I used the right library. And Ruby programmers probably don't want to draw to much attention to gems and the problems many Ruby devs have with dependency management.

But those are small issues. Over the next two days, I repeatedly heard echoes of Subramaniam's address in the conference's other talks. This is the sign of a successful keynote.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

May 22, 2012 7:53 PM

A Few Days at JRubyConf

It's been fourteen months since I last attended a conference. I decided to celebrate the end of the year, the end of my compiler course, and the prospect of writing a little code this summer by attending JRubyConf 2012. I've programmed a fair amount in Ruby but have only recently begun to play with JRuby, an implementation of Ruby in Java which runs atop the JVM. There are some nice advantages to this, including the ability to use Java graphics with Ruby models and the ability to do real concurrency. It also offers me a nice combination for the summer. I will be teaching our sophomore-level intermediate computing course this fall, which focuses in large part on OO design and Java implementation, as JRuby will let me program in Ruby while doing a little class prep at the same time.

the Stone Arch Bridge in Minneapolis

Conference organizer Nick Sieger opened the event with the obligatory welcome remarks. He said that he thinks the overriding theme of JRubyConf is being a bridge. This is perhaps a natural effect of Minneapolis, a city of many bridges, as the hometown of JRuby, its lead devs, and the conference. The image above is of the Stone Arch Bridge, as seen from the ninth level of the famed Guthrie Center, the conference venue. (The yellow tint is from the window itself.)

The goal for the conference is to be a bridge connecting people to technologies. But it also aims to be a bridge among people, promoting what Sieger called "a more sensitive way of doing business". Emblematic of this goal were its Sunday workshop, a Kids CodeCamp, and its Monday workshop, Railsbridge. This is my first open-source conference, and when I look around I see the issue that so many people talk about. Of 150 or so attendees, there must be fewer than one dozen women and fewer than five African-Americans. The computing world certainly has room to make more and better connections into the world.

My next few entries will cover some of the things I learn at the conference. I start with a smile on my face, because the conference organizers gave me a cookie when I checked in this morning:

the sugar cookie JRubyConf gave me at check-in

That seems like a nice way to say 'hello' to a newcomer.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

May 14, 2012 3:26 PM

Lessons Learned from Another Iteration of the Compiler Course

I am putting the wrap on spring semester, so that I can get down to summer duties and prep for fall teaching. Here are a few lessons I learned this spring.

•  A while back, I wrote briefly about re-learning the value of a small source language for the course. If I want to add a construct or concept, then I need also to subtract a corresponding load from language. In order to include imperative features, I need to eliminate recursive functions, or perhaps eliminate functions altogether. Eliminating recursion may be sufficient, as branching to a function is not much more complex than branching in loops. It is the stack of activation records that seems to slow down most students.

•  Using a free on-line textbook worked out okay. The main problem was that this particular book contained less implementation detail than books we have used in the past, such as Louden, and that hurt the students. We used Louden's TM assembly language and simulator, and the support I gave them for that stage of the compiler in particular was insufficient. The VM and assembly language themselves are simple enough, but students wanted more detailed examples of a code generator than I gave them.

•  I need to teach code generation better. I felt that way as the end of the course approached, and then several students suggested as much in our final session review. This is the most salient lesson I take from this iteration of the course.

I'm not sure at this moment if I need only to do a better job of explaining the process or if I need a different approach to the task more generally. That's something I'll need to think about between now and next time. I do think that I need to show them how to implement function calls in a bit more detail. Perhaps we could spend more time in class with statically-allocated activation records, and then let the students extend those ideas for a run-time stack and recursion.

•  For the first time ever, a few students suggested that I require something simpler than a table-driven parser. Of course, I can address several issues with parsing and code generation by using scaffolding: parser generators, code-generation frameworks and the like. But I still prefer that students write a compiler from scratch, even if only a modest one. There is something powerful in making something from scratch. A table-driven parser is a nice blend of simplicity (in algorithm) and complexity (in data) for learning how compilers really work.

I realize that I have to draw the abstraction line somewhere, and even after several offerings of the course I'm ready to draw it there. To make that work as well as possible, I may have to improve parts of the course to make it work better.

•  Another student suggestion that seems spot-on is that, as we learn each stage of the compiler, we take some time to focus on specific design decisions that the teams will have to make. This will alway them, as they said in their write-ups, "to make informed decisions". I do try to introduce key implementation decisions that they face and offer advice on how to proceed. Clearly I can do better. One way, I think, is to connect more directly with the programming styles they are working in.

~~~~

As usual, the students recognized some of the same shortcomings of the course that I noticed and suggested a couple more that had not occurred to me. I'm always glad I ask for their feedback, both open and anonymous. They are an indispensable source of information about the course.

Writing your first compiler is a big challenge. I can't help but recall something writer Samuel Delany said when asked if he "if it was fun" to write a set of novellas on the order of Eliot's The Waste Land, Pound's The Cantos, and Joyce's Ulysses:

No, not at all. ... But at least now, when somebody asks, "I wonder if Joyce could have done all the things he's supposed to have done in Ulysses," I can answer, "Yes, he could have. I know, because I tried it myself. It's possible."

Whatever other virtues there are in learning to write a compiler, it is valuable for computer science students to take on big challenges and know that it is possible to meet the challenge, because they have tried it themselves.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

May 01, 2012 8:56 AM

The Pleasure in a Good Name

Guile is a Scheme. It began life as GEL, which stood for GNU Extension Language. This brief history of Guile explains the change:

Due to a naming conflict with another programming language, Jim Blandy suggested a new name for GEL: "Guile". Besides being a recursive acronym, "Guile" craftily follows the naming of its ancestors, "Planner", "Conniver", and "Schemer". (The latter was truncated to "Scheme" due to a 6-character file name limit on an old operating system.) Finally, "Guile" suggests "guy-ell", or "Guy L. Steele", who, together with Gerald Sussman, originally discovered Scheme.

That is how you name a language, making it significant on at least three levels. Recursive acronyms are a staple of the GNU world, beginning with GNU itself. Guile recurses as the GNU Ubiquitous Intelligent Language for Extensions. Synonymny with Planner and Conniver keeps alive the historical connection to artificial intelligence, and is reinforced by the use of intelligent in the acronym. Finally, the homophonic connection to Steele is pure genius.

I bow to you, Mr. Blandy.

(While we are talking about words, I must say that I love the author's use of discovered in the passage quoted above. Most people say that Steele and Sussman "created" Scheme, or "designed" it, or "invented" it. But if you read Steele's and Gabriel's The Evolution of Lisp, you will see that discovery is a better label for what happened. Scheme was lurking in the implementation of Lisp's apply primitive and Carl Hewitt's theory of actors. Hewitt, of course, created Planner, which is another connection back to Guile.)


Posted by Eugene Wallingford | Permalink | Categories: Computing

April 24, 2012 4:55 PM

Recursive Discussions about Recursion

The SIGCSE listserv has erupted today with its seemingly annual discussion of teaching recursion. I wrote about one of the previous discussions a couple of years ago. This year's conversation has actually included a couple of nice ideas, so it was worth following along.

Along the way, one prof commented on an approach he has seen used to introduce students to recursion, often in a data structures course. First you cover factorial, gcd, and the Fibonacci sequence. Then you cover the Towers of Hanoi and binary search. Unfortunately, such an approach is all too common. The poster's wistful analysis:

Of the five problems, only one (binary search) is a problem students might actually want to solve. Only two (Fibonacci and Hanoi) are substantially clearer in recursive than iterative form, and both of them take exponential time. In other words, recursion is a confusing way to solve problems you don't care about, extremely slowly.

Which, frankly, I think is the lesson [some CS profs] want to convey.

And this on a day when I talked with my compiler students about how a compiler can transform many recursive programs into iterative ones, and even eliminate the cost of a non-recursive function call when it is in a tail position.

The quoted passage contains my favorite line of the week thus far: In other words, recursion is a confusing way to solve problems you don't care about, extremely slowly. If that's not the message you want to convey to your students, then please don't introduce them to recursion in this way. If that is the message you want to send to your students, then I am sad for you, but mostly for your students.

I sometimes wonder about the experiences that some computer scientists bring to the classroom. It only takes a little language processing to grok the value of recursion. And a data structures course is a perfectly good place for students to see and do a little language processing. Syntax, abstract or not, is a great computer science example of trees. And students get to learn a little more CS to boot!


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

April 15, 2012 8:10 PM

Learning From the Wheel of Reincarnation

Last week I was pointed in the direction of a forty-five old CACM paper. It had an innocuous title, On the Design of Display Processors, and was outside my areas of primary interest, so I might well have passed it by. But it was co-written by Ivan Sutherland, whose work Alan Kay has praised often, and it was recommended by someone on Kay's the Fundamentals of New Computing mailing list, where a lot of neat ideas are discussed. So I printed it out for my daily exercise bike ride. I'm glad I did.

Myer and Sutherland tell the story of needing a display system for a research computer. That was a bigger deal in 1967 than it is today, so they did some research of their own....

Finally we decided to design the processor ourselves, because only in this way, we thought, could we obtain a truly complete display processor. We approached the task by starting with a simple scheme and adding commands and features that we felt would enhance the power of the machine. Gradually the processor became more complex. We were not disturbed by this because computer graphics, after all, are complex. Finally the display processor came to resemble a full-fledged computer with some special graphics features. And then a strange thing happened. We felt compelled to add to the processor a second, subsidiary processor, which, itself, began to grow in complexity.

It was then that we discovered a disturbing truth. Designing a display processor can become a never-ending cyclical process. In fact, we found the process so frustrating that we have come to call it the "wheel of reincarnation." We spent a long time trapped on that wheel before we finally broke free. In the remainder of this paper we describe our experiences. We have written it in the hope that it may speed others on toward "Nirvana."

A mantra from the paper characterizes the authors' time on the wheel: "For just a little more money...". I'll bet that sounds familiar to a lot of researchers, not to mention all of us who buy computing equipment for labs and end users.

I was really happy to read this paper. It's an experience report in which the authors share honestly the mistakes they made. But they paid attention, recognized a pattern, and learned from it. Even better, they wrote what they learned, in hopes of teaching the rest of us.

The wheel of reincarnation is not limited to display design or hardware design. It occurs any place where we encounter complexity. We try to tame it, first by specializing and then by generalizing. The design of programming languages is just as prone to dizzying cycle.

(In software design, we have a related phenomenon, captured in Greenspun's Tenth Rule.)

In language design, we almost have to look for a fixed point at which we stabilize the pendulum between general and specialized. What we most often need as users is the ability to grow systems gracefully over time. This speaks to the value of a good macro system and good compiler design.

Reading this paper reminded me of a couple of lessons I've learned over the years:

  1. I should read and watch everything I can get my hands on from Sutherland, Robert Floyd, and other computer scientists of their generation. They were solving hard problems long ago, in the face of resource limitations few of us can imagine today.
  2. As someone tweeted recently, @fogus, I think: reading these old papers makes me think that I will never have an original idea in your life. But then they also teach me a lot and prepare me to have more and sometimes better ideas.
  3. I need to do my best to hang around with smart, curious people. It's old advice, I know, but it requires action and, in some environments, constant vigilance. Simply eavesdropping on the FONC mailing list raises the level of my brain's activity by a few degrees.

These papers also remind us of a valuable role that academics can play in the software and computing worlds, which are also heavily influenced by industry practitioners. We need to keep papers like these alive, so that the smart and curious people in our classes and in industry will read them. We never know when two ideas will crash into each other and lead to something new and better.


Posted by Eugene Wallingford | Permalink | Categories: Computing

April 11, 2012 4:06 PM

What Penn and Teller Have in Common With a Compilers Course

Early this morning (and I mean early!), Alfred Thompson posted What Do Magic and Computer Science Have in Common?, relaying that Alex Suter of Industrial Light & Magic will give the closing keynote at this summer's Computer Science and Information Technology conference. That sounds pretty cool. The title of his entry conjured up other thoughts for me, though, especially in light of something I said in class yesterday.

Recently, I used a superhero reference in a blog entry. That is how many people feel when they use a program to accomplish something meaningful -- like a superhero. I feel that way, too, sometimes. However, like many other people, I am more prone to magical imagery. To someone who has not learned to code, a program is like an incantation, capable of making the computer do something mystical.

There is a long tradition of magical imagery in computer science. The Hacker's Dictionary tells us that a wizard is someone who knows how a complex piece of software or hardware works (that is, who groks it). A wizard can do things that hackers and mere mortals cannot. The entry for "wizard" has links to other magical jargon, such as heavy wizardry incantation>/A> and magic itself.

Structure and Interpretation of Computer Programs

I tell my Programming Languages students that this is a course in which the "high priests" of computer science reveal their secrets, that after the course students will understand the magic embodied in the interpreters and compilers that process their programs. I should probably refer to wizards, rather high priests, given that so many of the course's ideas are covered masterfully in SICP.

Lots of CS courses reveal magic, or propose it. A course in compliers finishes the job of Programming Languages, driving program translation all the way down to hardware. Artificial Intelligence describes our best ideas for how to make a computer do human-like magic: reasoning, recognizing patterns, learning, and playing Jeopardy!.

In my compliers class yesterday, I was showing my students a technique for generating three-address code from an abstract syntax tree, based in large part on ideas found in the Dragon book (surely not a coincidence). I wrote on the board this template for a grammar rule denoting addition:

    E → E1 + E2

E.place := makeNewTemporaryIdentifier() E.code := [ E1.code ] [ E2.code ] emitCode( E.place " := " E1.place " + " E2.place )

When I finished, I stepped back, looked it over, and realized again just how un-magical that seems. Indeed, when in written in black on white, it looks pretty pedestrian.

the magician due of Penn and Teller

That made me think of another connection between magic and computer science, one that applies to practitioners and outsiders alike. Taking an AI course or a compilers course is like having Penn and Teller explain to you how they made a person disappear or a ball levitate in thin air. For some people, that kills any joy that might have in watching the act. They don't want to know. They want to be amazed. And, knowing that something is implemented -- often in a way that doesn't seem especially artful, performed with an obvious misdirection -- prevents them from being amazed.

That can happen in CS, too. My friends and I came in to our AI course wide-eyed and wanting to be amazed -- and to build amazing things. We studied search and logical resolution, Bayes' Theorem and decision tree induction. And it all looked so... pedestrian. Many of my friends lost their sense of wonder then and there. Without the magic of AI, they were just as interested in operating systems or databases. More interested, really, because AI had let them down. It all looked like parlor tricks.

But there is a second kind of person in the world. Lots of people love to watch Penn and Teller explain a trick. They want to know how it works. They want to watch again, knowing how it works, to see if they can notice the deception. If they don't, they are amazed again. If they do, though, they still feel wonder -- at the skill of the magicians, at the design of the illusion, and even at the way their mind wants to trick them at the moment of execution.

I am of this second kind, for magic and especially for computer science. Yes, I know that compilers and AI programs are, at their cores, implemented using techniques that don't always look all that impressive in the light of day. Sometimes, those techniques look pretty boring indeed.

Yet I am still amazed when a C compiler takes in my humble instructions and creates machine code that compresses a musical file or fills out my tax return. I am amazed when Watson crunches through gazillions of bytes of data in a second or two and beats two most worthy human opponents to the buzzer. I like to hear my friends and colleagues de-mystify their current projects with nitty-gritty details. I still feel wonder -- at their skill, at the cleverness of their designs, and even at the moment the program runs and makes something out of what seems like nothing.

That's one thing magic and computer science have in common.

~~~~

IMAGE 1: a photo of the cover of Structure and Interpretation of Computer Programs. Source: the book's website.

IMAGE 2: a publicity still of magicians Penn and Teller. Source: All-About-Magicians.com.


Posted by Eugene Wallingford | Permalink | Categories: Computing

April 06, 2012 4:29 PM

A Reflection on Alan Turing, Representation, and Universal Machines

Douglas Hofstadter speaking at UNI

The day after Douglas Hofstadter spoke here on assertions, proof's and Gödel's theorem, he gave a second public lecture hosted by the philosophy department. Ahead of time, we knew only that Hofstadter would reflect on Turing during his centennial. I went in expecting more on the Turing test, or perhaps a popular talk on Turing's proof of The Halting Problem. Instead, he riffed on Chapter 17 from I Am a Strange Loop.

In the end, we are self-perceiving, self-inventing, locked-in mirages that are little miracles of self-reference.

Turing, he said, is another peak in the landscape occupied by Tarski and Gödel, whose work he had discussed the night before. (As a computer scientist, I wanted to add to this set contemporaries such as Alonzo Church and Claude Shannon.) Hofstadter mentioned Turing's seminal paper about the Entscheidungsproblem but wanted to focus instead on the model of computation for which he is known, usually referred to by the name "Turing machine". In particular, he asked us to consider a key distinction that Turing made when talking about his model: that between dedicated and universal machines.

A dedicated machine performs one task. Human history is replete with dedicated machines, whether simple, like the wheel, or complex, such as a typewriter. We can use these tools with different ends in mind, but the basic work is fixed in their substance and structure.

The 21st-century cell phone is, in contrast, a universal machine. It can take pictures, record audio, and -- yes -- even be used as a phone. But it can also do other things for us, if we but go to the app store and download another program.

Hofstadter shared a few of his early personal experiences with programs enabling line printers to perform tasks for which they had not been specifically designed. He recalled seeing a two-dimensional graph plotted by "printing" mostly blank lines that contained a single *. Text had been turned into graphics. Taking the idea further, someone used the computer to print a large number of cards which, when given to members of the crowd at a football game, could be used to create a massive two-dimensional message visible from afar. Even further, someone used a very specific layout of the characters available on the line printer to produce a print-out that appeared from the other side of the room to be a black-and-white photograph of Raquel Welch. Text had been turned into image.

People saw each of these displays as images by virtue of our eyes and mind interpreting a specific configuration of characters in a certain way. We can take that idea down a level into the computer itself. Consider this transformation of bits:

0000 0000 0110 1011 → 0110 1011 0000 0000

A computer engineer might see this as a "left shift" of 8 bits. A computer programmer might see it as multiplying the number on the left by 256. A graphic designer might see us moving color from one pixel to another. A typesetter may see one letter being changed into another. What one sees depends on how one interprets what the data represent and what the process means.

Alan Turing was the first to express clearly the idea that a machine can do them all.

"Aren't those really binary numbers?", someone asked. "Isn't that real, and everything else interpretation?" Hofstadter said that this is a tempting perspective, but we need to keep in mind that they aren't numbers at all. They are, in most computers, pulses of electricity, or the states of electronic components, that we interpret as 0s and 1s.

After we have settled on interpreting those pulses or states as 0s and 1s, we then interpret configurations of 0s and 1s to mean something else, such as decimal numbers, colors, or characters. This second level of interpretation exposes the flaw in popular claims that computers can do "only" process 0s and 1s. Computers can deal with numbers, colors, or characters -- anything that can be represented in any way -- when we interpret not only what the data mean but also what the process means.

(In the course of talking representations, he threw in a cool numeric example: Given an integer N, factor it as 2^a * 3^b * 5^c *7^d ... and use [a.b.c.d. ...] to stand for N. I see a programming assignment or two lying in wait.)

The dual ideas of representation and interpretation take us into a new dimension. The Principia Mathematica describes a set of axioms and formal rules for reasoning about numeric structures. Gödel saw that it could be viewed at a higher level, as a system in its own right -- as a structure of integers. Thus the Principia can talk about itself. It is, in a sense, universal.

This is the launching point for Turing's greatest insight. In I Am a Strange Loop, Hofstadter writes:

Inspired by Gödel's mapping of PM into itself, Alan Turing realized that the critical threshold for this kind of computational universality comes exactly at the point where a machine is flexible enough to read and correctly interpret a set of data that describes its own structure. At this crucial juncture, a machine can, in principle, explicitly watch how it does any particular task, step by step. Turing realized that a machine that has this critical level of flexibility can imitate any other machine, no matter how complex the latter is. In other words, there is nothing more flexible than a universal machine. Universality is as far as you can go!

Alan Turing

Thus was Turing first person to recognize the idea of a universal machine, circa 1935-1936: that a Turing machine can be given, as input, data that encodes its own instructions. This is the beginning of perhaps the biggest of the Big Ideas of computer science: the duality of data and program.

We should all be glad he didn't patent this idea.

Turing didn't stop there, of course, as I wrote in my recent entry on the Turing test. He recognized that humans are remarkably capable and efficient representational machines.

Hofstadter illustrates this with the idea of "hub", a three-letter word that embodies an enormous amount of experience and knowledge, chunked in numerous ways and accreted slowly over time. The concept is assembled in our minds out of our experiences. It is a representation. Bound up in that representation is an understanding of ourselves as actors in certain kinds of interactions, such as booking a flight on an airplane.

It is this facility with representations that distinguishes us humans from dogs and other animals. They don't seem capable of seeing themselves or others as representations. Human beings, though, naturally take other people's representations into their own. This results in a range of familiarities and verisimilitude. We "absorb" some people so well that we feel we know them intimately. This is what we mean when we say that someone is "in our soul". We use the word 'soul' not in a religious sense; we are referring to our essence.

Viewed this way, we are all distributed beings. We are "out there", in other people, as well as "in here", in ourselves. We've all had dreams of the sort Hofstadter used as example, a dream in which his deceased father appeared, seemingly as real as he ever had been while alive. I myself recently dreamt that I was running, and the experience of myself was as real as anything I feel when I'm awake. Because we are universal machines, we are able to process the representations we hold of ourselves and of others and create sensations that feel just like the ones we have when we interact in the world.

It is this sense that we are self-representation machines that gives rise to the title of his book, "I am a strange loop". In Hofstadter's view, our identity is a representation of self that we construct, like any other representation.

This idea underlies the importance of the Turing test. It takes more than "just syntax" to pass the test. Indeed, syntax is itself more than "just" syntax! We quickly recurse into the dimension of representation, of models, and a need for self-reference that makes our syntactic rules more than "just" rules.

Indeed, as self-representation machines, we are able to have a sense of our own smallness within the larger system. This can be scary, but also good. It makes life seem precious, so we feel a need to contribute to the world, to matter somehow.

Whenever I teach our AI course, I encounter students who are, for religious or philosophical reasons, deeply averse to the idea of an intelligent machine, or even of scientific explanations of who we are. When I think about identity in terms of self-representation, I can't help but feel that, at an important level, it does not matter. God or not, I am in awe of who we are and how we got to here.

So, we owe Alan Turing a great debt. Building on the work of philosophers, mathematicians, and logicians, Turing gave us the essential insight of the universal machine, on which modern computing is built. He also gave us a new vocabulary with which to think about our identity and how we understand the world. I hope you can appreciate why celebrating his centennial is worthwhile.

~~~~

IMAGE 1: a photo of Douglas Hofstadter speaking at UNI, March 7, 2012. Source: Kevin C. O'Kane.

IMAGE 2: the Alan Turing centenary celebration. Source: 2012 The Alan Turing Year.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

April 04, 2012 4:39 PM

Computational Search Answers an Important Question

Update: Well, this is embarrassing. Apparently, Mat and I were the victims of a prank by the folks at ChessBase. You'd think that, after more than twenty-five years on the internet, I would be more circumspect at this time of year. Rather than delete the post, I will leave it here for the sake of posterity. If nothing else, my students can get a chuckle from their professor getting caught red-faced.

I stand behind my discussion of solving games, my recommendation of Rybka, and my praise for My 60 Memorable Games (my favorite chess book of all time. I also still marvel at the chess mind of Bobby Fischer.

~~~~

Thanks to reader Mat Roberts for pointing me to this interview with programmer Vasik Rajlich, which describes a recent computational result of his: one of the most famous openings in chess, the King's Gambit, is a forced draw.

Games are, of course, a fertile testbed for computing research, including AI and parallel computation. Many researchers make one of their goals to "solve" a game, that is, to show that, with best play by both players, a game has a particular outcome. Games with long histories and large communities of players naturally attract a lot of interest, and solving one of them is usually considered a valuable achievement.

For us in CS, interest grows as with the complexity of the game. Solving Connect Four was cool, but solving Othello on a full-sized board would be cooler. Almost five years ago, I blogged about what I still consider the most impressive result in this domain: the solving of checkers by Jonathan Schaeffer and his team at the University of Alberta.

the King's Gambit

The chess result is more limited. Rajlich, an International Master of chess and the programmer of the chess engine Rybka, has shown results only for games that begin 1.e4 e5 2.f4 exf4. If White plays 3.Nf3 -- the most common next move -- then Black can win with 3... d6. 3.Bc4 also loses. Only one move for White can force a draw, the uncommon 3.Be2. Keep in mind that these results all assume best play by both players from there on out. White can win, lose, or draw in all variations if either player plays a sub-optimal move.

I say "only" when describing this result because it leaves a lot of chess unsolved, all games starting with some other sequence of moves. Yet the accomplishment is still quite impressive! The King's Gambit is one of the oldest and most storied opening sequences in all of chess, and it remains popular to this day among players at every level of skill.

Besides, consider the computational resources that Rajlich had to use to solve even the King's Gambit:

... a cluster of computers, currently around 300 cores [created by Lukas Cimiotti, hooked up to] a massively parallel cluster of IBM POWER 7 Servers provided by David Slate, senior manager of IBM's Semantic Analysis and Integration department -- 2,880 cores at 4.25 GHz, 16 terabytes of RAM, very similar to the hardware used by IBM's Watson in winning the TV show "Jeopardy". The IBM servers ran a port of the latest version of Rybka, and computation was split across the two clusters, with the Cimiotti cluster distributing the search to the IBM hardware.

Oh, and this set up had to run for over four months to solve the opening. I call that impressive. If you want something less computationally intensive yet still able to beat you me and everybody we know at chess, you can by Rybka, a chess engine available commercially. (An older version is available for free!)

What effect will this result have on human play? Not much, practically speaking. Our brains aren't big enough or fast enough to compute all the possible paths, so human players will continue to play the opening, create new ideas, and explore the action in real time over the board. Maybe players with the Black pieces will be more likely to play one of the known winning moves now, but results will remain uneven between White and Black. The opening leads to complicated positions.

the cover of Bobby Fischer's 'My 60 Memorable Games'

If, like some people, you worry that results such as this one somehow diminish us as human beings, take a look again at the computational resources that were required to solve this sliver of one game, the merest sliver of human life, and then consider: This is not the first time that someone claimed the King's Gambit busted. In 1961, an eighteen-year-old U.S. chess champion named Bobby Fischer published an article claiming that 1.e4 e5 2.f4 exf4 3.Nf3 was a forced loss. His prescription? 3... d6. Now we know for sure. Like so many advances in AI, this one leaves me marveling at the power of the human mind.

Well, at least Bobby Fischer's mind.

~~~~

IMAGE 1: The King's Gambit. Source: Wikimedia Commons.

IMAGE 2: a photograph of the cover of my copy of My 60 Memorable Games by Bobby Fischer. Bobby analyzes a King's Gambit or two in this classic collection of games.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

April 03, 2012 4:00 PM

Intermediate Representations and Life Beyond the Compiler

In the simplest cases, a compiler can generate target code directly from the abstract syntax tree:

generating target code directly from the abstract syntax tree

In many cases, though, there are good reasons why we don't want to generate code for the target machine immediately. One is modularity. A big part of code generation for a particular target machine is machine-dependent. If we write a monolithic code generator, then we will have to reimplement the machine-independent parts every time we want to target a new machine.

Even if we stick with one back-end architecture, modularity helps us. Not all of the machine-dependent elements of code generation depend in the same way on the machine. If we write a monolithic code generator, then any small change in the target machine -- perhaps even a small upgrade in the processor -- can cause changes throughout our program. If instead we write a modular code generator, with modules that reflect particular shear layers in the generation process, a lá How Buildings Learn, then we may be able to contain changes in target machine specification to an easily identified subset of modules.

So, more generally we think of code generation in two parts:

  • one or more machine-independent transformations from an abstract syntax tree to intermediate representations of the program, followed by

  • one or more machine-dependent transformations from the final intermediate representation to machine code.

generating target code directly from the abstract syntax tree

Intermediate representations between the abstract syntax tree and assembly code have other advantages, too. In particular, they enable us to optimize code in machine-independent ways, without having to manipulate a complex target language.

In practice, an intermediate representation sometimes outlives the compiler for which it was created. Chris Clark described an example of this phenomenon in Build a Tree--Save a Parse:

Sometimes the intermediate language (IL) takes on a life of its own. Several systems that have multiple parsers, sophisticated optimizers, and multiple code generators have been developed and marketed commercially. Each of these systems has its own common virtual assembly language used by the various parsers and code generators. These intermediate languages all began connecting just one parser to one code generator.

P-code is an example IL that took on a life of its own. It was invented by Nicklaus Wirth as the IL for the ETH Pascal compiler. Many variants of that compiler arose [Ne179], including the USCD Pascal compiler that was used at Stanford to define an optimizer [Cho83]. Chow's compiler evolved into the MIPS compiler suite, which was the basis for one of the DEC C compilers -- acc. That compiler did not parse the same language nor use any code from the ETH compiler, but the IL survived.

Good language design usually pays off, sometimes in unexpected ways.

(If you like creating languages and writing language processors, Clark's paper is worth a read!)


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

March 30, 2012 5:22 PM

A Reflection on Alan Turing, the Turing Test, and Machine Intelligence

Alan Turing

In 1950, Alan Turing published a paper that launched the discipline of artificial intelligence, Computing Machinery and Intelligence. If you have not read this paper, go and do so. Now. 2012 is the centennial of Turing's birth, and you owe yourself a read of this seminal paper as part of the celebration. It is a wonderful work from a wonderful mind.

This paper gave us the Imitation Game, an attempt to replace the question of whether a computer could be intelligent by withn something more concrete: a probing dialogue. The Imitation became the Turing Test, now a staple of modern culture and the inspiration for contests and analogies and speculation. After reading the paper, you will understand something that many people do not: Turing is not describing a way for us to tell the difference between human intelligence and machine intelligence. He is telling us that the distinction is not as important as we seem to think. Indeed, I think he is telling us that there is no distinction at all.

I mentioned in an entry a few years ago that I always have my undergrad AI students read Turing's paper and discuss the implications of what we now call the Turing Test. Students would often get hung up on religious objections or, as noted in that entry, a deep and a-rational belief in "gut instinct". A few ended up putting their heads in the sand, as Turing knew they might, because they simply didn't want to confront the implication of intelligences other than our own. And yet they were in an AI course, learning techniques that enable us to write "intelligent" programs. Even students with the most diehard objections wanted to write programs that could learn from experience.

Douglas Hofstadter, who visited campus this month, has encountered another response to the Turing Test that surprised him. On his second day here, in honor of the Turing centenary, Hofstadter offered a seminar on some ideas related to the Turing Test. He quoted two snippets of hypothetical man-machoine dialogue from Turing's seminal paper in his classic Gödel, Escher, Bach. Over the the years, he occasionally runs into philosophers who think the Turing Test is shallow, trivial to pass with trickery and "mere syntax". Some are concerned that it explores "only behavior". Is behavior all there is? they ask.

As a computer programmer, the idea that the Turing test explores only behavior never bothered me. Certainly, a computer program is a static construct and, however complex it is, we can read and understand it. (Students who take my programming languages course learn that even another program can read and process programs in a helpful way.) This was not a problem for Hofstadter either, growing up as he did in a physicist's household. Indeed, he found Turing's formulation of the Imitation Game to be deep and brilliant. Many of us who are drawn to AI feel the same. "If I could write a program capable of playing the Imitation Game," we think, "I will have done something remarkable."

One of Hofstadter's primary goals in writing GEB was to make a compelling case form Turing's vision.

Douglas Hofstadter

Those of us who attended the Turing seminar read a section from Chapter 13 of Le Ton beau de Marot, a more recent book by Hofstadter in which he explores many of the same ideas about words, concepts, meaning, and machine intelligence as GEB, in the context of translating text from one language to another. Hofstadter said the focus in this book is on the subtlety of words and the ideas they embody, and what that means for translation. Of course, these are the some of the issues that underlie Turing's use of dialogue as sufficient for us to understand what it means to be intelligent.

In the seminar, he shared with us some of his efforts to translate a modern French poem into faithful English. His source poem had itself been translated from older French into modern French by a French poet friend of his. I enjoyed hearing him talk about "the forces" that pushed him toward and away from particular words and phrases. Le Ton beau de Marot uses creative dialogues of the sort seen in GEB, this time between the Ace Mechanical Translator (his fictional computer program) and a Dull Rigid Human. Notice the initials of his raconteurs! They are an homage to Turing. The human translator, Douglas R. Hofstadter himself, is cast in the role of AMT, which shares its initials with Alan M. Turing, the man who started this conversation over sixty years ago.

Like Hofstadter, I have often encountered people who object to the Turing test. Many of my AI colleagues are comfortable with a behavioral test for intelligence but dislike that Turing considers only linguistic behavior. I am comfortable with linguistic behavior because it captures what is for me the most important feature of intelligence: the ability to express and discuss ideas.

Others object that it sets too low a bar for AI, because it is agnostic on method. What if a program "passes the test", and when we look inside the box we don't understand what we see? Or worse, we do understand what we see and are unimpressed? I think that this is beside the point. Not to say that we shouldn't want to understand. If we found such I program, I think that we would make it an overriding goal to figure out how it works. But how an entity manages to be "intelligent" is a different question from whether it is intelligent. That is precisely Turing's point!

I agree with Brian Christian, who won the prize for being "The Most Human Human" in a competition based on Turing's now-famous test. In an interview with The Paris Review, he said,

Some see the history of AI as a dehumanizing narrative; I see it as much the reverse.

Turing does not diminish what it is to be human when he suggests that a computer might be able to carry on a rich conversation about something meaningful. Neither do AI researchers or teenagers like me, who dreamed of figuring just what it is that makes it possible for humans to do what we do. We ask the question precisely because we are amazed. Christian again:

We build these things in our own image, leveraging all the understanding of ourselves we have, and then we get to see where they fall short. That gap always has something new to teach us about who we are.

As in science itself, every time we push back the curtain, we find another layer of amazement -- and more questions.

I agree with Hofstadter. If a computer could do what it does in Turing's dialogues, then no one could rightly say that it wasn't "intelligent", whatever that might mean. Turing was right.

~~~~

PHOTOGRAPH 1: the Alan Turing centenary celebration. Source: 2012 The Alan Turing Year.

PHOTOGRAPH 2: Douglas Hofstadter in Bologna, Italy, 2002. Source: Wikimedia Commons.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

March 09, 2012 3:33 PM

This and That from Douglas Hofstadter's Visit

Update: In the original, I conflated two quotes in
"Food and Hygiene". I have un-conflated them.

In addition to his lecture on Gödel's incompleteness theorem, Douglas Hofstadter spent a second day on campus, leading a seminar and giving another public talk. I'll blog on those soon. In the meantime, here are a few random stories I heard and impressions I formed over the two days.

The Value of Good Names.   Hofstadter told a story about his "favorite chapter on Galois theory" (don't we all have one?), from a classic book that all the mathematicians in the room recognized. The only thing Hofstadter didn't like about this chapter was that it referred to theorems by number, and he could never remember which theorem was which. That made an otherwise good text harder to follow than it needed to be.

In contrast, he said, was a book by Galyan that gave each theorem a name, a short phrase evocative of what the theorem meant. So much better for reader! So he gave his students one semester an exercise to make his favorite chapter better: they were to give each of the numbered theorems in the chapter an evocative name.

This story made me think of my favorite AI textbook, Patrick Henry Winston's Artificial Intelligence. Winston's book stands out from the other AI books as quirky. He uses his own vocabulary and teaches topics very much in the MIT AI fashion. But he also gives evocative names to many of the big ideas he wants us to learn, among them the representation principle, the principle of least commitment, the diversity principle, and the eponymous "Winston's principle of parallel evolution". My favorite of all is the convergent intelligence principle:

The world manifests constraints and regularities. If a computer is to exhibit intelligence, it must exploit those constraints and regularities, no matter of what the computer happens to be made.

To me, that is AI.

Food and Hygiene.   The propensity of mathematicians to make their work harder for other people to understand, even other mathematicians, reminded Doug Shaw of two passages, from famed mathematicians Gian-Carlo Rota and André Weil. Rota said that we must guard ... against confusing the presentation of mathematics with the content of mathematics. More colorfully, Weil cautioned [If] logic is the hygiene of the mathematician, it is not his source of food. Theorems, proofs, and Greek symbols are mathematical hygiene. Pictures, problems, and understanding are food.

A Good Gig, If You Can Get It.   Hofstadter holds a university-level appointment at Indiana, and his research on human thought and the fluidity of concepts is wide enough to include everything under the sun. Last semester, he taught a course on The Catcher in the Rye. He and his students read the book aloud and discussed what makes it great. Very cool.

If You Need a Research Project...   At some time in the past, Hofstadter read, in a book or article about translating natural language into formal logic, that 'but' is simply a trivial alternative to 'and' and so can be represented as such. "Nonsense", he said! 'but' embodies all the complexity of human thought. "If we could write a program that could use 'but' correctly, we would have accomplished something impressive."

Dissatisfied.   Hofstadter uses that word a lot in conversation, or words like it, such as 'unsatisfying'. He does not express the sentiment in a whiny way. He says it in a curious way. His tone always indicates a desire to understand something better, to go deeper to the core of the question. That's a sign of a good researcher and a deep thinker.

~~~~

Let's just say that this was a great treat. Thanks to Dr. Hofstadter for sharing so much time with us here.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Teaching and Learning

March 07, 2012 5:35 PM

Douglas Hofstadter on Questions, Proofs, and Passion

In the spring of my sophomore year in college, I was chatting with the head of the Honors College at my alma mater. His son, a fellow CS major, had recently read what he considered a must-read book for every thinking computer scientist. I went over to library and checked it out in hardcopy. I thumbed through it, and it was love at first sight. So I bought the paperback and spent my summer studying it, line by line.

the cover of Godel, Escher, Bach

Gödel, Escher, Bach seemed to embody everything that excited me about computer science and artificial intelligence. It made, used, and deconstructed analogies. It talked about programming languages, and computer programs as models. Though I allowed myself to be seduced in grad school by other kinds of AI, I never felt completely satisfied. My mind and heart have never really left go of the feeling I had that summer.

Last night, I had the pleasure of seeing Douglas Hofstadter give the annual Hari Shankar Memorial Lecture here. This lecture series celebrates the beauty of mathematics and its accessibility to everyone. Hofstadter said that he was happy to honored to be asked to give such a public lecture, speaking primarily to non-mathematicians. Math is real; it is in the world. It's important, he said, to talk about it in ways that are accessible to all. His lecture would share the beauty of Gödel's Incompleteness Theorem. Rather than give a dry lecture, he told the story as his story, putting the important issues and questions into the context of his own life in math.

As a 14-year-old, he discovered a paperback copy of Gödel's Proof in a used bookstore. His father mentioned that one of the authors, Ernest Nagel, was one of his teachers and friends. Douglas was immediately fascinated. Gödel used a tool (mathematics) to study itself. It was "a strange loop".

As a child, he figured out that two twos is four. The natural next question is, "What is three threes?" But this left him dissatisfied, because two was still lurking in the question. What is "three three threes"? It wasn't even clear to him what that might mean.

But he was asking questions about patterns and and seeking answers. He was on his way to being a mathematician. How did he find answers? He understood science to be about experiments, so he looked for answers by examining a whole bunch of cases, until he had seen enough to convince himself that a claim was true.

He did not know yet what a proof was. There are, of course, many different senses of proof, including informal arguments and geometric demonstrations. Mathematicians use these, but they are not what they mean by 'proof'.

Douglas Hofstadter

So he explored problems and tried to find answers, and eventually he tried to prove his answers right. He became passionate about math. He was excited by every new discovery. (Pi!) In retrospect, his excitement does not surprise him. It took mathematicians hundreds of years to create and discover these new ideas. When he learned about them after the fact, they look like magic.

Hofstadter played with numbers. Squares. Triangular numbers. Primes. He noticed that 2^3 and 3^2 adjacent to one another and wondered if any other powers were adjacent.

Mathematicians have faith that there is an answer to questions like that. It may be 'yes', it may be 'no', but there's an answer. He said this belief is so integral to the mindset that he calls this the Mathematician's Credo:

If something is true, it has a proof, and if something has a proof, then it is true.

As an example, he wrote the beginning of the Fibonacci series on the chalk board: 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233. The list contains some powers of integers: 1, 8, and 144. Are there more squares? Are there more powers? Are there infinitely many? How often do they appear? It turns out that someone recently discovered that there are no more integer powers in the list. A mathematician may be surprised that this is true, but she would not be surprised that, if it is true, there is a proof of it.

Then he gave an open question as an example. Consider this variation of a familiar problem:

  • Rule 1: n → 2n
  • Rule 2: 3n+1 → n
  • Start with 1.

Rule 1 takes us from 1 to 2. Rule 1 takes us to 4. Rule 2 takes us to 1. We've already been there, so that's not very interesting. But Rule 1 takes us to 8.... And so on.

Hofstadter called the numbers we visited C-numbers. He then asked a question: Where can we go using these rules? Can we visit every integer? Which numbers are C-numbers? Are all integers C-numbers?

The answer is, we don't know. People have used computers to test all the integers up to a very large number (20 x 2^58) and found that we can reach every one of them from 1. So many people conjecture strongly that all integers are C-numbers. But we don't have a proof, so the purist mathematician will say only, "We don't know".

At this point in the talk, my mind wanders.... (Wonders?) It would be fun to write a program to answer, "Is n a C-number?" in Flair, the language for which my students this semester are writing a compiler. That would make a nice test program. Flair is a subset of Pascal without any data structures, so there is an added challenge... A danger of teaching a compilers course -- any course, really -- is that I would rather write programs than do almost anything else in the world.

One could ask the same question of the Fibonacci series: Is every number a Fibonacci number? It is relatively easy to answer this question with 'no'. The sequence grows larger in each new entry, so once you skip any number, you know it's not in the list. C-numbers are tougher. They grow and shrink. For any given number n, we can search the tree of values until we find it. But there is no proof for all n.

As one last bit of preparation, Hofstadter gave an informal proof of the statement, "There are an infinite number of prime numbers." The key is that his argument used one assumption (there are a finite number of primes) to destroy a necessary consequence of the same (p1*p2*...*pk+1 is prime).

From there, Hofstadter told a compact, relatively simple version of Tarski's undefinability theorem and, at the end, made the bridge to Gödel's theorem. I won't tell that story here, for a couple of reasons. First, this entry is already quite long. Second, Hofstadter himself has already told this story better than I ever could, in Gödel, Escher, Bach. You really should read it there.

This story gave him a way to tell us about the importance of the proof: it drives a wedge between truth and provability. This undermines the Mathematician's Credo. It also allowed him to demonstrate his fascination with Gödel's Proof so many years ago: it uses mathematical logic to say something interesting, powerful, and surprising about mathematical logic itself.

Hofstadter opened the floor to questions. An emeritus CS professor asked his opinion of computer proofs, such as the famous 1976 proof of the four color theorem. That proof depends on a large number of special cases and requires hundreds of pages of analysis. At first, Hofstadter said he doesn't have much of an opinion. Of course, such proofs require new elements of trust, such as trust that the program is correct and trust that the computer is functioning correctly. He is okay with that. But then he said that he finds such proofs to be unsatisfying. Invariably, they are a form of brute force, and that violates the spirit of mathematics that excites him. In the end, these proofs do not help him to understand why something isn true, and that is the whole point of exploring: to understand why.

This answer struck a chord in me. There are whole swaths of artificial intelligence that make me feel the same way. For example, many of my students are fascinated by neural networks. Sure, it's exciting any time you can build a system that solves a problem you care about. (Look ma, no hands!) But these programs are unsatisfying because they don't give me any insight into the nature of the problem, or into how humans solve the problem. If I ask a neural network, "Why did you produce this output for this input?", I can't expect an answer at a conceptual level. A vector of weights leaves me cold.

To close the evening, Hofstadter responded to a final question about the incompleteness theorem. He summarized Gödel's result in this way: Every interesting formal system says true things, but it does not say all true things. He also said that Tarski's result is surprising, but in a way comforting. If an oracle for T-numbers existed, then mathematics would be over. And that would be depressing.

As expected, I enjoyed the evening greatly. Having read GEB and taken plenty of CS theory courses, I already knew the proofs themselves, so the technical details weren't a big deal. What really highlighted the talk for me was hearing Hofstadter talk about his passions: where they came from, how he has pursued them, and how these questions and answers continue to excite him as they do. Listening to an accomplished person tell stories that make connections to their lives always makes me happy.

We in computer science need to do more of what people like Hofstadter do: talk about the beautiful ideas of our discipline to as many people as we can, in way that is accessible to all. We need a Sagan or a Hofstadter to share the beauty.

~~~~

PHOTOGRAPH 1: a photograph of the cover of my copy of Gödel, Escher, Bach.

PHOTOGRAPH 2: Douglas Hofstadter in Bologna, Italy, 2002. Source: Wikimedia Commons.


Posted by Eugene Wallingford | Permalink | Categories: Computing

February 14, 2012 3:51 PM

Beautiful Sentences, Programming Languages Division

From The Heroes of Java: Ward Cunningham (emphasis added):

Making a new language is the ultimate power move. There are lots of ways to do it. Get to know them all. A switch statement inside a for loop makes an interpreter. When you write that, ask yourself: what language am I interpreting?

Those last two sentences distill a few weeks of a course on programming languages into an idea that spans all of computer science. Beautiful.

As in most conversations with Ward, this one is full of good sentences, including:

The shortest path to exceeding expectations rarely goes through meeting expectations.

And:

When [pair programming] you can assume both people bring something to the collaboration. I'd rather find my colleague's strength or passion and learn from them as we go forward together.

In my experience, Ward brings the same mindset to all of his professional interactions. I am still a fan, though I have never asked for his autograph.


Posted by Eugene Wallingford | Permalink | Categories: Computing

January 10, 2012 4:05 PM

Looking Forward: Preparing for Compilers

Spring semester is underway. My compilers course met for the first time today. After all these years, I still get excited at the prospect of writing a compiler. On top of that, we get to talk about programming languages and programming all semester.

I've been preparing for the course since last semester, during the programming languages course I debriefed recently. I've written blog entries as I planned previous offerings of the compiler course, on topics such as short iterations and teaching by example, fifteen compilers in fifteen weeks and teaching the course backwards. I haven't written anything yet this time for one of the same reasons I haven't been writing about my knee rehab: I haven't had much to say. Actually, I have two small things.

First, on textbooks. I found that the textbook I've used for the last few offering of the course now costs students over $140, even at Amazon. That's no $274.70, but sheesh. I looked at several other popular undergrad compiler texts and found them all to be well over $100. The books my students might want to keep for their professional careers are not suitable for an undergrad course, and the ones that are suitable are expensive. I understand the reasons why yet can't stomach hitting my students with such a large bill. The Dragon book is the standard, of course, but I'm not convinced it's a good book for my audience -- too few examples, and so much material. (At least it's relatively inexpensive, at closer to $105.)

I found a few compiler textbooks available free on-line, including that $275 book I like. Ultimately I settled on Torben Mogensen's Basics of Compiler Design. It covers the basic material without too much fluff, though it lacks a running example with full implementation. I'll augment it with my own material and web readings. The price is certainly attractive. I'll let you know how it works out.

Second, as I was filing a pile of papers over break, I ran across the student assessments from last offering of the course. Perfect timing! I re-read them and am able to take into account student feedback. The last group was pretty pleased with the course and offered two broad suggestions for improvement: more low-level details and more code examples. I concur in both. It's easy when covering so many new ideas to stay at an abstract level, and the compiler course is no exception. Code examples help students connect the ideas we discuss with the reality of their own projects.

These are time consuming improvements to make, and time will be at a premium with a new textbook for the course. This new text makes them even more important, though, because it has few code examples. My goal is to add one new code example to each week of the course. I'll be happy if I manage one really good example every other week.

And we are off.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

December 01, 2011 3:06 PM

Programming for Everyone: Journalists

Jacob Harris begins his recent In Praise of Impractical Programming with a short discussion of how programming is becoming an integral part of the newsroom:

For the past few years, I've been working as a software developer in the newsroom, where perceptions of my kind have changed from novelty to a necessity. Recognizing this, some journalism schools now even require programming courses to teach students practical skills with databases or web frameworks. It's thrilling to contemplate a generation of web-hacking journalists -- but I wish we could somehow squeeze a little magic into their course load.

This seems like a natural evolutionary path that many industries will follow in the coming years or decades. At first it will be enough to use other people's tools. Then, practitioners will want to be able to write code in a constrained environment, such as a web framework or a database application. Eventually, I suspect that at least a few of the programming practitioners will tire of the constraints, step outside of the box, and write the code -- and maybe even the tools -- they want and need. If historians can do it, so can journalists.


Posted by Eugene Wallingford | Permalink | Categories: Computing

November 30, 2011 7:07 PM

A Definition of Design from Charles Eames

Paul Rand's logo for NeXT

Our council of department heads meets in the dean's conference room, in the same building that houses the Departments of Theater and Art, among others. Before this morning's meeting, I noticed an Edward Tufte poster on the wall and went out to take a look. It turns out that the graphic design students were exhibiting posters they had made in one of their classes, while studying accomplished designers such as Tufte and Paul Rand, the creator of the NeXT logo for Steve Jobs.

As I browsed the gallery, I came across a couple of posters on the work of Charles and Ray Eames. One of them prominently featured this quote from Charles:

Design is a plan for arranging elements in such a way as best to accomplish a particular purpose.

This definition works just as well for software design as it does for graphic design. It is good to be reminded occasionally how universal the idea of design is to the human condition.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 12, 2011 10:40 AM

Tools, Software Development, and Teaching

Last week, Bret Victor published a provocative essay on the future of interaction design that reminds us we should be more ambitious in our vision of human-computer interaction. I think it also reminds us that we can and should be more ambitious in our vision of most of our pursuits.

I couldn't help but think of how Victor's particular argument applies to software development. First he defines "tool":

Before we think about how we should interact with our Tools Of The Future, let's consider what a tool is in the first place.

I like this definition: A tool addresses human needs by amplifying human capabilities.

a tool addresses human needs by amplifying human capabilities

That is, a tool converts what we can do into what we want to do. A great tool is designed to fit both sides.

The key point of the essay is that our hands have much more consequential capabilities than our current interfaces use. They feel. They participate with our brains in numerous tactile assessments of the objects we hold and manipulate: "texture, pliability, temperature; their distribution of weight; their edges, curves, and ridges; how they respond in your hand as you use them". Indeed, this tactile sense is more powerful than the touch-and-slide interfaces we have now and, in many ways, is more powerful than even sight. These tactile senses are real, not metaphorical.

As I read the essay, I thought of the software tools we use, from language to text editors to development processes. When I am working on a program, especially a big one, I feel much more than I see. At various times, I experience discomfort, dread, relief, and joy.

Some of my colleagues tell me that these "feelings" are metaphorical, but I don't think so. A big part of my affinity for so-called agile approaches is how these sensations come into play. When I am afraid to change the code, it often means that I need to write more or better unit tests. When I am reluctant to add a new feature, it often means that I need to refactor the code to be more hospitable. When I come across a "code smell", I need to clean up, even if I only have time for a small fix. YAGNI and doing the simplest thing that can possibly work are ways that I feel my way along the path to a more complete program, staying in tune with the code as I go. Pair programming is a social practice that engages more of my mind than programming alone.

Victor closes with some inspiration for inspiration:

In 1968 -- three years before the invention of the microprocessor -- Alan Kay stumbled across Don Bitzer's early flat-panel display. Its resolution was 16 pixels by 16 pixels -- an impressive improvement over their earlier 4 pixel by 4 pixel display.

Alan saw those 256 glowing orange squares, and he went home, and he picked up a pen, and he drew a picture of a goddamn iPad.

We can think bigger about so much of what we do. The challenge I take from Victor's essay is to think about the tools I to teach: what needs do they fulfill, and how well do they amplify my own capabilities? Just as important are the tools we give our students as they learn: what needs do they fulfill, and how well do they amplify our students' capabilities?


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

November 09, 2011 4:29 PM

Sentences of the Day: Sheer Fun

From Averia, The Average Font:

Having found a simple process to use, I was ready to start. And after about a month of part-time slaving away (sheer fun! Better than any computer game) -- in the process of which I learned lots about bezier curves and font metrics -- I had a result.

Programmers love to slave away in their free time on projects that put fires in their bellies, with no slave driver other than their own passion.

The story of Averia is a worthy one to read, even if you are not particularly a font person. It's really about how the seed of an idea can grow as our initial efforts pull us deeper into the beautiful intricacies of a problem. It also reminds us how programs make nice testbeds for our experiments.


Posted by Eugene Wallingford | Permalink | Categories: Computing

November 05, 2011 10:09 AM

Is Computing Too Hard, Too Foreign, or Too Disconnected?

A lot of people are discussing a piece published in the New York Times piece yesterday, Why Science Majors Change Their Minds (It's Just So Darn Hard). It considers many factors that may be contributing to the phenomenon, such as low grades and insufficient work habits.

Grades are typically much lower in STEM departments, and students aren't used to getting that kind of marks. Ben Deaton argues that this sort of rigor is essential, quoting his undergrad steel design prof: "If you earn an A in this course, I am giving you a license to kill." Still, many students think that a low grade -- even a B! -- is a sign that they are not suited for the major, or for the specialty area. (I've had students drop their specialty in AI after getting a B in the foundations course.)

Most of the students who drop under such circumstances are more than capable of succeeding. Unfortunately, they have not usually developed the disciplined work habits they need to succeed in such challenging majors. It's a lot easier to switch to a different major where their current skills suffice.

I think there are two more important factors at play. On the first, the Times article paraphrases Peter Kilpatrick, Notre Dame's Dean of Engineering:

... it's inevitable that students will be lost. Some new students do not have a good feel for how deeply technical engineering is.

In computer science, our challenge is even bigger: most HS students don't have any clue at all what computer science is. My university is nearing the end of its fall visit days for prospective students, who are in the process of choosing a college and a major. The most common question I am asked is, "What is computer science?", or its cousin, "What do computer scientists do?". This question comes from even the brightest students, ones already considering math or physics. Even more students walk by the CS table with their parents with blank looks on their faces. I'm sure some are thinking, "Why consider a major I have no clue about?"

This issue also plagues students who decide to major in CS and then change their minds, which is the topic of the Times article. Students begin the major not really knowing what CS is, they find out that they don't like it as much as they thought they might, and they change. Given what they know coming into the university, it really is inevitable that a lot of students will start and leave CS before finishing.

On the second factor I think most important, here is the money paragraph from the Times piece:

But as Mr. Moniz (a student exceedingly well prepared to study engineering) sat in his mechanics class in 2009, he realized he had already had enough. "I was trying to memorize equations, and engineering's all about the application, which they really didn't teach too well," he says. "It was just like, 'Do these practice problems, then you're on your own.'" And as he looked ahead at the curriculum, he did not see much relief on the horizon.

I have written many times here about the importance of building instructions around problems, beginning with Problems Are The Thing. Students like to work on problems, especially problems that matter to someone in the world. Taken to the next level, as many engineering schools are trying to do, courses should -- whenever possible -- be built around projects. Projects ground theory and motivate students, who will put in a surprising effort on a project they care about or think matters in the world. Projects are also often the best way to help students understand why they are working so hard to memorize and practice tough material.

In closing, I can take heart that schools like mine are doing a better job retaining majors:

But if you take two students who have the same high school grade-point average and SAT scores, and you put one in a highly selective school like Berkeley and the other in a school with lower average scores like Cal State, that Berkeley student is at least 13 percent less likely than the one at Cal State to finish a STEM degree.

Schools tend to teach less abstractly than our research-focused sister schools. We tend to provide more opportunities early in the curriculum to work on projects and to do real research with professors. I think the other public universities in my state do a good job, but if a student is considering an undergrad STEM major, they will be much better served at my university.

There is one more reason for the better retention rate at the "less selective" schools: pressure. The students at the more selective schools are likely to be more competitive about grades and success than the students at the less selective schools. This creates an environment more conducive to learning for most students. In my department, we try not to "treat the freshman year as a 'sink or swim' experience and accept attrition as inevitable" for reasons of Darwinian competition. As the Times article says, this is both unfair to students and wasteful of resources.

By changing our curricula and focusing more on student learning than on how we want to teach, universities can address the problem of motivation and relevance. But that will leave us with the problem of students not really knowing what CS or engineering are, or just how technical and rigorous they need to be. This is an education problem of another sort, one situated in the broader population and in our HS students. We need to find ways to both share the thrill and help more people see just what the STEM disciplines are and what they entail.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 02, 2011 7:39 AM

Programming for All: will.i.am

While reading a bit about the recent flap over racism in the tech start-up world, I found this passage in a piece by Michael Arrington:

will.i.am was proposing an ambitious new idea to help get inner city youth (mostly minorities) to begin to see superstar entrepreneurs as the new role models, instead of NBA stars. He believes that we can effect real societal change by getting young people to learn how to program, and realize that they can start businesses that will change the world.

Cool. will.i.am is a pop star who has had the ear of a lot of kids over the last few years. I hope they listen to this message from him as much as they do to his music.


Posted by Eugene Wallingford | Permalink | Categories: Computing

October 31, 2011 3:47 PM

"A Pretty Good Lisp"

I occasionally read or hear someone say, "X is a pretty good Lisp", where X is a programming language. Usually, it's a newish language that is more powerful than the languages many of us learned in school. For a good example, see Why Ruby is an acceptable LISP. A more recent article, Ruby is beautiful (but I'm moving to Python) doesn't go quite that far. It says only "almost":

Ruby does not revel in structures or minutiae. It is flexible. And powerful. It really almost is a Lisp.

First, let me say that I like both of these posts. They tell us about how we can do functional programming in Ruby, especially through its support for higher-order functions. As a result, I have found both posts to be useful reading for students. And, of course, I love Ruby, and like Python well enough.

But that's not all there is to Lisp. It's probably not even the most important thing.

Kenny Tilton tells a story about John McCarthy's one-question rebuttal to such claims at the very end of his testimonial on adopting Lisp, Ooh! Ooh! My turn! Why Lisp?:

... [McCarthy] simply asked if Python could gracefully manipulate Python code as data.

"No, John, it can't," said Peter [Norvig] and nothing more...

That's the key: data == program. It really is the Big Idea that sets Lisp apart from the other programming languages we use. I've never been a 100% full-time Lisper, and as a result I don't think I fully appreciate the full power to which Lisp programmers put this language feature. But I've programmed enough with and without macros to be able to glimpse what they see in the ability to gracefully manipulate their code -- all code -- as data.

In the "acceptable Lisp" article linked above, Kidd does address this shortcoming and says that "Ruby gives you about 80% of what you want from macros". Ruby's rather diverse syntax lets us create readable DSLs such as Treetop and Rake, which is one of the big wins that Lisp and Scheme macros give us. In this sense, Ruby code can feel generative, much as macros do.

Unfortunately, Ruby, Python, and other "pretty good Lisps" miss out on the other side of the code-as-data equation, the side McCarthy drew out in his question: manipulation. Ruby syntax is too irregular to generate "by hand" or to read and manipulate gracefully. We can fake it, of course, but to a Lisp programmer it always feels fake.

I think what most people mean when they say a language is a pretty good Lisp is that it can be used as a pretty good functional programming language. But Lisp is not only an FP language. Many would claim it is not even primarily a functional programming language.

I love Ruby. But it's not a pretty good Lisp. It is a fine programming languages, perhaps my favorite these days, with strengths that take it beyond the system programming languages that most of us cut our teeth on. Among those strengths are excellent support for a functional programming style. It also has its weaknesses, like every other programming language.

Neither is Python a pretty good Lisp. Nor is most anything else, for that matter. That's okay.

All I ask is this: When you are reading articles like the ones linked above, don't dismiss every comment you see that says, "No, it's not, and here's why" as the ranting of a smug Lisp weenie. It may be a rant, and it may be written by a smug Lisp weenie. But it may instead be written by a perfectly sane programmer who is trying to teach you that there is more to Lisp than higher-order functions, and that the more you've missed is a whole lot more. We can learn from some of those comments, and think about how to make our programming languages even better.


Posted by Eugene Wallingford | Permalink | Categories: Computing

October 25, 2011 3:53 PM

On the Passing of John McCarthy

John McCarthy tribute -- 'you are doing it wrong'

It's been a tough couple of weeks for the computer science community. First we lost Steve Jobs, then Dennis Ritchie. Now word comes that John McCarthy, the creator of Lisp, died late Sunday night at the age of 84. I'm teaching Programming Languages this semester based on the idea of implementing small language interpreters, and we are using Scheme. McCarthy's ideas and language are at the heart of what my students and I are doing every day.

Scheme is a Lisp, so McCarthy is its grandfather. Lisp is different from just about every other programming language. It's not just the parentheses, which are only syntax. In Lisp and Scheme, programs and data are the same. To be more specific, the representation of a Lisp program is the the same representation used to represent Lisp data. The equivalence of data and program is one of the truly Big Ideas of computer science, one which I wrote about in Basic Concepts: The Unity of Data and Program. This idea is crucial to many areas of computer science, even ones in which programmers do not take direct advantage of it through their programming language.

We also owe McCarthy for the idea that we can write a language interpreter in the language being interpreted. Actually, McCarthy did more: he stated the features of Lisp in terms of the language features themselves. Such a program defines the language in which the program is written. This is the idea of meta-circular interpreter, in which two procedures:

  • a procedure that evaluates an expression, and
  • a procedure that applies a procedure to its arguments
recurse mutually to evaluate a program. This is one of the most beautiful ideas in computing, as well as serving as the mechanism and inspiration for modern-day interpreters and compilers.

Last week, the CS world lost Dennis Ritchie, the creator of the C programming language. By all accounts I've read and heard, McCarthy and Ritchie were very different kinds of people. Ritchie was an engineer through and through, while McCarthy was an academic's academic. So, too, are the languages they created very different. Yet they are without question the two most influential programming languages ever created. One taught us about simplicity and made programming across multiple platforms practical and efficient; the other taught us about simplicity made programming a matter of expressiveness and concision.

Though McCarthy created Lisp, he did not implement the first Lisp interpreter. As Paul Graham relates in Revenge of the Nerds, McCarthy first developed Lisp as a theoretical exercise, an attempt to create an alternative to the Turing Machine. Steve Russell, one of McCarthy's grad students, suggested that he could implement the theory in an IBM 704 machine language program. McCarthy laughed and told him, "You're confusing theory with practice..." Russell did it any way. (Thanks to Russell and the IBM 704, we also have car and cdr!) McCarthy and Russell soon discovered that Lisp was more powerful than the language they had planned to build after their theoretical exercise, and the history of computing was forever changed.

If you'd like, take a look at my Scheme implementation of John McCarthy's Lisp written in Lisp. It is remarkable how much can be built out of so little. Alan Kay has often compared this interpreter to Maxwell's equations in physics. To me, its parts usually feel like the basic particles out of which all matter is built. Out of these few primitives, all programs are built.

I first learned of McCarthy not from Lisp but from my first love, AI. McCarthy coined the term "Artificial Intelligence" when organizing (along with Minsky, Rochester, and Shannon) the 1956 Dartmouth conference that gave birth to the field. I studied McCarthy's work in AI using the language he had created. To me, he was a giant of AI long before I recognized that he was giant of programming languages, too. Like many pioneers of our field, he laid the groundwork in many subdisciplines. They had no choice; they had to build their work out of ideas using only the rawest materials. McCarthy is even credited with the first public descriptions of time-sharing systems and what we now call cloud computing. (For McCarthy's 1970-era predictions about home computers and the cloud, see his The Home Information Terminal, reprinted in 2000.)

Our discipline has lost a giant.


Posted by Eugene Wallingford | Permalink | Categories: Computing

October 24, 2011 7:38 PM

Simple/Complex Versus Easy/Hard

A few years ago, I heard a deacon give a rather compelling talk to a group of college students on campus. When confronted with a recommended way to live or act, students will often say that living or acting that way is hard. These same students are frustrated with the people who recommend that way of living or acting, because the recommenders -- often their parents or teachers -- act as if it is easy to live or act that way. The deacon told the students that their parents and teachers don't think it is easy, but they might well think it is simple.

How can this be? The students were confounding "simple" and "easy". A lot of times, life is simple, because we know what we should do. But that does not make life easy, because doing a simple thing may be quite difficult.

This made an impression on me, because I recognized that conflict in my own life. Often, I know just what to do. That part is simple. Yet I don't really want to do it. To do it requires sacrifice or pain, at least in the short term. To do it means not doing something else, and I am not ready or willing to forego that something. That part is difficult.

Switch the verb from "do" to "be", and the conflict becomes even harder to reconcile. I may know what I want to be. However, the gap between who I am and who I want to be may be quite large. Do I really want to do what it takes to get there? There may be a lot of steps to take which individually are difficult. The knowing is simple, but the doing is hard.

This gap surely faces college students, too, whether it means wanting to get better grades, wanting to live a healthier life, or wanting to reach a specific ambitious goal.

When I heard the deacon's story, I immediately thought of some of my friends, who like very much the idea of being a "writer" or a "programmer", but they don't really want to do the hard work that is writing or programming. Too much work, too much disappointment. I thought of myself, too. We all face this conflict in all aspects of life, not just as it relates to personal choices and values. I see it in my teaching and learning. I see it in building software.

I thought of this old story today when I watched Rich Hickey's talk from StrangeLoop 2011, Simple Made Easy. I had put off watching this for a few days, after tiring of a big fuss that blew up a few weeks ago over Hickey's purported views about agile software development techniques. I knew, though, that the dust-up was about more than just Hickey's talk, and several of my friends recommended it strongly. So today I watched. I'm glad I did; it is a good talk. I recommend it to you!

Based only on what I heard in this talk, I would guess that Hickey misunderstands the key ideas behind XP's practices of test-driven development and refactoring. But this could well be a product of how some agilistas talk about them. Proponents of agile and XP need to be careful not to imply that tests and refactoring make change or any other part of software development easy. They don't. The programmer still has to understand the domain and be able to think deeply about the code.

Fortunately, I don't base what I think about XP practices on what other people think, even if they are people I admire for other reasons. And if you can skip or ignore any references Hickey makes to "tests as guard rails" or to statements that imply refactoring is debugging, I think you will find this really is a very good talk.

Hickey's important point is that simple/complex and easy/hard are different dimensions. Simplicity should be our goal when writing code, not complexity. Doing something that is hard should be our goal when it makes us better, especially when it makes us better able to create simplicity.

Simplicity and complexity are about the interconnectedness of a system. In this dimension, we can imagine objective measures. Ease and difficulty are about what is most readily at hand, what is most familiar. Defined as they are in terms of a person's experience or environment, this dimension is almost entirely subjective.

And that is good because, as Hickey says a couple of times in the talk, "You can solve the familiarity problem for yourself." We are not limited to our previous experience or our current environment; we can take on a difficult challenge and grow.

a Marin mountain bike

Alan Kay often talks about how it is worth learning to play a musical instrument, even though playing is difficult, at least at the start. Without that skill, we are limited in our ability to "make music" to turning on the radio or firing up YouTube. With it, you are able make music. Likewise riding a bicycle versus walking, or learning to fly an airplane versus learning to drive a car. None of these skills is necessarily difficult once we learn them, and they enable new kinds of behaviors that can be simple or complex in their own right.

One of the things I try to help my students see is the value in learning a new, seemingly more difficult language: it empowers us to think new and different thoughts. Likewise making the move from imperative procedural style to OOP or to functional programming. Doing so stretches us. We think and program differently afterward. A bonus is that something that seemed difficult before is now less daunting. We are able to work more effectively in a bigger world.

In retrospect, what Hickey says about simplicity and complexity is actually quite compatible with the key principles of XP and other agile methods. Writing tests is a part of how we create systems that are as simple as we can in the local neighborhood of a new feature. Tests can also help us to recognize complexity as it seeps into our program, though they are not enough by themselves to help us see complexity. Refactoring is an essential part of how we eliminate complexity by improving design globally. Refactoring in the presence of unit tests does not make programming easy. It doesn't replace thinking about design; indeed, it is thinking about design. Unit tests and refactoring do help us to grapple with complexity in our code.

Also in retrospect, I gotta make sure I get down to St. Louis for StrangeLoop 2012. I missed the energy this year.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

October 19, 2011 1:37 PM

A Programming Digression: Benford's Law and Factorials

leading digits of factorials up to 500

This morning, John Cook posted a blog entry on the leading digits of factorials and how, despite what might be our intuition, they follow Benford's Law. He whipped up some Python code and showed the results of his run for factorials up to 500. I have linked to his graphic at the right.

As I am , I decided to whip up a quick Scheme version of Cook's experiment. He mentioned some implementation issues involving the sizes of integers and floating-point numbers in Python, and I wondered how well Scheme would fare.

For my first attempt, I did the simplest thing that would possibly work. I already had a tail-recursive factorial function and so wrote a procedure that would call it n times and record the first digit of each:

(define benford-factorials
  (lambda (n)
    (let ((counts (make-vector 10 0)))
      (letrec ((foreach
                 (lambda (n)
                   (if (zero? n)
                       counts
                       (let ((lead-digit (first-digit (factorial n))))
                         (vector-set! counts lead-digit
                                      (+ 1 (vector-ref counts lead-digit)))
                         (foreach (- n 1)))))))
        (foreach n)))))

This gets the answers for us:

     > (benford-factorials 500)
     #(0 148 93 67 38 34 43 24 28 25)

Of course, it is wildly inefficient. My naive implementation computes and acts on each factorial independently, which means that it recomputes (n-1)!, (n-2)!, ... for each value less than n. As a result, benford-factorials becomes unnecessarily sluggish for even relatively small values of n. How can I do better?

I decided to create a new factorial function, one that caches the smaller factorials it creates on the way to n!. I call it all-factorials-up-to:

(define all-factorials-up-to
  (lambda (n)
    (letrec ((aps (lambda (i acc)
                    (if (> i n)
                        acc
                        (aps (+ i 1)
                             (cons (* i (car acc)) acc))))))
      (aps 2 '(1)))))

Now, benford-factorials can use a more functional approach: map first-digit over the list of factorials, and then map a count incrementer over the list of first digits.

(define benford-factorials
  (lambda (n)
    (let ((counts (make-vector 10 0))
          (first-digits (map first-digit
                             (all-factorials-up-to n))))
      (map (lambda (digit)
             (vector-set! counts digit
                          (+ 1 (vector-ref counts digit))))
           first-digits)
      counts)))

(We can, of course, do without the temporary variable first-digit by dropping the first map right into the second. I often create an explaining temporary variable such as this one to make my code easier for me to write and read.)

How does this one perform? It gets the right answers and runs more comfortably on larger n:

     > (benford-factorials 500)
     #(0 148 93 67 38 34 43 24 28 25)
     > (benford-factorials 1000)
     #(0 293 176 124 102 69 87 51 51 47)
     > (benford-factorials 2000)
     #(0 591 335 250 204 161 156 107 102 94)
     > (benford-factorials 3000)
     #(0 901 515 361 301 244 233 163 147 135)
     > (benford-factorials 4000)
     #(0 1192 707 482 389 311 316 227 201 175)
     > (benford-factorials 5000)
     #(0 1491 892 605 477 396 387 282 255 215)

This procedure begins to be sluggish for n ≥ 3000 on my iMac.

Cook's graph shows how closely the predictions of Benford's Law fit for factorials up to 500. How well do the actual counts match the predicted values for the larger sets of factorials? Here is a comparison for n = 3000, 4000, and 5000:

     n = 3000
       digit        1   2   3   4   5   6   7   8   9
       actual     901 515 361 301 244 233 163 147 135
       predicted  903 528 375 291 238 201 174 153 137

n = 4000 digit 1 2 3 4 5 6 7 8 9 actual 1192 707 482 389 311 316 227 201 175 predicted 1204 704 500 388 317 268 232 205 183

n = 5000 digit 1 2 3 4 5 6 7 8 9 actual 1491 892 605 477 396 387 282 255 215 predicted 1505 880 625 485 396 335 290 256 229

That looks pretty close to the naked eye. I've always found Benford's Law to be almost magic, even though mathematicians can give a reasonable account of why it holds. Seeing it work so well with something seemingly as arbitrary as factorials only reinforces my sense of wonder.

If you would like play with these ideas, feel free to start with my Scheme code. It has everything you need to replicate my results above. If you improve on my code or take it farther, please let me know!


Posted by Eugene Wallingford | Permalink | Categories: Computing

October 17, 2011 4:46 PM

Computational Thinking Everywhere: Experiments in Education

I recently ran across Why Education Startups Do Not Succeed, based on the author's experience working as an entrepreneur in the education sector. He admits upfront that he isn't offering objective data to support his conclusions, so we should take them with a grain of salt. Still, I found his ideas interesting. Here is the take-home point in sentences:

Most entrepreneurs in education build the wrong type of business, because entrepreneurs think of education as a quality problem. The average person thinks of it as a cost problem.

That disconnect creates a disconnect between the expectations of sellers and buyers, which ends up hurting, even killing, most education start-ups.

The old AI guy in me latched on to this paragraph:

Interestingly, in the US, the people who are most willing to try new things are the poor and uneducated because they have a similar incentive structure to a person in rural India. Their default state is "screwed." If a poor person doesn't do something dramatic, they are going to stay screwed. Many parents and teachers in these communities understand this. So the communities are often willing to try new, experimental things -- online education, charter schools, longer school days, no summer vacation, co-op programs -- even if they may not work. Why? Because their students default state is "screwed", and they need something dramatically better. Doing something significantly higher quality is the only way to overcome the inertia of already being screwed. The affordable, but poor quality approaches just aren't good enough. These communities are on the hunt for dramatically better approaches and willing to try new things.

Local and global maxima in hill-climbing

I've seen other discussions of the economic behavior of people in the lowest socioeconomic categories that fit this model. Among them were the consumption of lottery tickets in lieu of saving, and more generally the trade-off between savings and consumption. If a small improvement won't help a people much, then it seems they are more likely willing to gamble on big improvements or to simply enjoy short-term rewards of spending.

This mindset immediately brought to mind the AI search technique known as hill climbing. When you know you are on a local maximum that is significantly lower than the global maximum, you are willing to take big steps in search of a better hill to climb, even if that weakens your position in the short-term. Baby steps won't get you there.

This is a small example of unexpected computational thinking in the real world. Psychologically, it seems, that we are often hill climbers.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

October 13, 2011 3:10 PM

Learning and New Kinds of Problems

I recently passed this classic by Reg Braithwaite to a grad student who is reading in the areas of functional programming and Ruby. I love how Braithwaite prefaces the technical content of the entry with an exhortation to learners:

... to obtain the deepest benefit from learning a new language, you must learn to think in the new language, not just learn to translate your favourite programming language syntax and idioms into it.

The more different the thing you are learning from what you already know, the more important this advice. You are already good at solving the problems your current languages solve well!

And worse, when a new tool is applied to a problem you think you know well, you will probably dismiss the things the new tool does well. Look at how many people dismiss brevity of code. Note that all of the people ignore the statistics about the constant ratio between bugs and lines of code use verbose languages. Look at how many people dismiss continuation-based servers as a design approach. Note that all of them use programming languages bereft of control flow abstractions.

Real programmers know Y.

This is great advice for people trying to learn functional programming, which is all the rage these days. Many people come to a language like Scheme, find it lacking for the problems they have been solving in Python, C, and Java, and assume something is wrong with Scheme, or with functional programming more generally. It's easy to forget that the languages you know and the problems you solve are usually connected in a variety of ways, not the least of which for university students is that we teach them to solve problems most easily solved by the languages we teach them!

If you keep working on the problems your current language solves well, then you miss out on the strengths of something different. You need to stretch not only your skill set but also your imagination.

If you buy this argument, schedule some time to work through Braithwaite's derivation of the Y combinator in Ruby. It will, as my daughter likes to say, make your brain hurt. That's a good thing. Just like with physical exercise, sometimes we need to stretch our minds, and make them hurt a bit, on the way to making them stronger.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

October 12, 2011 12:31 PM

Programming for Everyone -- Really?

TL;DR version: Yes.

Yesterday, I retweeted a message that is a common theme here:

Teaching students how to operate software, but not produce software, is like teaching kids to read & not write. (via @KevlinHenney)

It got a lot more action than my usual fare, both retweets and replies. Who knew? One of the common responses questioned the analogy by making another, usually of this sort:

Yeah, that would be like teaching kids how to drive a car, but not build a car. Oh, wait...

This is a sounds like a reasonable comparison. A car is a tool. A computer is a tool. We use tools to perform tasks we value. We do not always want to make our own tools.

But this analogy misses out on the most important feature of computation. People don't make many things with their cars. People make things with a computer.

When people speak of "using a computer", they usually mean using software that runs on a computer: a web browser, a word processor, a spreadsheet program. And people use many of these tools to make things.

As soon as we move into the realm of creation, we start to bump into limits. What if the tool we are given doesn't allow us to say or do what we want? Consider the spreadsheet, a general data management tool. Some people use it simply as a formatted data entry tool, but it is more. Every spreadsheet program gives us a formula language for going beyond what the creators of Excel or Numbers imagined.

But what about the rest of our tools? Must we limit what we say to what our tool affords us -- to what our tool builders afford us?

A computer is not just a tool. It is also a medium of expression, and an increasingly important one.

If you think of programming as C or Java, then the idea of teaching everyone to program may seem silly. Even I am not willing to make that case here. But there are different kinds of programming. Even professional programmers write code at many levels of abstraction, from assembly language to the highest high-level language. Non-programmers such as physicists and economists use scripting languages like Python. Kids of all ages are learning to program Scratch.

Scratch is a good example of what I was thinking when I retweeted. Scratch is programming. But Scratch is really a way to tell stories. Just like writing and speaking.

Alfred Thompson summed up this viewpoint succinctly:

[S]tudents need to be creators and not just consumers.

Kids today understand this without question. They want to make video mash-ups and interactive web pages and cutting-edge presentations. They need to know that they can do more than just use the tools we deign to give them.

One respondent wrote:

As society evolves there is an increasing gap between those that use technology and those that can create technology. Whilst this is a concern, it's not the lowest common denominator for communication: speaking, reading and writing.

The first sentence is certainly true. The question for me is: on which side of this technology divide does computing live? If you think of computation as "just" technology, then the second sentence seems perfectly reasonable. People use Office to do their jobs. It's "just a tool".

It could, however, be a better tool. Many scientists and business people write small scripts or programs to support their work. Many others could, too, if they had the skills. What about teachers? Many routine tasks could be automated in order to give them more time to do what they do best, teach. We can write software packages for them, but then we limit them to being consumers of what we provide. They could create, too.

Is computing "just tech", or more? Most of the world acts like it is the former. The result is, indeed, an ever increasing gap between the haves and the have nots. Actually, the gap is between the can dos and the cannots.

I, and many others, think computation is more than simply a tool. In the wake of Steve Jobs's death last week, many people posted his famous quote that computing is a liberal art. Alan Kay, one of my inspirations, has long preached that computing is a new medium on the order of reading and writing. The list of people in the trenches working to make this happen is too numerous to include.

More practically, software and computer technology are the basis of much innovation these days. If we teach the new medium to only a few, the "5 percent of the population over in the corner" to whom Jobs refers, we exclude the other 95% from participating fully in the economy. That restricts economic growth and hurts everyone. It is also not humane, because it restricts people's personal growth. Everyone has a right to the keys to the kingdom.

I stand in solidarity with the original tweeter and retweeter. Teaching students how to operate software, but not produce software, is like teaching kids to read but not to write. We can do better.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

October 04, 2011 4:43 PM

Programming in Context: Digital History

Last April I mentioned The Programming Historian, a textbook aimed at a specific set of non-programmers who want or need to learn how to program in order to do their job in the digital age. I was browsing through the textbook today and came across a paragraph that applies to more than just historians or so-called applied programmers:

Many books about programming fall into one of two categories: (1) books about particular programming languages, and (2) books about computer science that demonstrate abstract ideas using a particular programming language. When you're first getting started, it's easy to lose patience with both of these kinds of books. On the one hand, a systematic tour of the features of a given language and the style(s) of programming that it supports can seem rather remote from the tasks that you'd like to accomplish. On the other hand, you may find it hard to see how the abstractions of computer science are related to your specific application.

I don't think this feeling is limited to people with a specific job to do, like historians or economists. Students who come to the university intending to major in Computer Science lose patience with many of our CS1 textbooks and CS1 courses for the very same reasons. Focusing too much on all the features of a language is overkill when you are just trying to make something work. The abstractions we throw at them don't have a home in their understanding of programming or CS yet and so seem, well, too abstract.

Writing for the aspiring applied programmer has an advantage over writing for CS1: your readers have something specific they want to do, and they know just what it is. Turkel and MacEachern can teach a subset of several tools, including Python and Javascript, focused on what historians want to be able to do. Greg Wilson and his colleagues can teach what scientists want and need to know, even if the book is pitched more broadly.

In CS1, your students don't have a specific task in mind and do eventually need to take a systematic tour of a language's features and to learn a programming style or three. They do, eventually, need to learn a set of abstractions and make sense of them in the context of several languages. But when they start, they are much like any other person learning to program: they would like to do something that matters. The problems we ask them to solve matter.

Guzdial, Ericson, and their colleagues have used media computation as context in which to learn how to program, with the idea that many students, CS majors and non-majors alike, can be enticed to manipulate images, sounds, and video, the raw materials out of which students' digital lives are now constructed. It's not quite the same -- students still need to be enticed, rather than starting with their own motivation -- but it's a shorter leap to caring than the run-off-the-mill CS textbook has to make.

Some faculty argue that we need a CS0 course that all students take, in which they can learn basic programming skills in a selected context before moving onto the major's first course. The context can be general enough, say, media manipulation or simple text processing on the web, that the tools students learn will be useful after the course whether they continue on or not. Students who elect to major in CS move on to take a systematic tour of a language's features, to learn about OO or FP style, and to begin learning the abstractions of the discipline.

My university used to follow this approach, back in the early and mid 1990s. Students had to take a one-year HS programming course or a one-semester programming course at the university before taking CS1. We dropped this requirement when faculty began asking, why shouldn't we put the same care into teaching low-level programming skills in CS1 as we do into teaching CS0? The new approach hasn't always been as successful as we hoped, due to the difficulty of finding contexts that motivate students as well as we want, but I think the approach is fundamentally sound. It means that CS1 may not teach all the things that it did when the course had a prerequisite.

That said, students who take one of our non-majors programming courses, C and Visual Basic, and then move decide to major in CS perform better on average in CS1 than students who come in fresh. We have work to do.

Finally, one sentence from The Programming Historian made me smile. It embodies the "programming for all" theme that permeates this blog:

Programming is for digital historians what sketching is for artists or architects: a mode of creative expression and a means of exploration.

I once said that being able to program is like having superhuman strength. But it is both more mundane and more magical than that. For digital historians, being able to program means being able to do the mundane, everyday tasks of manipulating text. It also gives digital historians a way to express themselves creatively and to explore ideas in ways hard to imagine otherwise.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

October 03, 2011 7:20 AM

Softmax, Recursion, and Higher-Order Procedures

Update: This entry originally appeared on September 28. I bungled my blog directory and lost two posts, and the simplest way to get the content back on-line is to repost.

John Cook recently reported that he has bundled up some of his earlier writings about the soft maximum as a tech report. The soft maximum is "a smooth approximation to the maximum of two real variables":

    softmax(x, y) = log(exp(x) + exp(y))

When John posted his first blog entry about the softmax, I grabbed the idea and made it a homework problem for my students, who were writing their first Scheme procedures. I gave them a link to John's page, so they had access to this basic formula as well as a Python implementation of it. That was fine with me, because I was simply trying to help students become more comfortable using Scheme's unusual syntax:

    (define softmax
      (lambda (x y)
        (log (+ (exp x)
                (exp y)))))

On the next assignment, I asked students to generalize the definition of softmax to more than two variables. This gave them an opportunity to write a variable arity procedure in Scheme. At that point, they had seen only a couple simple examples of variable arity, such as this implementation of addition using a binary + operator:

    (define plus              ;; notice: no parentheses around
      (lambda args            ;; the args parameter in lambda
        (if (null? args)
            0
            (+ (car args) (apply plus (cdr args))) )))

Many students followed this pattern directly for softmax:

    (define softmax-var
      (lambda args
        (if (null? (cdr args))
            (car args)
            (softmax (car args)
                     (apply softmax-var (cdr args))))))

Some of their friends tried a different approach. They saw that they could use higher-order procedures to solve the problem -- without explicitly using recursion:

    (define softmax-var
      (lambda args
        (log (apply + (map exp args)))))

When students saw each other's solutions, they wondered -- as students often do -- which one is correct?

John's original blog post on the softmax tells us that the function generalizes as we might expect:

    softmax(x1, x2, ..., xn) = log(exp(x1) + exp(x2) + ... + exp(xn))

Not many students had looked back for that formula, I think, but we can see that it matches the higher-order softmax almost perfectly. (map exp args) constructs a list of the exp(xi) values. (apply + ...) adds them up. (log ...) produces the final answer.

What about the recursive solution? If we look at how its recursive calls unfold, we see that this procedure computes:

    softmax(x1, softmax(x2, ..., softmax(xn-1, xn)...))

This is an interesting take on the idea of a soft maximum, but it is not what John's generalized definition says, nor is it particularly faithful to the original 2-argument function.

How might we roll our own recursive solution that computes the generalized function faithfully? The key is to realize that the function needs to iterate not over the maximizing behavior but the summing behavior. So we might write:

    (define softmax-var
      (lambda args
        (log (accumulate-exps args))))

(define accumulate-exps (lambda (args) (if (null? args) 0 (+ (exp (car args)) (accumulate-exps (cdr args))))))

This solution turns softmax-var into interface procedure and then uses structural recursion over a flat list of arguments. One advantage of using an interface procedure is that the recursive procedure accumulate-exps no longer has to deal with variable arity, as it receives a list of arguments.

It was remarkable to me and some of my students just how close the answers produced by the two student implementations of softmax were, given how different the underlying behaviors are. Often, the answers were identical. When different, they differed only in the 12th or 15th decimal digit. As several blog readers pointed out, softmax is associative, so the two solutions are identical mathematically. The differences in the values of the functions result from the vagaries of floating-point precision.

The programmer in me left the exercise impressed by the smoothness of the soft maximum. The idea is resilient across multiple implementations, which makes it seem all the more useful to me.

More important, though, this programming exercise led to several interesting discussions with students about programming techniques, higher-order procedures, and the importance of implementing solutions that are faithful to the problem domain. The teacher in me left the exercise pleased.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

October 02, 2011 5:16 PM

Computing Everywhere: Economist Turns Programmer

This About Me page, on Ana Nelson's web site, is a great example of how computing can sneak up on people:

Having fallen in love with programming while studying for my Ph.D. in economics, I now develop open source software to explore and explain data. I am the creator of Dexy, a new tool for reproducible research and document automation.

A lot of disciplines explore and explain data, from particular domains and within particular models. I'm not surprised when I encounter someone in one of those disciplines who finds she likes exploring and explaining data more than the specific domain or model. Programming is a tool that lets rise above disciplinary silos and consider data, patterns, and ideas across the intellectual landscape.


Posted by Eugene Wallingford | Permalink | Categories: Computing

September 26, 2011 9:02 PM

Taking Computer Science Off the Beaten Track

I love this idea: a workshop at POPL 2012, the premier programming languages research conference, called Off the Beaten Track. Here is the gist.

Programming language researchers have the principles, tools, algorithms and abstractions to solve all kinds of problems, in all areas of computer science. However, identifying and evaluating new problems, particularly those that lie outside the typical core PL problems we all know and love, can be a significant challenge. Hence, the goal of this workshop is to identify and discuss problems that do not often show up in our top conferences, but where programming language researchers can make a substantial impact. The hope is that by holding such a forum and associating it directly with a top conference like POPL, we can slowly start to increase the diversity of problems that are studied by PL researchers and that by doing so we will increase the impact that our community has on the world.

I remember when I first started running across papers written by physicists on networks in social science and open-source software. Why were physicists writing these papers? They are out of their league. In fact, though they were something else: curious, and equipped with good tools for studying the problems. Good for them -- and good fro the rest of us. too, as they contributed to our understanding of how the world works.

Computer science has even better tools and methods for studying all manners of problems and systems, especially the more dynamic systems. Our ability to reify the language of any domain and then write programs to implement the semantics of the domain is one step up from the models that most mathematicians and physicists bring to the table.

Sometimes we forget the power we have in language. As Tim Ottinger tweeted today, "I think people have lost the idea that OO builds the language you implement in, as well as the product." We forget to use our own ability to create and use language even in an approach built on the premise! And of course we can go farther when we build architectures and virtual machines for domain-specific languages, rather than living inside a relatively restrictive model like Java.

The organizers of Off the Beaten Track remind us to think about the wealth of "principles, tools, algorithms, and abstractions" we possess and can bring to bear on problems far beyond the narrow technical area of programming languages research, from the natural sciences to art and music, from economics and the law to linguistics and education. They even acknowledge that we don't always appreciate the diversity in our own research field and so encourage submissions on "unusual compilers" and "underrepresented programming languages".

The last sentence in the passage above expresses an important ultimate goal: to increase the impact the programming languages community has on the world. I heartily support this goal and suggest that it is an important one not only for programming languages researchers. It is essential that many more of us in computer science look off the beaten track for ways to apply what we have learned to problems far beyond our own borders. If we start focusing on problems that matter to other people, problems that matter, we might just solve them.

My favorite line in the Off the Beaten Track home page is the last item in its bullet list of potential topic areas: Surprise us.. Indeed. Surprise us.


Posted by Eugene Wallingford | Permalink | Categories: Computing

September 23, 2011 3:52 PM

Grading and Learning in the Age of Social Media

Yesterday morning, I was grading the first quiz from my programming languages course and was so surprised by the responses to the first short-answer question that I tweeted in faux despair:

Wow. We discussed a particular something every day in class for three weeks. Student quiz answers give no evidence of this.

Colleagues around the globe consoled me and commiserated. But I forgot that I am also followed by several of my student, and their reaction was more like... panic. Even though I soon followed up with a tweet saying that their Scheme code made me happy, they were alarmed about that first tweet.

It's a new world. I never used to grade with my students watching or to think out loud while I was grading. Twitter changes that, unless I change me and stop using Twitter. On balance, I think I'm still better off. When I got to class, students all had smiles on their faces, some more nervous than others. We chatted. I did my best to calm them. We made a good start on the day with them all focused on the course.

We have reached the end of Week 5, one-third of the way through the course. Despite the alarm I set off in students' minds, they have performed on par with students in recent offerings over the first three homeworks and the first quiz. At this point, I am more concerned with my performance than theirs. After class yesterday, I was keenly aware of the pace of the session being out of sync with the learning curve of material. The places where I've been slowing down aren't always the best places to slow down, and the places where I've been speeding up (whether intentional or unintentional) aren't always the best places to speed up. A chat with one student that afternoon cemented my impression.

Even with years of experience, teaching is hard to get right. One shortcoming of teaching a course only every third semester is that the turnaround time on improvements is so long. What I need to do is use my realization to improve the rest of this offering, first of all this unit on recursive programming.

I spent some time early this week digging into Peter Norvig's Prescient but Not Perfect, a reconsideration of Christopher Strachey's 1966 Sci Am article and in particular Strachey's CPL program to play checkers. Norvig did usual wonderful job with the code. It's hard to find a CPL compiler these days, and has been since about 1980, so he wrote a CPL-to-Python translator, encoded and debugged Strachey's original program, and published the checkers program and a literate program essay that explains his work.

This is, of course, a great topic for a programming languages course. Norvig exemplifies the attitude I encourage to my students on Day 1: if you need a language processor, write one. It's just another program. I am not sure yet when I will bring this topic into my course; perhaps when we first talk in detail about interpreters, or perhaps when we talk about parsing and parser generators. (Norvig uses YAPPS, a Python parser generator, to convert a representation of CPL's grammar into a CPL parser written in Python.)

There are some days when I had designed all of my course sessions to be 60 minutes instead of 75 minutes, so that we had more lüft for opportunistic topics like this one. Or that I could design a free day into the course every 2-3 weeks for the same purpose. Alas, the CS curriculum depends on this course to expose students to a number of important ideas and practices, and the learning curve for some of the material is non-trivial. I'll do my best to provide at least a cursory coverage of Norvig's article and program. I hope that a few students will turn out to his approach to the world -- the computer scientist's mindset.

If nothing else, working through his paper and code excite me, and that will leak over into the rest of my work.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

September 16, 2011 4:13 PM

The Real Technological Revolution in the Classroom Hasn't Happened Yet

Earlier this month, the New York Times ran a long article exploring the topic of technology in K-12 classrooms, in particular the lack of evidence that the mad rush to create the "classroom of future" is having any effect on student learning. Standardized test scores are stagnant in most places, and in schools showing improvements, research has not been able to separate the effect of using technology from the effect of extra teacher training.

We should not be surprised. It is unlikely that simply using a new technology will have any effect on student learning. If we teach the same way and have students do the same things, then we should expect student learning to be about the same whether they are writing on paper, using a typewriter, or typing on a computer keyboard. There are certainly some very cool things one can do with, say, Keynote, and I think having those features available can augment a student's experience. But I doubt that those features can have a substantial effect on learning. Technology is like a small booster rocket for students who are already learning a lot and like training wheels for those who are not.

As I read that article, one fact screamed out at me. Computers are being used in classrooms everywhere for almost everything. Everything, that is, except the one thing that makes them useful at all: computer programming.

After reading the Times piece, Clive Thompson pulled the critical question out of it and asked, What can computers teach that textbooks and paper can't? Mark Guzdial has written thoughtfully on this topic in the past as well. Thompson offers two answers: teaching complexity and seeing patterns, (His third answer is a meta-answer, more effective communication between teacher and student.) We can improve both teaching complexity and seeing patterns by using the right software, but -- thankfully! -- Thompson points out that we can do even better if we teach kids even a little computer programming.

Writing a little code is a great vehicle for exploring a complex problem and trying to create and communicate understanding. Using or writing a program to visualize data and relationships among them is, too.

Of course, I don't have hard evidence for these claims, either. But it is human nature to want to tinker, to hack, to create. If I am going to go out on a limb without data, I'd rather do it with students creating tools that help them understand their world than with students mostly consuming media using canned tools. And I'm convinced that we can expand our students' minds more effectively by showing them how to program than by teaching most of what we teach in K-12. Programming can be tedious, like many learning tasks, and students need to learn how to work through tedium to deeper understanding. But programming offers rewards in a way and on a scale that, say, the odd problems at the end of a chapter in an algebra textbook can never do by themselves.

Mark Surman wrote a very nice blog entry this week, Mozilla as teacher, expressing a corporate vision for educating the web-using public that puts technology in context:

... people who make stuff on the internet are better creators and better online citizens if they know at least a little bit about the web's basic building blocks.

As I've written before, we do future teachers, journalists, artists, filmmakers, scientists, citizens, and curious kids a disservice if we do not teach them a little bit of code. Without this knowledge, they face unnecessary limits on their ability to write, create, and think. They deserve the ability to tinker, to hack, to trick out their digital worlds. The rest of us often benefit when they do, because some of things they create make all of our lives better.

(And increasingly, the digital world intersects with the physical world in surprising ways!)

I will go a step further than Surman's claim. I think that people who create and who are engaged with the world they inhabit have an opportunity to better citizens, period. They will be more able, more willing participants in civic life when they understand more clearly their connections to and dependence on the wider society around them. By giving them the tools they need to think more deeply and to create more broadly, education can enable them to participate in the economy of the world and improve all our lots.

I don't know if K-12 standardized test scores would get better if we taught more students programming, but I do think there would be benefits.

As I often am, I am drawn back to the vision Alan Kay has for education. We can use technology -- computers -- to teach different content in different ways, but ultimately it all comes back to new and better ways to think in a new medium. Until we make the choice to cross over into that new land, we can spend all the money we want on technology in the classroom and not change much at all.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

September 11, 2011 11:27 AM

Using Plain Text to Scrub Spreadsheet Files

For a few years, I have been saving many .xls as plain text. This is handy when I want to program against the data more often than I want to use spreadsheet tools. It also makes the data more accessible across apps and platforms, which is good now and in the future. While doing this, I have came across this technique that I find useful more generally. Maybe you will, too.

People use spreadsheets used for two purposes, structuring data and presenting data. Both Excel and Apple's Numbers offer as much or more functionality for presenting information as they do for manipulating it. For me, the presentation stuff often gets in the way of organizing and manipulating, both by cluttering the UI with commands I don't need to know about and by adding formatting information to my data. The result is a UI more complicated than I need and data files much larger than I need them to be.

When I run into one of those bloated files, I sometimes take a round trip:

  • export the file as .csv or .tsv
  • import that text file back into Numbers or OpenOffice as a spreadsheet file

The result is a clean data file, with the data and its basic structure, but nothing more. No text formatting, no variably spaced rows or columns, and no presentation widgets. When I do work with the data in the spreadsheet app, it's unadorned, just as I like it.


Posted by Eugene Wallingford | Permalink | Categories: Computing

September 09, 2011 2:19 PM

The Future of Your Data

In Liberating and future-proofing your research data, J.B. Deaton describes a recent tedious effort "to liberate several gigabytes of data from a directory of SigmaPlot files". Deaton works in Python, so he had to go through the laborious process of installing an evaluation copy of SigmaPlot, stepping through each SigmaPlot file workbook by workbook, exporting each to Excel, and then converting the files from Excel to CSV or some other format he could process in Python.

Of course, this was time spent not doing research.

All of this would have been a moot point if the data had been stored as CSV or plain text. I can open and process data stored in CSV on any operating system with a large number of tools, for free. And I am confident in 10 years time, I will be able to do the same.

This is a problem we face when we need to work with old data, as Deaton is doing. It's a problem we face when working with current data, too. I wrote recently about how doing a basic task of my job, such as scheduling courses each semester, in a spreadsheet gets in the way getting the job done as well as I might.

Had Deaton not taken the time to liberate his data, things could have been worse in the long run. Not only would the data have been unavailable to his current project, but it may well have fallen into disuse forever and eventually disappeared.

Kari Kraus wrote this week about the problem of data disappearing. One problem is the evolution of media:

When you saved that unpublished manuscript on [a 5-1/4" floppy disk], you figured it would be accessible forever. But when was the last time you saw a floppy drive?

Well, not a 5-1/4". I do have a 3-1/2" USB floppy drive at home and another in my office. But Kraus has a point. Most of the people creating data aren't CS professionals or techno-geeks. And while I do have a floppy drive, I never use them for my own data. Over the years, I've been careful to migrate my data, from floppy drives to zip drives, from CDs to large, replicated hard drives. Eventually it may live somewhere in the cloud, and I will certainly have to move it to the next new thing in hardware sometime in the future.

Deaton's problem wasn't hardware, though, and Kraus points out the bigger problem: custom encoded-data from application software:

If you don't have a copy of WordPerfect 2 around, you're out of luck.

The professional data that I have lost over the years hasn't been "lost" lost. The problem has always been with software. I, too, have occasionally wished I had a legacy of copy of WordPerfect lying around. My wife and I created a lot of files in pre-Windows WordPerfect back in the late 1980s, and I continued to use Mac versions of WP through the 1990s. As I moved to newer apps, I converted most of the files over, but every once in a while I still run across an old file in .wp format. At this point, it is rarely anything important enough to devote the time Deaton spent on his conversion experience. I choose to let that data die.

Fortunately, not all of my data from that era was encoded. I wrote most of grad school papers in nroff. That's also how I created our wedding invitations.

This is a risk we run as more of our world moves from paper to digital, even when it's just entertainment. Fortunately, for the last 5 years or so, I've been storing more and more of my data in plain text or, short of that, rich text. Like Deaton, I am pretty confident that I will be able to read and process that data 10 years hence. And, whenever possible, I have used an open file formats only policy with my colleagues.

Rather than having to liberate data in the future, it is wiser to let it live free from birth. That reduces friction now and later. Deaton offers a set of preferences that can help you keep your data as free as possible:

  • Open source beats closed source.
  • Ubiquitous beats niche software.
  • Automation/scripting beats manual processes.
  • Plain text beats binaries.
  • READMEs in every project directory.

That third bullet is good advice even if you are not a computer scientist. Deaton isn't. But you don't have to be a computer scientist to reap the benefits of a little programming!


Posted by Eugene Wallingford | Permalink | Categories: Computing

August 31, 2011 8:52 PM

Learning to Learn to Automate Work

Jon Udell recently wrote about the real problem with automating work: most of us don't know how. The problem isn't with using particular tools, which come and go, but with recognizing the power of information networks and putting data into a form from which it can be leveraged.

I want to apply his advice more often than I do. I have found that for me it is not enough simply to learn the principles.

The first challenge is fighting an uphill battle against institutional inertia. The university provides too much of its data in dead-tree form, and what data comes to us in digital form comes unstructured. Despite a desire to increase efficiency and decrease costs, a university is a big, slow organization. It takes a long time for its many bureaucracies to re-make themselves. It also takes a persistent, pervasive effort to change on the parts of many people. Too many administrators and faculty thrive in a papered society, which makes change even harder. This is the broad base of people who need to learn Udell's core principles of information networks.

The second challenge is my own habits. I may not be of my students' generation, but I've been in computer science for a long time, and I think I get the value of data. Even still, it's easy -- especially as department head -- to be sucked into institutional habits. My secretary and I are combating this by trying to convert as much data entering our office as possible into live, structured data. In the process, I am trying to teach her, a non-computer scientist, a bit about the principles of data and structured representation. We aren't worrying yet about networks and pub/sub, simply getting data into a named, structured form that supports computational processing.

Yet I need to change some of my own habits, too. When under time pressure, it's easy for me to, say, whip up assignments of graduate assistants to tasks and lab hours on a legal pad. Once the assignments are made, I can communicate a subset of the information in a couple of e-mail messages. The result is a lot of information and not a byte of structured data. Oh, and a lot of lost opportunities for using code to check consistency, make changes, publish the assignments in multiple forms, or reuse the data in adapted form next semester.

My next big opportunity to practice better what I preach is scheduling courses for spring semester. Instead of using spreadsheets as we have in the past, perhaps I should open up Dr. Racket and use it to record all the data we collect and create about the schedule. Scheme's reliance on the simple list as its primary data structure usually puts me in the mindset of grammars and syntax-driven programming. Sometimes, the best way to break a bad old habit is to create a good new one.

So, yes, we need to teach the principles of data networks in a systematic way to information technologists and everyone else. We also need to practice applying them and look for ways to help individuals and institutions alike change their habits.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Managing and Leading

August 18, 2011 3:52 PM

Some Thoughts on "Perlis Languages"

Alan Perlis

Fogus recently wrote a blog entry, Perlis Languages, that has traveled quickly through parts of software world. He bases his title on one of Alan Perlis's epigrams: "A language that doesn't affect the way you think about programming is not worth knowing." Long-time Knowing and Doing readers may remember this quote from my entry, Keeping Up Versus Settling Down. If you are a programmer, you should read Fogus's article, which lists a few languages he thinks might change how you think about programming.

There can be no single list of Perlis languages that works for everyone. Perlis says that a language is worth knowing if it affects how you think about programming. That depends on you: your background, your current stage of development as a programmer, and the kind of problems you work on every day. As an example, in the Java world, the rise of Scala and Clojure offered great opportunities for programmers to expand their thinking about programming. To Haskell and Scheme programmers, the opportunity was much smaller, perhaps non-existent.

The key to this epigram is that each programmer should be thinking about her knowledge and on the look out for languages that can expand her mind. For most of us, there is plenty of room for growth. We tend to work in one or two styles on a daily basis. Languages that go deep in a different style or make a new idea their basic building block can change us.

That said, some languages will show up lots of peoples' Perlis lists, if only because they are so different from the languages most people know and use on a daily basis. Lisp is one of the languages that used to be a near universal in this regard. It has a strangely small and consistent syntax, with symbols as first-order objects, multiple ways to work with functions, and macros for extending the language in a seamless way. With the appearance of Clojure, more and more people are being exposed to the wonders of Lisp, so perhaps won't be on everyone's Perlis list in 10 years. Fogus mentions Clojure only in passing; he has written one of the better early Clojure books, and he doesn't want to make a self-serving suggestion.

I won't offer my own Perlis list here. This blog often talks about languages that interest me, so readers have plenty of chances to hear my thoughts. I will add my thoughts about two of the languages Fogus mentions in his article.

Joy. *Love* it! It's one of my favorite little languages, and one that remains very different from what most programmers know. Scripting languages have put a lot of OOP and functional programming concepts before mainstream programmers across the board, but the idea of concatenative programming is still "out there" for most.

Fogus suggests the Forth programming language in this space. I cannot argue too strongly against this and have explained my own fascination with it in a previous entry. Forth is very cool. Still, I prefer Joy as a first step into the world of concatenative programming. It is clean, simple, and easy to learn. It is also easy to write a Joy interpreter in your favorite language, which I think is one of the best ways to grok a language in a deep way. As I mentioned in the Forth entry linked above, I spent a few months playing with Joy and writing an interpreter for it while on sabbatical a decade ago.

If you play with Joy and like it, you may find yourself wanting more than Joy offers. Then pick up Forth. It will not disappoint you.

APL. Fogus says, "I will be honest. I have never used APL and as a result find it impenetrable." Many things are incomprehensible before we try them. (A student or two will be telling me that Scheme is incomprehensible in the next few weeks...) I was fortunate to write a few programs in APL back in my undergrad programming languages course. I'm sure if I wrote a lot of APL it would become more understandable, but every time I return to the language, it is incomprehensible again to me for a while.

David Ungar told one of my favorite APL stories at OOPSLA 2003, which I mentioned in my report on his keynote address. The punchline of that story fits very well with the theme of so-called Perlis languages: "They could have done the same thing [I] did in APL -- but they didn't think of it!"

There are modern descendants of APL, but I still think there is something special about the language's unique character set. I miss the one-liners consisting or five or twenty Greek symbols, punctuation, and numbers, which accomplished unfathomable tasks such as implementing a set of accounting books.

I do second Fogus's reason for recommending APL despite never having programmed in it: creator Kenneth Iverson's classic text, A Programming Language. It is an unusually lucid account of the design of a programming language -- a new language, not an adaptation of a language we already know. Read it. I had the wonderful opportunity to meet Iverson when he spoke at Michigan State in the 1980s, as described in my entry on Iverson's passing.

... So, I encourage you to follow the spirit of Fogus's article, if not its letter. Find the languages that can change how you think, and learn them. I begin helping a new set of students on this path next week, when we begin our study of Scheme, functional programming, and the key ideas of programming languages and styles.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

August 03, 2011 7:55 PM

Psychohistory, Economics, and AI

Or, The Best Foresight Comes from a Good Model

Hari Seldon from the novel Foundation

In my previous entry, I mentioned re-reading Asimov's Foundation Trilogy and made a passing joke about psychohistory being a great computational challenge. I've never heard a computer scientist mention psychohistory as a primary reason for getting involved with computers and programming. Most of us were lucky to see so many wonderful and more approachable problems to solve with a program that we didn't need to be motivated by fiction, however motivating it might be.

I have, though, heard and read several economists mention that they were inspired to study economics by the ideas of psychohistory. The usual reason for the connection is that econ is the closest thing to psychohistory in modern academia. Trying to model the behavior of large groups of people, and reaping the advantages of grouping for predictability, is a big part of what macroeconomics does. (Asimov himself was most likely motivated in creating psychohistory by physics, which excels at predicting the behavior of masses of atoms over predicting the behavior of individual atoms.)

As you can tell from recent history, economists are no where near the ability to do what Hari Seldon did in Foundation, but then Seldon did his work more than 10,000 years in the future. Maybe 10,00 years from now economists will succeed as much and as well. Like my economist friends, I too am intrigued by economics, which also shares some important features in common with computer science, in particular a concern with the trade-offs among limited resources and the limits of rational behavior.

The preface to the third book in Asimov's trilogy, Second Foundation, includes a passage that caught my eye on this reading:

He foresaw (or he solved his [system's] equations and interpreted its symbols, which amounts to the same thing)...

I could not help but be struck by how this one sentence captured so well the way science empowers us and changes the intellectual world in which we live. Before the rapid growth of science and broadening of science education, the notion of foresight was limited to personal experience and human beings' limited ability to process that experience and generalize accurately. When someone had an insight, the primary way to convince others was to tell a good story. Foresight could be feigned and sold through stories that sounded good. With science, we have a more reliable way to assess the stories we are told, and a higher standard to which we can hold the stories we are told.

(We don't always do well enough in using science to make us better listeners, or better judges of purported foresights. Almost all of us can do better, both in professional settings and personal life.)

As a young student, I was drawn to artificial intelligence as the big problem to solve. Like economics, it runs directly into problems of limited resources and limited rationality. Like Asimov's quote above, it runs directly into the relationship between foresight and accurate models of the world. During my first few years teaching AI, I was often surprised by how fiercely my students defended the idea of "intuition", a seemingly magical attribute of men and women forever unattainable by computer programs. It did me little good to try to persuade them that their belief in intuition and "gut instinct" were outside the province of scientific study. Not only didn't they care; that was an integral part of their belief. The best thing I could do was introduce them to some of the techniques used to write AI programs and to show them such programs behaving in a seemingly intelligent manner in a situation that piqued my students' interest -- and maybe opened their minds a bit.

Over the course of teaching those early AI courses, I was eventually able to see one of the fundamental attractions I had to the field. When I wrote an AI program, I was building a model of intelligent behavior, much as Seldon's psychohistory involved building a model of collective human behavior. My inspiration did not come from Asimov, but it was similar in spirit to the inspiration my economist friends' drew from Asimov. I have never been discouraged or deterred by any arguments against the prospect of artificial intelligence, whether my students' faith-based reasons or by purportedly rational arguments such as John Searle's Chinese room argument. I call Searle's argument "purportedly rational" because, as it is usually presented, ultimately it rests on the notion that human wetware -- as a physical medium -- is capable of representing symbols in a way that silicon or other digital means cannot.

I always believed that, given enough time and enough computational power, we could build a model that approximated human intelligence as closely as we desired. I still believe this and enjoy watching (and occasionally participating in) efforts that create more and more intelligent programs. Unlike many, I am undeterred by the slow progress of AI. We are only sixty years into an enterprise that may take a few thousand years. Asimov taught me that much.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

June 06, 2011 9:12 PM

Magazines, PDF Files, and a Lifetime of Memories

I remember the day I received my first issue of Chess Life & Review magazine. It was the summer of 1979, in late June or early July. I had won a membership in the U.S. Chess Federation as part of a local goodwill tournament, by virtue of beating my good buddy and only competition for the junior prize. My victory entitled me to the membership as $20 of loot, which includes a portable set I use to this day and a notation book that records my games over a period of five or ten years.

Play it again, Sam.

That first issue arrived while I was at summer school. The cover heralded the upcoming U.S. Open championship, but inside the story of Montreal 1979, a super-GM tournament, captivated me with the games of familiar names (Karpov, Tal, Larsen, and Spassky) and new heroes (Portisch, Huuml;bner, Hort, and Ljubojevic). A feature article reviewed films of the 1940s that featured chess and introduced me to Humphrey Bogart's love of and skill at the game I loved to play. Bogart: the man's man, the tough-guy leading man at whose feet women swooned. Bogart! The authors of the magazine's regular columns became my fast friends, and for years thereafter I looked forward monthly to Andy Soltis's fun little stories, which always seemed to teach me something, and Larry Evans's Q-n-A column, which always seemed to entertain.

I was smitten, as perhaps only a young bookish kid can be.

Though I haven't played tournament chess regularly in three decades, I have remained an occasional player, a passionate programmer, and a lovestruck fan. And I've maintained my membership in the USCF, which entitles me to a monthly issue of Chess Life. Though life as husband, father, and professor leave me little time for the game I once played so much, every month I anticipate the arrival of my new issue, replete with new names and new games, tournament reports and feature articles, and regular columns that include Andy Soltis's "Chess to Enjoy". Hurray!

... which is all prelude to my current dilemma, a psychological condition that reveals me a man of my time and not a man of the future, or even the present. It's time to renew my USCF membership, and I am torn: do I opt for the membership that provides on-line access only to Chess Life?

For the last few years, ever since we moved into a new house and I cam face to face with just how much stuff I have, I've been in the process of cutting back. Even before then, I have made some society membership choices based in part on how little I need more piles of paper taking up space in my house and attention in my mind. This is the 21st century, right? I am a computer scientist, who deals daily in digital materials, who has disk space beyond his wildest dreams, whose students have effortlessly grown into a digital world that makes magazines seem like quaint compendia of the past. Right?

Yet I waffle. I can save roughly $7 a year by going paperless, which is a trifle, I know, but a prudent choice nonetheless. Right?

Undoubtedly, my CL&R-turned-CL collection takes up space. If I stem the tide of incoming issues, I can circumscribe the space needed to store my archive and devote future space to more worthy application. Perhaps I could even convert some of the archive into digital form and recoup space already spent?

This move would space, but if I am honest it does not free up all my attention. My magazines will join my music collection in the river of bits flowing into my future, being copied along from storage device to storage device, from medium to medium, and from software application to software application. I've lived through several generations of storage media, beginning in earnest with 5-1/4" floppies, and I'm sure I'll live through several more.

And what of changing formats? The text files that have followed me from college remain readable, for the most part, but not everything survives. For every few files I've converted from WordPerfect for DOS I have surely lost a file or two. Occasionally I run across one and ask myself, is it worth my time to try to open it and convert it to something more modern? I am sad to say that too often the answer is, well, no. This never happens to my books and magazines and pamphlets from that time. I choose to keep or to discard, and if I have it, I can read it. Where will PDF be in 50 years?

the cover of Bobby Fischer's 'My 60 Memorable Games'

I am also just old enough that I somewhat cherish having a life that is separate from my digital existence. When I have the chance to play chess these days, I still prefer to pull out a board and set up the pieces. The feel of the ivory or plastic or wood in my hands is part of the experience -- not essential to the experience, I suppose, in a cosmic sense, but a huge ingredient to my personal experience. I have been playing chess on computers since 1980 or so, which isn't much later than I began playing the game in earnest as in grade school, so I know that feeling, too. But feeling the pieces in my hand, poring over My 60 Memorable Games (another lifelong treasure from the booty that first brought me Chess Life) line by line in search of Bobby Fischer's magic... these are a part of the game for me.

Ultimately, that's where my renewal dilemma lies, too. My memories of checking the mailbox every day at that time of the month, eager to find the next issue of the magazine. The smell of the ink as I thumbed through the pages, peeking ahead at the delights that awaited me. The feel of the pages as I turned to the next column or article or advertisement. The joy of picking out an old issue, grabbing that magnetic portable set from 30-odd years ago, and settling into a comfortable chair for an evening of reminiscence and future-making. All are a part of what chess has been for me. A cache of PDF files, $22 over three years, and a little closet space hardly seem sufficient consideration.

Alas, we are all creatures of our own times, I no less than any man. Even though I know better, I find myself pulled backward in time just as much as Kurt Vonnegut, who occasionally waxed poetic about the future of printed book. Both Vonnegut and I realize that the future may well exceed our imaginations, but our presents retain the gifts of days past.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

May 02, 2011 3:52 PM

Thinking and Doing in the Digital Age

Last week, someone I follow tweeted this link in order to share this passage:

You will be newbie forever. Get good at the beginner mode, learning new programs, asking dumb questions, making stupid mistakes, soliciting help, and helping others with what you learn (the best way to learn yourself).

That blog entry is about inexorable change of technology in then modern world and how, if we want to succeed in this world, we need a mindset that accommodates change. We might even argue that we need a mindset that welcomes or seeks out change. To me, this is one of the more compelling reasons for us to broaden the common definition of the liberal arts to include computing and other digital forms of communication.

As much as I like the quoted passage, I liked a couple of others as much or more. Consider:

Understanding how a technology works is not necessary to use it well. We don't understand how biology works, but we still use wood well.

As we introduce computing and other digital media to more people, we need to balance teaching how to use new ideas and techniques and teaching underlying implementations. Some tools change how we work without us knowing how they work, or needing to know. It's easy for people like me to get so excited about, say, programming that we exaggerate its importance. Not everyone needs to program all the time.

Then again, consider this:

The proper response to a stupid technology is to make a better one yourself, just as the proper response to a stupid idea is not to outlaw it but to replace it with a better idea.

In the digital world as in the physical world, we are not limited by our tools. We can change how our tools work, through configuration files and scripts. We can make our own tools.

Finally, an aphorism that captures differences between how today's youth think about technology and how people my age often think (emphasis added):

Nobody has any idea of what a new invention will really be good for. To evaluate, don't think; try.

This has always been true of inventions. I doubt many people appreciated just how different the world would be after the creation of the automobile or the transistor. But with digital tools, the cost of trying things out has been driven so low, relative to the cost of trying things in the physical world, that the cost is effectively zero. In so many situations now, the net value of trying things exceeds the net value of thinking.

I know that sounds strange, and I certainly don't mean to say that we should all just stop thinking. That's the sort of misinterpretation too many people made of the tenets of extreme programming. But the simple fact is, thinking too much means waiting too long. While you are thinking -- waiting to start -- someone else is trying, learning faster, and doing things that matter.

I love this quote from Elisabeth Hendrickson, who reminded herself of the wisdom of "try; don't think" when creating her latest product:

... empirical evidence trumps speculation. Every. Single. Time.

The scientific method has been teaching us the value of empiricism over pure thought for a long time. In the digital world, the value is even more pronounced.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

April 19, 2011 6:04 PM

A New Blog on Patterns of Functional Programming

(... or, as my brother likes to say about re-runs, "Hey, it's new to me.")

I was excited this week to find, via my Twitter feed, a new blog on functional programming patterns by Jeremy Gibbons, especially an entry on recursion patterns. I've written about recursion patterns, too, though in a different context and for a different audience. Still, the two pieces are about a common phenomenon that occurs in functional programs.

I poked around the blog a bit and soon ran across articles such as Lenses are the Coalgebras for the Costate Comonad. I began to fear that the patterns on this blog would not be able to help the world come to functional programming in the way that the Gang of Four book helped the world come to object-oriented programming. As difficult as the GoF book was for every-day programmers to grok, it eventually taught them much about OO design and helped to make OO programming mainstream. Articles about coalgebras and the costate comonad are certainly of value, but I suspect they will be most valuable to an audience that is already savvy about functional programming. They aren't likely to reach every-day programmers in a deep way or help them learn The Functional Way.

But then I stumbled across an article that explains OO design patterns as higher-order datatype-generic programs. Gibbons didn't stop with the formalism. He writes:

Of course, I admit that "capturing the code parts of a pattern" is not the same as capturing the pattern itself. There is more to the pattern than just the code; the "prose, pictures, and prototypes" form an important part of the story, and are not captured in a HODGP representation of the pattern. So the HODGP isn't a replacement for the pattern.

This is one of the few times that I've seen an FP expert speak favorably about the idea that a design pattern is more than just the code that can be abstracted away via a macro or a type class. My hope rebounds!

There is work to be done in the space of design patterns of functional programming. I look forward to reading Gibbons's blog as he reports on his work in that space.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

April 14, 2011 10:20 PM

Al Aho, Teaching Compiler Construction, and Computational Thinking

Last year I blogged about Al Aho's talk at SIGCSE 2010. Today he gave a major annual address sponsored by the CS department at Iowa State University, one of our sister schools. When former student and current ISU lecturer Chris Johnson encouraged me to attend, I decided to drive over for the day to hear the lecture and to visit with Chris.

Aho delivered a lecture substantially the same as his SIGCSE talk. One major difference was that he repackaged it in the context of computational thinking. First, he defined computational thinking as the thought processes involved in formulating problems so that their solutions can be expressed as algorithms and computational steps. Then he suggested that designing and implementing a programming language is a good way to learn computational thinking.

With the talk so similar to the one I heard last year, I listened most closely for additions and changes. Here are some of the points that stood out for me this time around, including some repeated points:

  • One of the key elements for students when designing a domain-specific language is to exploit domain regularities in a way that delivers expressiveness and performance.
  • Aho estimates that humans today rely on somewhere between 0.5 and 1.0 trillion lines of software. If we assume that the total cost associated with producing each line is $100, then we are talking about a most serious investment. I'm not sure where he found the $100/LOC number, but...
  • Awk contains a fast, efficient regular expression matcher. He showed a figure from the widely read Regular Expression Matching Can Be Simple And Fast, with a curve showing Awk's performance -- quite close to Thompson NFA curve from the paper. Algorithms and theory do matter.
  • It is so easy to generate compiler front ends these days using good tools in nearly every implementation language. This frees up time in his course for language design and documentation. This is a choice I struggle with every time I teach compilers. Our students don't have as strong a theory background as Aho's do when they take the course, and I think they benefit from rolling their own lexers and parsers by hand. But I'm tempted by what we could with the extra time, including processing a more compelling source language and better coverage of optimization and code generation.
  • An automated build system and a complete regression test suite are essential tools for compiler teams. As Aho emphasized in both talks, building a compiler is a serious exercise in software engineering. I still think it's one of the best SE exercises that undergrads can do.
  • The language for quantum looks cool, but I still don't understand it.

After the talk, someone asked Aho why he thought functional programming languages were becoming so popular. Aho's answer revealed that he, like any other person, has biases that cloud his views. Rather than answering the question, he talked about why most people don't use functional languages. Some brains are wired to understand FP, but most of us are wired for, and so prefer, imperative languages. I got the impression that he isn't a fan of FP and that he's glad to see it lose out in the social darwinian competition among languages.

If you'd like to see an answer to the question that was asked, you might start with Guy Steel's StrangeLoop 2010 talk. Soon after that talk, I speculated that documenting functional design patterns would help ease FPs into the mainstream.

I'm glad I took most of my day for this visit. The ISU CS department and chair Dr. Carl Chang graciously invited me to attend a dinner this evening in honor of Dr. Aho and the department's external advisory board. This gave me a chance to meet many ISU CS profs and to talk shop with a different group of colleagues. A nice treat.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

April 06, 2011 7:10 PM

Programming for All: The Programming Historian

Knowing how to program is crucial for doing advanced research with digital sources.

This is the opening line of William Turkel's how-to for digital researchers in the humanities, A Workflow for Digital Research Using Off-the-Shelf Tools. This manual helps such researchers "Go Digital", which is, he says, the future in disciplines such as history. Turkel captures one of the key changes in mindset that faces humanities scholars as they make the move to the digital world:

In traditional scholarship, scarcity was the problem: travel to archives was expensive, access to elite libraries was gated, resources were difficult to find, and so on. In digital scholarship, abundance is the problem. What is worth your attention or your trust?

Turkel suggests that programs -- and programming -- are the only way to master the data. Once you do master the data, you can be an even more productive researcher than you were in the paper-only world.

I haven't read any of Turkel's code yet, but his how-to shows a level of sophistication as a developer. I especially like that his first step for the newly-digital is:

Start with a backup and versioning strategy.

There are far too many CS grads who need to learn the version control habit, and a certain CS professor has been bitten badly by a lack of back-up strategy. Turkel wisely makes this Job #1 for digital researchers.

The how-to manual does not currently have a chapter on programming itself, he does talk about using RSS feeds to do work for you and about and about measuring and refactoring constantly -- though at this point he is talking about one's workflow, not one's programs. Still, it's a start.

As soon as I have some time, I'm going to dig into Turkel's The Programming Historian, "an open-access introduction to programming in Python, aimed at working historians". I think there is a nice market for many books like this.

This how-to pointed me toward a couple of tools I might add to my own workflow. One is Feed43, a web-based tool to create RSS feeds for any web page (an issue I've discussed here before). On first glance, Feed43 looks a little complex for beginners, but it may be worth learning. The manual also reminded me of Mendeley an on-line reference manager. I've been looking for a new tool to manage bibliographies, so I'll give it another look.

But the real win here is a path for historians into the digital world and then into programming -- because it makes historians more powerful at what they do. Superhuman strength for everyone!


Posted by Eugene Wallingford | Permalink | Categories: Computing

March 24, 2011 10:23 PM

Teachers and Programming Languages as Permission Givers

Over spring break, I read another of William Zinsser's essays at The American Scholar, called Permission Givers. Zinsser talks about importance of people who give others permission to do, to grow, and to explore, especially in a world that offers so many freedoms but is populated with people and systems that erect barriers at every turn.

My first reaction to the paper was as a father. I have recognized our elementary and high schools as permission-denying places in a way I didn't experience them as a student myself, and I've watched running the gauntlet of college admissions cause a bright, eager, curious child to wonder whether she is good enough after all. But my rawest emotions were fear and hope -- fear that I had denied my children permission too often, and hope that on the whole I had given them permission to do what they wanted to do and become who they can be. I'm not talking about basic rules; some of those are an essential part of learning discipline and even cultivating creativity. I mean encouraging the sense of curiosity and eagerness that happy, productive people carry through life.

The best teachers are permission givers. They show students some of what is possible and then create conditions in which students can run with ideas, put them together and take them apart, and explore the boundaries of their knowledge and their selves. I marvel when I see students creating things of beauty and imagination; often, there is a good teacher to be found there as well. I'm sad whenever I see teachers who care deeply about students and learning but who sabotage their students' experience by creating "a long trail of don'ts and can'ts and shouldn'ts", by putting subtle roadblocks along the path of advancement.

I don't think that by nature I am permission giver, but over my career as a teacher I think I've gotten better. At least now I am more often aware of when I'm saying 'no' in subtle and damaging ways, so that I can change my behavior, and I am more often aware of the moments when the right words can help a student create something that matters to them.

In the time since I read the essay, another strange connection formed in my mind: Some programming languages are permission givers. Some are not.

Python is a permission giver. It doesn't erect many barriers that get in the way of the novice, or even the expert, as she explores ideas. Ruby is a permission giver, too, but not to the extent that Python is. It's enough more complex syntactically and semantically that things don't always work the way one first suspects. As a programmer, I prefer Ruby for the expressiveness it affords me, but I think that Python is the more empowering language for novices.

Simplicity and consistency seem to be important features of permission-giving languages, but they are probably not sufficient. Another of my favorite languages, Scheme, is simple and offers a consistent model of programming and computation, but I don't think of it as a permission giver. Likewise Haskell.

I don't think that the tired argument between static typing and dynamic typing is at play here. Pascal had types but it was a permission giver. Its descendant Ada, not so much.

I know many aficionados of other languages often feel differently. Haskell programmers will tell me that their language makes them so productive. Ada programmers will tell me how their language helps them build reliable software. I'm sure they are right, but it seems to me there is a longer learning curve before some languages feel like permission givers to most people.

I'm not talking about type safety, power, or even productivity. I'm talking about the feeling people have when they are deep in the flow of programming and reach out for something they want but can't quite name... and there it is. I admit, too, that I also have beginners in mind. Students who are learning to program, more than experts, need to be given permission to experiment and persevere.

I also admit that this idea is still new in mind and is almost surely colored heavily by my own personal experiences. Still, I can't shake the feeling that there is something valuable in this notion of language as permission giver.

~~~~

If nothing else, Zinsser's essay pointed me toward a book I'd not heard of, Michelle Feynman's Reasonable Deviations from the Beaten Track, a collection of the personal and professional letters written by her Nobel Prize-winning father. Even in the most mundane personal correspondence, Richard Feynman tells stories that entertain and illuminate. I've only begun reading and am already enjoying it.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

March 22, 2011 4:45 PM

Encounters with Large Numbers and the Limits of Programs

Yesterday, this xkcd illustration of radiation doses made the round of Twitter. My first thought was computational: this graph is a great way to help students see what "order of magnitude" means and how the idea matters to our understanding of a real-world phenomenon.

Late yesterday afternoon, one of my colleagues stopped by to describe a Facebook conversation he had been having with a few of our students, and in particular one of out better students. This student announced that he was going to write a program to generate all possible brackets for the men's NCAA basketball tournament. My colleague said, "Um, there are 2 to the 67th power brackets", to which the student responded, "Yeah, I know, that's why I'm going to write a program. There are too many to do by hand." From this followed a discussion of just how big 2**67 is and how long it would take a program to generate all the brackets. Even using a few heuristics to trim the problem down, such as always picking a 1-seed to beat a 16-seed, the number is astronomically large. (Or, as Richard Feynman suggests, "economically large".)

Sometimes, even good students can gain a better understanding of a concept by encountering it in the wild. This is perhaps even more often true when the idea is unintuitive or beyond our usual experience.


Posted by Eugene Wallingford | Permalink | Categories: Computing

March 10, 2011 9:21 PM

SIGCSE Day 2 -- Limited Exposure

For a variety of reasons, I am scheduled for only two days at SIGCSE this year. I did not realize just how little time that is until I arrived and started trying to work in all the things I wanted to do: visit the exhibits, attend a few sessions and learn a new thing or two, and -- most important -- catch up with several good friends.

It turns out that's hard to do in a little more than a day. Throw in a bout of laryngitis in the aftermath of a flu-riddled week, and the day passed even more quickly. Here are a few ideas that stood out from sessions on either end of the day.

Opening Keynote Address

Last March I blogged about Matthias Felleisen winning ACM's Outstanding Educator Award. This morning, Felleisen gave the opening address for the conference, tracing the evolution of his team's work over the last fifteen years in a smooth, well-designed talk. One two-part idea stood out for me: design a smooth progression of teaching languages that are neither subset nor superset of any particular industrial-strength language, then implement them, so that your tools can support student learning as well as possible.

Matthias's emphasis on the smooth progression reminds me of Alan Kay's frequent references to the fact that English-speaking children learn the same language used by Shakespeare to write our greatest literature, growing into it over time. One of his goals for Smalltalk, or whatever replaces it, is a language that allows children to learn programming and grow smoothly into more powerful modes of expression as their experience and cognitive skills grow.

Two Stories from Scratch

At the end of the day, I listened in on a birds-of-a-feather session about Scratch, mostly in K-12 classrooms. One HS teacher described how his students learn to program in Scratch and then move onto a "real language". As they learn concepts and vocabulary in the new language, he connects the new terms back to their concrete experiences in Scratch. This reminded me of a story in one of Richard Feynman's books, in which he outlines his father's method of teaching young Richard science. He didn't put much stock in learning the proper names of things at first, instead helping his son to learn about how things work and how they relate to one another. The names come later, after understanding. One of the advantages of a clean language such as Scratch (or one of Felleisen's teaching languages) is that it enables students to learn powerful ideas by using them, not by memorizing their names in some taxonomy.

Later in the session, Brian Harvey told the story of a Logo project conducted back in the 1970s, in which each 5th-grader in a class was asked to write a Logo program to teach a 3rd-grader something about fractions. An assignment so wide open gave every student a chance to do something interesting, whatever they themselves knew about fractions. I need to pull this trick out of my teaching toolbox a little more often.

(If you know of a paper about this project, please send me a pointer. Thanks.)

~~~~

There is one unexpected benefit of a short stay: I am not likely to leave any dynamite blog posts sitting in the queue to be written, unlike last year and 2008. Limited exposure also limits the source of triggers!


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

March 09, 2011 11:31 PM

SIGCSE Day 1 -- Innovative Approaches for Introducing CS

SIGCSE 2011 in Dallas, Texas

I'm in Dallas for a couple of days for SIGCSE 2011. I owe my presence to Jeff Forbes and Owen Astrachan, who organized a pre-conference workshop on innovative approaches for introducing computer science and provided support for its participants, courtesy of their NSF projects.

The Sheraton Dallas is a big place, and I managed to get lost on the way to the workshop this morning. As I entered the room fifteen minutes late, Owen was just finishing up talking about something called the Jinghui Rule. I still don't know what it is, but I assume it had something to do with us not being able to use our laptops during much of the day. This saves you from reading a super-long breakdown of the day, which is just as well. The group will produce a report soon, and I'm sure Jeff and Owen will do more complete job than I might -- not least of which because we all produced summaries of our discussion throughout the day, presented them to the group as a whole, and submitted them to our leaders for their use.

The topics we discussed were familiar ones, including problems, interdisciplinary approaches, integrative approaches, motivating students, and pedagogical issues. Even still, the discussions were often fresh, as most everyone in the room wrestles with these topics in the trenches and is constantly trying new things.

I did take a few notes the old-fashioned way about some things that stood out to me:

  • Owen captured the distinction between "interdisciplinary" and "integrative" well; here is my take. Interdisciplinary approaches pull ideas from other areas of study into our CS courses as a way to illustrate or motivate ideas. Integrative approaches push CS techniques out into courses in other areas of study where they become a native part of how people in those disciplines work.
  • Several times during the day people mentioned the need to "document best practices" of various sorts. Joe Bergin was surely weeping gently somewhere. We need more than disconnected best practices; we need a pattern language or two for designing certain kinds of courses and learning experiences.
  • Several times during the day talk turned to what one participant termed student-driven discovery learning. Alan Kay's dream of an Exploratorium never strays far from my mind, especially when we talk about problem-driven learning. We seem to know what we need to do!
  • A group of us discussed problems and big data in a "blue sky" session, but the talk was decidedly down-to-earth: the need to format, sanitize, and package data sets for use in the classroom.
  • One of the biggest challenges we face is the invisibility of computing today. Most everyone at the workshop today views computing's ubiquity as a great opportunity, and I often feel the same way. But I fear the reality is that, for most everyone else, computing has disappeared into the background noise of life. Convincing them that it is cool to understand how, say, Facebook works may be a tougher task than we realize.

Finally, Ge Wang demoed some of the cool things you can do with an iPhone using apps like those from Smule. Wow. That was cool.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

February 24, 2011 4:17 PM

What are Java, C++, and Prolog?

That's the correct question for the clue, "The languages used to write Watson", courtesy of the IBM Watson team answering questions in the usual way on Reddit. I'm looking forward to learning more about how Watson operates at the software level, especially how it handles the uncertainty of multiple knowledge sources working in parallel on imprecise data.

I meet with prospective students and their parents frequently as part of on- and off-campus recruiting events. Last week and this are Up Close Days on campus, when students who have been admitted to the university but not yet accepted the offer visit in hopes of making their decision. I usually try to use part of our time together to help them see what computer science is and what graduates of our program can do, because so few of either the students or parents have any idea. With the match on my mind for teaching and research, it occurred to me that the timing of the match offered a perfect opportunity.

Here was a story that everyone had heard about, the kind that captures the attention of even the least technically minded among us: a battle of man versus machine in an arena that has historical been ours alone. Watson is not only a triumph of computer science; it covers the full spectrum of what we do, from racks of servers and an open-source operating system, through algorithms and data structures and programs that embody them, to the dreams of artificial intelligence and the promise of applications that help people all over the world. So I built a short presentation around the match, showing short clips from the show, offering commentary at key moments, taking wide-eyed questions at any time, and tying what the program was doing to what computer science is and what computer science graduates do.

Last week's first session went very well. If nothing else, all of our visitors could see my excitement at what we do and what remains to be done. Sharing the thrill, indeed!

Yesterday, Lance Fortnow tweeted:

Asked: What is Computer Science? Answered: Everything that happens after you ask a question to Google until you get a result.

I don't know what all the future implications of Watson's performance will be for AI, CS, or the world. I do know that, for now, the match has given us a compelling way to talk to others about computer science ripped right from the headlines and our TV screens. When we can connect our story to something like Google, we have a chance of helping others to get what we do. When the story connects to something that has already excited our audience, our job is perhaps a little bit easier.


Posted by Eugene Wallingford | Permalink | Categories: Computing

February 18, 2011 4:51 PM

The Dawn of a New Age

Jeopardy! champ Ken Jennings passes the baton to Watson

My world this week has been agog with the Watson match on Jeopardy!. The victory by a computer over the game's two greatest champions may well signal a new era, in much the same way as the rise of the web and the advent of search. I'd like to collect my thoughts before writing anything detailed about the match itself.

In anticipation of match, last week my Intelligent Systems students and I began to talk about some of the techniques that were likely being used by Watson under the hood. When I realized how little they had learned in their AI course about reasoning in the face of uncertainty, I dusted off an old lecture from my AI course, circa 2001. With a few tweaks and additions, it held up well. Among its topics were probability and Bayes' Law. In many ways, this material is more timely today than it was then.

Early in the week, as I began to think about what this match would mean for computing and for the world, I was reminded by Peter Norvig's The Machine Age that, in so many ways, the Jeopardy! extravaganza heralds a change already in progress. If you haven't read it yet, you should.

The shift from classical AI to the data-driven AI that underlies the advances Norvig lists happened while I was in graduate school. I saw glimpses of it at conferences on expert systems in finance and accounting, where the idea of mining reams of data seemed to promise new avenues for decision makers in business. The data might be audited financial statements of public corporations or, more tantalizing, collected from grocery store scanners. But I was embedded pretty deeply in a particular way of thinking about AI, and I missed the paradigm shift.

What is most ironic for me is that my own work, which involved writing programs that could construct legal arguments using both functional knowledge of a domain and case law, was most limited by the problem that we see addressed in systems like Google and Watson: the ability to work with the staggering volume of text that makes up our case law. Today's statistical techniques for processing language make extending my work and seeing how well it works in large domains possible in a way I could only dream of. And I never even dreamed of doing it myself, such was my interest in classical AI. (I just needed a programmer, right?)

There is no question that data-driven, statistical AI has proven to be our most successful way to build intelligent systems, particularly those of a certain scale. As an engineering approach, it has won. It may well turn out to be the best scientific approach to understanding intelligence as well, but... For an old classicist like me, there is something missing. Google is more idiot savant than bon vivant; more Rain Man than Man on the Street.

There was a telling scene in one of the many short documentary films about Watson in which the lead scientist on the project said something like, "People ask me if I know why Watson gave the answer it did. I don't how it got the answer right. I don't know how it got the answer wrong." His point was that Watson is so complex and so data-rich it can surprise us with its answers. This is true, of course, and an important revelation to many people who think that, because a computer can only do what we program it to do, it can never surprise or create.

But immediately I was thinking about another sense of that response. When Watson asks "What is Toronto? in a Final Jeopardy! on "U.S. Cities", I want to ask it, "What in the world were you thinking?" If it were a human player, it might be able to tell about its reasoning process. I'd be able to learn from what it did right or wrong. But if I ask most computer programs based on statistical computations over large data sets "Why?" I can't get much more than the ranked lists of candidates we saw at the bottom of the screen on Jeopardy!

There is still a romantic part of me that wants to understand what it means to think and reason at a conscious level. Perhaps my introspection misleads me with explanations constructed post hoc, but it sure seems like I am able to think at a level above my neurons firing. That feeling is especially strong when I perform more complex tasks, such as writing a program or struggling to understand a new idea.

So, when it comes to the science of AI, I still hope for more. Maybe our statistical systems will become complex and data-rich enough that they will be able to explain their reasoning in a meaningful way. It's also possible that my desire is nothing more than a form of chauvinism for my own species, that the desire to sit around and talk about stuff, including how and why we think the way we do, is a quaint feature peculiar to humans. I don't know the answer to this question, but I can't shake the deep belief in an architecture of thought and intelligent behavior that accounts for metacognition.

In any case, it was fun to watch Watson put on such an impressive show!

~~~~

If you are interested in such things, you might want to (re-)read Newell and Simon's 1975 Turing Award lecture, Computer Science as Empirical Inquiry: Symbols and Search. It's a classic of the golden era of AI.


Posted by Eugene Wallingford | Permalink | Categories: Computing

February 11, 2011 4:15 PM

DIY Empirical Analysis Via Scripting

In my post on learning in a code base, I cited Michael Feathers's entry on measuring the closure of code. Michael's entry closes with a postscript:

It's relatively easy to make these diagrams yourself. grep the log of your VCS for lines that depict adds and modifications. Strip everything except the file names, sort it, run it through 'uniq -c', sort it again, and then plot the first column.

Ah, the Unix shell. A few years ago I taught a one-unit course on bash scripting, and I used problems like this as examples in class. Many students are surprised to learn just how much you can do with a short pipeline of Unix commands, operating on plain text data pulled from any source.

You can also do this sort of thing almost as easily in a more full-featured scripting language, such as Python or Ruby. That is one reason languages like them are so attractive to me for teaching programming in context.

Of course, using a powerful, fun language in CS1 creates a new set of problems for us. A while back, a CS educator on the SIGCSE mailing list pointed out one:

Starting in Python postpones the discovery that "CS is not for me".

After years of languages such as C++, Java, and Ada in CS1, which hastened the exit of many a potential CS major, it's ironic that our new problem might be students succeeding too long for their own good. When they do discover that CS isn't for them, they will be stuck with the ability to write scripts and analyze data.

With all due concern for not wasting students' time, this is a problem we in CS should willingly accept.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

February 10, 2011 4:04 PM

This and That: Problems, Data, and Programs

Several articles caught my eye this week which are worth commenting on, but at this point none has triggered a full entry of its own. Some of my favorite bloggers do what they call "tab sweeps", but I don't store cool articles in browser tabs. I cache URLs and short notes to myself. So I'll sweep up three of my notes as a single entry, related to programming.

Programmer as Craftsman

Seth Godin writes about:

... the craftsperson, someone who takes real care and produces work for the ages. Everyone else might be a hack, or a factory guy or a suit or a drone, but a craftsperson was someone we could respect.

There's a lot of talk in the software development world these days about craftsmanship. All the conversation and all the hand-waving boil down to this. A craftsman is the programmer we all respect and the programmer we all want to be.

Real Problems...

Dan Meyer is an erstwhile K-12 math teacher who rails against the phony problems we give kids when we ask them to learn math. Textbooks do so in the name of "context". Meyer calls it "pseudocontext". He gives an example in his entry Connect These Two Dots, and then explains concisely what is wrong with pseudocontext:

Pseudocontext sends two signals to our students, both false:
  • Math is only interesting in its applications to the world, and
  • By the way, we don't have any of those.

Are we really surprised that students aren't motivated to practice and develop their craft on such nonsense? Then we do the same things to CS students in our programming courses...

... Are Everywhere These Days

Finally, Greg Wilson summarizes what he thinks "computational science" means in one of his Software Carpentry lessons. It mostly comes down to data and how we understand it:

It's all just data.

Data doesn't mean anything on its own -- it has to be interpreted.

Programming is about creating and composing abstractions.

...

The tool shapes the hand.

We drown in data now. We collect faster than we can understand it. There is room for more programmers, better programmers, across the disciplines and in CS.

We certainly shouldn't be making our students write Fahrenheit-to-Celsius converters or processing phony data files.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

February 07, 2011 9:03 PM

Teaching and Learning in a Code Base

In a pair of tweets today, Brian Marick offered an interesting idea for designing instruction for programmers:

A useful educational service: examine a person's codebase. Devise a new feature request that would be hard, given existing code and skill...

... Keep repeating as the codebase and skill improve. Would accelerate a programmer's skill at dealing with normal unexpected change.

This could also be a great way to help each programmer develop competencies that are missing from his or her skill set. I like how this technique would create an individualized learning for each student. The cost, of course, is in the work needed by the instructor to study the codebases and devise the feature requests. With a common set of problems to work on, over time an instructor might be able to develop a checklist of (codebase characteristic, feature request) pairs that covered a lot of the instructional space. This idea definitely deserves some more thought!

Of course, we can sometimes analyze valuable features of a codebase with relatively simple programs. Last month, Michael Feathers blogged about measuring the closure of code, in which he showed how we can examine the Open/Closed Principle in a codebase by extracting and plotting the per-file commit frequencies of source files in a project's version control repository. Feathers discussed how developers could use this information intentionally to improve the quality of their code. I think this sort of analysis could be used to great effect in the classroom. Students could see the OCP graphically for a number of projects and, combined with their programming knowledge of the projects, begin to appreciate what the OCP means to a programmer.

A serendipitous side effect would be for students to experience CS as an empirical discipline. This would help us prepare developers more readily in sync with Feathers's use of analytical data in their practice and CS grads who understand the ways in which CS can and should be an empirical endeavor.

I actually blogged a bit about studying program repositories last semester, for the purpose of understanding how to design better programming languages. That work used program repositories for research purposes. What I like about Marick's and Feathers's recent ideas is that they bring to mind how studying a program repository can aid instruction, too. This didn't occur to me so much back when one of my grad students studied relationships among open-source software packages with automated analysis of a large codebase. I'm glad to have received a push in that direction now.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

February 03, 2011 3:30 PM

Science and Engineering in CS

A long discussion on the SIGCSE members listserv about math requirements for CS degrees has drifted, as most curricular discussions seem to do, to "What is computer science?" Somewhere along the way, someone said, "Computer Science *is* a science, by name, and should therefore be one by definition". Brian Harvey responded:

The first thing I tell my intro CS students is "Computer Science isn't a science, and it isn't about computers." (It should be called "information engineering.")

I think that this assertion is wrong, at least without a couple of "only"s thrown in, but it is a great way to start a conversation with students.

I've been seeing the dichotomy between CS as science and CS as system-building again this semester in my Intelligent course. The textbook my students used in their AI course last semester is, like nearly every undergrad AI text, primarily an introduction to the science of AI: a taxonomy of concepts, results of research that help to define and delimit the important ideas. It contains essentially no pragmatic results for building intelligent systems. Sure, students learn about state-space search, logic as a knowledge representation, planning, and learning, along with algorithms for the basic methods of the field. But they are not prepared for the fact that, when they try to implement search or logical inference for a given problem, they still have a huge amount of work to do, with little guidance from the text.

In class today, we discussed this gap in two contexts: the gap one sees between low-level programming and high-level programming languages, and the difference between general-purpose languages and domain-specific languages.

My students seemed to understand my point of view, but I am not sure they really grok it. That happens best after they gain experience writing code and feel the gap while making real systems run. This is one of the reasons I'm such a believer in projects, real problems, and writing code. We don't always understand ideas until we see them in a concrete.

I don't imagine that intro CS students have any of the experience they need to understand the subtleties academics debate about what computer science is or what computer scientists. We are almost surely better off asking them to do something that matters them, whether a small problem or a larger project. In these problems and projects, students can learn from us and from their own work what CS is and how computer scientists think.

Eventually, I hope that the students writing large-ish AI programs in my course this semester learn just how much more there is to writing an intelligent system than just implementing a general-purpose algorithm from their text. The teams that are using pre-existing packages as part of they system might even learn that integrating software systems is "more like performing a heart transplant than snapping together LEGO blocks". (Thanks to John Cook for that analogy.)


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

January 31, 2011 4:12 PM

Side Thoughts on Tasks, Methods, and Intelligent Systems

I've been enjoying teaching Intelligent Systems again this semester. A big part of the fun is re-reading old papers. I had a lot of fun re-reading and thinking about the ideas in William Clancey's classic paper, Heuristic Classification. Long-time readers may recall me mentioning this paper in a long-ago post about one of computing's basic concepts, the unity of data and program.

My students didn't read this paper, but they did read a chapter from Peter Jackson's Introduction to Expert Systems that discusses it in some detail. The chapter is called "Heuristic Classification", too, but it's really about much more: the distinction between tasks and methods, the distinction between analytic tasks and synthesis tasks, and Clancey's heuristic classification method itself.

This one paper set me off on a mini-lecture that could have been much longer. To me, it felt like a technical reminiscence, a story of ideas that set the stage for my own work on design patterns of knowledge-based systems. Eventually, I had to cut my story short, both in the interest of having time to do other things in class that day and in the interest of not boring students with my own deep fascination with these topics. I hope I said enough to give them a sense of how important these ideas are and to whet the appetite of any student who might have his or her own fascination with the ideas.

Our reading and our conversation that day brought to my mind a rich set of ideas for students of intelligent systems to think about, such as:

  • The distinction between task (what) and method (how) is, at one level, no big deal. Students begin learning about this in their intro course, see its idea more clearly in their data structures course, and study the lifecycle issues involved with both in Software Engineering. But understanding the specification for a software system is often a challenge; it's easy to focus on the wrong task. The problem is made more difficult in many AI software projects because we are working in domains we understand less well, on problems at the edge of our intellectual abilities.
  • Many problems in the world are really composites of other, more primitive tasks. For example, a tutoring system must diagnose "bugs" in a student's thinking, develop a plan to fix the bug, and monitor student performance while solving problems. When we decompose a task in this way, we are in effect talking about a method for performing the task, but at a level that commits only to the exchange of information among high-level modules.
  • Where do learning systems fit into our classification of tasks. In one sense, a learning system "goes meta", because it creates a system capable of performing one of the tasks in the taxonomy. In another, though, it, too, is performing a task. But which one?

Many of these go beyond my current interests precisely because they are at the beginning of their studies, focused on building a particular system. They may find their interest in these ideas piqued after they build one of these systems, or three, and realize just how many degrees of freedom they face when writing a program that behaves "intelligently".


Posted by Eugene Wallingford | Permalink | Categories: Computing

December 15, 2010 4:30 PM

You May Be A Programmer If ... #1

... you get to the code generation part of the compiler course you are teaching. You realize that you have forgotten the toy assembly language from the textbook for its toy virtual machine, so you have to relearn it. You think, "It's gonna be fun to write programs in this language again."

Assembly language? A toy assembly language? Really?

I just like to program. Besides, after programming in Klein this semester, a language I termed as an integer assembly language, moving to a very RISC assembly doesn't seem like that big a step down. Assembly language can be fun, though I don't think I'd want to program in it for a living!


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

December 10, 2010 3:43 PM

Mean Time to Competence and Mean Time to Mastery

I'm on a mailing list with a bunch of sports fans, many of whom are also CS academics and techies. Yesterday, one of the sysadmins posted a detailed question about a problem he was having migrating a Sun Solaris server to Ubuntu, due to a conflict between default /etc/group values on the two operating systems. For example, the staff group has a value of 50 on Ubuntu and a value of 10 on Solaris. On Ubuntu, the value 10 identifies the uucp group.

Another of the sysadmins on the list wrote an even more detailed answer, explaining how group numbers are supposed to work in Unix, describing an ideal solution, and outlining a couple of practical approaches involving files such as /etc/nsswitch.conf.

After I read the question and the answer, I sent my response to the group:

Thank goodness I'm a CS professor and don't have to know how all this works.

I was joking, of course. Some people love to talk about how CS profs are disconnected from computing in the real world, and this is the sort of real-world minutia that CS profs might not know, or even have to know if they teach courses on algorithms, intro programming, or software engineering. After seeing my friends' exchange and seeing all that Unix guru-speak, I was happy to play to the stereotype.

Of course, the numbers used to implement Unix group numbers really are minutia and something only practicing sysadmins would need to know. The professor who teaches our systems courses is deeply versed in these details, as are the prof or two who manage servers for their courses and research. There certainly are CS academics divorced from reality, but you can say that of any group of people. Most know what they need to know, and a bit more.

Later in the day, I became curious about the problem my friends had discussed, so I dug in, studied a bit, and came to understand the problem and candidate solutions. Fun stuff.

Our department has recently been discussing our curriculum and in particular the goals of our B.A. and B.S. programs: What are the desired outcomes of each program? That is jargon for the simpler, "When students graduate, what would we like for them to be able to do?" For departments like ours, that means skills such as being able to write programs of a certain size, design a database, and choose appropriate data structures for solving a specific problem.

I was thinking about the Unix exchange on my mailing list in this context. Let's be honest... There is a lot of computer science, especially CS as it is applied in specific technology, that I don't know. Should I have known to fix my friend's problem? Should our students? What can we reasonably expect of our students or of ourselves as faculty?

Obviously, we professors can't know everything, and neither can our students. That is true in any discipline and especially in one like CS, which changes and grows so fast. This is one of the reasons it is so important for us to define clearly what we expect our programs to achieve. The space of computing knowledge and skills is large and growing. Without a pretty good idea of what we are hoping to achieve with our courses and curricula, our students could wander around in the space aimlessly and not having anything coherent to show for the time or effort. Or the money they spent paying tuition.

So, when I first read my friends' messages about Unix groups, I didn't know how to solve the problem. And that's okay, because I can't know everything about CS, let alone every arcane detail of every Unix distro out in the world. But I do have a CS degree and lots of experience. What difference should that make when I approach problems like this? If one of our graduates confronts this situation or one like it, how will they be difference from the average person on the street, or even the average college graduate?

Whatever specific skills our graduates have, I think that they should be able to come up to speed on computing problems relatively quickly. They should have enough experience with a broad set of CS domains and enough theoretical background to be able to make sense of unfamiliar problems and understand candidate solutions. They should be able to propose solutions that make sense to a computer scientist, even if the solutions lack a detailed knowledge of the domain.

That is, CS graduates should have a relatively low mean time to competence in most sub-areas of computing, even the ones they have not studied in detail yet.

For a smaller set of sub-areas, our students should also have a relatively low mean time to mastery. These are the areas they have studied in some detail, either in class or through project work, but which they have not yet mastered. A CS degree should put them in a position to master them more quickly than most educated non-computer scientists.

Mean time to competence (MTTC) and mean time to mastery (MTTM) are actually a big part of how I distinguish a university education from a community college education when I speak to prospective students and their parents, though I have never used those terms before. They always wonder about the value of technical certifications, which community college programs often stress, and why my department does not make study for certification exams an explicit goal for students.

We hope, I tell them, to put the student in a position of being ready to prepare for any certification exam in relatively short order, rather than spending a couple of years preparing them to take a specific exam. We also hope that the knowledge and experience they gain will prepare them for the inevitable developments in our discipline that will eventually make any particular certification obsolete.

I am not certain if mean time to competence and mastery are student learning outcomes in the traditional educational jargon sense of the word, or whether they are abstractions of several more concrete outcomes. In any case, I am left thinking about how we can help to create these outcomes in students and how we can know whether we are successful or not. (The agile developer in me says, if we can't figure out how to test our program, we need to break the feature down into smaller, more concrete steps.)

Whatever the practical challenges of curriculum design and outcomes, I think MTTC and MTTM are essential results for a CS program to generate. Indeed, they are the hallmark of a university education in any discipline and of education in general.

Now, to figure out how to do that.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

November 30, 2010 5:08 PM

No More Bugs

Maurice Wilkes, one of computer science's pioneers, passed away yesterday at the grand age of 97. Wilkes spent most of his career working on computer systems, so I never studied his work in the detail that I studied Backus's or Floyd's. He is perhaps best known as the creator of the EDSAC, the first practical (if slow!) stored program computer, in May 1949. Wilkes also had an undeniable impact on programming as the primary developer of the idea of microprogramming, whereby the CPU is controlled by a program stored in ROM.

Finally, every programmer must hold a special place in his or her heart for this, Wilkes's most quoted aphorism:

... the realization came over me that a good part of my life was going to be spent finding errors in my own programs.

All programmers have their Wilkes Moment sooner or later.


Posted by Eugene Wallingford | Permalink | Categories: Computing

November 19, 2010 4:45 PM

Debugging the Law

Debugging the Law

Recently, Kevin Carey's Decoding the Value of Computer Science got a lot of play among CS faculty I know. Carey talks about how taking a couple of computer programming courses way back at the beginning of his academic career has served him well all these years, though he ended up majoring in the humanities and working in the public affairs sector. Some of my colleagues suggested that this article gives great testimony about the value of computational thinking. But note that Carey didn't study abstractions about computation or theory or design or abstraction. He studied BASIC and Pascal. He learned computer programming.

Indeed, programming plays a central role in the key story within the story. In his first job out of grad school, Carey encountered a convoluted school financing law in my home state of Indiana. He wrote code to simulate the law in SAS and, between improving his program and studying the law, he came to understand the convolution so well that he felt confident writing a simpler formula "from first principles". His formula became the basis of an improved state law.

That's right. His code was so complicated and hard to maintain, he through the system away and wrote a new one. Every programmer has lived this experience with computer code. Carey tried to debug a legal code and found its architecture to be so bad that he was better off creating a new one.

CS professors should use this story every time they try to sell the idea of universal computer programming experience to the rest of the university!

The idea of refactoring legal code via a program that implements it is not really new. When I studied logic programming and Prolog in grad school, I read about the idea of expressing law as a Prolog programming and using the program to explore its implications. Later, I read examples where Prolog was used to do just that. The AI and law community still works on problems of this sort. I should dig into some of the recent work to see what progress, if any, has been made since I moved away from that kind of work.

My doctoral work involved modeling and reasoning about legal arguments, which are very much like computer programs. I managed to think in terms of argumentation patterns, based on the work of Stephen Toulmin (whose work I have mentioned here before). I wish I had been smart or insightful enough to make the deeper connection from argumentation to software development ideas such as architecture and refactoring. It seems like there is room for some interesting cross-fertilization.

(As always, if you know about work in this domain, please let me know!)


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

November 18, 2010 3:43 PM

The Will to Run, or Do Anything Else

In "How Do You Do It?", an article in the latest issue of Running Times about how to develop the intrinsic motivation to do crazy things like run every morning at 5:00 AM, ultrarunner Eric Grossman writes:

The will to run emerges gradually where we cultivate it. It requires humility -- we can't just decide spontaneously and make it happen. Yet we must hold ourselves accountable for anything about which we can say, "I could have done differently."

Cultivation, humility, patience, commitment, accountability -- all features of developing the habits I need to run on days I'd rather stay in bed. After a while, you do it, because that's what you do.

I think this paragraph is true of whatever habit of thinking an doing that you are trying to develop, whether it's object-oriented programming, playing piano, or test-driven design.

~~~~

Eugene speaking at Tech Talk Cedar Valley, 2010/11/17

Or functional programming. Last night I gave a talk at Tech Talk Cedar Valley, a monthly meet-up of tech and software folks in the region. Many of these developers are coming to grips with a move from Java to Scala and are peddling fast to add functional programming style to their repertoires. I was asked to talk about some of the basic ideas of functional programming. My talk was called "Don't Drive on the Railroad Tracks", referring to Bill Murray's iconic character in the movie Groundhog Day. After hundreds or thousands of days reliving February 2 from the same starting point, Phil Connors finally comes to understand the great power of living in a world without side effects. I hope that my talk can help software developers in the Cedar Valley reach that state of mind sooner than a few years from now.

If you are interested, check out the slides of the talk (also available on SlideShare) and the code. both Ruby and Scheme, that I used to illustrate some of the ideas.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

November 13, 2010 9:28 AM

Shape, or Be Shaped

Playwright Arthur Miller is often quoted as saying:

Man must shape his tools lest they shape him.

I read this again yesterday, in the online book Focus, while listening to faculty at a highly-ranked local school talk about the value of a liberal arts education. The quote reminds me about one of the reasons I so like being a computer scientist. I can shape my tools. If I need a new tool, or even a new kind of tool, I can make it.

Our languages are tools, too. We can shape them, grow them, change them. We can create new ones. (Thanks, Matz.)

Via the power of the Internet I am continuously surrounded by colleagues smarter and more motivated than I doing the same. I've been enjoying watching Brian Marick tweet about his thoughts and decision making as he implements Midje in Clojure. His ongoing dialog reminds me that I do not have to settle.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

November 03, 2010 2:12 PM

Ideas from Readers on Recent Posts

A few recent entries have given rise to interesting responses from readers. Here are two.

Fat Arrows

Relationships, Not Characters talked about how the most important part of design often lies in the space between the modules we create, whether objects or functions, not the modules themselves. After reading this, John Cook reminded me about an article by Thomas Guest, Distorted Software. Near the end of that piece, which talks about design diagrams, Guest suggests that the arrows in application diagrams should be larger, so that they would be proportional to the time their components take to develop. Cook says:

We typically draw big boxes and little arrows in software diagrams. But most of the work is in the arrows! We should draw fat arrows and little boxes.

I'm not sure that would make our OO class diagrams better, but it might help us to think more accurately!

My Kid Could Do That

Ideas, Execution, and Technical Achievement wistfully admitted that knowing how to build Facebook or Twitter isn't enough to become a billionaire. You have to think to do it. David Schmüdde mentioned this entry in his recent My Kid Could Do That, which starts:

One of my favorite artists is Mark Rothko. Many reject his work thinking that they're missing some genius, or offended that others see something in his work that they don't. I don't look for genius because genuine genius is a rare commodity that is only understood in hindsight and reflection. The beauty of Rothko's work is, of course, its simplicity.

That paragraph connects with one of the key points of my entry: Genius is rare, and in most ways irrelevant to what really matters. Many people have ideas; many people have skills. Great things happen when someone brings these ingredients together and does something.

Later, he writes:

The real story with Rothko is not the painting. It's what happens with the painting when it is placed in a museum, in front of people at a specific place in the world, at a specific time.

In a comment on this post, I thanked Dave, and not just because he discusses my personal reminiscence. I love art but am a novice when it comes to understanding much of it. My family and I saw an elaborate Rothko exhibit at the Smithsonian this summer. It was my first trip to the Smithsonian complex -- a wonderful two days -- and my first extended exposure to Rothko's work. I didn't reject his art, but I did leave the exhibit puzzled. What's the big deal?, I wondered. Now I have a new context in which to think about that question and Rothko's art. I didn't expect the new context to come from a connection a reader made to my post on tech start-up ideas that change the world!

I am glad to know that thinkers like Schmüdde are able to make connections like these. I should note that he is a professional artist (both visual and aural), a teacher, and a recovering computer scientist -- and a former student of mine. Opportunities to make connections arise when worlds collide.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Personal, Software Development

October 28, 2010 4:27 PM

Ideas, Execution, and Technical Achievement

Four or five years ago, my best buddy on campus and I were having lunch at our favorite Chinese buffet. He looked up between bites of General Tsao's and asked, "Why didn't you and I sit down five years ago and write Facebook?"

You see, he is an awesome programmer and has worked with me enough to know that I do all right myself. At various times, both of us have implemented bits and pieces of the technology that makes up Facebook. It doesn't look like all that big a deal.

I answered, "Because we didn't think of it."

The technical details may or may not have been a big deal. Once implemented, they look straightforward. In any case, though, the real reason was that it never occurred to us to write Facebook. We were techies who got along nicely with the tools available to us in 1999 or 2000, such as e-mail, wiki, and the web. If we needed to improve our experience, we did so by improving our tools. Driven by one of his own itches, Troy had done his M.S. research with me as his advisor, writing a Bayesian filter to detect spam. But neither of us thought about supplanting e-mail with a new medium.

We had the technical skills we needed to write Facebook. We just didn't have the idea of Facebook. Turns out, that matters.

That lunch conversation comes into my mind every so often. It came back yesterday when I read Philip Greenspun's blog entry on The Social Network. Greenspun wrote one of my favorite books, Philip and Alex's Guide to Web Publishing, which appeared in 1998 and which describes in great detail (and with beautiful photos) how to implement web community software. When his students ask how he feels about Zuckerberg getting rich without creating anything "new", Greenspun gives a wonderfully respectful and dispassionate answer: "I didn't envision every element of Facebook." Then he explains what he means.

Technically, Greenspun was positioned as well or better than my buddy and I to write Facebook. But he didn't the idea, either, at least not to the same level as Zuckerberg. Having the core of an idea is one thing. Developing it to the point that it becomes a platform that changes the world in which it lives is another. Turns out, that matters, too.

I like Lawrence Lessig's most succinct summation of what makes Zuckerberg writing Facebook a notable achievement: He did it. He didn't just have an idea, or talk about it, or dream about it. He implemented it.

That's what great hackers do.

Watch this short video to hear Zuckerberg himself say why he built. His answer is also central to the hacker ethic: Because he wanted to.

(Also read through to the end of Lessig's article for a key point that many people miss when they think about the success and achievement of things like Facebook and Twitter and Napster: The real story is not the invention.

Zuckerberg may or may not be a genius? I don't know or care. That is a word that modern culture throws around far too carelessly these days. I will say this. I don't think that creating Facebook is in itself sufficient evidence for concluding so. A lot of people have cool ideas. A more select group of people write the code to make their ideas come alive. Those people are hackers. Zuckerberg is clearly a great hacker.

I'm not a big Facebook user, but it has been on my mind more than usual the last couple of days. Yesterday was my birthday, and I was overwhelmed by all the messages I received from Facebook friends wishing me a happy day. They came from all corners of the country; from old grade-school friends I haven't seen in over thirty years; from high school and college friends; from professional friends and acquaintances. These people all took the time to type a few words of encouragement to someone hundreds of miles away in the middle of the Midwest. I felt privileged and a little humbled.

Clearly, this tool has made the world a different place. The power of the social network lies in the people. The technology merely enables the communication. That's a significant accomplishment, even if most of the effects are beyond what the creator imagined. That's the power of a good idea.

~~~~

All those years ago, my buddy and I talked about how little technical innovation there was in Facebook. Greenspun's answer reminds us that there was some. I think there is another element to consider, something that was a driving force at StrangeLoop: big data. The most impressive technical achievement of Facebook and smaller web platforms such as Twitter is the scale at which they operate. They've long ago outgrown naive implementations and have had to try to offer uninterrupted service in the face of massive numbers of users and exponential growth. Solving the problems associated with operating at such a scale is an ongoing technical challenge and a laudable achievement in its own right.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

October 25, 2010 4:50 PM

A Programming Digression: Farey Sequences

Farey sequences as Ford circles

Last Wednesday, I saw John Cook's article on rational approximations using Farey's algorithm, and it led me on a programming journey. You might find it entertaining.

First, some background. The students in my compilers course are writing a compiler for a simple numerical language I call Klein. Klein is a minimal functional language inspired by Doug Baldwin's MinimL, designed to give compiler students experience with all the important issues in the course while not duplicating very much. The goal is that students be able to write a complete and correct compiler for the language from scratch in a semester. Among Klein's limitations are:

  • Repetition is done with recursion. There are no loops.
  • There are only two data types, integers and booleans.
  • All binding of values to names is done via function calls. There are no local variables, no assignments statements, and no sequences.
  • There is only one primitive function, print, that can be called only at the top of a function definition.

As you can imagine, it is difficult to find good, large problems for which we can write Klein programs. So I'm always on the look-out for numerical algorithms that can be implemented in Klein.

When I saw Cook's article and Python program, I got to thinking... Wouldn't this be a cool application for Klein?

Of course, I faced a few problems. Cook's program takes as arguments a real number in [0..1), r, and an integer N. It returns a fraction that is the rational number closest in value to r with a denominator that no bigger than N. Klein does not have floating-point values, so I would have to fake even the real value being approximated! Without assignment statements and sequences of statements, let alone Python's multiple assignments and multiple returns, I would have to convert the algorithm's loop to a set of functions for computing the individual components of the converging bounds. Finally, without floats again, I would also have to compare the bounding fractions to the faked real number as fractions.

Those are just the sort of challenges that intrigue a programmer. I spent a couple of hours on Thursday slowly refactoring Cook's code to a Klein-like subset of Python. Instead of passing r to the main function, I fake it by passing in two arguments: the digits to the right of the decimal point as an integer, and a power of 10 to indicate its magnitude. These serve as numerator and denominator of a rational approximation, albeit with a very denominator. For example, r = 0.763548745, I pass in 7635487 and 10,000,000. The third argument to the function is N. Here is an example:

     >>> farey( 127, 1000, 74)
     8
     63

8/63 is the best rational approximation for 0.127 with a denominator ≤ 74. My code prints the 8 and returns the 63, because that is best way for an equivalent Klein program to work.

Here is Cook's example of 1/e with a denominator bounded by 100:

     >>> farey( 367879, 1000000, 100 )
     32
     87

Take a look at my Python code if you are easily amused. In one part of the program, I duplicated code with only small changes to the resulting functions, farey_num() andfarey_den. In another section, I avoided creating duplicate functions by adding a selector argument that I allowed me to use one bit of code for two different calculations. while_loop_for returns the next value for one of a, b, c, and d, depending on the value of its first argument. For some reason, I am fond of the code that replaces Cook's mediant variable.

At this point, the port to Klein was straightforward.

In the end, there is so much code to do what is basically a simple task: to compute row N of this tree on an as-needed basis, in the service of a binary search:

Farey sequences as a lattice

Programming in Klein feels a lot like programming in an integer assembly language. That can be an interesting experience, but the real point of the language is to exercise students as they write a compiler. Still, I find myself now wanting to implement refactoring support for Klein, in order to make such digressions flow faster in the future!

I enjoyed this little diversion so much that I've been buttonholing every colleague who walks through the door. I decided I'd do the same to you.


Posted by Eugene Wallingford | Permalink | Categories: Computing

October 21, 2010 4:45 PM

On Kazoos and Violins

Coming out of Strange Loop, I have been reading more about Scala and Clojure, both of which bring a lot of cool functional programming ideas to the JVM world. A lot of people have seem to have made deeper inroads into FP via Scala than via Clojure. That is certainly true among the companies in my area. That is not too surprising. Scala looks more like what they are used to than Clojure. It feels more comfortable.

This context is probably one of the reasons that I so liked this quote I read yesterday from Brenton Ashworth, a serial contributor to the Clojure community:

I'm now convinced that choosing a language based on how easy it is to read or write is a very bad idea. What if we chose musical instruments based on how easy they are to play? Everyone would playing kazoos. You can't think deeply about music if all you know is a kazoo.

This reminds me of a story Alan Kay often tells about violins. I have alluded to it at least once before:

And we always need to keep in mind the difference between essential and gratuitous complexity. I am often reminded of Alan Kay's story about learning to play violin. Of course it's hard to learn. But the payoff is huge.

New tools often challenge us, because knowing them well will change us. And such change is a part of progress. Imagine if no one had wanted to learn to fly an airplane because it was too different from driving a car or a horse.

Don't get me wrong: I am not comparing Scala to a kazoo and Clojure to a violin! I don't know either language well enough to make grand pronouncements about them, but my small bit of study tells me that both are powerful, valuable languages, languages worth knowing. I'm simply concerned that too many people opt out of learning Clojure well because of how it looks. As Ashworth says in the sentence immediately following the passage quoted above, "Skill at reading and writing code is learned."

You can do it! Don't settle on something only because something else looks unfamiliar. By taking the easier path in this moment, you may be making your path harder in the long run.


Posted by Eugene Wallingford | Permalink | Categories: Computing

October 21, 2010 8:50 AM

Strange Loop Redux

StrangeLoop 2010 logo

I am back home from St. Louis and Des Moines, up to my next in regular life. I recorded some of my thoughts and experiences from Strange Loop in a set of entries here:

Unlike most of the academic conferences I attend, Strange Loop was not held in a convention center or in a massive conference hotel. The primary venue for the conference was the Pageant Theater, a concert nightclub in the Delmar Loop:

The Pageant Theater

This setting gave the conference's keynotes something of an edgy feel. The main conference lodging was the boutique Moonrise Hotel a couple of doors down:

The Pageant Theater

Conference session were also held in the Moonrise and in the Regional Arts Commission building across the street. The meeting rooms in the Moonrise and the RAC were ordinary, but I liked being in human-scale buildings that had some life to them. It was a refreshing change from my usual conference venues.

It's hard to summarize the conference in only a few words, other than perhaps to say, "Two thumbs up!" I do think, though, that one of the subliminal messages in Guy Steele's keynote is also a subliminal message of the conference. Steele talked for half an hour about a couple of his old programs and all of his machinations twenty-five or forty years to make them run in the limited computing environments of those days. As he went to all the effort to reconstruct the laborious effort that went into those programs in the first place, the viewer can't help but feel that the joke's on him. He was programming in the Stone Age!

But then he gets to the meat of his talk and shows us that how we program now is the relic of a passing age. For all the advances we have made, we still write code that transitions from state to state to state, one command at a time, just like our cave-dwelling ancestors in the 1950s.

It turns out that the joke is on us.

The talks and conversations at Strange Loop were evidence that one relatively small group of programmers in the midwestern US are ready to move into the future.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Software Development

October 16, 2010 7:59 PM

Strange Loop This and That

Some miscellaneous thoughts after a couple of days in the mix...

Pertaining to Knowing and Doing

** Within the recurring theme of big data, I still have a few things to study: MongoDB, CouchDB, and FlockDB. I also learned about Pig, a language I'd never heard of before. I think I need to learn it.

** I need to be sure that my students learn about the idea of multimethods. Clojure has brought them back into mainstream discussion.

** Kevin Weil, who spoke about NoSQL at Twitter, told us that his background is math and physics. Not CS. Yet another big-time developer who came to programming from another domain because they had real problems to solve.

Pertaining to the Conference

** The conference served soda all day long, from 8:00 in the morning to the end of the day. Hurray! My only suggestion: add Diet Mountain Dew to the selections.

** The conference venues consist of two rooms in a hotel, two rooms in a small arts building, and the auditorium of the Pageant Theater. The restrooms are all small. During breaks, the line for the men's room was, um, long. The women in attendance came and went without concern. This is exactly opposite of what one typically sees out in public. The women of Strange Loop have their revenge!

** This is the first time I have ever attended a conference with two laptop batteries. And I saw that it was good. Now, I just have to find out why every couple of weeks my keyboard and trackpad freeze up and effectively lock me out. Please, let it not be a failing mother board...

Pertaining to Nothing But Me

** Like every conference, Strange Loop fills the silence between sessions with a music loop. The music the last two days has been aimed at its audience, which is mostly younger and mostly hipper than I am. I really enjoyed it. I even found a song that will enter my own rotation, "Hey, Julie" by Fountains of Wayne. You can, of course, listen to it on YouTube. I'll have to check out more Fountains of Wayne later.

** On Twitter, I follow a relatively small number of people, mostly professional colleagues who share interesting ideas and links. I also follow a few current and former students. Rounding out the set are a couple connections I made with techies through others, back when Twitter was small. I find that I enjoy their tweets even though I don't know them, or perhaps because I don't.

On Thursday, it occurred to me: Maybe it would be fun to follow some arbitrary incredibly popular person. During one of the sessions, we learned that Lady Gaga has about 6,500,000 followers, surpassing Ashton Kutcher's six million. I wonder it would be like to have their tweets flowing in a stream with those of Brian Marick and Kevlin Henney, Kent Beck and Michael Feathers.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

October 16, 2010 9:09 AM

Strange Loop, Day 2 Afternoon

After eating some dandy deep-dish BBQ chicken at Pi Pizzeria with the guys from T8 Webware (thanks, Wade!), I returned for a last big run of sessions. I'll save the first session for last, because my report of it is the longest.

Android Squared

I went to this session because so many of my students want me to get down in the trenches with phone programming. I saw a few cool tools, especially RetroFit, a new open-source framework for Android. There are not enough hours in a day for me to explore every tool out there. Maybe I can have a student do an Android project.

Java Puzzlers

And I went to this session because I am weak. I am a sucker for silly programming puzzles, especially ones that take advantage of the dark corners of our programming languages. This session did not disappoint in this regard. Oh, the tortured code they showed us! I draw from this experience a two-part moral:

  1. Bad programmers can write really bad code, especially in a complex language.
  2. A language that is too complex makes bad programmers of us all.

Brian Marick on Outside-In TDD

Marick demoed a top-down | outside-in style of TDD in Clojure using Midje, his homebrew test package. This package and style make heavy use of mock objects. Though I've dinked around a bit in Scala, I've done almost nothing in Clojure, so I'll have to try this out. The best quote of the session echoed a truth about all programming: You should have no words in your test that are not specifically about the test.

Douglas Crockford, Open-Source Heretic

Maybe my brain was fried by the end of the two days, or perhaps I'm simply not clever enough. While I able to chuckle several times through this closing keynote, I never saw the big picture or the point of the talk. There were plenty of interesting, if disconnected, stories and anecdotes. I enjoyed Crockford's coverage of several historical mark-up languages, including Runoff and Scribe. (Runoff was the forebear of troff, a Unix utility I used throughout grad school -- I even produced my wedding invitations using it! Fans of Scribe should take a look at Scribble, a mark-up tool built-on top of Racket.) He also told an absolutely wonderful story about Grace Murray Hopper's A-0, the first compiler-like tool and likely the first open-source software project.

Panel on the Future of Languages

Panels like this often don't have a coherent summary. About all I can do is provide a couple of one-liners and short answers to a couple of particularly salient questions.

Joshua Bloch: Today's patterns are tomorrow's language features. Today's bugs are tomorrow's type system features.

Douglas Crockford: Javascript has become what Java was meant to be, the language that runs everywhere, the assembly language of the web.

Someone in the audience asked, "Are changes in programming languages driven by academic discovery or by practitioner pain?" Guy Steele gave the best answer: The evolution of ideas is driven by academics. Uptake is driven by practitioner needs.

So, what is the next big thing in programming languages? Some panelists gave answers grounded in today's problems: concurrency, a language that could provide and end-to-end solution for the web, and security. One panelists offered laziness. I think that laziness will change how many programmers think -- but only after functional programming has blanketed the mainstream. Collectively, several panelists offered variations of sloppy programming, citing as early examples Erlang's approach to error recovery, NoSQL's not-quite-data consistency, and Martin Rinard's work on acceptability-oriented computing.

The last question from the audience elicited some suggestions you might be able to use. What language, obscure or otherwise, should people learn in order to learn the language you really want them to learn? For this one, I'll give you a complete list of the answers:

  • Io. (Bruce Tate)
  • Rebol. (Douglas Crockford)
  • Forth. Factor. (Alex Payne)
  • Scheme. Assembly. (Josh Bloch)
  • Clojure. Haskell. (Guy Steele)

I second all of these suggestions. I also second Steele's more complete answer: Learn any three languages you do not know. The comparisons and contrasts among them will teach you more than any one language can.

Panel moderator Ted Neward closed the session with a follow-up question: "But what should the Perl guys learn while they are waiting for Perl 6?" We are still waiting for the answer.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

October 15, 2010 11:46 PM

Strange Loop, Day 2 Morning

Perhaps my brain is becoming overloaded, or I have been less disciplined in picking talks to attend, or the slate of sessions for Day 2 is less coherent than Day 1's. But today has felt scattered, and so far less satisfying than yesterday. Still, I have had some interesting thoughts.

Billy Newport on Enterprise NoSQL

This was yet another NoSQL talk, but not, because it was different than the preceding ones at the conference. This talk was not about any particular technologies. It was about mindsets.

Newport explained that NoSQL means not only SQL. These two general approaches to data storage offer complementary strengths and weaknesses. This means that they are best used in different contexts.

I don't do enough programming for big data apps to appreciate all the details of this talk. Actually, I understood most of the basic concepts, but they soon starting blurring in my mind, because I don't have personal experience on which to hang them. A few critical points stood out:

  • In the SQL world, the database is the "system of record" for all data, so consistency is a given. In the NoSQL world, having multiple systems of record is normal. In order to ensure consistency, the application uses business rules to bring data back into sync. This requires a big mind shift for SQL guys.

  • In the SQL world, the row is a bottleneck. In the NoSQL world, any node can handle the request. So there is not a bottleneck, which means the NoSQL approach scales transparently. J But see the first bullet.

These two issues are enough to see one of Newport's key points. The differences between the two worlds is not only technical but also cultural. SQL and NoSQL programmers use different vocabulary and have different goals. Consider that "in NoSQL, 'query' is a dirty word". NoSQL programmers do everything they can to turn queries into look-ups. For the SQL programmer, the query is a fundamental concept.

The really big idea I took away from this talk is that SQL and NoSQL solve different problems. The latter optimizes for one dominant question, while the former seeks to support an infinite number of questions. Most of the database challenges facing NoSQL shops boil down to this: "What happens if you ask a different question?"

Dean Wampler on Scala

The slot in which this tutorial ran was longer than the other sessions at the conference. This allowed Wampler to cover a lot of details about Scala. I didn't realize how much of an "all but the kitchen sink" language Scala is. It seems to include just about every programming language feature I know about, drawn from just about every programming language I know about.

I left the talk a bit sad. Scala contains so much. It performs so much syntactic magic, with so many implicit conversions and so many shortcuts. On the one hand, I fear that large Scala programs will overload programmers' minds the way C++ does. On the other, I worry that its emphasis on functional style will overload programmers' minds the way Haskell does.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

October 15, 2010 12:34 PM

Guy Steele's Own Strange Loop

The last talk of the afternoon of Day 1 was a keynote by Guy Steele. My notes for his talk are not all that long, or at least weren't when I started writing. However, as I expected, Steele presented a powerful talk, and I want to be able to link directly to it later.

Steele opened with a story about a program he wrote forty years ago, which he called the ugliest program he ever wrote. It fit on a single punch card. To help us understand this program, he described in some detail the IBM hardware on which it ran. One problem he faced as a programmer is that the dumps were undifferentiated streams of bytes. Steele wanted line breaks, so he wrote an assembly language program to do that -- his ugliest program.

Forty years later, all he has is the punch card -- no source. Steele's story then turned into CSI: Mainframe. He painstakingly reverse-engineered his code from punches on the card. We learned about instruction format, data words, register codes... everything we needed to know how this program managed to dump memory with newlines and fit on a single card. The number of hacks he used, playing on puns between op codes and data and addresses, was stunning. That he could resurrect these memories forty years later was just as impressive.

I am just old enough to have programmed assembly for a couple of terms on punch cards. This talk brought back memories, even how you can recognize data tables on a card by the unused rows where there are no op codes. What a wonderful forensics story.

The young guys in the room liked the story, but I think some were ready for the meet of the talk. But Steele told another, about a program for computing sin 3x on a PDP-11. To write this program, Steele took advantage of changes in the assembly languages between the IBM mainframe and the PDP-11 to create more readable code. Still, he had to use several idioms to make it run quickly in the space available.

These stories are all about automating resource management, from octal code to assemblers on up to virtual memory and garbage collection. These techniques let the programmer release concerns about managing memory resources to tools. Steele's two stories demonstrate the kind of thinking that programmers had to do back in the days when managing memory was the programmer's job. It turns out that the best way to think about memory management is not to think about it at all.

At this point, Steele closed his own strange loop back to the title of his talk. His thesis is this: the best way to think about parallel programming is not to have to.

If we program using a new set of idioms, then parallelism can be automated in our tools. The idioms aren't about parallelism; they are more like functional programming patterns that commit the program less to underlying implementation.

There are several implications of Steele's thesis. Here are two:

  • Accumulators are bad. Divide and conquer is good.

  • Certain algebraic properties of our code are important. Programmers need to know and preserve them in then code they write.

Steele illustrated both of these implications by solving an example problem that would fit nicely in a CS1 course: finding all the words in a string. With such a simple problem, everyone in the room has an immediate intuition about how to solve it. And nearly everyone's intuition produces a program using accumulators that violates several important algebraic properties that our code might have.

One thing I love about Steele's talks: he grounds ideas in real code. He developed a complete solution to the problem in Fortress, the language Steele and his team have been creating at Sun/Oracle for the last few years. I won't try to reproduce the program or the process. I will say this much. One, the process demonstrated a wonderful interplay between functions and objects. Two, in the end, I felt like we had just used a process very similar to the one I use when teaching students to create this functional merge sort function:

    (define mergesort
       (lambda (lst)
          (merge-all (map list lst))))

Steele closed his talk with the big ideas that his programs and stories embody. Among the important algebraic properties that programs should have whenever possible are ones we all learned in grade school, explicitly or implicitly. Though they may still sound scary, they all have intuitive common meanings:

  • associative -- grouping don't matter
  • commutative -- order doesn't matter
  • idempotent -- duplicates don't matter
  • identity -- this value doesn't matter
  • zero -- other values don't matter

Steele said that "wiggle room" was the key buzzword to take away from his talk. Preserving invariants of these algebraic properties give the compiler wiggle room to choose among alternative ways to implement the solution. In particulars, associativity and commutativity give the compiler wiggle room to parallelize the implementation.

(Note that the merge-all operation in my mergesort program satisfies all five properties.)

One way to convert an imperative loop to a parallel solution is to think in terms of grouping and functions:

  1. Bunch mutable state together as a state "value".
  2. Look at the loop as an application of one or more state transformation functions.
  3. Look for an efficient way to compose these transformation functions into a single function.

The first two steps are relatively straightforward. The third step is the part that requires ingenuity!

In this style of programming, associative combining operators are a big deal. Creating new, more diverse associative combining operators is the future of programming. Creating new idioms -- the patterns of programs written in this style -- is one of our challenges. Good programming languages of the future will provide, encourage, and enforce invariants that give compilers wiggle room.

In closing, Steele summarized our task as this: We need to do for processor allocation what garbage collection did for memory allocation. This is essential in a world in which we have parallel computers of wildly different sizes. (Multicore processors, anyone?)

I told some of the guys at the conference that I go to hear Guy Steele irrespective of his topic. I've been fortunate enough to be in a small OOPSLA workshop on creativity with Steele, gang-writing poetry and Sudoku generators, and I have seen him speak a few times over the years. Like his past talks, this talk makes me think differently about programs. It also crystallizes several existing ideas in a way that clarifies important questions.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

October 14, 2010 11:17 PM

Strange Loop 2010, Day 1 Afternoon

I came back from a lunchtime nap ready for more ideas.

You Already Use Closures in Ruby

This is one of the talks I chose for a specific personal reason. I was looking for ideas I might use in a Cedar Valley Tech Talk on functional programming later this month, and more generally in my programming languages course. I found one, a simple graphical notation for talking about closures as a combination code and environment. Something the speaker said about functions sharing state also gave me an idea for how to use the old koan on the dual "a closure is poor man's object / an object is poor man's closure".

NoSQL at Twitter

Speaker Kevin Weil started by decrying his own title. Twitter uses MySQL and relational databases everywhere. They use distributed data stores for applications that need specific performance attributes.

Weil traced Twitter's evolution toward different distributed data solutions. They started with Syslog for logging applications. It served there early needs but didn't scale well. They then moved to Scribe, which was created by the Facebook team to solve the same problem and then open-sourced.

This move led to a realization. Scribe solved Twitter's problem and opened new vistas. It made logging data so easy, that they started logging more. Having more data gave them better insight into the behavior of their users, behaviors they didn't even know to look for before. Now, data analytics is one of Twitter's most interesting areas.

The Scribe solution works, scaling with no changes to the architecture as data throughput doubles; they just add more servers. But this data creates a 'write' problem: giving today's takes technology, it takes something like 42 hours to write 12 TB to a single hard drive. This led Twitter to add Hadoop to its toolset. Hadoop is both scalable and powerful. Weil mentioned an a 4,000-node cluster at Yahoo! that had sorted one terabyte of integers in 62 seconds.

The rest of Weil's talk focused on data analytics. The key point underlying all he said was this: It is easy to answer questions. It is hard to ask the right questions. This makes experimental programming valuable, and by extension a powerful scripting language and short turnaround times. They need time to ask a lot of questions, looking for good ones and refining promising questions into more useful ones. Hadoop is a Java platform, which doesn't fit those needs.

So, Twitter added Pig, a high-level language that sits atop Hadoop. Programs written in Pig are easy to read and almost English-like. Equivalent SQL programs would probably be shorter, but Pig compiles to MapReduce jobs that run directly on Hadoop. Pig extracts a performance penalty, but the Twitter team doesn't mind. Weil captured why in another pithy sentence: I don't mind if a 10-minute job runs in 12 minutes if it took me 10 minutes to write the script instead of an hour.

Twitter works on several kinds of data-analytic problems. A couple stood out:

  • correlating big data. How do different kinds of user behave -- mobile, web, 3rd-party clients? What features hook users? How do user cohorts work? What technical details go wrong at the same time, leading to site problems?

  • research on big data. What can we learn from a users' tweets, the tweets of those they follow, or the tweets of those who follow them? What can we learn from asymmetric follow relationships about social and personal interests?

As much as Weil had already described, there was more! HBase, Cassandra, FlockDB, .... Big data means big problems and big opportunities, which lead to hybrid solutions that optimize competing forces. Interesting stuff.

Solutions to the Expression Problem

This talk was about Clojure, which interests me for obvious reasons, but the real reason I chose this talk was that I wanted to know what is the expression problem! Like many in the room, I had experienced the expression problem without knowing it by this name:

The Expression Problem is a new name for an old problem. The goal is to define a datatype by cases, where one can add new cases to the datatype and new functions over the datatype, without recompiling existing code, and while retaining static type safety (e.g., no casts).

Speaker Chris Houser used an example in which we need to add a behavior to an existing class that is hard or impossible to modify. He then stepped through four possible solutions: the adapter pattern and monkey patching, which are available in languages like Java and Ruby, and multimethods and protocols, which are available in Clojure.

I liked two things about this talk. First, he taught his a "failure-driven" way: pose a problem, solve it using a known technique, expose a weakness, and move on to a more effective solution. I often use this technique when teaching design patterns. Second, the breadth of the problem and its potential solutions encouraged language geeks to talk ideas in language design. The conversation included not only Java and Clojure but also Javascript, C#, and macros.

Guy Steele

My notes on this talk aren't that long, but it was important enough to have its own entry. Besides, this entry is already long enough!


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

October 14, 2010 10:02 PM

Strange Loop 2010, Day 1 Morning

I arrived in the University City district this morning ready for a different kind of conference. The program is an interesting mix of hot topics in technology, programming languages, and computing. My heuristic for selecting talks to attend was this: Of the talks in each time slot, which discusses the biggest idea I need to know more about? The only exceptions will be talks that fill a specific personal need, in teaching, research, or personal connections.

A common theme among the talks I attended today was big data. Here are some numbers I jotted down. They may be out of date by now, or I may have made an error in some of the details, but the orders of magnitude are impressive:

  • Hilary Mason reported that bit.ly processes 10s of millions URLs per day, 100s of millions of events per day, and 1,000s of millions data points each day.
  • Google processes 24 petabytes a day, which amounts to 8 exabytes a year.
  • Digg has a 3 terabyte Cassandra database. This is dwarfed by Facebook's 150-node Cassandra cluster that comprises 150 terabytes.
  • Twitter users generate 12 terabytes of data each day -- the equivalent of 17,000 CDs -- and more than 4 petabytes each year. Also astounding: the data rate is doubling multiple times a year.

All of these numbers speak to the need for more CS programs to introduce students to data-intensive computing.

Hilary Mason on Machine Learning

Mason is a data scientist at bit.ly, a company that faces "wicked hard problems" at scale. These problems arise from the combination of algorithms, on-demand computing, and "data, data, data". Mason gave an introduction to machine learning as applied to some of these problems. She had a few quotable lines:

  • AI was "founded on a remarkable conceit". This reminded me of a recent entry.
  • "Academics like to create new terms, because if they catch on the academics get prizes."
  • "Only a Bayesian can tell you why if there's a 50-50 chance that something will happen, then 90% of the time it will."

She also sprinkled her top with programming tips. One of the coolest was that you can append a "+" to the end of any bit.ly URL to get analytic metrics for it.

Gradle

I had planned to go to a talk on Flickr's architecture, but I talked so long in the break that the room full before I got there. So I stopped in on a talk by Ken Sipe on Gradle, a scriptable Java build system built on top of Groovy. He had one very nice quote as well: "I can't think of a real use of this idea. I just like showing it off." The teacher and hacker in me smiled.

Eben Hewitt on Adopting Cassandra

Even if you are not Google, you may have to process a lot of data. Hewitt talked about some of the efforts made to scale relational databases to handle big data (query tuning, indexes, vertical scaling, shards, and denormalization). This sequence of fixes made me think of epicycles, fixes applied to orbits of heavenly bodies in an effort to match observed data. At some point, you start looking for a theory that fits better up front.

That's what happened in the computing world, too. Soon there were a number of data systems defined mostly by what they are not: "NoSQL". That idea is not new. Lotus Notes used a document-oriented storage system; Smalltalk images store universes of objects. Now, as the problem of big data becomes more prevalent, new and different implementations have been proposed.

Hewitt talked about Cassandra and what distinguishes it from other approaches. He called Cassandra the love child of the Google BigTable paper (2006) and the Amazon Dynamo paper (2007). He also pointed out some of the limitations that circumscribe its applicability. In the NoSQL approaches, there is "no free lunch": you give up many relational DB advantages, such as transactions, joins, and ad hoc queries.

He did advocates one idea I'll need to read more about: that we should shift our attention from the CAP theorem of distributed data systems, which is useful but misses some important dynamic distinctions, with Abadi's PACELC model: If the data store is Partitioned, then you trade off between Availability and Consistency; Else you trade off between Latency and Consistency.


Posted by Eugene Wallingford | Permalink | Categories: Computing

October 06, 2010 11:24 AM

Studying Program Repositories

Hackage, through Cabal

Last week, Garrett Morris presented an experience report at the 2010 Haskell Symposium entitled Using Hackage to Inform Language Design, which is also available at Morris's website. Hackage is an online repository for the Haskell programming community. Morris describes how he and his team studied programs in the Hackage repository to see how Haskell programmers work with a type-system concept known as overlapping instances. The details of this idea aren't essential to this entry, but if you'd like to learn more, check out Section 6.2.3 in this user guide.)

In Morris's search, he sought answers to three questions:

  • What proportion of the total code on Hackage uses overlapping instances?
  • In code that uses overlapping instances, how many instances overlap each other?
  • Are there common patterns among the uses of overlapping instances?

Morris and his team used what they learned from this study to design the class system for their new language. They found that their language did not need to provide the full power of overlapping instances in order to support what programmers were really doing. Instead, they developed a new idea they call "instance chains" that was sufficient to support a large majority of the uses of overlapping instances. They are confident that their design can satisfy programmers because the design decision reflects actual uses of the concept in an extensive corpus of programs.

I love this kind of empirical research. One of the greatest assets the web gives us is access to large corpora of code: SourceForge and GitHub are examples large public repositories, and there are an amazing number of smaller domain-specific repositories such as Hackage and RubyGems. Why design languages or programming tools blindly when we can see how real programmers work through the code they write?

The more general notion of designing languages via observing behavior, forming theories, and trying them out is not new. In particular, I recommend the classic 1981 paper Design Principles Behind Smalltalk. Dan Ingalls describes the method by which the Smalltalk team grew its language and system as explicitly paralleling the scientific method. Morris's paper talks about a similar method, only with the observation phase grounded in an online corpus of code.

Not everyone in computer science -- or outside CS -- thinks of this method as science. Just this weekend Dirk Riehle blogged about the need to broaden the idea of science in CS. In particular, he encourages us to include exploration as a part of our scientific method, as it provides a valuable way for us to form the theories that we will test using the sort of experiments that everyone recognizes as science.

Dirk Riehle's cycle of theory formation and validation

Unfortunately, too many people in computer science do not take this broad view. Note that Morris published his paper as an experience report at a symposium. He would have a hard time trying to get an academic conference program committee to take such a paper in its research track, without first dressing it up in the garb of "real" research.

As I mentioned in an earlier blog entry, one of my grad students, Nate Labelle, did an M.S. thesis a few years ago based on information gathered from the study of a large corpus of programs. Nate was interested in dependencies among open-source software packages, so he mapped the network of relationships within several different versions of Linux and within a few smaller software packages. This was the raw data he used to analyze the mathematical properties of the dependencies.

In that project, we trolled repositories looking for structural information about the code they contained. Morris's work studied Hackage to learn about the semantic content of the programs. While on sabbatical several years ago, I started a much smaller project of this sort to look for design patterns in functional programs. That project was sidetracked by some other questions I ran across, but I've always wanted to get back to it. I think there is a lot we could learn about functional programming in this way that would help us to teach a new generation of programmers in industry.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

September 29, 2010 9:27 PM

The Manifest Destiny of Computer Science?

The June 2010 issue of Communications of the ACM included An Interview with Ed Feigenbaum, who is sometimes called the father of expert systems. Feigenbaum was always an ardent promoter of AI, and time doesn't seem to have made him less brash. The interview closes with the question, "Why is AI important?" The father of expert systems pulls no punches:

There are certain major mysteries that are magnificent open questions of the greatest import. Some of the things computer scientists study are not. If you're studying the structure of databases -- well, sorry to say, that's not one of the big magnificent questions.

I agree, though occasionally I find installing and configuring Rails and MySQL on my MacBook Pro to be one of the great mysteries of life. Feigenbaum is thinking about the questions that gave rise to the field of artificial intelligence more than fifty years ago:

I'm talking about mysteries like the initiation and development of life. Equally mysterious is the emergence of intelligence. Stephen Hawking once asked, "Why does the universe even bother to exist?" You can ask the same question about intelligence. Why does intelligence even bother to exist?

That is the sort of question that captivates a high school student with an imagination bigger than his own understanding of the world. Some of those young people are motivated by a desire to create an "ultra-intelligent computer:, as Feigenbaum puts it. Others are motivated more by the second prize on which AI founders set their eyes:

... a very complete model of how the human mind works. I don't mean the human brain, I mean the mind: the symbolic processing system.

That's the goal that drew the starry-eyed HS student who became the author of this blog into computer science.

Feigenbaum closes his answer with one of the more bodacious claims you'll find in any issue of Communications:

In my view the science that we call AI, maybe better called computational intelligence, is the manifest destiny of computer science.

There are, of course, many areas of computer science worthy of devoting one's professional life to. Over the years I have become deeply interested in questions related to language, expressiveness, and even the science or literacy that is programming. But it is hard for me to shake the feeling, still deep in my bones, that the larger question of understanding what we mean by "mind" is the ultimate goal of all that we do.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

September 26, 2010 7:12 PM

Competition, Imitation, and Nothingness on a Sunday Morning

Joseph Brodsky said:

Another poet who really changed not only my idea of poetry, but also my perception of the world -- which is what it's all about, ya? -- is Tsvetayeva. I personally feel closer to Tsvetayeva -- to her poetics, to her techniques, which I was never capable of. This is an extremely immodest thing to say, but, I always thought, "Can I do the Mandelstam thing?" I thought on several occasions that I succeeded at a kind of pastiche.

But Tsvetayeva. I don't think I ever managed to approximate her voice. She was the only poet -- and if you're a professional that's what's going on in your mind -- with whom I decided not to compete.

Tsvetayeva was one of Brodsky's closest friends in Russia. I should probably read some of her work, though I wonder how well the poems translate into English.

~~~~~~~~~~

absolutely nothing: next 22 miles

When I am out running 22 miles, I have lots of time to think. This morning, I spent a few minutes thinking about Brodsky's quote, in the world of programmers. Every so often I have encountered a master programmer whose work changes my perception of world. I remember coming across several programs by Ward Cunningham's, including his wiki, and being captivated by its combination of simplicity and depth. Years before that, the ParcPlace Smalltalk image held my attention for months as I learned what object-oriented programming really was. That collection of code seemed anonymous at first, but I later learned its history and and became a fan of Ingalls, Maloney, and the team. I am sure this happens to other programmers, too.

Brodsky also talks about his sense of competition with other professional poets. From the article, it's clear that he means not a self-centered or destructive competition. He liked Tsvetayeva deeply, both professionally and personally. The competition he felt is more a call to greatness, an aspiration. He was following the thought, "That is beautiful" with "I can do that" -- or "Can I do that?"

I think programmers feel this all the time, whether they are pros or amateurs. Like artists, many programmers learn by imitating the code they see. These days, the open-source software world gives us so many options! See great code; imitate great code. Find a programmer whose work you admire consistently; imitate the techniques, the style, and, yes, the voice. The key in software, as in art, is finding the right examples to imitate.

Do programmers ever choose not to compete in Brodsky's sense? Maybe, maybe not. There are certainly people whose deep grasp of computer science ideas usually feels beyond my reach. Guy Steele comes to mind. But I think for programmers it's mostly a matter of time. We have to make trade-offs between learning one thing well or another.

~~~~~~~~~~

22 miles is a long run. I usually do only one to two runs that long during my training for a given marathon. Some days I start with the sense of foreboding implied by the image above, but more often the run is just there. Twenty-two miles. Run.

This time the morning was chill, 40 degrees with a bright sun. The temperature had fallen so quickly overnight that the previous day's rain had condensed in the leaves of every tree and bush, ready to fall like a new downpour at the slightest breeze.

This is my last long run before taking on Des Moines in three weeks. It felt neutral and good at the same time. It wasn't a great run, like my 20-miler two weeks ago, but it did what it needed to do: stress my legs and mind to run for about as long as the marathon will be. And I had plenty of time to think through the nothingness.

Now begins my taper, an annual ritual leading to a race. The 52 miles I logged this week will seep into my body for the next ten days or so as it acclimates to the stress. Now, I will pare back my mileage and devote a few more short and medium-sized runs to converting strength into the speed.

~~~~~~~~~~

The quote that opens this entry comes from Joseph Brodsky, The Art of Poetry No. 28, an interview in The Paris Review by Sven Birkerts in December 1979. I like very much to hear writers talk about how they write, about other writers, and about the culture of writing. This long interview repaid me several times for the time I spent reading.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Running, Teaching and Learning

September 20, 2010 4:28 PM

Alan Kay on "Real" Object-Oriented Programming

In Alan Kay's latest comment-turned-blog entry, he offers thoughts in response to Moti Ben-Ari's CACM piece about object-oriented programming. Throughout, he distinguishes "real oop" from what passes under the term these days. His original idea of object-oriented software is really quite simple to express: encapsulated modules all the way down, with pure messaging. This statement boils down to its essence many of the things he has written and talked about over the years, such as in his OOPSLA 2004 talks.

This is a way to think about and design systems, not a set of language features grafted onto existing languages or onto existing ways of programming. He laments what he calls "simulated data structure programming", which is, sadly, the dominant style one sees in OOP books these days. I see this style in nearly every OOP textbook -- especially those aimed at beginners, because those books generally start with "the fundamentals" of the old style. I see it in courses at my university, and even dribbling into my own courses.

One of the best examples of an object-oriented system is one most people don't think of as a system at all: the Internet:

It has billions of completely encapsulated objects (the computers themselves) and uses a pure messaging system of "requests, not commands", etc.

Distributed client-server apps make good examples and problems in courses on OO design precisely because they separate control of the client and the server. When we write OO software in which we control both sides of the message, it's often too tempting to take advantage of how we control both objects and to violate encapsulation. These violations can be quite subtle, even when we take into account idioms such as access methods and the Law of Demeter. To what extent does one component depend on how another component does its job? The larger the answer, the more coupled the components.

Encapsulation isn't an end unto itself, of course. Nor are other features of our implementation:

The key to safety lies in the encapsulation. The key to scalability lies in how messaging is actually done (e.g., maybe it is better to only receive messages via "postings of needs"). The key to abstraction and compactness lies in a felicitous combination of design and mathematics.

I'd love to hear Kay elaborate on this "felicitous combination of design and mathematics"... I'm not sure just what he means!

As an old AI guy, I am happy to see Kay make reference to the Actor model proposed by Carl Hewitt back in the 1970s. Hewitt's ideas drew some of their motivation from the earliest Smalltalk and gave rise not only to Hewitt's later work on concurrent programming but also Scheme. Kay even says that many of Hewitt's ideas "were more in the spirit of OOP than the subsequent Smalltalks."

Another old AI idea that came to my mind as I read the article was blackboard architecture. Kay doesn't mention blackboards explicitly but does talk about how messaging might better be if instead of an object sent messages to specific targets they might "post their needs". In a blackboard system, objects capable of satisfying needs monitor the blackboard and offer to respond to a request as they are able. The blackboard metaphor maintains some currency in the software world, especially in the distributed computing world; it even shows up as an architectural pattern in Pattern-Oriented Software Architecture. This is a rich metaphor with much room for exploration as a mechanism for OOP.

Finally, as a CS educator, I could help but notice Kay repeating a common theme of his from the last decade, if not longer:

The key to resolving many of these issues lies in carrying out education in computing in a vastly different way than is done today.

That is a tall order all its own, much harder in some ways than carrying out software development in a vastly different way than is done today.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 10, 2010 2:22 PM

Recursion, Trampolines, and Software Development Process

Yesterday I read an old article on tail calls and trampolining in Scala by Rich Dougherty, which summarizes nicely the problem of recursive programming on the JVM. Scala is a functional language, which lends itself to recursive and mutually recursive programs. Compiling those programs to run on the JVM presents problems because (1) the JVM's control stack is shallow and (2) the JVM doesn't support tail-call optimization. Fortunately, Scala supports first-class functions, which enables the programmer to implement a "trampoline" that avoids the growing the stack. The resulting code is harder to understand and so to maintain, but it runs without growing the control stack. This is a nice little essay.

Dougherty's conclusion about trampoline code being harder to understand reminded me of a response by reader Edward Coffin to my July entry on CS faculty sending bad signals about recursion. He agreed that recursion usually is not a problem from a technical standpoint but pointed out a social problem (paraphrased):

I have one comment about the use of recursion in safety-critical code, though: it is potentially brittle with respect to changes made by someone not familiar with that piece of code, and brittle in a way that makes breaking the code difficult to detect. I'm thinking of two cases here: (1) a maintainer unwittingly modifies the code in a way that prevents the compiler from making the formerly possible tail-call optimization and (2) the organization moves to a compiler that doesn't support tail-call optimization from one that did.

Edward then explained how hard it is to warn the programmers that they have just made changes to the code that invalidate essential preconditions. This seems like a good place to comment the code, but we can't rely on programmers paying attention to such comments, even that the comments will accompany the code forever. The compiler may not warn us, and it may be hard to write test cases that reliably fail when the optimization is missed. Scala's @tailrec annotation is a great tool to have in this situation!

"Ideally," he writes, "these problems would be things a good organization could deal with." Unfortunately, I'm guessing that most enterprise computing shops are probably not well equipped to handle them gracefully, either by personnel or process. Coffin closes with a pragmatic insight (again, paraphrased):

... it is quite possible that [such organizations] are making the right judgement by forbidding it, considering their own skill levels. However, they may be using the wrong rationale -- "We won't do this because it is generally a bad idea." -- instead of the more accurate "We won't do this because we aren't capable of doing it properly."

Good point. I don't suppose it's reasonable for me or anyone to expect people in software shops to say that. Maybe the rise of languages such and Scala and Clojure will help both industry and academia improve the level of developers' ability to work with functional programming issues. That might allow more organizations to use a good technical solution when it is suitable.

That's one of the reasons I still believe that CS educators should take care to give students a balanced view of recursive programming. Industry is beginning to demand it. Besides, you never know when a young person will get excited about a problem whose solution feels so right as a recursion and set off to write a program to grow his excitement. We also want our graduates to be able to create solutions to hard problems that leverage the power of recursion. We need for students to grok the Big Idea of recursion as a means for decomposing problems and composing systems. The founding of Google offers an instructive example of using inductive definition recursion, as discussed in this Scientific American article on web science:

[Page and Brin's] big insight was that the importance of a page -- how relevant it is -- was best understood in terms of the number and importance of the pages linking to it. The difficulty was that part of this definition is recursive: the importance of a page is determined by the importance of the pages linking to it, whose importance is determined by the importance of the pages linking to them. [They] figured out an elegant mathematical way to represent that property and developed an algorithm they called PageRank to exploit the recursiveness, thus returning pages ranked from most relevant to least.

Much like my Elo ratings program that used successive approximation, PageRank may be implemented in some other way, but it began as a recursive idea. Students aren't likely to have big recursive ideas if we spend years giving them the impression it is an esoteric technique best reserved for their theory courses.

So, yea! for Scala, Clojure, and all the other languages that are making recursion respectable in practice.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

September 07, 2010 4:53 PM

Quick Thoughts on "Computational Thinking"

When Mark Guzdial posted his article Go to the Data, offering "two stories of (really) computational thinking", I thought for sure that I'd write something on the topic soon. Over four and a half weeks have passed, and I have not written yet. That's because, despite thinking about it on and off ever since, I still don't have anything deep or detailed to say. I'm as confused as Mark says he and Barb are, I guess.

Still, I can't seem to shake these two simple thoughts:

  • Whatever else else it may be, "computational thinking" has to involve, well, um, computation. As several commenters on his piece point out, lots of scientists collect and analyze data. Computer science connects data and algorithm in a way that other disciplines don't.

  • If a bunch of smart, well-informed people can discuss a name like "computational thinking" for several years and still be confused, maybe the problem is with the name. Maybe it really is just another name for a kind of computer science, generalized so that we can speak meaningfully of non-CS types and non-programmers doing it. I believe strongly that computing offers a new medium for expressing and creating ideas but, as for the term computational thinking, maybe there's no there there.

I do agree with Mark that we runners can learn a lot by looking back at our running logs! An occasional short program can often expose truths that elude the naked eye.


Posted by Eugene Wallingford | Permalink | Categories: Computing

September 01, 2010 9:12 PM

The Beginning of a New Project

"Tell me you're not playing chess," my colleague said quizzically.

But I was. My newest grad student and I were sitting in my office playing a quick couple of games of progressive chess, in which I've long been interested. In progressive chess, white makes one move, then black makes two; white makes three moves, then black makes four. The game proceeds in this fashion until one of the players delivers checkmate or until the game ends in any other traditional way.

This may seem like a relatively simple change to the rules of the game, but the result is something that almost doesn't feel like chess. The values of the pieces changes radically, as does the value of space and the meaning of protection. That's why we needed to play a couple of games: to acquaint my student with how different it is from the classical chess I know and love and which has played since a child.

For his master's project, the grad student wanted to do something in the general area of game-playing and AI, and we both wanted to work on a problem that is relatively untouched, where a few cool discoveries are still accessible to mortals. Chess, the fruit fly of AI from the 1950s into the 1970s, long ago left the realm where newcomers could make much of a contribution. Chess isn't solved in the technical sense, as checkers is, but the best programs now outplay even the best humans. To improve on the state of the art requires specialty hardware or exquisitely honed software.

Progressive chess, on the other hand, has a funky feel to it and looks wide open. We are not yet aware of much work that has been done on it, either in game theory or automation. My student is just beginning his search of the literature and will know soon how much has been done and what problems have been solved, if any.

That is why we were playing chess in my office on a Wednesday afternoon, so that we could discuss some of the ways in which we will have to think differently about this problem as we explore solutions. Static evaluation of positions is most assuredly different from what works in classical chess, and I suspect that the best ways to search the state space will be quite different, too. After playing only a few games, my student proposed a new way to do search to capitalize on progressive chess's increasingly long sequences of moves by one player. I'm looking forward to exploring it further, giving it a try in code, and finding other new ideas!

I may not be an AI researcher first any more, but this project excites me. You never know what you will discover until you wander away from known territory, and this problem offers us a lot of unknowns.

And I'll get to say, "Yes, we are playing chess," every once in a while, too.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

August 31, 2010 7:18 PM

Notes from a Master Designer

Someone tweeted recently about a recent interview with Fred Brooks in Wired magazine. Brooks is one of the giants of our field, so I went straight to the page. I knew that I wanted to write something about the interview as soon as I saw this exchange, which followed up questions about how a 1940s North Carolina schoolboy ended up working with computers:

Wired: When you finally got your hands on a computer in the 1950s, what did you do with it?

Brooks: In our first year of graduate school, a friend and I wrote a program to compose tunes. The only large sample of tunes we had access to was hymns, so we started generating common-meter hymns. They were good enough that we could have palmed them off to any choir.

It never surprises me when I learn that programmers and computer scientists are first drawn to software by a desire to implement creative and intelligent tasks. Brooks was first drawn to computers by a desire to automatic data retrieval, which at the time must have seemed almost as fantastic as composing music. In an Communications of the ACM interview printed sometime last year, Ed Feigenbaum called AI the "manifest destiny" of computer science. I often think he is right. (I hope to write about that interview soon, too.)

But that's not the only great passage in Brooks's short interview. Consider:

Wired: You say that the Job Control Language you developed for the IBM 360 OS was "the worst computer programming language ever devised by anybody, anywhere." Have you always been so frank with yourself?

Brooks: You can learn more from failure than success. In failure you're forced to find out what part did not work. But in success you can believe everything you did was great, when in fact some parts may not have worked at all. Failure forces you to face reality.

As an undergrad, I took a two-course sequence in assembly language programming and JCL on an old IBM 370 system. I don't know how much the JCL on that machine had advanced beyond Brooks's worst computer programming language ever devised, if it had at all. But I do know that the JCL course gave me a negative-split learning experience unlike any I had ever had before or have had since. As difficult as that was, I will be forever grateful for Dr. William Brown, a veteran of the IBM 360/370 world, and what he taught me that year.

There are at least two more quotables from Brooks that are worth hanging on my door some day:

Great design does not come from great processes; it comes from great designers.

Hey to Steve Jobs.

The insight most likely to improve my own work came next:

The critical thing about the design process is to identify your scarcest resource.

This one line will keep me on my toes for many projects to come.

If great design comes from great designers, then how can the rest of us work toward the goal of becoming a great designer, or at least a better one?

Design, design, and design; and seek knowledgeable criticism.

Practice, practice, practice. But that probably won't be enough. Seek out criticism from thoughtful programmers, designers, and users. Listen to what they have to say, and use it to improve your practice.

A good start might be to read this interview and Brooks's books.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

August 24, 2010 4:30 PM

Dreaming, Doing, Perl, and Language Translation

Today, I quoted Larry Wall's 2000 Atlanta Linux Showcase Talk in the first day of my compilers course. In that talk, he gives a great example of using a decompiler to port code -- in this case, from Perl 5 to Perl 6. While re-reading the talk, I remembered something that struck me as wrong when I read it the first time:

["If you can dream it, you can do it"--Walt Disney]

"If you can dream it, you can do it"--Walt Disney. Now this is actually false (massive laughter). I think Walt was confused between necessary and sufficient conditions. If you *don't* dream it, you can't do it; that is certainly accurate.

I don't think so. I think this is false, too. (Laugh now.)

It is possible to do things you don't dream of doing first. You certainly have to be open to doing things. Sometimes we dream something, set out to do it, and end up doing something else. The history of science and engineering are full of accidents and incidental results.

I once was tempted to say, "If you don't start it, you can't do it; that is certainly accurate." But I'm not sure that's true either, because of the first "it". These days, I'm more inclined to say that if you don't start doing something, you probably won't do anything.

Back to Day 1 of the compilers: I do love this course. The Perl quote in my lecture notes is but one element in a campaign to convince my students that this isn't just a compilers course. The value in the course material and in the project itself go far beyond the creation of an old-style source language-to-machine language translator. Decompilers, refactoring browsers, cross-compilers, preprocessors, interpreters, and translators for all sorts of domain-specific languages -- a compilers course will help you learn about all of these tools, both how they work and how to build them. Besides, there aren't many better ways to consolidate your understanding of the breadth of computer science than to build a compiler.

The official title of my department's course is "Translation of Programming Languages". Back in 1994, before the rebirth of mainstream language experimentation and the growth of interest in scripting languages and domain-specific languages, this seemed like a daring step. These days, the title seems much more fitting than "Compiler Construction". Perhaps my friend and former colleague Mahmoud Pegah and I had a rare moment of foresight. More likely, Mahmoud had the insight, and I was simply wise enough to follow.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General, Teaching and Learning

August 20, 2010 3:30 PM

Simplicity, Complexity, and Good Enough Chess Ratings

Andrew Gelman writes about a competition offered by Kaggle to find a better rating system for chess. The Elo system has been used for the last 40+ years with reasonable success. In the era of big data, powerful ubiquitous computers, and advanced statistical methods, it turns out that we can create a rating system that predicts more accurately the performance of players on games in the near-future. Very cool. I'm still enough of chess geek that I want to know just when Capablanca surpassed Lasker and how much better Fischer was than his competition in the 1972 challenger's matches. I've always had an irrational preference for ridiculously precise values.

Even as we find systems that perform better, I find myself still attached to Elo. I'm sure part of it is that I grew up with Elo ratings as a player, and read Elo's The Rating of Chess Players, Past and Present as a teen.

But there's more. I've also written programs to implement the rating system, including the first program I ever wrote out of passion. Writing the code to assign initial ratings to a pool of players based on the outcomes of games played among them required me to do something I didn't even know was possible at the time: start a process that wasn't guaranteed to stop. I learned about the idea of successive approximations and how my program would have to settle for values that fit the data well enough. This was my first encounter with epsilon, and my first non-trivial use of recursion. Yes, I could have written a loop, but the algorithm seemed so clear written recursively. Such experiences stick with a person.

There is still more, though, beyond my personal preferences and experiences. Compared to most of the alternatives that do a better job objectively, the Elo system is simple. The probability curve is simple enough for anyone to understand, and the update process is basic arithmetic. Even better, there is a simple linear approximation of the curve that made it possible for a bunch of high school kids with no interest in math to update ratings based on games played at the club. We posted a small table of expected values based on rating differences at the front of the room and maintained the ratings on index cards. (This is a different sort of index-card computing than I wrote about long ago.) There may have been more accurate systems we could have run, but the math behind this one was so simple, and the ratings were more than good enough for our purposes. I am guessing that the Elo system is more than good enough for most people's purposes.

Simple and good enough is a strong combination. Perhaps the Elo system will turn out to be the Newtonian physics of ratings. We know there are a better, more accurate models, and we use them whenever we need something very accurate. Otherwise, we stick to the old model and get along just fine almost all the time.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

August 17, 2010 5:33 PM

Does Computer Science Have a Culture?

Zed Shaw is known for his rants. I've enjoyed many of them over the years. However, his Go To University, Not For CS hits awfully close to home. I love his defense of a university education, but he doesn't have much good to say about computer science programs. This is the punchline:

This is why you go to a University and also why you should not study Computer Science. Except for a few places like MIT, Computer Science is a pointless discipline with no culture. Everything is borrowed from somewhere else. It's the ultimate post modern discipline, and nobody teaching it seems to know what the hell "post-modernism" even means.

He is perhaps a bit harsh, yet what counterargument might we offer? If you studied computer science, did your undergrad alma mater or your graduate school have a CS culture? Did any of your professors offer a coherent picture of CS as a serious intellectual discipline, worthy of study independent of specific technologies and languages?

In graduate school, my advisor and I talked philosophically about CS, artificial intelligence, and knowledge in a way that stoked my interest in computing as a coherent discipline. A few of my colleagues shared our interests, but many of fellow graduate students were more interested in specific problems and solutions. They viewed our philosophical explorations as diversions from the main attraction.

Unfortunately, when I look around at undergrad CS programs, I rarely see a CS culture. This true of what I see at my own university, at my friends' schools, and at schools I encounter professionally. Some programs do better than others, but most of us could do better. Some of our students would appreciate the intellectual challenge that is computer science beyond installing the latest version of Linux or making Eclipse work with SVN.

Shaw offers one sentence of great advice for those of us thinking about undergrad curriculum:

... the things that are core to Computer Science like language design, parsing, or state machines, aren't even taught unless you take an "advanced" course.

I feel his pain. Few schools seem willing to design a curriculum built around core ideas packaged any differently from the way they were packaged in 1980. Students can graduate from most CS programs in this country without studying language design or parsing in any depth.

I can offer one counterpoint: some of us do know what post-modernism is and means. Larry Wall calls Perl the first postmodern computer language. More insightful to me, though, is work by James Noble and Robert Biddle, in particular Notes on Notes on Postmodern Programming, which I mentioned briefly a few years ago.

Shaw is right: there can be great value in studying at a university. We need to make sure that computer science students receive all of the value they should.


Posted by Eugene Wallingford | Permalink | Categories: Computing

August 13, 2010 8:42 PM

Learning from Projects in Industry Training

Former student and current ThoughtWorker Chris Turner sent me an article on ThoughtWorks University's new project-based training course. I mentioned Chris once before, soon after he joined ThoughtWorks. (I also mentioned his very cool research on "zoomable" user interfaces, still one of my all-time favorite undergrad projects.)

Chris tool one of my early offerings of agile software development, one that tried to mix traditional in-class activities with a "studio approach" to a large team project. My most recent offering of the course turned the knobs a bit higher, with two weeks of lecture and pair learning exercises followed by two weeks of intensive project. I really like the results of the new course but wonder how I might be able to do the same kind of thing during the regular semester, when students take five courses and typically spend only three hours a week in class over fifteen weeks.

The ThoughtWorks U. folks do not work under such constraints and have even more focused time available than my one-month course. They bring students in for six weeks of full-time work. Not surprisingly they came to question the effectiveness of their old approach: five weeks of lecture and learning activities followed by a one-week simulation of a project. Most of the learning, it seemed, happened in context during the week-long project. Maybe they should expand the project? But... there is so much content to teach!

Eventually they asked themselves the $64,000 Question:

"What if we don't teach this at all? What's the worst that can happen?"

I love this question. When trying to balance practice in context with yet another lecture, university professors should ask this question about each element of the courses they teach. Often the answer is that students will have to learn the concept from their experience on real projects. Maybe students need more experience on real projects, not more lecture and more homework problems from the back of the textbook chapter!

The folks at TWU redesigned their training program for developers to consist of two weeks of training and four weeks of project work. And they -- and their students -- seem pleased with the results.

... information in context trumped instruction out of context in a huge way. The project was an environment for students to fail in safety. Failure created the need for people to learn and a catalyst for us to coach and teach. A real project environment also allowed students to learn to learn.

This echoes my own experience and is one of the big reasons I think so much about project-based courses. Students still need to learn ideas and concepts, and some will need more direct individual assistance to pick them. The ThoughtWorks folks addressed that need upfront:

We also created several pieces of elearning to help students gain some basic skills when they needed them. Coupled with a social learning platform and a 6:1 student-coach ratio, we were looking at a program that focussed heavily on individualisation as against an experience that was one-size-fits-all-but-fits-nobody. Even with the elearning, we ensured that we were pragmatic in partnering with external content providers whose content met our quality standards.

This is a crucial step, and one that I would like to improve before I teach my agile course again. I found lots of links to on-line resources students could use to learn about agile and XP, but I need to create better materials in some areas and create materials to fill gaps in the accessible web literature. If I want to emphasize the project in my compiler course even more, I will need to create a lot of new materials. What I'd really like to do is create active e-learning resources, rather than text to read. The volume, variety, and quality of supporting materials is even more important if we want to make projects the central activity in courses for beginners.

By the way, I also love the phrase "one-size-fits-all-but-fits-nobody".

When faculty who teach more traditional courses in more traditional curricula hear stories such as this one from TWU, they always ask me the same question: How much does the success of such an industry training program depend on "basic knowledge" students learned in traditional courses? I wonder the same thing. Could we start CS1 or CS2 with two weeks of regular classes followed by four weeks of project? What would work and what wouldn't? Could we address the weaknesses to make the idea work? If we could, student motivation might reach a level higher than we see now. Even better, student learning might be improved as they encounter ideas as they need them to solve problems that matter. (For an opinion to the contrary, see Moti Ben-Ari's comments as reported by Mark Guzdial.)

School starts in a week, so my thoughts have turned to my compiler course. This course already based on one of the classic project experiences that CS students can have. There is a tendency to think all is well with the basic structure of the course and that we should leave it alone. That's not really my style. Having taught compilers any times, I know my course's strengths and weaknesses and know that it can be improved. The extent to which I change it is always a open question.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

August 10, 2010 3:36 PM

In Praise of Attacking Hard Problems

With the analysis of Deolalikar's P != NP paper now under way in earnest, I am reminded of a great post last fall by Lance Fortnow, The Humbling Power of P v NP. Why should every theorist try to prove P = NP and P != NP?

Not because you will succeed but because you will fail. No matter what teeth you sink into P vs NP, the problem will bite back. Think you solved it? Even better. Not because you did but because when you truly understand why your proof has failed you will have achieved enlightenment.

You might even succeed, though I'm not sure if the person making the attempt achieves the same kind of enlightenment in that case.

Even if Deolalikar's proof holds up, Fortnow's short essay will still be valuable and true.

We'll just use a different problem as our standard.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

August 02, 2010 1:54 PM

Computing and Education in My News

My newreader and inbox are full of recent articles about computing and education in the news. First, there is a New York Times Technology section piece on Scott McNealy's open-source textbook project, Currwiki. When I first read this, I thought for sure I would blog on the idea of free and open-source textbooks today. The more I thought about it, and especially the more I tried to write about it, the less I found I have to say right now. Mark Guzdial has already responded with a few concerns he has about open-source textbooks. Guzdial conflates "open source" with "free", as does the Times piece, though McNealy's project seems to be mostly about offering low-cost or free alternatives to increasingly expensive school books. Most of Guzdial's concerns echo the red flags people have raised about free and open-source software in the past, and we see the effect FOSS has had in the world.

Maybe I'll have something cogent to say some other day, but for now, all I can think is, "Is that a MacBookPro in the photo of McNealy and his son?" If so, even well-placed pencil holder can't hide the truth!

Then there is a blog entry at Education Week on the Computer Science Education Act, a bill introduced in the U.S. House of Representatives last week aimed at improve the state of K-12 CS education. Again, any initial excitement to write at length on this topic faded as I thought more about it. This sort of bill is introduced all the time in Congress with little or no future, so until I see this one receive serious attention from House leaders I'll think of it as mostly good PR for computer science. I do not generally think that legislation of this kind has a huge effect on practice in the schools, which are much too complicated to be pushed off course by a few exploratory grants or a new commission. That said, it's nice that a few higher-ups in education might think deeply about the role CS might and could play in 21st-century K-12 education. This ain't 1910, folks.

Finally, here's one that I can blog about with excitement and a little pride: One of my students, Nick Cash, has been named one of five finalists in Entrepreneur Magazine's Entrepreneur of 2010 contest. Nick is one of those bright guys for who our education system is a poor fit, because he is thinking bigger thoughts than "when is the next problem set due?" He has been keeping me apprised of his start-up every so often, but things change so fast that it is hard for me to keep up.

One of the things that makes me proud is the company he is keeping in that final five. Maryland and Michigan are big-time universities with big-time business schools. Though you may not have heard of Babson College, it has long had one of the top-ranking undergraduate entrepreneurship programs in the country. (I'm that in part because I double-majored in accounting at Ball State University, which also has a top-ranked entrepreneurship center for undergrads.) UNI has been doing more to support student entrepreneurship over the last few years, including an incubator for launching start-ups. Still, Nick has made it to the finals against students who come from better-funded and better-known programs. That says even more about his accomplishment.

Nick's company, Book Hatchery, is certainly highly relevant in today's digital publishing market. I'll be wishing him well in the coming years and helping in any way he asks. Check out the link above and, if you are so inclined, cast a vote for his start-up in the contest!


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

July 28, 2010 2:26 PM

Sending Bad Signals about Recursion

A couple of weeks ago there was a sustained thread on the SIGCSE mailing list on the topic of teaching recursion. Many people expressed strongly held opinions, though most of those were not applicable outside the poster's own context. Only a few of the views expressed were supported by educational research.

My biggest take-away from the discussion was this: I can understand why CS students at so many schools have a bad impression of recursion. Like so many students I encounter, the faculty who responded expressed admiration for the "beauty and elegance" of recursion but seemed to misunderstand at a fundamental level how to use it as a practical tool.

The discussion took a brief side excursion into the merits of Towers of Hanoi as a useful example for teaching recursion to students in the first year. It is simple and easy for students to understand, said the proponents, so that makes it useful. In my early years of teaching CS1/CS2, I used Towers as an example, but long ago I came to believe that real problems are more compelling and provide richer context for learning. (My good friend Owen Astrachan has been sounding the clarion call on this topic for a few years now, including a direct dig on the Towers of Hanoi!)

My concern with the Towers is more specific when we talk about recursion. One poster remarked that this problem helped students to see that recursion can be slow:

Recursion *is* slow if you're solving a problem that is exponentially hard like Hanoi. You can't solve it faster than the recursive solution, so I think Hanoi is a perfectly fine example of recursion.

M. C. Escher's 'Drawing Hands'

This, I think, is one of the attitudes that gives our students an unduly bad impression of recursion, because it confounds problem and solution. Most students leave their first-year courses thinking that recursion is slow and computationally expensive. This is in part an effect of the kinds of problems we solve with recursion there. The first examples we show students of loops tend not to solve exponentially hard problems. This leads students to infer that loops are fast and recursion is slow, when the computational complexity was a feature of the problems, not the solutions. A loop-based solution to Towers would be slow and use a lot of space, too! We can always tell our students about the distinction, but they see so few examples of recursion that they are surely left with a misimpression, through no fault of their own.

Another poster commented that he had once been a consultant on a project at a nuclear reactor. One of the programmers proudly showed one of their programs that used recursion to solve one of their problems. By using recursion, they had been able to construct a straightforward inductive proof of the code's correctness. The poster chided the programmer, because the code was able to overflow the run-time stack and fail during execution. He encouraged them to re-write the code using a subset of looping constructs that enables proofs over the limited set of programs it generates. Recursion cannot be used in real-time systems, he asserted, for just this reason.

Now, I don't want run-time errors in the code that runs our nuclear reactors or any other real-time system, for that matter, but that conclusion is a long jump from the data. I wrote to this faculty member off-list and asked whether the programming language in question forbids, allows, or requires the compiler to optimize recursive code or, more specifically, tail calls. With tail call optimization, a compiler can convert a large class of recursive functions to a non-recursive run-time implementation. This means that the programmer could have both a convincing inductive proof of the code's correctness and a guarantee that the run-time stack will never grow beyond the initial stack frame.

The answer was, yes, this is allowed, and the standard compilers provide this as an option. But he wasn't interested in discussing the idea further. Recursion is not suitable for real-time systems, and that's that.

It's hard to imagine students developing a deep appreciation for recursion when their teachers believe that recursion is inappropriate independent of any evidence otherwise. Recursion has strengths and weaknesses, but the only strengths most students seem to learn about are its beauty and its elegance. Those are code words in many students' minds for "impractical" and, when combined with a teacher's general attitude toward the technique, surely limit our students' ability to get recursion.

I'm not claiming that it's easy for students to learn recursion, especially in the first year, when we tend to work with data that make it hard to see when recursion really helps. But it's certainly possible to help students move from naive recursive solutions to uses of an accumulator variable that enable tail-recursive implementations. Whether that is a worthwhile endeavor in the first year, given everything else we want to accomplish there, is the real question. It is also the question that underlay the SIGCSE thread. But we need to make sure that our students understand recursion and know how to use it effectively in code before they graduate. It's too powerful a tool to be missing from their skill set when they enter the workforce.

As I opened this entry, though, I left the discussion not very hopeful. The general attitude of many instructors may well get in the way of achieving that goal. When confronted with evidence that one of their beliefs is a misconception, too many of them shrugged their shoulders or actively disputed the evidence. The facts interfered with what they already know to be true!

There is hope, though. One of Ursula Wolz's messages was my favorite part of the conversation. She described a study she conducted in grad school teaching recursion to middle-schoolers using simple turtle graphics. From the results of that study and her anecdotal experiences teaching recursion to undergrads, she concluded:

Recursion is not HARD. Recursion is accessible when good models of abstraction are present, students are engaged and the teacher has a broad rather than narrow agenda.

Two important ideas stand out of this quote for me. First, students need to have access to good models of abstraction. I think this can be aided by using problems that are rich enough to support abstractions our students can comprehend. Second, the teacher must have a broad agenda, not a narrow one. To me, this agenda includes not only the educational goals for the lesson but also general message that we want to send our students. Even young learners are pretty\ good at sensing what we think about the material we are teaching. If we convey to them that recursion is beautiful, elegant, hard, and not useful, then that's what they will learn.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

July 26, 2010 3:28 PM

Digital Cameras, Electric Guitars, and Programming

Eric Clapton at the Crossroads Festival 2010

I often write here about programming for everyone, or at least for certain users. Won't that put professional programmers out of business? Here is a great answer to a related question from a screenwriter using an analogy to music:

Well, if I gave you an electric guitar, would you instantly become Eric Clapton?

There is always room for specialists and artists, people who take literacy to a higher level. We all can write a little bit, and pretty soon everyone will be blogging, tweeting, or Facebooking, but we still need Shakespeare, James Joyce, and Kurt Vonnegut. There is more to writing than letters and sentences. There is more to programming than tokens and procedures. A person with ideas can create things we want to read and use.

Sometimes the idea is as simple as hooking up two existing ideas. I may be late to the party, but @bc_l is simply too cool:

I'm GNU bc on twitter! DM me your math and I'll tell you the answer. (by @hmason)

@hmason is awesome.

On a more practical note, I use dc as the target language for a simple demo compiler in my compilers course, following the lead of Fischer, Cytron, and LeBlanc in Crafting a Compiler. I'm considering using the new edition of this text in my course this fall, in part because of its support for virtual machines as targets and especially the JVM. I like where my course has been the last couple of offerings, but this seems like an overdue change in my course's content. I may as well start moving the course. Eventually, targeting multi-core architectures will be essential.

If I want to help students who dream of being Eric Clapton with a keyboard, I gotta keep moving.

~~~~

(The image above is courtesy of PedalFreak at flickr, with a Attribution-NoDerivs 2.0 Generic license.)


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

July 22, 2010 4:19 PM

Days of Future Passed: Standards of Comparison and The Value of Limits

This morning, a buddy of mine said something like this as part of a group e-mail discussion:

For me, the Moody Blues are a perfect artist for iTunes. Obviously, "Days of Future Passed" is a classic, but everything else is pretty much in the "one good song on an album" category.

This struck as a reflection of an interesting way in which iTunes has re-defined how we think about music. There are a lot of great albums that everyone should own, but even for fans most artists produce only a song or two worth keeping over the long haul. iTunes makes it possible to cherry-pick individual songs in a way that relegates albums to second thought. A singer or band have achieved something notable if people want to buy the whole album.

That's not the only standard measure I encountered in that discussion.

After the same guy said, "The perfect Moody Blues disc collection is a 2-CD collection with the entirety of 'Days of Future Passed' and whatever else you can fit", another buddy agreed and went further (again paraphrased):

"Days of Future Passed" is just over half an 80-minute CD, and then I grabbed another 8 or 9 songs. That worked out right for me.

Even though CDs are semi-obsolete in this context, they still serve a purpose, as a sort of threshold for asking the question "How much music from this band do I really want to rip?"

When I was growing up, the standard was the 90-minute cassette tape. Whenever I created a collection for a band from a set of albums I did not want to own, I faced two limits: forty-five minutes on a side, and ninety minutes total. Those constraints caused me many moments of uncertainty as I tried to cull my list of songs into two lists that fit. Those moments were fun, though, too, because I spent a lot of time thinking about the songs on the bubble, listening and re-listening until I could make a comfortable choice. Some kids love that kind of thing.

Then, for a couple of decades the standard was the compact disc. CDs offered better quality with no halfway cut, but only about eighty minutes of space. I had to make choices.

When digital music leapt from the CD to the hard drive, something strange happened. Suddenly we were talking about gigabytes. And small numbers of gigabytes didn't last long. From 4- and 8-gigabyte devices we quickly jumped to iPods with a standard capacity of 160GB. That's several tens of thousands of songs! People might fill their iPods with movies, but most people won't ever need to fill them with the music they listen to on any regular basis. If they do, they always have the hard drive on the computer they sync the iPod with. Can you say "one terabyte", boys and girls?

The computer drives we use for music got so large so fast that they are no longer useful as the arbitrary limit on our collections. In the long run, that may well be a good thing, but as someone who has lived on both sides of the chasm, I feel a little sadness. The arbitrary limits imposed by LPs, cassettes, and CDs caused us to be selective and sometimes even creative. This is the same thing we hear from programmers who had to write code for machines with 128K of memory and 8 Mhz processors. Constraints are a source of creativity and freedom.

It's funny how the move to digital music has created one new standard of comparison via the iTunes store and destroyed another via effectively infinite hard drives. We never know quite how we and our world will change in response to the things we build. That's part of the fun, I think.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

July 21, 2010 4:17 PM

Two Classic Programs Available for Study

I just learned that the Computer History Museum has worked with Apple Computer to make source code for MacPaint and QuickDraw available to the public. Both were written by Bill Atkinson for the original Mac, drawing on his earlier work for the Lisa. MacPaint was the iconic drawing program of the 1980s. The utility and quality of this one program played a big role in the success of the Mac. Andy Hertzfeld, another Apple pioneer, credited QuickDraw for the success of the Mac for its speed at producing the novel interface that defined the machine to the public. These programs were engineering accomplishments of a different time:

MacPaint was finished in October 1983. It coexisted in only 128K of memory with QuickDraw and portions of the operating system, and ran on an 8 Mhz processor that didn't have floating-point operations. Even with those meager resources, MacPaint provided a level of performance and function that established a new standard for personal computers.

Though I came to Macs in 1988 or so, I was never much of a MacPaint user, but I was aware of the program through friends who showed me works they created using it. Now we can look under the hood to see how the program did what it did. Atkinson implemented MacPaint in one 5,822-line Pascal program and four assembly language files for the Motorola 6800 totaling 3,583 lines. QuickDraw consists of 17,101 lines of Motorola 6800 assembly in thirty-seven modules.

I speak Pascal fluently and am eager to dig into the main MacPaint program. What patterns will I recognize? What design features will surprise me, and teach me something new? Atkinson is a master programmer, and I'm sure to learn plenty from him. He was working in an environment that so constrained his code's size that he had to do things differently than I ever think about programming.

This passage from the Computer History Museum piece shares a humorous story that highlights how Atkinson spent much of his time tightening up his code:

When the Lisa team was pushing to finalize their software in 1982, project managers started requiring programmers to submit weekly forms reporting on the number of lines of code they had written. Bill Atkinson thought that was silly. For the week in which he had rewritten QuickDraw's region calculation routines to be six times faster and 2000 lines shorter, he put "-2000" on the form. After a few more weeks the managers stopped asking him to fill out the form, and he gladly complied.

This reminded me of one of my early blog entries about refactoring. Code removed is code earned!

I don't know assembly language nearly as well as I know Pascal, let alone Motorola 6800 assembly, but I am intrigued by the idea of being able to study more than 20,000 lines of assembly language that work together on a common task and which also exposes a library API for other graphics programs. Sounds like great material for a student research project, or five...

I am a programmer, and I love to study code. Some people ask why anyone would want to read listings of any program, let alone a dated graphics program from more than twenty-five years ago. If you use software but don't write it, then you probably have no reason to look under this hood. But keep in mind that I study how computation works and how it solves problems in a given context, especially when it has limited access to time, space, or both.

But... People write programs. Don't we already know how they work? Isn't that what we teach CS students, at least ones in practical undergrad departments? Well, yes and no. Scientists from other disciplines often ask this question, not as a question but as an implication that CS is not science. I have written on this topic before, including this entry about computation in nature. But studying even human-made computation is a valuable activity. Building large systems and building tightly resource-constrained programs are still black arts.

Many programmers could write a program with the functionality of MacPaint these days, but only a few could write a program that offers such functionality under similar resource limitations. That's true even today, more than two decades after Atkinson and others of his era wrote programs like this one. Knowledge and expertise matter, and most of it is hidden away in code that most of us never get to see. Many of the techniques used by masters are documented either not well or not at all. One of the goals of the software patterns community is to document techniques and the design knowledge needed to use them effectively. And one of the great services of the free and open-source software communities is to make programs and their source code accessible to everyone, so that great ideas are available to anyone willing to work to find them -- by reading code.

Historically, engineering has almost always run ahead of science. Software scientists study source code in order to understand how and why a program works, in a qualitatively different way than is possible by studying a program from the outside. By doing so, we learn about both engineering (how to make software) and science (the abstractions that explain how software works). Whether CS is a "natural" science or not, it is science, and source code embodies what it studies.

For me, encountering the release of source code for programs such as MacPaint feels something like a biologist discovering a new species. It is exciting, and an invitation to do new work.

Update: This is worth an update: a portrait of Bill Atkinson created in MacPaint. Well done.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

July 19, 2010 4:48 PM

"I Blame Computers."

I spent the weekend in southwestern Ohio at Hueston Woods State Park lodge with a bunch of friends from my undergrad days. This group is the union of two intersecting groups of friends. I'm a member of only one but was good friends with the two main folks in the intersection. After over twenty years, with close contact every few years, we remain bonded by experiences we shared -- and created -- all those years ago.

The drives to and from the gathering were more eventful than usual. I was stopped by a train at same railroad crossing going both directions. On the way there, a semi driver intentionally ran me off the road while I was passing him on the right. I don't usually do that, but he had been driving in the left lane for quite a while, and none too fast. Perhaps I upset him, but I'm not sure how. Then, on the way back, I drove through one of the worst rainstorms I've encountered in a long while. It was scarier than most because it hit while I was on a five-lane interstate full of traffic in Indianapolis. The drivers of my hometown impressed me by slowing down, using their hazard lights, and cooperating. That was a nice counterpoint to my experience two days earlier.

Long ago, my mom gave me the New Testament of the Bible on cassette tape. (I said it was long ago!) When we moved to a new house last year, I came across the set again and have had it in pile of stuff to handle ever since. I was in an unusual mood last week while packing for the trip and threw the set in the car. On the way to Ohio, I listened to Gospel of Matthew. I don't think I have ever heard or read an entire gospel in one setting before. After hearing Matthew, I could only think, "This is a hard teaching." (That is a line from another gospel, by John, the words and imagery of which have always intrigued me more than the other gospels.)

When I arrived on Friday, I found that the lodge did offer internet service to the rooms, but at an additional cost. That made it easier for me to do what I intended, which was to spend weekend off-line and mostly away from the keyboard. I enjoyed the break. I filled my time with two runs (more on them soon) and long talks with friends and their families.

Ironically, conversation late on Saturday night turned to computers. The two guys I was talking with are lawyers, one for the Air Force at Wright Patterson Air Force Base and one for a U.S. district court in northern Indiana. Both lamented the increasing pace of work expected by their clients. "I blame computers," said one of the guys.

In the old days, documents were prepared, duplicated, and mailed by hand. The result was slow turnaround times, so people came to expect slow turnaround. Computers in the home and office, the Internet, and digital databases have made it possible to prepare and communicate documents almost instantly. This has contributed to two problems they see in their professional work. First, the ease of copy-and-paste has made it even easier to create documents that are bloated or off-point. This can be used to mislead, but in their experience the more pernicious problem is lack of thoughtfulness and understanding.

Second, the increased speed of communication has led to a change in peoples' expectations about response. "I e-mailed you the brief this morning. Have you resolved the issue this morning?" There is increasing pressure to speed up the work cycle and respond faster. Fortunately, both report that these pressures come only from outside. Neither the military brass nor the circuit court judges push them or their staff to work faster, and in fact encourage them to work with prudence and care. But the pressure on their own staff from their clients grows.

Many people lash out and blame "computers" for whatever ills of society trouble them. These guys are bright, well-read, and thoughtful, and I found their concerns about our legal system to be well thought out. They are deeply concerned by what the changes mean for the cost and equitability of the justice the system can deliver. The problem, of course, is not with the computers themselves but with how we use them, and perhaps with how they change us. For me as a computer scientist, that conversation was a reminder that writing a program does not always solve our problems, and sometimes it creates new ones. The social and cultural environments in which programs operate are harder to understand and control than our programs. People problems can be much harder to solve than technical problems. Often, when we solve technical problems, we need to be prepared for unexpected effects on how people work and think.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

July 04, 2010 3:49 PM

Sharing Skills and Joys with Teachers

For many of the departments in my college, a big part of summer is running workshops for K-12 teachers. Science education programs reside in the science departments here, and summer is the perfect time for faculty in biology, physical science, and the like to work with their K-12 colleagues and keep them up to date on both their science and the teaching of it.

Not so in Computer Science; we don't have a CS education program. My state does not certify CS teachers, so their is no demand from teachers for graduate credits to keep licenses up to date. Few schools teach any CS or even computer programming these days at all, so there aren't even many teachers interested in picking up CS content or methods for their own courses. My department did offer a teaching minor for undergrads and a master's degree in CS education for many years in the 1980s and 1990s, but the audience slowly dried up and we dropped the programs.

Still, several CS faculty and I have long talked about how valuable it might be for recruiting to share the thrill of computing with high school teachers. With a little background, they might be more motivated -- and better equipped -- to share that excitement with their students.

Google CS4HS logo

That's what makes this summer so exciting. With support from Google's CS4HS program, we are offering a workshop for middle and high school teachers, Introducing Computing via Scratch and Simulation. Because there are not many CS-only teachers in our potential audience, we pitched the workshop to science and math teachers, with a promise that the workshop would help them learn how to use computing to demonstrate concepts in their disciplines and to build simple simulations for their students. We have also had some positive experiences working with middle-school social science students at the local lab school, so we included social science and humanities teachers in our call for participants. (Scratch is a great tool for story telling!)

Our workshop will reflect a basic tenet my colleagues and I hold: the best way to excite people about computing is to show them its power. Our main focus is on how they can use CS to teach their science, math, and other courses better. But we will also begin to hint at how they can use simple scripts to make their lives better, whether to prepare data for classroom examples or to handle administrative tasks. Scratch will be the main medium we teach them, but we will also point them toward something like Python. Glitz gets people's attention, but glitz fades quickly when the demands of daily life return to the main stage. The power of computing may help keep their attention.

Google is expanding this program, which they piloted last year at a couple of top schools. This year, several elite schools are offering workshops, but also several schools like mine, schools working in the trenches both of CS but also teacher preparation. As the old teachers' college in my state, we prepare a large percentage of the K-12 teachers here.

The grant from Google, awarded through a competitive proposal process, helped us to take the big step of developing a workshop and trying to sell it to the teachers of our area. We were not sure how many teachers would be willing to spend four days (three this summer and a follow-up day in the fall) to study computer science. The Google award allowed us to offer small stipends to lower the barrier to attendance. We also kept expenses low by donating instructor time, which allowed us to offer the workshop at the lowest cost possible to teachers. They result was promising: more teachers signed up than we have stipends for.

Next comes the fun part: preparing and teaching the workshop!


Posted by Eugene Wallingford | Permalink | Categories: Computing

June 08, 2010 4:46 PM

No, the RICO Act Does Not Apply

The mafioso over at PLT have done it. They have renamed their suite of languages, editors, tools, and teaching materials from the collective "PLT Scheme" to Racket. As they explain on their web site, the Scheme part of the old name had grown increasingly less useful for identifying their work and the languages they had created. As a result, they had to continually distinguish their work from the work of the mainstream Scheme community, which was similarly disadvantaged by having the PLT community's work consume much of the mindshare around the name Scheme. A name change solves both problems... with the new cost of having to re-brand PLT Scheme, Dr. Scheme, Mr. Ed, mzscheme, and the rest of their suite.

This is the first time I have ever seen a programming language community try to rebrand its language, so this will be fun. I suspect the PLT folks will be just fine. They are both great researchers and tool builders, and they are remarkably energetic in disseminating their ideas. I don't think there is any danger they will lose many of their current devotees, if any. The real question is whether the new name will make it easier or harder to bring new users on board. Because "Racket is a Scheme" and Racket is not called Scheme, I figure the name change is at worst a wash and at best a perceptive move. Over the next couple of years, we'll see which is true.

If nothing else, this is likely to force me out of my holding pattern with PLT Scheme 4.2.0 and into the modern era of PLT, er, Racket 5.0. The #lang construct is such a nice solution to several different issues in creating languages and building large systems. Now I'll have to try it out for real! I'll also have to convert all of the code for my programming languages course, which will give me a little more incentive to make some larger changes in what I teach in the course and how I teach it. That's good news, too -- an extra shot of energy.


Posted by Eugene Wallingford | Permalink | Categories: Computing

May 25, 2010 4:16 PM

A Book on Alan Kay / Alan Kay on Science and Engineering

Everyone already seems to know about this, but I didn't until yesterday, when I saw a note from Hal Perkins...

In honor of Alan Kay's 70th birthday, a bunch of his colleagues and best friends have published Points of View, "a collection of previously-unpublished essays". The list of authors ranges from CS greats such as Ivan Sutherland, Butler Lampson, and Leonard Kleinrock, to luminaries from other human endeavors such as Betty Edwards and Quincy Jones. The list of essay titles makes me want to drop all of my work and read it from cover to cover right now. There may be a second printing of the book, but it is available in PDF from the website for free.

Very cool.

I've been meaning to post a short snippet from Kay for the last few weeks. He wrote a guest entry on Mark Guzdial's blog about the danger of trying to make learning 'too simple', a common theme in how Kay talks about understanding and building systems. The passage that really attracted me was on something else, though, in a comment responding to other commenters' comments. Considering that Kay's blog entry itself began as a comment intended for another entry, we have more than enough indirection to make us computer scientists happy for weeks!

In his comment, Kay describes some of his team's current work at Viewpoints, in which they extract from programs of millions of code an essential core that is executable, readable, and yet several orders of magnitude smaller than the original. He uses this description as a vehicle for explaining science and engineering:

In this direction -- where we have phenomena already, and our task is to make a much more compact, understandable and debuggable model -- we are doing actual science (this is what science does and is).

But we can also use the model idea to go in the other direction -- to come up with an idea, make a runnable debuggable model of it -- and then use this as a guide and reference ... for manifesting the ... artifact -- here, we are doing invention and engineering.

Computer scientists are often asked to explain why CS is a science. Kay has always been very clear that science is about method, not subject matter. His explanation of science and engineering nicely captures how CS can be science, how CS can be engineering, and how CS can be both. The last these is part of what distinguishes computer science from other disciplines, precisely because CS is about models of systems.

For me, this simple example was the big win from Kay's guest post on Mark's blog. It's also a fitting example of how Kay thinks about the world and why accomplished people would take the time to write essays to celebrate his birthday.


Posted by Eugene Wallingford | Permalink | Categories: Computing

April 22, 2010 8:36 PM

At Some Point, You Gotta Know Stuff

A couple of days ago, someone tweeted a link to Are you one of the 10% of programmers who can write a binary search?, which revisits a passage by Jon Bentley from twenty-five years ago. Bentley observed back than that 90% of professional programmers were unable to produce a correct version of binary search, even with a couple of hours to work. I'm guessing that most people who read Bentley's article put themselves in the elite 10%.

Mike Taylor, the blogger behind The Reinvigorated Programmer, challenged his readers. Write your best version of binary search and report the results: is it correct or not? One of his conditions was that you were not allowed to run tests and fix your code. You had to make it run correctly the first time.

Writing a binary search is a great little exercise, one I solve every time I teach a data structures course and occasionally in courses like CS1, algorithms, and any programming language- or style-specific course. So I picked up the gauntlet.

You can see my solution in a comment on the entry, along with a sheepish admission: I inadvertently cheated, because I didn't read the rules ahead of time! (My students are surely snickering.) I wrote my procedure in five minutes. The first test case I ran pointed out a bug in my stop condition, (>= lower upper). I thought for a minute or so, changed the condition to (= lower (- upper 1)), and the function passed all my tests.

In a sense, I cheated the intent of Bentley's original challenge in another way. One of the errors he found in many professional developers' solution was an overflow when computing the midpoint of the array's range. The solution that popped into my mind immediately, (lower + upper)/2, fails when lower + upper exceeds the size of the variable used to store the intermediate sum. I wrote my solution in Scheme, which handle bignums transparently. My algorithm would fail in any language that doesn't. And to be honest, I did not even consider the overflow issue; having last read Bentley's article many years ago, I had forgotten about that problem altogether! This is yet another good reason to re-read Bentley occasionally -- and to use languages that do heavy lifting for you.

But.

One early commenter on Taylor's article said that the no-tests rule took away some of my best tools and his usual way of working. Even if he could go back to basics, working in an unfamiliar probably made him less comfortable and less likely to produce a good solution. He concluded that, for this reason, a challenge with a no-tests rule is not a good test of whether someone is a good programmer.

As a programmer who prefers an agile style, I felt the same way. Running that first test, chosen to encounter a specific possibility, did exactly what I had designed it to do: expose a flaw in my code. It focused my attention on a problem area and caused me to re-examine not only the stopping condition but also the code that changed the values of lower and upper. After that test, I had better code and more confidence that my code was correct. I ran more tests designed to examine all of the cases I knew of at the time.

As someone who prides himself in his programming-fu, though, I appreciated the challenge of trying to design a perfect piece of code in one go: pass or fail.

This is a conundrum to me. It is similar to a comment that my students often make about the unrealistic conditions of coding on an exam. For most exams, students are away from their keyboards, their IDEs, their testing tools. Those are big losses to them, not only in the coding support they provide but also in the psychological support they provide.

The instructor usually sees things differently. Under such conditions, students are also away from Google and from the buddies who may or may not be writing most of their code in the lab. To the instructor, This nakedness is a gain. "Show me what you can do."

Collaboration, scrapheap programming, and search engines are all wonderful things for software developers and other creators. But at some point, you gotta know stuff. You want to know stuff. Otherwise you are doomed to copy and paste, to having to look up the interface to basic functions, and to being able to solve only those problems Google has cached the answers to. (The size of that set is growing at an alarming rate.)

So, I am of two minds. I agree with the commenter who expressed concern about the challenge rules. (He posted good code, if I recall correctly.) I also think that it's useful to challenge ourselves regularly to solve problems with nothing but our own two hands and the cleverness we have developed through practice. Resourcefulness is an important trait for a programmer to possess, but so are cleverness and meticulousness.

Oh, and this was the favorite among the ones I read:

I fail. ... I bring shame to professional programmers everywhere.

Fear not, fellow traveler. However well we delude ourselves about living in a Garrison Keillor world, we are all in the same boat.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

April 20, 2010 9:58 PM

Computer Code in the Legal Code

You may have heard about a recent SEC proposal that would require issuers of asset-backed securities to submit "a computer program that gives effect to the flow of funds". What a wonderful idea!

I have written a lot in this blog about programming as a new medium, a way to express processes and the entities that participate in them. The domain of banking and finance is a natural place for us to see programming enter into the vernacular as the medium for describing problems and solutions more precisely. Back in the 1990s, Smalltalk had a brief moment in the sunshine as the language of choice used by financial houses in the creation of powerful, short-lived models of market behavior. Using a program to describe models gave the investors and arbitrageurs not only a more precise description of the model but also a live description, one they could execute against live data, test and tinker with, and use as an active guide for decision-making.

We all know about the role played by computer models in the banking crisis over the last few years, but that is an indictment of how the programs were used and interpreted. The use of programs itself was and is the right way to try to understand a complex system of interacting, independent agents manipulating complex instruments. (Perhaps we should re-consider whether to traffic in instruments so complex that they cannot be understand without executing a complex program. But that is a conversation for another blog entry, or maybe a different blog altogether!)

What is the alternative to using a program to describe the flow of funds engendered by a particular asset-backed security? We could describe these processes using text in a natural language such as English. Natural language is supremely expressive but fraught with ambiguity and imprecision. Text descriptions rely on the human reader to do most of the work figuring out what they mean. They are also prone to gratuitous complexity, which can be used to mislead unwary readers.

We could also describe these processes using diagrams, such as a flow chart. Such diagrams can be much more precise than text, but they still rely on the reader to "execute" them as she reads. As the diagrams grow more complex, the more difficult it is for the reader to interpret the diagram correctly.

A program has the virtue of being both precise and executable. The syntax and semantics of a programming are (or at least can be) well-defined, so that a canonical interpreter can execute any program written in the language and determine its actual value. This makes describing something like the flow of funds created by a particular asset-backed security as precise and accurate as possible. A program can be gratuitously complex, which is a danger. Yet programmers have at their disposal tools for removing gratuitous complexity and focusing on the essence of a program, moreso than we have for manipulating text.

The behavior of the model can still be complex and uncertain, because it depends on the complexity and uncertainty of the environment in which it operates. Our financial markets and the economic world in which asset-backed securities live are enormously complex! But at least we have a precise description of the process being proposed.

As one commentator writes:

When provisions become complex beyond a point, computer code is actually the simplest way to describe them... The SEC does not say so, but it would be useful to add that if there is a conflict between the software and textual description, the software should prevail.

Using a computer program in this way is spot on.

After taking this step, there are still a couple of important issues yet to decide. One is: What programming language should we use? A lot of CS people are talking about the proposal's choice of Python as the required language. I have grown to like Python quite a bit for its relative clarity and simplicity, but I am not prepared to say that it is the right choice for programs that are in effect "legal code". I'll let people who understand programming language semantics better than I make technical recommendations on the choice of language. My guess is that a language with a simpler, more precisely defined semantics would work better for this purpose. I am, of course, partial to Scheme, but a number of functional languages would likely do quite nicely.

Fortunately, the SEC proposal invites comments, so academic and industry computer scientists have an opportunity to argue for a better language. (Computer programmers seem to like nothing more than a good argument about language, even writing programs in their own favorite!)

The most interesting point here, though, is not the particular language suggested but that the proposers suggest any programming language at all. They recognize how much more effectively a computer program cab describe a process than text or diagrams. This is a triumph in itself.

Other people are reminding us that mortgage-backed CDOs at the root of the recent financial meltdown were valued by computer simulations, too. This is where the proposal's suggestion that the code be implemented in open-source software shines. By making the source code openly available, everyone has the opportunity and ability to understand what the models do, to question assumptions, and even to call the authors on the code's correctness or even complexity. The open source model has worked well in the growth of so much of our current software infrastructure, including the simple in concept but complex in scale Internet. Having the code for financial models be open brings to bear a social mechanism for overseeing the program's use and evolution that is essential in a market that should be free and transparent.

This is also part of the argument for a certain set of languages as candidates for the code. If the language standard and implementations of interpreters are open and subject to the same communal forces as the software, this will lend further credibility to the processes and models.

I spend a lot of time talking about code in this blog. This is perhaps the first time I have talked about legal code -- and even still I get to talk about good old computer code. It's good to see programs recognized for what they are and can be.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

April 06, 2010 7:36 PM

Shame at Causing Loved Ones Harm

The universe tends toward maximum irony.
Don't push it.
-- JWZ

I have had Jamie Zawinski's public service announcement on backups sitting on desk since last fall. I usually keep my laptop and my office machine pretty well in sync, so I pretty much always have a live back-up. But some files live outside the usual safety zone, such as a temporary audio files on my desktop, which also contains one or two folders of stuff. I knew I need to be more systematic and complete in safeguarding myself from disk failure, so printed Zawinski's warning and resolved to Do the Right Thing.

Last week, I read John Gruber's ode to backups and disk recovery. This article offers a different prescription but the same message. You must be 100% backed up, including even the files that you are editing now in the minutes or hours before the next backup. Drives fail. Be prepared.

Once again, I was energized to Do the Right Thing. I got out a couple of external drives that I had picked out for a good price recently. The plan was to implement a stable, complete backup process this coming weekend.

The universe tends toward maximum irony. Don't push it.

If the universe were punishing me for insufficient respect for its power, you would think that the hard drive in either my laptop or my office machine would have failed. But both chug along just fine. Indeed, I still have never had a hard drive fail in any of my personal or work computers.

It turns out that the universe's sense of irony is much bigger than my machines.

On Sunday evening, the hard drive in our family iMac failed. I rarely use this machine and store nothing of consequence there. Yet this is a much bigger deal.

My wife lost a cache of e-mail, an address book, and a few files. She isn't a big techie, so she didn't have a lot to lose there. We can reassemble the contact information at little cost, and she'll probably use this as a chance to make a clean break from Eudora and POP mail and move to IMAP and mail in the cloud. In the end, it might be a net wash.

My teenaged daughters are a different story. They are from a new generation and live a digital life. They have written a large number of papers, stories, and poems, all of which were on this machine. They have done numerous projects for schools and extracurricular activities. They have created artwork using various digital tools. They have taken photos. All on this machine, and now all gone.

I cannot describe how I felt when I first realized what had happened, or how I feel now, two days later. I am the lead techie in our house, the computer science professor who knows better and preaches better, the husband and father who should be taking care of what matters to his family. This is my fault. Not that the hard drive failed, because drives fail. It is my fault that we don't have a reliable, complete backup of all the wonderful work my daughters have created.

Fortunately, not all is lost. At various times, we have copied files to sundry external drives and servers for a variety of reasons. I sometimes copy poetry and stories and papers that I especially like onto my own machines, for easy access. The result is a scattering of files here and there, across a half dozen machines. I will spend the next few days reassembling what we have as best I can. But it will not be all, and it will not be enough.

The universe maximized its irony this time around by getting me twice. First, I was gonna do it, but didn't.

That was just the head fake. I was not thinking much at all about our home machine. That is where the irony came squarely to rest.

Shut up. I know things. You will listen to me. Do it anyway.

Trust Zawinski, Gruber, and every other sane computer user. Trust me.

Do it. Run; don't walk. Whether your plan uses custom tools or a lowly cron running rysnc, do it now. Whether you go as far as using a service such as dropbox to maintain working files or not, set up an automatic, complete, and bootable backup of your hard drives.

I know I can't be alone. There must be others like me out there. Maybe you used to maintain automatic and complete system backups and for whatever reason fell out of the habit. Maybe you have never done it but know it's the right thing to do. Maybe, for whatever reason, you have never thought about a hard drive failing. You've been lucky so far and don't even know that your luck might change at any moment.

Do it now, before dinner, before breakfast. Do it before someone you love loses valuable possessions they care deeply about.

I will say this: my daughters have been unbelievable through all this. Based on what happened Sunday night, I certainly don't deserve their trust or their faith. Now it's time to give them what they deserve.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Personal

March 31, 2010 3:22 PM

"Does Not Play Well With Others"

Today I ran across a recent article by Brian Hayes on his home-baked graphics. Readers compliment him all the time on the great graphics in his articles. How does he do it? they ask. The real answer is that he cares what they look like and puts a lot of time into them. But they want to know what tools he uses. The answer to that question is simple: He writes code!

His graphics code of choice is PostScript. But, while PostScript is a full-featured postfix programming language, it isn't the sort of language that many people want to write general-purpose code in. So Hayes took the next natural step for a programmer and built his own language processor:

... I therefore adopted the modus operandi of writing a program in my language of choice (usually some flavor of Lisp) and having that program write a PostScript program as its output. After doing this on an ad hoc basis a few times, it became clear that I should abstract out all the graphics-generating routines into a separate module. The result was a program I named lips (for Lisp-to-PostScript).

Most of what lips does is trivial syntactic translation, converting the parenthesized prefix notation of Lisp to the bracketless postfix of PostScript. Thus when I write (lineto x y) in Lisp, it comes out x y lineto in PostScript. The lips routines also take care of chores such as opening and closing files and writing the header and trailer lines required of a well-formed PostScript program.

Programmers write code to solve problems. More often than many people, including CS students, realize, programmers write a language processor or even create a little language of their own to make solving the more convenient. We have been covering the idea of syntactic abstractions in my programming languages course for the last few weeks, and Hayes offers us a wonderful example.

Hayes describes his process and programs in some detail, both lips and his homegrown plotting program plot. Still, he acknowledges that the world has changed since the 1980s. Nowadays, we have more and better graphics standards and more and better tools available to the ordinary programmer -- many for free.

All of which raises the question of why I bother to roll my own. I'll never keep up -- or even catch up -- with the efforts of major software companies or the huge community of open-source developers. In my own program, if I want something new -- treemaps? vector fields? the third dimension? -- nobody is going to code it for me. And, conversely, anything useful I might come up with will never benefit anyone but me.

Why, indeed? In my mind, it's enough simply to want to roll my own. But I also live in the real world, where time is a scarce resource and the list of things I want to do grows seemingly unchecked by any natural force. Why then? Hayes answers that question in a way that most every programmer I know will understand:

The trouble is, every time I try working with an external graphics package, I run into a terrible impedance mismatch that gives me a headache. Getting what I want out of other people's code turns out to be more work than writing my own. No doubt this reveals a character flaw: Does not play well with others.

That phrase stood me up in my seat when I read it. Does not play well with others. Yep, that's me.

Still again, Hayes recognizes that something will have to give:

In any case, the time for change is coming. My way of working is woefully out of date and out of fashion.

I don't doubt that Hayes will make a change. Programmers eventually get the itch even with their homebrew code. As technology shifts and the world changes, so do our needs. I suspect, though, that his answer will not be to start using someone else's tools. He is going to end up modifying his existing code, or writing new programs all together. After all, he is a programmer.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development

March 30, 2010 8:57 PM

Matthias Felleisen Wins the Karl Karlstrom Award

Early today, Shriram Krishnamurthi announced on the PLT Scheme mailing list that Matthias Felleisen had won the Karl Karlstrom Outstanding Educator Award. The ACM presents this award annually

to an outstanding educator who is .. recognized for advancing new teaching methodologies, or effecting new curriculum development or expansion in Computer Science and Engineering; or making a significant contribution to the educational mission of the ACM.

A short blurb in the official announcement touts Felleisen "for his visionary and long-term contributions to K-12 outreach programs, innovative textbooks, and pedagogically motivated software". Krishnamurthi, in his message to the community borne out of Felleisen's leadership and hard work, said it nicely:

Everyone on this list has been touched, directly or indirectly, by Matthias Felleisen's intellectual leadership: from his books to his software to his overarching vision and singular execution, as well as his demand that the rest of us live up to his extraordinary standards.

Matthias Felleisen

As an occasional Scheme programmer and a teacher of programmers, I have been touched by Felleisen's work regularly over the last 20 years. I first read "The Little Lisper" long before I knew Matthias, and it changed how I approached programming with inductive data types. I assign "The Little Schemer" as the only textbook for my programming languages course, which introduces and uses functional programming. I have always felt as if I could write my own materials to teach functional programming and the languages content of the course, but "The Little Schemer" is a tour de force that I want my students to read. Of course, we also use Dr. Scheme and all of its tools for writing Scheme programs, though we barely scratch the surface of what it offers in our one-semester course.

We have never used Felleisen's book "How to Design Programs" in our introductory courses, but I consider its careful approach to teaching software design one of the most important intro CS innovations of the last twenty years. Back in the mid-1990s, when my department was making one of its frequent changes to the first-year curriculum, I called Matthias to ask his advice. Even after he learned that we were not likely to adopt his curriculum, he chatted me for a while and offered me pedagogical advice and even strategic advice my making a case for a curriculum based in a principle outside any given language.

That's one of the ironic things about Felleisen's contribution: He is most closely associated with Scheme and tools built in and for Scheme, but his TeachScheme! project is explicitly not about Scheme. (The "!" is even pronounced "not", a programming pun using the standard C meaning of the symbol.) TeachScheme! uses Scheme as a tool for creating languages targeted at novices who progress through levels of understanding and complexity. Just today in class, I talked with my students about Scheme's mindset of bringing to users of a language the same power available to language creators. This makes it an ideal intellectual tool for implementing Felleisen's curriculum, even as its relative lack of popularity has almost certainly hindered adoption of the curriculum more widely.

As my department has begun to reach out to engage K-12 students and teachers, I have come to appreciate just how impressive the TeachScheme! outreach effort is. This sort of engagement requires not only a zeal for the content but also sustained labor. Felleisen has sustained both his zeal and his hard work, all the while building a really impressive group of graduate students and community supporters. The grad students all seem to earn their degrees, move on as faculty to other schools, and yet remain a part of the effort.

Closer to my own work, I continue to think about the design recipe, which is the backbone of the HtDP curriculum. I remain convinced that this idea is compatible with the notion of elementary patterns, and that the design recipe can be integrated with a pattern language of novice programs harmoniously to create an even more powerful model for teaching new programmers how to design programs.

As Krishnamurthi wrote to the PLT Scheme developer and user communities, Felleisen's energy and ideas have enriched my work. I'm happy to see the ACM honor him for his efforts.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

March 28, 2010 6:15 PM

SIGCSE Day 3 -- Interdisciplinary Research

[A transcript of the SIGCSE 2010 conference: Table of Contents]

My SIGCSE seem to come to an abrupt end Saturday morning. After a week of long days, I skipped a last run in Milwaukee and slept in a bit. The front desk messed up my bill, so I spent forty-five minutes wangling for my guaranteed rate. As a result, I missed the Nifty Assignments session and arrived just in time for some time meeting with colleagues at the exhibits during the morning break. This left me time for one last session before my planned pre-luncheon departure.

I chose to attend the special session called Interdisciplinary Computing Education for the Challenges of the Future, with representatives of the National Science Foundation who have either carried out funded interdisciplinary research or who have funded and managed interdisciplinary programs. The purpose of the session was to discuss...

... the top challenges and potential solutions to the problem of educating students to develop interdisciplinary computing skills. This includes a computing perspective to interdisciplinary problems that enables us to think deeply about difficult problems and at the same time engage well under differing disciplinary perspectives.

This session contributed to the conference buzz of computer science looking outward, both for research and education. I found it quite interesting and, of course, think the problems it discussed are central to what people in CS should be thinking about these days. I only wish my mind had been more into the talk that morning.

Three ideas stayed with me as my conference closed:

  • One panelist made a great comment in the spirit of looking outward. Paraphrase: While we in CS argue about what "computational thinking" means, we should embrace the diversity of computational thinking done out in the world and reach out to work with partners in many disciplines.

  • Another panelist commented on the essential role that computing plays in other disciplines. He used biology as his example. Paraphrase: To be a biologist these days requires that you understand simulation, modeling, and how to work with large databases. Working with large databases is the defining characteristic of social science these days.

  • Many of the issues that challenge computer scientists who want to engage in interdisciplinary research of this sort are ones we have encountered for a long time. For instance, how can a computer scientist find the time to gain all of the domain knowledge she needs?

    Other challenges follow from how people on either side of the research view the computer scientist's role. Computer science faculties that make tenure and promotion decisions often do not see research value in interdisciplinary research. The folks on the applied side often contribute to this by viewing the computer science as a tool builder or support person, not as an integral contributor to solving the research problem. I have seen this problem firsthand while helping members of my department's faculty try to contribute to projects outside of our department.

This panel was a most excellent way to end my conference, with many thoughts about how to work with CS colleagues to develop projects that engage colleagues in other disciplines.

Pretty soon after the close of this session I was on the road home, whether repacked my bags and headed off for a few days of spring break with my family in St. Louis. This trip was a wonderful break, though ten days too early to see my UNI Panthers end their breakout season with an amazing run.


Posted by Eugene Wallingford | Permalink | Categories: Computing

March 24, 2010 7:42 PM

SIGCSE Day 2 -- Al Aho on Teaching Compiler Construction

[A transcript of the SIGCSE 2010 conference: Table of Contents]

Early last year, I wrote a blog entry about using idea's from Al Aho's article, Teaching the Compilers Course, in the most recent offering of my course. When I saw that Aho was speaking at SIGCSE, I knew I had to go. As Rich Pattis told me in the hallway after the talk, when you get a chance to hear certain people speak, you do. Aho is one of those guys. (For me, so is Pattis.)

The talk was originally scheduled for Thursday, but persistent fog over southeast Wisconsin kept several people from arriving at the conference on time, including Aho. So the talk was rescheduled for Fri. I still had to see it, of course, so I skipped the attention-grabbing "If you ___, you might be a computational thinker".

Aho's talk covered much of the same ground as his inroads paper, which gave me the luxury of being able to listen more closely to his stories and elaborations than to the details. The talk did a nice job of putting the compiler course into its historical context and tried to explain why we might well teach a course very different -- yet in many ways similar -- to the course we taught forty, twenty-five, or even ten years ago.

He opened with lists of the top ten programming languages in 1970 and 2010. There was no overlap, which introduced Aho's first big point: the landscape of programming languages has changes in a big way since the beginning of our discipline, and there have been corresponding changes in the landscape of compilers. The dimensions of change are many: the number of languages, the diversity of languages, the number and kinds of applications we write. The growth in number and diversity applies not only to the programming languages we use, which are the source language to a compiler, but also to the target machines and the target languages produced by compilers.

From Aho's perspective, one of the most consequential changes in compiler construction has been the rise of massive compiler collections such as gcc and LLVM. In most environments, writing a compiler is no longer a matter of "writing a program" as much a software engineering exercise: work with a large existing system, and add a new front end or back end.

So, what should we teach? Syntax and semantics are fairly well settled as matter of theory. We can thus devote time to the less mathematical parts of the job, such as the art of writing grammars. Aho noted that in the 2000s, parsing natural languages is mostly a statistical process, not a grammatical one, thanks to massive databases of text and easy search. I wonder if parsing programming languages will ever move in this direction... What would that mean in terms of freer grammar, greater productivity, or confusion?

With the availability of modern tools, Aho advocates an agile "grow a language" approach. Using lex and yacc, students can quickly produce a compiler in approximately 20 lines of code. Due to the nature of syntax-directed translation, which is closely related to structural recursion, we can add new productions to a grammar with relative ease. This enables us to start small, to experiment with different ideas.

The Dragon book circa 2010 adds many new topics to its previous editions. It just keeps getting thicker! It covers much more material, both breadth and depth, than can be covered in the typical course, even with graduate students. This gives instructors lots of leeway in selecting a subset around which to build a course. The second edition already covers too much material for my undergrad course, and without enough of the examples that many students need these day. We end up selecting such a small subset of the material that the price of the book is too high for the number of pages we actually used.

The meat of the talk matched the meat of his paper: the compiler course he teaches these days. Here are a few tidbits.

On the Design of the Course

  • Aho claims that, through all the years, every team has delivered a working system. He attributes this to experience teaching the course and the support they provide students.
  • Each semester, he brings in at least one language designer in as a guest speaker, someone like Stroustrup or Gosling. I's love to do this but don't have quite the pull, connections, or geographical advantage of Aho. I'll have to be creative, as I was the last time I taught agile software development and arranged a phone conference with Ken Auer.
  • Students in the course become experts in one language: the one they create. They become much more knowledgable in several others: the languages they to to write, build, and test their compiler.

On System Development

  • Aho sizes each student project at 3,000-6,000 LOC. He uses Boehm's model to derive a team size of 5, which fits nicely with his belief that 5 is the ideal team size.
  • Every team member must produce at least 500 lines of code on the project. I have never had an explicit rule about this in the past, but experience in my last two courses with team projects tells me that I should.
  • Aho lets teams choose their own technology, so that they can in the way that makes them most comfortable. One serendipitous side effect of this choice is that requires him to stay current with what's going on in the world.
  • He also allows teams to build interpreters for complex languages, rather than full-blown compilers. He feels that the details of assembly language get in the way of other important lessons. (I have not made that leap yet.)

On Language Design

  • One technique he uses to scope the project is to require students to identify an essential core of their language along with a list of extra features that they will implement if time permits. In 15 years, he says, no team has ever delivered an extra feature. That surprises me.
  • In order to get students past the utopian dream of a perfect language, he requires each team to write two or three programs in their language to solve representative problems in the language's domain. This makes me think of test-first design -- but of the language, not the program!
  • Aho believes that students come to appreciate our current languages more after designing a language and grappling with the friction between dreams and reality. I think this lesson generalizes to most forms of design and creation.

I am still thinking about how to allow students to design their own language and still have the time and energy to produce a working system in one semester. Perhaps I could become more involved early in the design process, something Aho and his suite of teaching assistants can do, or even lead the design conversation.

On Project Management

  • "A little bit of process goes a long way" toward successful delivery and robust software. The key is finding the proper balance between too much process, which stifles developers, and too little, which paralyzes them.
  • Aho has experimented with different mechanisms for organizing teams and selecting team leaders. Over time, he has found it best to let teams self-organize. This matches my experience as well, as long as I keep an eye out for obviously bad configurations.
  • Aho devotes one lecture to project management, which I need to do again myself. Covering more content is a siren that scuttles more student learning than it buoys.

~~~~

Aho peppered his talk with several reminiscences. He told a short story about lex and how it was extended with regular expressions from egrep by Eric Schmidt, Google CEO. Schmidt worked for Aho as a summer intern. "He was the best intern I ever had." Another interesting tale recounted one of his doctoral student's effort to build a compiler for a quantum computer. It was interesting, yes, but I need to learn more about quantum computing to really appreciate it!

My favorite story of the day was about awk, one of Unix's great little languages. Aho and his colleagues Weinberger and Kernighan wrote awk for their own simple data manipulation tasks. They figured they'd use it to write throwaway programs of 2-5 lines each. In that context, you can build a certain kind of language and be happy. But as Aho said, "A lot of the world is data processing." One day, a colleague came in to his office, quite perturbed at a bug he had found. This colleague had written a 10,000-line awk program to do computer-aided design. (If you have written any awk, you know just how fantabulous this feat is.) In a context where 10K-line programs are conceivable, you want a very different sort of language!

The awk team fixed the bug, but this time they "did it right". First, they built a regression test suite. (Agile Sighting 1: continuous testing.) Second, they created a new rule. To propose a new language feature for awk, you had to produce regression tests for it first. (Agile Sighting 2: test-first development.) Aho has built this lesson into his compiler course. Students must write their compiler test-first and instrument their build environments to ensure that the tests are run "all of the time". (Agile Sighting 3: continuous integration.)

An added feature of Aho's talk over his paper was three short presentations from members of a student team that produced PixelPower, a language which extends C to work with a particular graphics library. They shared some of the valuable insights from their project experience:

  • They designed their language to have big overlap with C. This way, they had an existing compiler that they understood well and could extend.
  • The team leader decided to focus the team, not try to make everyone happy. This is a huge lesson to learn as soon as you can, one the students in my last compiler course learned perhaps a bit too late. "Getting things done," Aho's students said, "is more important than getting along."
  • The team kept detailed notes of all their discussions and all their decisions. Documentation of process is in many ways much more important than documentation of code, which should be able to speak for itself. My latest team used a wiki for this purpose, which was a good idea they had early in the semester. If anything, they learned that they should have used it more frequently and more extensively.

One final note to close this long report. Aho had this to say about the success of his course:

If you make something a little better each semester, after a while it is pretty good. Through no fault of my own this course is very good now.

I think Aho's course is good precisely because he adopted this attitude about its design and implementation. This attitude serves us well when designing and implementing software, too: Many iterations. Lots of feedback. Collective ownership of the work product.

An hour well spent.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

March 22, 2010 4:35 PM

SIGCSE Day 2 -- Reimagining the First Course

[A transcript of the SIGCSE 2010 conference: Table of Contents]

The title of this panel looked awfully interesting, so I headed off to it despite not knowing just what it was about. It didn't occur to me until I saw the set of speakers at the table that it would be about an AP course! As I have written before, I don't deal much with AP, though most of my SIGCSE colleagues do. This session turned out to be a good way to spend ninety minutes, both as a window into some of the conference buzz and as a way to see what may be coming down the road in K-12 computer science in a few years.

What's up? A large and influential committee of folks from high schools, universities, and groups such as the ACM and NSF are designing a new course. It is intended as an alternative to the traditional CS1 course, not as a replacement. Rather than starting with programming or mathematics as the foundation, of the the course, the committee is first identifying a set of principles of computing and then designing a course to teach these principles. Panel leader Owen Astrachan said that the are engineering a course, given the national scale of the project and the complexity of creating something that works at lots of schools and for lots of students.

Later, I hope to discuss the seven big ideas and the seven essential practices of computational thinking that serve as the foundation for this course, but for now you should read them for yourself. At first blush, they seem like a reasonable effort to delineate what computing means in the world and thus what high school graduates these days should know about the technology that circumscribes their lives. They emphasize creativity, innovation, and connections across disciplines, all of which can be lost when we first teach students a programming language and whip out "Hello, World" and the Towers of Hanoi.

Universities have to be involved in the design and promotion of this new course because it is intended for advanced placement, and that mean that it must earn college credit. Why does the AP angle matter? Right now, because it is the only high school CS course that counts at most universities. It turns out that advanced placement into a major matters less to many parents and HS students than the fact that the course carries university credit. Placement is the #1 reason that HS students take AP courses, but university credit is not too far behind.

For this reason, any new high school CS course that does not offer college credit will be hard to sell to any K-12 school district. (This is especially true in a context where even the existing AP CS is taught in only 7% of our high schools.) That's not too high a hurdle. At the university level, it is much easier to have an AP course approved for university or even major elective credit than it is to have a course approved for advanced placement in the major. So the panel encouraged university profs in the audience to do what they can at their institutions to prepare the way.

Someone on the panel may have mentioned the possibility of having a principles-based CS AP course count as a general education course. At my school we were successful a couple of years ago at having a CS course on simulation and modeling added as one of the courses that satisfies the "quantitative reasoning" requirement in our Liberal Arts Core. I wonder how successful we could be at having a course like the new course under development count for LAC credit. Given the current climate around our core, I doubt we could get a high school AP course to count, because it would not be a part of the shared experience our students have at the university.

The most surprising part of this panel was the vibe in the room. Proposals such as this one that tinker with the introductory course in CS usually draw a fair amount of skepticism and outright opposition. This one did not. The crowd seemed quite accepting, even when the panel turned its message into one of advocacy. They encouraged audience members to become advocates for this course and for AP CS more generally at their schools. They asked us not to tear down these efforts, but to join in and help make the course better. Finally, they asked us to join the College Board, the CS Teachers Association, and the ACM in presenting a united front to our universities, high schools, and state governments about the importance and role of computing in the K-12 curriculum. The audience seemed as if it was already on board.

In closing, there were two memorable quotes from the panel. First, Jan Cuny, currently a program officer for CISE at the National Science Foundation, addressed concern that all the talk these days about the "STEM disciplines" often leaves computing out of the explicit discussion:

There is a C in STEM. Nothing will happen in the S, the T, the E, or the M without the C.

I've been telling everyone at my university this for the last several years, and most are open to the broadening of the term when they are confronted with this truth.

Second, the front-runner for syllogism of the year is this gem from Owen Astrachan. Someone in the audience asked, "If this new course is not CS1, then is it CS0?" (CS0 is a common moniker for university courses taken by non-majors before they dive into the CS major-focused CS1 course.) Thus spake Owen:


     This course comes before CS1.
     0 is the only number less than 1.
Therefore, this course is CS0.

This was only half of Owen's answer, but it was the half that made me laugh.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

March 20, 2010 9:04 PM

SIGCSE -- What's the Buzz?

[A transcript of the SIGCSE 2010 conference: Table of Contents]

Some years, I sense a definite undercurrent to the program and conversation at SIGCSE. In 2006, it was Python on the rise, especially as a possible language for CS1. Everywhere I turned, Python was part of the conversation or a possible answer to the questions people were asking. My friend Jim Leisy was already touting a forthcoming CS1 textbook that has since risen to the top of its niche. It pays for me to talk to people who are looking into the future!

Other years, the conference is melange of many ideas with little or no discernible unifier. SIGCSE 2008 seemed to want to be about computing with big data, but even a couple of high-profile keynote addresses couldn't put that topic onto everyone's lips. 2007 brought lots of program about "computational thinking", but there was no buzz. Big names and big talks don't create buzz; buzz comes from the people.

This year's wanna-be was concurrency. There were papers and special sessions about its role in the undergrad curriculum, and big guns Intel, Google, and Microsoft were talking it up. But conversation in the hallways and over drinks never seemed to gravitate toward concurrency. The one place where I saw a lot of people gathering to discuss it was a BoF on Thursday night, but even then the sense was more anticipation than activity. What are others doing? Maybe next year.

The real buzz this year was CS, and CS ed, looking outward. Consistent with recent workshops like SECANT, SIGCSE 2010 was full of talk about computer science interacting with other disciplines, especially science but also the arts. Some of this talk was about how CS can affect science education, and some was about how other disciplines can affect CS education.

But there was also a panel on integrating computing research and development with R&D in other sciences. While this may look like a step outside of SIGCSE's direct sphere of interest, it is essential that we keep the sights of our CS ed efforts out in the world where our graduates work. Increasingly, this is again in applications where computing is integral to how others do their jobs.

An even bigger outward-looking buzz coalesced around CS educators working with K-12 schools, teachers, and students. A big part of this involved teaching computer science in middle schools and high schools, including AP CS. But it also involved the broader task of teaching all students about "computational thinking": what it means, how to do it, and maybe even how to write programs that automate it. Such a focus on the general education of students is real evidence of looking outward. This isn't about creating more CS majors in college, though few at SIGCSE would object to that. It's about creating college students better prepared for every major and career they might pursue.

To me, this is a sign of how we in CS ed are maturing, going from a concern primarily for our own discipline to one for computing's larger role in the world. That is an ongoing theme of these blog, it seems, so perhaps I am suffering from confirmation bias. But I found it pretty exciting to see so many people working so hard to bring computing into classrooms and into research labs as a fundamental tool rather than as a separate discipline.


Posted by Eugene Wallingford | Permalink | Categories: Computing

March 11, 2010 8:33 PM

SIGCSE Day One -- The Most Influential CS Ed Papers

[A transcript of the SIGCSE 2010 conference: Table of Contents]

This panel aimed to start the discussion of how we might identify which CS education papers have had the greatest influence on our practice of CS education. Each panelist produced a short list of candidates and also suggested criteria and principles that the community might use over time. Mike Clancy made explicit the idea that we should consider both papers that affect how we teach and papers that affect what we teach.

This is an interesting= process that most areas of CS eventually approach. A few years ago, OOPSLA began selecting a paper from the OOPSLA conference ten years prior that had had the most influence on OO theory or practice. That turns out to be a nice starting criterion for selection: wait ten years so that we have some perspective on a body of work and an opportunity to gather data on the effects of the work. Most people seem to think that ten years is long enough to wait.

You can see the list of papers, books, and websites offered by the panelists on this page. The most impassioned proposal was Eric Roberts's tale of how much Rich Pattis's Karel the Robot affects Stanford's intro programming classes to this day, over thirty years after Rich first created Karel.

I was glad to see several papers by Eliot Soloway and his students on the list. Early in my career, Soloway had a big effect on how I thought about novice programmers, design, and programming patterns. My patterns work was also influenced strongly by Linn and Clancy's The Case for Case Studies of Programming Problems, though I do not think I have capitalized on that work as much as I could have.

Mark Guzdial based his presentation on just this idea: our discipline in general does not fully capitalize on great work that has come before. So he decided to nominate the most important papers, not the most influential. What papers should we be using to improve our theory and practice?

I know Anderson's cognitive tutors work well, from the mid-1990s when I was preparing to move my AI research toward intelligent tutoring systems. The depth and breadth of that work is amazing.

Some of my favorite papers showed up as runners-up on various lists, including Gerald Weinberg's classic The Psychology of Programming. But I was especially thrilled when, in the post-panel discussion, Max Hailperin suggested Robert Floyd's Turing Award lecture, The Paradigms of Programming. I think this is one of the all-time great papers in CS history, with so many important ideas presented with such clarity. And, yes, I'm a fan.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

March 11, 2010 7:47 PM

SIGCSE Day One -- What Should Everyone Know about Computation?

[A transcript of the SIGCSE 2010 conference: Table of Contents]

This afternoon session was a nice follow up to the morning session, though it had a focus beyond the interaction of computation and the sciences: What should every liberally-educated college graduate know about computation? This almost surely is not the course we start CS majors with, or even the course we might teach scientists who will apply computing in a technical fashion. We hope that every student graduates with an understanding of certain ideas from history, literature, math, and science. What about computation?

Michael Goldweber made an even broader analogy in his introduction. In the 1800s, knowledge about farming was pervasive throughout society, even among non-farmers. This was important for people, even city folk, to understand the world in which they lived. Just as agriculture once dominated our culture, so does technology now. To understand the world in which they live, people these days need to understand computation.

Ultimately, I found this session disappointing. We heard a devil's advocate argument against teaching any sort of "computer literacy"; a proposal that we teach all students what amounts to an applied, hand-waving algorithms course; and a course that teaches abstraction in contexts that connects with students. There was nothing wrong with these talks -- they were all entertaining enough -- but they didn't shed much new light on what is a difficult question to answer.

Henry Walker did say a few things that resonated with me. One, he reminded us that there is a difference between learning about science and doing science. We need to be careful to design courses that do one of these well. Two, he tried to explain why computer science is the right discipline for teaching problem solving as a liberal art, such as how a computer program can illustrate the consequences specific choices, the interaction of effects, and especially the precision with which we must use language to describe processes in a computer program. Walker was the most explicit of the panelists in treating programming as fundamental to what we offer the world.

In a way unlike many other disciplines, writing programs can affect how we think in other areas. A member of the audience pointed out CS also fundamentally changes other disciplines by creating new methodologies that are unlike anything that had been practical before. His example was the way in which Google processes and translates language. Big data and parallel processing have turned the world of linguistics away from Chomskian approach and toward statistical models of understanding and generating language.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

March 11, 2010 7:04 PM

SIGCSE Day One -- Computation and The Sciences

[A transcript of the SIGCSE 2010 conference: Table of Contents]

I chose this session over a paper session on Compilers and Languages. I can always check those papers out in the proceedings to see if they offer any ideas I can borrow. This session connects back to my interests in the role of computing across all disciplines, and especially to my previous attendance at the SECANT workshops on CS and science in 2007 and 2008. I'm eager to hear about how other schools are integrating CS with other disciplines, doing data-intensive computing with their students, and helping teachers integrate CS into their work. Two of these talks fit this bill.

In then first, Ali Erkan talked about the need to prepare today's to work on large projects that span several disciplines. This is more difficult than simply teaching programming languages, data structures, and algorithms. It requires students to have disciplinary expertise beyond CS, the ability to do "systems thinking", and the ability to translate problems and solutions across the cultural boundaries of the disciplines. A first step is to have students work on problems that are bigger than the students of any single discipline can solve. ( Astrachan's Law marches on!)

Erkin then described an experiment at Ithaca College in which four courses run as parallel threads against a common data set of satellite imagery: ecology, CS, thermodynamics, and calculus. Students from any course can consult students in the other courses for explanations from those disciplines. Computer science students in a data structures course use the data not only to solve the problem but also to illustrate ideas, such as memory usage of depth-first and breadth-first searches of a grid of pixels. They can also apply more advanced ideas, such as data analysis techniques to smooth curves and generate 3D graphs.

I took away two neat points from this talk. The first was a link to The Cryosphere Today, a wonderful source of data on arctic and antarctic ice coverage for students to work with. The second was a reminder that writing programs to solve a problem or illustrate a data set helps students to understand the findings of other sciences. Raw data become real for them in writing and running their code.

In the second paper, Craig Struble described a three-day workshop for introducing computer science to high school science teachers. Struble and his colleagues at Marquette offered the workshop primarily for high school science teachers in southeast Wisconsin, building on the ideas described in A Novel Approach to K-12 CS Education: Linking Mathematics and Computer Science. The workshop had four kinds of sessions:

  • tools: science, simulation, probability, Python, and VPython
  • content: mathematics, physics, chemistry, and biology
  • outreach: computing careers, lesson planning
  • fun: CS unplugged activities, meals and other personal interaction with the HS teachers

This presentation echoed some of what we have been doing here. Wisconsin K-12 education presents the same challenge that we face in Iowa: there are very few CS courses in the middle- or high schools. The folks at Marquette decided to attack the challenge in the same way we have: introduce CS and the nebulous "computational thinking" through K-12 science classes. We are planning to offer a workshop for middle- and high school teachers. We are willing to reach an audience wider than science teachers and so will be showing teachers how to use Scratch to create simulations, to illustrate math concepts, and even to tell stories.

I am also wary of one of the things the Marquette group learned in follow-up with the teachers who attended their workshop. Most teachers likes it and learned a lot, but many are not able to incorporate what they learn into their classes. Some face time constraints from a prescribed curriculum, while others are limited by miscellaneous initiatives external to their curriculum that are foisted on them by their school. That is a serious concern for us as we try to help teachers do cool things with CS that change how they teach their usual material.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

March 10, 2010 8:35 PM

SIGCSE DAY 0 -- Media Computation Workshop

[A transcript of the SIGCSE 2010 conference: Table of Contents]

I headed to SIGCSE a day early this year in order to participate in a couple of workshops. The first draw was Mark Guzdial's and Barbara Ericson's workshop using media computation to teach introductory computing to both CS majors and non-majors. I have long been a fan of this work but have never seen them describe it. This seemed like a great chance to learn a little from first principles and also to hear about recent developments in the media comp community.

Because I taught a CS1 course in Java, using media comp four years ago, I was targeted by other media comp old-timers as a media comp old-timer. They decided, with Mark's blessing, to run a parallel morning session with the goal of pooling experience and producing a resource of value to the community.

When the moment came for the old-timers to break out on their own, I packed up my laptop, stood to leave -- and stopped. I felt like spending the morning as a beginner. This was not an entirely irrational decision. First, while I have done Java media comp, I have never worked with the original Python materials or the JES programming environment students use to do media comp in Python. Second, I wanted to see Mark present the material -- material he has spent years developing and for which he has great passion. I love to watch master teachers in action. Third, I wanted to play with code!

Throughout the morning, I diddled in JES with Python code to manipulate images, doing things I've done many times in Java. It was great fun. Along the way, I picked up a few nice quotes, ideas, and background facts:

  • Mark: "For liberal arts students and business students, the computer is not a tool of calculation but a tool of communication.

  • The media comp data structures book is built largely on explaining the technology needed to create the wildebeest stampede in The Lion King. (Check out this analysis, which contains a description of the scene in the section "Building the Perfect Wildebeests".)

  • We saw code that creates a grayscale version of an image attuned to human perception. The value used for each color in a pixel weights its original values as 0.299*red + 0.587*blue + 0.114*green. This formula reinforces the idea that there are an infinite number of weightings we can use to create grayscale. There are, of course, only a finite number of grayscale versions of an image, though that number is very large: 256 raised to a power equal to the number of pixels in the image.

  • After creating several Python methods that modify an image, non-majors eventually bump into the need to return a value, often a new image. Deciding when a function should return a value can be tough, especially for non-CS folks. Mark uses this rule of thumb to get them started: "If you make an image in the method, return it."

  • Mark and Barb use Making of "The Matrix" to take the idea of chromakey beyond the example everyone seems to know, TV weather forecasters.

  • Using mirroring to "fix" a flawed picture leads to a really interesting liberal arts discussion: How do you know when a picture is fake? This is a concept that every person needs to understand these days, and understanding the computations that can modify an image enables us to understand the issues at a much deeper level.

  • Mark showed an idea proposed to him by students at one of his workshops for minority high school boys: when negating an image, change the usual 255 upper bound to something else, say, 180. This forces many of the resulting values to 0 and behaves like a strange posterize function!

I also learned about Susan Menzel's work at Indiana University to port media computation to Scheme. This is the second such project I've heard of, after Sam Rebelsky's work at Grinnell College connecting Scheme to Gimp.

Late in the morning, we moved on to sound. Mark demonstrated some wonderful tools for playing with and looking at sound. He whistled, sang, hummed, and played various instruments into his laptop's microphone, and using their MediaTools (written in Squeak) we could see the different mixes of tones available in the different sounds. These simple viewers enable us to see that different instruments produce their own patterns of sounds. As a relative illiterate in music, I only today understood how it is that different musical instruments can produce sounds of such varied character.

The best quote of the audio portion of the morning was, "Your ears are all about logarithms." Note systems with halving and doubling of frequencies across sets of notes is not an artifact of culture but an artifact of how the human ear works!

This was an all-day workshop, but I also had a role as a sage elder at the New Educators Roundtable in the afternoon, so I had to duck out for a few hours beginning with lunch. I missed out on several cool presentations, including advanced image processing ideas such as steganography and embossing, but did get back in time to hear how people are now using media computation to teach data structures ideas such as linked lists and graphs. Even with a gap in the day, this workshop was a lot of fun, and valuable as we consider expanding my department's efforts to teach computing to humanities students.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

March 10, 2010 7:40 PM

Notes on SIGCSE 2010: Table of Contents

The set of entries cataloged here records some of my thoughts and experiences at SIGCSE 2010, in Milwaukee, Wisconsin, March 10-13. I'll update it as I post new essays about the conference.

Primary entries:

Ancillary entries:


Posted by Eugene Wallingford | Permalink | Categories: Computing, Running, Software Development, Teaching and Learning

March 07, 2010 5:45 PM

Programming as Inevitable Consequence

My previous entry talked about mastering tools and improving process on the road to achievement. Garry Kasparov wrote of "programming yourself" as the way to make our processes better. To excel, we must program ourselves!

One way to do that is via computation. Humans use computers all the time now to augment their behavior. Chessplayers are a perfect example. Computers help us do what we do better, and sometimes they reconfigure us, changing who we are and what we do. Reconfigured well, a person or group of people can push their capabilities beyond even what human experts can do -- alone, together, or with computational help.

But what about our tools? How many chessplayers, or any other people for that matter, program their computers these days as a means of making the tools they need, or the tools they use better? This is a common lament among certain computer scientists. Ian Bogost reminds us that writing programs used to be an inevitable consequence of using computers. Computer manufacturers used to make writing programs a natural step in our mastery of the machine they sold us. They even promoted the personal computer as part of how we became more literate. Many of us old-timers tell stories of learning to program so that we could scratch some itch.

It's not obvious that we all need to be able to program, as long as the tools we need to use are created for us by others. Mark Guzdial discusses his encounters with the "user only" point of view in a recent entry motivated by Bogost's article. As Mark points out, though, the computer may be different than a bicycle and our other tools. Most tools extend our bodies, but the computer extends our minds. We can program our bodies by repetition and careful practice, but the body is not as malleable as the mind. With the right sort of interaction with the world, we seem able to amplify our minds in ways much different than what a bicycle can do for our legs.

Daniel Lemire expresses it nicely and concisely: If you understand an idea, you can implement it in software. To understand an idea is to be able to write a program. The act of writing itself gives rise to a new level of understanding, to a new way of describing and explaining the idea. But there is more than being able to write code. Having ideas and being able to program is, for so many people, a sufficient condition to want to program: Sometimes to scratch an itch; sometimes to understand better; and sometimes simply to enjoy the act.

This feeling is universal. As I wrote not long ago, computing has tools and ideas that make people feel superhuman. But there is more! As Thomas Guest reminds us, "ultimately, the power of the programmer is what matters". The tools help to make us powerful, true, but they also unleash power that is already within is.

By the way, I strongly recommend Guest's blog, Word Aligned. Guest doesn't write as frequently as some bloggers, but when he does, it is technically solid, deep, and interesting.


Posted by Eugene Wallingford | Permalink | Categories: Computing

March 05, 2010 9:21 PM

Mastering Tools and Improving Process

Today, a student told me that he doesn't copy and paste code. If he wants to reuse code verbatim, he requires himself to type it from scratch, character by character. This way, he forces himself to confront the real cost of duplication right away. This may motivate him to refactor as soon as he can, or to reconsider copying the code at all and write something new. In any case, he has paid a price for copying and so has to take it seriously.

The human mind is wonderfully creative! I'm not sure I could ma