June 26, 2013 2:30 PM

An Opportunity to Learn, Born of Deprivation

Earlier this summer, my daughter was talking about something one of her friends had done with Instagram. As a smug computer weenie, I casually mentioned that she could do that, too.

She replied, "Don't taunt me, Dad."

You see, no one in our family has a cell phone, smart or otherwise, so none of us use Instagram. That's not a big deal for dear old dad, even though (or perhaps because) he's a computer scientist. But she is a teenager growing up in an entirely different world, filled with technology and social interaction, and not having a smart phone must surely seem like a form of child abuse. Occasionally, she reminds us so.

This gave me a chance to explain that Instagram filters are, at their core, relatively simple little programs, and that she could learn to write them. And if she did, she could run them on almost any computer, and make them do things that even Instagram doesn't do.

I had her attention.

So, this summer I am going to help her learn a little Python, using some of the ideas from media computation. At the end of our first pass, I hope that she will be able to manipulate images in a few basic ways: changing colors, replacing colors, copying pixels, and so on. Along the way, we can convert color images to grayscale or sepia tones, posterize images, embed images, and make simple collages.

That will make her happy. Even if she never feels the urge to write code again, she will know that it's possible. And that can be empowering.

I have let my daughter know that we probably will not write code that does as good a job as what she can see in Instagram or Photoshop. Those programs are written by pros, and they have evolved over time. I hope, though, that she will appreciate how simple the core ideas are. As James Hague said in a recent post, then key idea in most apps require relatively few lines of code, with lots and lots of lines wrapped around them to handle edge cases and plumbing. We probably won't write much code for plumbing... unless she wants to.

Desire and boredom often lead to creation. They also lead to the best kind of learning.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Software Development, Teaching and Learning

June 25, 2013 2:46 PM

Data Ingestion

This sentence in Reid Draper's Data Traceability made me laugh recently:

I previously worked in the data ingestion team at a music data company.

Nice turn of phrase. I suppose that another group digests the data, and yet another expels it.

Draper's sentence came to mind again yesterday while I was banging my head on a relatively simple problem, transforming a CSV file generated by my university's information system, replete with embedded quotes and commas, into something more manageable. As data ingestion goes, this isn't much of a problem at all. There are plenty of libraries that do the heavy lifting for you, in most any language you choose, Ruby included.

Of course, I was just writing a quick-and-dirty script, so I was rolling my own CSV-handling code. As usual, "quick and dirty" is often dirty, but rarely quick. I tweeted a bit of my frustration, in response to which @geoffwozniak wrote:

Welcome to the world of enterprise data ingress.

If I had to deal with these files everyday, I might head for the egress. ... or master a good library, so that I could bang my head on more challenging data ingestion problems.

Posted by Eugene Wallingford | Permalink | Categories: Software Development

June 20, 2013 12:32 PM

Agile Moments: Stories, Tests, and Refactoring in Visual Design

In A Place for Sharing Ideas and Stories, designers Teehan+Lax tell the story of their role in creating Medium, "a better place to read and write things that matter". The section "Forging ahead", about features added to the platform after its launch, made me think of some of the ideas we use when designing code test-first, only at a much higher level.

We reset by breaking the team up into new feature teams. Each feature team would have at least one designer, one front-end developer and a back-end developer. Some teams would take on multiple features depending on their complexity. We used one page briefs that were easy to write, easy to understand and helped guide the teams when working through their feature(s).

They consisted of questions like:

  • Who is this page for?
  • What problem does this page solve for the user?
  • How do we know they need it?
  • What is the primary action we want users to take on this page?
  • What might prompt a user to take this action?
  • How will we know that this page is doing what we want it to do?

This bullet list embodies several elements of agile development. For each feature, the brief acts like a story card that boils the feature down to a clear need of the user, a clear action, and, most important in my mind, a test: How will we know that this page is doing what we want it to do? In a lot of my work, this is a crucial element. As Kent Beck says, "How will I know I'm done?"

The paragraph preceding the list highlights a couple of other attributes common to agile development. One, teams are working on stories in parallel on a common artifact. Two, the teams include a designer, a front-end developer, and a back-end developer. The team doesn't include a user, which can be a huge advantage for developers, but the author mentions elsewhere that nearly everyone on the team was a user:

As the internal product progressed and its features and capabilities became clearer, we would reduce the amount of ad hoc meetings and focus on getting stuff built into the product so we could actually use it. Talking about work is great at first, but usage is what breathes life into the product.

The author also stresses the value of physical co-location of the designers and developers over even well-supported electronic communication, which echoes for me the value of having a user in the room with the developers.

What happens as features were enhanced or added by separate teams? "... maintaining some sort of design integrity across the entire product."

This thing was about to get full of user choices (read: complexity) in a hurry -- The product was now at a critical point in its life.

Of course, complexity and incoherence can creep into products even when they are designed and built by one team, when it works on multiple features in rapid succession.

The solution for Medium sounded familiar:

We took a week off current fixes and features and focused on redesigning three pages from scratch: Home, Collection and Post. Some of it went live, some of it went away.


We hated to see some of the stuff we'd designed (and even built) not go live, but it needed to die so the product could grow through simplification.

This reminded me a lot like refactoring, even with differences from the way we refactor at the code level. As teams added features without considering the effect of the changes on the global structure of the product, they accumulated something akin to "design debt". So after a while they dedicated time to paying off the debt and bringing back to the product the sense of wholeness that had been lost. I am curious to know whether the teams ever looked back at the page briefs to verify that the redesigned pages still did what they wanted them to do. That would be the equivalent of "running the tests".

We programmers really do have it nice. Lines and units of code are a bit more separable than the visual design elements of a product. This allows us to refactor all the time, if we are so inclined, not just in batch after a large set of changes. The presence of concrete tests, written as code, allows us to test the efficacy of our changes relatively easily before we move forward. While down in the trenches writing code, it's easy to forget just how liberating -- and empowering -- this combination of separability and testability are for design and redesign.

It's also easy to forget sometimes that similar challenges face designers and creators across domains and disciplines. Many of the same themes run through stories the things we create, whether software in the small, software in the large, or physical artifacts. Reading Teahan+Lax's story reminded me of that, too.

Posted by Eugene Wallingford | Permalink | Categories: Software Development

June 14, 2013 2:48 PM

The History of Achievement in AI

... often looks something like this:

  1. X := some task that people do well, but not computers.
  2. "It would really be impressive if a computer could X."
  3. Computer does X.
  4. "That's not intelligent. The computer is only doing search (or number crunching, or ...)."
  5. X := something else.
  6. Go to 2.

A common variation of this pattern is to replace Step 3 with a different dodge:

"That's no big deal. X doesn't really require intelligence."

In either case, the target moves.

Occasionally, the critic must admit, if grudgingly, that the task requires intelligence, whatever that means, and that the computer performs it well. But there is still one last move available to deflect the achievement from the computer:

"This is a human achievement. People had to program the computer."

I suspect that until a computer learns everything it knows from scratch -- whatever that means -- this pattern will repeat. We humans have an image to protect.


Postscript. I wrote this after reading a short interview interview with playwright Matt Charman, who has dramatized Deep Blue's epic 1997 match win over world chess champion Garry Kasparov. Note that Charman does not employ the dodges I list. He simply chose to focus on the human personalities involved in the drama. And those personalities are worthy of exploration, especially the fascinating Kasparov!

Posted by Eugene Wallingford | Permalink | Categories: Computing

June 13, 2013 3:01 PM

It's Okay to Talk About Currying!

James Hague offers some sound advice for writing functional programming tutorials. I agree with most of it, having learned the hard way by trying to teach functional style to university students for many years. But I disagree with one of his suggestions: I think it's okay to talk about currying.

Hague's concern with currying is practical:

Don't get so swept up in that theory that you forget the obvious: in any programming language ever invented, there's already a way to easily define functions of multiple arguments. That you can build this up from more primitive features is not useful or impressive to non-theoreticians.

Of course, my context is a little different. We teach functional programming in a course on programming languages, so a little theory is important. We want students not only to be able to write code in a functional style but also to understand some of the ideas at the foundation of the languages they use. We also want them to understand a bit about how different programming styles relate to one another.

But even in the context of teaching people to think functionally and to write code in that style, I think it's okay to talk about currying. Indeed, it is essential. Currying is not simply a theoretical topic. It is a valuable programming technique.

Here is an example. When we write a language interpreter, we often write a procedure names eval-exp. It takes two arguments: an expression to evaluate, and a list of variable/value bindings.

   (define eval-exp
     (lambda (exp env)

The binding list, sometimes called an environment, is a map of names declared in the local block to their values, along with the bindings from the blocks that contain the local block. Each time the interpreter enters a new block, it pushes a new set of name/value pairs onto the binding list and recurses.

To evaluate a function call for which arguments are passed by value, the interpreter must first evaluate all of the function's arguments. As the arguments are all in the same block, they are evaluated using the same binding list. We could write a new procedure to evaluate the arguments recursively, but this seems like a great time to map a procedure over a list: (map eval-exp args), get a list of the results, and pass them to the code that applies the function to them.

We can't do that, though, because eval-exp is a two-argument procedure, and map works only with a one-argument procedure. But the same binding list is used to evaluate each of the expressions, so that argument to eval-exp is effectively a constant for the purposes of the mapping operation.

So we curry eval-exp:

   (define eval-exp-with
     (lambda (bindings)
       (lambda (exp)
         (eval-exp exp bindings))))

... to create the one-argument evaluator that we need, and we use it to evaluate the arguments with map:

   ; in eval-exp
   (map (eval-exp-with env) arguments)

In most functional languages, we can use a nameless lambda to curry eval-exp "in place" and avoid writing an explicit helper function:

   ; an alternative approach in eval-exp
   (map (lambda (exp)
          (eval-exp exp bindings))

This doesn't look much like currying because we never created the procedure that takes the bindings argument. But we can reach this same piece of code by writing eval-exp-with, calling it in eval-exp, and then using program derivation to substitute the value of the call for the call itself. This is actually a nice connection to be able to make in a course about programming languages!

When I deliver short tutorials on functional style, currying often does not make the cut, because there are so many cool and useful ideas to cover. But it doesn't take long writing functional code before currying becomes useful. As this example shows, currying is a practical tool for the functional programmer to have in his or her pocket. In FP, currying isn't just theory. It's part of the style.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns

June 10, 2013 2:41 PM

Unique in Exactly the Same Way

Ah, the idyllic setting of my youth:

When people refer to "higher education" in this country, they are talking about two systems. One is élite. It's made up of selective schools that people can apply to -- schools like Harvard, and also like U.C. Santa Cruz, Northeastern, Penn State, and Kenyon. All these institutions turn most applicants away, and all pursue a common, if vague, notion of what universities are meant to strive for. When colleges appear in movies, they are verdant, tree-draped quadrangles set amid Georgian or Gothic (or Georgian-Gothic) buildings. When brochures from these schools arrive in the mail, they often look the same. Chances are, you'll find a Byronic young man reading "Cartesian Meditations" on a bench beneath an elm tree, or perhaps his romantic cousin, the New England boy of fall, a tousle-haired chap with a knapsack slung back on one shoulder. He is walking with a lovely, earnest young woman who apparently likes scarves, and probably Shelley. They are smiling. Everyone is smiling. The professors, who are wearing friendly, Rick Moranis-style glasses, smile, though they're hard at work at a large table with an eager student, sharing a splayed book and gesturing as if weighing two big, wholesome orbs of fruit. Universities are special places, we believe: gardens where chosen people escape their normal lives to cultivate the Life of the Mind.

I went to a less selective school than the ones mentioned here, but the vague ideal of higher education was the same. I recognized myself, vaguely, in the passage about the tousle-haired chap with a knapsack, though on a Midwestern campus. I certainly pined after a few lovely, earnest young women with a fondness for scarves and the Romantic poets in my day. These days, I have become the friendly, glasses-wearing, always-smiling prof in the recruiting photo.

The descriptions of movie scenes and brochures, scarves and Shelley and approachable professors, reminded me most of something my daughter told me as she waded through recruiting literature from so many schools a few years ago, "Every school is unique, dad, in exactly the same way." When the high school juniors see through the marketing facade of your pitch, you are in trouble.

That unique-in-the-same-way character of colleges and university pitches is a symptom of what lies at the heart of the coming "disruption" of what we all think of as higher education. The traditional ways for a school to distinguish itself from its peers, and even from schools it thinks of as lesser rivals, are becoming less effective. I originally wrote "disappearing", but they are now ubiquitous, as every school paints the same picture, stresses the same positive attributes, and tries not to talk too much about the negatives they and their peers face. Too many schools chasing too few tuition-paying customers accelerates the process.

Trying to protect the ideal of higher education is a noble effort now being conducted in the face of a rapidly changing landscape. However, the next sentence of the recent New Yorker article Laptop U, from which the passage quoted above comes, reminds us:

But that is not the kind of higher education most Americans know. ...

It is the other sort of higher education that will likely be the more important battleground on which the higher ed is disrupted by technology.

We are certainly beginning to have such conversations at my school, and we are starting to hear rumblings from outside. My college's dean and our new university president recently visited the Fortune 100 titan that dominates local industry. One of the executives there gave them several documents they've been reading there, including "Laptop U" and the IPPR report mentioned in it, "An Avalanche is Coming: Higher Education and the Revolution Ahead".

It's comforting to know your industry partners value you enough to want to help you survive a coming revolution. It's also hard to ignore the revolution when your partners begin to take for granted that it will happen.

Posted by Eugene Wallingford | Permalink | Categories: General, Teaching and Learning

June 07, 2013 1:53 PM

Sentences to Ponder

Henry Rollins:

When one beast dumps you, summon the guts to find another. If it tries to kill you, the party has definitely started. Otherwise, life is a slow retirement.

Rollins is talking about why he's not making music anymore, but his observation applies to other professions. We all know programmers who are riding out the long tail of an intellectual challenge that died long ago. College professors, too.

I have to imagine that this is a sad life. It certainly leaves a lot of promise unfulfilled.

If you think you have a handle on the beast, then the beast has probably moved on. Find a new beast with which to do battle.

Posted by Eugene Wallingford | Permalink | Categories: General, Managing and Leading

June 05, 2013 1:52 PM

I Fooled Around and Fell in Love

Cue the Elvin Bishop [ video ]...

I smile whenever I see this kind of statement on a website's About page:

Erika Carlson was studying clinical psychology in 2011, when she wrote her first line of Python code. She fell in love with programming, decided to change paths, and is now a software developer at Pillar Technology.

I fell in love upon writing my first line of code, too.

Not everyone will have the same reaction Erika and I had, but it's good that we give people at least an opportunity to learn how to program. Knowing that someone might react this way focuses my mind on giving novice programmers a good enough experience that they can, if they are so inclined.

My teaching should never get in the way of true love.

Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

June 04, 2013 2:43 PM

A Simple Confession

My Unix-toting brethren may revoke my CS card for saying this, but I really do like to install programs this way:

    Installing ApplicationX

1. Open the disk image 2. Drag ApplicationX to your Applications folder 3. Eject the disk image

The app loses points if I really have to drag it to the Applications folder. The Desktop should do.

I understand the value in ./configure and ./make and setting paths and... but it sure is nice when I don't have to use them.

Posted by Eugene Wallingford | Permalink | Categories: General

June 03, 2013 2:52 PM

Reflection on Zimmer's Open Letter and Student Questions

Carl Zimmer recently wrote an open letter to science students and teachers to address a disturbing trend: students cold-mailing him to ask for information. He stresses that he and his fellow science writers like to interact with students, but only after students have done some work on their own and have started to develop their own questions. The offending messages can be boiled down to the anti-pattern:

I have homework. I need information from you.

The actual messages can be a lot more entertaining. Be sure to check out at least the opening of his piece, which reprises a multi-day exchange with a random high school student.

I'm not an accomplished popular science writer like Zimmer, but as a CS professor at a public university I receive occasional messages of the sort Zimmer describes from high school students across our region. I try to help out as best I can, and the volume is not so large that I get burnt out trying to guide the student to a more fruitful exchange than "Tell me something" followed by "Here is some information you could have read for yourself".

Fortunately, most of the messages of this sort that reach my inbox come from students in my own courses. Well, it's unfortunate that I receive these messages at all, because they are often a symptom of laziness and presumptuousness. Most often, though, they are simply a sign of bad habits learned in their previous years of schooling. The fortunate part is that I have a chance to help students learn new, better habits of intellectual discipline and discourse.

My approach is a lot like the one Zimmer relates in his exchange with young Davis, so much so that a former student of mine forwarded me a link to Zimmer's piece and said "The exchange at the beginning reminded me of you."

But as a classroom teacher, I have an opportunity that internet celebrities don't: I get to see the question-askers in the classroom two or three times a week and in office hours whenever students avail themselves of the resource. I can also ask students to come see me outside of class for longer conversations.

Talking face-to-face can help students know for certain that I really do care about their curiosity and learning, even as I choose consciously not to fulfill their request until they have something specific to ask. They have to invest something in the conversation, too, and demonstrate their investment by having more to say than simply, "I don't get it."

(Talking face-to-face also helps me have a better idea of when a student has done his or her work and yet still is struggling to the point of not knowing what to say other than "I don't get it." Sometimes, I need to go farther than halfway to help a student in real need of help.)

Most students are open to the idea that college is different from their past experience and that it's time for them to start taking more control of their learning. They learn new habits and are able to participate in meaningful exchanges about material with which they have already engaged.

A few continue to struggle, never moving far beyond "Give me information". I don't know whether their previous schooling has driven them into such a deep rut that they can't get out, or whether even with different schooling they would have been poorly suited for university study. Fortunately, these students are quite few among our student body. Most really do just need a gentle push in the right direction.

Posted by Eugene Wallingford | Permalink | Categories: Teaching and Learning