May 23, 2017 2:50 PM

Becoming a Manager Changed My Perspective

My perspective on the way the university works changed after I became department head. This paragraph describes rather well one of the changes:

The other major challenge is coming to terms with all of the constraints within which decisions are made. From the outside, it's often easy to criticize actual decisions when contrasted when imagined ideal outcomes. But when you know why those ideals have to be imaginary, and you have to maneuver within much narrower confines than you might have imagined, you start to understand the "why" behind some patterns. That can be frustrating, but if you treat it as a series of puzzles, it can be fun. Any idiot can get good results with infinite resources and discretion, but how can you improve results with flat funding, contradictory policies, and prickly personalities? That's where the challenge comes in.

Yes, the university could do everything in its power to meet the needs and desires of Computer Science. It turns out, though, that other departments have needs and desires, too. Sometimes, those needs are more critical at this moment than our are.

You would think that CS faculty, who identify and measure trade-offs as a part of their academic discipline, would grok the notion of trade-offs inside the institution more easily. It took me a while to really get. Now, a big part of my job is helping other faculty to see it, too, all the while advocating effectively for my department.

The paragraph quoted above is by Matt Reed, from How to.... This sentence from the same piece expresses one of the biggest challenges to my peace of mind and daily sense of accomplishment after becoming department head:

The victories in administrative roles tend to be vicarious, rather than personal or direct.

The way I help faculty and students most as department head is by helping to reduce or eliminate friction that slows them down and wastes their energy. My biggest wins are usually when our faculty and students accomplish something big.

The other most salient challenge to my professional peace of mind: lack of closure. Almost nothing is ever finished, no battle ever won; it all starts again tomorrow.


Posted by Eugene Wallingford | Permalink | Categories: Managing and Leading

May 21, 2017 10:07 AM

Computer Programs Have Much to Learn, and Much to Teach Us

In his recent interview with Tyler Cowen, Garry Kasparov talks about AI, chess, politics, and the future of creativity. In one of the more intriguing passages, he explains that building databases for chess endgames has demonstrated how little we understand about the game and offers insight into how we know that chess-playing computer programs -- now so far beyond humans that even the world champion can only score occasionally against commodity programs -- still have a long way to improve.

He gives as an example a particular position with a king, two rooks, a knight on one side versus a king and two rooks on the other. Through the retrograde analysis used to construct endgame databases, we know that, with ideal play by both sides, the stronger side can force checkmate in 490 moves. Yes, 490. Kasparov says:

Now, I can tell you that -- even being a very decent player -- for the first 400 moves, I could hardly understand why these pieces moved around like a dance. It's endless dance around the board. You don't see any pattern, trust me. No pattern, because they move from one side to another.
At certain points I saw, "Oh, but white's position has deteriorated. It was better 50 moves before." The question is -- and this is a big question -- if there are certain positions in these endgames, like seven-piece endgames, that take, by the best play of both sides, 500 moves to win the game, what does it tell us about the quality of the game that we play, which is an average 50 moves? [...]
Maybe with machines, we can actually move our knowledge much further, and we can understand how to play decent games at much greater lengths.

But there's more. Do chess-playing computer programs, so much superior to even the best human players, understand these endgames either? I don't mean "understand" in the human sense, but only in the sense of being able to play games of that quality. Kasparov moves on to his analysis of games between the best programs:

I think you can confirm my observations that there's something strange in these games. First of all, they are longer, of course. They are much longer because machines don't make the same mistakes [we do] so they could play 70, 80 moves, 100 moves. [That is] way, way below what we expect from perfect chess.
That tells us that [the] machines are not perfect. Most of those games are decided by one of the machines suddenly. Can I call it losing patience? Because you're in a position that is roughly even. [...] The pieces are all over, and then suddenly one machine makes a, you may call, human mistake. Suddenly it loses patience, and it tries to break up without a good reason behind it.
That also tells us [...] that machines also have, you may call it, psychology, the pattern and the decision-making. If you understand this pattern, we can make certain predictions.

Kasparov is heartened by this, and it's part of the reason that he is not as pessimistic about the near-term prospects of AI as some well-known scientists and engineers are. Even with so-called deep learning, our programs are only beginning to scratch the surface of complexity in the universe. There is no particular reason to think that the opaque systems evolved to drive our cars and fly our drones will be any more perfect in their domains than our game-playing programs, and we have strong evidence from the domain of games that programs are still far from perfect.

On a more optimistic note, advances in AI give us an opportunity to use programs to help us understand the world better and to improve our own judgment. Kasparov sees this in chess, in the big gaps between the best human play, the best computer play, and perfect play in even relatively simple positions; I wrote wistfully about this last year, prompted by AlphaGo's breakthrough. But the opportunity is much more valuable when we move beyond playing games, as Cowen alluded in an aside during Kasparov's explanation: Imagine how bad our politics will look in comparison to computer programs that do it well! We have much to learn.

As always, this episode of Conversations with Tyler was interesting and evocative throughout. If you are a chess player, there is an special bonus. The transcript includes a pointer to Kasparov's Immortal Game against Veselin Topalov at Wijk aan Zee in 1999, along with a discussion of some of Kasparov's thoughts on the game beginning with the pivotal move 24. Rxd4. This game, an object of uncommon beauty, will stand as an eternal reminder why, even in the face of advancing AI, it will always matter that people play and compete and create.

~~~~

If you enjoyed this entry, you might also like Old Dreams Live On. It looks more foresightful now that AlphaGo has arrived.


Posted by Eugene Wallingford | Permalink | Categories: Computing, General

May 16, 2017 10:57 AM

The Big O

I recently ran across a newspaper column about triple-doubles, a secondary statistic in basketball that tracks when a player reaches "double figures" in three of the major primary stats. The column included this short passage about the great Oscar Robertson:

You probably know that Robertson averaged triple-doubles for an entire season, 1961-62. But did you know that he averaged triple-doubles over the cumulative first five seasons of his NBA career, from 1960-61 through '64-65? In that stretch Robertson averaged 30.3 points, 10.4 rebounds, and 10.6 assists.

This is indeed an amazing accomplishment. But it was not news to me.

I grew up in Indiana, a hotbed of basketball, with the sort of close personal attachment to Robertson that only a young sports fan can have. Like me, the "Big O" was from Indianapolis. He had led Crispus Attacks High School to two straight state championships in the late 1950s and helped lead the University of Cincinnati to national prominence in the early 1960s. When I first became aware of basketball as a young boy, Robertson was deep into a stellar pro career. He was one of my early sports idols.

Later, Robertson and his legacy played an unexpected role in my life. When I interviewed for the scholarship that would pay my way through college, I found that the interviewer, the dean of the Honors College, was a former Division III basketball player from Michigan who had gone on to earn a PhD in history from the University of Maryland. Our conversation quickly turned to basketball and our mutual admiration for the Big O, but it was not all sports talk. Robertson's career as a black player in the 1950s and '60s launched us into a discussion of urban segregation, race relations, and the role of sport in culture.

After the interview, I wondered if it had been wise tactially to talk so much about basketball. I guess the conversation went well enough, though.

Folks today can have Michael Jordan and LeBron. They are undeniably great players, but Oscar Robertson will always be my standard for all-around play -- and a touchpoint in my own life.


Posted by Eugene Wallingford | Permalink | Categories: Personal

May 09, 2017 1:22 PM

Pair Programming, Script Writing Edition

Screenwriter Ken Levine answers one of his Friday Questions about how he and writing partner David Isaacs worked:

We always worked on the same script. And we always worked together in the room. Lots of teams will divide up scenes, write separately, then return to either polish it together or rewrite each other's scenes on their own. We wrote head-to-head. To us the value of a partnership is to get immediate feedback from someone you trust, and more importantly, have someone to go to lunch with.

It sounds like Levine and Isaacs (MASH, Cheers, Frasier, ...) discovered the benefits of pair programming in their own line of work.

I liked the second part of his answer, too, about whether they ever gave up on a script once they starting writing it:

Nothing gets done unless both team members are committed to it. Once we began to write a spec there was never any discussion of just junking or tabling it to work on something else. We would struggle at times with the story or certain jokes but we always fought our way through it. Wrestling scripts to the ground is excellent training for when you do go on staff.

Wrestling code to the ground is excellent training for what you have to do as a developer, too. On those occasions when what you thought was a good idea turns out to be a bad one, it is wise to pitch it and move on. But it's too easy to blame difficulty in the trenches on the idea. Often, the difficulty is a hint that you need to work harder or dig deeper. Pairing with another programmer often provides the support you need to stick with it.


Posted by Eugene Wallingford | Permalink | Categories: Software Development

May 04, 2017 4:07 PM

Tweak

In an essay in The Guardian, writer George Saunders reflects on having written his first novel after many years writing shorter fiction. To a first approximation, he found the two experiences to be quite similar. In particular,

What does an artist do, mostly? She tweaks that which she's already done.

I read this on a day when I had just graded thirty-plus programming assignments from my junior/senior-level Programming Languages courses, and this made me think of student programmers. My first thought was snarky, and only partly tongue-in-cheek: Students write and then submit. Who has the time or interest to tweak?

My conscience quickly got the better of me, and I admitted that this was unfair. In a weak moment at the end of a long day, it's easy to be dismissive and not think about students as people who face all sorts of pressures both in and out of the classroom. Never forget Hanlon's Razor, my favorite formulation of which is:

Never attribute to malice or stupidity that which can be explained by moderately rational individuals following incentives in a complex system of interactions.

Even allowing the snark, my first thought was inaccurate. The code students submit is often the end result of laborious tweaking. The thing is, most students tweak only while the code gives incorrect answers. In the worst case, some students tweak and tweak, employing an age-old but highly inefficient software development methodology: Make another change and see if it works.

This realization brought to mind Kent Beck's Rules of Simple Design:

  1. passes the tests
  2. reveals intention
  3. has no duplication
  4. has the fewest elements possible

Most students are under time pressures that make artistry a luxury good; they are happy to find time to make their code work at all. If the code passes the tests, it's probably good enough for now.

But there is more to the student's willingness to stop tinkering so soon than just external pressures. It takes a lot of programming experience and a fair amount of time to come to even appreciate Rules 2 through 4. Why does it matter if code reveals the programmer's intention, in terms of either art or engineering? What's the big deal about a little duplication? The fewest elements? -- making that happen takes time that could be spent on something much more interesting.

I am coming to think of Kent's rules as a sort of Bloom's taxonomy for the development of programming expertise. Students start at Level 1, happy to write code that achieves its stated purpose. As they grow, programmers move through the rules, mastering deeper levels of understanding of design, simplicity, and, yes, artistry. They don't move through the stages in a purely linear fashion, but they do tend to master the rules in roughly the order listed above.

Today is a day of empathy for my novice programmers. As I write this, they are writing the final exam in my course. I hope that in a few weeks, after the blur of a busy semester settles in their minds, they reflect a bit and see that they have made progress as programmers -- and that they can now ask better questions about the programming languages they use than they could at the beginning of the course.


Posted by Eugene Wallingford | Permalink | Categories: Software Development, Teaching and Learning

April 28, 2017 11:27 AM

Data Compression and the Complexity of Consciousness

Okay, so this is cool:

Neuroscientists stimulate the brain with brief pulses of energy and then record the echoes that bounce back. Dreamless sleep and general anaesthesia return simple echoes; brains in conscious states produce more complex patterns. Then comes a little inspiration from data compression:

Excitingly, we can now quantify the complexity of these echoes by working out how compressible they are, similar to how simple algorithms compress digital photos into JPEG files. The ability to do this represents a first step towards a "consciousness-meter" that is both practically useful and theoretically motivated.

This made me think of Chris Ford's StrangeLoop 2015 talk about using compression to understand music. Using compressibility as a proxy for complexity gives us a first opportunity to understand all sorts of phenomena about which we are collecting data. Kolmogorov complexity is a fun tool for programmers to wield.

The passage above is from an Aeon article on the study of consciousness. I found it an interesting read from beginning to end.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Patterns, Software Development

April 15, 2017 10:39 AM

Science Seeks Regularity

A week or so ago I tweeted that Carver Mead was blowing my mind: an electron a mile long! I read about that idea in this Spectator interview that covers both Mead's personal life and his professional work in engineering. Well worth a read.

Mead is not satisfied with the current state of physics and biology, or at least with the incomplete theories that we seem to have accepted in lieu of a more coherent conceptual understanding of how the world works. Ultimately, he sides with Einstein in his belief that there is a more coherent explanation:

I think Einstein was being a scientist in the truest sense in his response to the Copenhagen interpretation. He said that none of us would be scientists if deep down we didn't believe there is a set of regularities in the operation of physical law. That is a matter of faith. It is not something anybody has proven, but none of us would be scientists if we didn't have that faith.

Like Einstein, Mead believes that unpredictability at the lowest levels of a system does not imply intrinsic uncertainty. We need a different view that brings regularities to the forefront of our theories.

I also like this line from near the end of the interview:

People don't even know where to put the decimal point.

Mead says this as part of his assurance that artificial intelligence is nowhere near the level of what even the fruit fly can do, let alone the human brain. A lot has happened in AI during fifteen years since this interview; a computer program even beats our best players in Go now. Still, there is so much that we don't understand and cannot replicate.

I wonder if Mead's "decimal point" aphorism also might apply, metaphorically, to his view of the areas of science in which we have settled for, or are currently stuck with, unsatisifying theories. Our mathematical models cover a lot of ground, decimal point-wise, but there is still a simpler, more coherent picture to see. Maybe, though, that is the engineer in Mead showing through.


Posted by Eugene Wallingford | Permalink | Categories: General

April 07, 2017 1:33 PM

Two Very Different Kinds of Student

The last sentence of each of these passages reminds me of some of the students over the years.

First this, from Paul Callaghan's The Word Chain Kata:

One common way to split problems is via the "generate and test" pattern, where one component suggests results and the other one discards invalid ones. (In my experience, undergrad programmers are experts in this pattern, although they tend to use it as a [software development] methodology--but that's another story.)

When some students learn to program for the first time, they start out by producing code that looks like something they have seen before, trying it out, and then tinkering with it until it works or until they become exasperated. (I always hope that they act on their exasperation by asking me for help, rather than by giving up.) These students usually understand little bits of the code locally, but they don't really understand the program or function as a whole. Yet, somehow, they find a way to make it work.

It's surprising how far some students can get in a course of study by programming this way. (That's why Callaghan calling the approach a methodology made me smile.) It's unfortunate, too, because eventually the approach hits a wall when problems and domains become more challenging. Or when they run into a course where they program in Racket, in which one misplaced parenthesis can cause an otherwise perfect piece of code to implode. Lisp-like languages do not provide a supportive environment for this kind of "programming by wandering around".

And then there's this, from Andy Hertzfeld's fun little story about the writing of the classic manual Inside Macintosh:

Pretty soon, I figured out that if Caroline had trouble understanding something, it probably meant that the design was flawed. On a number of occasions, I told her to come back tomorrow after she asked a penetrating question, and revised the API to fix the flaw that she had pointed out. I began to imagine her questions when I was coding something new, which made me work harder to get things clearer before I went over them with her.

In this story, Caroline is not a student, but a young new writer assigned to the Mac documentation team. Still, she reminds me of students who are as delightful to work with as generate-and-test programmers can be frustrating. These students pay attention. They ask good questions, ones that often challenge the unstated assumptions underlying what we have explained before. At first, this can seem frustrating to us teachers, because we have to formulate answers for things that should be obvious. But that's the point: they aren't obvious, at least not to everyone, and us thinking they are obvious is inhibiting our teaching.

Last semester, I had one team of students in my compilers class that embodied this attitude. They asked questions no one had ever bothered to ask me before. At first, I thought, "How can these guys not understand such basic material?" Like Hertzfeld, though, pretty soon I figured out that their questions were exposing holes or weaknesses in my lectures, examples, and sample code. I began to anticipate their questions as I prepared for class. Their questions helped me see ways to make my course better.

As along so many other dimensions, part of the challenge in teaching CS is the wide variation in the way students study, learn, and approach their courses. It is also a part of the great fun of teaching, especially when you encounter the Carolines and Averys who push me to get better.


Posted by Eugene Wallingford | Permalink | Categories: Software Development, Teaching and Learning

April 02, 2017 12:02 PM

Reading an Interview with John McPhee Again, for the First Time

"This weekend I enjoyed Peter Hessler's interview of McPhee in The Paris Review, John McPhee, The Art of Nonfiction No. 3."

That's a direct quote from this blog. Don't remember it? I don't blame you; neither do I. I do remember blogging about McPhee back when, but as I read the same Paris Review piece again last Sunday and this, I had no recollection of reading it before, no sense of déjà vu at all.

Sometimes having a memory like mine is a blessing: I occasionally get to read something for the first time again. If you read my blog, then you get to read my first impressions for a second time.

I like this story that McPhee told about Bob Bingham, his editor at The New Yorker:

Bingham had been a writer-reporter at The Reporter magazine. So he comes to work at The New Yorker, to be a fact editor. Within the first two years there, he goes out to lunch with his old high-school friend Gore Vidal. And Gore says, What are you doing as an editor, Bobby? What happened to Bob Bingham the writer? And Bingham says, Well, I decided that I would rather be a first-rate editor than a second-rate writer. And Gore Vidal draws himself up and says, And what is wrong with a second-rate writer?

I can just hear the faux indignation in Vidal's voice.

McPhee talked a bit about his struggle over several years to write a series of books on geology, which had grown out of an idea for a one-shot "Talk of the Town" entry. The interviewer asked him if he ever thought about abandoning the topic and moving on to something he might enjoy more. McPhee said:

The funny thing is that you get to a certain point and you can't quit. Because I always worried: if you quit, you'll quit again. The only way out was to go forward, to learn your way and write your way out of it.

I know that feeling. Sometimes, I really do need to quit something and move on, but I always wonder whether quitting this time will make it easier to do next time. Because sometimes, I need to stick it out and, as McPhee says, learn my way out of the difficulty. I have no easy answers for knowing when quitting is the right thing to do.

Toward the end of the interview, the conversation turned to the course McPhee teaches at Princeton, once called "the literature of fact". The university first asked him to teach on short notice, over the Christmas break in 1974, and he accepted immediately. Not everyone thought it was a good idea:

One of my dear friends, an English teacher at Deerfield, told me: Do not do this. He said, Teachers are a dime a dozen -- writers aren't. But my guess is that I've been more productive as a writer since I started teaching than I would have been if I hadn't taught. In the overall crop rotation, it's a complementary job: I'm looking at other people's writing, and the pressure's not on me to do it myself. But then I go back quite fresh.

I know a lot of academics who feel this way. Then again, it's a lot easier to stay fresh in one's creative work if one has McPhee's teaching schedule, rather than a full load of courses:

My schedule is that I teach six months out of thirty-six, and good Lord, that leaves a lot of time for writing, right?

Indeed it does. Indeed it does.

On this reading of the interview, I marked only two passages that I wrote about last time. One came soon after the above response, on how interacting with students is its own reward. The other was a great line about the difference between mastering technique and having something to say: You demonstrated you know how to saddle a horse. Now go find the horse.

That said, I unconsciously channeled this line from McPhee just yesterday:

Writing teaches writing.

We had a recruitment event on campus, and I was meeting with a dozen or so prospective students and their entourages. We were talking about our curriculum, and I said a few words about our senior project courses. Students generally like these courses, even though they find them difficult. The students have never had to write a big program over the course of several months, and it's harder than it looks. The people who hire our graduates like these courses, too, because they know that these courses are places where students really begin to learn to program.

In the course of my remarks, I said something to the effect, "You can learn a lot about programming in classes where you study languages and techniques and theory, but ultimately you learn to write software by writing software. That's what the project courses are all about." There were a couple of experienced programmers in the audience, and they were all nodding their heads. They know McPhee is right.


Posted by Eugene Wallingford | Permalink | Categories: General, Software Development, Teaching and Learning

March 29, 2017 4:16 PM

Working Through A Problem Manually

This week, I have been enjoying Eli Bendersky's two-article series "Adventures in JIT Compilation":

Next I'll follow his suggestion and read the shorter How to JIT - An Introduction.

Bendersky is a good teacher, at least in the written form, and I am picking up a lot of ideas for my courses in programming languages and compilers. I recommend his articles and his code highly.

In Part 2, Bendersky says something that made me think of my students:

One of my guiding principles through the field of programming is that before diving into the possible solutions for a problem (for example, some library for doing X) it's worth working through the problem manually first (doing X by hand, without libraries). Grinding your teeth over issues for a while is the best way to appreciate what the shrinkwrapped solution/library does for you.

The presence or absence of this attitude is one of the crucial separators among CS students. Some students come into the program with this mindset already in place, and they are often the ones who advance most quickly in the early courses. Other students don't have this mindset, either by interest or by temperament. They prefer to solve problems quickly using canned libraries and simple patterns. These students are often quite productive, but they sometimes soon hit a wall in their learning. When a student rides along the surface of what they are told in class, never digging deeper, they tend to have a shallow knowledge of how things work in their own programs. Again, this can lead to a high level of productivity, but it also produces brittle knowledge. When something changes, or the material gets more difficult, they begin to struggle. A few of the students eventually develop new habits and move nicely into the group of students who likes to grind. The ones who don't make the transition continue to struggle and begin to enjoy their courses less.

There is a rather wide variation among undergrad CS students, both in their goals and in their preferred styles or working and learning. This variation is one of the challenges facing profs who hope to reaching the full spectrum of students in their classes. And helping students to develop new attitudes toward learning and doing is always a challenge.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning