A colleague sent me the following exchange from his class, with the tag line "Best comments of the day." His students were working in groups to design a Java program for Conway's Game of Life.
Student 1: I can't comprehend what you are saying.
Student 2: The board doesn't have to be rectangular, does it?
Instructor: In Conway's design, it was. But abstractly, no.
Student 3: So you could have a board of different shapes, or you could even have a three-dimensional "board". Each cell knows its neighbors even if we can't easily display it to the user.
Instructor: Sure, "neighbor" is an abstract concept that you can implement differently depending on your need.
Student 2: I knew there was a reason I took linear algebra.
Student 1: Ok. So let's only allow rectangular boards.
Maybe Student 1 still can't comprehend what everyone is saying... or perhaps he or she understands perfectly well and is a pragmatist. YAGNI for the win!
It always makes me happy when a student encounters a situation in which linear algebra is useful and recognizes its applicability unprompted.
I salute all three of these students, and the instructor who is teaching the class. A good day.
This morning I read three pieces with some connection to universities and learning. Each had a one passage that made me smart off silently as I pedaled.
Boyarin describes his own research as not merely interdisciplinary but "deeply post-disciplinary." (He jokes that when he first came to Berkeley, his dream was to be 5 percent in 20 departments.)
Good luck getting tenure that way, dude.
"Deeply post-disciplinary" is a great bit of new academic jargon. Universities are very much organized by discipline. Figuring out how to support scholars who work outside the lines is a perpetual challenge, one that we really should address at scale if we want to enable universities to evolve.
From this article on Bernie Sanders's free college plan:
Big-picture principles are important, but implementation is important, too.
Hey, maybe he just needs a programmer.
Implementing big abstractions is hard enough when the substance is technical. When you throw in social systems and politics, implementing any idea that deviates very far from standard practice becomes almost impossible. Big Ball of Mud, indeed.
Being taught is the intellectual analog of being loved.
I'll remind my students of this tomorrow when I give them Exam 3, on syntactic abstraction. "I just called to say 'I love you'."
Asimov is right. When I think back on all my years in school, I feel great affection for so many of my teachers, and I recall feeling their affection for me. Knowledge is not only power, says Asimov; it is happiness. When people help me learn they offer me knew ways to be happy.
( The Foundation Trilogy makes me happy, too.)
Dan Luu's blog is usually both entertaining and insightful. We Only Hire The Trendiest is no exception.
I did my undergrad at Wisconsin, which is one of the 25 schools that claims to be a top 10 cs/engineering school...
Even though he comes from one of the best CS schools, Luu recognizes that grads of many other schools are well prepared for careers in industry. Companies that bias their hiring toward the top schools miss out on a lot of "talent". Then again, they miss out on a lot of great talent in many other ways, too. For example:
A typical rejection reason was something like "we process millions of transactions per day here and we really need someone with more relevant experience who can handle these things without ramping up".
The people you don't hire won't have to ramp up. There is only one problem: You didn't hire anyone, so there's no way they can help you handle these things. Maybe you should hire people with strong foundations, a little curiosity, and a little drive. Those people can develop all the relevant experience you need.
As Luu says, "It's much easier to hire people who are underrated, especially if you're not paying market rates." That's where the Moneyball analogy comes in handy. I wonder if any company is doing sabermetric-like analysis of software developers? If so, they could develop a durable competitive advantage in hiring.
Software entrepreneurs and companies in my part of the world, the American Midwest, have always faced stiff challenges when it comes to hiring and retaining tech talent from the coasts. Luu reminds them that they also have an opportunity: find underrated people from non-elite schools who want to stay in the Midwest, and then develop a team of professionals who rival in performance anything you can find on the coasts.
In my town, Banno is a great example of a company that has accepted this challenge. It's not easy, but with patience and strong leadership, it seems to be working.
Somehow, I recently came across a link to Stay on the Bus, an excerpt from a commencement speech Arno Rafael Minkkinen gave at the New England School of Photography in June 2004. It is also titled "The Helsinki Bus Station Theory: Finding Your Own Vision in Photography". I almost always enjoy reading the thoughts of an artist on his or her own art, and this was no exception. I also usually hear echoes of what I feel about my own arts and avocations. Here are three.
What happens inside your mind can happen inside a camera.
This is one of the great things about any art. What happens inside your mind can happen in a piano, on a canvas, or in a poem. When people find the art that channels their mind best, beautiful things -- and lives -- can happen.
One of the things I like about programming is that is really a meta-art. Whatever happens in your mind can happen inside a camera, inside a piano, on a canvas, or in a poem. Yet whatever happens inside a camera, inside a piano, or on a canvas can happen inside a computer, in the form of a program. Computing is a new medium for experimenting, simulating, and creating.
Teachers who say, "Oh, it's just student work," should maybe think twice about teaching.
Amen. There is no such thing as student work. It's the work our students are ready to make at a particular moment in time. My students are thinking interesting thoughts and doing their best to make something that reflects those thoughts. Each program, each course in their study is a step along a path.
All work is student work. It's just that some of us students are at a different point along our paths.
Georges Braque has said that out of limited means, new forms emerge. I say, we find out what we will do by knowing what we will not do.
This made me think of an entry I wrote many years ago, Patterns as a Source of Freedom. Artists understand better than programmers sometimes that subordinating to a form does not limit creativity; it unleashes it. I love Minkkinen's way of saying this: we find out what we will do by knowing what we will not do. In programming as in art, it is important to ask oneself, "What will I not do?" This is how we discover we will do, what we can do, and even what we must do.
Those are a few of the ideas that stood out to me as I read Minkkinen's address. The Helsinki Bus Station Theory is a useful story, too.
The San Antonio Spurs perennially challenge for the championship of the National Basketball Association. Like most great NBA teams, they have several excellent players. However, the foundation of their success isn't the high-flying sort of offense that people often associate with professional basketball, but rather a meticulous defense, the sort of defense people usually associate with defensive specialists and hard-working journeymen.
What's the secret? This article tells us there is no magic:
It's easy to think some form of incomprehensible genius is responsible for the subtle components of an elite defense, but in all reality, doing the obvious thing and doing it well (which is the hard part) is often all it takes.
This is true of so many things in life. Figure out what you need to do to excel, and then practice it -- both in preparation for the act and in the act itself.
It's not very exciting, but grunt work and attention to detail are usually the primary components of excellence. A little luck helps, of course; the Spurs were able to draft all-time great Tim Duncan as David Robinson's career was winding down. But even that luck is tinged with an unexciting message... What makes Duncan great isn't flashy style and supernatural skills. It's mostly doing the obvious things and doing them well.
courtesy of the American Go Association
In Why AlphaGo Matters, Ben Kamphaus writes:
AlphaGo recognises strong board positions by first recognizing visual features in the board. It's connecting movements to shapes it detects. Now, we can't see inside AlphaGo unless DeepMind decides they want to share some of the visualizations of its intermediate representations. I hope they do, as I bet they'd offer a lot of insight into both the game of Go and how AlphaGo specifically is reasoning about it.
I'm not sure seeing visualizations of AlphaGo's intermediate representations would offer much insight into either the game of Go or how AlphaGo reasons about it, but I would love to find out.
One of the things that drew me to AI when I was in high school and college was the idea that computer programs might be able to help us understand the world better. At the most prosaic level, I though this might happen in what we had to learn in order to write an intelligent program, and in how we structured the code that we wrote. At a more interesting level, I thought that we might have a new kind of intelligence with which to interact, and this interaction would help us to learn more about the domain of the program's expertise.
Alas, computer chess advanced mostly by making computers that were even faster at applying the sort of knowledge we already have. In other domains, neural networks and then statistical approaches led to machines capable of competent or expert performance, but their advances were opaque. The programs might shed light on how to engineer systems, but the systems themselves didn't have much to say to us about their domains of expertise or competence.
Intelligent programs, but no conversation. Even when we play thousands of games against a chess computer, the opponent seems otherworldly, with no new principles emerging. Perhaps new principles are there, but we cannot see them. Unfortunately, chess computers cannot explain their reasoning to us; they cannot teach us. The result is much less interesting to me than my original dreams for AI.
Perhaps we are reaching a point now where programs such as AlphaGo can display the sort of holistic, integrated intelligence that enables them to teach us something about the game -- even if only by playing games with us. If it turns out that neural nets, which are essentially black boxes to us, are the only way to achieve AI that can work with us at a cognitive level, I will be chagrined. And most pleasantly surprised.
All last spring, I planned to blog about my Programming Languages class but never got around to writing more than a couple of passing thoughts. I figured this spring would be different, yet here we are at the end of Week 9 and I've not even mentioned the course. This is how guys like me fail in personal relationships. We think a lot of good thoughts but don't follow through with our time and attention.
I am tempted to say that this has been a seesaw semester, but on reflection things have gone pretty well. I have a good group of students, most of whom are engaged in the topic and willing to interact in class. (Lack of interest and engagement was a bit of a problem last spring.) We've had fun in and out of class.
The reason it sometimes feels like I'm on a seesaw is that the down moments stand out more in my memory than they deserve to. You'd think that, as long as I've been teaching, I would have figured out how to manage this dynamic more effectively. Maybe it's just a part of being a teacher. We want things to go perfectly for our students, and when they don't we have to wonder why not.
One source of concern this semester has been the average exam score for the first two tests. They have been lower than historic averages in the course. Have I changed my expectations? Have I done something differently in the classroom? After taking a hard look back on my previous semesters' notes and assignments, I think not. My job now is to help this class reach the level I think they can reach. What can I do differently going forward? What can they do differently, and how do I help them do it?
I know they are capable of growth. Early last month, PhD Comics ran a strip titled The Five Most Typed Words in Academia. The winner was "Sorry for the late reply". At the time, the five most common words my students had said to me in the young semester were:
I didn't read the instructions.
For example, the student would say, "This problem was hard because I didn't know how to take the remainder properly." Me: "I gave that information in the instructions for the assignment." Student: "Oh, I didn't read the instructions."
Fortunately, we seem to moved beyond that stage of our relationship, as most students have come to see that the assignment may actually include some clues to help them out. Or maybe they've just stopped telling me that they don't read the instructions. If so, I appreciate them sparing my feelings.
A related problem is a perennial issue in this course: We learn a new technique, and some students choose not to use it. Writing code to process language expressions is a big part of the course, so we study some structural recursion patterns for processing a grammar specified in BNF. Yet a few students insist on whacking away at the problem with nothing more than a cond expression and a brave heart. When I ask them why, they sometimes say:
I didn't have time to use the technique we learned in class, so...
... so they spent twice as long trying to find their way to a working solution. Even when they find one, the code is generally unreadable even to them. Functional programming is hard, they say.
Fortunately, again, many seem to moved beyond this stage and are now listening to their data structures. The result is beautiful code: short, clear, and expressive. Grading such programs is a pleasure.
It recently occurred to me, though, that I have been guilty of the "I didn't have time to use your technique..." error myself. While trying to improve the recursive programming unit of the course over the last few years, I seem to have invented my own version of the Design Recipe from How to Design Programs. In the spirit of Greenspun's Tenth Rule, I have probably reinvented an ad hoc, informally-specified, bug-ridden, pedagogically less sound version of the Design Recipe.
As we used to say in Usenet newsgroups, "Pot. Kettle, Black." Before the next offering of the course, I intend to do my homework properly and find a way to integrate HtDP's well-tested technique into my approach.
These things stand out in my mind, yet I think that the course is going pretty well. And just when I begin to wonder whether I've been letting the class down, a few students stop in my office and say that this is one of their favorite courses CS courses ever. They are looking forward to the compiler course this fall. I'm not sure students realize the effect such words have on their instructors -- at least on this one.
Off to spring break we go. When we get back: lexical addressing and (eventually) a look at stateful programming. Fun and surprise await us all.
I'm on a mailing list of sports fans who happen also to be geeks of various kinds, including programmers and puzzle nuts. Last Friday, one of my friends posted this link and puzzle to the list:
Two players go on a hot new game show called "Higher Number Wins". The two go into separate booths, and each presses a button, and a random number between zero and one appears on a screen. (At this point, neither knows the other's number, but they do know the numbers are chosen from a standard uniform distribution.) They can choose to keep that first number, or to press the button again to discard the first number and get a second random number, which they must keep. Then, they come out of their booths and see the final number for each player on the wall. The lavish grand prize -- a case full of gold bouillon -- is awarded to the player who kept the higher number. Which number is the optimal cutoff for players to discard their first number and choose another? Put another way, within which range should they choose to keep the first number, and within which range should they reject it and try their luck with a second number?
From there, the conversation took off quiickly with a lot of intuition and analysis. There was immediate support for the intuitive threhold of 0.5, which a simple case analysis shows to give the maximum expected value for a player, 0.625. Some head-to-head analysis of various combinations, however, showed other values winning more often than 0.5, with values around 0.6 doing the best.
What was up? One of these analyses was wrong, but we weren't sure which. One list member, who had built a quick model in Excel, said,
I think the optimum may be somewhere around .61, but I'm damned if I know why.
I can't help thinking we're ignoring something game theoretical. I'm expecting we've all arrived at the most common wrong answer.
To confirm his suspicions, this person went off and wrote a C program -- a "terrible, awful, ugly, horrible C program" -- to compute all the expected values for all possible head-to-head cases. He announced,
We have a winner, according to my program. ... 61 wins.
He posted a snippet of the output from his program, which showed a steady rise in the win percentages for cutoffs up to a threshold of 0.61, which beat the 99 other cutoffs, with a steady decline for cutoffs thereafter.
Before reading the conversation on the mailing list, I discussed the puzzle with my wife. We were partial to 0.5, too, but that seemed too simple... So I sat down and wrote a program of my own.
My statistics skills are not as strong as many of my friends, and for this reason I like to write programs that simulate the situation at hand. My Racket program creates players who use all possible thresholds, plays matches of 1,000,000 games between each pair, and tallies up the results. Like the C program written by my buddy, my program is quick and dirty; it replays all hundred combinations on each pass, without taking advantage of the fact that the tournament matrix is symmetric. It's slower than it needs to be, but it gets the job done.
Player 57 defeats 95 opponents. Player 58 defeats 96 opponents. Player 59 defeats 99 opponents. Player 60 defeats 100 opponents. Player 61 defeats 98 opponents. Player 62 defeats 96 opponents. Player 63 defeats 93 opponents.
The results of my simulation mirrored the results of the brute-force case analysis. In simulation, 0.6 won, with 0.59 and 0.61 close behind. The two approaches gave similar enough results that it's highly likely there are bugs in neither program -- or both!
Once my friends were confident that 0.5 was not the winner, they were able to diagnose the error in the reasoning that made us think it was the best we could do: Although a player's choice of strategies is independent of the other player's choice, we cannot treat the other player's value as a uniform distribution over [0..1]. That is true only when they choose a threshold of 0 or 1.
In retrospect, this seems obvious, and maybe it was obvious to my mathematician friends right off the bat. But none of us on the mailing list is a professional statistician. I'm proud that we all stuck with the problem until we understood what was going on.
I love how we can cross-check our intuitions about puzzles like this with analysis and simulation. There is a nice interplay between theory and empirical investigation here. A simple theory, even if incomplete or incorrect, gives us a first approximation. Then we run a test and use the results to go back and re-think our theory. The data helped us see the holes in our thinking. What works for puzzles also works for hairier problems out in the world, too.
And we created the data we ended by writing a computer program. You know how much I like to do that. This the sort of situation we see when writing chess-playing programs and machine learning programs: We can write programs that are smarter than we are by starting from much simpler principles that we know and understand.
This experience is also yet another reminder of why, if I ever go freelance as a consultant or developer, I plan to team up with someone who is a better mathematician than I am. Or at least find such a person to whom I can sub-contract a sanity check.
In Sledgehammers vs Nut Crackers, Thomas Guest talks about pulling awk of the shelf to solve a fun little string-processing problem. He then shows a solution in Python, one of his preferred general-purpose programming languages. Why bother with languages like awk when you have Python at the ready? Guest writes:
At the outset of this post I admitted I don't generally bother with awk. Sometimes, though, I encounter the language and need to read and possibly adapt an existing script. So that's one reason to bother. Another reason is that it's elegant and compact. Studying its operation and motivation may help us compose and factor our own programs -- programs far more substantial than the scripts presented here, and in which there will surely be places for mini-languages of our own.
As I watch some of my students struggle this semester to master Racket, recursive programming, and functional style, I offer them hope that learning a new language and a new style will make them better Python and Java programmers, even if they never write another Racket or Lisp program again. The more different ways we know how to think about problems and solutions, the more effective we can be as solvers of problems. Of course, Racket isn't a special purpose language, and a few students find they so like the new style that they stick with the language as their preferred tool.
Experienced programmers understand what Guest is saying, but in the trenches of learning, it can be hard to appreciate the value of knowing different languages and being able to think in different ways. My sledgehammer works fine, my students say; why am I learning to use a nutcracker? I have felt that sensation myself.
I try to keep this feeling in mind as my students work hard to master a new way of thinking. This helps me empathize with their struggle, all the while knowing that Racket will shape how some of them think about every program they write in the future.