Knowing and Doing

Like me, you probably see references to this classic quote from Donald Rumsfeld all the time:

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say, we know there are some things we do not know. But there are also unknown unknowns -- the ones we don't know we don't know.

I recently ran across it again in an old Epsilon Theory post that uses it to frame the difference between decision making under risk (the known unknowns) and decision-making under uncertainty (the unknown unknowns). It's a good read.

Seeing the passage again for the umpteenth time, it occurred to me that no one ever seems to talk about the fourth quadrant in that grid: the unknown knowns. A quick web search turns up a few articles such as this one, which consider unknown knowns from the perspective of others in a community: maybe there are other people who know something that you do not. But my curiosity was focused on the first-person perspective that Rumsfeld was implying. As a knower, what does it mean for something to be an unknown known?

My first thought was that this combination might not be all that useful in the real world, such as the investing context that Ben Hunt writes about in Epsilon Theory. Perhaps it doesn't make any sense to think about things you don't know that you know.

As a student of AI, though, I suddenly made an odd connection ... to explanation-based learning. As I described in a blog post twelve years ago:

Back when I taught Artificial Intelligence every year, I used to relate a story from Russell and Norvig when talking about the role knowledge plays in how an agent can learn. Here is the quote that was my inspiration, from Pages 687-688 of their 2nd edition:

Sometimes one leaps to general conclusions after only one observation. Gary Larson once drew a cartoon in which a bespectacled caveman, Zog, is roasting his lizard on the end of a pointed stick. He is watched by an amazed crowd of his less intellectual contemporaries, who have been using their bare hands to hold their victuals over the fire. This enlightening experience is enough to convince the watchers of a general principle of painless cooking.

I continued to use this story long after I had moved on from this textbook, because it is a wonderful example of explanation-based learning.

In a mathematical sense, explanation-based learning isn't learning at all. The new fact that the program learns follows directly from other facts and inference rules already in its database. In EBL, the program constructs a proof of a new fact and adds the fact to its database, so that it is ready-at-hand the next time it needs it. The program has compiled a new fact, but in principle it doesn't know anything more than it did before, because it could always have deduced that fact from things it already knows.

As I read the Epsilon Theory article, it struck me that EBL helps a learner to surface unknown knowns by using specific experiences as triggers to combine knowledge it already into a piece of knowledge that is usable immediately without having to repeat the (perhaps costly) chain of inference ever again. Deducing deep truths every time you need them can indeed be quite costly, as anyone who has ever looked at the complexity of search in logical inference systems can tell you.

When I begin to think about unknown knowns in this way, perhaps it does make sense in some real-world scenarios to think about things you don't know you know. If I can figure it all out, maybe I can finally make my fortune in the stock market.

Earlier this week, there was a meme on Twitter where people gave one-line advice to young students as they stepped onto a college campus as first-years, to help them enjoy and benefit from their college years. I didn't have anything clever or new to say, so I didn't join in, but something I read this morning triggered a bit of old wisdom that I wish more students would try to live out. In tweet-size form, it might be: "Learn the basics, struggle a bit, then ask questions." Here's the blog-size version.

In Tyler Cowen's conversation with Google economist Hal Varian, Cowen asks about a piece of advice Varian had once given to graduate students: "Don't look at the literature too soon." Is that still good advice, and why? Yes, Varian replied...

VARIAN: Because if you look at the literature, you'll see this completely worked-out problem, and you'll be captured by that person's viewpoint. Whereas, if you flounder around a little bit yourself, who knows? You might come across a completely different phenomenon. Now, you do have to look at the literature. I want to emphasize that. But it's a good idea to wrestle with a problem a little bit on your own before you adopt the standard viewpoint.

Grad students are often trying to create new knowledge, so it's best for them not to lock themselves into existing ways of thinking too soon. Thus: Don't look at the literature too soon.

I work mostly with undergrads, who study in a different context than grad students. But I think that the core of Varian's advice works well for undergrads, too: Start by learning a few basic ideas in class. Then try to solve problems. Then ask questions.

Undergrads are usually trying to master foundational material, not create new knowledge, so it's tempting to want to jump straight to answers. But it's still to valuable approach the task of learning as a process of building one's own understanding of problems before seeking answers. Banging on a bunch of problems helps us to build instincts about what the important issues and to explore the fuzzy perimeter between the basics and the open questions that will vex us after we master them. That happens best when we don't see a solution right away, when what we learned in class doesn't seem to point us directly to a solution and we have to find our own way.

But do ask questions! A common theme among students who struggle in my courses is the belief they just have to work harder or longer on a problem. Too many times I've had a student tell me "I spent an hour on each of the five homework problems." Really? My goal is for each problem to take 15 minutes or less. After half an hour, or maybe a second attempt the next day, maybe you are missing something small but important. Ask a question; maybe a little nudge can put you on the right track. Sometimes, your question will help me realize that it's the problem which is flawed and needs a tweak!

Back at the beginning of the process, too strong a belief in the ability to figure things out on one's own creates a different sort of breakdown in the learning process: It can be tempting to skip over what you read in your textbook and what you learn in class, and start trying to solving problems. "It's basic material, right? I'll figure it out." You might, but that's taking the idea to an unhealthy level. There's a difference between seeking answers too soon and trying to solve problems without the basic tools you need. Trust your profs a little bit... In class, they are trying to give you the basic tools you need to solve interesting problems.

There's nothing new here. But let's be honest; there isn't much new to be found in ways to learn. Even in the digital age, the basic tenets remain true. That's why I extol curiosity and persistence and why I'd rather be Mr. Miyagi than an answer machine. Learning will be uncomfortable. The trick is to find a way to balance the curiosity with the discomfort, the not-knowing with the receiving of answers and moving on. I wish I had great advice for how to find that balance, but I think people ultimately have to do that for themselves. We benefit by being part of a community of learners, but we each learn in our own ways and on our own time.

Actually, writing up this post has led me to a goal for myself as a teacher this year, and which may be good advice for my fellow teachers: Be more explicit about my expectations of students. This is true both at the micro-level of, say, how much time to spend on homework problems before seeking help, and at the macro-level of how to approach learning. If I want students to do something, I should at least remove the barriers between what they are thinking they should do and what I would like for them to do.

So there's some advice for students and some advice for teachers. Let's enjoy the new year and learn together.

I hope to eventually write up a reflection on my first Dagstuhl seminar, but for now I have a short story about how I encountered a new idea three times in ten days, purely by coincidence. Actually, the idea is over one hundred fifty years old but, as my brother often says, "Hey, it's new to me."

On the second day of Dagstuhl, Mark Guzdial presented a poster showing several inspirations for his current thinking about task-specific programming languages. In addition to displaying screenshots of two cool software tools, the poster included a picture of an old mechanical device that looked both familiar and strange. Telegraphy had been invented in the early 1840s, and telegraph operators needed some way to type messages. But how? The QWERTY keyboard was not created for the typewriter until the early 1870s, and no other such devices were in common use yet. To meet the need, Royal Earl House adapted a portion of a piano keyboard to create the input device for the "printing telegraph", or teleprinter. The photo on Mark's poster looked similar to the one on Wikipedia page for the teleprinter.

There was a need for a keyboard thirty years before anyone designed a standard typing interface, so telegraphers adapted an existing tool to fit their needs. What if we are in that same thirty-year gap in the design of programming languages? This has been one of Mark's inspirations as he works with non-computer scientists on task-specific programming languages. I had never seen an 1870s teleprinter before and thought its keyboard to be a rather ingenious way to solve a very specific problem with a tool borrowed from another domain.

When Dagstuhl ended, my wife and I spent another ten days in Europe on a much-needed vacation. Our first stop was Paris, and on our first full day there we visited the museum of the Conservatoire National des Arts et Métiers. As we moved into the more recent exhibits of the museum, what should I see but...

... a Hughes teleprinter with piano-style keyboard, circa 1975. Déjà vu! I snapped a photo, even though the device was behind glass, and planned to share it with Mark when I got home.

We concluded our vacation with a few days in Martinici, Montenegro, the hometown of a department colleague and his wife. They still have a lot of family in the old country and spend their summers there working and relaxing. On our last day in this beautiful country, we visited its national historical museum, which is part of the National Museum of Montenegro in the royal capital of Cetinje. One of the country's most influential princes was a collector of modern technology, and many of his artifacts are in the museum -- including:

This full-desk teleprinter was close enough to touch and examine up close. (I didn't touch!) The piano keyboard on the device shows the wear of heavy use, which brings to mind each of my laptops' keyboards after a couple of years. Again, I snapped a photo, this time in fading light, and made a note to pass it on.

In ten days, I went from never having heard much about a "printing telegraph" to seeing a photo of one, hearing how it is an inspiration for research in programming language design, and then seeing two such devices that had been used in the 19th-century heyday of telegraphy. It was an unexpected intersection of my professional and personal lives. I must say, though, that having heard Mark's story made the museum pieces leap into my attention in a way that they might not have otherwise. The coincidence added a spark to each encounter.

As so often, Marvin Minsky loved to tell us about the beauty of programming. Kids love to play with construction sets like Legos, TinkerToys, and Erector sets. Programming provides an infinite construction kit: you never run out of parts!

In the linked essay, which was published as a preface to a 1986 book about Logo, Minsky tells several stories. One of the stories relates that once, as a small child, he built a large tower out of TinkerToys. The grownups who saw it were "terribly impressed". He inferred from their reaction that:

some adults just can't understand how you can build whatever you want, so long as you don't run out of sticks and spools.

Kids get it, though. Why do so many of us grow out of this simple understanding as we get older? Whatever its cause, this gap between children's imaginations and the imaginations of adults around them creates a new sort of problem when we give the children a programming language such as Logo or Scratch. Many kids take to these languages just as they do to Legos and TinkerToys: they're off to the races making things, limited only by their expansive imaginations. The memory on today's computers is so large that children never run out of raw material for writing programs. But adults often don't possess the vocabulary for talking with the children about their creations!

... many adults just don't have words to talk about such things -- and maybe, no procedures in their heads to help them think of them. They just do not know what to think when little kids converse about "representations" and "simulations" and "recursive procedures". Be tolerant. Adults have enough problems of their own.

Minsky thinks there are a few key ideas that everyone should know about computation. He highlights two:

Computer programs are societies. Making a big computer program is putting together little programs.

Any computer can be programmed to do anything that any other computer can do--or that any other kind of "society of processes" can do.

He explains the second using ideas pioneered by Alan Turing and long championed in the popular sphere by Douglas Hofstadter. Check out this blog post, which reflects on a talk Hofstadter gave at my university celebrating the Turing centennial.

The inability of even educated adults to appreciate computing is a symptom of a more general problem. As Minsky says toward the end of his essay, People who don't appreciate how simple things can grow into entire worlds are missing something important. If you don't understand how simple things can grow into complex systems, it's hard to understand much at all about modern science, including how quantum mechanics accounts for what we see in the world and even how evolution works.

You can usually do well by reading Minsky; this essay is a fine example of that. It comes linked to an afterword written by Alan Kay, another computer scientist with a lot to say about both the beauty of computing and its essential role in a modern understanding of the world. Check both out.