Knowing and Doing

Yes, according to Jeff Jarvis. That is one of his unpopular opinions about AI:

Machines should have the same right to learn as humans; to say otherwise is to set a dangerous precedent for humans. If we say that a machine is not allowed to learn, to read, to extract knowledge from existing content and adapt it to other uses, then I fear it would not be a long leap to declare what we as humans are not allowed to read, see, or know some things. This puts us in the odd position of having to defend the machine's rights so as to protect our own.

I have been staying mostly out of the AI debate these days, except as devil's advocate inside my head to most every arguments I see. However, I must admit that Jarvis's assertion seems on the right trackto me. The LLMs aren't memorizing the text they process so much as breaking it down into little bits and distributing the bits across elements that enable a level of synthesis different from regurgitation. That's why they get so many facts wrong. That sounds a lot like what people do when they read and assimilate new material into their heads. (Sadly, we get a lot of facts wrong, too.)

My support for this assertion rests on something that Jarvis says at the end of his previous bullet:

I am no lawyer but I believe training machines on any content that is lawfully acquired so it can be inspired to produce new content is not a violation of copyright. Note my italics.

I, too, am not a lawyer. Good read from Jarvis, as usual.

I'm not sure what to think of the fact that my bank says it received my money NaN years ago:

At least NaN hasn't show up as my account balance yet! I suppose that if it were the result of an overflow, at least I'd know what it's like to be fabulously wealthy.

(For my non-technical readers, NaN stands for "Not a Number", and is used in computing interpreted as a value that is not defined or not representable. You may be able to imagine why seeing this in a bank transaction would be disconcerting to a programmer!)

The last day of a conference is often a wildcard. If it ends early enough, I can often drive or fly home afterward, which means that I can attend all of the conference activities. If not, I have to decide between departing the next day or cutting out early. When possible, I stay all day. Sometimes, as with StrangeLoop, I can stay all day, skip only the closing keynote, and drive home into the night.

With virtual PyCon, I decided fairly early on that I would not be able to attend most of the last day. This is Sunday, I have family visiting, and work looms on the horizon for Monday. An afternoon talk or two is all I will be able to manage.

Talk 1: A Pythonic Full-Text Search

The idea of this talk was to use common Python tools to implement full-text search of a corpus. It turns out that the talk focused on websites, and thus on tools common to Python web developers: PostgreSQL and Django. It also turns out that django.contrib.postgres provides lots of features for doing search out of the box. This talk showed how to use them.

This was an interesting talk, but not immediately useful to me. I don't work with Django, and I use SQLite more often then PostgreSQL for my personal work. There may be some support for search in django.contrib.sqlite, if it exists, but the speaker said that I'd likely have to implement most of the search functionality myself. Even so, I enjoyed seeing what was possible with modules already available.

Talk 2: Using Python to Help the Unhoused

I thought I was done for the conference, but I decided I could listen in one one more talk while making dinner. This non-technical session sounded interesting:

How a group of volunteers from around the globe use Python to help an NGO in Victoria, BC, Canada to help the unhoused. By building a tool to find social media activity on unhoused in the Capitol Region, the NGO can use a dashboard of results to know where to move their limited resources.

With my attention focused on the Sri Lankan dal with coconut-lime kale in my care, I didn't take detailed notes this time, but I did learn about the existence of Statistics Without Borders, which sounds like a cool public service group that needs to exist in 2023. Otherwise, the project involved scraping Twitter as a source of data about the needs of the homeless in Victoria, and using sentiment analysis to organize the data. Filtering the data to zero in on relevant data was their biggest challenge, as keyword filters passed through many false positives.

At this point, the developers have given their app to the NGO and are looking forward to receiving feedback, so that they can make any improvements that might be needed.

This is a nice project by folks giving back to their community, and a nice way to end the conference.

~~~~~

I was a PyCon first-timer, attending virtually. The conference talks were quite good. Thanks to everyone who organized the conference and created such a complete online experience. I didn't use all of the features available, but what I did use worked well, and the people were great. I ended up with links to several Python projects to try out, a few example scripts for PyScript and Mermaid that I cobbled together after the talks, and lots of syntactic abstractions to explore. Three days well spent.

Another way that attending a virtual conference is like the in-person experience: you can oversleep, or lose track of time. After a long morning of activity before the time-shifted start of Day 2, I took a nap before the 11:45 talk, and...

Talk 1: Python's Syntactic Sugar

Grrr. I missed the first two-thirds of this talk, which I greatly anticipated, but I slept longer than I planned. My body must have needed more than I was giving it.

I saw enough of the talk, though, to know I want to watch the video on YouTube when it shows up. This topic is one of my favorite topics in programming languages: What is the smallest set of features we need to implement the rest of the language? The speaker spent a couple of years implementing various Python features in terms of others, and arrived at a list of only ten that he could not translate away. The rest are sugar. I missed the list at the beginning of the talk, but I gathered a few of its members in the ten minutes I watched: while, raise, and try/except.

I love this kind of exercise: "How can you translate the statement if X: Y into one that uses only core features?" Here's one attempt the speaker gave:

    try:
        while X:
	    Y
	    raise _DONE
    except _DONE:
        None
 

I was today days old when I learned that Python's bool subclasses int, that True == 1, and that False == 0. That bit of knowledge was worth interrupting my nap to catch the end of the talk. Even better, this talk was based on a series of blog posts. Video is okay, but I love to read and play with ideas in written form. This series vaults to the top of my reading list for the coming days.

Talk 2: Subclassing, Composition, Python, and You

Okay, so this guy doesn't like subclasses much. Fair enough, but... some of his concerns seem to be more about the way Python classes work (open borders with their super- and subclasses) than with the idea itself. He showed a lot of ways one can go wrong with arcane uses of Python subclassing, things I've never thought to do with a subclass in my many years doing OO programming. There are plenty of simpler uses of inheritance that are useful and understandable.

Still, I liked this talk, and the speaker. He was honest about his biases, and he clearly cares about programs and programmers. His excitement gave the talk energy. The second half of the talk included a few good design examples, using subclassing and composition together to achieve various ends. It also recommended the book Architecture Patterns with Python. I haven't read a new software patterns book in a while, so I'll give this one a look.

Toward the end, the speaker referenced the line "Software engineering is programming integrated over time." Apparently, this is a Google thing, but it was new to me. Clever. I'll think on it.

Talk 3: How We Are Making CPython Faster -- Past, Present and Future

I did not realize that, until Python 3.11, efforts to make the interpreter had been rather limited. The speaker mentioned one improvement made in 3.7 to optimize the typical method invocation, obj.meth(arg), and one in 3.8 that sped up global variable access by using a cache. There are others, but nothing systematic.

At this point, the talk became mutually recursive with the Friday talk "Inside CPython 3.11's New Specializing, Adaptive Interpreter". The speaker asked us to go watch that talk and return. If I were more ambitious, I'd add a link to that talk now, but I'll let you any of you are interested to visit yesterday's post and scroll down two paragraphs.

He then continued with improvements currently in the works, including:

• efforts to optimize over larger regions, such as the different elements of a function call
• use of partial evaluation when possible
• specialization of code
• efforts to speed up memory management and garbage collection

He also mentions possible improvements related to C extension code, but I didn't catch the substance of this one. The speaker offered the audience a pithy takeaway from his talk: Python is always getting faster. Do the planet a favor and upgrade to the latest version as soon as you can. That's a nice hook.

There was lots of good stuff here. Whenever I hear compiler talks like this, I almost immediately start thinking about how I might work some of the ideas into my compiler course. To do more with optimization, we would have to move faster through construction of a baseline compiler, skipping some or all of the classic material. That's a trade-off I've been reluctant to make, given the course's role in our curriculum as a compilers-for-everyone experience. I remain tempted, though, and open to a different treatment.

Talk 4: The Lost Art of Diagrams: Making Complex Ideas Easy to See with Python

Early on, this talk contained a line that programmers sometimes need to remember: Good documentation shows respect for users. Good diagrams, said the speaker, can improve users' lives. The talk was a nice general introduction to some of the design choices available to us as we create diagrams, including the use of color, shading, and shapes (Venn diagrams, concentric circles, etc.). It then discussed a few tools one can use to generate better diagrams. The one that appealed most to me was Mermaid.js, which uses a Markdown-like syntax that reminded me of GraphViz's Dot language. My students and use GraphViz, so picking up Mermaid might be a comfortable task.

~~~~~

My second day at virtual PyCon confirmed that attending was a good choice. I've seen enough language-specific material to get me thinking new thoughts about my courses, plus a few other topics to broaden the experience. A nice break from the semester's grind.

One of great benefits of a virtual conference is accessibility. I can hop from Iowa to Salt Lake City with the press of a button. The competing cost to virtual conference is that I am accessible ... from elsewhere.

On the first day of PyCon, we had a transfer orientation session that required my presence in virtual Iowa from 10:00 AM-12:00 noon local time. That's 9:00-11:00 Mountain time, so I missed Ned Batchelder's keynote and the opening set of talks. The rest of the day, though, I was at the conference. Virtual giveth, and virtual taketh away.

Talk 1: Inside CPython 3.11's New Specializing, Adaptive Interpreter

As I said yesterday, I don't know Python -- tools, community, or implementation -- intimately. That means I have a lot to learn in any talk. In this one, Brandt Bucher discussed the adaptive interpreter that is part of Python 3.11, in particular how the compiler uses specialization to improve its performance based on run-time usage of the code.

Midway through the talk, he referred us to a talk on tomorrow's schedule. "You'll find that the two talks are not only complementary, they're also mutually recursive." I love the idea of mutually recursive talks! Maybe I should try this with two sessions in one of my courses. To make it fly, I will need to make some videos... I wonder how students would respond?

This online Python disassembler by @pamelafox@fosstodon.org popped up in the chat room. It looks like a neat tool I can use in my compiler course. (Full disclosure: I have been following Pamela on Twitter and Mastodon for a long time. Her posts are always interesting!)

Talk 2: Build Yourself a PyScript

PyScript is a Javascript module that enables you to embed Python in a web page, via WebAssembly. This talk described how PyScript works and showed some of the technical issues in writing web apps.

Some of this talk was over my head. I also do not have deep experience programming in the web. It looks like I will end up teaching a beginning web development course this fall (more later), so I'll definitely be learning more about HTML, CSS, and Javascript soon. That will prepare me to be more productive using tools like PyScript.

Talk 3: Kill All Mutants! (Intro to Mutation Testing)

Our test suites are often not strong enough to recognize changes in our code. The talk introduced mutation testing, which modifies code to test the suite. I didn't take a lot of notes on this one, but I did make a note to try mutation testing out, maybe in Racket.

Talk 4: Working with Time Zones: Everything You Wish You Didn't Need to Know

Dealing with time zones is one of those things that every software engineer seems to complain about. It's a thorny problem with both technical and social dimensions, which makes it really interesting for someone who loves software design to think about.

This talk opened with example after example of how time zones don't behave as straightforwardly as you might think, and then discussed Python's newest time zone library, pytz.

My main takeaways from this talk: pytz looks useful, and I'm glad I don't have to deal with time zones on a regular basis.

Talk 5: Pythonic Functional (iter)tools for your Data Challenges

This is, of course, a topic after my heart. Functional programming is a big part of my programming languages course, and I like being able to show students Python analogues to the Racket ideas they are learning. There was not much new FP content for me here, but I did learn some new Python functions from itertools that I can use in class -- and in my own code. I enjoyed the Advent of Code segment of the talk, in which the speaker applied Python to some of the 2021 challenges. I use an Advent of Code challenge or two each year in class, too. The early days of the month usually feature fun little problems that my students can understand quickly. They know how to solve them imperatively in Python, but we tackle them functionally in Racket.

Most of the FP ideas needed to solve them in Python are similar, so it was fun to see the speaker solve them using itertools. Toward the end, the solutions got heavy quickly, which must be how some of my students feel when we are solving these problems in class.

~~~~~

Between work in the morning and the conference afternoon and evening, this was a long day. I have a lot of new tools to explore.

Last night, I posted on Mastodon:

Heading off to #PyConUS in the morning, virtually. I just took my first tour of Hubilo, the online platform. There's an awful lot going on, but you need a lot of moving parts to produce the varied experiences available in person. I'm glad that people have taken up the challenge.

Why PyCon? I'm not an expert in the language, or as into the language as I once was with Ruby. Long-time readers may recall that I blogged about attending JRubyConf back in 2012. Here is a link to my first post from that conference. You can scroll up from there to see several more posts from the conference.

However, I do write and read a lot of Python code, because our students learn it early and use it quite a bit. Besides, it's a fun language and has a large, welcoming community. Like many language-specific conferences, PyCon includes a decent number of talks about interpreters, compilers, and tools, which are a big part of my teaching portfolio these days. The conference offers a solid virtual experience, too, which makes it attractive to attend while the semester is still going on.

My goals for attending PyCon this year include:

• learning some things that will help me improve my programming languages and compiler courses,
• learning some things that make me a better Python programmer, and
• simply enjoying computer science and programming for a few days, after a long and occasionally tedious year. The work doesn't go away while I am at a conference, but my mind gets to focus on something else -- something I enjoy!

More about today's talks tomorrow.

On Friday, I wrote a note to myself about updating an upcoming class session:

Clean out the old cruft. Simplify, simplify, simplify! I want students to grok the idea and the implementation. All that Lisp and Scheme history is fun for me, but it gets in the students' way.

This is part of an ongoing battle for me. Intellectually, I know to design class sessions and activities focused on where students are and what they need to do in order to learn. Yet it continually happens that I strike upon a good approach for a session, and then over the years I stick a little extra in here and there; within a few iterations I have a big ball of mud. Or I fool myself that some bit of history that I find fascinating is somehow essential for students to learn about, too, so I keep it in a session. Over the years, the history grows more and more distant; the needs of the session evolve, but I keep the old trivia in there, filling up time and distracting my students.

It's hard.

The specific case in question here is a session in my programming languages course called Creating New Syntax. The session has the modest goal of introducing students to the idea of using macros and other tools to define new syntactic constructs in a language. My students come into the course with no Racket or Lisp experience, and only a few have enough experience with C/C++ that they may have seen its textual macros. My plan for this session is to expose them to a few ideas and then to demonstrate one of Racket's wonderful facilities for creating new syntax. Given the other demands of the course, we don't have time to go deep, only to get a taste.

[In my dreams, I sometimes imagine reorienting this part of my course around something like Matthew Butterick's Beautiful Racket... Maybe someday.]

Looking at my notes for this session on Friday, I remembered just how overloaded and distracting the session has become. Over the years, I've pared away the extraneous material on macros in Lisp and C, but now it has evolved to include too many ideas and incomplete examples of macros in Racket. Each by itself might make for a useful part of the story. Together, they pull attention here and there without ever closing the deal.

I feel like the story I've been telling is getting in the way of the one or two key ideas about this topic I want students to walk away from the course with. It's time to clean the session up -- to make some major changes -- and tell a more effective story.

The specific idea I seized upon on Friday is an old idea I've had in mind for a while but never tried: adding a Python-like for-loop:

    (for i in lst: (sqrt i))

[Yes, I know that Racket already has a fine set of for-loops! This is just a simple form that lets my students connect their fondness for Python with the topic at hand.]

This functional loop is equivalent to a Racket map expression:

    (map (lambda (i)
           (sqrt i))
	 lst)

We can write a simple list-to-list translator that converts the loop to an equivalent map:

    (define for-to-map
      (lambda (for-exp)
        (let ((var (second for-exp))
              (lst (fourth for-exp))
              (exp (sixth for-exp)))
          (list 'map
                (list 'lambda (list var) exp)
                lst))))

This code handles only the surface syntax of the new form. To add it to the language, we'd have to recursively translate the form. But this simple function alone demonstrates the idea of translational semantics, and shows just how easy it can be to convert a simple syntactic abstraction into an equivalent core form.

Racket, of course, gives us better options! Here is the same transformer using the syntax-rules operator:

    (define-syntax for-p
      (syntax-rules (in :)
        ( (for-p var in lst : exp)
            (map (lambda (var) exp) lst) )  ))

So easy. So powerful. So clear. And this does more than translate surface syntax in the form of a Racket list; it enables the Racket language processor to expand the expression in place and execute the result:

    > (for-p i in (range 0 10):
        (sqrt i))
    '(0
      1
      1.4142135623730951
      ...
      2.8284271247461903
      3)

This small example demonstrates the key idea about macros and syntax transformers that I want students to take from this session. I plan to open the session with for-p, and then move on to range-case, a more complex operator that demonstrates more of syntax-rules's range and power.

This sets me up for a fun session in a little over a week. I'm excited to see how it plays with students. Renewed simplicity and focus should help.

In this episode of Conversations With Tyler, Cowen asks economist Jeffrey Sachs if he agrees with several other economists' bearish views on a particular issue. Sachs says they "have been wrong ... for 20 years", laughs, and goes on to say:

They just got it wrong time and again. They had failed to understand, and the same with [another economist]. It's the same story. It doesn't fit our model exactly, so it can't happen. It's got to collapse. That's not right. It's happening. That's the story of our time. It's happening.

"It doesn't fit our model, so it can't happen." But it is happening.

When your model keeps saying that something can't happen, but it keeps happening anyway, you may want to reconsider your model. Otherwise, you may miss the dominant story of the era -- not to mention being continually wrong.

Sachs spends much of his time with Cowen emphasizing the importance of context in determining which model to use and which actions to take. This is essential in economics because the world it studies is simply too complex for the models we have now, even the complex models.

I think Sachs's insight applies to any discipline that works with people, including education and software development.

The topic of education even comes up toward the end of the conversation, when Cowen asks Sachs how to "fix graduate education in economics". Sachs says that one of its problems is that they teach econ as if there were "four underlying, natural forces of the social universe" rather than studying the specific context of particular problems.

He goes on to highlight an approach that is affecting every discipline now touched by data analytics:

We have so much statistical machinery to ask the question, "What can you learn from this dataset?" That's the wrong question because the dataset is always a tiny, tiny fraction of what you can know about the problem that you're studying.

Every interesting problem is bigger than any dataset we build from it. The details of the problem matter. Again: context. Sachs suggests that we shouldn't teach econ like physics, with Maxwell's overarching equations, but like biology, with the seemingly arbitrary details of DNA.

In my mind, I immediately began thinking about my discipline. We shouldn't teach software development (or econ!) like pure math. We should teach it as a mix of principles and context, generalities and specific details.

There's almost always a tension in CS programs between timeless knowledge and the details of specific languages, libraries, and tools. Most of students don't go on to become theoretical computer scientists; they go out to work in the world of messy details, details that keep evolving and morphing into something new.

That makes our job harder than teaching math or some sciences because, like economics:

... we're not studying a stable environment. We're studying a changing environment. Whatever we study in depth will be out of date. We're looking at a moving target.

That dynamic environment creates a challenge for those of us teaching software development or any computing as practiced in the world. CS professors have to constantly be moving, so as not to fall our of date. But they also have to try to identify the enduring principles that their students can count on as they go on to work in the world for several decades.

To be honest, that's part of the fun for many of us CS profs. But it's also why so many CS profs can burn out after 15 or 20 years. A never-ending marathon can wear anyone out.

Anyway, I found Cowens' conversation with Jeffrey Sachs to be surprisingly stimulating, both for thinking about economics and for thinking about software.

I finally read Rudolf Winestock's 2011 essay The Lisp Curse, which is summarized in one line:

Lisp is so powerful that problems which are technical issues in other programming languages are social issues in Lisp.

It seems to me that Racket and Clojure have overcome the curse. Racket was built by a small team that grew up in academia. Clojure was designed and created by an individual. Yet they are both 100% solutions, not the sort of one-off 80% personal solutions that tend to plague the Lisp world.

But the creators went further: They also attracted and built communities.

The Racket and Clojure communities consist of programmers who care about the entire ecosystem. The Racket community welcomes and helps newcomers. I don't move in Clojure circles, but I see and hear good things from people who do.

Clojure has made a bigger impact commercially, of course. Offering a high level of performance and running on the JVM have their advantages. I doubt either will ever displace Java or the other commercial behemoths, but they appear to have staying power. They earned that status by solving both technical issues and social issues.

Poet Marvin Bell, in his contribution to the collection Writers on Writing:

The future belongs to the helpless. I am often presented that irresistible question asked by the beginning poet: "Do you think I am any good?" I have learned to reply with a question: "If I say no, are you going to quit?" Because life offers any of us many excuses to quit. If you are going to quit now, you are almost certainly going to quit later. But I have concluded that writers are people who you cannot stop from writing. They are helpless to stop it.

Reading that passage brought to mind Ted Gioia's recent essay on musicians who can't seem to retire. Even after accomplishing much, these artists seem never want to stop doing their thing.

Just before starting Writers on Writing, I finished Kurt Vonnegut's Sucker's Portfolio, a slim 2013 volume of six stories and one essay not previously published. The book ends with an eighth piece: a short story unfinished at the time of Vonnegut's death. The story ends mid-sentence and, according to the book's editor, at the top of an unfinished typewritten page. In his mid-80s, Vonnegut was creating stories t the end.

I wouldn't mind if, when it's my time to go, folks find my laptop open to some fun little programming project I was working on for myself. Programming and writing are not everything there is to my life, but they bring me a measure of joy and satisfaction.

~~~~~

This week was a wonderful confluence of reading the Bell, Gioia, and Vonnegut pieces around the same time. So many connections... not least of which is that Bell and Vonnegut both taught at the Iowa Writers' Workshop.

There's also an odd connection between Vonnegut and the Gioia essay. Gioia used a quip attributed to the Roman epigrammist Martial:

Fortune gives too much to many, but enough to none.

That reminded me of a story Vonnegut told occasionally in his public talks. He and fellow author Joseph Heller were at a party hosted by a billionaire. Vonnegut asked Heller, "How does it make you feel to know that guy made more money yesterday than Catch-22 has made in all the years since it was published?" Heller answered, "I have something he'll never have: the knowledge that I have enough."

There's one final connection here, involving me. Marvin Bell was the keynote speaker at Camouflage: Art, Science & Popular Culture an international conference organized by graphic design prof Roy Behrens at my university and held in April 2006. Participants really did come from all around the world, mostly artists or designers of some sort. Bell read a new poem of his and then spoke of:

the ways in which poetry is like camouflage, how it uses a common vocabulary but requires a second look in order to see what is there.

The other night, I wanted to have some fun, so I implemented a few Racket functions to implement Flavius Josephus's sieve. Think the sieve of Eratosthenes, but with each pass applied to the numbers left after the previous pass:

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ...
    1   3   5   7   9    11    13    15 ...  - after pass 1 (remove every 2nd)
    1   3       7   9          13    15 ...  - after pass 2 (remove every 3rd)
    1   3       7              13    15 ...  - after pass 3 (remove every 4th)
    ...

My code isn't true to the mathematical ideal, because it doesn't process the unbounded sequence of natural numbers. (Haskell, with lazy evaluation, might be a better choice for that!) Instead, it works on a range [1..n] for a given n. That's plenty good enough for me to see how the sequence evolves.

I ended up using a mix of higher-order functions and recursive functions. When it comes to time to filter out every kth item in the previous sequence, I generate a list of booleans with false in every kth position and true elsewhere. For example, '(#t #t #f #t #t #f ...) is the mask for the pass that filters out every third item. Then, this function:

    (define filter-with-mask
      (lambda (lon mask)
        (map car
             (filter (lambda (p) (cdr p))
                     (map cons lon mask)))))

I won't be surprised if my mathematician friends, or readers who are better Racket programmers, can suggest a more elegant way to "strain" a list, but, hey, I'm a programmer. This is a simple enough approach that does the job. The code is reasonably fast and flexible.

In any case, it was a nice way to spend an hour after a long day on campus. My wife just smiled when I told what I was going to do to relax that evening. She knows me well.

Catching up on articles in my newsreader and ran across Commands I Use by @gvwilson. That sounded like fun, and I was game:

    $ history | awk '{print $2}' | sort | uniq -c | sort -nr > commands.txt

The first four items on my list are essentially the same as Wilson's, and there are a lot of other similarities, too. I don't think this is surprising, given how Unix works and how much sense git makes for software developers to use.

• git    - same caveat as Wilson. Next time, I may look at field and flesh this out.
• ll    - my shorthand for ls -al
• emacs    - less of a cheat for me. I mostly edit in emacs.
• cd
• mv
• rmbak    - my shorthand for rm *~, a form of Wilson's clean
• cpsync    - my shorthand for copying a file to a folder for syncing to my office machine
• popd
• pushd    - I pushd and popd a lot...
• open
• dirs    - ... which means occasionally checking the stack
• mvsync    - similar to cpsync but also moves the file (often from the desktop to its permanent home)
• tgz    - a 5-line script that bundles the sync folder used by cpsync and mvsync
• cp
• cls
• pwd
• close-journal.py    - a substantial Python script; part of my homegrown family accounting system
• rm    - aliased to ask for confirmation before deleting
• cat
• /bin/rm    - the built-in command nukes a file with no shame
• more
• xattr
• python3    - same caveat as Wilson
• gzt    - an unbundling script
• mkdir
• gooffice    - my shorthand for sshing into my office machine

It's interesting to see that I use rm and /bin/rm in roughly even measure. I would have guessed that I used the guarded command in higher proportion.

At the bottom of the tally are a few items I don't use often, or don't generally launch from the command line:

• touch
• racket
• npm
• idle3
• chmod

... and a bunch of typos, including:

• pops
• ce
• nv
• nl
• mdir
• emaxs, emavs, emacd,

That was fun! Thanks to Greg for the prompt.

Today I received an email message similar to this:

I didn't do very well in my first semester, so I'm looking for ways to do better this time around. Do you have any ideas about study resources or tips for succeeding in CS courses?

As an advisor, I'm occasionally asked by students for advice of this sort. As department head, I receive even more queries, because early on I am the faculty member students know best, from campus visits and orientation advising.

When such students have already attempted a CS course or two, my first step is always to learn more about their situation. That way, I can offer suggestions suited to their specific needs.

Sometimes, though, the request comes from a high school student, or a high school student's parent: What is the best way to succeed as a CS student?

To be honest, most of the advice I give is not specific to a computer science major. At a first approximation, what it takes to succeed as a CS student is the same as what it takes to succeed as a student in any major: show up and do the work. But there are a few things a CS student does that are discipline-specific, most of which involve the tools we use.

I've decided to put together a list of suggestions that I can point our students to, and to which I can refer occasionally in order to refresh my mind. My advice usually includes one or all of these suggestions, with a focus on students at the beginning of our program:

• Go to every class and every lab session. This is #0 because it should go without saying, but sometimes saying it helps. Students don't always have to go to their other courses every day in order to succeed.


• Work steadily on a course. Do a little work on your course, both programming and reading or study, frequently -- every day, if possible. This gives your brain a chance to see patterns more often and learn more effectively. Cramming may help you pass a test, but it doesn't usually help you learn how to program or make software.


• Ask your professor questions sooner rather than later. Send email. Visit office hours. This way, you get answers sooner and don't end up spinning your wheels while doing homework. Even worse, feeling confused can lead you to shying away from doing the work, which gets in the way of #1.


• Get to know your programming environment. When programming in Python, simply feeling comfortable with IDLE, and with the file system where you store your programs and data, can make everything else seem easier. Your mind doesn't have to look up basic actions or worry about details, which enables you to make the most of your programming time: working on the assigned task.


• Spend some of your study time with IDLE open. Even when you aren't writing a program, the REPL can help you! It lets you try out snippets of code from your reading, to see them work. You can run small experiments of your own, to see whether you understand syntax and operators correctly. You can make up your own examples to fill in the gaps in your understanding of the problem.
  
  Getting used to trying things out in the interactions window can be a huge asset. This is one of the touchstones of being a successful CS student.

That's what came to mind at the end of a Friday, at the end of a long week, when I sat down to give advice to one student. I'd love to hear your suggestions for improving the suggestions in my list, or other bits of advice that would help our students. Email me your ideas, and I'll make my list better for anyone who cares to read it.

Cory Doctorow, in a recent New Yorker interview reminisces about learning to program. The family had a teletype and modem.

My mom was a kindergarten teacher at the time, and she would bring home rolls of brown bathroom hand towels from the kid's bathroom at school, and we would feed a thousand feet of paper towel into the teletype and I would get a thousand feet of computing after school at the end of the day.

Two things:

• Tsk, tsk, Mom. Absconding with school supplies, even if for a noble cause! :-) Fortunately, the statute of limitations on pilfering paper hand towels has likely long since passed.


• I love the idea of doing "a thousand feet of computing" each day. What a wonderful phrase. With no monitor, the teletype churns out paper for every line of code, and every line the code produces. You know what they say: A thousand feet a day makes a happy programmer.

The entire interview is a good read on the role of computing in modern society. The programmer in me also resonated with this quote from Doctorow's 2008 novel, Little Brother:

If you've never programmed a computer, there's nothing like it in the whole world. It's awesome in the truest sense: it can fill you with awe.

My older daughter recommended Little Brother to me when it first came out. I read many of her recommendations promptly, but for some reason this one sits on my shelf unread. (The PDF also sits in my to-read/ folder, unread.) I'll move it to the top of my list.

Ethan Mollick tells us how to generate prompts for programs like ChatGPT and DALL-E: give direct and detailed instructions.

Don't ask it to write an essay about how human error causes catastrophes. The AI will come up with a boring and straightforward piece that does the minimum possible to satisfy your simple demand. Instead, remember you are the expert and the AI is a tool to help you write. You should push it in the direction you want. For example, provide clear bullet points to your argument: write an essay with the following points: -Humans are prone to error -Most errors are not that important -In complex systems, some errors are catastrophic -Catastrophes cannot be avoided

But even the results from such a prompt are much less interesting than if we give a more explicit prompt. Fo instance, we might add:

use an academic tone. use at least one clear example. make it concise. write for a well-informed audience. use a style like the New Yorker. make it at least 7 paragraphs. vary the language in each one. end with an ominous note.

This reminds me of setting essay topics for students, either for long-form writing or for exams. If you give a bland uninteresting question, you will generally get a bland uninteresting answer. Such essays are hard to evaluate. A squooshy question allows the student to write almost anything in response. Students are usually unhappy in this scenario, too, because they don't know what you want them to write, or how they will be evaluated.

Asking a human a more specific question has downsides, though. It increases the cognitive load placed on them, because there are more things for them to be thinking about as they write. Is my tone right? Does this sound like the New Yorker? Did I produce the correct number of paragraphs? Is my essay factually accurate? (ChatGPT doesn't seem to worry about this one...) The tradeoff is clearer expectations. Many students prefer this trade, at least on longer form assignments when they have time to consider the specific requests. A good spec reduces uncertainty.

Maybe these AI programs are closer to human than we think after all. (Some people don't worry much about correctness either.)

~~~~

On a related note: As I wrote elsewhere, I refuse to call ChatGPT or any program "an AI". The phrase "artificial intelligence" is not a term for a specific agent; it is the name of an idea. Besides, none of these programs are I yet, only A.

Someone on Mastodon posted a link to a 2021 survey of how the parsers for major languages are implemented. Are they written by hand, or automated by a parser generator? The answer was mixed: a few are generated by yacc-like tools (some of which were custom built), but many are written by hand, often for speed.

My two favorite notes:

Julia's parser is handwritten but not in Julia. It's in Scheme!

Good for the Julia team. Scheme is a fine language in which to write -- and maintain -- a parser.

Not only [is Clang's parser] handwritten but the same file handles parsing C, Objective-C and C++.

I haven't clicked through to the source code for Clang yet but, wow, that must be some file.

Finally, this closing comment in the paper hit close to the heart:

Although parser generators are still used in major language implementations, maybe it's time for universities to start teaching handwritten parsing?

I have persisted in having my compiler students write table-driven parsers by hand for over fifteen years. As I noted in this post at the beginning of the 2021 semester, my course is compilers for everyone in our major, or potentially so. Most of our students will not write another compiler in their careers, and traditional skills like implementing recursive descent and building a table-driven program are valuable to them more generally than knowing yacc or bison. Any of my compiler students who do eventually want to use a parser generator are well prepared to learn how, and they'll understand what's going on when they do, to boot.

My course is so old school that it's back to the forefront. I just had to be patient.

(I posted the seeds of this entry on Mastodon. Feel free to comment there!)

Yesterday, Ben Fulton posted on Mastodon:

TIL: C++ has a mismatch algorithm that returns the first non-equal pair of elements from two sequences. ...

C++'s mismatch was new to me, too, so I clicked through to the spec on cppreference.com to read a bit more. I learned that mismatch is an algorithm implemented as as a template function with several different signatures. My thoughts turned immediately to my spring course, Programming Languages, which starts with an introduction to Racket and functional programming. mismatch would make a great example or homework problem for my students, as they learn to work with Racket lists and functions! I stopped working on what I was doing and used the C++ spec to draw up a family of functions for my course:

    ; Return the first mismatching pair of elements from two lists.
    ; Compare using eq?.
    ;   (mismatch lst1 lst2)
    ;
    ; Compare using a given binary predicate comp?.
    ;   (mismatch comp? lst1 lst2)
    ;
    ; Compare using a given binary predicate comp?,
    ; as a higher-order function.
    ;   ((make-mismatch comp?) lst1 lst2)
    ;
    ; Return the first mismatching pair of elements from two ranges,
    ; also as a higher-order function.
    ; If last2 is not provided, it denotes first2 + (last1 - first1).
    ;   (make-mismatch first1 last1 first2 [last2]) -> (f lst1 lst2)

Of course, this list is not exhaustive, only a start. With so many related possibilities, mismatch will make a great family of examples or homework problems for the course! What a fun distraction from the other work in my backlog.

Ben's post conveniently arrived in the middle of an email discussion with the folks who teach our intro course, about ChatGPT and the role it will play in Intro. I mentioned ChatGPT in a recent post suggesting that we all think about tools like ChatGPT and DALL-E from the perspective of cultural adaptation: how do we live with new AI tools knowing that we change our world to accommodate our technologies? In that post, I mentioned only briefly the effect that these tools will have on professors, their homework assignments, and the way we evaluate student competencies and performance. The team preparing to teach Intro this spring has to focus on these implications now because they affect how the course will work. Do we want to mitigate the effects of ChatGPT and, if so, how?

I think they have decided mostly to take a wait-and-see approach this semester. We always have a couple of students who do not write their own code, and ChatGPT offers them a new way not to do so. When we think students have not written the code they submitted, we talk with them. In particular, we discuss the code and ask the student to explain or reason about it.

Unless the presence of ChatGPT greatly increases the number of students submitting code they didn't write, this approach should continue to work. I imagine we will be fine. Most students want to learn; they know that writing code is where they learn the most. I don't expect that access to ChatGPT is going to change the number of students taking shortcuts, at least not in large numbers. Let's trust our students as we keep a watchful eye out for changes in behavior.

The connection between mismatch and the conversation about teaching lies in the role that a family of related functions such as mismatch can play in building a course that is more resistant to the use of AI assistants in a way that harms student learning. I already use families of related function specs as a teaching tool in my courses, for purely pedagogical reasons. Writing different versions of the same function, or seeing related functions used to solve slightly different problems, is a good way to help students deepen understanding of an idea or to help them make connections among different concepts. My mismatches give me another way to help students in Programming Languages learn about processing lists, passing functions as arguments, returning functions as values, and accepting a variable number of arguments. I'm curious to see how this family of functions works for students.

A set of related functions also offers a tool both for helping professors determine whether students have learned to write code. We already ask students in our intro course to modify code. Asking students to convert a function with one spec into a function with a slightly different spec, like writing different versions of the same function, give them the chance benefit from their understanding the existing code. It is easier for a programmer to modify a function if they understand it. The existing code is a scaffold that enables the student to focus on the single feature or concept they need to write the new code.

Students who have not written code like the code they are modifying have a harder time reading and modifying the given code, especially when operating under any time or resource limitation. In a way, code modification exercises do something simpler to asking students to explain code to us: the modification task exposes when students don't understand code they claim to have written.

Having ChatGPT generate a function for you won't be as valuable if you will soon be asked to explain the code in detail or to modify the code in a way that requires you understand it. Increasing the use of modification tasks is one way to mitigate the benefits of a student having someone else write the code for them. Families of functions such as mismatch above are a natural source of modification tasks.

Beyond the coming semester, I am curious how our thinking about writing code will evolve in the presence of ChatGPT-like tools. Consider the example of auto-complete facilities in our editors. Few people these days think of using auto-complete as cheating, but when it first came out many professors were concerned that using auto-complete was a way for students not to learn function signatures and the details of standard libraries. (I'm old enough to still have a seed of doubt about auto-complete buried somewhere deep in my mind! But that's just me.)

If LLM-based tools become the new auto-complete, one level up from function signatures, then how we think about programming will probably change. Likewise how we think about teaching programming... or not. Did we change how we teach much as a result of auto-complete?

The existence of ChatGPT is a bit disconcerting for today's profs, but the long-term implications are kind of interesting.

In the meantime, coming across example generators like C++'s mismatch helps me deal with the new challenge and gives me unexpected fun writing code and problem descriptions.

My social media feeds are full of ChatGPT screenshots and speculation these days, as they have been with LLMs and DALL-E and other machine learning-based tools for many months. People wonder what these tools will mean for writers, students, teachers, artists, and anyone who produces ordinary text, programs, and art.

These are natural concerns, given their effect on real people right now. But if you consider the history of human technology, they miss a bigger picture. Technologies often eliminate the need for a certain form of human labor, but they just as often create a new form of human labor. And sometimes, they increase the demand for the old kind of labor! If we come to rely on LLMs to generate text for us, where will we get the text with which to train them? Maybe we'll need people to write even more replacement-level prose and code!

As Robin Sloan reminds us in the latest edition of his newsletter, A Year of New Avenues, we redesign the world to fit the technologies we create and adopt.

Likewise, here's a lesson from my work making olive oil. In most places, the olive harvest is mechanized, but that's only possible because olive groves have been replanted to fit the shape of the harvesting machines. A grove planted for machine harvesting looks nothing like a grove planted for human harvesting.

Which means that our attention should be on how programs like GPT-2 might lead us to redesign the world we live and work in better to accommodate these new tools:

For me, the interesting questions sound more like

• What new or expanded kinds of human labor might AI systems demand?
• What entirely new activities do they suggest?
• How will the world now be reshaped to fit their needs?

That last question will, on the timescale of decades, turn out to be the most consequential, by far. Think of cars ... and of how dutifully humans have engineered a world just for them, at our own great expense. What will be the equivalent, for AI, of the gas station, the six-lane highway, the parking lot?

Many professors worry that ChatGPT makes their homework assignments and grading rubrics obsolete, which is a natural concern in the short run. I'm old enough that I may not live to work in a world with the AI equivalent of the gas station, so maybe that world seems too far in the future to be my main concern. But the really interesting questions for us to ask now revolve around how tools such as these will lead us to redesign our worlds to accommodate and even serve them.

Perhaps, with a little thought and a little collaboration, we can avoid engineering a world for them at our own great expense. How might we benefit from the good things that our new AI technologies can provide us while sidestepping some of the highest costs of, say, the auto-centric world we built? Trying to answer that question is a better long-term use of our time and energy that fretting about our "Hello, world!" assignments and advertising copy.

In the essay "On Societies as Organisms", Lewis Thomas says that we "violate science" when we try to read human meaning into the structures and behaviors of insects. But it's hard not to:

Ants are so much like human beings as to be an embarrassment. They farm fungi, raise aphids as livestock, launch armies into wars, use chemical sprays to alarm and confuse enemies, capture slaves. The families of weaver ants engage in child labor, holding their larvae like shuttles to spin out the thread that sews the leaves together for their fungus gardens. They exchange information ceaselessly. They do everything but watch television.

I'm not sure if humans should be embarrassed for still imitating some of the less savory behaviors of insects, or if ants should be embarrassed for reflecting some of the less savory behaviors of humans.

Biology has never been my forte, so I've read and learned less about it than many other sciences. Enjoying chemistry a bit at least helped keep me within range of the life sciences. I was fortunate to grow up in the Digital Age.

But with many people thinking the 21st century will the Age of Biology, I feel like I should get more in tune with the times. I picked up Thomas's now classic The Lives of a Cell, in which the quoted essay appears, as a brief foray into biological thinking about the world. I'm only a few pages in, but it is striking a chord. I can imagine so many parallels with computing and software. Perhaps I can be as at home in the 21st century as I was in the 20th.

I've been catching up on some items in my newsreader that went unread last summer while I rode my bike outdoors rather than inside. This passage from a blog post by Fred Wilson at AVC touched on a personal habit I've been working on:

I can't imagine an effective exec team that isn't in person together at least once a month.

I sometimes fall into a habit of saying or thinking "I can't imagine...". I'm trying to break that habit.

I don't mean to pick on Wilson, whose short posts I enjoy for insight into the world of venture capital. "I can't imagine" is a common trope in both spoken and written English. Some writers use it as a rhetorical device, not as a literal expression. Maybe he meant it that way, too.

For a while now, though, I've been trying to catch myself whenever I say or think "I can't imagine...". Usually my mind is simply being lazy, or too quick to judge how other people think or act.

It turns out that I usually can imagine, if I try. Trying to imagine how that thinking or behavior makes sense helps me see what other people might be thinking, what their assumptions or first principles are. Even when I end up remaining firm in my own way of thinking, trying to imagine usually puts me in a better position to work with the other person, or explain my own reasoning to them more effectively.

Trying to imagine can also give me insight into the limits of my own thinking. What assumptions am I making that lead me to have confidence in my position? Are those assumptions true? If yes, when might they not to be true? If no, how do I need to update my thinking to align with reality?

When I hear someone say, "I can't imagine..." I often think of Russell and Norvig's textbook Artificial Intelligence: A Modern Approach, which I used for many years in class [1]. At the end of one of the early chapters, I think, they mention critics of artificial intelligence who can't imagine the field of AI ever accomplishing a particular goal. They respond cheekily to the effect, This says less about AI than it says about the critics' lack of imagination. I don't think I'd ever seen a textbook dunk on anyone before, and as a young prof and open-minded AI researcher, I very much enjoyed that line [2].

Instead of saying "I can't imagine...", I am trying to imagine. I'm usually better off for the effort.

~~~~

[1] The Russell and Norvig text first came out in 1995. I wonder if the subtitle "A Modern Approach" is still accurate... Maybe theirs is now a classical approach!

[2] I'll have to track that passage down when I am back in my regular office and have access to my books. (We are in temporary digs this fall due to construction.) I wonder if AI has accomplished the criticized goal in the time since Russell and Norvig published their book. AI has reached heights in recent years that many critics in 1995 could not imagine. I certainly didn't imagine a computer program defeating a human expert at Go in my lifetime, let alone learning to do so almost from scratch! (I wrote about AlphaGo and its intersection with my ideas about AI a few times over the years: [ 01/2016 | 03/2016 | 05/2017 | 05/2018 ].)

This weekend, I learned that Kathleen Booth, a British mathematician and computer scientist, invented assembly language. An October 29 obituary reported that Booth died on September 29 at the age of 100. By 1950, when she received her PhD in applied mathematics from the University of London, she had already collaborated on building at least two early digital computers. But her contributions weren't limited to hardware:

As well as building the hardware for the first machines, she wrote all the software for the ARC2 and SEC machines, in the process inventing what she called "Contracted Notation" and would later be known as assembly language.

Her 1958 book, Programming for an Automatic Digital Calculator, may have been the first one on programming written by a woman.

I love the phrase "Contracted Notation".

Thanks to several people in my Twitter feed for sharing this link. Here's hoping that Twitter doesn't become uninhabitable, or that a viable alternative arises; otherwise, I'm going to miss out on a whole lotta learning.

I've been meaning to write a post about my fall compilers course since the beginning of the semester but never managed to set aside time to do anything more than jot down a few notes. Now we are at the end of Week 9 and I just must write. Long-time readers know what motivates me most: a fun program to write in my student's source language!

TFW you run across a puzzle and all you want to do now is write a program to solve it. And teach your students about the process.
-- https://twitter.com/wallingf/status/1583841233536884737

Yesterday, it wasn't a puzzle so much as discovering a new kind of number, Carmichael numbers. Of course, I didn't discover them (neither did Carmichael, though); I learned of them from a Quanta article about a recent proof about these numbers that masquerade as primes. One way of defining this set comes from Korselt:

A positive composite integer n is a Carmichael number if and only if it has multiple prime divisors, no prime divisor repeats, and for each prime divisor p, p-1 divides n-1.

This definition is relatively straightforward, and I quickly imagined am imperative solution with a loop and a list. The challenge of writing a program to verify a number is a Carmichael number in my compiler course's source language is that it has neither of these things. It has no data structures or even local variables; only basic integer and boolean arithmetic, if expressions, and function calls.

Challenge accepted. I've written many times over the years about the languages I ask my students to write compilers for and about my adventures programming in them, from Twine last year through Flair a few years ago to a recurring favorite, Klein, which features prominently in popular posts about Farey sequences and excellent numbers.

This year, I created a new language, Graphene, for my students. It is essentially a small functional subset of Google's new language Carbon. But like it's predecessors, it is something of an integer assembly language, fully capable of working with statements about integers and primes. Korselt's description of Carmichael numbers is right in Graphene's sweet spot.

As I wrote in the post about Klein and excellent numbers, my standard procedure in cases like this is to first write a reference program in Python using only features available in Graphene. I must do this if I hope to debug and test my algorithm, because we do not have a working Graphene compiler yet! (I'm writing my compiler in parallel with my students, which is one of the key subjects in my phantom unwritten post.) I was proud this time to write my reference program in a Graphene-like subset of Python from scratch. Usually I write a Pythonic solution, using loops and variables, as a way to think through the problem, and then massage that code down to a program using a limited set of concepts. This time, I started with short procedural outline:

    # walk up the primes to n
    #   - find a prime divisor p:
    #     - test if a repeat         (yes: fail)
    #     - test if p-1 divides n-1  (no : fail)
    # return # prime divisors > 1

As I tested the code, it occurred to me that my students have a chance to one-up standard Python. Its rather short standard stack depth prevented my program from reaching even n=1000. When I set sys.setrecursionlimit(5000), my program found the first five Carmichael numbers: 561, 1105, 1729, 2465, and 2821. Next come 6601 and 8911; I'll need a lot more stack frames to get there.

All those stack frames are unnecessary, though. My main "looping" function is beautifully tail recursive: two failure cases, the final answer case checking the number of prime divisors, and two tail-recursive calls that move either to the next prime as potential factor or to the next quotient when we find one. If my students implement proper tail calls -- an optimization that is optional in the course but encouraged by their instructor with gusto -- then their compiler will enable us to solve for values up to the maximum integer in the language, 2³¹-1. We'll be limited only by the speed of the target language's VM, and the speed of the target code the compiler generates. I'm pretty excited.

Now I have to resist the urge to regale my students with this entire story, and with more details about how I approach programming in a language like Graphene. I love to talk shop with students about design and programming, but our time is limited... My students are already plenty busy writing the compiler that I need to run my program!

This lark resulted in an hour or so writing code in Python, a few more minutes porting to Graphene, and an enjoyable hour writing this blog post. As the song says, it was a good day.

In The Orchid Thief, there is a passage where author Susan Orlean describes a drive across south Florida on her way to a state preserve, where she'll be meeting an orchid hunter. She ends the passage this way:

The land was marble-smooth and it rolled without a pucker to the horizon. My eyes grazed across the green band of ground and the blue bowl of sky and then lingered on a dead tire, a bird in flight, an old fence, a rusted barrel. Hardly any cars came toward me, and I saw no one in the rearview mirror the entire time. I passed so many vacant acres and looked past them to so many more vacant acres and looked ahead and behind at the empty road and up at the empty sky; the sheer bigness of the world made me feel lonely to the bone. The world is so huge that people are always getting lost in it. There are too many ideas and things and people, too many directions to go. I was starting to believe that the reason it matters to care passionately about something is that it whittles the world down to a more manageable size. It makes the world seem not huge and empty but full of possibility. If I had been an orchid hunter I wouldn't have see this space as sad-making and vacant--I think I would have seen it as acres of opportunity where the things I loved were waiting to be found.

John Laroche, the orchid hunter at the center of The Orchid Thief, comes off as obsessive, but I think many of us know that condition. We have found an idea or a question or a problem that grabs our attention, and we work on it for years. Sometimes, we'll follow a lead so far down a tunnel that it feels a lot like the swamps Laroche braves in search of the ghost orchid.

Even a field like computer science is big enough that it can feel imposing if a person doesn't have a specific something to focus their attention and energy on. That something doesn't have to be forever... Just as Laroche had cycled through a half-dozen obsessions before turning his energy to orchids, a computer scientist can work deeply in an area for a while and then move onto something else. Sometimes, there is a natural evolution in the problems one focuses on, while other times people choose to move into a completely different sub-area. I see a lot of people moving into machine learning these days, exploring how it can change the sub-field they used to focus exclusively on.

As a prof, I am fortunate to be able to work with young adults as they take their first steps in computer science. I get to watch many of them find a question they want to answer, a problem they want to work on for a few years, or an area they want to explore in depth until they master it. It's also sad, in a way, to work with a student who never quite finds something that sparks their imagination. A career in software, or anything, really, can look as huge and empty as Orlean's drive through south Florida if someone doesn't care deeply about something. When they do, the world seems not huge and empty, but full of possibility.

I'm about halfway through The Orchid Thief and am quite enjoying it.

Chad Orzel, getting ready to begin his 22nd year as a faculty member at Union College, muses:

It's really hard to see myself in the "grizzled veteran" class of faculty, though realistically, I'm very much one of the old folks these days. I am to a new faculty member starting this year as someone hired in 1980 would've been to me when I started, and just typing that out makes me want to crumble into dust.

I'm not the sort who likes to one-up another blogger, but... I can top this, and crumble into a bigger, or at least older, pile of dust.

In May, I finished my 30th year as a faculty member. I am as old to a 2022 hire as someone hired in 1962 would have been to me. Being in computer science, rather than physics or another of the disciplines older than CS, this is an even bigger gap culturally than it first appears. The first Computer Science department in the US was created in 1962. In 1992, my colleagues who started in the 1970s seemed pretty firmly in the old guard, and the one CS faculty member from the 1960s had just retired, opening the line into which I was hired.

Indeed, our Department of Computer Science only came into existence in 1992. Prior to that, the CS faculty had been offering CS degrees for a little over a decade as part of a combined department with Mathematics. (Our department even has a few distinguished alums who graduated pre-1981, with CS degrees that are actually Math degrees with a "computation emphasis".) A new department head and I were hired for the department's first year as a standalone entity, and then we hired two more faculty the next year to flesh out our offerings.

So, yeah. I know what Chad means when he says "just typing that out makes me want to crumble into dust", and then some.

On the other hand, it's kind of cool to see how far computer science has come as an academic discipline in the last thirty years. It's also cool that I am still be excited to learn new things and to work with students as they learn them, too.

Morgan Housel's recent piece on experts trying too hard includes a sentence that made me think of what I teach:

A doctor once told me the biggest thing they don't teach in medical school is the difference between medicine and being a doctor — medicine is a biological science, while being a doctor is often a social skill of managing expectations, understanding the insurance system, communicating effectively, and so on.

Most of the grads from our CS program go on to be software developers or to work in software-adjacent jobs like database administrator or web developer. Most of the rest work in system administration or networks. Few go on to be academic computer scientists. As Housel's doctor knows about medicine, there is a difference between academic computer science and being a software developer.

The good news is, I think the CS profs at many schools are aware of this, and the schools have developed computer science programs that at least nod at the difference in their coursework. The CS program at my school has a course on software engineering that is more practical than theoretical, and another short course that teaches practical tools such as version control, automated testing, and build tools, and skills such as writing documentation. All of our CS majors complete a large project course in which students work in teams to create a piece of software or a secure system, and the practical task of working as a team to deliver a piece of working software is a focus of the course. On top of those courses, I think most of our profs try to keep their courses real for students they know will want to apply what they learn in Algorithms, say, or Artificial Intelligence in their jobs as developers.

Even so, there is always a tension in classes between building foundational knowledge and building practical skills. I encounter this tension in both Programming Languages and Translation of Programming Languages. There are a lot of cool things we could learn about type theory, some of which might turn out to be quite useful in a forty-year career as a developer. But any time we devote to going deeper on type theory is time we can't devote to the concrete languages skills of a software developer, such as learning and creating APIs or the usability of programming languages.

So, we CS profs have to make design trade-offs in our courses as we try to balance the forces of students learning computer science and students becoming software developers. Fortunately, we learn a little bit about recognizing, assessing, and making trade-offs in our work both as computer scientists and as programmers. That doesn't make it easy, but at least we have some experience for thinking about the challenge.

The sentence quoted above reminds me that other disciplines face a similar challenge. Knowing computer science is different from being a software developer, or sysadmin. Knowing medicine is different from being a doctor. And, as Housel explains so well in his own work, knowing finance is different from being an investor, which is usually more about psychology and self-awareness than it is about net present value or alpha ratios. (The current stock market is a rather harsh reminder of that.)

Thanks to Housel for the prompt. The main theme of his piece — that experts makes mistakes that novices can't make, which leads to occasional unexpected outcomes — is the topic for another post.

In my previous post, I wrote joyously of a fun bit of programming: implementing ordered pairs using sets.

Alas, there was a bug in my solution. Thanks to Carl Friedrich Bolz-Tereick for finding it so quickly:

Heh, this is fun, great post! I wonder what happens though if a = b? Then the set is {{a}}. first should still work, but would second fail, because the set difference returns the empty set?

Carl Friedrich had found a hole in my small set of tests, which sufficed for my other implementations because the data structures I used separate cells for the first and second parts of the pair. A set will do that only if the first and second parts are different!

Obviously, enforcing a != b is unacceptable. My first code thought was to guard second()'s behavior:

    if my formula finds a result
       then return that result
       else return (first p)

This feels like a programming hack. Furthermore, it results in an impure implementation: it uses a boolean value and an if expression. But it does seem to work. That would have to be good enough unless I could find a better solution.

Perhaps I could use a different representation of the pair. Helpfully, Carl Friedrich followed up with pointers to several blog posts by Mark Dominus from last November that looked at the set encoding of ordered pairs in some depth. One of those posts taught me about another possibility: Wiener pairs. The idea is this:

    (a,b) = { {{a},∅}, {{b}} }

Dominus shows how Wiener pairs solve the a == b edge case in Kuratowski pairs, which makes it a viable alternative.

Would I ever have stumbled upon this representation, as I did onto the Kuratowski pairs? I don't think so. The representation is more complex, with higher-order sets. Even worse for my implementation, the formulas for first() and second() are much more complex. That makes it a lot less attractive to me, even if I never want to show this code to my students. I myself like to have a solid feel for the code I write, and this is still at the fringe of my understanding.

Fortunately, as I read more of Dominus's posts, I found there might be a way to save my Kuratowski-style solution. It turns out that the if expression I wrote above parallels the set logic used to implement a second() accessor for Kuratowski pairs: a choice between the set that works for a != b pairs and a fallback to a one-set solution.

From this Dominus post, we see the correct set expression for second() is:

... which can be simplified to:

The latter expression is useful for reasoning about second(), but it doesn't help me implement the function using set operations. I finally figured out what the former equation was saying: if (∪ p) is same as (∩ p), then the answer comes from (∩ p); otherwise, it comes from their difference.

I realized then that I could not write this function purely in terms of set operations. The computation requires the logic used to make this choice. I don't know where the boundary lies between pure set theory and the logic in the set comprehension, but a choice based on a set-empty? test is essential.

In any case, I think I can implement the my understanding of the set expression for second() as follows. If we define union-minus-intersection as:

    (set-minus (apply set-union aSet)
               (apply set-intersect aSet))

    (second p) = (if (set-empty? union-minus-intersection)
                     (set-elem (apply set-intersect aSet))
                     (set-elem union-minus-intersection))

The then clause is the same as the body of first(), which must be true: if the union of the sets is the same as their intersection, then the answer comes from the interesection, just as first()'s answer does.

It turns out that this solution essentially implements my first code idea above: if my formula from the previous blog entry finds a result, then return that result. Otherwise, return first(p). The circle closes.

Success! Or, I should: Success!(?) After having a bug in my original solution, I need to stay humble. But I think this does it. It passes all of my original tests as well as tests for a == b, which is the main edge case in all the discussions I have now read about set implementations of pairs. Here is a link to the final code file, if you'd like to check it out. I include the two simple test scenarios, for both a == b and a == b, as Rackunit tests.

So, all in all, this was a very good week. I got to have some fun programming, twice. I learned some set theory, with help from a colleague on Twitter. I was also reacquainted with Mark Dominus's blog, the RSS feed for which I had somehow lost track of. I am glad to have it back in my newsreader.

This experience highlights one of the selfish reasons I like for students to ask questions in class. Sometimes, they lead to learning and enjoyment for me as well. (Thanks, Henry!) It also highlights one of the reasons I like Twitter. The friends we make there participate in our learning and enjoyment. (Thanks, Carl Friedrich!)

Yesterday's session of my course was a quiz preceded by a fun little introduction to data abstraction. As part of that, I use a common exercise. First, I define a simple API for ordered pairs: (make-pair a b), (first p), and (second p). Then I ask students to brainstorm all the ways that they could implement the API in Racket.

They usually have no trouble thinking of the data structures they've been using all semester. Pairs, sure. Lists, yes. Does Racket have a hash type? Yes. I remind my students about vectors, which we have not used much this semester. Most of them haven't programmed in a language with records yet, so I tell them about structs in C and show them Racket's struct type. This example has the added benefit of seeing that Racket generates constructor and accessor functions that do the work for us.

The big payoff, of course, comes when I ask them about using a Racket function -- the data type we have used most this semester -- to implement a pair. I demonstrate three possibilities: a selector function (which also uses booleans), message passaging (which also uses symbols), and pure functions. Most students look on these solutions, especially the one using pure functions, with amazement. I could see a couple of them actively puzzling through the code. That is one measure of success for the session.

This year, one student asked if we could use sets to implement a pair. Yes, of course, but I had never tried that... Challenge accepted!

While the students took their quiz, I set myself a problem.

The first question is how to represent the pair. (make-pair a b) could return {a, b}, but with no order we can't know which is first and which is second. My students and I had just looked at a selector function, (if arg a b), which can distinguish between the first item and the second. Maybe my pair-as-set could know which item is first, and we could figure out the second is the other.

So my next idea was (make-pair a b) → {{a, b}, a}. Now the pair knows both items, as well as which comes first, but I have a type problem. I can't apply set operations to all members of the set and will have to test the type of every item I pull from the set. This led me to try (make-pair a b) → {{a}, {a, b}}. What now?

My first attempt at writing (first p) and (second p) started to blow up into a lot of code. Our set implementation provides a way to iterate over the members of a set using accessors named set-elem and set-rest. In fine imperative fashion, I used them to start whacking out a solution. But the growing complexity of the code made clear to me that I was programming around sets, but not with sets.

When teaching functional programming style this semester, I have been trying a new tactic. Whenever we face a problem, I ask the students, "What function can help us here?" I decided to practice what I was preaching.

Given p = {{a}, {a, b}}, what function can help me compute the first member of the pair, a? Intersection! I just need to retrieve the singleton member of ∩ p:

    (first p) = (set-elem (apply set-intersect p))

What function can help me compute the second member of the pair, b? This is a bit trickier... I can use set subtraction, {a, b} - {a}, but I don't know which element in my set is {a, b} and which is {a}. Serendipitously, I just solved the latter subproblem with intersection.

Which function can help me compute {a, b} from p? Union! Now I have a clear path forward: (∪ p) – (∩ p):

    (second p) = (set-elem
                   (set-minus (apply set-union p)
                              (apply set-intersect p)))

I implemented these three functions, ran the tests I'd been using for my other implementations... and they passed. I'm not a set theorist, so I was not prepared to prove my solution correct, but the tests going all green gave me confidence in my new implementation.

Last night, I glanced at the web to see what other programmers had done for this problem. I didn't run across any code, but I did find a question and answer on the mathematics StackExchange that discusses the set theory behind the problem. The answer refers to something called "the Kuratowski definition", which resembles my solution. Actually, I should say that my solution resembles this definition, which is an established part of set theory. From the StackExchange page, I learned that there are other ways to express a pair as a set, though the alternatives look much more complex. I didn't know the set theory but stumbled onto something that works.

My solution is short and elegant. Admittedly, I stumbled into it, but at least I was using the tools and thinking patterns that I've been encouraging my students to use.

I'll admit that I am a little proud of myself. Please indulge me! Department heads don't get to solve interesting problems like this during most of the workday. Besides, in administration, "elegant" and "clever" solutions usually backfire.

I'm guessing that most of my students will be unimpressed, though they can enjoy my pleasure vicariously. Perhaps the ones who were actively puzzling through the pure-function code will appreciate this solution, too. And I hope that all of my students can at least see that the tools and skills they are learning will serve them well when they run into a problem that looks daunting.

Wordle has been the popular game du jour for a while now. Whenever this sort of thing happens, CS profs think, "How can I turn this into an assignment?" I've seen a lot of discussion of ways to create a Wordle knock-off assignment for CS 1. None of this conversation has interested me much. First, I rarely teach CS 1 these days, and a game such as Wordle doesn't fit into the courses I do teach right now. Second, there's a certain element of Yogi Berra wisdom driving me away from Wordle: "It's so popular; no one goes there any more."

Most important, I had my students implementing Mastermind in my OO course back in the 2000-2005 era. I've already had most of the fun I can have with this game as an assignment. And it was a popular one with students, though challenging. The model of the game itself presents challenges for representation and algorithm, and the UI has the sort of graphic panache that tends to engage students. I remember receiving email from a student who had transferred to another university, asking me if I would still help her debug and improve her code for the assignment; she wanted to show it to her family. (Of course I helped!)

However I did see something recently that made me think of a cool assignment: 5x6 Art, a Twitter account that converts paintings and other images into minimalist 5 block-by-6 block abstract grids. The connection to Wordle is in the grid, but the color palette is much richer. Like any good CS prof, I immediately asked myself, "How can I turn this into an assignment?"

I taught our intro course using the media computation approach popularized by Mark Guzdial back in 2006. In that course, my students processed images such as the artwork displayed above. They would have enjoyed this challenge! There are so many cool ways to think about creating a 5x6 abstraction of an input image. We could define a fixed palette of n colors, then map the corresponding region of the image onto a single block. But how to choose the color?

We could compute the average pixel value of the range and then choose the color in the palette closest to that value. Or we could create neighborhoods of different sizes around all of the palette colors so that we favor some colors for the grid over others. What if we simply compute the average pixel for the region and use that as the grid color? That would give us a much larger but much less distinct set of possible colors. I suspect that this would produce less striking outputs, but I'd really like to try the experiment and see the grids it produces.

What if we allowed ourselves a bigger grid, for more granularity in our output images? There are probably many other dimensions we could play with. The more artistically inclined among you can surely think of interesting twists I haven't found yet.

That's some media computation goodness. I may assign myself to teach intro again sometime soon just so that I can develop and use this assignment. Or stop doing other work for a day or two and try it out on my own right now.

Or, Plain Text and Spreadsheets -- Giving Up and Giving In

One day a couple of weeks ago, a colleague and I were discussing a student. They said, I think I had that student in class a few semesters ago, but I can't find the semester with his grade."

My first thought was, "I would just use grep to...". Then I remembered that my colleagues all use Excel for their grades.

The next day, I saw Derek Sivers' recent post that gives the advice I usually give when asked: Write plain text files.

Over my early years in computing, I lost access to a fair bit of writing that was done using various word processing applications. All stored data in proprietary formats. The programs are gone, or have evolved well beyond the version and OS I was using at the time, and my words are locked inside. Occasionally I manage to pull something useful out of one of those old files, but for the most part they are a graveyard.

No matter how old, the code and essays I wrote in plaintext are still open to me. I love being able to look at programs I wrote for my undergrad courses (including the first parser I ever wrote, in Pascal) and my senior honors project (an early effort to implement Swiss System pairings for chess tournament). All those programs have survived the move from 5-1/4" floppies, through several different media, and still open just fine in emacs. So do the files I used to create our wedding invitations, which I wrote in troff(!).

The advice to write in plain text transfers nicely from proprietary formats on our personal computers to tools that run out on the web. The same week as Sivers posted his piece, a prolific Goodreads reviewer reported losing all his work when Goodreads had a glitch. The reviewer may have written in plain text, but his reviewers are buried in someone else's system.

I feel bad for non-tech folks when they lose their data to a disappearing format or app. I'm more perplexed when a CS prof or professional programmer does. We know about plain text; we know the history of tools; we know that our plain text files will migrate into the future with us, usable in emacs and vi and whatever other plain text editors we have available there.

I am not blind, though, to the temptation. A spreadsheet program does a lot of work for us. Put some numbers here, add a formula or two over there, and boom! your grades are computed and ready for entry -- into the university's appalling proprietary system, where the data goes to die. (Try to find a student's grade from a forgotten semester within that system. It's a database, yet there are no affordances available to users for the simplest tasks...)

All of my grade data, along with most of what I produce, is in plain text. One cost of this choice is that I have to write my own code to process it. This takes a little time, but not all that much, to be honest. I don't need all of Numbers or Excel; all I need most of the time is the ability to do simple computations and a bit of sorting. If I use a comma-separated values format, all of my programming languages have tools to handle parsing, so I don't even have to do much input processing to get started. If I use Racket for my processing code, throwing a few parens into the mix enables Racket to read my files into lists that are ready for mapping and filtering to my heart's content.

Back when I started professoring, I wrote all of my grading software in whatever language we were using in the class in that semester. That seemed like a good way for me to live inside the language my students were using and remind myself what they might be feeling as they wrote programs. One nice side effect of this is that I have grading code written in everything from Cobol to Python and Racket. And data from all those courses is still searchable using grep, filterable using cut, and processable using any code I want to write today.

That is one advantage of plain text I value that Sivers doesn't emphasize: flexibility. Not only will plain text survive into the future... I can do anything I want with it. I don't often feel powerful in this world, but I feel powerful when I'm making data work for me.

In the end, I've traded the quick and easy power of Excel and its ilk for the flexible and power of plain text, at a cost of writing a little code for myself. I like writing code, so this sort of trade is usually congenial to me. Once I've made the trade, I end up building a set of tools that I can reuse, or mold to a new task with minimal effort. Over time, the cost reaches a baseline I can live with even when I might wish for a spreadsheet's ease. And on the day I want to write a complex function to sort a set of records, one that befuddles Numbers's sorting capabilities, I remember why I like the trade. (That happened yet again last Friday.)

A recent tweet from Michael Nielsen quotes physicist Steven Weinberg as saying, "This is often the way it is in physics -- our mistake is not that we take our theories too seriously, but that we do not take them seriously enough." I think this is often true of plain text: we in computer science forget to take its value and power seriously enough. If we take it seriously, then we ought to be eschewing the immediate benefits of tools that lock away our text and data in formats that are hard or impossible to use, or that may disappear tomorrow at some start-up's whim. This applies not only to our operating system and our code but also to what we write and to all the data we create. Even if it means foregoing our commercial spreadsheets except for the most fleeting of tasks.

My recent post on what language or tool I should dive into next got some engagement on Twitter, with many helpful suggestions. Thank you all! So I figure I should post a quick update to report what I'm thinking at this point.

In that post, I mentioned JavaScript and Pharo by name, though I was open to other ideas. Many folks pointed out the practical value of JavaScript, especially in a context where many of my students know and use it. Others offered lots of good ideas in the Smalltalk vein, both Pharo and several lighter-weight Squeaks. A couple of folks recommended Glamorous Toolkit (GToolkit), from @feenkcom, which I had not heard of before.

I took to mind several of the suggestions that commenters made about the how to think about making the decision. For example, there is more overhead to studying Pharo and GToolkit than JavaScript or one of the lighter-weight Squeaks. Choosing one of the latter would make it easier to tinker. I think some of these comments had students in mind, but they are true even for my own study during the academic semester. Once I get into a term (my course begins one week from today), my attention gets pulled in many directions for fifteen or sixteen weeks. Being able to quickly switch contexts when jumping into a coding session means that I can jump more often and more productively.

Also, as Glenn Vanderburg pointed out, JavaScript and Pharo aren't likely to teach me much new. I have a lot of background with Smalltalk and, in many ways, JavaScript is just another language. The main benefit of working with either would be practical, not educational.

GToolkit might teach me something, though. As I looked into GToolkit, it became more tempting. The code is Smalltalk, because it is implemented in Pharo. But the project has a more ambitious vision of software that is "moldable": easier to understand, easier to figure out. GToolkit builds on Smalltalk's image in the direction of a computational notebook, which is an idea I've long wanted to explore. (I feel a little guilty that I haven't look more into the work that David Schmüdde has done developing a notebook in Clojure.) GToolkit sounds like a great way for me to open several doors at once and learn something new. To do it justice, though, I need more time and focus to get started.

So I have decided on a two-pronged approach. I will explore JavaScript during the spring semester. This will teach me more about a language and ecosystem that are central to many of my students' lives. There is little overhead to picking it up and playing with it, even during the busiest weeks of the term. I can have a little fun and maybe make some connections to my programming languages course along the way. Then for summer, I will turn my attention to GToolkit, and perhaps a bigger research agenda.

I started playing with JavaScript on Tuesday. Having just read a blog post on scripting to compute letter frequencies in Perl, I implemented some of the same ideas in JavaScript. For the most part, I worked just as my students do: relying on vague memories of syntax and semantics and, when that failed, searching about for examples online.

A couple of hours working like this refreshed my memory on the syntax I knew from before and introduced me to some features that were new to me. It took a few minutes to re-internalize the need for those pesky semicolons at the end of every line... The resulting code is not much more verbose than Perl. I drifted pretty naturally to using functional programming style, as you might imagine, and it felt reasonably comfortable. Pretty soon I was thinking more about the tradeoff between clarity and efficiency in my code than about syntax, which is a good sign. I did run into one of JavaScript's gotchas: I used for...in twice instead of for...of and was surprised by the resulting behavior. Like any programmer, I banged my head on wall for a few minutes and then recovered. But I have to admit that I had fun. I like to program.

I'm not sure what I will write next, or when I will move into the browser and play with interface elements. Suggestions are welcome!

I am pretty sure, though, that I'll start writing unit tests soon. I used SUnit briefly and have a lot of experience with JUnit. Is JSUnit a thing?

When I was in high school, the sponsor of our our Math Club, Mr. Harpring, liked to give books as prizes and honors for various achievements. One time, he gave me Women in Mathematics, by Lynn Osen. It introduced me to Émilie du Châtelet, Sophie Germain, Emmy Noether, and a number of other accomplished women in the field. I also learned about some cool math ideas.

Another time, I received The Unexpected Hanging and Other Mathematical Diversions, a collection of Martin Gardner's columns from Scientific American. One of the chapters was about Hexapawn, a simple game played with chess pawns on a 3x3 board. The chapter described an analog computer that learned how to play a perfect game of Hexapawn. I was amazed.

I played a lot of chess in high school and was already interested in computer chess programs. Now I began to wonder what it would be like to write a program that could learn to play chess... I suspect that Gardner's chapter planted one of the seeds that grew into my study of computer science in college. (It took a couple of years, though. From the time I was eight years old, I had wanted to be an architect, and that's where my mind was focused.)

As I wrote those words, it occurred to me that I may have written about the Gardner book before. Indeed I have, in a 2013 post on building the Hexapawn machine. Some experiences stay with you.

They also intersect with the rest of the world. This week, I read Jeff Atwood's recent post about his project to bring the 1973 book BASIC Computer Games into the 21st century. This book contains the source code of BASIC programs for 101 simple games. The earliest editions of this book used a version of BASIC before it included the GOSUB command, so there are no subroutines in any of the programs! Atwood started the project as a way to bring the programs in this book to a new audience, using modern languages and idioms.

You may wonder why he and other programmers would feel so fondly about BASIC Computer Games to reimplement its programs in Java or Ruby. They feel about these books the way I felt about The Unexpected Hanging. Books were the Github of the day, only in analog form. Many people in the 1970s and 1980s got their start in computing by typing these programs, character for character, into their computers.

I was not one of those people. My only access to a computer was in the high school, where I took a BASIC programming class my junior year. I had never seen a book like BASIC Computer Games, so I wrote all my programs from scratch. As mentioned in an old OOPSLA post from 2005, the first program I wrote out of passion was a program to implement a ratings system for our chess club. Elo ratings were great application for a math student and beginning programmer.

Anyway, I went to the project's Github site to check out what was available and maybe play a game or two. And there it was: Hexapawn! Someone has already completed the port to Python, so I grabbed it and played a few games. The text interface is right out of 1973, as promised. But the program learns, also as promised, and eventually plays a perfect game. Playing it brings back memories of playing my matchbox computer from high school. I wonder now if I should write my own program that learns Hexapawn faster, hook it up with the program from the book, and let them duke it out.

Atwood's post brought to mind pleasant memories at a time when pleasant memories are especially welcome. So many experiences create who we are today, yet some seem to have made an outsized contribution. Learning BASIC and reading Martin Gardner's articles are two of those for me.

Reading that blog post and thinking about Hexapawn also reminded me of Mr. Harpring and the effect he had on me as a student of math and as a person. The effects of a teacher in high school or grade school can be subtle and easy to lose track of over time. But they can also be real and deep, and easy not to appreciate fully when we are living them. I wish I could thank Mr. Harpring again for the books he gave me, and for the gift of seeing a teacher love math.

I've never been one to write year-end retrospectives on my blog, or prospective posts about my plans for the new year. That won't change this year, this post notwithstanding.

I will say that 2021 was a weird year for me, as it was for many people. One positive was purchasing a 29" ultra-wide monitor for work at home, seen in this post from my Strange Loop series. Programming at home has been more fun since the purchase, as have been lecture prep, data-focused online meetings, and just about everything. The only downside is that it's in my basement office, which hides me away. When I want to work upstairs to be with family, it's back to the 15" laptop screen. First-world problems.

Looking forward, I'm feeling a little itchy. I'll be teaching programming languages again this spring and plan to inject some new ideas, but the real itch is: I am looking for a new project to work on, and a new language to study. This doesn't have to be a new language, just one that one I haven't gone deep on before. I have considered a few, including Swift, but right now I am thinking of Pharo and JavaScript.

Thinking about mastering JavaScript in 2022 feels like going backward. It's old, as programming languages go, and has been a dominant force in the computing world for well over a decade. But it's also the most common language many of my students know that I have never gone deep on. There is great value in studying languages for their novel ideas and academic interest, but there is also value in having expertise with a language and toolchain that my students already care about. Besides, I've really enjoyed reading about work on JIT compilation of JavaScript over the years, and it's been a long time since I wrote code in a prototype-based OO language. Maybe it's time to build something useful in JavaScript.

Studying Pharo would be going backward for me in a different way. Smalltalk always beckons. Long-time followers of this blog have read many posts about my formative experiences with Smalltalk. But it has been twenty years since I lived in an image every day. Pharo is a modern Smalltalk with a big class library and a goal of being suitable for mission-critical systems. I don't need much of a tug; Smalltalk always beckons.

My current quandary brings to mind a dinner at a Dagstuhl seminar in the summer of 2019 (*). It's been a while now, so I hope my memory doesn't fail me too badly. Mark Guzdial was talking about a Pharo MOOC he had recently completed and how he was thinking of using the language to implement a piece of software for his new research group at Michigan, or perhaps a class he was teaching in the fall. If I recall correctly, he was torn between using Pharo and... JavaScript. He laid out some of the pros and cons of each, with JavaScript winning out on several pragmatic criteria, but his heart was clearly with Pharo. Shriram Krishnamurthi gently encouraged Mark to follow his heart: programming should be joyful, and programming languages allow us to build in languages that give us enjoyment. I seconded the (e)motion.

And here I sit mulling a similar choice.

Maybe I can make this a two-language year.

~~~~~

(*) Argh! I never properly blogged about about this seminar, on the interplay between notional machines and programming language semantics, or the experience of visiting Europe for the first time. I did write one post that mentioned Dagstuhl, Paris, and Montenegro, with an expressed hope to write more. Anything I write now will be filtered through two and a half years of fuzzy memory, but it may be worth the time to get it down in writing before it's too late to remember anything useful. In the meantime: both the seminar and the vacation were wonderful! If you are ever invited to participate in a Dagstuhl seminar, consider accepting.

The week after Strange Loop has been a blur of catching up with all the work I didn't do while attending the conference, or at least trying. That is actually good news for my virtual conference: despite attending Strange Loop from the comfort of my basement, I managed not to get sucked into the vortex of regular business going on here.

A few closing thoughts on the conference:

• Speaking of "the comfort of my basement", here is what my Strange Loop conference room looked like:

The big screen is a 29" ultra-wide LG monitor that I bought last year on the blog recommendation of Robert Talbert, which has easily been my best tech purchase of the pandemic. On that screen you'll see vi.to, the streaming platform used by Strange Loop, running in Safari. To its right, I have emacs open on a file of notes and occasionally an evolving blog draft. There is a second Safari window open below emacs, for links picked up from the talks and the conference Slack channels.

On the MacBookPro to left, I am running Slack, another emacs shell for miscellaneous items, and a PDF of the conference schedule, marked up with the two talks I'm considering in each time slot.

That set-up served me well. I can imagine using it again in the future.

• Attending virtually has its downsides, but also its upsides. Saturday morning, one attendee wrote in the Slack #virtual-attendees channel:

Virtual FTW! Attending today from a campsite in upstate New York and enjoying the fall morning air

I was not camping, but I experienced my own virtual victories at lunch time, when I was able to go for a walk with my wife on our favorite walking trails.

• I didn't experience many technical glitches at the conference. There were some serious AV issues in the room during Friday's second slot. Being virtual, I was able to jump easily into and out of the room, checking in on another talk while they debugged on-site. In another talk, we virtual attendees missed out on seeing the presenter's slides. The speaker's words turned out to be enough for me to follow. Finally, Will Byrd's closing keynote seemed to drop its feed a few times, requiring viewers to refresh their browsers occasionally. I don't have any previous virtual conferences to compare to, but this all seemed pretty minor. In general, the video and audio feedbacks were solid and of high fidelity.

• One final note, not related to The Virtual Experience. Like many conferences, Strange Loop has so many good talks that I usually have to choose among two or three talks I want to see in each slot. This year, I kept track of alt-Strange Loop, the schedule of talks I didn't attend but really wanted to. Comparing this list to the list of talks I did attend gives a representative account of the choices I faced. It also would make for a solid conference experience in its own right:

• FRI 02 -- Whoops! I Rewrote it in Rust (Brian Martin)
• FRI 03 -- Keeping Your Open Source Project Accessible to All (Treva Williams)
• FRI 04 -- Impacting Global Policy by Understanding Litter Data (Sean Doherty)
• FRI 05 -- Morel, A Functional Query Language (Julian Hyde)
• FRI 06 -- Software for Court Appointed Special Advocates (Linda Goldstein)
• SAT 02 -- Asami: Turn your JSON into a Graph in 2 Lines (Paula Gearon)
• SAT 03 -- Pictures Of You, Pictures Of Me, Crypto Steganography (Sean Marcia)
• SAT 04 -- Carbon Footprint Aware Software Development Tejas Chopra
• SAT 05 -- How Flutter Can Change the Future of Urban Communities (Edward Thornton)
• SAT 06 -- Creating More Inclusive Tech Spaces: Paths Forward (Amy Wallhermfechtel)

There is a tie for the honor of "talk I most wanted to see but didn't": Wallhermfechtel on creating more inclusive tech spaces and Marcia on crypto steganography. I'll be watching these videos on YouTube some time soon!

As I mentioned in Day 1's post, this year I tried to force myself out of usual zone, to attend a wider range of talks. Both lists of talks reflect this mix. At heart I am an academic with a fondness for programming languages. The tech talks generally lit me up more. Even so, I was inspired by some of the talks focused on community and the use of technology for the common good. I think I used my two days wisely.

That is all. Strange Loop sometimes gives me the sort of inspiration overdose that Molly Mielke laments in this tweet. This year, though, Strange Loop 2021 gave me something I needed after eighteen months of pandemic (and even more months of growing bureaucracy in my day job): a jolt of energy, and a few thoughts for the future.

I am usually tired on the second day of a conference, and today was no exception. But the day started and ended with talks that kept my brain alive.

• "Poems in an Accidental Language" by Kate Compton -- Okay, so that was a Strange Loop keynote. When the video goes live on YouTube, watch it. I may blog more about the talk later, but for now know only that it included:

• "Evenings of Recreational Ontology" (I also learned about Google Sheet parties)
• "fitting an octopus into an ontology"
• "Contemplate the universe, and write an API for it."

• Quantum computing is one of those technical areas I know very little about, maybe the equivalent of a 30-minute pitch talk. I've never been super-interested, but some of my students are. So I attended "Practical Quantum Computing Today" to see what's up these days. I'm still not interested in putting much of my time into quantum computing, but now I'm better informed.

• Before my lunch walk, I attended a non-technical talk on "tech-enabled crisis response". Emma Ferguson and Colin Schimmelfing reported on their experience doing something I'd like to be able to do: spin up a short-lived project to meet a critical need, using mostly free or open-source tools. For three months early in the COVID pandemic, their project helped deliver ~950,000 protective masks from 7,000 donors to 6,000 healthcare workers. They didn't invent new tech; they used existing tools and occasionally wrote some code to connect such tools.

My favorite quote from the talk came when Ferguson related the team's realization that they had grown too big for the default limits on Google Sheets and Gmail. "We thought, 'Let's just pay Google.' We tried. We tried. But we couldn't figure it out." So they built their own tool. It is good to be a programmer.

• After lunch, Will Crichton live-coded a simple API in Rust, using traits (Rust's version of interfaces) and aggressive types. He delivered almost his entire talk within emacs, including an ASCII art opening slide. It almost felt like I was back in grad school!

• In "Remote Workstations for Discerning Artists", Michelle Brenner from Netflix described the company's cloud-based infrastructure for the workstations used by the company's artists and project managers. This is one of those areas that is simply outside my experience, so I learned a bit. At the top level, though, the story is familiar: the scale of Netflix's goals requires enabling artists to work wherever they are, whenever they are; the pandemic accelerated a process that was already underway.

• Eric Gade gave another talk in the long tradition of Alan Kay and a bigger vision for computing. "Authorship Environments: In Search of the 'Personal' in Personal Computing" started by deconstructing Steve Jobs's "bicycle for the mind" metaphor (he's not a fan of what most people take as the meaning) and then moved onto the idea of personal computing as literacy: a new level at which to interrogate ideas, including one's own.

This talk included several inspirational quotes. My favorite was was from Adele Goldberg:

There's all these layers in everything we do... We have to learn how to peel.

As with most talks in this genre, I left feeling like there is so much more to be done, but frustrated at not knowing how to do it. We still haven't found a way to reach a wide audience with the empowering idea that there is more to computing than typing into a Google doc or clicking in a web browser.

• The closing keynote was delivered by Will Byrd. "Strange Dreams of Stranger Loops" took Douglas Hofstadter's Gödel, Escher, Bach as its inspiration, fitting both for the conference and for Byrd's longstanding explorations of relational programming. His focus today: generating quines in mini-Kanren, and discussing how quines enable us to think about programs, interpreters, and the strange loops at the heart of GEB.

As with the opening keynote I may blog more about this talk later. For now I give you two fun items:

• Byrd expressed his thanks to D((a(d|n))oug), a regular expression that matches on Byrd (his father), Friedman (his academic mentor), and Hofstadter (his intellectual inspiration).
• While preparing his keynote, Byrd clains to have suffered from UDIS: Unintended Doug Intimidation Syndrome. Hofstader is so cultured, so well-read, and so deep a thinker, how can the rest of us hope to contribute?

Strange Loop 2021 has ended. I "hit the road" by walking upstairs to make dinner with my wife.

On this first day of my first virtual conference, I saw a number of Strange Loop-y talks: several on programming languages and compilers, a couple by dancers, and a meta-talk speculating on the future of conferences.

• I'm not a security guy or a cloud guy, so the opening keynote "Why Security is the Biggest Benefit of Using the Cloud" by AJ Yawn gave me a chance to hear what people in this space think and talk about. Cool trivia: Yawn played a dozen college basketball games for Leonard Hamilton at Florida State. Ankle injuries derailed his college hoops experience, and now he's a computer security professional.

• Richard Marmorstein's talk, "Artisanal, Machine-Generated API Libraries" was right on topic with my compiler course this semester. My students would benefit from seeing how software can manipulate AST nodes when generating target code.

Marmorstein uttered two of the best lines of the day:

• "I could tell you a lot about Stripe, but all you need to know is Stripe has an API."
• "Are your data structures working for you?"

I've been working with students all week trying to help them see how an object in their compiler such as a token can help the compiler do its job -- and make the code simpler to boot. Learning software design is hard.

• I learned a bit about the Nim programming language from Aditya Siram. As you might imagine, a language designed at the nexus Modula/Oberon, Python, and Lisp appeals to me!

• A second compiler-oriented talk, by Richard Feldman, demonstrated how opportunistic in-place mutation, a static optimization, can help a pure functional program outperform imperative code.

• After the talk "Dancing With Myself", an audience member complimented Mariel Pettee on "nailing the Strange Loop talk". The congratulations were spot-on. She hit the technical mark by describing the use of two machine learning techniques, variational auto encoding and graph neural networks. She hit the aesthetic mark by showing how computer models can learn and generate choreography. When the video for this talk goes live, you should watch.

Pettee closed with the expansive sort of idea that makes Strange Loop a must-attend conference. Dance has no universal language for "writing" choreography, and video captures only a single instance or implementation of a dance, not necessarily the full intent of the choreographer. Pettite had expected her projects to show how machine learning can support invention and co-creation, but now she sees how work like this might provide a means of documentation. Very cool. Perhaps CS can help to create a new kind of language for describing dance and movement.

• I attended Laurel Lawson's "Equitable Experiential Access: Audio Description" to learn more about ways in which videos and other media can provide a fuller, more equitable experience to everyone. Equity and inclusion have become focal points for so much of what we do at my university, and they apply directly to my work creating web-based materials for students. I have a lot to learn. I think one of my next steps will be to experience some of web pages (session notes, assignments, resource pages) solely through a screen reader.

• Like all human activities, traditional in-person conferences offer value and extract costs. Crista Lopes used her keynote closing Day 1 to take a sober look at the changes in their value and their costs in the face of technological advances over the last thirty years.

If we are honest with ourselves, virtual conferences are already able to deliver most of the value of in-person conferences (and, in some ways, provide more value), at much lower cost. The technology of going virtual is the easy part. The biggest challenges are social.

~~~~~

A few closing thoughts as Day 1 closes.

As Crista said, "Taking paid breaks in nice places never gets old." My many trips to OOPSLA and PLoP provided me with many wonderful physical experiences. Being in the same place with my colleagues and friends was always a wonderful social experience. I like driving to St. Louis and going to Strange Loop in person; sitting in my basement doesn't feel the same.

With time, perhaps my expectations will change.

It turns out, though, that "virtual Strange Loop" is a lot like "in-person Strange Loop" in one essential way: several cool new ideas arrive every hour. I'll be back for Day Two.

After a couple of years away, I am attending Strange Loop. 2018 seems so long ago now...

Last Wednesday morning, I hopped in my car and headed south to Strange Loop 2018. It had been a few years since I'd listened to Zen and the Art of Motorcycle Maintenance on a conference drive, so I popped it into the tape deck (!) once I got out of town and fell into the story. My top-level goal while listening to Zen was similar to my top-level goal for attending Strange Loop this year: to experience it at a high level; not to get bogged down in so many details that I lost sight of the bigger messages. Even so, though, a few quotes stuck in my mind from the drive down. The first is an old friend, one of my favorite lines from all of literature:

Assembly of Japanese bicycle require great peace of mind.

The other was the intellectual breakthrough that unified Phaedrus's philosophy:

Quality is not an object; it is an event.

This idea has been on my mind in recent months. It seemed a fitting theme, too, for Strange Loop.

There will be no Drive South in 2021. For a variety of reasons, I decided to attend the conference virtually. The persistence of COVID is certainly one of big the reasons. Alex and the crew at Strange Loop are taking all the precautions one could hope for to mitigate risk, but even so I will feel more comfortable online this year than in rooms full of people from across the country. I look forward to attending in person again soon.

Trying to experience the conference at a high level is again one of my meta-level goals for attending. The program contains so many ideas that are new to me; I think I'll benefit most by opening myself to areas I know little or nothing about and seeing where the talks lead me.

This year, I have a new meta-level goal: to see what it is like to attend a conference virtually. Strange Loop is using Vito as its hub for streaming video and conference rooms and Slack as its online community. This will be my first virtual conference, and I am curious to see how it feels. With concerns such as climate change, public health, and equity becoming more prominent as conference-organizing committees make their plans, I suspect that we will be running more and more of our conferences virtually in the future, especially in CS. I'm curious to see how much progress has been made in the last eighteen months and how much room we have to grow.

This topic is even on the program! Tomorrow's lineup concludes with Crista Lopes speaking on the future of conferences. She's been thinking about and helping to implement conferences in new ways for a few years, so I look forward to hearing what she has to say.

Whatever the current state of virtual conferences, I fully expect that this conference will be a worthy exemplar. It always is.

So, I'm off to Strange Loop for a couple of days. I'll be in my basement.

Well, a month has passed. Already, the first week of classes are in the books. My compiler course is off to as good a start as one might hope.

Week 1 of the course is an orientation to the course content and project. Content-wise, Day 1 offers a bird's-eye view of what a compiler does, then Day 2 tries to give a bird's-eye view of how a compiler works. Beginning next week, we go deep on the stages of a compiler, looking at techniques students can use to implement their compiler for a small language. That compiler project is the centerpiece and focus of the course.

Every year, I think about ways to shake up this course. (Well, not last year, because we weren't able to offer it due to COVID.) As I prepared for the course, I revisited this summary of responses to a Twitter request from John Regehr: What should be taught in a modern undergrad compiler class? It was a lot of fun to look back through the many recommendations and papers linked there. In the end, though, the response that stuck with me came from Celeste Hollenbeck, who "noted the appeal of focusing on the basics over esoterica": compilers for the masses, not compilers for compiler people.

Our class is compilers for everyone in our major, or potentially so. Its main role in our curriculum is to be one of four so-called project courses, which serve as capstones for a broad set of electives. Many of the students in the course take it to satisfy their project requirement, others take it to satisfy a distribution requirement, and a few take it just because it sounds like fun.

The course is basic, and a little old-fashioned, but that works for us. The vast majority of our students will never write a compiler again. They are in the course to learn something about how compilers work conceptually and to learn what it is like to build a large piece of software with a team. We talk about modern compiler technology such as LLVM, but working with such complicated systems would detract from the more general goals of the course for our students. Some specific skills for writing lexers and scanners, a little insight into how compilers work, and experience writing a big program with others (and living with design decisions for a couple of months!) are solid outcomes for an undergrad capstone project.

That's not to say that some students don't go on to do more with compilers... Some do. A few years ago, one of our undergrads interviewed his way into an internship with Sony PlayStation's compiler team, where he now works full time. Other students have written compilers for their own languages, including one that was integrated as a scripting language into a gaming engine he had built. In that sense, the course seems to serve the more focused students well, too.

Once more unto the breach, dear friends, once more...
-- Henry V

So, we are off. I still haven't described the source language my students will be processing this semester, as promised in my last post. Soon. Since then, though, I wrote a bunch of small programs in the language just to get a feel for it. That's as much fun as a department head gets to have most days these days.

On teaching, via Robert Talbert:

Look at the course you teach most often. If you had the power to remove one significant topic from that course, what would it be, and why?

I have a high degree of autonomy in most of the courses I teach, so power isn't the limiting factor for me. Time is a challenge to making big changes, of course. Gumption is probably what I need most right now. Summer is a great time for me to think about this, both for my compiler course this fall and programming languages next spring.

On research, via Kris Micinski:

i remember back to Dana Scott's lecture on the history of the lambda calculus where he says, "If Turing were alive today, I don't know what he'd be doing, but it wouldn't be recursive function theory." I think about that a lot.

Now I am, too. Seriously. I'm no Turing, but I have a few years left and some energy to put into something that matters. Doing so will require some gumption to make other changes in my work life first. I am reaching a tipping point.

A couple of weeks ago, a former student emailed me after many years. Felix immigrated to the US from the Sudan back in the 1990s and wound up at my university, where he studied computer science. While in our program, he took a course or two with me, and I supervised his undergrad research project. He graduated and got busy with life, and we lost touch.

He emailed to let me know that he was about to defend his Ph.D. dissertation, titled "Efficient Reconstruction and Proofreading of Neural Circuits", at Harvard. After graduating from UNI, he programmed at DreamWorks Interactive and EA Sports, before going to grad school and working to "unpack neuroscience datasets that are almost too massive to wrap one's mind around". He defended his dissertation successfully this week.

Congratulations, Dr. Gonda!

Felix wrote initially to ask permission to acknowledge me in his dissertation and defense. As I told him, it is an honor to be remembered so fondly after so many years. People often talk about how teachers affect their students' futures in ways that are often hard to see. This is one of those moments for me. Arriving at the end of what has been a challenging semester in the classroom for me, Felix's note boosted my spirit and energizes me a bit going into the summer.

If you'd like to learn more about Felix and his research, here is his personal webpage The Harvard School of Engineering also has a neat profile of Felix that shows you what a neat person he is.

In a Paris Review interview, Fran Lebowitz joked about the challenge of writing:

Every time I sit at my desk, I look at my dictionary, a Webster's Second Unabridged with nine million words in it and think, All the words I need are in there; they're just in the wrong order.

Unfortunately, thinks this computer scientist, writing is a computationally more intense task than simply putting the words in the right order. We have to sample with replacement.

Computational complexity is the reason we can't have nice things.

Last summer, I tried something new: I stored the entire directory of materials for my database course in Git. This included all of my code, the course website, and everything else. It worked well.

The idea came from a post or tweet many years ago by Martin Fowler who, if I recall correctly, had put his entire home directory under version control. It sounded like the potential advantages might be worth the cost, so I made a note to try it myself sometime. I wasn't quite ready last summer to go all the way, so I took a baby step by creating my new course directory as a git repo and growing it file by file.

My context is pretty simple. I do almost all of my work on a personal MacBook Pro or a university iMac in my office. My main challenge is to keep my files in sync. When I make changes to a small number of files, or when the stakes of a missing file are low, copying files by hand works fine, with low overhead and no tooling necessary.

When I make a lot of changes in a short period of time, however, as I sometimes do when writing code or building my website, doing things by hand becomes more work. And the stakes of losing code or web pages are a lot higher than losing track of some planning notes or code I've been noodling with. To solve this problem, for many years I have been using rsync and a couple of simple shell scripts to manage code directories and my course web sites.

So, the primary goal for using Git in a new workflow was to replace rsync. Not being a Git guru, as many of you are, I figured that this would also force me to live with git more often and perhaps expand my tool set of handy commands.

My workflow for the semester was quite simple. When I worked in the office, there were four steps:

1. git merge laptop
2. [ do some work ]
3. git commit
4. git push

On my laptop, the opening and closing git commands changed:

1. git pull origin main
2. [ do some work ]
3. git commit
4. git push origin laptop

My work on a course is usually pretty straightforward. The most common task is to create files and record information with commit. Every once in a while, I had to back up a step with checkout.

You may say, "But you are not using git for version control!" You would be correct. The few times I checked out an older version of a file, it was usually to eliminate a spurious conflict, say, a .DS_Store file that was out of sync. Locally, I don't need a lot of version control, but using Git this way was a form of distributed version control, making sure that, wherever I was working, I had the latest version of every file.

I think this is a perfectly valid way to use Git. In some ways, Git is the new Unix. It provided me with a distributed filesystem and a file backup system all in one. The git commands ran effectively as fast as their Unix counterparts. My repo was not very much bigger than the directory would have been on its own, and I always had a personal copy of the entire repo with me wherever I went, even if I had to use another computer.

Before I started, several people reminded me that Git doesn't always work well with large images and binaries. That didn't turn out to be much of a problem for me. I had a couple of each in the repo, but they were not large and never changed. I never noticed a performance hit.

The most annoying hiccup all semester was working with OS X's .DS_Store files, which record screen layout information for OS X. I like to keep my windows looking neat and occasionally reorganize a directory layout to reflect what I'm doing. Unfortunately, OS X seems to update these files at odd times, after I've closed a window and pushed changes. Suddenly the two repos would be out of sync only because one or more .DS_Store files had changed after the fact. The momentary obstacle was quickly eliminated with a checkout or two before merging. Perhaps I should have left the .DS_Stores untracked...

All in all, I was pretty happy with the experience. I used more git, more often, than ever before and thus am now a bit more fluent than I was. (I still avoid the hairier corners of the tool, as all right-thinking people do whenever possible.) Even more, the repository contains a complete record of my work for the semester, false starts included, with occasional ruminations about troubles with code or lecture notes in my commit messages. I had a little fun after the semester ended looking back over some of those messages and making note of particular pain points.

The experiment went well enough that I plan to track my spring course in Git, too. This will be a bigger test. I've been teaching programming languages for many years and have a large directory of files, both current and archival. Not only are there more files, there are several binaries and a few larger images. I'm trying decide if I should put the entire folder into git all at once upfront or start with an empty folder a lá last semester and add files as I want or need them. The latter would be more work at early stages of development but might be a good way to clear out the clutter that has built up over twenty years.

If you have any advice on that choice, or any other, please let me know by email or on Twitter. You all have taught me a lot over the years. I appreciate it.

In the middle of an old post about computing an "impossible" integral, John Cook says:

In the artificial world of the calculus classroom, everything works out nicely. And this is a shame.

When I was a student, I probably took comfort in the fact that everything was supposed to work out nicely on the homework we did. There *was* a solution; I just had to find the pattern, or the key that turned the lock. I suspect that I was a good student largely because I was good at finding the patterns, the keys.

It wasn't until I got to grad school that things really changed, and even then course work was typically organized pretty neatly. Research in the lab was very different, of course, and that's where my old skills no longer served me so well.

In university programs in computer science, where many people first learn how to develop software, things tend to work out nicely. That is a shame, too. But it's a tough problem to solve.

In most courses, in particular introductory courses, we create assignments with that have "closed form" solutions, because we want students to practice a specific skill or learn a particular concept. Having a fixed target can be useful in achieving the desired outcomes, especially if we want to help students build confidence in their abilities.

It's important, though, that we eventually take off the training wheels and expose students to messier problems. That's where they have an opportunity to build other important skills they need for solving problems outside the classroom, which aren't designed by a benevolent instructor to have follow a pattern. As Cook says, neat problems can create a false impression that every problem has a simple solution.

Students who go on to use calculus for anything more than artificial homework problems may incorrectly assume they've done something wrong when they encounter an integral in the wild.

CS students need experience writing programs that solve messy problems. In more advanced courses, my colleagues and I all try to extend students' ability to solve less neatly-designed problems, with mixed results.

It's possible to design a coherent curriculum that exposes students to an increasingly messy set of problems, but I don't think many universities do this. One big problem is that doing so requires coordination across many courses, each of which has its own specific content outcomes. There's never enough time, it seems, to teach everything about, say, AI or databases, in the fifteen weeks available. It's easier to be sure that we cover another concept than it is to be sure students take a reliable step along the path from being able to solve elementary problems to being able to solve to the problems they'll find in the wild.

I face this set of competing forces every semester and do my best to strike a balance. It's never easy.

Courses that involve large systems projects are one place where students in my program have a chance to work on a real problem: writing a compiler, an embedded real-time system, or an AI-based system. These courses have closed form solutions of sorts, but the scale and complexity of the problems require students to do more than just apply formulas or find simple patterns.

Many students thrive in these settings. "Finally," they say, "this is a problem worth working on." These students will be fine when they graduate. Other students struggle when they have to do battle for the first time with an unruly language grammar or a set of fussy physical sensors. One of my challenges in my project course is to help this group of students move further along the path from "student doing homework" to "professional solving problems".

That would be a lot easier to do if we more reliably helped students take small steps along that path in their preceding courses. But that, as I've said, is difficult.

This post describes a problem in curriculum design without offering any solutions. I will think more about how I try to balance the forces between neat and messy in my courses, and then share some concrete ideas. If you have any approaches that have worked for you, or suggestions based on your experiences as a student, please email me or send me a message on Twitter. I'd love to learn how to do this better.

I've written a number of posts over the years that circle around this problem in curriculum and instruction. Here are three:

• Problems Are The Thing
• Problem-Based Universities and Other Radical Changes
• The Tension Among Motivation, Real Problems, and Hard Work

Note: The following comes from the bottom of my previous post. It gets buried there beneath a lot of code and thinking out loud, but it's a message that stands on its own.

~~~~~

I demo'ed a variation of my database-briven passphrase generator to my students as we closed the course last week. It let me wrap up my time with them by reminding them that they are developing skills that can change how they see every problem they encounter in the future.

Knowing how to write programs gives you a new power. Whenever you encounter a problem, you can ask yourself, "How might a program help me solve this?"

The same is true for many more specialized CS skills. People who know how to create a language and implement an interpreter can ask themselves, "How might a language help me solve this problem?" That's one of the outcomes, I hope, of our course in programming languages.

The same is true for databases, too. Whenever you encounter a problem, you can ask yourself, "Can a database help me solve this?"

Computer science students can use the tools they learn each semester to represent and interpret information. That's a power they can use to solve many problems. It's easy to lose sight of that fact during a busy semester and worth reflecting on in calmer moments.

A long semester -- shorter in time than usual by more than a week, but longer psychologically than any in a long time -- is coming to an end. Teaching databases for the first time was a lot of fun, though the daily grind of preparing so much new material in real time wore me down. Fortunately, I like to program enough that there were moments of fun scattered throughout the semester as I played with SQLite for the first time.

In the spirit of the Three Bears pattern, I looked for opportunities all semester to use SQLite to solve a problem. When I read about the Diceware technique for generating passphrases, I found one.

Diceware is a technique for generating passphrases using dice to select words from the "Diceware Word List", in which each word is paired with a five digit number. All of the digits are between one and six, so five dice rolls are all you need to select a word from the list. Choose a number of words for the passphrase, roll your dice, and select your words.

The Diceware Word List is a tab-separated file of dice rolls and words. SQLite can import TSV data directly into a table, so I almost have a database. I had to preprocess the file twice to make it importable. First, the file is wrapped as a PGP signed message, so I stripped the header and footer by hand, to create diceware-wordlist.txt.

Second, some of the words in this list contain quote characters. Like many applications, SQLite struggles with CSV and TSV files that contain embedded quote characters. There may be some way to configure it to handle these files gracefully, but I didn't bother looking for one. I just replaced the ' and " characters with _ and __, respectively:

    cat diceware-wordlist.txt \
      | sed "s/\'/_/g"        \
      | sed 's/\"/__/g'       \
      > wordlist.txt

Now the file is ready to import:

    sqlite> CREATE TABLE WordList(
       ...>    diceroll CHAR(5),
       ...>    word VARCHAR(30),
       ...>    PRIMARY KEY(diceroll)
       ...>    );

    sqlite> .mode tabs
    sqlite> .import 'wordlist.txt' WordList

    sqlite> SELECT * FROM WordList
       ...> WHERE diceroll = '11113';
    11113 a_s

That's one of the words that used to contain an apostrophe.

So, I have a dice roll/word table keyed on the dice roll. Now I want to choose words at random from the table. To do that, I needed a couple of SQL features we had not used in class: random numbers and string concatenation. The random() function returns a big integer. A quick web search showed me this code to generate a random base-10 digit:

    SELECT abs(random())%10
    FROM (SELECT 1);

    SELECT 1+abs(random())%6
    FROM (SELECT 1);

I need to evaluate this query repeatedly, so I created a view that wraps the code in what acts, effectively, a function:

    sqlite> CREATE VIEW RandomDie AS
       ...>   SELECT 1+abs(random())%6 AS n
       ...>   FROM (SELECT 1);

Aliasing the random value as 'n' is important because I need to string together a five-roll sequence. SQL's concatenation operator helps there:

    SELECT 'Eugene' || ' ' || 'Wallingford';

I can use the operator to generate a five-character dice roll by selecting from the view five times...

    sqlite> SELECT n||n||n||n||n FROM RandomDie;
    21311
    sqlite> SELECT n||n||n||n||n FROM RandomDie;
    63535

... and then use that phrase to select random words from the list:

    sqlite> SELECT word FROM WordList
       ...> WHERE diceroll =
       ...>         (SELECT n||n||n||n||n FROM RandomDie);
    fan

Hurray! Diceware defaults to three-word passphrases, so I need to do this three times and concatenate.

This won't work...

    sqlite> SELECT word, word, word FROM WordList
       ...> WHERE diceroll =
       ...>         (SELECT n||n||n||n||n FROM RandomDie);
    crt|crt|crt

    sqlite> CREATE VIEW OneRoll AS
       ...>   SELECT word FROM WordList
       ...>   WHERE diceroll =
       ...>           (SELECT n||n||n||n||n FROM RandomDie);

OneRoll acts like a table that returns a random word:

    sqlite> SELECT word FROM OneRoll;
    howdy
    sqlite> SELECT word FROM OneRoll;
    scope
    sqlite> SELECT word FROM OneRoll;
    snip

Almost there. Now, this query generates three-word passphrases:

    sqlite> SELECT Word1.word || ' ' || Word2.word || ' ' || Word3.word FROM
       ...>   (SELECT * FROM OneRoll) AS Word1,
       ...>   (SELECT * FROM OneRoll) AS Word2,
       ...>   (SELECT * FROM OneRoll) AS Word3;
    eagle crab pinch

Yea! I saved this query in gen-password.sql and saved the SQLite database containing the table WordList and the views RandomDie and OneRoll as diceware.db. This lets me generate passphrases from the command line:

    $ sqlite3 diceware.db < gen-password.sql
    ywca maine over

Yes, this is a lot of work to get a simple job done. Maybe I could do better with Python and a CSV reader package, or some other tools. But that wasn't the point. I was revisiting SQL and learning SQLite with my students. By overusing the tools, I learned them both a little better and helped refine my sense of when they will and won't be helpful to me in the future. So, success.

~~~~~

I demo'ed a variation of this to my students on the last day of class. It let me wrap up my time with them by pointing out that they are developing skills which can change how they see every problem they encounter in the future.

Knowing how to write programs gives you a new power. Whenever you encounter a problem, you can ask yourself, "How might a program help me solve this?" I do this daily, both as faculty member and department head.

The same is true for many more specialized CS skills. People who know how to create a language and implement an interpreter can ask themselves, "How might a language help me solve this problem?" That's one of the outcomes, I hope, of our course in programming languages.

The same is true for databases. When I came across a technique for generating passphrases, I could ask myself, "How might a database help me build a passphrase generator?"

Computer science students can use the tools they learn each semester to represent and interpret information. That's a power they can use to solve many problems. It's easy to lose sight of this incredible power during a hectic semester, and worth reflecting on in calmer moments after the semester ends.

People often look at the difference between the highest-rated male chess player in a group and the highest-rated female chess player in the same group and conclude that there is a difference between the abilities of men and women to play chess, despite the fact that there are usually many, many more men in the group than women. But that's not even good evidence that there is an achievement gap. From What Gender Gap in Chess?:

It's really quite simple. Let's say I have two groups, A and B. Group A has 10 people, group B has 2. Each of the 12 people gets randomly assigned a number between 1 and 100 (with replacement). Then I use the highest number in Group A as the score for Group A and the highest number in Group B as the score for Group B. On average, Group A will score 91.4 and Group B 67.2. The only difference between Groups A and B is the number of people. The larger group has more shots at a high score, so will on average get a higher score. The fair way to compare these unequally sized groups is by comparing their means (averages), not their top values. Of course, in this example, that would be 50 for both groups -- no difference!

I love this paragraph. It's succinct and uses only the simplest ideas from probability and statistics. It's the sort of statistics that I would hope our university students learn in their general education stats course. While learning a little math, students can also learn about an application that helps us understand something important in the world.

The experiment described is also simple enough for beginning programmers to code up. Over the years, I've used problems like this with intro programming students in Pascal, Java, and Python, and with students learning Scheme or Racket who need some problems to practice on. I don't know whether learning science supports my goal, but I hope that this sort of problem (with suitable discussion) can do double duty for learners: learn a little programming, and learn something important about the world.

With educational opportunities like this available to us, we really should be able to turn graduates who have a decent understanding of why so many of our naive conclusions about the world are wrong. Are we putting these opportunities to good use?

I recently read a Five Books interview about the best books on philosophical wonder. One of the books recommended by philosopher Eric Schwitzgebel was Diaspora, a science fiction novel by Greg Egan I've never read. The story unfolds in a world where people are able to destroy their physical bodies to upload themselves into computers. Unsurprisingly, this leads to some fascinating philosophical possibilities:

Well, for one thing you could duplicate yourself. You could back yourself up. Multiple times.

And then have divergent lives, as it were, in parallel but diverging.

Yes, and then there'd be the question, "do you want to merge back together with the person you diverged from?"

Egan wrote Diaspora before the heyday of distributed version control, before darcs and mercurial and git. With distributed VCS, a person could checkout a new personality, or change branches and be a different person every day. We could run diffs to figure out what makes one version of a self so different from another. If things start going too wrong, we could always revert to an earlier version of ourselves and try again. And all of this could happen with copies of the software -- ourselves -- running in parallel somewhere in the world.

And then there's Git. Imagine writing such a story now, with Git's complex model of versioning and prodigious set of commands and flags. Not only could people branch and merge, checkout and diff... A person could try something new without ever committing changes to the repository. We'd have to figure out what it means to push origin or reset --hard HEAD. We'd be able to rewrite history by rebasing, amending, and squashing. A Git guru can surely explain why we'd need to --force-with-lease or --unset-upstream, but even I can imagine the delightful possibilities of git stash in my personal improvement plan.

Perhaps the final complication in our novel would involve a merge so complex that we need a third-party diff tool to help us put our desired self back together. Alas, a Python library or Ruby gem required by the tool has gone stale and breaks an upgrade. Our hero must find a solution somewhere in her tree of blobs, or be doomed to live a forever splintered life.

If you ever see a book named Dreaming in Git or Bug Report on an airport bookstore's shelves, take a look. Perhaps I will have written the first of my Git fantasies.

It's been a long time since I was excited by a new piece of software the way I was excited by Loglo, Avi Bryant's new creation. Loglo is "LOGO for the Glowforge", an experimental programming environment for creating SVG images. That's not a problem I need to solve, but the way Loglo works drew me in immediately. It consists of a stack programming language and a set of primitives for describing vector graphics, integrated into a spreadsheet interface. It's the use of a stack language to program a spreadsheet that excites me so much.

Actually, it's the reverse relationship that really excites me: using a spreadsheet to build and visualize a stack-based program. Long-time readers know that I am interested in this style of programming (see Summer of Joy for a post from last year) and sometimes introduce it in my programming languages course. Students understand small examples easily enough, but they usually find it hard to grok larger programs and to fully appreciate how typing in such a language can work. How might Loglo help?

In Loglo, a cell can refer to the values produced by other cells in the familiar spreadsheet way, with an absolute address such as "a1" or "f2". But Loglo cells have two other ways to refer to other cell's values. First, any cell can access the value produced by the cell to its left implicitly, because Loglo leaves the result of a cell's computation sitting on top of the stack. Second, a cell can access the value produced by the cell above it by using the special variable "^". These last two features strike me as a useful way for programmers to see their computations grow over time, which can be an even more powerful mode of interaction for beginners who are learning this programming style.

Stack-oriented programming of this sort is concatenative: programs are created by juxtaposing other programs, with a stack of values implicitly available to every operator. Loglo uses the stack as leverage to enable programmers to build images incrementally, cell by cell and row by row, referring to values on the stack as well as to predecessor cells. The programmer can see in a cell the value produced by a cumulative bit of code that includes new code in the cell itself. Reading Bryant's description of programming in Loglo, it's easy to see how this can be helpful when building images. I think my students might find it helpful when learning how to write concatenative programs or learning how types and programs work in a concatenative language.

For example, here is a concatenative program that works in Loglo as well as other stack-based languages such as Forth and Joy:

 2 3 + 5 * 2 + 6 / 3 /

Loglo tells us that it computes the value 1.5:

This program consists of eleven tokens, each of which is a program in its own right. More interestingly, we can partition this program into smaller units by taking any subsequences of the program:

 2 3 + 5 *   2 + 6 /   3 /
 ---------   -------   ---

By making the intermediate results visible to the programmer, this interface might help programmers better see how pieces of a concatenative program work and learn what the type of a program fragment such as 2 + 6 / (in cell B1 above) or 3 / is. Allowing locally-relative references on a new row will, as Avi points out, enable an incremental programming style in which the programmer uses a transformation computed in one cell as the source for a parameterized version of the transformation in the cell below. This can give the novice concatenative programmer an interactive experience more supportive than the usual REPL. And Loglo is a spreadsheet, so changes in one cell percolate throughout the sheet on each update!

Am I the only one who thinks this could be a really cool environment for programmers to learn and practice this style of programming?

Teaching concatenative programming isn't a primary task in my courses, so I've never taken the time to focus on a pedagogical environment for the style. I'm grateful to Avi for demonstrating a spreadsheet model for stack programs and stimulating me to think more about it.

For now, I'll play with Loglo as much as time permits and think more about its use, or use of a tool like it, in my courses. There are couple of features I'll have to get used to. For one, it seems that a cell can access only one item left on the stack by its left neighbor, which limits the kind of partial functions we can write into cells. Another is that named functions such as rotate push themselves onto the stack by default and thus require a ! to apply them, whereas operators such as + evaluate by default and thus require quotation in a {} block to defer execution. (I have an academic's fondness for overarching simplicity.) Fortunately, these are the sorts of features one gets used to whenever learning a new language. They are part of the fun.

Thinking beyond Loglo, I can imagine implementing an IDE like this for my students that provides features that Loglo's use cases don't require. For example, it would be cool to enable the programmer to ctrl-click on a cell to see the type of the program it contains, as well as an option to see the cumulative type along the row or built on a cell referenced from above. There is much fun to be had here.

To me, one sign of a really interesting project is how many tangential ideas flow out of it. For me, Loglo is teeming with ideas, and I'm not even in its target demographic. So, kudos to Avi!

Now, back to administrivia and that database course...

There's value to going into a field that you find difficult to grasp, as long as you're willing to be persistent. Even better, others can benefit from your persistence, too.

In an old essay, James Propp notes that working in a field where you lack intuition can "impart a useful freedom from prejudice". Even better...

... there's value in going into a field that you find difficult to grasp, as long as you're willing to be really persistent, because if you find a different way to think about things, something that works even for someone like you, chances are that other people will find it useful too.

This reminded me of a passage in Bob Nystroms's post about his new book, Crafting Interpreters. Nystrom took a long time to finish the book in large part because he wanted the interpreter at the end of each chapter to compile and run, while at the same time growing into the interpreter discussed in the next chapter. But that wasn't the only reason:

I made this problem harder for myself because of the meta-goal I had. One reason I didn't get into languages until later in my career was because I was intimidated by the reputation compilers have as being only for hardcore computer science wizard types. I'm a college dropout, so I felt I wasn't smart enough, or at least wasn't educated enough to hack it. Eventually I discovered that those barriers existed only in my mind and that anyone can learn this.

Some students avoid my compilers course because they assume it must be difficult, or because friends said they found it difficult. Even though they are CS majors, they think of themselves as average programmers, not "hardcore computer science wizard types". But regardless of the caliber of the student at the time they start the course, the best predictor of success in writing a working compiler is persistence. The students who plug away, working regularly throughout the two-week stages and across the entire project, are usually the ones who finish successfully.

One of my great pleasures as a prof is seeing the pride in the faces of students who demo a working compiler at the end of the semester, especially in the faces of the students who began the course concerned that they couldn't hack it.

As Propp points out in his essay, this sort of persistence can pay off for others, too. When you have to work hard to grasp an idea or to make something, you sometimes find a different way to think about things, and this can help others who are struggling. One of my jobs as a teacher is to help students understand new ideas and use new techniques. That job is usually made easier when I've had to work persistently to understand the idea myself, or to find a better way to help the students who teach me the ways in which they struggle.

In Nystrom's case, his hard work to master a field he didn't grasp immediately pays of for his readers. I've been following the growth of Crafting Interpreters over time, reading chapters in depth whenever I was able. Those chapters were uniformly easy to read, easy to follow, and entertaining. They have me thinking about ways to teach my own course differently, which is probably the highest praise I can give as a teacher. Now I need to go back and read the entire book and learn some more.

Teaching well enough that students grasp what they thought was not graspable and do what they thought was not doable is a constant goal, rarely achieved. It's always a work in progress. I have to keep plugging away.

I blogged weekly about the sudden switch to remote instruction starting in late March, but only for three weeks. I stopped mostly because my sense of disorientation had disappeared. Teaching class over Zoom started to feel more normal, and my students and I got back into the usual rhythm. A few struggled in ways that affected their learning and performance, and a smaller few thrived. My experience was mostly okay: some parts of my work suffered as I learned how to use tools effectively, but not having as many external restrictions on my schedule offset the negatives. Grades are in, summer break begins to begin, and at least some things are right with the world.

Fall offers something new for me to learn. My fall compilers course had a lower enrollment than usual and, given the university's current financial situation, I had to cancel it. This worked out fine for the department, though, as one of our adjunct instructors asked to take next year off in order to deal with changes in his professional and personal lives. So there was a professor in need of a course, and a course in need of a professor: Database Systems.

Databases is one of the few non-systems CS courses that I have never taught as a prof or as a grad student. It's an interesting course, mixing theory and design with a lot of practical skills that students and employers prize. In this regard, it's a lot of like our OO design and programming course in Java, only with a bit more visible theory. I'm psyched to give it a go. At the very least, I should be able to practice some of those marketable skills and learn some of the newer tools involved.

As with all new preps, this course has me looking for ideas. I'm aware of a few of the standard texts, though I am hoping to find a good open-source text online, or a set of online materials out of which to assemble the readings my students will need for the semester. I'm going to be looking scouting for all the other materials I need to teach the course as well, including examples, homework assignments, and projects. I tend to write a lot of my own stuff, but I also like to learn from good courses and good examples already out there. Not being a database specialist, I am keen to see what specialists think is important, beyond what we find in traditional textbooks.

Then there is the design of the course itself. Teaching a course I've never taught before means not having an old course design to fall back on. This means more work, of course, but is a big win for curious mind. Sometimes, it's fun to start from scratch. I have always found instructional design fascinating, much like any kind of design, and building a new course leaves open a lot of doors for me to learn and to practice some new skills.

COVID-19 is a big part of why I am teaching this course, but it is not done with us. We still do not know what fall semester will look like, other than to assume that it won't look like a normal semester. Will be on campus all semester, online all semester, or a mix of both? If we do hold instruction on campus, as most universities are hoping to do, social distancing requirements will require us to do some things differently, such as meeting students in shifts every other day. This uncertainty suggests that I should design a course that depends less on synchronous, twice-weekly, face-to-face direct instruction and more on ... what?

I have a lot to learn about teaching this way. My university is expanding its professional development offerings this summer and, in addition to diving deep into databases and SQL, I'll be learning some new ways to design a course. It's exciting but also means a bit more teaching prep than usual for my summer.

This is the first entirely new prep I've taught in a while. I think the most recent was the fall of 2009, when I taught Software Engineering for the first and only time. Looking back at the course website reminds me that I created this delightful logo for the course:

So, off to work I go. I could sure use your help. Do you know of model database courses that I should know about? What database concepts and skills should CS graduates in 2021 know? What tools should they be able to use? What has changed in the world since I last took database courses that must be reflected in today's database course? Do you know of a good online textbook for the course, or a print book that my students would find useful and be willing to pay for?

If you have any ideas to share, feel free to email me or contact me on Twitter. If not for me, do it for my students!

This week I read one of Craig Mod's old essays and found a great line, one that everyone who writes programs for other people should keep front of mind:

When it comes to software that people live in all day long, a 3% increase in fun should not be dismissed.

Working hard to squeeze a bit more speed out of a program, or to create even a marginally better interaction experience, can make a huge difference to someone who uses that program everyday. Some people spend most of their professional days inside one or two pieces of software, which accentuates further the human value of Mod's three percent. With shelter-in-place and work-from-home the norm for so many people these days, we face a secondary crisis of software that is no fun.

I was probably more sensitive than usual to Mod's sentiment when I read it... This week I used Blackboard for the first time, at least my first extended usage. The problem is not Blackboard, of course; I imagine that most commercial learning management systems are little fun to use. (What a depressing phrase "commercial learning management system" is.) And it's not just LMSes. We use various PeopleSoft "campus solutions" to run the academic, administrative, and financial operations on our campus. I always feel a little of my life drain away whenever I spend an hour or three clicking around and waiting inside this large and mostly functional labyrinth.

It says a lot that my first thought after downloading my final exams on Friday morning was, "I don't have to login to Blackboard again for a good long while. At least I have that going for me."

I had never used our LMS until this week, and then only to create a final exam that I could reliably time after being forced into remote teaching with little warning. If we are in this situation again in the fall, I plan to have an alternative solution in place. The programmer in me always feels an urge to roll my own when I encounter substandard software. Writing an entire LMS is not one of my life goals, so I'll just write the piece I need. That's more my style anyway.

Later the same morning, I saw this spirit of writing a better program in a context that made me even happier. The Friday of finals week is my department's biennial undergrad research day, when students present the results of their semester- or year-long projects. Rather than give up the tradition because we couldn't gather together in the usual way, we used Zoom. One student talked about alternative techniques for doing parallel programming in Python, and another presented empirical analysis of using IR beacons for localization of FIRST Lego League robots. Fun stuff.

The third presentation of the morning was by a CS major with a history minor, who had observed how history profs' lectures are limited by the tools they had available. The solution? Write better presentation software!

As I watched this talk, I was proud of the student, whom I'd had in class and gotten to know a bit. But I was also proud of whatever influence our program had on his design skills, programming skills, and thinking. This project, I thought, is a demonstration of one thing every CS student should learn: We can make the tools we want to use.

This talk also taught me something non-technical: Every CS research talk should include maps of Italy from the 1300s. Don't dismiss 3% increases in fun wherever they can be made.

This passage from Remembering the LAN recalls an earlier time that feels familiar:

My father, a general practitioner, used this infrastructure of cheap 286s, 386s, and 486s (with three expensive laser printers) to write the medical record software for the business. It was used by a dozen doctors, a nurse, and receptionist. ...

The business story is even more astonishing. Here is a non-programming professional, who was able to build the software to run their small business in between shifts at their day job using skills learned from a book.

I wonder how many hobbyist programmers and side-hustle programmers of this sort there are today. Does programming attract people the way it did in the '70s or '80s? Life is so much easier than typing programs out of Byte or designing your own BASIC interpreter from scratch. So many great projects out on Github and the rest of the web to clone, mimic, adapt. I occasionally hear a student talking about their own projects in this way, but it's rare.

As Crawshaw points out toward the end of his post, the world in which we program now is much more complex. It takes a lot more gumption to get started with projects that feel modern:

So much of programming today is busywork, or playing defense against a raging internet. You can do so much more, but the activation energy required to start writing fun collaborative software is so much higher you end up using some half-baked SaaS instead.

I am not a great example of this phenomenon -- Crawshaw and his dad did much more -- but even today I like to roll my own, just for me. I use a simple accounting system I've been slowly evolving for a decade, and I've cobbled together bits and pieces of my own tax software, not an integrated system, just what I need to scratch an itch each year. Then there are all the short programs and scripts I write for work to make Spreadsheet City more habitable. But I have multiple CS degrees and a lot of years of experience. I'm not a doctor who decides to implement what his or her office needs.

I suspect there are more people today like Crawshaw's father than I hear about. I wish it were more of a culture that we cultivated for everyone. Not everyone wants to bake their own bread, but people who get the itch ought to feel like the world is theirs to explore.

This was a weird week. It started with preparations for spring break and an eye on the news. It turned almost immediately into preparations for at least two weeks of online courses and a campus on partial hiatus. Of course, we don't know how the COVID-19 outbreak will develop over the next three weeks, so we may be facing the remaining seven weeks of spring semester online, with students at a distance.

Here are three pieces that helped me get through the week.

Even If You Believe

From When Bloom Filters Don't Bloom:

Advanced data structures are very interesting, but beware. Modern computers require cache-optimized algorithms. When working with large datasets that do not fit in L3, prefer optimizing for a reduced number of loads over optimizing the amount of memory used.

I've always liked the Bloom filter. It seems such an elegant idea. But then I've never used one in a setting where performance mattered. It still surprises me how well current architectures and compilers optimize performance for us in ways that our own efforts can only frustrate. The article is also worth reading for its link to a nice visualization of the interplay among the parameters of a Bloom Filter. That will make a good project in a future class.

Even If You Don't Believe

From one of Tyler Cowen's long interviews:

Niels Bohr had a horseshoe at his country house across the entrance door, a superstitious item, and a friend asked him, "Why do you have it there? Aren't you a scientist? Do you believe in it?" You know what was Bohr's answer? "Of course I don't believe in it, but I have it there because I was told that it works, even if you don't believe in it."

You don't have to believe in good luck to have good luck.

You Gotta Believe

From Larry Tesler's annotated manual for the PUB document compiler:

In 1970, I became disillusioned with the slow pace of artificial intelligence research.

The commentary on the manual is like a mini-memoir. Tesler writes that he went back to the Stanford AI lab in the spring of 1971. John McCarthy sent him to work with Les Earnest, the lab's chief administrator, who had an idea for a "document compiler", a lá RUNOFF, for technical manuals. Tesler had bigger ideas, but he implemented PUB as a learning exercise. Soon PUB had users, who identified shortcomings that were in sync with Tesler's own ideas.

The solution I favored was what we would now call a WYSIWYG interactive text editing and page layout system. I felt that, if the effect of any change was immediately apparent, users would feel more in control. I soon left Stanford to pursue my dream at Xerox PARC (1973-80) and Apple Computer (1980-1997).

Thus began the shift to desktop publishing. And here I sit, in 2020, editing this post using emacs.

I saw Robin Sloan's An App Can Be a Home-Cooked Meal floating around Twitter a few days back. It really is quite good; give it a read if you haven't already. This passage captures a lot of the essay's spirit in only a few words:

The exhortation "learn to code!" has its foundations in market value. "Learn to code" is suggested as a way up, a way out. "Learn to code" offers economic leverage, a squirt of power. "Learn to code" goes on your resume.

But let's substitute a different phrase: "learn to cook." People don't only learn to cook so they can become chefs. Some do! But far more people learn to cook so they can eat better, or more affordably, or in a specific way. Or because they want to carry on a tradition. Sometimes they learn just because they're bored! Or even because -- get this -- they love spending time with the person who's teaching them.

Sloan expresses better than I ever have an idea that I blog about every so often. Why should people learn to program? Certainly it offers a path to economic gain, and that's why a lot of students study computer science in college, whether as a major, a minor, or a high-leverage class or two. There is nothing wrong with that. It is for many a way up, a way out.

But for some of us, there is more than money in programming. It gives you a certain power over the data and tools you use. I write here occasionally about how a small script or a relatively small program makes my life so much easier, and I feel bad for colleagues who are stuck doing drudge work that I jump past. Occasionally I'll try to share my code, to lighten someone else's burden, but most of the time there is such a mismatch between the worlds we live in that they are happier to keep plugging along. I can't say that I blame them. Still, if only they could program and used tools that enabled them to improve their work environments...

But... There is more still. From the early days of this blog, I've been open with you all:

Here's the thing. I like to write code.

One of the things that students like about my classes is that I love what I do, and they are welcome to join me on the journey. Just today a student in my Programming Languages drifted back to my office with me after class , where we ended up talking for half an hour and sketching code on a whiteboard as we deconstructed a vocabulary choice he made on our latest homework assignment. I could sense this student's own love of programming, and it raised my spirits. It makes me more excited for the rest of the semester.

I've had people come up to me at conferences to say that the reason they read my blog is because they like to see someone enjoying programming as much as they do. many of them share links with their students as if to say, "See, we are not alone." I look forward to days when I will be able to write in this vein more often.

Sloan reminds us that programming can be -- is -- more than a line on a resume. It is something that everyone can do, and want to do, for a lot of different reasons. It would be great if programming "were marbled deeply into domesticity and comfort, nerdiness and curiosity, health and love" in the way that cooking is. That is what makes Computing for All really worth doing.

Campaign Security is a Wood Chipper for Your Hopes and Dreams

Practical campaign security is a wood chipper for your hopes and dreams. It sits at the intersection of 19 kinds of status quo, each more odious than the last. You have to accept the fact that computers are broken, software is terrible, campaign finance is evil, the political parties are inept, the DCCC exists, politics is full of parasites, tech companies are run by arrogant man-children, and so on.

This piece from last year has some good advice, plenty of sarcastic humor from Maciej, and one remark that was especially timely for the past week:

You will fare especially badly if you have written an app to fix politics. Put the app away and never speak of it again.

Know the Difference Between Neurosis and Process

In a conversation between Tom Waits and Elvis Costello from the late 1980s, Waits talks about tinkering too long with a song:

TOM: "You have to know the difference between neurosis and actual process, 'cause if you're left with it in your hands for too long, you may unravel everything. You may end up with absolutely nothing."

In software, when we keep code in our hands for too long, we usually end up with an over-engineered, over-abstracted boat anchor. Let the tests tell you when you are done, then stop.

Sometimes, Work is Work

People say, "if you love what you do you'll never work a day in your life." I think good work can be painful--I think sometimes it feels exactly like work.

Some weeks more than others. Trust me. That's okay. You can still love what you do.

In 1957, Dan McCracken published Digital Computer Programming, perhaps the first book on the new art of programming. His book shows that the roots of extreme programming run deep. In this passage, McCracken encourages both the writing of tests before the writing of code and the involvement of the customer in the software development process:

The first attack on the checkout problem may be made before coding is begun. In order to fully ascertain the accuracy of the answers, it is necessary to have a hand-calculated check case with which to compare the answers which will later be calculated by the machine. This means that stored program machines are never used for a true one-shot problem. There must always be an element of iteration to make it pay. The hand calculations can be done at any point during programming. Frequently, however, computers are operated by computing experts to prepare the problems as a service for engineers or scientists. In these cases it is highly desirable that the "customer" prepare the check case, largely because logical errors and misunderstandings between the programmer and customer may be pointed out by such procedure. If the customer is to prepare the test solution is best for him to start well in advance of actual checkout, since for any sizable problem it will take several days or weeks to calculate the test.

I don't have a copy of this book, but I've read a couple of other early books by McCracken, including one of his Fortran books for engineers and scientists. He was a good writer and teacher.

I had the great fortune to meet Dan at an NSF workshop in Clemson, South Carolina, back in the mid-1990s. We spent many hours in the evening talking shop and watching basketball on TV. (Dan was cheering his New York Knicks on in the NBA finals, and he was happy to learn that I had been a Knicks and Walt Frazier fan in the 1970s.) He was a pioneer of programming and programming education who was willing to share his experience with a young CS prof who was trying to figure out how to teach. We kept in touch by email thereafter. It was honor to call him a friend.

You can find the above quotation in A History of Test-Driven Development (TDD), as Told in Quotes, by Rob Myers. That post includes several good quotes that Myers had to cut from his upcoming book on TDD. "Of course. How else could you program?"

This tweet and this blog entry on the first use of the term "programming language" evoked responses from readers with some new history and some prior occurrences.

Doug Moen pointed me to the 1956 Fortran manual from IBM, Chapter 2 of which opens with:

Any programming language must provide for expressing numerical constants and variable quantities.

I was aware of the Fortran manual, which I link to in the notes for my compiler course, and its use of the term. But I had been linking to a document dated October 1957, and the file at fortran.com is dated October 15, 1956. That beats the January 1957 Newell and Shaw paper by a few months.

As Moen said in his email, "there must be earlier references, but it's hard to find original documents that are early enough."

The oldest candidate I have seen comes from @matt_dz. His tweet links to this 1976 Stanford tech report, "The Early Development of Programming Languages", co-authored by Knuth. On Page 26, it refers to work done by Arthur W. Burks in 1950:

In 1950, Burks sketched a so-called "Intermediate Programming Language" which was to be the step one notch above the Internal Program Language.

Unfortunately, though, this report's only passage from Burke refers to the new language as "Intermediate PL", which obscures the meaning of the 'P'. Furthermore, the title of Burke's paper uses "program" in the language's name:

Arthur W. Burks, "An intermediate program language as an aid in program synthesis", Engineering Research Institute, Report for Burroughs Adding Machine Company (Ann Arbor, Mich.: Univ. of Michigan, 1951), ii+15 pp.

The use of "program language" in this title is consistent with the terminology in Burks's previous work on an "Internal Program Language", to which Knuth et al. also refer.

Following up on the Stanford tech report, Douglas Moen found the book Mathematical Methods in Program Development, edited by Manfred Broy and Birgit Schieder. It includes a paper that attempts "to identify the first 'programming language', and the first use of that term". Here's a passage from Page 219, via Google Books:

There is not yet an indication of the term 'programming languages'. But around 1950 the term 'program' comes into wide use: 'The logic of programming electronic digital computers' (A. W. Burks 1950), 'Programme organization and initial orders for the EDSAC' (D. J. Wheeler 1950), 'A program composition technique' (H. B. Curry 1950), 'La conception du programme' (Corrado Bohm 1952), and finally 'Logical or non-mathematical programmes' (C. S. Strachey 1952).

And then, on Page 224, it comments specifically on Burks's work:

A. W. Burks ('An intermediate program language as an aid in program synthesis', 1951) was among the first to use the term program(ming) language.

The parenthetical in that phrase -- "the first to use the term program(ming) language" -- leads me to wonder if Burks may use "program language" rather than "programming language" in his 1951 paper.

Is it possible that Knuth et al. retrofitted the use of "programming language" onto Burks's language? Their report documents the early development of PL ideas, not the history of the term itself. The authors may have used a term that was in common parlance by 1976 even if Burks had not. I'd really like to find an original copy of Burks's 1951 ERI report to see if he ever uses "programming language" when talking about his Intermediate PL. Maybe after the holiday break...

In any case, the use of program language by Burks and others circa 1950 seems to be the bridge between use of the terms "program" and "language" independently and the use of "programming language" that soon became standard. If Burke and his group never used the new term for its intermediate PL, it's likely that someone else did between 1951 and release of the 1956 Fortran manual.

There is so much to learn. I'm glad that Crista Lopes tweeted her question on Sunday and that so many others have contributed to the answer!

Yesterday, Crista Lopes asked a history question on Twitter:

Hey, CS History Twitter: I just read Iverson's preface of his 1962 book carefully, and suddenly this occurred to me: did he coin the term "programming language"? Was that the first time a programming language was called "programming language"?

In a follow-up, she noted that McCarthy's CACM paper on LISP from roughly the same time called Lisp a 'programming system'", not a programming language.

I had a vague recollection from my grad school days that Newell and Simon might have used the term. I looked up IPL, the Information Processing Language they created in the mid-1950s with Shaw. IPL pioneered the notion of list processing, though at the level of assembly language. I first learned of it while devouring Newell and Simon's early work on AI and reading every thing I could find about programs such as the General Problem Solver and Logic Theorist.

That wikipedia page has a link to this unexpected cache of documents on IPL from Newell, Simon, and Shaw's days at Rand. The oldest of these is a January 1957 paper, Programming the Logic Theory Machine, by Newell and Shaw that was presented at the Western Joint Computer Conference (WJCC) the next month. It details their efforts to build computer systems to perform symbolic reasoning, as well as the language they used to code their programs.

There it is on Page 5: a section titled "Requirements for the Programming Language". They even define what they mean by programming language:

We can transform these statements about the general nature of the program of LT into a set of requirements for a programming language. By a programming language we mean a set of symbols and conventions that allows a programmer to specify to the computer what processes he wants carried out.

Other than the gendered language, that definition works pretty well even today.

The fact that Newell and Shaw defined "programming language" in this paper indicates that the term probably was not in widespread use at the time. The WJCC was a major computing conference of the day. The researchers and engineers who attended it would likely be familiar with common jargon of the industry.

Reading papers about IPL is an education across a range of ideas in computing. Researchers at the dawn of computing had to contend with -- and invent -- concepts at multiple levels of abstraction and figure out how to implement the on machines with limited size and speed. What a treat these papers are.

I love to read original papers from the beginning of our discipline, and I love to learn about the history of words. A few of my students do, too. One student stopped in after the last day of my compilers course this semester to thank me for telling stories about the history of compilers occasionally. Next semester, I teach our Programming Languages and Paradigms course again, and this little story might add a touch of color to our first days together.

All this said, I am neither a historian of computer science nor a lexicographer. If you know of an earlier occurrence of the term "programming language" than Newell and Shaw's from January 1957, I would love to hear from you by email or on Twitter.

I like tennis. The Tennis Abstract blog helps me keep up with the game and indulge my love of sports stats at the same time. An entry earlier this month gave a gentle introduction to Elo ratings as they are used for professional tennis:

One of the main purposes of any rating system is to predict the outcome of matches--something that Elo does better than most others, including the ATP and WTA rankings. The only input necessary to make a prediction is the difference between two players' ratings, which you can then plug into the following formula:

1 - (1 / (1 + (10 ^ (difference / 400))))

This formula always makes me smile. The first computer program I ever wrote because I really wanted to was a program to compute Elo ratings for my high school chess club. Over the years I've come back to Elo ratings occasionally whenever I had an itch to dabble in a new language or even an old favorite. It's like a personal kata of variable scope.

I read the Tennis Abstract piece this week as my students were finishing up their compilers for the semester and as I was beginning to think of break. Playful me wondered how I might implement the prediction formula in my students' source language. It is a simple functional language with only two data types, integers and booleans; it has no loops, no local variables, no assignments statements, and no sequences. In another old post, I referred to this sort of language as akin to an integer assembly language. And, heaven help me, I love to program in integer assembly language.

To compute even this simple formula in Klein, I need to think in terms of fractions. The only division operator performs integer division, so 1/x for any x gives 0. I also need to think carefully about how to implement the exponentiation 10 ^ (difference / 400). The difference between two players' ratings is usually less than 400 and, in any case, almost never divisible by 400. So My program will have to take an arbitrary root of 10.

Which root? Well, I can use our gcd() function (implemented using Euclid's algorithm, of course) to reduce diff/400 to its lowest terms, n/d, and then compute the dth root of 10^n. Now, how to take the dth root of an integer for an arbitrary integer d?

Fortunately, my students and I have written code like this in various integer assembly languages over the years. For instance, we have a SQRT function that uses binary search to hone in on the integer closest to the square of a given integer. Even better, one semester a student implemented a square root program that uses Newton's method:

   xn+1 = xn - f(xn)/f'(xn)

That's just what I need! I can create a more general version of the function that uses Newton's method to compute an arbitrary root of an arbitrary base. Rather than work with floating-point numbers, I will implement the function to take its guess as a fraction, represented as two integers: a numerator and a denominator.

This may seem like a lot of work, but that's what working in such a simple programming language is like. If I want my students' compilers to produce assembly language that predicts the result of a professional tennis match, I have to do the work.

This morning, I read a review of Francis Su's new popular math book, Mathematics for Human Flourishing. It reminds us that math isn't about rules and formulas:

Real math is a quest driven by curiosity and wonder. It requires creativity, aesthetic sensibilities, a penchant for mystery, and courage in the face of the unknown.

Writing my Elo rating program in Klein doesn't involve much mystery, and it requires no courage at all. It does, however, require some creativity to program under the severe constraints of a very simple language. And it's very much true that my little programming diversion is driven by curiosity and wonder. It's fun to explore ideas in a small space uses limited tools. What will I find along the way? I'll surely make design choices that reflect my personal aesthetic sensibilities as well as the pragmatic sensibilities of a virtual machine that know only integers and booleans.

As I've said before, I love to program.

This morning I read an old blog post by Michael Feathers, The Flawed Theory Behind Unit Testing. It discusses what makes TDD and Clean Room software development so effective for writing code with fewer defects: they define practices that encourage developers to work in a continuous state of reflection about their code. The post holds up well ten years on.

The line that lit my mind up, though, was this one:

John Nolan gave his developers a challenge: write OO code with no getters.

Twenty-plus years after the movement of object-oriented programming into the mainstream, this still looks like a radical challenge to many people. "Whenever possible, tell another object to do something rather than ask for its data." This sort of behavioral abstraction is the heart of OOP and the source of its design power. Yet it is rare to find big Java or C++ systems where most classes don't provide public accessors. When you open that door, client code will walk in -- even if you are the person writing the client code.

Whenever I look at a textbook intended for teaching undergraduates OOP, I look to see how it introduces encapsulation and the use of "getters" and "setters". I'm usually disappointed. Most CS faculty think doing otherwise would be too extreme for relative beginners. Once we open the door , though, it's a short step to using (gulp) instanceof to switch on kinds of objects. No wonder that some students are unimpressed and that too many students don't see any much value in OO programming, which as they learn it doesn't feel much different from what they've done before but which puts new limitations on them.

To be honest, though, it is hard to go Full Metal OOP. Nolan was working with professional programmers, not second-year students, and even so programming without getters was considered a challenge for them. There are certainly circumstances in which the forces at play may drive us toward cracking the door a bit and letting an instance variable sneak out. Experienced programmers know how to decide when the trade-off is worth it. But our understanding the downside of the trade-off is improved after we know how to design independent objects that collaborate to solve problems without knowing anything about the other objects beyond the services they provide.

Maybe we need to borrow an idea from the TDD As If You Meant It crowd and create workshops and books that teach and practice OOP as if we really meant it. Nolan's challenge above would be one of the central tenets of this approach, along with the Polymorphism Challenge and other practices that look odd to many programmers but which are, in the end, the heart of OOP and the source of its design power.

~~~~~

If you like this post, you might enjoy The Summer Smalltalk Taught Me OOP. It's isn't about OOP itself so much as about me throwing away systems until I got it right. But the reason I was throwing systems away was that I was still figuring out how to build an object-oriented system after years programming procedurally, and the reason I was learning so much was that I was learning OOP by building inside of Smalltalk and reading its standard code base. I'm guessing that code base still has a lot to teach many of us.

Jan Schauma recently posted a list of one hundred Falsehoods CS Students (Still) Believe Upon Graduating. There is much good fun here, especially for a prof who tries to help CS students get ready for the world, and a fair amount of truth, too. I will limit my brief comments to three items that have been on my mind recently even before reading this list.

18. 'Email' and 'Gmail' are synonymous.

CS grads are users, too, and their use of Gmail, and systems modeled after it, contributes to the truths of modern email: top posting all the time, with never a thought of trimming anything. Two-line messages sitting atop icebergs of text which will never be read again, only stored in the seemingly infinite space given us for free.

Of course, some of our grads end up in corporate and IT, managing email as merely one tool in a suite of lowest-common-denominator tools for corporate communication. The idea of email as a stream of text that can, for the most part, be read as such, is gone -- let alone the idea that a mail stream can be processed by programs such as procmail to great benefit.

I realize that most users don't ask for anything more than a simple Gmail filter to manage their mail experience, but I really wish it were easier for more users with programming skills to put those skills to good use. Alas, that does not fit into the corporate IT model, and not even the CS grads running many of these IT operations realize or care what is possible.

38. Employers care about which courses they took.

It's the time of year when students register for spring semester courses, so I've been meeting with a lot of students. (Twice as many as usual, covering for a colleague on sabbatical.) It's interesting to encounter students on both ends of the continuum between not caring at all what courses they take and caring a bit too much. The former are so incurious I wonder how they fell into the major at all. The latter are often more curious but sometimes are captive to the idea that they must, must, must take a specific course, even if it meets at a time they can't attend or is full by the time they register.

I do my best to help them get into these courses, either this spring or in a late semester, but I also try to do a little teaching along the way. Students will learn useful and important things in just about every course they take, if they want to, and taking any particular course does not have to be either the beginning or the end of their learning of that topic. And if the reason they think they must take a particular course is because future employers will care, they are going to be surprised. Most of the employers who interview our students are looking for well-rounded CS grads who have a solid foundation in the discipline and who can learn new things as needed.

90. Two people with a CS degree will have a very similar background and shared experience/knowledge.

This falsehood operates in a similar space to #38, but at the global level I reached at the end of my previous paragraph. Even students who take most of the same courses together will usually end their four years in the program with very different knowledge and experiences. Students connect with different things in each course, and these idiosyncratic memories build on one another in subsequent courses. They participate in different extracurricular activities and work different part-time jobs, both of shape and augment what they learn in class.

In the course of advising students over two, three, or four years, I try to help them see that their studies and other experiences are helping them to become interesting people who know more than they realize and who are individuals, different in some respects from all their classmates. They will be able to present themselves to future employers in ways that distinguish them from everyone else. That's often the key to getting the job they desire now, or perhaps one they didn't even realize they were preparing for while exploring new ideas and building their skillsets.

In an old interview at Alphachatterbox, economist Brad DeLong adds another programming tale to the annals of unintended consequences:

So the Sage Air Defense system, which never produced a single usable line of software running on any piece of hardware -- we spent more on the Sage Air Defense System than we did on the entire Manhattan Project. And it was in one sense the ultimate government Defense Department boondoggle. But on the other hand it trained a whole generation of computer programmers at a time when very little else was useful that computer programmers could exercise their skills on.

And by the time the 1960s rolled around we not only ... the fact that Sage had almost worked provided say American Airlines with the idea that maybe they should do a computer-driven reservations system for their air travel, which I think was the next big Manhattan Project-scale computer programming project.

And as that moved on the computer programmers began finding more and more things to do, especially after IBM developed its System 360.

And we were off and running.

As DeLong says earlier in the conversation, this development upended IBM president Thomas Watson's alleged claim that there was "a use for maybe five computers in the world". This famous quote is almost certainly an urban legend, but Watson would not have been as off-base as people claim even if he had said it. In the 1950s, there was not yet a widespread need for what computers did, precisely because most people did not yet understand how computing could change the landscape of every activity. Training a slew of programmers for a project that ultimately failed had the unexpected consequence of creating the intellectual and creative capital necessary to begin exploring the ubiquitous applications of computing. Money unexpectedly well spent.

Today's lecture notes for my course include a link to @KentBeck's article on Prune, which I still enjoy.

The line that merits its link in today's session is:

We wrote an ugly, fragile state machine for our typeahead, which quickly became a source of pain and shame.

My students will soon likely experience those emotions about the state machines; they are building for lexers for their semester-long compiler project. I reassure them: These emotions are normal for programmers.

Like me, you probably see references to this classic quote from Donald Rumsfeld all the time:

There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say, we know there are some things we do not know. But there are also unknown unknowns -- the ones we don't know we don't know.

I recently ran across it again in an old Epsilon Theory post that uses it to frame the difference between decision making under risk (the known unknowns) and decision-making under uncertainty (the unknown unknowns). It's a good read.

Seeing the passage again for the umpteenth time, it occurred to me that no one ever seems to talk about the fourth quadrant in that grid: the unknown knowns. A quick web search turns up a few articles such as this one, which consider unknown knowns from the perspective of others in a community: maybe there are other people who know something that you do not. But my curiosity was focused on the first-person perspective that Rumsfeld was implying. As a knower, what does it mean for something to be an unknown known?

My first thought was that this combination might not be all that useful in the real world, such as the investing context that Ben Hunt writes about in Epsilon Theory. Perhaps it doesn't make any sense to think about things you don't know that you know.

As a student of AI, though, I suddenly made an odd connection ... to explanation-based learning. As I described in a blog post twelve years ago:

Back when I taught Artificial Intelligence every year, I used to relate a story from Russell and Norvig when talking about the role knowledge plays in how an agent can learn. Here is the quote that was my inspiration, from Pages 687-688 of their 2nd edition:

Sometimes one leaps to general conclusions after only one observation. Gary Larson once drew a cartoon in which a bespectacled caveman, Zog, is roasting his lizard on the end of a pointed stick. He is watched by an amazed crowd of his less intellectual contemporaries, who have been using their bare hands to hold their victuals over the fire. This enlightening experience is enough to convince the watchers of a general principle of painless cooking.

I continued to use this story long after I had moved on from this textbook, because it is a wonderful example of explanation-based learning.

In a mathematical sense, explanation-based learning isn't learning at all. The new fact that the program learns follows directly from other facts and inference rules already in its database. In EBL, the program constructs a proof of a new fact and adds the fact to its database, so that it is ready-at-hand the next time it needs it. The program has compiled a new fact, but in principle it doesn't know anything more than it did before, because it could always have deduced that fact from things it already knows.

As I read the Epsilon Theory article, it struck me that EBL helps a learner to surface unknown knowns by using specific experiences as triggers to combine knowledge it already into a piece of knowledge that is usable immediately without having to repeat the (perhaps costly) chain of inference ever again. Deducing deep truths every time you need them can indeed be quite costly, as anyone who has ever looked at the complexity of search in logical inference systems can tell you.

When I begin to think about unknown knowns in this way, perhaps it does make sense in some real-world scenarios to think about things you don't know you know. If I can figure it all out, maybe I can finally make my fortune in the stock market.

I hope to eventually write up a reflection on my first Dagstuhl seminar, but for now I have a short story about how I encountered a new idea three times in ten days, purely by coincidence. Actually, the idea is over one hundred fifty years old but, as my brother often says, "Hey, it's new to me."

On the second day of Dagstuhl, Mark Guzdial presented a poster showing several inspirations for his current thinking about task-specific programming languages. In addition to displaying screenshots of two cool software tools, the poster included a picture of an old mechanical device that looked both familiar and strange. Telegraphy had been invented in the early 1840s, and telegraph operators needed some way to type messages. But how? The QWERTY keyboard was not created for the typewriter until the early 1870s, and no other such devices were in common use yet. To meet the need, Royal Earl House adapted a portion of a piano keyboard to create the input device for the "printing telegraph", or teleprinter. The photo on Mark's poster looked similar to the one on Wikipedia page for the teleprinter.

There was a need for a keyboard thirty years before anyone designed a standard typing interface, so telegraphers adapted an existing tool to fit their needs. What if we are in that same thirty-year gap in the design of programming languages? This has been one of Mark's inspirations as he works with non-computer scientists on task-specific programming languages. I had never seen an 1870s teleprinter before and thought its keyboard to be a rather ingenious way to solve a very specific problem with a tool borrowed from another domain.

When Dagstuhl ended, my wife and I spent another ten days in Europe on a much-needed vacation. Our first stop was Paris, and on our first full day there we visited the museum of the Conservatoire National des Arts et Métiers. As we moved into the more recent exhibits of the museum, what should I see but...

... a Hughes teleprinter with piano-style keyboard, circa 1975. Déjà vu! I snapped a photo, even though the device was behind glass, and planned to share it with Mark when I got home.

We concluded our vacation with a few days in Martinici, Montenegro, the hometown of a department colleague and his wife. They still have a lot of family in the old country and spend their summers there working and relaxing. On our last day in this beautiful country, we visited its national historical museum, which is part of the National Museum of Montenegro in the royal capital of Cetinje. One of the country's most influential princes was a collector of modern technology, and many of his artifacts are in the museum -- including:

This full-desk teleprinter was close enough to touch and examine up close. (I didn't touch!) The piano keyboard on the device shows the wear of heavy use, which brings to mind each of my laptops' keyboards after a couple of years. Again, I snapped a photo, this time in fading light, and made a note to pass it on.

In ten days, I went from never having heard much about a "printing telegraph" to seeing a photo of one, hearing how it is an inspiration for research in programming language design, and then seeing two such devices that had been used in the 19th-century heyday of telegraphy. It was an unexpected intersection of my professional and personal lives. I must say, though, that having heard Mark's story made the museum pieces leap into my attention in a way that they might not have otherwise. The coincidence added a spark to each encounter.

As so often, Marvin Minsky loved to tell us about the beauty of programming. Kids love to play with construction sets like Legos, TinkerToys, and Erector sets. Programming provides an infinite construction kit: you never run out of parts!

In the linked essay, which was published as a preface to a 1986 book about Logo, Minsky tells several stories. One of the stories relates that once, as a small child, he built a large tower out of TinkerToys. The grownups who saw it were "terribly impressed". He inferred from their reaction that:

some adults just can't understand how you can build whatever you want, so long as you don't run out of sticks and spools.

Kids get it, though. Why do so many of us grow out of this simple understanding as we get older? Whatever its cause, this gap between children's imaginations and the imaginations of adults around them creates a new sort of problem when we give the children a programming language such as Logo or Scratch. Many kids take to these languages just as they do to Legos and TinkerToys: they're off to the races making things, limited only by their expansive imaginations. The memory on today's computers is so large that children never run out of raw material for writing programs. But adults often don't possess the vocabulary for talking with the children about their creations!

... many adults just don't have words to talk about such things -- and maybe, no procedures in their heads to help them think of them. They just do not know what to think when little kids converse about "representations" and "simulations" and "recursive procedures". Be tolerant. Adults have enough problems of their own.

Minsky thinks there are a few key ideas that everyone should know about computation. He highlights two:

Computer programs are societies. Making a big computer program is putting together little programs.

Any computer can be programmed to do anything that any other computer can do--or that any other kind of "society of processes" can do.

He explains the second using ideas pioneered by Alan Turing and long championed in the popular sphere by Douglas Hofstadter. Check out this blog post, which reflects on a talk Hofstadter gave at my university celebrating the Turing centennial.

The inability of even educated adults to appreciate computing is a symptom of a more general problem. As Minsky says toward the end of his essay, People who don't appreciate how simple things can grow into entire worlds are missing something important. If you don't understand how simple things can grow into complex systems, it's hard to understand much at all about modern science, including how quantum mechanics accounts for what we see in the world and even how evolution works.

You can usually do well by reading Minsky; this essay is a fine example of that. It comes linked to an afterword written by Alan Kay, another computer scientist with a lot to say about both the beauty of computing and its essential role in a modern understanding of the world. Check both out.

He applied to switch his major from mathematics to computer science, but the authorities forbade it. "That is what tipped me to accept the idea that perhaps Russia is not the best place for me," he says. "When they wouldn't allow me to study computer science."

-- Sergey Aleynikov, as told to Michael Lewis and reported in Chapter 5 of Flash Boys.

In The Narluga Is a Strange Beluga-Narwhal Hybrid, Ed Yong tells the story of a narluga, the offspring of a beluga father and a narwhal mother:

Most of its DNA was a half-and-half mix between the two species, but its mitochondrial DNA -- a secondary set that animals inherit only from their mothers -- was entirely narwhal.

This strange hybrid had a mouth and teeth unlike either of its parents, the product of an unexpected DNA computation:

It's as if someone took the program for creating a narwhal tusk and ran it in a beluga's mouth.

The analogy to software doesn't end there, though...

There's something faintly magical about that. This fluky merger between two species ended up with a mouth that doesn't normally exist in nature but still found a way of using it. It lived neither like a beluga nor a narwhal, but it lived nonetheless.

Fluky and abnormal; a one-off, yet it adapts and survives. That sounds like a lot of the software I've used over the years and, if I'm honest, like some of the software I've written, too.

That said, nature is amazing.

I wasn't getting any work done today on my to-do list, so I decided to write some code.

One of my learning exercises to open the Summer of Joy is to solve the term frequency problem from Crista Lopes's Exercises in Programming Style. Joy is a little like Scheme: it has a lot of cool operations, especially higher-order operators, but it doesn't have much in the way of practical level tools for basic tasks like I/O. To compute term frequencies on an arbitrary file, I need to read the file onto Joy's stack.

I played around with Joy's low-level I/O operators for a while and built a new operator called readfile, which expects the pathname for an input file on top of the stack:

    DEFINE readfile ==
        (* 1 *)  [] swap "r" fopen
        (* 2 *)  [feof not] [fgets swap swonsd] while
        (* 3 *)  fclose.

The first line leaves an empty list and an input stream object on the stack. Line 2 reads lines from the file and conses them onto the list until it reaches EOF, leaving a list of lines under the input stream object on the stack. The last line closes the stream and pops it from the stack.

This may not seem like a big deal, but I was beaming when I got it working. First of all, this is my first while in Joy, which requires two quoted programs. Second, and more meaningful to me, the loop body not only works in terms of the dip idiom I mentioned in my previous post, it even uses the higher-order swonsd operator to implement the idiom. This must be how I felt the first time I mapped an anonymous lambda over a list in Scheme.

readfile leaves a list of lines on the stack. Unfortunately, the list is in reverse order: the last line of the file is the front of the list. Besides, given that Joy is a stack-based language, I think I'd like to have the lines on the stack itself. So I noodled around some more and implemented the operator pushlist:

    DEFINE pushlist ==
        (* 1 *)  [ null not ] [ uncons ] while
        (* 2 *)  pop.

Look at me... I get one loop working, so I write another. The loop on Line 1 iterates over a list, repeatedly taking (head . tail) and pushing head and tail onto the stack in that order. Line 2 pops the empty list after the loop terminates. The result is a stack with the lines from the file in order, first line on top:

    line-n ... line-3 line-2 line-1

Put readfile and pushlist together:

    DEFINE fileToStack == readfile pushlist.

I'll admit that I'm pleased with myself, but I suspect that this code can be improved. Joy has a lot of dandy higher-order operators. There is probably a better way to implement pushlist and maybe even readfile. I won't be surprised if there is a more idiomatic way to implement the two that makes the two operations plug together with less rework. And I may find that I don't want to leave bare lines of text on the stack after all and would prefer having a list of lines. Learning whether I can improve the code, and how, are tasks for another day.

My next job for solving the term frequency problem is to split the lines into individual words, canonicalize them, and filter out stop words. Right now, all I know is that I have two more functions in my toolbox, I learned a little Joy, and writing some code made my day better.

"Elementary" ideas are really hard & need to be revisited
& explored & re-revisited at all levels of mathematical
sophistication. Doing so actually moves math forward.
-- James Tanton

Three summers ago, I spent a couple of weeks re-familiarizing myself with the concatenative programming language Joy and trying to go a little deeper with the style. I even wrote a few blog entries, including a few quick lessons I learned in my first week with the language. Several of those lessons hold up, but please don't look at the code linked there; it is the raw code of a beginner who doesn't yet get the idioms of the style or the language. Then other duties at work and home pulled me away, and I never made the time to get back to my studies.

I have dubbed this the Summer of Joy. I can't devote the entire summer to concatenative programming, but I'm making a conscious effort to spend a couple of days each week in real study and practice. After only one week, I have created enough forward momentum that I think about problems and solutions at random times of the day, such as while walking home or making dinner. I think that's a good sign.

An even better sign is that I'm starting to grok some of the idioms of the style. Joy is different from other concatenative languages like Forth and Factor, but it shares the mindset of using stack operators effectively to shape the data a program uses. I'm finally starting to think in terms of dip, an operator that enables a program to manipulate data just below the top of the stack. As a result, a lot of my code is getting smaller and beginning to look like idiomatic Joy. When I really master dip and begin to think in terms of other "dipping" operators, I'll know I'm really on my way.

One of my goals for the summer is to write a Joy compiler from scratch that I can use as a demonstration in my fall compiler course. Right now, though, I'm still in Joy user mode and am getting the itch for a different sort of language tool... As my Joy skills get better, I find myself refactoring short programs I've written in the past. How can I be sure that I'm not breaking the code? I need unit tests!

So my first bit of tool building is to implement a simple JoyUnit. As a tentative step in this direction, I created the simplest version of RackUnit's check-equal? function possible:

    DEFINE check-equal == [i] dip i =.

    [ 2 square ] [ 4 ] check-equal.

This is, of course, only the beginning. Next I'll add a message to display when a test fails, so that I can tell at a glance which tests have failed. Eventually I'll want my JoyUnit to support tests as objects that can be organized into suites, so that their results can be tallied, inspected, and reported on. But for now, YAGNI. With even a few simple functions like this one, I am able to run tests and keep my code clean. That's a good feeling.

To top it all off, implementing JoyUnit will force me to practice writing Joy and push me to extend my understanding while growing the set of programming tools I have at my disposal. That's another good feeling, and one that might help me keep my momentum as a busy summer moves on.

In an answer on Quora from earlier this year:

There are several modern APL-like languages today -- such as J and K -- but I would criticize them as being too much like the classic APL. It is possible to extract what is really great from APL and use it in new language designs without being so tied to the past. This would be a great project for some grad students of today: what does the APL perspective mean today, and what kind of great programming language could be inspired by it?

The APL perspective was more radical even twenty years ago, before MapReduce became a thing and before functional programming ascended. When I was an undergrad, though, it seemed otherworldly: setting up a structure, passing it through a sequence of operators that changed its shape, and then passing it through a sequence of operators that folded up a result. We knew we weren't programming in Fortran anymore.

I'm still fascinated by APL, but I haven't done a lot with it in the intervening years. These days I'm still thinking about concatenative programming in languages like Forth, Factor, and Joy, a project I reinitiated (and last blogged about) three summers ago. Most concatenative languages work with an implicit stack, which gives it a very different feel from APL's dataflow style. I can imagine, though, that working in the concision and abstraction of concatenative languages for a while will spark my interest in diving back into APL-style programming some day.

Kay's full answer is worth a read if only for the story in which he connects Iverson's APL notation, and its effect on how we understand computer systems, to the evolution of Maxwell's equations. Over the years, I've heard Kay talk about McCarthy's Lisp interpreter as akin to Maxwell's equations, too. In some ways, the analogy works even better with APL, though it seems that the lessons of Lisp have had a larger historical effect to date.

Perhaps that will change? Alas, as Kay says in the paragraph that precedes his challenge:

As always, time has moved on. Programming language ideas move much slower, and programmers move almost not at all.

Kay often comes off as pessimistic, but after all the computing history he has lived through (and created!), he has earned whatever pessimism he feels. As usual, reading one of his essays makes me want to buckle down and do something that would make him proud.

Dick Gabriel writes, in Lessons From The Science of Nothing At All:

Nevertheless, the spreadsheet was something never seen before. A chart indicating the 64 greatest events in accounting and business history contains VisiCalc.

This reminds me of a line from The Tao of Pooh:

Take the path to Nothing, and go Nowhere until you reach it.

A lot of research is like this, but even more so in computer science, where the things we produce are generally made out of nothing. Often, like VisiCalc, they aren't really like anything we've ever seen or used before either.

From James Propp's Prof. Engel's Marvelously Improbable Machines:

Chip-firing has been rediscovered independently in three different academic communities: mathematics, physics, and computer science. However, its original discovery by Engel is in the field of math education, and I strongly feel that Engel deserves credit for having been the first to slide chips around following these sorts of rules. This isn't just for Engel's sake as an individual; it's also for the sake of the kind of work that Engel did, blending his expertise in mathematics with his experience in the classroom. We often think of mathematical sophistication as something that leads practitioners to create concepts that can only be understood by experts, but at the highest reaches of mathematical research, there's a love of clarity that sees the pinnacle of sophistication as being the achievement of hard-won simplicity in settings where before there was only complexity.

First of all, Petri nets! I encountered Petri nets for the first time in a computer architecture course, probably as a master's student, and it immediately became my favorite thing about the course. I was never much into hardware and architecture, but Petri nets showed me a connection back to graph theory, which I loved. Later, I studied how to apply temporal logic to modeling hardware and found another way to appreciate my architecture courses.

But I really love the point that Propp makes in this paragraph and the section it opens. Most people think of research and teaching as being different sort of activities. But the kind of thinking one does in one often crosses over into the other. The sophistication that researchers have and use help us make sense of complex ideas and, at their best, help us communicate that understanding to a wide audience, not just to researchers at the same level of sophistication. The focus that teachers put on communicating challenging ideas to relative novices can encourage us to seek new formulations for a complex idea and ways to construct more complex ideas out of the new formulations. Sometimes, that can lead to an insight we can use in research.

In recent years, my research has benefited a couple times from trying to explain and demonstrate concatenative programming, as in Forth and Joy, to my undergraduate students. These haven't been breakthroughs of the sort that Engel made with his probability machines, but they've certainly help me grasp in new ways ideas I'd been struggling with.

Propp argues convincingly that it's important that we tell stories like Engel's and recognize that his breakthrough came as a result of his work in the classroom. This might encourage more researchers to engage as deeply with their teaching as with their research. Everyone will benefit.

Do you know any examples similar to the one Propp relates, but in the field of computer science? If so, I would love to hear about them. Drop me a line via email or Twitter.

Oh, and if you like Petri nets, probability, or fun stories about teaching, do read Propp's entire piece. It's good fun and quite informative.

Andy Ko, in SIGCSE 2019 report:

I always have to warn my students before they attend SIGCSE that it's not a place for deep and nuanced discussions about learning, nor is it a place to get critical feedback about their ideas.

It is, however, a wonderful place to be immersed in the concerns of CS teachers and their perceptions of evidence.

I'm not sure I agree that one can't have deep, nuanced discussions about learning at SIGCSE, but it certainly is not a research conference. It is a great place to talk to and learn from people in the trenches teaching CS courses, with a strong emphasis on the early courses. I have picked up a lot of effective, creative, and inspiring ideas at SIGCSE over the years. Putting them onto sure scientific footing is part of my job when I get back.

~~~~~

Stephen Kell, in Some Were Meant for C (PDF), an Onward! 2017 essay:

Unless we can understand the real reasons why programmers continue to use C, we risk researchers continuing to solve a set of problems that is incomplete and/or irrelevant, while practitioners continue to use flawed tools.

For example,

... "faster safe languages" is seen as the Important Research Problem, not better integration.

... whereas Kell believes that C's superiority in the realm of integration is one of the main reasons that C remains a dominant, essential systems language.

Even with the freedom granted by tenure, academic culture tends to restrict what research gets done. One cause is a desire to publish in the best venues, which encourages work that is valued by certain communities. Another reason is that academic research tends to attract people who are interested in a certain kind of clean problem. CS isn't exactly "round, spherical chickens in a vacuum" territory, but... Language support for system integration, interop, and migration can seem like a grungier sort of work than most researchers envisioned when they went to grad school.

"Some Were Meant for C" is an elegant paper, just the sort of work, I imagine, that Richard Gabriel had when envisioned the essays track at Onward. Well worth a read.

This morning, while riding the exercise bike, I read two items within twenty minutes or so that formed a nice juxtaposition for our age. First came The Cost of Distraction, an old blog post by L.M. Sacasas that reconsiders Kurt Vonnegut's classic story, "Harrison Bergeron" (*). In the story, it is 2081, and the Handicapper General of the United States ensures equality across the land by offsetting any advantages any individual has over the rest of the citizenry. In particular, those of above-average intelligence are required to wear little earpieces that periodically emit high-pitched sounds to obliterate any thoughts in progress. The mentally- and physically-gifted Harrison rebels, to an ugly end.

Soon after came Ian Bogost's Apple's AirPods Are an Omen, an article from last year that explores the cultural changes that are likely to ensue as more and more people wear AirPods and their ilk. ("Apple's most successful products have always done far more than just make money, even if they've raked in a lot of it....") AirPods free the wearer in so many ways, but they also bind us to ubiquitous distraction. Will we ever have a free moment to think deeply when our phones and laptops now reside in our heads?

As Sacasas says near the end of his post,

In the world of 2081 imagined by Vonnegut, the distracting technology is ruthlessly imposed by a government agency. We, however, have more or less happily assimilated ourselves to a way of life that provides us with regular and constant distraction. We have done so because we tend to see our tools as enhancements.

Who needs a Handicapper General when we all walk down to the nearest Apple Store or Best Buy and pop distraction devices into our own ears?

Don't get me wrong. I'm a computer scientist, and I love to program. I also love the productivity my digital tools provide me, as well as the pleasure and comfort they afford. I'm not opposed to AirBuds, and I may be tempted to get a pair someday. But there's a reason I don't carry a smart phone and that the only iPod I've ever owned is 1GB first-gen Shuffle. Downtime is valuable, too.

(*) By now, even occasional readers know that I'm a big Vonnegut fan who wrote a short eulogy on the occasion of his death, nearly named this blog after one of his short stories, and returns to him frequently.

I love this line from Organizational Debt:

So my proposal for Rust 2019 is not that big of a deal, I guess: we just need to redesign our decision making process, reorganize our governance structures, establish new norms of communication, and find a way to redirect a significant amount of capital toward Rust contributors.

A solid understatement usually makes me smile. Decision-making processes, governance structure, norms of communication, and compensation for open-source developers... no big deal, indeed. We all await the results. If the results come with advice that generalizes beyond a single project, especially the open-source compensation thing, all the better.

Communication is a big part of the recommendation for 2019. Changing how communication works is tough in any organization, let alone an organization with distributed membership and leadership. In every growing organization there eventually comes the time for intentional systems of communication:

But we've long since reached the point where coordinating our design vision by osmosis is not working well. We need an active and intentional circulatory system for information, patterns, and frameworks of decision making related to design.

I'm not a member of the Rust community, only an observer. But I know that the language inspires some programmers, and I learned a bit about its tool chain and community support a couple of years ago when an ambitious student used it successfully to implement his compiler in my course. It's the sort of language we need, being created in what looks to be an admirable way. I wish the Rust team well as they tackle their organizational debt and tackle their growing pains.

More consonance with Paul Romer, via his conversation with Tyler Cowen: They were discussing how hard it is to learn read English than other languages, due to its confusing orthography and in particular the mismatch between sounds and their spellings. We could adopt a more rational way to spell words, but it's hard to change the orthography of large language spoken by a large, scattered population. Romer offered a computational solution:

It would be a trivial translation problem to let some people write in one spelling form, others in the other because it would be word-for-word translation. I could write you an email in rationalized spelling, and I could put it through the plug-in so you get it in traditional spelling. This idea that it's impossible to change spelling I think is wrong. It's just, it's hard, and we should -- if we want to consider this -- we should think carefully about the mechanisms.

This sounds similar to a common problem and solution in the software development world. Programmers working in teams often disagree about the orthography of code, not the spelling so much as its layout, the use of whitespace, and the placement of punctuation. Being programmers, we often address this problem computationally. Team members can stylize their code anyway they see fit but, when they check it into the common repository, they run it through a language formatter. Often, these formatters are built into our IDEs. Nowadays, some languages even come with a built-in formatting tool, such as Go and gofmt.

Romer's email plug-in would play a similar role in human-to-human communication, enabling writers to use different spelling systems concurrently. This would make it possible to introduce a more rational way to spell words without having to migrate everyone to the new system all at once. There are still challenges to making such a big change, but they could be handled in an evolutionary way.

Maybe Romer's study of Python is turning him into a computationalist! Certainly, being a programmer can help a person recognize the possibility of a computational solution.

Add this idea to his recent discovery of C.S. Peirce, and I am feeling some intellectual kinship to Romer, at least as much as an ordinary CS prof can feel kinship to a Nobel Prize-winning economist. Then, to top it all off, he lists Slaughterhouse-Five as one of his two favorite novels. Long-time readers know I'm a big Vonnegut fan and nearly named this blog for one of his short stories. Between Peirce and Vonnegut, I can at least say that Romer and I share some of the same reading interests. I like his tastes.

This morning I read Tyler Cowen's conversation with Paul Romer. At one point, Romer talks about being introduced to C.S. Peirce, who had deep insights into "abstraction and how we use abstraction to communicate" (a topic Romer and Cowen discuss earlier in the interview). Romer is clearly enamored with Peirce's work, but he's also fascinated by the fact that, after a long career thinking about a set of topics, he could stumble upon a trove of ideas that he didn't even know existed:

... one of the joys of reading -- that's not a novel -- but one of the joys of reading, and to me slightly frightening thing, is that there's so much out there, and that a hundred years later, you can discover somebody who has so many things to say that can be helpful for somebody like me trying to understand, how do we use abstraction? How do we communicate clearly?

But the joy of scholarship -- I think it's a joy of maybe any life in the modern world -- that through reading, we can get access to the thoughts of another person, and then you can sample from the thoughts that are most relevant to you or that are the most powerful in some sense.

This process, he says, is the foundation for how we transmit knowledge within a culture and across time. It's how we grow and share our understanding of the world. This is a source of great joy for scholars and, really, for anyone who can read. It's why so many people love books.

Romer's interest in Peirce calls to mind my own fascination with his work. As Romer notes, Peirce had a "much more sophisticated sense about how science proceeds than the positivist sort of machine that people describe". I discovered Peirce through an epistemology course in graduate school. His pragmatic view of knowledge, along with William James's views, greatly influenced how I thought about knowledge. That, in turn, redefined the trajectory by which I approached my research in knowledge-based systems and AI. Peirce and James helped me make sense of how people use knowledge, and how computer programs might.

So I feel a great kinship with Romer in his discovery of Peirce, and the joy he finds in scholarship.

Friday was a long day, but a good one. The talks I saw were a bit more diverse than on Day One: a couple on language design (though even one of those covered a lot more ground than that), one on AI, one on organizations and work-life, and one on theory:

• "All the Languages Together", by Amal Ahmed, discussed a problem that occurs in multi-language systems: when code written in one language invalidates the guarantees made by code written in the other. Most languages are not designed with this sort of interoperability baked in, and their FFI escape hatches make anything possible within foreign code. As a potential solution, Ahmed offered principled escape hatches designed with specific language features in mind. The proposed technique seems like it could be a lot of work, but the research is in its early stages, so we will learn more as she and her students implement the idea.

This talk is yet another example of how so many of our challenges in software engineering are a result of programming language design. It's good to see more language designers taking issues like these seriously, but we have a long way to go.

• I really liked Ashley Williams's talk on on the evolution of async in Javascript and Rust. This kind of talk is right up my alley... Williams invoked philosophy, morality, and cognitive science as she reviewed how two different language communities incorporated asynchronous primitives into their languages. Programming languages are designed, to be sure, but they are also the result of "contingent turns of history" (a lá Foucault). Even though this turned out to be more of a talk about the Rust community than I had expected, I enjoyed every minute. Besides, how can you not like a speaker who says, "Yes, sometimes I'll dress up as a crab to teach."?

(My students should not expect a change in my wardrobe any time soon...)

• I also enjoyed "For AI, by AI", by Connor Walsh. The talk's subtitle, "Freedom & Evolution of the Algopoetic Avant-Garde", was a bit disorienting, as was its cold open, but the off-kilter structure of the talk was easy enough to discern once Walsh got going: first, a historical review of humans making computers write poetry, followed by a look at something I didn't know existed... a community of algorithmic poets — programs — that write, review, and curate poetry without human intervention. It's a new thing, of Walsh's creation, that looks pretty cool to someone who became drunk on the promise of AI many years ago.

I saw two other talks the second day:

• the after-lunch address by Philip Wadler, "Categories for the Working Hacker", which I wrote about separately
• Rachel Krol's Some Things May Never Get Fixed, about how organizations work and how developers can thrive despite how they work

I wish I had more to say about the last talk but, with commitments at home, the long drive beckoned. So, I departed early, sadly, hopped in my car, headed west, and joined the mass exodus that is St. Louis traffic on a Friday afternoon. After getting past the main crush, I was able to relax a bit with the rest of Zen and the Art of Motorcycle Maintenance.

Even a short day at Strange Loop is a big win. This was the tenth Strange Loop, and I think I've been to five, or at least that's what my blog seems to tell me. It is awesome to have a conference like this in Middle America. We who live here benefit from the opportunities it affords us, and maybe folks in the rest of the world get a chance to see that not all great computing ideas and technology happen on the coasts of the US.

When is Strange Loop 2019?

Philip Wadler is a rockstar to the Strange Loop crowd. His 2015 talk on propositions as types introduced not a few developers to one of computer science's great unities. This year, he returned to add a third idea to what is really a triumvirate: categories. With a little help from his audience, he showed that category theory has elements which correspond directly to ...

• logical 'and', which models the record (or tuple, or pair) data type
• logical 'or', which models the union (or variant record) data type
• a map, which models the function data type

Wadler is an entertaining teacher; I recommend the video of his talk! But he is also as quotable as any CS prof I've encountered in a long while. Here is a smattering of his great lines from "Categories for the Working Hacker":

If you can use math to do something, do it. It will make your life better.

That's the great thing about math. It lets you see something obvious after only thirty or forty years.

Pick your favorite algebra. If you don't have one, get one.

Let's do that in Java. That's what you should always do when you learn a new idea: do it in Java.

That's what category theory is really about: avoiding traffic jams.

Sums are the secret origin of folds.

If you don't understand this, I don't mind, because it's Java.

While watching the presentation, I created a one-liner of my own: Surprise! If you do something that matches exactly what Haskell does, Haskell code will be much shorter than Java code.

This was a very good talk; I enjoyed it quite a bit. However, I also left the room with a couple of nagging questions. The talk was titled "Categories for the Working Hacker", and it did a nice job of presenting some basic ideas from category theory in a way that most any developer could understand, even one without much background in math. But... How does this knowledge make one a better hacker? Armed with this new, entertaining knowledge, what are software developers able to do that they couldn't do before?

I have my own ideas for answers to these questions, but I would love to hear Wadler's take.

This talk, the first of the afternoon on Day 1, opened with a familiar image: René Magritte's "this is not a pipe" painting, next to a picture of an actual pipe from some e-commerce site. Throughout the talk, speaker David Schmüdde returned to the distinction between thing and referent as he looked at the phenomenon of software users who used -- misused -- software to do something other than intended by the designer. The things they did were, or became, art.

First, a disclaimer: David is a former student of mine, now a friend, and one of my favorite people in the world. I still have in my music carousel a CD labeled "Schmudde Music!!" that he made for me just before he graduated and headed off to a master's program in music at Northwestern.

I often say in my conference reports that I can't do a talk justice in a blog entry, but it's even more true of a talk such as this one. Schmüdde demonstrated multiple works of art, both static and dynamic, which created a vibe that loses most of its zing when linearized in text. So I'll limit myself here to a few stray observations and impressions from the talk, hoping that you'll be intrigued enough to watch the video when it's posted.

Art is a technological endeavor. Rembrandt and hip hop don't exist without advances in art-making technology.

Misuse can be a form of creative experimentation. Check out Jodi, a website created in 1995 and still available. In the browser, it seems to be a work of ASCII art, but show the page source... (That's a lot harder these days than it was in 1995.) Now that is ASCII art.

Schmüdde talked about another work of from the same era, entitled Rain. It used slowness -- of the network, of the browser -- as a feature. Old HTML (or was it a bug in an old version of Netscape Navigator?) allowed one HEAD tag in a file with multiple BODY tags. The artist created such a document that, when loaded in sequence, gave the appearance of rain falling in the browser. Misusing the tools under the conditions of the day enabled the artist to create an animation before animated GIFs, Flash, and other forms of animation existed.

The talk followed with examples and demos of other forms of software misuse, which could:

• find bugs in a system
• lead to new system features
• illuminate a system in ways not anticipated by the software's creator

Accidental misuse is life. We expect it. Intentional misuse is, or can be, art. It can surprise us.

What does art preservation look like for these works? The original hardware and software systems often are obsolete or, more likely, gone. To me, this is one of the great things about computers: we can simulate just about anything. Digital art preservation becomes a matter of simulating the systems or interactions that existed at the time the art was created. We are back to Magritte's pipe... This is not a work of art; it is a pointer to a work of art.

It is, of course, harder to recreate the experience of the art from the time it was created, but isn't this true of all art? Each of us experiences a work of art anew each time we encounter it. Our experience is never the same as the experience of those who were present when the work was first unveiled. It's often not even the same experience we ourselves had yesterday.

Schmüdde closed with a gentle plea to the technologists in the room to allow more art into their process. This is a new talk, and he was a little concerned about his ending. He may find a less abrupt way to end in the future, but to be honest, I though what he did this time worked well enough for the day.

Even taking my friendship with the speaker into account, this was the talk of the conference for me. It blended software, users, technology, ideas, programming, art, the making of things, and exploring software at its margins. These ideas may appear at the margin, but they often lie at the core of the work. And even when they don't, they surprise us or delight us or make us think.

This talk was a solid example of what makes Strange Loop a great conference every year. There were a couple of other talks this year that gave me a similar vibe, for example, Hannah Davis's "Generating Music..." talk on Day 1 and Ashley Williams's "A Tale of Two asyncs" talk on Day 2. The conference delivers top-notch technical content but also invites speakers who use technology, and explore its development, in ways that go beyond what you find in most CS classrooms.

For me, Day One of the conference ended better than most: over a beer with David at Flannery's with good conversation, both about ideas from his talk and about old times, families, and the future. A good day.

The opening keynote this year was by Simon Peyton Jones of Microsoft Research, well known in the programming languages for Haskell and many other things. But his talk was about something considerably less academic: "Shaping Our Children's Education in Computing", a ten-year project to reform the teaching of computing in the UK primary and secondary schools. It was a wonderful talk, full of history, practical advice, lessons learned, and philosophy of computing. Rather than try to summarize everything Peyton Jones said, I will let you watch the video when it is posted (which will be as early as next week, I think).

I would, though, like to highlight one particular part of the talk, the way he describes computer science to a non-CS audience. This is an essential skill for anyone who wants to introduce CS to folks in education, government, and the wider community who often see CS as either hopelessly arcane or as nothing more than a technology or a set of tools.

Peyton Jones characterized computing as being about information, computation, and communication. For each, he shared one or two ways to discuss the idea with an educated but non-technical audience. For example:

• Information.   Show two images, say the Mona Lisa and a line drawing of a five-pointed star. Ask which contains more information. How can we tell? How can we compare the amounts? How might we write that information down?


• Computation.   Use a problem that everyone can relate to, such as planning a trip to visit all the US state capitals in the fewest miles or sorting a set of numbers. For the latter, he used one of the activities from CS Unplugged on sorting networks as an example.


• Communication.   Here, Peyton Jones used the elegant and simple idea underlying the Diffie Hellman algorithm for sharing secret as his primary example. It is simple and elegant, yet it's not at all obvious to most people who don't already know it that the problem can be solved at all!

In all three cases, it helps greatly to use examples from many disciplines and to ask questions that encourage the audience to ask their own questions, form their own hypotheses, and create their own experiments. The best examples and questions actually enable people to engage with computing through their own curiosity and inquisitiveness. We are fascinated by computing; other people can be, too.

There is a huge push in the US these days for everyone to learn how to program. This creates a tension among many of us computer scientists, who know that programming isn't everything that we do and that its details can obscure CS as much as they illuminate it. I thought that Peyton Jones used a very nice analogy to express the relationship between programming and CS more broadly: Programming is to computer science as lab work is to physics. Yes, you could probably take lab work out of physics and still have physics, but doing so would eviscerate the discipline. It would also take away a lot of what draws people to the discipline. So it is with programming and computer science. But we have to walk a thin line, because programming is seductive and can ultimately distract us from the ideas that make programming so valuable in the first place.

Finally, I liked Peyton Jones's simple summary of the reasons that everyone should learn a little computer science:

• Everyone should be able to create digital media, not just consume it.
• Everyone should be able to understand their tools, not just use them.
• People should know that technology is not magic.

Oh, and yes, a few people will get jobs that use programming skills and computing knowledge. People in government and business love to hear that part.

Regular readers of this blog know that I am a sucker for aphorisms. Peyton Jones dropped a few on us, most earnestly when encouraging his audience to participate in the arduous task of introducing and reforming the teaching CS in the schools:

• "If you wait for policy to change, you'll just grow old. Get on with it."
• "There is no 'them'. There is only us."

It's easy to admire great researchers who have invested so much time and energy into solving real-world problems, especially in our schools. As long as this post is, it covers only a few minutes from the middle of the talk. My selection and bare-bones outline don't do justice to Peyton Jones's presentation or his message. Go watch the talk when the video goes up. It was a great way to start Strange Loop.

Last Wednesday morning, I hopped in my car and headed south to Strange Loop 2018. It had been a few years since I'd listened to Zen and the Art of Motorcycle Maintenance on a conference drive, so I popped it into the tapedeck (!) once I got out of town and fell into the story. My top-level goal while listening to Zen was similar to my top-level goal for attending Strange Loop this year: to experience it at a high level; not to get bogged down in so many details that I lost sight of the bigger messages. Even so, though, a few quotes stuck in my mind from the drive down. The first is an old friend, one of my favorite lines from all of literature:

Assembly of Japanese bicycle require great peace of mind.

The other was the intellectual breakthrough that unified Phaedrus's philosophy:

Quality is not an object; it is an event.

On the first day of the conference, I saw mostly a mixture of compiler talks and art talks, including:

• @mraleph's "Six Years of Dart", in which he reminisced on the evolution of the language, its ecosystem, and its JIT. I took at least one cool idea from this talk. When he compared the performance of two JITs, he gave a histogram comparing their relative performances, rather than an average improvement. A new system often does better on some programs and worse on others. An average not only loses information; it may mislead.

• Jason Dagit's "Your Secrets are Safe with Julia", about a system that explores the use of homomorphic encryption to to compile secure programs. In this context, the key element of security is privacy. As Dagit pointed out, "trust is not transitive", which is especially important when it comes to sharing a person's health data.

• I just loved Hannah Davis's talk on "Generating Music From Emotion". She taught me about data sonification and its various forms. She also demonstrated some of her attempts to tease multiple dimensions of human emotion out of large datasets and to use these dimensions to generate music that reflects the data's meaning. Very cool stuff. She also showed the short video Dragon Baby, which made me laugh out loud.

• I also really enjoyed "Hackett: A Metaprogrammable Haskell", by Alexis King. I've read about this project on the Racket mailing list for a few years and have long admired King's ability in posts there to present complex ideas clearly and logically. This talk did a great job of explaining that Haskell deserves a powerful macro system like Racket's, that Racket's macro system deserves a powerful type system like Haskell's, and that integrating the two is more challenging than simply adding a stage to the compiler pipeline.

I saw two other talks the first day:

• the opening keynote address by Simon Peyton Jones, "Shaping Our Children's Education in Computing" [ link ]
• David Schmüdde, "Misuser" [ link ]

I had almost forgotten how many different kinds of cool ideas I can encounter in a single day at Strange Loop. Thursday was a perfect reminder.

This fall I am again teaching our course in compiler development. Working in teams of two or three, students will implement from scratch a complete compiler for a simple functional language that consists of little more than integers, booleans, an if statement, and recursive functions. Such a language isn't suitable for much, but it works great for writing programs that do simple arithmetic and number theory. In the past, I likened it to an integer assembly language. This semester, my students are compiling a Pascal-like language of this sort that call Flair.

If you've read my blog much in the falls over the last decade or so, you may recall that I love to write code in the languages for which my students write their compilers. It makes the language seem more real to them and to me, gives us all more opportunities to master the language, and gives us interesting test cases for their scanners, parsers, type checkers, and code generators. In recent years I've blogged about some of my explorations in these languages, including programs to compute Farey numbers and excellent numbers, as well as trying to solve one of my daughter's AP calculus problems.

When I run into a problem, I usually get an itch to write a program, and in the fall I want to write it in my students' language.

Yesterday, I began writing my first new Flair program of the semester. I ran across this tweet from James Tanton, which starts:

N is "special" if, in binary, N has a 1s and b 0s and a & b are each factors of N (so non-zero).

So, 10 is special because:

• In binary, 10 is 1010.
• 1010 contains two 1s and two 0s.
• Two is a factor of 10.

9 is not special because its binary rep also contains two 1s and two 0s, but two is not a factor of 9. 3 is not special because its binary rep has no 0s at all.

My first thought upon seeing this tweet was, "I can write a Flair program to determine if a number is special." And that is what I started to do.

Flair doesn't have loops, so I usually start every new program by mapping out the functions I will need simply to implement the definition. This makes sure that I don't spend much time implementing loops that I don't need. I ended up writing headers and default bodies for three utility functions:

• convert a decimal number to binary
• count the number of times a particular digits occurs in a number
• determine if a number x divides evenly into a number n

With these helpers, I was ready to apply the definition of specialness:

    return divides(count(1, to_binary(n)), n)
       and divides(count(0, to_binary(n)), n)

Calling to_binary on the same argument is wasteful, but Flair doesn't have local variables, either. So I added one more helper to implement the design pattern "Function Call as Variable Assignment", apply_definition:

    function apply_definition(binary_n : integer, n : integer) : boolean

    return apply_definition(to_binary(n), n)

This is only the beginning. I still have a lot of work to do to implement to_binary, count and divides, using recursive function calls to simulate loops. This is another essential design pattern in Flair-like languages.

As I prepared to discuss my new program in class today, I found bug: My divides test was checking for factors of binary_n, not the decimal n. I also renamed a function and one of its parameters. Explaining my programs to students, a generalization of rubber duck debugging, often helps me see ways to make a program better. That's one of the reasons I like to teach.

Today I asked my students to please write me a Flair compiler so that I can run my program. The course is officially underway.

Or: How to Learn Physics, Professional Golfer Edition

Bryson DeChambeau is a professional golfer, in the news recently for consecutive wins in the FedExCup playoff series. But he can also claim an unusual distinction as a student of physics:

In high school, he rewrote his physics textbook.

DeChambeau borrowed the textbook from the library and wrote down everything from the 180-page book into a three-ring binder. He explains: "My parents could have bought one for me, but they had done so much for me in golf that I didn't want to bother them in asking for a $200 book. ... By writing it down myself I was able to understand things on a whole comprehensive level.

I imagine that copying texts word-for-word was a more common learning strategy back when books were harder to come by, and perhaps it will become more common again as textbook prices rise and rise. There is certainly something to be said for it. Writing by hand takes time, and all the while our brains can absorb terms, make connections among concepts, and process the material into long-term memory. Zed Shaw argues for this as a great way to learn computer programming, implementing it as a pedagogical strategy in his "Learn <x> the Hard Way" series of books. (See Learn Python the Hard Way as an example.)

I don't think I've ever copied a textbook word-for-word, and I never copied computer programs from "Byte" magazine, but I do have similar experiences in note taking. I took elaborate notes all through high school, college, and grad school. In grad school, I usually rewrote all of my class notes -- by hand; no home PC -- as I reviewed them in the day or two after class. My clean, rewritten notes had other benefits, too. In a graduate graph algorithms course, they drew the attention of a classmate who became one of my best friends and were part of what attracted the attention of the course's professor, who asked me to consider joining his research group. (I was tempted... Graph algorithms was one of my favorite courses and research areas!)

I'm not sure many students these days benefit from this low-tech strategy. Most students who take detailed notes in my course seem to type rather than write which, if what I've read is correct, has fewer cognitive advantages. But at least those students are engaging with the material consciously. So few students seem to take detailed notes at all these days, and that's a shame. Without notes, it is harder to review ideas, to remember what they found challenging or puzzling in the moment, and to rehearse what they encounter in class into their long-term memories. Then again, maybe I'm just having a "kids these days" moment.

Anyway, I applaud DeChambeau for saving his parents a few dollars and for the achievement of copying an entire physics text. He even realized, perhaps after the fact, that it was an excellent learning strategy.

(The above passage is from The 11 Most Unusual Things About Bryson DeChambeau. He sounds like an interesting guy.)

If you don't sit facing the window, you could be in any town.

I read that line this morning in Maybe the Cumberland Gap just swallows you whole, where it is a bittersweet observation of the similarities among so many dying towns across Appalachia. It's a really good read, mostly sad but a little hopeful, that applies beyond one region or even one country.

My mind is self-centered, though, and immediately reframed the sentence in a way that cast light on my good fortune.

I just downloaded a couple of papers on return-oriented programming so that I can begin working with an undergraduate on an ambitious research project. I have a homework assignment to grade sitting in my class folder, the first of the semester. This weekend, I'll begin to revise a couple of lectures for my compiler course, on NFAs and DFAs and scanning text. As always, there is a pile of department work to do on my desk and in my mind.

I live in Cedar Falls, Iowa, but if I don't sit facing the window, I could be in Ames or Iowa City, East Lansing or Durham, Boston or Berkeley. And I like the view out of my office window very much, thank you, so I don't even want to trade.

Heading into a three-day weekend, I realize again how fortunate I am. Do I put my good fortune to good enough use?

As I a way to get into the right frame of mind for the new semester and the next iteration of my compiler course, I read Michael Hicks's Software Security is a Programming Languages Issue this morning. Hicks incorporates software security into his courses on the principles of programming languages, with two lectures on security before having students study and use Rust. The article has links to lecture slides and supporting material, which makes it a post worth bookmarking.

I started thinking about adding LangSec to my course late in the spring semester, as I brainstormed topics that might spice the rest of the course up for both me and my students. However, time was short, so I stuck with a couple of standalone sessions on topics outside the main outline: optimization and concatenative languages. They worked fine but left me with an itch for something new.

I think I'll use the course Hicks and his colleagues teach as a starting point for figuring out how I might add to next spring's course. Students are interested in security, it's undoubtedly an essential issue for today's grads, and it is a great way to demonstrate how the design of programming languages is more than just the syntax of a loop or the lambda calculus.

Hicks's discussion of Rust also connects with my fall course. Two years ago, an advanced undergrad used Rust as the implementation language for his compiler. He didn't know the language but wanted to pair it with Haskell in his toolbox. The first few weeks of the project were a struggle as he wrestled with mastering ownership and figuring out some new programming patterns. Eventually he hit a nice groove and produced a working compiler with only a couple of small holes.

I was surprised how easy it was for me install the tools I needed to compile, test, and explore his code. That experience increased my interest in learning the language, too. Adding it to my spring course would give me the last big push I need to buckle down.

This summer has been a blur of administrative stuff, expected and unexpected. The fall semester brings the respite of work I really enjoy: teaching compilers and writing some code. Hurray!

I just finished David Mamet's Three Uses of the Knife, a wide-ranging short book with the subtitle: "on the nature and purpose of drama". It is an extended essay on how we create and experience drama -- and how these are, in the case of great drama, the same journey.

Even though the book is only eighty or so pages, Mamet characterizes drama in so many ways that you'll have to either assemble a definition yourself or accept the ambiguity. Among them, he says that the job of drama and art is to "delight" us and that "the cleansing lesson of the drama is, at its highest, the worthlessness of reason."

Mamet clearly believes that drama is central to other parts of life. Here's a cynical example, about politics:

The vote is our ticket to the drama, and the politician's quest to eradicate "fill in the blank", is no different from the promise of the superstar of the summer movie to subdue the villain -- both promise us diversion for the price of a ticket and a suspension of disbelief.

As reader, I found myself using the book's points to ruminate about other parts of life, too. Consider the first line of the second essay:

The problems of the second half are not the problems of the first half.

Mamet uses this to launch into a consideration of the second act of a drama, which he holds equally to be a consideration of writing the second act of a drama. But with fall semester almost upon us, my thoughts jumped immediately to teaching a class. The problems of teaching the second half of a class are quite different from the problems of teaching the first half. The start of a course requires the instructor to lay the foundation of a topic while often convincing students that they are capable of learning it. By midterm, the problems include maintaining the students' interest as their energy flags and the work of the semester begins to overwhelm them. The instructor's energy -- my energy -- begins to flag, too, which echoes Mamet's claim that the journey of the creator and the audience are often substantially the same.

A theme throughout the book is how people immerse themselves in story, suspending their disbelief, even creating story when they need it to soothe their unease. Late in the book, he connects this theme to religious experience as well. Here's one example:

In suspending their disbelief -- in suspending their reason, if you will -- for a moment, the viewers [of a magic show] were rewarded. They committed an act of faith, or of submission. And like those who rise refreshed from prayers, their prayers were answered. For the purpose of the prayer was not, finally, to bring about intercession in the material world, but to lay down, for the time of the prayer, one's confusion and rage and sorrow at one's own powerlessness.

This all makes the book sound pretty serious. It's a quick read, though, and Mamet writes with humor, too. It feels light even as it seems to be a philosophical work.

The following paragraph wasn't intended as humorous but made me, a computer scientist, chuckle:

The human mind cannot create a progression of random numbers. Years ago computer programs were created to do so; recently it has been discovered that they were flawed -- the numbers were not truly random. Our intelligence was incapable of creating a random progression and therefore of programming a computer to do so.

This reminded me of a comment that my cognitive psychology prof left on the back of an essay I wrote in class. He wrote something to the effect, "This paper gets several of the particulars incorrect, but then that wasn't the point. It tells the right story well." That's how I felt about this paragraph: it is wrong on a couple of important facts, but it advances the important story Mamet is telling ... about the human propensity to tell stories, and especially to create order out of our experiences.

Oh, and thanks to Anna Gát for bringing the book to my attention, in a tweet to Michael Nielsen. Gát has been one of my favorite new follows on Twitter in the last few months. She seems to read a variety of cool stuff and tweet about it. I like that.

• Why I Don't Love Gödel, Escher, Bach

I saw a lot of favorable links to this post a while back and finally got around to it. Meh. I generally agree with the author: GEB is verbose in places, there's a lot of unnecessary name checking, and the dialogues that lead off each chapter are often tedious. I even trust the author's assertion that Hofstadter's forays beyond math, logic, and computers are shallow.

So what? Things don't have to be perfect for me to like them, or for them to make me think. GEB was a swirl of ideas that caused me to think and helped me make a few connections. I'm sure if I read the book now that I would feel differently about it, but reading it when I did, as an undergrad CS major thinking about AI and the future, it energized me.

I do thank the author for his pointer (in a footnote) to Vi Hart's wonderful Twelve Tones. You should watch it. Zombie Schonberg!

• The Web Aesthetic

This post wasn't quite what I expected, but even a few years old it has something to say to web designers today.

Everything on the web ultimately needs to degrade down to plain text (images require alt text; videos require transcripts), so the text editor might just become the most powerful app in the designer's toolbox.

• XP Challenge: Compilers

People outside the XP community often don't realize how seriously the popularizers of XP explored the limitations of their own ideas. This page documents one of several challenges that push XP values and practices to the limits: When do they break down? Can they be adapted successfully to the task? What are the consequences of applying them in such circumstances?

Re-reading this old wiki page was worth it if only for this great line from Ron Jeffries:

The point of XP is to win, not die bravely.

Yes.

How's this for a job description: "The successful candidate will be able to hit a fungo, throw batting practice, and program in SQL."

We decided that in the minor leagues, we would hire an extra coach at each level. The requirements for that coach were that he had to be able to hit a fungo, throw batting practice, and program in SQL. It's a hard universe to find where those intersect, but we were able to find enough of them--players who had played in college that maybe played one year in the minors who had a technical background and could understand analytics.

The technical skills are not enough by themselves, though. In order to turn a baseball franchise into a data-informed enterprise, you have to change the culture of the team in the trenches, working with the people who have to change their own behavior. Management must take the time necessary to guide the organization's evolution.

The above passage is from How the Houston Astros are winning through advanced analytics. I picked it up expecting a baseball article, or perhaps a data analytics article, but it reads like a typical McKinsey Report piece. It was an interesting read, but for different reasons than I had imagined.

Some thoughts on studying computer science from Gian-Carlo Rota:

A large fraction of MIT undergraduates major in computer science or at least acquire extensive computer skills that are applicable in other fields. In their second year, they catch on to the fact that their required courses in computer science do not provide the whole story. Not because of deficiencies in the syllabus; quite the opposite. The undergraduate curriculum in computer science at MIT is probably the most progressive and advanced such curriculum anywhere. Rather, the students learn that side by side with required courses there is another, hidden curriculum consisting of new ideas just coming into use, new techniques and that spread like wildfire, opening up unsuspected applications that will eventually be adopted into the official curriculum.

Keeping up with this hidden curriculum is what will enable a computer scientist to stay ahead in the field. Those who do not become computer scientists to the second degree risk turning into programmers who will only implement the ideas of others.

MIT is, of course, an exceptional school, but I think Rota's comments apply to computer science at most schools. So much learning of CS happens in the spaces between courses: in the lab, in the student lounge, at meetings of student clubs, at part-time jobs, .... That can sometimes be a challenge for students who don't have much curiosity, or develop one as they are exposed to new topics.

As profs, we encourage students to be aware of all that is going on in computer science beyond the classroom and to take part in the ambient curriculum to the extent they are able. Students who become computer scientists only to the first degree can certainly find good jobs and professional success, but there are more opportunities open at the second degree. CS can also be a lot more fun there.

From an old New Yorker article about Spotify:

YouTube, which is by far the largest streaming-music site in the world (it wasn't designed that way--that's just what it became)...

Companies starting in one line of business and evolving into something else is nothing new. I mean, The Connecticut Leather Company became Coleco and made video game consoles. But there's something about software that make this sort of evolution seem so normal. We build a computer system to solve one problem and find that our users -- who have needs and desires that neither we nor they fully comprehend -- use it to solve a different problem. Interesting times. Don't hem yourself in, and don't hem your software in, or the people who use it.

I just read Pops, Michael Chabon's recent book of essays on fatherhood. The first essay, which originally appeared as an article in GQ, includes this parenthetical about his tour of Paris fashion week with his son:

-- a special mapping algorithm seemed to have been employed to ensure that every show was held as far as possible from its predecessor and its successor on the schedule --

My first thought was to approach this problem greedily: Start with the first show, then select a second show that is as far away as possible, then select a third show that is as far away as possible from that one, and so on, until all of the shows had been scheduled. But then I figured that a schedule so generated might seem laborious to travel at first, when there are plenty of faraway shows to choose from, but it might eventually start to seem pretty reasonable as the only shows left to schedule are relatively close.

We can generate a more wholly sort of unsatisfactory schedule by maximizing the total travel time of the circuit. That's the Traveling Salesman Problem, inverted. Taking this approach, our algorithm is quite simple. We start with the usual nxn matrix d, where d[i,j] equals the distance between shows i and j. Then:

1. Replace the distance between every two show locations d[i,j], with -(d[i,j]), its additive inverse.
2. Call your favorite TSP solver with the new graph.

Easy! I leave implementation of the individual steps as an exercise for the reader.

(By the way, Chabon's article is a sweet story about an already appreciative dad coming to appreciate his son even better. If you like that sort of thing, give it a read.)

In Adaptation-Executers, not Fitness-Maximizers, Eliezer Yudkowsky talks about how evolution has led to human traits that may no longer be ideal in the our current environment. He also talks about tools, though, and this summary sentence made me think of programs:

So the screwdriver's cause, and its shape, and its consequence, and its various meanings, are all different things; and only one of these things is found within the screwdriver itself.

I often fall victim to thinking that the meaning of software is at least somewhat inherent in its code, but that really is what the designer intended as its use -- a mix of what Yudkowsky calls its cause and its consequence. These are things that exist only in the mind of the designer and the user, not in the computational constructs that constitute the code.

When approaching new software, especially a complex piece of code with many parts, it's helpful to remember that it doesn't really have objective meaning or consequences, only those intended by its designers and those exercised by its users. Over time, the users' conception tends to drive the designers' conception as they put the software to particular uses and even modify it to better suit these new uses.

Perhaps software is best thought of as an adaptation-executer, too, and not as a fitness-maximizer.

A lot of people I know have been discussing last week's NY Times op-ed about recent advances in neural networks and what they mean for AI. The article even sparked conversation among colleagues from my grad school research lab and among my PhD advisor's colleagues from when he was in grad school. It seems that many of us are frequently asked by non-CS folks what we think about recent advances in AI, from AlphaGo to voice recognition to self-driving cars. My answers sound similar to what some of my old friends say. Are we now afraid of AI being able to take over the world? Um, no. Do you think that the goals of AI are finally within reach? No. Much remains to be done.

I rate my personal interest in recent deep learning advances as meh. I'm not as down on the current work as the authors of the Times piece seem to be; I'm just not all that interested. It's wonderful as an exercise in engineering: building focused systems that solve a single problem. But, as the article points out, that's the key. These systems work in limited domains, to solve limited problems. When I want one of these problems to be solved, I am thankful that people have figured out how to solve and make it commercially available for us to use. Self-driving cars, for instance, have the potential to make the world safer and to improve the quality of my own life.

My interest in AI, though, has always been at a higher level: understanding how intelligence works. Are there general principles that govern intelligent behavior, independent of hardware or implementation? One of the first things to attract me to AI was the idea of writing a program that could play chess. That's an engineering problem in a very narrow domain. But I soon found myself drawn to cognitive issues: problem-solving strategies, reflection, explanation, conversation, symbolic reasoning. Cognitive psychology was one of my favorite courses in grad school in large part because it tried to connect low-level behaviors in the human brain connected to the symbolic level. AlphaGo is exceedingly cool as a game player, but it can't talk to me about Go, and for me that's a lot of the fun of playing.

In an email message earlier this week, my quick take on all this work was: We've forgotten the knowledge level. And for me, the knowledge level is what's most interesting about AI.

That one-liner oversimplifies things, as most one-liners do. The AI world hasn't forgotten the knowledge level so much as moved away from it for a while in order to capitalize on advances in math and processing power. The results have been some impressive computer systems. I do hope that the pendulum swings back soon as AI researchers step back from these achievements and builds some theories at the knowledge level. I understand that this may not be possible, but I'm not ready to give up on the dream yet.

I've been reading my way through Frank Chimero's talks online and ran across a great bit on maps and interaction design in What Screens Want. One of the paragraphs made me think about the abstractions that show up in CS courses:

When I realized that, a little light went off in my head: a map's biases do service to one need, but distort everything else. Meaning, they misinform and confuse those with different needs.

CS courses are full of abstractions and models of complex systems. We use examples, often simplified, to expose or emphasize a single facet a system, as a way to help students cut through the complexity. For example, compilers and full-strength interpreters are complicated programs, so we start with simple interpreters operating over simple languages. Students get their feet wet without drowning in detail.

In the service of trying not to overwhelm students, though, we run the risk of distorting how they think about the parts we left out. Worse, we sometimes distort even their thinking about the part we're focusing on, because they don't see its connections to the more complete picture. There is an art to identifying abstractions, creating examples, and sequencing instruction. Done well, we can minimize the distortions and help students come to understand the whole with small steps and incremental increases in size and complexity.

At least that's what I think on my good days. There are days and even entire semesters when things don't seem to progress as smoothly as I hope or as smoothly as past experience has led me to expect. Those days, I feel like I'm doing violence to an idea when I create an abstraction or adopt a simplifying assumption. Students don't seem to be grokking the terrain, so change the map. We try different problems or work through more examples. It's hard to find the balance sometimes between adding enough to help and not adding so much as to overwhelm.

The best teachers I've encountered know how to approach this challenge. More importantly, they seem to enjoy the challenge. I'm guessing that teachers who don't enjoy it must be frustrated a lot. I enjoy it, and even so there are times when this challenge frustrates me.

For years now, I've been listening to many people -- smart, accomplished people -- feverishly proclaim that functional programming is here to right the wrongs of object-oriented programming. For many years before that, I heard many people -- smart, accomplished people -- feverishly proclaim that object-oriented programming was superior to functional programming, an academic toy, for building real software.

Alas, I don't have a home in the battle between OOP and FP. I like and program in both styles. So it's nice whenever I come across something like Alan Kay's recent post on Quora, in response to the question, "Why is functional programming seen as the opposite of OOP rather than an addition to it?" He closes with a paragraph I could take on as my credo:

So: both OOP and functional computation can be completely compatible (and should be!). There is no reason to munge state in objects, and there is no reason to invent "monads" in FP. We just have to realize that "computers are simulators" and figure out what to simulate.

As in many things, Kay encourages to go beyond today's pop culture of programming to create a computational medium that incorporates big ideas from the beginning of our discipline. While we work on those ideas, I'll continue to write programs in both styles, and to enjoy them both. With any luck, I'll bounce between mindsets long enough that I eventually attain enlightenment, like the venerable master Qc Na. (See the koan at the bottom of that link.)

Oh: Kay really closes his post with

I will be giving a talk on these ideas in July in Amsterdam (at the "CurryOn" conference).

If that's not a reason to go to Amsterdam for a few days, I don't know what is. Some of the other speakers looks pretty good, too.

Yesterday morning I read The Good Room, a talk Frank Chimero gave last month. Early on in the talk, Chimero says:

Let me start by stating something obvious: in the last decade, technology has transformed from a tool that we use to a place where we live.

This sentence jumped off the page both for the content of the assertion and for the decade time frame with which he bounds it. In the fall of 2003, I taught a capstone course for non-majors that is part of my university's liberal arts core. The course, titled "Environment, Technology, and Society", brings students from all majors on campus together in a course near the end of their studies, to apply their general education and various disciplinary expertises to problems of some currency in the world. As you might guess from the title, the course focuses on problems at the intersection of the natural environment, technology, and people.

My offering of the course put on a twist on the usual course content. We focused on the man-made environment we all live in, which even by 2003 had begun to include spaces carved out on the internet and web. The only textbook for the course was Donald Norman's The Design of Everyday Things, which I think every university graduate should have read. The topics for the course, though, had a decided IT flavor: the effect of the Internet on everyday life, e-commerce, spam, intellectual property, software warranties, sociable robots, AI in law and medicine, privacy, and free software. We closed with a discussion of what an educated citizen of the 21st century ought to know about the online world in which they would live in order to prosper as individuals and as a society.

The change in topic didn't excite everyone. A few came to the course looking forward to a comfortable "save the environment" vibe and were resistant to considering technology they didn't understand. But most were taking the course with no intellectual investment at all, as a required general education course they didn't care about and just needed to check off the list. In a strange way, their resignation enabled them to engage with the new ideas and actually ask some interesting questions about their future.

Looking back now after fifteen years , the course design looks pretty good. I should probably offer to teach it again, updated appropriately, of course, and see where young people of 2018 see themselves in the technological world. As Chimero argues in his talk, we need to do a better job building the places we want to live in -- and that we want our children to live in. Privacy, online peer pressure, and bullying all turned out differently than I expected in 2003. Our young people are worse off for those differences, though I think most have learned ways to live online in spite of the bad neighborhoods. Maybe they can help us build better places to live.

Chimero's talk is educational, entertaining, and quotable throughout. I tweeted one quote: "How does a city wish to be? Look to the library. A library is the gift a city gives to itself." There were many other lines I marked for myself, including:

• Penn Station "resembles what Kafka would write about if he had the chance to see a derelict shopping mall." (I'm a big Kafka fan.)
• "The wrong roads are being paved in an increasingly automated culture that values ease."

Theoretical physicist Marcelo Gleiser, in The More We Know, the More Mystery There Is:

But even if we did [bring the four fundamental forces together in a common framework], and it's a big if right now, this "unified theory" would be limited. For how could we be certain that a more powerful accelerator or dark matter detector wouldn't find evidence of new forces and particles that are not part of the current unification? We can't. So, dreamers of a final theory need to recalibrate their expectations and, perhaps, learn a bit of epistemology. To understand how we know is essential to understand how much we can know.

People are often surprised to hear that, in all my years of school, my favorite course was probably PHL 440 Epistemology, which I took in grad school as a cognate to my CS courses. I certainly enjoyed the CS courses I took as a grad student, and as an undergrad, too, and but my study of AI was enhanced significantly by courses in epistemology and cognitive psychology. The prof for PHL 440, Dr. Rich Hall, became a close advisor to my graduate work and a member of my dissertation committee. Dr. Hall introduced me to the work of Stephen Toulmin, whose model of argument influenced my work immensely.

I still have the primary volume of readings that Dr. Hall assigned in the course. Looking back now, I'd forgotten how many of W.V.O. Quine's papers we'd read... but I enjoyed them all. The course challenged most of my assumptions about what it means "to know". As I came to appreciate different views of what knowledge might be and how we come by it, my expectations of human behavior -- and my expectations for what AI could be -- changed. As Gleiser suggests, to understand how we know is essential to understanding what we can know, and how much.

Gleiser's epistemology meshes pretty well with my pragmatic view of science: it is descriptive, within a particular framework and necessarily limited by experience. This view may be why I gravitated to the pragmatists in my epistemology course (Peirce, James, Rorty), or perhaps the pragmatists persuaded me better than the others.

In any case, the Gleiser interview is a delightful and interesting read throughout. His humble of science may get you thinking about epistemology, too.

... and, yes, that's the person for whom a quine in programming is named. Thanks to Douglas Hofstadter for coining the term and for giving us programming nuts a puzzle to solve in every new language we learn.

Racket -- "A Programmable Programming Language" -- is the cover story for next month's Communications of the ACM. The new issue is already featured on the magazine's home page, including a short video in which Matthias Felleisen explains the idea of code as more than a machine artifact.

My love of Racket is no surprise to readers of this blog. Still one of my favorite old posts here is The Racket Way, a write-up of my notes from Matthew Flatt's talk of the same name at StrangeLoop 2012. As I said in that post, this was a deceptively impressive talk. I think that's especially fitting, because Racket is a deceptively impressive language.

One last little bit of love from a recent message to the Racket users mailing list... Stewart Mackenzie describes his feelings about the seamless interweaving of Racket and Typed Racket via a #lang directive:

So far my dive into Racket has positive. It's magical how I can switch from untyped Racket to typed Racket simply by changing #lang. Banging out my thoughts in a beautiful lisp 1, wave a finger, then finger crack to type check. Just sublime.

That's what you get when your programming language is as programmable as your application.

Guile Scheme guru Andy Wingo recently wrote a post about langsec, the idea that we can bake system security into our programs by using languages that support proof of correctness. Compilers can then be tools for enforcing security. Wingo is a big fan of the langsec approach but, in light of the Spectre and Meltdown vulnerabilities, is pessimistic that it really matter anymore. If bad actors can exploit the hardware that executes our programs, then proving that the code is secure doesn't do much good.

I've read a few blog posts and tweets that say Wingo is too pessimistic, that efforts to make our languages produce more secure code will still pay off. I think my favorite such remark, though, is a comment on Wingo's post itself, by Thomas Dullien:

I think this is too dark a post, but it shows a useful shock: Computer Science likes to live in proximity to pure mathematics, but it lives between EE and mathematics. And neglecting the EE side is dangerous - which not only Spectre showed, but which should have been obvious at the latest when Rowhammer hit.

There's actual physics happening, and we need to be aware of it.

It's easy for academics, and even programmers who work atop an endless stack of frameworks, to start thinking of programs as pure abstractions. But computer programs, unlike mathematical proofs, come into contact with real, live hardware. It's good to be reminded sometimes that computer science isn't math; it lives somewhere between math and engineering. That is good in so many ways, but it also has its downsides. We should keep that in mind.

In this Los Angeles Review of Books interview, novelist Jenny Offill says:

I was reading a poet from the Tang dynasty... One of his lines from, I don't know, page 812, was "No new feelings". When I read that I laughed out loud. People have been writing about the same things since the invention of the written word. The only originality comes from the language itself.

After a week revising lecture notes and rewriting a recruiting talk intended for high school students and their parents, I know just what Offill and that Tang poet mean. I sometimes feel the same way about code.

This was posted on the Racket mailing list recently:

"The Little Schemer" starts slow for people who have programmed before, but seeing that I am only half-way through and already gained some interesting knowledge from it, one should not underestimate the acceleration in this book.

The Little Schemer is the only textbook I assign in my Programming Languages course. These students usually have only a little experience: often three semesters, two in Python and one in Java; sometimes just the two in Python. A few of the students who work in the way the authors intend have an A-ha! experience while reading it. Or maybe they are just lucky... Other students have only a WTF? experience.

Still, I assign the book, with hope. It's relatively inexpensive and so worth a chance that a few students can use it to grok recursion, along with a way of thinking about writing functions that they haven't seen in courses or textbooks before. The book accelerates from the most basic ideas of programming to "interesting" knowledge in a relatively short number of pages. Students who buy in to the premise, hang on for the ride, and practice the ideas in their own code soon find that they, too, have accelerated as programmers.

In this interview with Adam Grant, Walter Jacobson talks about some of the things he learned while writing biographies of Benjamin Franklin, Albert Einstein, Steve Jobs, and Leonardo da Vinci. A common theme is that all four were curious and interested in a wide range of topics. Toward the end of the interview, Jacobson says:

We of humanities backgrounds are always doing the lecture, like, "We need to put the 'A' in 'STEM', and you've got to learn the arts and the humanities." And you get big applause when you talk about the importance of that.

But we also have to meet halfway and learn the beauty of math. Because people tell me, "I can't believe somebody doesn't know the difference between Mozart and Haydn, or the difference between Lear and Macbeth." And I say, "Yeah, but do you know the difference between a resistor and a transistor? Do you know the difference between an integral and a differential equation?" They go, "Oh no, I don't do math, I don't do science." I say, "Yeah, but you know what, an integral equation is just as beautiful as a brush stroke on the Mona Lisa." You've got to learn that they're all beautiful.

Appreciating that beauty made Leonardo a better artist and Jobs a better technologist. I would like for the students who graduate from our CS program to know some literature, history, and art and appreciate their beauty. I'd also like for the students who graduate from our university with degrees in literature, history, art, and especially education to have some knowledge of calculus, the Turing machine, and recombinant DNA, and appreciate their beauty.

I've not read either of Helen DeWitt's novels, but this interview from 2011 makes her sound like a technophile. When struggling to write, she finds inspiration in her tools:

What is to be done?

Well, there are all sorts of technical problems to address. So I go into Illustrator and spend hours grappling with the pen tool. Or I open up the statistical graphics package R and start setting up plots. Or (purists will be appalled) I start playing around with charts in Excel.

... suddenly I discover a brilliant graphic solution to a problem I've been grappling with for years! How to display poker hands graphically in a way that sets a series of strong hands next to the slightly better hands that win.

Other times she feels the need for a prop, a lá Olivier:

I may have a vague idea about a character -- he is learning Japanese at an early age, say. But I don't know how to make this work formally, I don't know what to do with the narrative. I then buy some software that lets me input Japanese within my word-processing program. I start playing around, I come up with bits of Japanese. And suddenly I see that I can make visible the development of the character just by using a succession of kanji! I don't cut out text -- I have eliminated the need for 20 pages of text just by using this software.

Then she drops a hint about a work in progress, along with a familiar name:

Stolen Luck is a book about poker using Tuftean information design to give readers a feel for both the game and the mathematics.

Dewitt sounds like my kind of person. I wonder if I would like her novels. Maybe I'll try Lightning Rods first; it sounds like an easier read than The Last Samurai.

I recently read an old conversation between Neil Gaiman and Kazuo Ishiguro that started out as a discussion of genre but covered a lot of ground, including how stories mutate over time, and that the time scale of stories is so much larger than that of human lives. Here are a few of the passages about stories and time:

NG   Stories are long-lived organisms. They're bigger and older than we are.

NG   You sit there reading Pepys, and just for a minute, you kind of get to be 350, 400 years older than you are.

KI   There's an interesting emotional tension that comes because of the mismatch of lifespans in your work, because an event that might be tragic for one of us may not be so for the long-lived being.

KI   I'm often asked what my attitude is to film, theatrical, radio adaptations of my novels. It's very nice to have my story go out there, and if it's in a different form, I want the thing to mutate slightly. I don't want it to be an exact translation of my novel. I want it to be slightly different, because in a very vain kind of way, as a storyteller, I want my story to become like public property, so that it gains the status where people feel they can actually change it around and use it to express different things.

This last comment by Ishiguro made me think of open-source software. It can be adapted by anyone for almost any use. When we fork a repo and adapt it, how often does it grow into something new and considerably different? I often tell my compiler students about the long, mutated life of P-code, which was related by Chris Clark in a 1999 SIGPLAN Notices article:

P-code is an example [compiler intermediate representation] that took on a life of its own. It was invented by Nicklaus Wirth as the IL for the ETH Pascal compiler. Many variants of that compiler arose [Ne179], including the USCD Pascal compiler that was used at Stanford to define an optimizer [Cho83]. Chow's compiler evolved into the MIPS compiler suite, which was the basis for one of the DEC C compilers -- acc. That compiler did not parse the same language nor use any code from the ETH compiler, but the IL survived.

That's not software really, but a language processed by several generations of software. What are other great examples of software and languages that mutated and evolved?

We have no history with 100-year-old software yet, of course, let alone 300- or 1000-year-old software. Will we ever? Software is connected to the technology of a given time in ways that stories are not. Maybe, though, an idea that is embodied in a piece of software today could mutate and live on in new software or new technology many decades from now? The internet is a system of hardware and software that is already evolving into new forms. Will the world wide web continue to have life in a mutated form many years hence?

The Gaiman/Ishiguro conversation turned out to be more than I expected when I first found it. Good stuff. Oh, and as I wrap up this post, this passage resonates with me:

NG   I know that when I create a story, I never know what's going to work. Sometimes I will do something that I think was just a bit of fun, and people will love it and it catches fire, and sometimes I will work very hard on something that I think people will love, and it just fades: it never quite finds its people.

Been there, done that, my friend. This pretty well describes my experience blogging and tweeting all these years, and even writing for my students. I am a less reliable predictor of what will connect with readers than my big ego would ever have guessed.

Earlier this week I learned about Kaprekar numbers when someone re-tweeted this my way:

Kaprekar numbers are numbers whose square in that base can be split into 2 parts that add up to the original number

So, 9 is a Kaprekar number, because 9 squared is 81 and 8+1 equals 9. 7777 is, too, because 7777 squared is 60481729 and 6048 + 1729 equals 7777.

This is the sort of numerical problem that is well-suited for the language my students are writing a compiler for this semester. I'm always looking out for fun little problems that I can to test their creations. In previous semesters, I've blogged about computing Farey sequences and excellent numbers for just this purpose.

Who am I kidding. I just like to program, even in small language that feels like integer assembly language, and these problems are fun!

So I sat down and wrote Klein functions to determine if a given number is a Kaprekar number and to generate all of the Kaprekar numbers less than a given number. I made one small change to the definition, though: I consider only numbers whose squares consist of an even-number of digits and thus can be split in half, a lá excellent numbers.

Until we have a complete compiler for our class language, I always like to write a reference program in a language such as Python so that I can validate my logic. I had a couple of meetings this morning, which gave just the time I needed to port my solution to a Klein-like subset of Python.

When I finished my program, I still had a few meeting minutes available, so I started generating longer and longer Kaprekar numbers. I noticed that there are a bunch more 6-digit Kaprekar numbers than at any previous length:

 1: 1
 2: 3
 3: 2
 4: 5
 5: 4
 6: 24

I started wondering why that might be... and then realized that there are a lot more 6-digit numbers overall than 5-digit -- ten times as many, of course. (D'oh!) My embarrassing moment of innumeracy didn't kill my curiosity, though. How does that 24 compare to the trend line of Kaprekar numbers by length?

 1: 1    of        9  0.11111111
 2: 3    of       90  0.03333333
 3: 2    of      900  0.00222222
 4: 5    of     9000  0.00055555
 5: 4    of    90000  0.00004444
 6: 24   of   900000  0.00002666

There is a recognizable drop-off at each length up to six, where the percentage is an order of magnitude different than expected. Are 6-digit numbers a blip or a sign of a change in the curve? I ran another round. This took much longer, because my Klein-like Python program has to compute operations like length recursively and has no data structures for caching results. Eventually, I had a count:

 7: 6    of  9000000  0.00000066

A big drop, back in line with the earlier trend. One more round, even slower.

 8: 21   of 90000000  0.00000023

Another blip in the rate of decline. This calls for some more experimentation... There is a bit more fun to have the next time I have a couple of meetings to fill.

Image: courtesy of the Simpsons wiki.

sneaking up on the Gates-Hillman Complex from Forbes St., Pittsburgh, PA

I had a couple of hours yesterday between the end of the CS education summit and my shuttle to the airport. Rather than sit in front of a computer for two more hours, I decided to take advantage of my location, wander over to the Carnegie Mellon campus, and take a leisurely walk through the Gates Center for Computer Science. I'm glad I did.

At the beginning of my tour, I was literally walking in circles, from the ground-level entrance shown in its Wikipedia photo up to where the CS offices seem to begin, up on the fourth floor. This is one of those buildings that looks odd from the outside and is quite confusing on the inside, at least to the uninitiated. But everyone inside seemed to feel at home, so maybe it works.

It didn't take long before my mind was flooded by memories of my time as a doctoral student. Michigan State's CS program isn't as big as CMU's, but everywhere I looked I saw familiar images: Students sitting in their labs or in their offices, one or two or six at a time, hacking code on big monitors, talking shop, or relaxing. The modern world was on display, too, with students lounging comfy chairs or sitting in a little coffee shop, laptops open and earbuds in place. That was never my experience as a student, but I know it now as a faculty member.

I love to wander academic halls, in any department, really, and read what is posted on office doors and hallway walls. At CMU, I encountered the names of several people whose work I know and admire. They came from many generations... David Touretzky, whose Lisp textbook taught me a few things about programming. Jean Yang, whose work on programming languages I find cool. (I wish I were going to SPLASH later this month...) Finally, I stumbled across the office of Manuel Blum, the 1995 Turing Award winner. There were a couple of posters outside his door showing the work of his students on problems of cryptography and privacy, and on the door itself were several comic strips. The punchline of one read, "I'll retire when it stops being fun." On this, even I am in sync with a Turing Award winner.

Everywhere I turned, something caught my eye. A pointer to the Newell/Simon bridge... Newell-and-Simon, the team, were the like the Pied Piper to me when I began my study of AI. A 40- or 50-page printout showing two old researchers (Newell and Simon?) playing chess. Plaques in recognition of big donations that had paid for classrooms, labs, and auditoria, made by Famous People who were either students or faculty in the school.

CMU is quite different from my school, of course, but there are many other schools that give off a similar vibe. I can see why people want to be at an R-1, even if they aspire to be teachers more than research faculty. There is so much going on. People, labs, sub-disciplines, and interdisciplinary projects. Informal talks, department seminars, and outside speakers. Always something going on. Ideas. Energy.

On the ride to the airport later in the day, I sat in some slow, heavy traffic going one direction and saw slower, heavier traffic going in the other. As much as I enjoyed the visit, I was glad to be heading home.

The opening talk of the CS Education Summit this week considered the challenges facing CS education in a time of surging enrollments and continued concerns about the diversity of the CS student population. In the session that followed, Eric Roberts and Jodi Tims presented data that puts the current enrollment surge into perspective, in advance of a report from the National Academy of Science.

In terms of immediate takeaway, Eric Roberts's comments were gold. Eric opened with Stein's Law: If something is unsustainable, it will stop. Stein was an economist whose eponymous law expresses one of those obvious truths we all seem to forget about in periods of rapid change: If something cannot go on forever, it won't. You don't have to create a program to make it stop. A natural corollary is: If it can't go on for long, you don't need a program to deal with it. It will pass soon.

Why is that relevant to the summit? Even without continued growth, current enrollments in CS majors is unsustainable for many schools. If the past is any guide, we know that many schools will deal with unsustainable growth by limiting the number of students who start or remain in their major.

Roberts has studied the history of CS boom-and-bust cycles over the last thirty years, and he's identified a few common patterns:

• Limiting enrollments is how departments respond to enrollment growth. They must: the big schools can't hire faculty fast enough, and most small schools can't hire new faculty at all.


• The number of students graduating with CS degrees drops because we limit enrollments. Students do not stop enrolling because the number of job opportunities goes down or any other cause.
  
  After the dot-com bust, there was a lot of talk about offshoring and automation, but the effects of that were short-term and rather small. Roberts's data shows that enrollment crashes do not follow crashes in job openings; they follow enrollment caps. Enrollments remain strong wherever they are not strictly limited.


• When we limit enrollments, the effect is bigger on women and members of underserved communities. These students are more likely to suffer from impostor syndrome, stereotype bias, and other fears, and the increased competitiveness among students for fewer openings combines with discourages them from continuing.

So the challenge of booming enrollments exacerbates the challenge to increase diversity. The boom might decrease diversity, but when it ends -- and it will, if we limit enrollments -- our diversity rarely recovers. That's the story of the last three booms.

In order to grow capacity, the most immediate solution is to hire more professors. I hope to write more about that soon, but for now I'll mention only that the problem of hiring enough faculty to teach all of our students has at east two facets. The first is that many schools simply don't have the money to hire more faculty right now. The second is that there aren't enough CS PhDs to go around. Roberts reported that, of last year's PhD grads, 83% took positions at R1 schools. That leaves 17% for the rest of us. "Non-R1 schools can expect to hire a CS PhD every 27 years." Everyone laughed, but I could see anxiety on more than a few faces.

The value of knowing this history is that, when we go to our deans and provosts, we can do more than argue for more resources. We can show the effect of not providing the resources needed to teach all the students coming our way. We won't just be putting the brakes on local growth; we may be helping to create the next enrollment crash. At a school like mine, if we teach the people of our state that we can't handle their CS students, then the people of our state will send their students elsewhere.

The problem for any one university, of course, is that it can act only based on its own resources and under local constraints. My dean and provost might care a lot about the global issues of demand for CS grads and need for greater diversity among CS students. But their job is to address local issues with their own (small) pool of money.

I'll have to re-read the papers Roberts has written about this topic. His remarks certainly gave us plenty to think about, and he was as engaging as ever.

Today and tomorrow, I am at a CS Education Summit in Pittsburgh. I've only been to Pittsburgh once before, for ICFP 2002 (the International Conference on Functional Programming) and am glad to be back. It's a neat city.

The welcome address for the summit was given by Dr. Farnam Jahanian, the interim president at Carnegie Mellon University. Jahanian is a computer scientist, with a background in distributed computing and network security. His resume includes a stint as chair of the CS department at the University of Michigan and a stint at the NSF.

Welcome addresses for conferences and workshops vary in quality. Jahanian gave quite a good talk, putting the work of the summit into historical and cultural context. The current boom in CS enrollments is happening at a time when computing, broadly defined, is having an effect in seemingly all disciplines and all sectors of the economy. What does that mean for how we respond to the growth? Will we see that the current boom presages a change to the historical cycle of enrollments in coming years?

Jahanian made three statements in particular that for me capture the challenge facing CS departments everywhere and serve as a backdrop for the summit:

• "We have to figure out how to teach all of these students."
  
  Unlike many past enrollment booms, "all of these students" this time comprises two very different subsets: CS majors and non-majors. We have plenty of experience teaching CS majors, but how do you structure your curriculum and classes when you have three times as many majors? When numbers go up far enough fast enough, many schools have a qualitatively different problem.
  
  Most departments have far less experience teaching computer science (not "literacy") to non-majors. How do you teach all of these students, with different backgrounds and expectations and needs? What do you teach them?


• "This is an enormous responsibility."
  
  Today's graduates will have careers for 45 years or more. That's a long time, especially in a world that is changing ever more rapidly, in large part due to our own discipline. How different are the long-term needs of CS majors and non-majors? Both groups will be working and living for a long time after they graduate. If computing remains a central feature of the world in the future, how we respond to enrollment growth now will have an outsized effect on every graduate. An enormous responsibility, indeed.


• "We in CS have to think about impending cultural changes..."
  
  ... which means that we computer science folks will need to have education, knowledge, and interests much broader than just CS. People talk all the time about the value of the humanities in undergraduate education. This is a great example of why. One bit of good news: as near as I can tell, most of the CS faculty in this room, at this summit, do have interests and education bigger than just computer science (*). But we have to find ways to work these issues into our classrooms, with both majors and non-majors.

Thus the idea of a CS education summit. I'm glad to be here.

(*) In my experience, it is much more likely to find a person with a CS or math PhD and significant educational background in the humanities than to find a person with a humanities PhD and significant educational background in CS or math (or any other science, for that matter). One of my hopes for the current trend of increasing interest in CS among non-CS majors is that we an close this gap. All of the departments on our campuses, and thus all of our university graduates, will be better for it.

Yesterday, I mentioned rewriting the rules for computing FIRST and FOLLOW sets using only "plain English". As I was refactoring my descriptions, I realized that one of the reasons students have difficulty with many textbook treatments of the algorithms is that the books give complete and correct definitions of the sets upfront. The presence of X := ε rules complicates the construction of both sets, but they are unnecessary to understanding the commonsense ideas that motivate the sets. Trying to deal with ε too soon can interfere with the students learning what they need to learn in order to eventually understand ε!

When I left the ε rules out of my descriptions, I ended up with what I thought were an approachable set of rules:

• The FIRST set of a terminal contains only the terminal itself.


• To compute FIRST for a non-terminal X, find all of the grammar rules that have X on the lefthand side. Add to FIRST(X) all of the items in the FIRST set of the first symbol of each righthand side.


• The FOLLOW set of the start symbol contains the end-of-stream marker.


• To compute FOLLOW for a non-terminal X, find all of the grammar rules that have X on the righthand side. If X is followed by a symbol in the rule, add to FOLLOW(X) all of the items in the FIRST set of that symbol. If X is the last symbol in the rule, add to FOLLOW(X) all of the items in the FOLLOW set of the symbol on the rule's lefthand side.

These rules are incomplete, but they have offsetting benefits. Each of these cases is easy to grok with a simple example or two. They also account for a big chunk of the work students need to do in constructing the sets for a typical grammar. As a result, they can get some practice building sets before diving into the gnarlier details ε, which affects both of the main rules above in a couple of ways.

These seems like a two-fold application of the Concrete, Then Abstract pattern. The first is the standard form: we get to see and work with accessible concrete examples before formalizing the rules in mathematical notation. The second involves the nature of the problem itself. The rules above are the concrete manifestation of FIRST and FOLLOW sets; students can master them before considering the more abstract ε cases. The abstract cases are the ones that benefit most from using formal notation.

I think this is an example of another pattern that works well when teaching. We might call it "Learn Exceptions Later", "Handle Exceptions Later", "Save Exceptions For Later", or even "Treat Exceptions as Exceptions". (Naming things is hard.) It is often possible to learn a substantial portion of an idea without considering exceptions at all, and doing so prepares students for learning the exceptions anyway.

I guess I now have at least one idea for my next PLoP paper.

Ironically, writing this post brings to mind a programming pattern that puts exceptions up top, which I learned during the summer Smalltalk taught me OOP. Instead of writing code like this:

    if normal_case(x) then
       // a bunch
       // of lines
       // of code
       // processing x
    else
       throw_an_error

    if abnormal_case(x) then
       throw_an_error

    // a bunch
    // of lines
    // of code
    // processing x

This pattern has served me well over the years, far beyond Smalltalk.

I've been meaning to blog about my compilers course for more than a month, but life -- including my compilers course -- have kept me busy. Here are three quick notes to prime the pump.

• I recently came across Lindsey Kuper's My First Fifteen Compilers and thought again about this unusual approach to a compiler course: one compiler a week, growing last week's compiler with a new feature or capability, until you have a complete system. Long, long-time readers of this blog may remember me writing about this idea once over a decade ago.
  
  The approach still intrigues me. Kuper says that it was "hugely motivating" to have a working compiler at the end of each week. In the end I always shy away from the approach because (1) I'm not yet willing to adopt for my course the Scheme-enabled micro-transformation model for building a compiler and (2) I haven't figured out how to make it work for a more traditional compiler.
  
  I'm sure I'll remain intrigued and consider it again in the future. Your suggestions are welcome!


• Last week, I mentioned on Twitter that I was trying to explain how to compute FIRST and FOLLOW sets using only "plain English". It was hard. Writing a textual description of the process made me appreciate the value of using and understanding mathematical notation. It is so expressive and so concise. The problem for students is that it is also quite imposing until they get it. Before then, the notation can be a roadblock on the way to understanding something at an intuitive level.
  
  My usual approach in class to FIRST and FOLLOW sets, as for most topics, is to start with an example, reason about it in commonsense terms, and only then to formalize. The commonsense reasoning often helps students understand the formal expression, thus removing some of its bite. It's a variant of the "Concrete, Then Abstract" pattern.
  
  Mathematical definitions such as these can motivate some students to develop their formal reasoning skills. Many people prefer to let students develop their "mathematical maturity" in math courses, but this is really just an avoidance mechanism. "Let the Math department fail them" may solve a practical problem, sometimes we CS profs have to bite the bullet and help our students get better when they need it.


• I have been trying to write more code for the course this semester, both for my enjoyment (and sanity) and for use in class. Earlier, I wrote a couple of toy programs such as a Fizzbuzz compiler. This weekend I took a deeper dive and began to implement my students' compiler project in full detail. It was a lot of fun to be deep in the mire of a real program again. I have already learned and re-learned a few things about Python, git, and bash, and I'm only a quarter of the way in! Now I just have to make time to do the rest as the semester moves forward.

In her post, Kuper said that her first compiler course was "a lot of hard work" but "the most fun I'd ever had writing code". I always tell my students that this course will be just like that for them. They are more likely to believe the first claim than the second. Diving in, I'm remembering those feelings firsthand. I think my students will be glad that I dove in. I'm reliving some of the challenges of doing everything that I ask them to do. This is already generating a new source of empathy for my students, which will probably be good for them come grading time.

I have finally started reading Mindstorms. I hope to write short reflections as I complete every few pages or anytime I come across something I feel compelled to write about in the moment. This is the first entry in the imagined series.

In the introduction, Papert says:

We shall see again and again that the consequences of mathophobia go far beyond obstructing the learning of mathematics and science. The interact with other endemic "cultural toxins", for example, with popular theories of aptitudes, to contaminate peoples' images of themselves as learners. Difficulty with school math is often the first step of an invasive intellectual process that leads us all to define ourselves as bundles of aptitudes and ineptitudes, as being "mathematical" or "not mathematical", "artistic" or "not artistic", "musical" or "not musical", "profound" or "superficial", "intelligent" or "dumb". Thus deficiency becomes identity, and learning is transformed from the early child's free exploration of the world to a chore beset by insecurities and self-imposed restrictions.

This invasive intellectual process has often deeply affected potential computer science students long before they reach the university. I would love to see Papert's dream made real early enough that young people can imagine being a computer scientist earlier. It's hard to throw of the shackles after they take hold.

The thing that sticks out as I read the first few pages of Mindstorms is its focus on the power of affect in learning. I don't recall conscious attention to my affect having much of a role in my education; it seems I was in a continual state of "cool, I get to learn something". I didn't realize at the time just what good fortune it was to have that as a default orientation.

I'm also struck by Papert's focus on the role of physicality in learning, how we often learn best when the knowledge has a concrete manifestation in our world. I'll have to think about this more... Looking back now, abstraction always seemed natural to me.

Papert's talk of love -- falling in love with the thing we learn about, but also with the thing we use to learn it -- doesn't surprise me. I know these feelings well, even from the earliest experiences I had in kindergarten.

An outside connection that I will revisit: Frank Oppenheimer's exploratorium, an aspiration I learned about from Alan Kay. What would a computational exploratorium look like?

In his conversation with Tyler Cowen, Ben Sasse talks a bit about how students learn in our schools of public policy, business, and law:

We haven't figured out in most professional schools how to create apprenticeship models where you cycle through different aspects of what doing this kind of work will actually look like. There are ways that there are tighter feedback loops at a med school than there are going to be at a policy school. There are things that I don't think we've thought nearly enough about ways that professional school models should diverge from traditional, theoretical, academic disciplines or humanities, for example.

We see a similar continuum in what works best, and what is needed, for learning computer science and learning software engineering. Computer science education can benefit from the tighter feedback loops and such that apprenticeship provides, but it also has a substantial theoretical component that is suitable for classroom instruction. Learning to be a software engineer requires a shift to the other end of the continuum: we can learn important things, in the classroom, but much of the important the learning happens in the trenches, making things and getting feedback.

A few universities have made big moves in how they structure software engineering instruction, but most have taken only halting steps. They are often held back by a institutional loyalty to the traditional academic model, or out of sheer curricular habit.

The one place you see apprenticeship models in CS is, of course, graduate school. Students who enter research work in the lab under the mentorship of faculty advisors and more senior grad students. It took me a year or so in graduate school to figure out that I needed to begin to place more focus on my research ideas than on my classes. (I hadn't committed to a lab or an advisor yet.)

In lieu of a changed academic model, internships of the sort I mentioned recently can be really helpful for undergrad CS students looking to go into software development. Internships create a weird tension for faculty... Most students come back from the workplace with a new appreciation for the academic knowledge they learn in the classroom, which is good, but they also back to wonder why more of their schoolwork can't have the character of learning in the trenches. They know to want more!

Project-based courses are a way for us to bring the value of apprenticeship to the undergraduate classroom. I am looking forward to building compilers with ten hardy students this fall.

This morning, I tweeted a quote from Sherry Turkle's Remembering Seymour Papert that struck a chord with a few people: "Seymour Papert saw that the computer would make it easier for thinking itself to become an object of thought." Here is another passage that struck a chord with me:

At the time of the juggling lesson, Seymour was deep in his experiments into what he called 'loud thinking'. It was what he was asking my grandfather to do. What are you trying? What are you feeling? What does it remind you of? If you want to think about thinking and the real process of learning, try to catch yourself in the act of learning. Say what comes to mind. And don't censor yourself. If this sounds like free association in psychoanalysis, it is. (When I met Seymour, he was in analysis with Greta Bibring.) And if it sounds like it could you get you into personal, uncharted, maybe scary terrain, it could. But anxiety and ambivalence are part of learning as well. If not voiced, they block learning.

It occurred to me that I blog as a form of "loud thinking". I don't write many formal essays or finished pieces for my blog these days. Mostly I share thoughts as they happen and think out loud about them in writing. Usually, it's just me trying to make sense of ideas that cross my path and see where they fit in with the other things I'm learning. I find that helpful, and readers sometimes help me by sharing their own thoughts and ideas.

When I first read the phrase "loud thinking", it felt awkward, but it's already growing on me. Maybe I'll try to get my compiler students to do some loud thinking this fall.

By the way, Turkle's entire piece is touching and insightful. I really liked the way she evoked Papert's belief that we "love the objects we think with" and "think with the objects we love". (And not just because I'm an old Smalltalk programmer!) I'll let you read the rest of the piece yourself to appreciate both the notion and Turkle's storytelling.

Now, for a closing confession: I have never read Mindstorms. I've read so much about Papert and his ideas over the years, but the book has never made it to the top of my stack. I pledge to correct this egregious personal shortcoming and read it as soon as I finish the novel on my nightstand. Maybe I'll think out loud about it here soon.

It's become commonplace of late to promote programming as fun! and something everyone will want to learn, if only they had a chance. Now, I love to program, but I've also been teaching long enough to know that not everyone takes naturally to programming. Sometimes, they warm up to it later in their careers, and sometimes, they never do.

This Quartz article takes the conventional wisdom to task as misleading:

Insisting on the glamour and fun of coding is the wrong way to acquaint kids with computer science. It insults their intelligence and plants the pernicious notion in their heads that you don't need discipline in order to progress. As anyone with even minimal exposure to making software knows, behind a minute of typing lies an hour of study.

But the author does think that people should understand code and what it means to program, but not because they necessarily will program very much themselves:

In just a few years, understanding programming will be an indispensable part of active citizenship.

This is why it's important for people to learn about programming, and why it's so important not to sell it in a way that ambushes students when they encounter it for the first time. Software development is both technically and ethically challenging. All citizens will be better equipped to participate in the world if they understand these challenges at some level. Selling the challenges short makes it harder to attract people who might be interested in the ethical challenges and harder to retain people turned off by technical challenges they weren't expecting.

In The Secret Origin Story of the iPhone, we hear about the day Steve Jobs told Bas Ording, one of Apple's UI "wizards", ...

... to make a demo of scrolling through a virtual address book with multitouch. "I was super-excited," Ording says. "I thought, Yeah, it seems kind of impossible, but it would be fun to just try it." He sat down, "moused off" a phone-size section of his Mac's screen, and used it to model the iPhone surface. He and a scant few other designers had spent years experimenting with touch-based user interfaces -- and those years in the touchscreen wilderness were paying off.

I'm guessing that a lot of programmers understand what Ording felt in that moment: "It seems kind of impossible, but it would be fun to just try it..." But he and his team were ready. Sometimes, you work in the wilderness a while, and suddenly it all pays off.

Via Pinboard's creator, the always entertaining Maciej Cegłowski:

Pinboard is not much more than a thin wrapper around some carefully tuned database queries.

You are ready to make your millions. Now all you need is an idea.

Earlier this month, James Iry tweeted:

Every CS degree covers fancy data structures. But what trips up more programmers? Times. Dates. Floats. Non-English text. Currencies.

I would like to add names to Iry's list. As a person who goes by his middle name and whose full legal name includes a suffix, I've seen my name mangled over the years in ways you might not imagine -- even by a couple of computing-related organizations that shall remain nameless. (Ha!) And my name presents only a scant few of the challenges available when we consider all the different naming conventions around the world.

This topic would make a great course for undergrads. We could call it "Humble Data Types" or "Mundane Data Types". My friends who program for a living know that these are practical data types, the ones that show up in almost all software and which consume an inordinate amount of time. That's why we see pages on the web about "falsehoods programmers believe" about time, names, and addresses -- another one for our list!

It might be hard to sell this course to faculty. They are notoriously reluctant to add new courses to the curriculum. (What would it displace?) Such conservatism is well-founded in a discipline that moves quickly through ideas, but this is a topic that has been vexing programmers for decades.

It would also be hard to sell the course to students, because it looks a little, well, mundane. I do recall a May term class a few years ago in which a couple of programmers spent days fighting with dates and times in Ruby while building a small accounting system. That certainly created an itch, but I'm not sure most students have enough experience with such practical problems before they graduate.

Maybe we could offer the course as continuing education for programmers out in the field. They are the ones who would appreciate it the most.

From Compress to Impress, on Jeff Bezos's knack for encoding important strategies in concise, memorable form:

As a hyper intelligent person, Jeff didn't want lossy compression or lazy thinking, he wanted the raw feed in a structured form, and so we all shifted to writing our arguments out as essays that he'd read silently in meetings. Written language is a lossy format, too, but it has the advantage of being less forgiving of broken logic flows than slide decks.

Ask any intro CS student: Even less forgiving of broken logic than prose is the computer program.

Programs are not usually the most succinct way to express an idea, but I'm often surprised by how little work it takes to express an idea about a process in code. When a program is a viable medium for communicating an idea, it provides value in many dimensions. You can run a program, which makes the code's meaning observable in its behavior. A program lays bare logic and assumptions, making them observable, too. You can tinker with a program, looking at variations and exploring their effects.

The next time you have an idea about a process, try to express it in code. A short bit prose may help, too.

None of this is intended to diminish the power of using rhetorical strategies to communicate at scale and across time, as described in the linked post. It's well worth a read. From the outside looking in, Bezos seems to be a remarkable leader.

Last week I read a tweet that linked to an article by Paul Romer. He is an economist currently working at the World Bank, on leave from his chair at NYU. Romer writes well, so I found myself digging deeper and reading a couple of his blog articles. One of them, Writing, struck a chord with me both as a writer and as a computer scientist.

Consider:

The quality of written prose should be higher in documents that will have many readers.

This is true of code, too. If a piece of code will be read many times, whether by one person or several, then each minute spent making it shorter and clearer improves reading comprehension every single time. That's even more important in code than in text, because so often we read code in order to change it. We need to understand it at even deeper level to ensure that our changes have the intended effect. Time spent making code better repays itself many times over.

Romer caused a bit of a ruckus when he arrived at the World Bank by insisting, to some of his colleagues' displeasure, that everyone in his division writer clearer, more concise reports. His goal was admirable: He wanted more people to be able to read and understand these reports, because they deal with policies that matter to the public.

He also wanted people to trust what the World Bank was saying by being able more readily to see that a claim was true or false. His article looks at two different examples that make a claim about the relationship between education spending and GDP per capita. He concludes his analysis of the examples with:

In short, no one can say that the author of the second claim wrote something that is false because no one knows what the second claim means.

In science, writing clearly builds trust. This trust is essential for communicating results to the public, of course, because members of the public do not generally possess the scientific knowledge they need to assess the truth of claim directly. But it is also essential for communicating results to other scientists, who must understand the claims at a deeper level in order to support, falsify, and extend them.

In the second half of the article, Romer links to a study of the language used in World Bank's yearly reports. It looks at patterns such as the frequency of the word "and" in the reports and the ratio of nouns to verbs. (See this Financial Times article for a fun little counterargument on the use of "and".)

Romer wants this sort of analysis to be easier to do, so that it can be used more easily to check and improve the World Bank's reports. After looking at some other patterns of possible interest, he closes with this:

To experiment with something like this, researchers in the Bank should be able to spin up a server in the cloud, download some open-source software and start experimenting, all within minutes.

Wonderful: a call for storing data in easy-to-access forms and a call for using (and writing) programs to analyze text, all in the name not of advancing economics technically but of improving its ability to communicate its results. Computing becomes a tool integrated into the process of the World Bank doing its designated job. We need more leaders in more disciplines thinking this way. Fortunately, we hear reports of such folks more often these days.

Alas, data and programs were not used in this way when Romer arrived at the World Bank:

When I arrived, this was not possible because people in ITS did not trust people from DEC and, reading between the lines, were tired of dismissive arrogance that people from DEC displayed.

One way to create more trust is to communicate better. Not being dismissively arrogant is, too, though calling that sort of behavior out may be what got Romer in so much hot water with the administrators and economists at the World Bank in the first place.

In his recent interview with Tyler Cowen, Garry Kasparov talks about AI, chess, politics, and the future of creativity. In one of the more intriguing passages, he explains that building databases for chess endgames has demonstrated how little we understand about the game and offers insight into how we know that chess-playing computer programs -- now so far beyond humans that even the world champion can only score occasionally against commodity programs -- still have a long way to improve.

He gives as an example a particular position with a king, two rooks, a knight on one side versus a king and two rooks on the other. Through the retrograde analysis used to construct endgame databases, we know that, with ideal play by both sides, the stronger side can force checkmate in 490 moves. Yes, 490. Kasparov says:

Now, I can tell you that -- even being a very decent player -- for the first 400 moves, I could hardly understand why these pieces moved around like a dance. It's endless dance around the board. You don't see any pattern, trust me. No pattern, because they move from one side to another.

At certain points I saw, "Oh, but white's position has deteriorated. It was better 50 moves before." The question is -- and this is a big question -- if there are certain positions in these endgames, like seven-piece endgames, that take, by the best play of both sides, 500 moves to win the game, what does it tell us about the quality of the game that we play, which is an average 50 moves? [...]

Maybe with machines, we can actually move our knowledge much further, and we can understand how to play decent games at much greater lengths.

But there's more. Do chess-playing computer programs, so much superior to even the best human players, understand these endgames either? I don't mean "understand" in the human sense, but only in the sense of being able to play games of that quality. Kasparov moves on to his analysis of games between the best programs:

I think you can confirm my observations that there's something strange in these games. First of all, they are longer, of course. They are much longer because machines don't make the same mistakes [we do] so they could play 70, 80 moves, 100 moves. [That is] way, way below what we expect from perfect chess.

That tells us that [the] machines are not perfect. Most of those games are decided by one of the machines suddenly. Can I call it losing patience? Because you're in a position that is roughly even. [...] The pieces are all over, and then suddenly one machine makes a, you may call, human mistake. Suddenly it loses patience, and it tries to break up without a good reason behind it.

That also tells us [...] that machines also have, you may call it, psychology, the pattern and the decision-making. If you understand this pattern, we can make certain predictions.

Kasparov is heartened by this, and it's part of the reason that he is not as pessimistic about the near-term prospects of AI as some well-known scientists and engineers are. Even with so-called deep learning, our programs are only beginning to scratch the surface of complexity in the universe. There is no particular reason to think that the opaque systems evolved to drive our cars and fly our drones will be any more perfect in their domains than our game-playing programs, and we have strong evidence from the domain of games that programs are still far from perfect.

On a more optimistic note, advances in AI give us an opportunity to use programs to help us understand the world better and to improve our own judgment. Kasparov sees this in chess, in the big gaps between the best human play, the best computer play, and perfect play in even relatively simple positions; I wrote wistfully about this last year, prompted by AlphaGo's breakthrough. But the opportunity is much more valuable when we move beyond playing games, as Cowen alluded in an aside during Kasparov's explanation: Imagine how bad our politics will look in comparison to computer programs that do it well! We have much to learn.

As always, this episode of Conversations with Tyler was interesting and evocative throughout. If you are a chess player, there is an special bonus. The transcript includes a pointer to Kasparov's Immortal Game against Veselin Topalov at Wijk aan Zee in 1999, along with a discussion of some of Kasparov's thoughts on the game beginning with the pivotal move 24. Rxd4. This game, an object of uncommon beauty, will stand as an eternal reminder why, even in the face of advancing AI, it will always matter that people play and compete and create.

~~~~

If you enjoyed this entry, you might also like Old Dreams Live On. It looks more foresightful now that AlphaGo has arrived.

Okay, so this is cool:

Neuroscientists stimulate the brain with brief pulses of energy and then record the echoes that bounce back. Dreamless sleep and general anaesthesia return simple echoes; brains in conscious states produce more complex patterns. Then comes a little inspiration from data compression:

Excitingly, we can now quantify the complexity of these echoes by working out how compressible they are, similar to how simple algorithms compress digital photos into JPEG files. The ability to do this represents a first step towards a "consciousness-meter" that is both practically useful and theoretically motivated.

This made me think of Chris Ford's StrangeLoop 2015 talk about using compression to understand music. Using compressibility as a proxy for complexity gives us a first opportunity to understand all sorts of phenomena about which we are collecting data. Kolmogorov complexity is a fun tool for programmers to wield.

The passage above is from an Aeon article on the study of consciousness. I found it an interesting read from beginning to end.

This week, I have been enjoying Eli Bendersky's two-article series "Adventures in JIT Compilation":

• Part 1 - An Interpreter
• Part 2 - An x64 JIT

Bendersky is a good teacher, at least in the written form, and I am picking up a lot of ideas for my courses in programming languages and compilers. I recommend his articles and his code highly.

In Part 2, Bendersky says something that made me think of my students:

One of my guiding principles through the field of programming is that before diving into the possible solutions for a problem (for example, some library for doing X) it's worth working through the problem manually first (doing X by hand, without libraries). Grinding your teeth over issues for a while is the best way to appreciate what the shrinkwrapped solution/library does for you.

The presence or absence of this attitude is one of the crucial separators among CS students. Some students come into the program with this mindset already in place, and they are often the ones who advance most quickly in the early courses. Other students don't have this mindset, either by interest or by temperament. They prefer to solve problems quickly using canned libraries and simple patterns. These students are often quite productive, but they sometimes soon hit a wall in their learning. When a student rides along the surface of what they are told in class, never digging deeper, they tend to have a shallow knowledge of how things work in their own programs. Again, this can lead to a high level of productivity, but it also produces brittle knowledge. When something changes, or the material gets more difficult, they begin to struggle. A few of the students eventually develop new habits and move nicely into the group of students who likes to grind. The ones who don't make the transition continue to struggle and begin to enjoy their courses less.

There is a rather wide variation among undergrad CS students, both in their goals and in their preferred styles or working and learning. This variation is one of the challenges facing profs who hope to reaching the full spectrum of students in their classes. And helping students to develop new attitudes toward learning and doing is always a challenge.

As I got ready for class yesterday morning, I decided to refactor a piece of code. No big deal, right? It turned out to be a bigger deal than I expected. That's part of the fun of programming.

The function in question is a lexical addresser for a little language we use as a specimen in my Programming Languages course. My students had been working on a design, and it was time for us to build a solution as a group. Looking at my code from the previous semester, I thought that changing the order of two cases would make for a better story in class. The cases are independent, so I swapped them and ran my tests.

The change broke my code. It turns out that the old "else" clause had been serving as a convenient catch-all and was only working correctly due to an error in another function. Swapping the cases exposed the error.

Ordinarily, this wouldn't be a big deal, either. I would simply fix the code and give my students a correct solution. Unfortunately, I had less than an hour before class, so I now found myself in a scramble to find the bug, fix it, and make the changes to my lecture notes that had motivated the refactor in the first place. Making changes like this under time pressure is rarely a good idea... I was tempted to revert to the previous version, teach class, and make the change after class. But I am a programmer, dogged and often foolhardy, so I pressed on. With a few minutes to spare, I closed the editor on my lecture notes and synced the files to my teaching machine. I was tired and still had a little nervous energy coursing through me, but I felt great. That's part of the fun of programming.

I will say this: Boy, was I glad to have my test suite! It was incomplete, of course, because I found an error in my program. But the tests I did have helped me to know that my bug fix had not broken something else unexpectedly. The error I found led to several new tests that make the test suite stronger.

This experience was fresh in my mind this morning when I read "Physics Was Paradise", an interview with Melissa Franklin, a distinguished experimental particle physicist at Harvard. At one point, Franklin mentioned taking her first physics course in high school. The interviewer asked if physics immediately stood out as something she would dedicate her life to. Franklin responded:

Physics is interesting, but it didn't all of a sudden grab me because introductory physics doesn't automatically grab people. At that time, I was still interested in being a writer or a philosopher.

I took my first programming class in high school and, while I liked it very much, it did not cause me to change my longstanding intention to major in architecture. After starting in the architecture program, I began to sense that, while I liked architecture and had much to learn from it, computer science was where my future lay. Maybe somewhere deep in my mind was memory of an experience like the one I had yesterday, battling a piece of code and coming out with a sense of accomplishment and a desire to do battle again. I didn't feel the same way when working on problems in my architecture courses.

Intro CS, like intro physics, doesn't always snatch people away from their goals and dreams. But if you enjoy the fun of programming, eventually it sneaks up on you.

At one point in this SOHP interview, Douglas Engelbart describes his work at the Ames Research Center after graduating from college. He was an electrical engineer, building and maintaining wind tunnels, paging systems, and other special electronics. Looking to make a connection between this job and his future work, the interviewer asked, "Did they have big computers running the various operations?" Engelbart said:

I'll tell you what a computer was in those days. It was an underpaid woman sitting there with a hand calculator, and they'd have rooms full of them, that's how they got their computing done. So you'd say, "What's your job?" "I'm a computer."

Later in the interview, Engelbart talks about how his experience working with radar in the Navy contributed to his idea for a symbol-manipulating system that could help people deal with complexity and urgency. He viewed the numeric calculations done by the human computers at Ames as being something different. Still, I wonder how much this model of parallel computing contributed to his ideas, if only implcitly.

In The Victorian Internet, Tom Standage, says this about the unexpected challenge facing William Cooke and Samuel Morse, the inventors of the telegraph:

[They] had done the impossible and constructed working telegraphs. Surely the world would fall at their feet. Building the prototypes, however, turned out to be the easy part. Convincing people of their significance was far more of a challenge.

That must be what it feels like to be Ivan Sutherland. Or Alan Kay, for that matter.

A key passage from Peter Seibel's 2014 essay, Code Is Not Literature:

But then it hit me. Code is not literature and we are not readers. Rather, interesting pieces of code are specimens and we are naturalists. So instead of trying to pick out a piece of code and reading it and then discussing it like a bunch of Comp Lit. grad students, I think a better model is for one of us to play the role of a 19th century naturalist returning from a trip to some exotic island to present to the local scientific society a discussion of the crazy beetles they found: "Look at the antenna on this monster! They look incredibly ungainly but the male of the species can use these to kill small frogs in whose carcass the females lay their eggs."

The point of such a presentation is to take a piece of code that the presenter has understood deeply and for them to help the audience understand the core ideas by pointing them out amidst the layers of evolutionary detritus (a.k.a. kludges) that are also part of almost all code. One reasonable approach might be to show the real code and then to show a stripped down reimplementation of just the key bits, kind of like a biologist staining a specimen to make various features easier to discern.

My scientist friends often like to joke that CS isn't science, even as they admire the work that computer scientists and programmers do. I think Seibel's essay expresses nicely one way in which studying software really is like what natural scientists do. True, programs are created by people; they don't exist in the world as we find it. (At least programs in the sense of code written by humans to run on a computer.) But they are created under conditions that look a lot more like biological evolution than, say, civil engineering.

As Hal Abelson says in the essay, most real programs end up containing a lot of stuff just to make it work in the complex environments in which they operate. The extraneous stuff enables the program to connect to external APIs and plug into existing frameworks and function properly in various settings. But the extraneous stuff is not the core idea of the program.

When we study code, we have to slash our way through the brush to find this core. When dealing with complex programs, this is not easy. The evidence of adaptation and accretion obscures everything we see. Many people do what Seibel does when they approach a new, hairy piece of code: they refactor it, decoding the meaning of the program and re-coding it in a structure that communicates their understanding in terms that express how they understand it. Who knows; the original program may well have looked like this simple core once, before it evolved strange appendages in order to adapt to the condition in which it needed to live.

The folks who helped to build the software patterns community recognized this. They accepted that every big program "in the wild" is complex and full of cruft. But they also asserted that we can study such programs and identify the recurring structures that enable complex software both to function as intended and to be open to change and growth at the hands of programmers.

One of the holy grails of software engineering is to find a way to express the core of a system in a clear way, segregating the extraneous stuff into modules that capture the key roles that each piece of cruft plays. Alas, our programs usually end up more like the human body: a mass of kludges that intertwine to do some very cool stuff just well enough to succeed in a harsh world.

And so: when we read code, we really do need to bring the mindset and skills of a scientist to our task. It's not like reading People magazine.

In Links 2013, Bret Victor offers these bits of advice for how to read Alan Kay's writings and listen to his talks:

As you read and watch Alan Kay, try not to think about computational technology, but about a society that is fluent in thinking and debating in the dimensions opened up by the computational medium.

Don't think about "coding" (that's ink and metal type, already obsolete), and don't think about "software developers" (medieval scribes only make sense in an illiterate society).

I have always been inspired and challenged by Kay's work. One of second-order challenges I face is to remember that his vision is not ultimately about people like me writing programs. It's about a culture in which every person can use computational media the way we all use backs of envelopes, sketch books, newspapers, and books today. Computation can change the way we think and exchange ideas.

Then again, it's hard enough to teach CS students to program. That is a sign that we still have work to do in understanding programming better, and also in thinking about the kind of tools we build and use. In terms of Douglas Engelbart's work, also prominently featured among Victor's 2013 influences -- we need to build tools to improve how we program before we can build tools to "improve the improving".

Links 2013 could be the reading list for an amazing seminar. There are no softballs there.

I was recently reading an old bit-player entry on computing number factoids when I ran across a paragraph that expresses an all-too-common phenomenon of the modern world:

If I had persisted in my wrestling match, would I have ultimately prevailed? I'll never know, because in this era of Google and MathOverflow and StackExchange, a spoiler lurks around every cybercorner. Before I could make any further progress, I stumbled upon pointers to the work of Ira Gessel of Brandeis, who neatly settled the matter ... more than 40 years ago, when he was an undergraduate at Harvard.

The matter in this case was recognizing whether an arbitrary n is a Fibonacci number or not, but it could be have been just about anything. If you need an answer to almost any question these days, it's already out there, right a your fingertips.

Google and StackExchange and MathOverflow are a boon for knowing, but not so much for doing. Unfortunately, doing often leads to a better kind of knowing. Jumping directly to the solution can rob us of some important learning. As Hayes reminds us in his articles, it also can also deprive us of a lot of fun.

You can still learn by doing and have a lot of fun doing it today -- if you can resist the temptation to search. After you struggle for a while and need some help, then having answers at our fingertips becomes a truly magnificent resource and can help us get over humps we could never have gotten over so quickly in even the not-the-so-distant past.

The new world puts a premium on curiosity, the desire to find answers for ourselves. It also values self-denial, the ability to delay gratification while working hard to find answer that we might be able to look up. I fear that this creates a new gap for us to worry about in our education systems. Students who are curious and capable of self-denial are a new kind of "haves". They have always had a leg up in schools, but ubiquitous information magnifies the gap.

Being curious, asking questions, and wanting to create (not just look up) answers have never been more important to learning.

Coming into the semester, I didn't have any students doing their undergraduate research under my supervision. That frees up some time each week, which is nice, but leaves my semester a bit poorer. Working with students one-on-one is one of the best parts of this job, even more so in relief against administrative duties. Working on these projects makes my weeks better, even when I don't have as much time to devote to them as I'd like.

Yesterday, a student walked in with a project that makes my semester a little busier -- and much more interesting. Last summer, he implemented some ideas on extensible effects in Haskell and has some ideas for ways to make the system more efficient.

This student knows a lot more about extensible effects and Haskell than I do, so I have some work to do just to get ready to help. I'll start with Extensible Effects: An Alternative to Monad Transformers, the paper by Oleg Kiselyov and his colleagues that introduced the idea to the wider computing community. This paper builds on work by Cartwright and Felleisen, published over twenty years ago, which I'll probably look at, too. The student has a couple of other things for me to read, which will appear in his more formal proposal this week. I expect that these papers will make my brain hurt, in the good way, and am looking forward to diving in.

In the big picture, most undergrad projects in my department are pretty minor as research goes. They are typically more D than R, with students developing something that goes beyond what they learn in any course and doing a little empirical analysis. The extensible effects project is much more ambitious. It builds on serious academic research. It works on a significant problem and proposes something new. That makes the project much more exciting for me as the supervisor.

I hope to report more later, as the semester goes on.

Good advice in this paragraph, paraphrased lightly from Modern Garbage Collection:

Garbage collection is a hard problem, really hard, one that has been studied by an army of computer scientists for decades. Be very suspicious of supposed breakthroughs that everyone else missed. They are more likely to just be strange or unusual tradeoffs in disguise, avoided by others for reasons that may only become apparent later.

It's wise always to be on the lookout for "strange or unusual tradeoffs in disguise".

In the Paris Review's The Art of Fiction No. 183, the interviewer asks Tobias Wolff how he balances writing with university teaching. Wolff figures that teaching is a pretty good deal:

When I think about the kinds of jobs I've had and the ways I've lived, and still managed to get work done--my God, teaching in a university looks like easy street. I like talking about books, and I like encountering other smart, passionate readers, and feeling the friction of their thoughts against mine. Teaching forces me to articulate what otherwise would remain inchoate in my thoughts about what I read. I find that valuable, to bring things to a boil.

That reflects how I feel, too, as someone who loves to do computer science and write programs. As a teacher, I get to talk about cool ideas every day with my students, to share what I learn as I write software, and to learn from them as they ask the questions I've stopped asking myself. And they pay me. It's a great deal, perhaps the optimal point in the sort of balance that Derek Sivers recommends.

Wolff immediately followed those sentences with a caution that also strikes close to home:

But if I teach too much it begins to weigh on me--I lose my work. I can't afford to do that anymore, so I keep a fairly light teaching schedule.

One has to balance creative work with the other parts of life that feed the work. Professors at research universities, such as Wolff at Stanford, have different points of equilibrium available to them than profs at teaching universities, where course loads are heavier and usually harder to reduce.

I only teach one course a semester, which really does help me to focus creative energies around a smaller set of ideas than a heavier load does. Of course, I also have the administrative duties of a department head. They suffocate time and energy in a much less productive way than teaching does. (That's the subject of another post.)

Why can't Wolff afford to teach too many courses anymore? I suspect the answer is time. When you reach a certain age, you realize that time is no longer an ally. There are only so many years left, and Wolff probably feels the need to write more urgently. This sensation has been seeping into my mind lately, too, though I fear perhaps a bit too slowly.

~~~~

(I previously quoted Wolff from the same interview in a recent entry about writers who give advice that reminds us that there is no right way to write all programs. A lot of readers seemed to like that one.)

The latest edition of my compiler course has wrapped, with grades submitted and now a few days distance between us and the work. The course was successful in many ways, even though not all of the teams were all able to implement the entire compiler. That mutes the students' sense of accomplishment sometimes, but it's not unusual for at least some of the teams to have trouble implementing a complete code generator. A compiler is a big project. Fifteen weeks is not a lot of time. In that time, students learn a lot about compilers, and also about how to work as a team to build a big program using some of the tools of modern software development. In general, I was quite proud of the students' efforts and progress. I hope they were proud of themselves.

One of the meta-lessons students tend to learn in this course is one of the big lessons of any project-centered course:

... making something is a different learning experience from remembering something.

I think that a course like this one also helps most of them learn something else even more personal:

... the discipline in art-making is exercised from within rather than without. You quickly realize that it's your own laziness, ignorance, and sloppiness, not somebody else's bad advice, that are getting in your way. No one can write your [program] for you. You have to figure out a way to write it yourself. You have to make a something where there was a nothing.

"Laziness", "ignorance", and "sloppiness" seem like harsh words, but really they aren't. They are simply labels for weaknesses that almost all of us face when we first learn to create things on our own. Anyone who has written a big program has probably encountered them in some form.

I learned these lessons as a senior, too, in my university's two-term project course. It's never fun to come up short of our hopes or expectations. But most of us do it occasionally, and never more reliably than we are first learning how to make something significant. It is good for us to realize early on our own responsibility for how we work and what we make. It empowers us to take charge of our behavior.

The quoted passages are, with the exception of the word "program", taken from Learn by Painting, a New Yorker article about "Leap Before You Look: Black Mountain College, 1933-1957", an exhibit at the Institute of Contemporary Art in Boston. Black Mountain was a liberal arts college with a curriculum built on top of an unusual foundation: making art. Though the college lasted less than a quarter century, its effects were felt across most of art disciplines in the twentieth century. But its mission was bigger: to educate citizens, not artists, through the making art. Making something is a different learning experience from remembering something, and BMC wanted all of its graduates to have this experience.

The article was a good read throughout. It closes with a comment on Black Mountain's vision that touches on computer science and reflects my own thinking about programming. This final paragraph begins with a slight indignity to us in CS but turns quickly into an admiration:

People who teach in the traditional liberal-arts fields today are sometimes aghast at the avidity with which undergraduates flock to courses in tech fields, like computer science. Maybe those students see dollar signs in coding. Why shouldn't they? Right now, tech is where value is being created, as they say. But maybe students are also excited to take courses in which knowing and making are part of the same learning process. Those tech courses are hands-on, collaborative, materials-based (well, virtual materials), and experimental -- a digital Black Mountain curriculum.

When I meet with prospective students and their parents, I stress that, while computer science is technical, it is not vocational. It's more. Many high school students sense this already. What attracts them to the major is a desire to make things: games and apps and websites and .... Earning potential appeals to some of them, of course, but students and parents alike seem more interested in something else that CS offers them: the ability to make things that matter in the modern world. They want to create.

The good news suggested in "Learn by Painting", drawing on the Black Mountain College experiment, is that learning by making things is more than just that. It is a different and, in most ways, more meaningful way to learn about the world. It also teaches you a lot about yourself.

I hope that at least a few of my students got that out of their project course with me, in addition to whatever they learned about compilers.

~~~~

IMAGE. The main building of the former Black Mountain College, on the grounds of Camp Rockmont, a summer camp for boys. Courtesy of Wikipedia. Public domain.

A friend of mine recently shared a link to Radio Garden on a mailing list (remember those?), and in the ensuing conversation, another friend wrote:

I remember when I was a kid playing with my Dad's shortwave radio and just being flabbergasted when late one night I tuned in a station from Peru. Today you can get on your computer and communicate instantly with any spot on the globe, and that engenders no sense of wonder at all.

Such is the nature of advancing technology. Everyone becomes acclimated to amazing new things, and pretty soon they aren't even things any more.

Teachers face a particularly troublesome version of this phenomenon. Teach a subject for a few years, and pretty soon it loses its magic for you. It's all new to your students, though, and if you can let them help you see it through their eyes, you can stay fresh. The danger, though, is that it starts to look pretty ordinary to you, even boring, and you have a hard time helping them feel the magic.

If you read this blog much, you know that I'm pretty easy to amuse and pretty easy to make happy. Even so, I have to guard against taking life and computer science for granted.

Earlier this week, I was reading one reading one of the newer tutorials in Matthew Butterick's Beautiful Racket, Imagine a language: wires. In it, he builds a DSL to solve one of the problems in the 2015 edition of Advent of Code, Some Assembly Required. The problem is fun, specifying a circuit in terms of a small set of operations for wires and gates. Butterick's approach to solving it is fun, too: creating a DSL that treats the specification of a circuit as a program to interpret.

This is no big deal to a jaded old computer scientist, but remember -- or imagine -- what this solution must seem like to a non-computer scientist or to a CS student encountering the study of programming languages for the first time. With a suitable interpreter, every dataset is a program. If that isn't amazing enough, some wires datasets introduce sequencing problems, because the inputs to a gate are defined in the program after the gate. Butterick uses a simple little trick: define wires and gates as functions, not data. This simple little trick is really a big idea in disguise: Functions defer computation. Now circuit programs can be written in any order and executed on demand.

Even after all these years, computing's most mundane ideas can still astonish me sometimes. I am trying to keep my sense of wonder high and to renew it whenever it starts to flag. This is good for me, and good for my students.

~~~~

P.S. As always, I recommend Beautiful Racket, and Matthew Butterick's work more generally, quite highly. He has a nice way of teaching useful ideas in a way that appreciates their beauty.

P.P.S. The working title of this entry was "Paging Louis C.K., Paging Louis C.K." That reference may be a bit dated by now, but still it made me smile.

Earlier this week, Rands tweeted:

Tinkering is a deceptively high value activity.

... to which I followed up:

Which is why a language that enables tinkering is a deceptively high value tool.

I thought about these ideas a couple of days later when I read The Running Conversation in Your Head and came across this paragraph:

The idea is not that you need language for thinking but that when language comes along, it sure is useful. It changes the way you think, it allows you to operate in different ways because you can use the words as tools.

This is how I think about programming in general and about new, and better, programming languages in particular. A programmer can think quite well in just about any language. Many of us cut our teeth in BASIC, and simply learning how to think computationally allowed us to think differently than we did before. But then we learn a radically different or more powerful language, and suddenly we are able to think new thoughts, thoughts we didn't even conceive of in quite the same way before.

It's not that we need the new language in order to think, but when it comes along, it allows us to operate in different ways. New concepts become new tools.

I am looking forward to introducing Racket and functional programming to a new group of students this spring semester. First-class functions and higher-order functions can change how students think about the most basic computations such as loops and about higher-level techniques such as OOP. I hope to do a better job this time around helping them see the ways in which it really is different.

To echo the Running Conversation article again, when we learn a new programming style or language, "Something really special is created. And the thing that is created might well be unique in the universe."

I'm reminded of a student I met with once who told me that he planned to go to law school, and then a few minutes later, when going over a draft of a lab report, said "Yeah... Grammar isn't really my thing." Explaining why I busted up laughing took a while.

When I ask prospective students why they decided not to pursue a CS degree, they often say things to the effect of "Computer science seemed cool, but I heard getting a degree in CS was a lot of work." or "A buddy of mine told me that programming is tedious." Sometimes, I meet these students as they return to the university to get a second degree -- in computer science. Their reasons for returning vary from the economic (a desire for better career opportunities) to personal (a desire to do something that they have always wanted to do, or to pursue a newfound creative interest).

After you've been in the working world a while, a little hard work and some occasional tedium don't seem like deal breakers any more.

Such conversations were on my mind as I read physicist Chad Orzel's recent Science Is Not THAT Special. In this article, Orzel responds to the conventional wisdom that becoming a scientist and doing science involve a lot of hard work that is unlike the exciting stuff that draws kids to science in the first place. Then, when kids encounter the drudgery and hard work, they turn away from science as a potential career.

Orzel's takedown of this idea is spot on. (The quoted passage above is one of the article's lighter moments in confronting the stereotype.) Sure, doing science involves a lot of tedium, but this problem is not unique to science. Getting good at anything requires a lot of hard work and tedious attention to detail. Every job, every area of expertise, has its moments of drudgery. Even the rare few who become professional athletes and artists, with careers generally thought of as dreams that enable people to earn a living doing the thing they love, spend endless hours engaged in the drudgery of practicing technique and automatizing physical actions that become their professional vocabulary.

Why do we act as if science is any different, or should be?

Computer science gets this rap, too. What could be worse than fighting with a compiler to accept a program while you are learning to code? Or plowing threw reams of poorly documented API descriptions to plug your code into someone's e-commerce system?

Personally, I can think of lots of things that are worse. I am under no illusion, however, that other professionals are somehow shielded from such negative experiences. I just prefer my pains to theirs.

Maybe some people don't like certain kinds of drudgery. That's fair. Sometimes we gravitate toward the things whose drudgery we don't mind, and sometimes we come to accept the drudgery of the things we love to do. I'm not sure which explains my fascination with programming. I certainly enjoy the drudgery of computer science more than that of most other activities -- or at least I suffer it more gladly.

I'm with Orzel. Let's be honest with ourselves and our students that getting good at anything takes a lot of hard work and, once you master something, you'll occasionally face some tedium in the trenches. Science, and computer science in particular, are not that much different from anything else.

I've been wanting to write a blog entry or two lately about my compiler course and about papers I've read recently, but I've not managed to free up much time as semester winds down. That's one of the problems with having Big Ideas to write about: they seem daunting and, at the very least, take time to work through.

So instead here are two brief notes about articles that crossed my newsfeed recently and planted themselves in my mind. Perhaps you will enjoy them even without much commentary from me.

• A Student's Unusual Proof Might Be A Better Proof

I asked a student to show that between any two rationals is a rational.

She did the following: if x < y are rational then take δ << y-x and rational and use x+δ.

I love the student's two proofs in article! Student programmers are similarly creative. Their unusual solutions often expose biases in my thinking and give me way to think about a problem. If nothing else, they help to understand better how students think about ideas that I take for granted.

• Numberless Word Problems

Some girls entered a school art competition. Fewer boys than girls entered the competition.

She projected her screen and asked, "What math do you see in this problem?"

Pregnant pause.

"There isn't any math. There aren't any numbers."

I am fascinated by the possibility of adapting this idea to teaching students to think like a programmer. In an intro course, for example, students struggle with computational ideas such as loops and functions even though they have a lot of experience with these ideas embodied in their daily lives. Perhaps the language we use gets in the way of them developing their own algorithmic skills. Maybe I could use computationless word problems to get them started?

I'm giving serious thought to ways I might use this approach to help students learn functional programming in my Programming Languages course this spring. The authors describes how to write numberless word problems, and I'm wondering how I might bring the philosophy to computer science. If you have any ideas, please share!

This morning I read this blog post by Dan Luu on the use of randomized algorithms in cache eviction. He mentions a paper by Mitzenmacher, Richa, and Sitaraman called The Power of Two Random Choices that explains a cool effect we see when we limit our options when choosing among multiple options. Luu summarizes the result:

The mathematical intuition is that if we (randomly) throw n balls into n bins, the maximum number of balls in any bin is O(log n / log log n) with high probability, which is pretty much just O(log n). But if (instead of choosing randomly) we choose the least loaded of k random bins, the maximum is O(log log n / log k) with high probability, i.e., even with two random choices, it's basically O(log log n) and each additional choice only reduces the load by a constant factor.

Luu used this result to create a cache eviction policy that outperforms random eviction across the board and competes closely with the traditional LRU policy. It chooses two pages at random and evicts the least-recently used of the two. This so-called 2-random algorithm slightly outperforms LRU in larger caches and slightly underperforms LRU in smaller caches. This trade-off may be worth making because, unlike LRU, random eviction policies degrade gracefully as loops get large.

The power of two random choices has potential application in any context that fits the balls-and-bins model, including load balancing. Luu mentions less obvious application areas, too, such as circuit routing.

Very cool indeed. I'll have to look at Mitzenmacher et al. to see how the math works, but first I may try the idea out in some programs. For me, the programming is even more fun...

I am a regular reader of John Regehr's blog, which provides a steady diet of cool compiler conversation. One of Regehr's frequent topics is undefined behavior in programming languages, and what that means for implementing and testing compilers. A lot of those blog entries involve C and C++, which I don't use all that often any more, so reading them is more spectator sport than contact sport.

This week, I got see how capricious C++ compilers can feel up close.

My students are implementing a compiler for a simple subset of a Pascal-like language. We call the simplest program in this language print-one:

    $ cat print-one.flr
    program main();
      begin
        return 1
      end.

One of the teams is writing their compiler in C++. The team completed its most recent version, a parser that validates its input or reports an error that renders its input invalid. They were excited that it finally worked:

    $ g++ -std=c++14 compiler.cpp -o compiler
    $ compiler print-one.flr 
    Valid flair program

They had tested their compiler on two platforms:

• a laptop running OS X v10.11.5 and gcc v4.9.3
• a desktop running Ubuntu v14.04 and gcc v4.8.4

I sat down at my desktop computer to exercise their compiler.

    $ g++ compiler.cpp -o compiler
    In file included from compiler.cpp:7:
    In file included from ./parser.cpp:3:
    In file included from ./ast-utilities.cpp:4:
    ./ast-utilities.hpp:7:22: warning: in-class initialization of non-static data
          member is a C++11 extension [-Wc++11-extensions]
        std::string name = "Node";
                    ^
    [...]
    24 warnings generated.

Oops, I forgot the -std=c++14 flag. Still, it compiled, and all of the warnings come from a part of the code has no effect on program validation. So I tried the executable:

    $ compiler print-one.flr 
    ERROR at line #3 -- unexpected <invalid>  1
    Invalid flair program

Hmm. The warnings are unrelated to part of the executable that I am testing, but maybe they are creating a problem. So I recompile with the flag:

    $ g++ -std=c++14 compiler.cpp -o compiler
    error: invalid value 'c++14' in '-std=c++14'

What? I check my OS and compiler specs:

    $ sw_vers -productVersion
    10.9.5
    $ g++ --version
    Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
    Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
    [...]

Oh, right, Apple doesn't ship gcc any more; it ships clang and link gcc to the clang exe. I know my OS is a bit old, but it still seems odd that the -std=c++14 flag isn't supported. I google for an answer (thanks, StackOverflow!) and find that that I need to use -std=c++1y. Okay:

    $ g++ -std=c++1y compiler.cpp -o compiler
    $ compiler print-one.flr 
    ERROR at line #3 -- unexpected <invalid>  1
    Invalid flair program

Now the student compiler compiles but gives incorrect, or at least unintended, behavior. I'm surprised that both my clang and the students' gcc compile their compiler yet produce executables that give different answers. I know that gcc and clang aren't 100% compatible, but my students are using a relatively small part of C++. How can this be?

Maybe it has something to do with how clang processes the c++1y standard flag. So I backed up to the previous standard:

    $ g++ -std=c++0x compiler.cpp -o compiler
    $ compiler print-one.flr 
    ERROR at line #3 -- unexpected <invalid>  1
    Invalid flair program

Yes, that's c++0x, not c++0y. The student compiler still compiles and still gives incorrect or unintended behavior. Maybe it is a clang problem? I upload their code to our student server, which runs Linux and gcc:

    $ cat /etc/debian_version 
    8.1
    $ g++ --version
    [...]
    gcc version 4.7.2 (Debian 4.7.2-5)

This version of gcc doesn't support either c++14 or c++1?, so I fell back to c++0x:

    $ g++ -std=c++0x compiler.cpp -o compiler
    $ compiler print-one.flr 
    Valid flair program

Hurray! I can test their code.

I'm curious. I have a Macbook Pro running a newer version of OS X. Maybe...

    $ sw_vers -productVersion
    ProductName:Mac OS X
    ProductVersion:10.10.5
    BuildVersion:14F2009
    $ g++ --version
    Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/c++/4.2.1
    Apple LLVM version 7.0.2 (clang-700.1.81)
    [...]

    $ g++ -std=c++14 compiler.cpp -o compiler
    $ compiler print-one.flr 
    Valid flair program

Now, the c++14 flag works, and it produces a compiler that produces the correct behavior -- or at least the intended behavior.

I am curious about this anomaly, but not curious enough to research the differences between clang and gcc, the differences between the different versions of clang, or what Apple or Debian are doing. I'm also not curious enough to figure out which nook of C++ my students have stumbled into that could expose a rift in the behavior of these various C++ compilers, all of which are standard tools and pretty good.

At least now I remember what it's like to program in a language with undefined behavior and can read Regehr's blog posts with a knowing nod of the head.

I enjoyed this interview with Atul Gawande by Ezra Klein. When talking about making mistakes, Gawande notes that humans have enough knowledge to cut way down on errors in many disciplines, but we do not always use that knowledge effectively. Mistakes come naturally from the environments in which we work:

We're all set up for failure under the conditions of complexity.

Mistakes are often more a matter of discipline and attention to detail than a matter of knowledge or understanding. Klein captures the essence of Gawande's lesson in one of his questions:

We have this idea that competence is not making mistakes and getting everything right. [But really...] Competence is knowing you will make mistakes and setting up a context that will help reduce the possibility of error but also help deal with the aftermath of error.

In my experience, this is a hard lesson for computer science students to grok. It's okay to make mistakes, but create conditions where you make as few as possible and in which you can recognize and deal with the mistakes as quickly as possible. High-discipline practices such as test-first and pair programming, version control, and automated builds make a lot more sense when you see them from this perspective.

There is an open thread on the SIGCSE mailing list called "Forget the language wars, try the math wars". Faculty are discussing how to justify math requirements on a CS major, especially for students who "just want to be programmers". Some people argue that math requirements are a barrier to recruiting students who can succeed in computer science, in particular calculus.

Somewhere along the line, Cay Horstmann wrote a couple of things I agree with. First, he said that he didn't want to defend the calculus requirement because most calculus courses do not teach students how to think "but how to follow recipes". I have long had this complaint about calculus, especially as it's taught in most US high schools and universities. Then he wrote something more positive:

What I would like to see is teaching math alongside with programming. None of my students were able to tell me what sine and cosine were good for, but when they had to program a dial [in a user interface], they said "oh".

Couldn't that "oh" have come earlier in their lives? Why don't students do programming in middle school math? I am not talking large programs--just a few lines, so that they can build models and intuition.

I agree wholeheartedly. And even if students do not have such experiences in their K-12 math classes, the least we could do help them have that "oh" experience earlier in their university studies.

My colleagues and I have been discussing our Discrete Structures course now for a few weeks, including expected outcomes, its role as a prerequisite to other courses, and how we teach it. I have suggested that one of the best ways to learn discrete math is to connect it with programs. At our university, students have taken at least one semester of programming (currently, in Python) before they take Discrete. We should use that to our advantage!

A program can help make an abstract idea concrete. When learning about set operations, why do only paper-and-pencil exercises when you can use simple Python expressions in the REPL? Yes, adding programming to the mix creates new issues to deal with, but if designed well, such instruction could both improve students' understanding of discrete structures -- as Horstmann says, helping them build models and intuition -- and give students more practice writing simple programs. An ancillary benefit might be to help students see that computer scientists can use computation to help them learn new things, thus preparing for habits that can extend to wider settings.

Unfortunately, the most popular Discrete Structures textbooks don't help much. They do try to use CS-centric examples, but they don't seem otherwise to use the fact that students are CS majors. I don't really blame them. They are writing for a market in which students study many different languages in CS 1, so they can't (and shouldn't) assume any particular programming language background. Even worse, the Discrete Structures course appears at different places throughout the CS curriculum, which means that textbooks can't assume even any particular non-language CS experience.

Returning to Horstmann's suggestion to augment math instruction with programming in K-12, there is, of course, a strong movement nationally to teach computer science in high school. My state has been disappointingly slow to get on board, but we are finally seeing action. But most of the focus in this nationwide movement is on teaching CS qua CS, with less interest in emphasis on integrating CS into math and other core courses.

For this reason, let us again take a moment to thank the people behind the Bootstrap project for leading the charge in this regard, helping teachers use programming in Racket to teach algebra and other core topics. They are even evaluating the efficacy of the work and showing that the curriculum works. This may not surprise us in CS, but empirical evidence of success is essential if we hope to get teacher prep programs and state boards of education to take the idea seriously.

An oldie but goodie from Andrew Tanenbaum:

Actually, MINIX 3 and my research generally is **NOT** about microkernels. It is about building highly reliable, self-healing, operating systems. I will consider the job finished when no manufacturer anywhere makes a PC with a reset button. TVs don't have reset buttons. Stereos don't have reset buttons. Cars don't have reset buttons. They are full of software but don't need them. Computers need reset buttons because their software crashes a lot. I know that computer software is different from car software, but users just want them both to work and don't want lectures why they should expect cars to work and computers not to work. I want to build an operating system whose mean time to failure is much longer than the lifetime of the computer so the average user never experiences a crash.

I remember loving MINIX 1 (it was just called Minix then, of course) when I first learned it in grad school. I did not have any Unix experience coming out of my undergrad and had only begun to feel comfortable with BSD Unix in my first few graduate courses. Then I was assigned to teach the Operating Systems course, working with one of the CS faculty. He taught me a lot, but so did Tanenbaum -- through Minix. That is one of the first times I came to really understand that the systems we use (the OS, the compiler, the DBMS) were just programs that I could tinker with, modify, and even write.

Operating systems is not my area, and I have no expertise for evaluating the whole microkernel versus monolith debate. But I applaud researchers like Tanenbaum who are trying to create general computer systems that don't need to be rebooted. I'm a user, too.

My recent post about the two languages resonated in my mind with an article I finished reading the day I wrote the post: Two Heads, about the philosophers Paul and Pat Churchland. The Churchlands have been on a forty-year quest to change the language we use to describe our minds, from popular terms based in intuition and introspection to terms based in the language of neuroscience. Changing language is hard under any circumstances, and it is made harder when the science they need is still in its infancy. Besides, maybe more traditional philosophers are right and we need our traditional vocabulary to make sense of what it feels like to be human?

The New Yorker article closes with these paragraphs, which sounds as if they are part of a proposal for a science fiction novel:

Sometimes Paul likes to imagine a world in which language has disappeared altogether. We know that the two hemispheres of the brain can function separately but communicate silently through the corpus callosum, he reasons. Presumably, it will be possible, someday, for two separate brains to be linked artificially in a similar way and to exchange thoughts infinitely faster and more clearly than they can now through the muddled, custom-clotted, serially processed medium of speech. He already talks about himself and Pat as two hemispheres of the same brain. Who knows, he thinks, maybe in his children's lifetime this sort of talk will not be just a metaphor.

If, someday, two brains could be joined, what would be the result? A two-selved mutant like Joe-Jim, really just a drastic version of Siamese twins, or something subtler, like one brain only more so, the pathways from one set of neurons to another fusing over time into complex and unprecedented arrangements? Would it work only with similar brains, already sympathetic, or, at least, both human? Or might a human someday be joined to an animal, blending together two forms of thinking as well as two heads? If so, a philosopher might after all come to know what it is like to be a bat, although, since bats can't speak, perhaps he would be able only to sense its batness without being able to describe it.

(Joe-Jim is a character from a science fiction novel, Robert Heinlein's Orphans of the Sky.)

What a fascinating bit of speculation! Can anyone really wonder why kids are drawn to science fiction?

Let me add my own speculation to the mix: If we do ever find a way to figure out what it's like to be a bat, people will find a way to idescribe what it's like to be a bat. They will create the language they need. Making language is what we do.

If Carroll's deconstruction of the simulation argument is right, then the more trouble we have explaining consciousness, the more that should push us to believe we're in a ground-level simulation. There's probably a higher-level version of physics in which consciousness makes sense. Our own consciousness is probably being run in a world that operates on that higher-level law. And we're stuck in a low-resolution world whose physics doesn't allow consciousness -- because if we weren't, we'd just keep recursing further until we were.

-- Scott Alexander, The View From Ground Level

In the latest installment of "You Haven't Seen That Yet?", I watched the film Inception yesterday. There was only one person watching, but still the film gets two hearty thumbs-up. All those Ellen Pages waking up, one after the other...

Over the last few years, I've heard many references to the idea from physics that we are living in a simulation, that our universe is a simulation created by beings in another universe. It seems that some physicists think and talk about this a lot, which seems odd to me. Empiricism can't help us much to unravel the problem; arguments pro and con come down to the sort of logical arguments favored by mathematicians and philosophers, abstracted away from observation of the physical world. It's a fun little puzzle, though. The computer science questions are pretty interesting, too.

Ideas like this are old hat to those of us who read a lot of science fiction growing up, in particular Philip K. Dick. Dick's stories were often predicated on suspending some fundamental attribute of reality, or our perception of it, and seeing what happened to our lives and experiences. Now that I have seen Memento (a long-time favorite of mine) and Inception, I'm pretty happy. What Philip K. Dick was with the written word to kids of my generation, Christopher Nolan is on film to a younger generation. I'm glad I've been able to experience both.

~~~~

The photo above comes from Matt Goldberg's review of Inception. It shows Arthur, the character played by Joseph Gordon-Levitt, battling with a "projection" in three-dimensional space that folds back on itself. Such folding is possible in dream worlds and is an important element in designing dreams that enable all the cool mind games that are central to the film.

In The Security of Our Election Systems, Bruce Schneier says that we no longer have time to sound alarm about security flaws in our election systems and hope that government and manufacturers will take action. Instead...

We must ignore the machine manufacturers' spurious claims of security, create tiger teams to test the machines' and systems' resistance to attack, drastically increase their cyber-defenses and take them offline if we can't guarantee their security online.

How about this:

The students in my department love to compete in cyberdefense competitions (CDCs), in which they are charged with setting up various systems and then defending them against attack from experts for some period, say, twenty-four hours. Such competitions are growing in popularity across the country.

Maybe we should run a CDC with the tables turned. Manufacturers are required to set up their systems and to run the full set of services they promise when they sell the systems to government agencies. Students across the US would then be given a window of twenty-fours or more to try to crack the systems, with the manufacturers or even our election agencies trying to keep their systems up and running securely. Any vulnerabilities that the students find would be made public, enabling the manufacturers to fix them and the state agencies to create and set up new controls.

Great idea or crazy fantasy?

I've been reading a bunch of the essays on David Chapman's Meaningness website lately, after seeing a link to one on Twitter. (Thanks, @kaledic.) This morning I read How To Think Real Good, about one of Chapman's abandoned projects: a book of advice for how to think and solve problems. He may never write this book as he once imagined it, but I'm glad he wrote this essay about the idea.

First of all, it was a fun read, at least for me. Chapman is a former AI researcher, and some of the stories he tells remind me of things I experienced when I was in AI. We were even in school at about the same time, though in different parts of the country and different kinds of program. His work was much more important than mine, but I think at some fundamental level most people in AI share common dreams and goals. It was fun to listen as Chapman reminisced about knowledge and AI.

He also introduced me to the dandy portmanteau anvilicious. I keep learning new words! There are so many good ones, and people make up the ones that don't exist already.

My enjoyment was heightened by the fact that the essay stimulated the parts of my brain that like to think about thinking. Chapman includes a few of the heuristics that he intended to include in his book, along with anecdotes that illustrate or motivate them. Here are three:

All problem formulations are "false", because they abstract away details of reality.

Solve a simplified version of the problem first. If you can't do even that, you're in trouble.

Probability theory is sometimes an excellent way of dealing with uncertainty, but it's not the only way, and sometimes it's a terrible way.

He elaborates on the last of these, pointing out that probability theory tends to collapse many different kinds of uncertainty into a single value. This does not work all that well in practice, because different kinds of uncertainty often need to be handles in very different ways.

Chapman has a lot to say about probability. This essay was prompted by what he sees as an over-reliance of the rationalist community on a pop version of Bayesianism as its foundation for reasoning. But as an old AI researcher, he knows that an idea can sound good and fail in practice for all sorts of reasons. He has also seen how a computer program can make clear exactly what does and doesn't work.

Artificial intelligence has always played a useful role as a reality check on ideas about mind, knowledge, reasoning, and thought. More generally, anyone who writes computer programs knows this, too. You can make ambiguous claims with English sentences, but to write a program you really have to have a precise idea. When you don't have a precise idea, your program itself is a precise formulation of something. Figuring out what that is can be a way of figuring out what you were really thing about in the first place.

This is one of the most important lessons college students learn from their intro CS courses. It's an experience that can benefit all students, not just CS majors.

Chapman also includes a few heuristics for approaching the problem of thinking, basically ways to put yourself in a position to become a better thinker. Two of my favorites are:

Try to figure out how people smarter than you think.

Find a teacher who is willing to go meta and explain how a field works, instead of lecturing you on its subject matter.

This really is good advice. Subject matter is much easier to come by than deep understanding of how the discipline work, especially in these days of the web.

The word meta appears frequently throughout this essay. (I love that the essay is posted on the metablog/ portion of his site!) Chapman's project is thinking about thinking, a step up the ladder of abstraction from "simply" thinking. An AI program must reason; an AI researcher must reason about how to reason.

This is the great siren of artificial intelligence, the source of its power and also its weaknesses: Anything you can do, I can do meta.

I think this gets at why I enjoyed this essay so much. AI is ultimately the discipline of applied epistemology, and most of us who are lured into AI's arms share an interest in what it means to speak of knowledge. If we really understand knowledge, then we ought to be able to write a computer program that implements that understanding. And if we do, how can we say that our computer program isn't doing essentially the same thing that makes us humans intelligent?

As much as I love computer science and programming, my favorite course in graduate school was an epistemology course I took with Prof. Rich Hall. It drove straight to the core curiosity that impelled me to study AI in the first place. In the first week of the course, Prof. Hall laid out the notion of justified true belief, and from there I was hooked.

A lot of AI starts with a naive feeling of this sort, whether explicitly stated or not. Doing AI research brings that feeling into contact with reality. Then things gets serious. It's all enormously stimulating.

Ultimately Chapman left the field, disillusioned by what he saw as a fundamental limitation that AI's bag of tricks could never resolve. Even so, the questions that led him to AI still motivate him and his current work, which is good for all of us, I think.

This essay brought back a lot of pleasant memories for me. Even though I, too, am no longer in AI, the questions that led me to the field still motivate me and my interests in program design, programming languages, software development, and CS education. It is hard to escape the questions of what it means to think and how we can do it better. These remain central problems of what it means to be human.

We have a fair number of students on campus outside of CS who want to become web designers, but few of them think they should learn to program. Some give it a try when one of our communications profs tells them how exciting and liberating it can be. In general, though, it's a hard sell. Programming sounds boring to them, full of low-level details better left to techies over in computer science.

This issue pervades the web design community. In The Bomb in the Garden, Matthew Butterick does a great job of explaining why the web as a design medium is worth saving, and pointing to ways in which programming can release the creativity we need to keep it alive.

Which brings me to my next topic--what should designers know about programming?

And I know that some of you will think this is beating a dead horse. But when we talk about restoring creativity to the web, and expanding possibilities, we can't avoid the fact that just like the web is a typographic medium, it's also a programmable medium.

And I'm a designer who actually does a lot of programming in my work. So I read the other 322,000 comments about this on the web. I still think there's a simple and non-dogmatic answer, which is this:

You don't have to learn programming, but don't knock it till you try it.

It's fun for me when one of the web design students majoring in another department takes his or her first programming class and is sparked by the possibilities that writing a program opens up. And we in CS are happy to help them go deeper into the magic.

Butterick speaks truth when he says he's a designer who does a lot of programming in his work. Check out Pollen, the publishing system he created to write web-based books. Pollen's documentation says that it "helps authors make functional and beautiful digital books". That's true. It's a very nice package.

I recently came across an old entry on Jonathan Blow's blog called John Carmack on Inlined Code. The bulk of the entry consists an even older email message that Carmack, lead programmer on video games such as Doom and Quake, sent to a mailing list, encouraging developers to consider inlining function calls as a matter of style. This email message is the earliest explanation I've seen of Carmack's drift toward functional programming, seeking to as many of its benefits as possible even in the harshly real-time environment of game programming.

The article is a great read, with much advice borne in the trenches of writing and testing large programs whose run-time performance is key to their success. Some of the ideas involve programming language:

It would be kind of nice if C had a "functional" keyword to enforce no global references.

... while others are more about design style:

The function that is least likely to cause a problem is one that doesn't exist, which is the benefit of inlining it.

... and still others remind us to rely on good tools to help avoid inevitable human error:

I now strongly encourage explicit loops for everything, and hope the compiler unrolls it properly.

(This one may come in handy as I prepare to teach my compiler course again this fall.)

This message-within-a-blog-entry itself quotes another email message, by Henry Spencer, which contains the seeds of a programming challenge. Spencer described a piece of flight software written in a particularly limiting style:

It disallowed both subroutine calls and backward branches, except for the one at the bottom of the main loop. Control flow went forward only. Sometimes one piece of code had to leave a note for a later piece telling it what to do, but this worked out well for testing: all data was allocated statically, and monitoring those variables gave a clear picture of most everything the software was doing.

Wow: one big loop, within which all control flows forward. To me, this sounds like a devilish challenge to take on when writing even a moderately complex program like a scanner or parser, which generally contain many loops within loops. In this regard, it reminds me of the Polymorphism Challenge's prohibition of if-statements and other forms of selection in code. The goal of that challenge was to help programmers really grok how the use of substitutable objects can lead to an entirely different style of program than we tend to create with traditional procedural programming.

Even though Carmack knew that "a great deal of stuff that goes on in the aerospace industry should not be emulated by anyone, and is often self destructive", he thought that this idea might have practical value, so he tried it out. The experience helped him evolve his programming style in a promising direction. This is a great example of the power of the pedagogical pattern known as Three Bears: take an idea to its extreme in order to learn the boundaries of its utility. Sometimes, you will find that those boundaries lie beyond what you originally thought.

Carmack's whole article is worth a read. Thanks to Jonathan Blow for preserving it for us.

~~~~

The image above is an example of the cover art for the "Commander Keen" series of video games, courtesy of Wikipedia. John Carmack was also the lead programmer for this series. What a remarkable oeuvre he has produced.

I recently wrote about some experiences programming in Joy, in which I use Joy to solve five problems that make up a typical homework assignment early in my Programming Languages course. These problems introduce my students to writing simple functions in a functional style, using Racket. Here is my code, if you care to check it out. I'm just getting back to stack programming, so this code can surely be improved. Feel free to email me suggestions or tweet me at @wallingf!

What did these problems teach me about Joy?

• The language's documentation is sparse. Like my students, I had to find out which Joy primitives were available to me. It has a lot of the basic arithmetic operators you'd expect, but finding them meant searching through a plain-text file. I should write Racket-caliber documentation for the language to support my own work.


• The number and order of the arguments to a function matters a lot. A function that takes several arguments can become more complicated than the corresponding Racket function, especially if you need to use them multiple times. I encountered this on my first day back to the language. In Racket, this problem requires a compound expression, but it is relatively straightforward, because arguments have names. With all its arguments on the stack, a Joy function has to do more work simply to access values, let alone replicate them for multiple uses.


• A slight difference in task can lead to a large change in the code. For Problem 4, I implemented operators for modular addition, subtraction, and multiplication. +mod and *mod were elegant and straightforward. -mod was a different story. Joy has a rem operator that operates like Racket's remainder, but it has no equivalent to modulo. The fact that rem returns negative values means that I need a boolean expression and quoted programs and a new kind of thinking. This militates for a bigger shift in programming style right away.


• I miss the freedom of Racket naming style. This isn't a knock on Joy, because most every programming language restricts severely the set of characters you can use in identifiers. But after being able to name functions +mod, in-range?, and int->char in Racket, the restrictions feel even more onerous.


• As in most programming styles, the right answer in Joy is often to write the correct helpers. The difference in level of detail between +mod and *mod on the one hand and -mod on the other indicates that I am missing solution. A better approach is to implement a modulo operator and use it to write all three mod operators. This will hide lower-level details in a general-purpose operator. modulo would make a nice addition to a library of math operators.


• Even simple problems can excite me about the next step. Several of these solutions, especially the mod operators, cry out for higher-order operators. In Racket, we can factor out the duplication in these operators and create a function that generates these functions for us. In Joy, we can do it, too, using quoted programs of the sort you see in the code for -mod. I'll be moving on to quoted programs in more detail soon, and I can't wait... I know that they will push me farther along the transition to the unique style of stack programming.

It's neat for me to be reminded that even the simplest little functions raise interesting design questions. In Joy, use of a stack for all data values means that identifying the most natural order for the arguments we make available to an operators can have a big effect on the ability to read and write code. In what order will arguments generally appear "in the wild"?

In the course of experimenting and trying to debug my code (or, even more frustrating, trying to understand why the code I wrote worked), I even wrote my first general utility operator:

    DEFINE clear  == [] unstack.

It clears the stack so that I can see exactly what the code I'm about to run creates and consumes. It's the first entry in my own little user library, named utilities.joy.

Fun, fun, fun. Have I ever said that I like to write programs?

Painter Adolph Gottlieb was dismissive of art school in the 1950s:

I'd have done what I'm doing now twenty years ago if I hadn't had to go through that crap. What is the function of the art school today? To confuse the student. To make a living for the teacher. The painter can learn from museums -- probably it is the only way he can to learn. All artists have to solve their problems in the context of their own civilization, painting what their time permits them to paint, extending the boundaries a little further.

It isn't much of a stretch to apply this to computer programming in today's world. We can learn so much these days from programs freely available on GitHub and elsewhere on the web. A good teacher can help, but in general is there a better way to learn how to make things than to study existing works and to make our own?

Most importantly, today's programmers-to-be have to solve their problems in the context of their own civilization: today's computing, whether that's mobile or web or Minecraft. University courses have a hard time keeping up with constant change in the programming milieu. Instead, they often rely on general principles that apply across most environments but which seem lifeless in their abstraction.

I hope that, as a teacher, I add some value for the living I receive. Students with interests and questions and goals help keep me and my courses alive. At least I can set a lower bound of not confusing my students any more than necessary.

~~~~

(The passage above is quoted from Conversations with Artists, Selden Rodman's delightful excursion through the art world of the 1950s, chatting with many of the great artists of the era. It's not an academic treatise, but rather more an educated friend chatting with creative friends. I would thank the person who recommended this, but I have forgotten whose tweet or article did.)

Last time, I wrote about returning to the programming language Joy, without giving too many details of the language or its style. Today, I'll talk a bit more what it means to say that Joy is a "stack language" and show how I sometimes think about the stack as I am writing a simple program. The evolution of the stack is helping me to think about how data types work in this language and to get a sense of why languages such as Joy are called concatenative.

Racket and other Lisp-like languages use prefix notation, unlike the more common infix notation we use in C, Java, and most mainstream languages. A third alternative is postfix notation, in which operators follow their operands. For example, this postfix expression is a valid Joy program:

    2 3 +

computes 2 + 3.

Postfix expressions are programs in a stack-based language. They correspond to a postfix traversal of a program tree. Where is the stack? It is used to interpret the program:

• 2 is pushed onto the stack.
• 3 is pushed onto the stack.
• + is an operator. An operator pops its arguments from the stack, computes a result, and pushes the result on to the stack. + is a two-argument function, so it pops two arguments from the stack, computes their sum, and pushes the 5 back onto the stack.

Longer programs work the same way. Consider 2 3 + 5 *:

• 2 is pushed onto the stack.
• 3 is pushed onto the stack.
• + pops the 2 and the 3, computes a 5, and pushes it on to the stack.
• 5 is pushed onto the stack.
• * pops the two 5s, computes a 25, and pushes it on to the stack.

This program is equivalent to 2 + 3 - 5, er, make that (2 + 3) * 5. As long as we know the arity of each procedure, postfix notation requires no rules for the precedence of operators.

The result of the program can be the entire stack or the top value on the stack. At the end of a program, the stack could finish with more than one value on it:

    2 3 + 5 * 6 3 /

leaves a stack consisting of 25 2.

Adding an operator to the end of that program can return us to a single-value stack:

    2 3 + 5 * 6 3 / -

leaves a stack of one value, 23.

This points out an interesting feature of this style of programming: we can compose programs simply by concatenating their source code. 2 * is a program: "double my argument", where all arguments are read from the stack. If we place it at the end of 2 3 + 5 * 6 3 / -, we get a new program, one that computes the value 46.

This gives us our first hint as to why this style is called concatenative programming.

As I am re-learning Joy's style, I think a lot about the stack, which holds the partial results of the computation I am trying to program. I often find myself simulating the state of stack to help me keep track of what to do next, whether on paper or in a comment on my code.

As an example, think back to the wind chill problem I discussed last time: given the the air temperature T and the wind speed V, compute the wind chill index T'.

    T' = 35.74 + 0.6215·T - 35.75·V0.16 + 0.4275·T·V0.16

This program expects to find T and V on top of the stack:

    T V

The program 0.16 pow computes x^0.16 for the 'x' on top of the stack, so it leaves the stack in this state:

    T V'

In Joy documentation, the type of a program is often described with a before/after picture of the stack. The dup2 operator we saw last time is described as:

    dup2 :: a b -> a b a b

because it assumes two arbitrary values on top of the stack and leaves the stack with those values duplicated. In this fashion, we could describe the program 0.16 pow using

    0.16 pow :: N -> N

It expects a number N to be on top of the stack and leaves the stack with another number on top.

When I'm using these expressions to help me follow the state of my program, I sometimes use problem-specific names or simple modifiers to indicate changes in value or type. For the wind-chill program, I thought of the type of 0.16 pow as

    T V -> T V'

Applied to this stack, dup2 converts the stack as

    T V' -> T V' T V'

If we concatenate these two programs and evaluate them against the original stack, we get:

    [ initial stack]         T V
    0.16 pow              -> T V'
    dup2                  -> T V' T V'

I've been preserving these stack traces in my documentation, for me and any students who might end up reading my code. Here is the definition of wind-chill, taken directly from the source file:

    DEFINE wind-chill == 0.16 pow        (* -> T V'        *)
                         dup2            (* -> T V' T V'   *)
                         * 0.4275 *      (* -> T V' t3     *)
                         rollup          (* -> t3 T V'     *)
                         35.75 *         (* -> t3 T t2     *)
                         swap            (* -> t3 t2 T     *)
                         0.6215 *        (* -> t3 t2 t1    *)
                         35.74           (* -> t3 t2 t1 t0 *)
                         + swap - +.     (* -> T'          *)

After the dup2 step, the code alternates between computing a term of the formula and rearranging the stack for the next computation.

Notice the role concatenation plays. I can solve each substep in almost any order, paste the resulting little programs together, and boom! I have the final solution.

I don't know if this kind of documentation helps anyone but me, or if I will think it is as useful after I feel more comfortable in the style. But for now, I find it quite helpful. I do wonder whether thinking in terms of stack transformation may be helpful as I prepare to look into what type theory means for a language like Joy. We'll see.

I am under no pretense that this is a great Joy program, even in the context of a relative novice figuring things out. I'm simply learning, and having fun doing it.

I learned about the programming language Joy in the early 2000s, when I was on sabbatical familiarizing myself with functional programming. It drew me quickly into its web. Joy is a different sort of functional language, one in which function composition replaces function application as the primary focus. At that time, I wrote a bunch of small Joy programs and implemented a simple interpreter in PLT Scheme. After my sabbatical, though, I got pulled in a lot of different directions and lost touch with Joy. I saw it only every two or three semesters when I included it in my Programming Languages course. (The future of programming might not look like you expect it to....)

This spring, I felt Joy's call again and decided to make time to dive back into the language. Looking back over my notes from fifteen years ago, I'm surprised at some of the neat thoughts I had back then and at some of the code I wrote. Unfortunately, I have forgotten most of what I learned, especially about higher-order programming in Joy. I never reached the level of a Joy master anyway, but I feel like I'm starting from scratch. That's okay.

On Thursday, I sat down to write solutions in Joy for a few early homework problems from my Programming Languages course. These problems are intended to help my students learn the basics of functional programming and Racket. I figured they could help me do the same in Joy before I dove deeper, while also reminding me of the ways that programming in Joy diverges stylistically from more traditional functional style. As a bonus, I'd have a few more examples to show my students in class next time around.

It didn't take me long to start having fun. I'll talk more in upcoming posts about Joy, this style of programming, and -- I hope -- some of my research. For now, though, I'd like to tell you about one experience I had on my first day without getting into too many details.

In one homework problem, we approximate the wind chill index using this formula:

    T' = 35.74 + 0.6215·T - 35.75·V0.16 + 0.4275·T·V0.16

In Joy, we don't pass arguments to functions as in most other languages. Its operators pop their arguments from a common data stack and push their results back on to the stack. Many of Joy's operators manipulate the data stack: creating, rearranging, and deleting various items. For example, the dup operator makes a copy of the item on top of the stack, the swap operator swaps the top two items on the stack, and the rolldown operator moves the top two items on the stack below the third.

A solution to the wind-chill problem will expect to find T and V on top of the stack:

    T V

    T V'

The formula uses these values twice, so I really need two copies of each:

    T V' T V'

With a little work, I found that this sequence of operations does the trick:

    swap dup rolldown dup rolldown swap

From there, it didn't take long to find a sequence of operators that consumed these four values and left T' on the stack.

As I looked back over my solution, I noticed the duplication of dup rolldown in the longer expression shown above and thought about factoring it out. Giving that sub-phrase a name is hard, though, because it isn't all that meaningful on its own. However, the whole phrase is meaningful, and probably useful in a lot of other contexts: it duplicates the top two items on the stack. So I factored the whole phrase out and named it dup2:

    DEFINE dup2 == swap dup rolldown dup rolldown swap.

As soon as my fingers typed "dup2", though, my mind recognized it. Surely I had seen it before... So I went looking for "dup2" in Joy's documentation. It is not a primitive operator, but it is defined in the language's initial library, inilib.joy, a prelude that defines syntactic sugar in Joy itself:

    dup2 ==  dupd dup swapd

This definition uses two other stack operators, dupd and swapd, which are themselves defined using the higher-order operator dip. I'll be getting to higher-order operators soon enough, I thought; for now I was happy with my longer but conceptually simpler solution.

I was not surprised to find dup2 already defined in Joy. It defines a lot of cool stack operators, the sorts of operations every programmer needs to build more interesting programs in the language. But I was a little disappointed that my creation wasn't new, in the way that only a beginner can be disappointed when he learns that his discovery isn't new. My disappointment was more than offset by the thought that I had recognized an operator that the language's designer thought would be useful. I was already starting to feel like a Joy programmer again.

It was a fun day, and a much-needed respite from administrative work. I intend for it to be only the first day of many more days programming with Joy.

Oberon is a software stack created by Niklaus Wirth and his lab at ETH Zürich. Lukas Mathis describes some of what makes Oberon unusual, including the fact that its desktop is "an infinitely large two-dimensional space on which windows ... can be arranged":

It's incredibly easy to move around on this plane and zoom in and out of it. Instead of stacking windows, hiding them behind each other (which is possible in modern versions of Oberon), you simply arrange them next to each other and zoom out and in again to switch between them. When people held presentations using Oberon, they would arrange all slides next to each other, zoom in on the first one, and then simply slide the view one screen size to the right to go to the next slide.

This sounds like interacting with Google Maps, or any other modern map app. I wonder if anyone else is using this as a model for user interaction on the desktop?

Check out Mathis's article for more. The section "Everything is a Command Line" reminds me of workspaces in Smalltalk. I used to have several workspaces open, with useful snippets of code just waiting for me to do it. Each workspace window was like a custom text-based menu.

I've always liked the idea of Oberon and even considered using the programming language in my compilers course. (I ended up using a variant.) A version of Compiler Construction is now available free online, so everyone can see how Wirth's clear thinking lead to a sparse, complete, elegant whole. I may have to build the latest installment of Oberon and see what all they have working these days.

The latest example comes from FiveThirtyEight.com, an organization that is built on the idea of data-driven approaches to old problems:

Almost all data about politics that you encounter comes from polls and surveys of individuals or else from analysis of geographic units such as precincts, counties and states. Individual data and geographic data do not capture the essential networks in which we all live -- households and friendships and communities. But other and newer kinds of data -- such as voter files that connect individuals to their households or network data that capture online connections -- revolutionize how we understand politics. By the end of this election cycle, expect to see many more discoveries about the social groupings that define our lives.

Computational research enables us to ask entirely new questions -- both ones that were askable before but not feasible to answer and ones that would not have been conceived before. Even if the question begins as whimsically as "How often do Democrats and Republicans marry one another?"

Back in December 2007, I commented on this in the context of communications studies and public relations. One of our CS master's students, Sergey Golitsynskiy, had just defended an M.A. thesis in communications studies that investigated the blogosphere's impact on a particular dust-up in the public relations world. Such work has the potential to help refine the idea of "the general public" within public relations, and even its nature of "publics". (Sergey is now a tenure-track professor here in communications studies.)

Data that encodes online connections enables us to explore network effects that we can't even see with simpler data. As the 538 piece says, this will revolutionize how we understand politics, along with so many other disciplines.

StrangeLoop 2015 is long in the books for most people, but I occasionally still think about some of the things I learned there. Chris Ford's recent blog post reminded me that I had a draft about his talk waiting to be completed and posted. Had I published this post closer to the conference, I would have called it Kolmogorov Music: Compression and Understanding.

(This is the second follow-up post about a StrangeLoop 2015 talk that is still on my mind. The previous follow-up was about Peter Alvaro's talk on languages and distributed systems.)

Chris Ford stepped the podium in front of an emacs buffer. "Imagine a string of g's," he said, "infinitely long, going in both directions." This is an infinite string, he pointed out, with an 11-word description. That's the basic idea of Kolmogorov complexity, and the starting point for his talk.

I first read about Kolmogorov complexity in a couple of papers by Gregory Chaitin that I found on the early web back in the 1990s. It fascinated me then, and I went into "Kolmogorov Music", Ford's talk, with high hopes. It more than delivered. The talk was informative, technically clear, and entertaining.

Ford uses Clojure for this work in order to write macros. They allow him to talk about code at two levels: source and expansion. The source macro is his description of some piece of music, and the expansion is the music itself, "executable" by an interpreter.

He opened by demo'ing some cool music, including a couple of his own creations. Then he began his discussion of how complex a piece of music is. His measure of complexity is the ratio of the length of the evaluated data (the music) to the length of the macro (the program that generates it). This means that complexity is relative, in part, to the language of expression. If we used a language other than Clojure, the ratios would be different.

Once we settle on a programming language, we can compare the relative complexity of two pieces of music. This also gives rise to cool ideas such as conditional complexity, based on the distance between the programs that encode two pieces of music.

Compression algorithms do something quite similar: exploit our understanding of data to express it in fewer bytes. Ford said that he based his exploration on the paper Analysis by Compression by David Meredith, a "musicologist with a computational bent". Meredith thinks of listening to music as a model-building process that can be described using algorithms.

Programs have more expressive power than traditional music notation. Ford gave as an example clapping music that falls farther and farther behind itself as accompaniment continues. It's much easier to write this pattern using a programming language with repetition and offsets than using musical notation.

Everything has been cool so far. Ford pushed on to more coolness.

A minimalist idea can be described briefly. As Borges reminds us in The Library of Babel, a simple thing can contain things that are more complex than itself. Ford applied this idea to music. He recalled Carl Sagan's novel Contact, in which the constant pi was found to contain a hidden message. Inspired by Sagan, Ford looked to the Champernowne constant, a number created by concatenating all integers in succession -- 0.12345678910111213141516..., and turned it into music. Then he searched it for patterns.

Ford found something that sounded an awful lot like "Blurred Lines", a pop hit by Robin Thicke in 2013, and played it for us. He cheekily noted that his Champernowne song infringes the copyright on Thicke's song, which is quite humorous given the controversial resemblance of Thicke's song to "Got to Give It Up", a Marvin Gaye tune from 1977. Of course, Ford's song is infinitely long, so it likely infringes the copyright of every song ever written! The good news for him is that it also subsumes every song to be written in the future, offering him the prospect of a steady income as an IP troll.

Even more than usual, my summary of Ford's talk cannot possibly do it justice, because he shows code and plays music! Let me echo what was a common refrain on Twitter immediately after his talk at StrangeLoop: Go watch this video. Seriously. You'll get to see him give a talk using only emacs and a pair of speakers, and hear all of the music, too. Then check out Ford's raw material. All of his references, music, and code are available on his Github site.

After that, check out his latest blog entry. More coolness.

In Prime After Prime (great title!), Brian Hayes boils down into two sentences the fundamental challenge that faces people doing research:

What I find most surprising about the discovery is that no one noticed these patterns long ago. They are certainly conspicuous enough once you know how to look for them.

It would be so much easier to form hypotheses and run tests if interesting hypotheses were easier to find.

Once found, though, we can all see patterns. When they can be computed, we can all write programs to generate them! After reading a paper about the strong correlations among pairs of consecutive prime numbers, Hayes wrote a bunch of programs to visualize the patterns and to see what other patterns he might find. A lot of mathematicians did the same.

Evidently that was a common reaction. Evelyn Lamb, writing in Nature, quotes Soundararajan: "Every single person we've told this ends up writing their own computer program to check it for themselves."

Being able to program means being able to experiment with all kinds of phenomena, even those that seemingly took genius to discover in the first place.

Actually, though, Hayes's article gives a tour of the kind of thinking we all can do that can yield new insights. Once he had implemented some basic ideas from the research paper, he let his imagination roam. He tried different moduli. He visualized the data using heat maps. When he noticed some symmetries in his tables, he applied a cyclic shift to the data (which he termed a "twist") to see if some patterns were easier to identify in the new form.

Being curious and asking questions like these are one of the ways that researchers manage to stumble upon new patterns that no one has noticed before. Genius may be one way to make great discoveries, but it's not a reliable one for those of us who aren't geniuses. Exploring variations on a theme is a tactic we mortals can use.

Some of the heat maps that Hayes generates are quite beautiful. The image above is a heat map of the normalized counts of consecutive eight-digit primes, taken modulo 31. He has more fun making images of his twists and with other kinds of primes. I recommend reading the entire article, for its math, for its art, and as an implicit narration of how a computational scientist approaches a cool result.

Yesterday, William Stein's talk about the origins of SageMath spread rapidly through certain neighborhoods of Twitter. It is a thorough and somewhat depressing discussion of how hard it is to develop open source software within an academic setting. Writing code is not part of the tenure reward system or the system for awarding grants. Stein has tenure at the University of Washington but has decided that he has to start a company, SageMath, work for it full-time in order to create a viable open source alternative to the "four 'Ma's": Mathematica, Matlab, Maple, and Magma.

Stein's talk reminded me of something I read earlier this year, from a talk by Matthew Butterick:

"Information wants to be expensive, because it's so valuable ... On the other hand, information wants to be free, because the cost of getting it out is getting lower ... So you have these two fighting against each other."

This was said by a guy named Stewart Brand, way back in 1984.

So what's the message here? Information wants to be free? No, that's not the message. The message is that there are two forces in tension. And the challenge is how to balance the forces.

Proponents of open source software -- and I count myself one -- are often so glib with the mantra "information wants to be free" that we forget about the opposing force. Wolfram et al. have capitalized quite effectively on information's desire to be expensive. This force has an economic power that can overwhelm purely communitarian efforts in many contexts, to the detriment of open work. The challenge is figuring out how to balance the forces.

In my mind, Mozilla stands out as the modern paradigm of seeking a way to balance the forces between free and expensive, creating a non-profit shell on top of a commercial foundation. It also seeks ways to involve academics in process. It will be interesting to see whether this model is sustainable.

Oh, and Stewart Brand. He pointed out this tension thirty years ago. I recently recommended How Buildings Learn to my wife and thought I should look back at the copious notes I took when I read it twenty years ago. But I should read the book again myself; I hope I've changed enough since then that reading it anew brings new ideas to mind.

Yesterday, I wrote me some Java. It was fun.

A few days ago, I started wondering if there was something unique I could send my younger daughter for her birthday today. My daughters and I were all born in presidential election years, which is neat little coincidence. This year's election is special for the birthday girl: it is her first opportunity to vote for the president. She has participated in the process throughout, which has seen both America's most vibrant campaign for progressive candidate in at least forty years and the first nomination of a woman by a major party. Both of these are important to her.

In the spirit of programming and presidential politics, I decided to write a computer program to convert images into the style of Shepard Fairey's iconic Obama "Hope" poster and then use it to create a few images for her.

I dusted off Dr. Java and fired up some code I wrote when I taught media computation in our intro course many years ago. It had been a long time since I had written any Java at all, but it came back just like riding a bike. More than decade of writing code in a language burns some pretty deep grooves in the mind.

I found RGB values to simulate the four colors in Fairey's poster in an old message to the mediacomp mailing list:

    Color darkBlue  = new Color(0, 51, 76);
    Color lightBlue = new Color(112, 150, 158);
    Color red       = new Color(217, 26, 33);
    Color yellow    = new Color(252, 227, 166);

Then came some experimentation...

• First, I tried turning each pixel into the Fairey color to which it was closest. That gave an image that was grainy and full of lines, almost like a negative.


• Then I calculated the saturation of each pixel (the average of its RGB values) and translated the pixel into one of the four colors depending on which quartile it was in. If the saturation was less than 256/4, it went dark blue; if it was less than 256/2, it went red; and so on. This gave a better result, but some images ended up having way too much of one or two of the colors.


• Attempt #2 skews the outputs because in many images (most?) the saturation values are not distributed evenly over the four quartiles. So I wrote a function to "normalize" the quartiles. I recorded all of the saturation values in the image and divided them evenly across the four colors. The result was an image with an equal numbers of pixels assigned to each of the four colors.

I liked the outputs of this third effort quite a bit, at least for the photos I gave it as input. Two of them worked out especially well. With a little doctoring in Photoshop, they would have an even more coherent feel to them, like an artist might produce with a keener eye. Pretty good results for a few fun minutes of programming.

Now, let's hope my daughter likes them. I don't think she's ever received a computer-generated present before, at least not generated by a program her dad wrote!

The images I created were gifts to her, so I'll not share them here. But if you've read this far, you deserve a little something, so I give you these:

Now that is change we can all believe in.

"John McCarthy presents Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I" courtesy of Classic Programmer Paintings

Earlier this week, Alan Kay was answering questions on Hacker News and mentioned Lisp 1.5:

This got deeper if one was aware of how Lisp 1.5 had been implemented with the possibility of late bound parameter evaluation ...

Kay mentions Lisp, and especially Lisp 1.5, often whenever he is talking about the great ideas of computing. He sometimes likens McCarthy's universal Lisp interpreter to Maxwell's equations in physics -- a small, simple set of equations that capture a huge amount of understanding and enable a new way of thinking. Late-bound evaluation of parameters is one of the neat ideas you can find embedded in that code.

The idea of a universal Lisp interpreter is pretty simple: McCarthy defined the features of Lisp in terms of the language features themselves. The interpreter consists of two main procedures:

• a procedure that evaluates an expression, and
• a procedure that applies a procedure to its arguments.

These procedures recurse mutually to evaluate a program.

This is one of the most beautiful ideas in computing, one that we take for granted today.

The syntax and semantics of Lisp programs are so sparse and so uniform that the McCarthy's universal Lisp interpreter consisted of about one page of Lisp code. Here that page is: Page 13 of the Lisp 1.5 Programmer's Manual, published in 1962.

You may see this image passed around the Twitter and the web these whenever Lisp 1.5 is mentioned. But the universal Lisp interpreter is a program. Why settle for a JPG image?

While preparing for the final week of my programming languages course this spring, I sat down and implemented the Lisp interpreter on Page 13 of the Lisp 1.5 manual in universal-lisp-interpreter.rkt, using Racket.

I tried to reproduce the main procedures from the manual as faithfully as I could. You see the main two functions underlying McCarthy's idea: "evaluate an expression" and "apply a function to its arguments". The program assumes the existence of only a few primitive forms from Racket:

• the functions cons, car, cdr, atom, and eq?
• the form lambda, for creating functions
• the special forms quote and cond
• the values 't and nil

't means true, and nil means both false and the empty list. My Racket implementation uses #t and #f internally, but they do not appear in the code for the interpreter.

Notice that this interpreter implements all of the language features that it uses: the same five primitive functions, the same two special forms, and lambda. It also defines label, a way to create recursive functions. (label offers a nice contrast to the ways we talk about implementing recursive functions in my course.)

The interpreter uses a few helper functions, which I also define as in the manual. evcon evaluates a cond expression, and evlis evaluates a list of arguments. assoc looks up the value for a key in an "association list", and pairlis extends an existing association list with new key/value pairs. (In my course, assoc and pairlis correspond to basic operations on a finite function, which we use to implement environments.)

I enjoyed walking through this code briefly with my students. After reading this code, I think they appreciated anew the importance of meaningful identifiers...

The code works. Open it up in Racket and play with a Lisp from the dawn of time!

It really is remarkable how much can be built out of so little. I sometimes think of the components of this program as the basic particles out of which all computation is built, akin to an atomic theory of matter. Out of these few primitives, all programs can be built.

In a Startup School interview earlier this year, Paul Graham reminds software developers of an uncomfortable truth:

Still, to this day, one of the big things programmers do not get is how traumatized users have been by bad and hard-to-use software. The default assumption of users is, "This is going to be really painful, and in the end, it's not going to work."

I have encountered this trauma even more since beginning to work with administrators on campus a decade ago. "Campus solutions" track everything from enrollments to space usage. So-called "business-to-business software" integrates purchasing with bookkeeping. Every now and then the university buys and deploys a new system, to manage hiring, say, or faculty travel. In almost every case, interacting with the software is painful for the user, and around the edges it never quite seems to fit what most users really want.

When administrators or faculty relate their latest software-driven pain, I try to empathize while also bring a little perspective to their concerns. These systems address large issues, and trying to integrate them into a coherent whole is a very real challenge, especially for an understaffed group of programmers. Sometimes, the systems are working exactly as they should to implement an inconvenient policy. Unfortunately, users don't see the policy on a daily basis; they see the inconvenient and occasionally incomplete software that implements it.

Yet there are days when even I have to complain out loud. Using software can be painful.

Today, though, I offer a story of nascent redemption.

After reviewing some enrollment data earlier this spring, my dean apologized in advance for any errors he had made in the reports he sent to the department heads. Before he can analyze the data, he or one of the assistant deans has to spend many minutes scavenging through spreadsheets to eliminate rows that are not part of the review. They do this several times a semester, which adds up to hours of wasted time in the dean's office. The process is, of course, tedious and error-prone.

I'm a programmer. My first thought was, "A program can do this filtering almost instantaneously and never make an error."

In fact, a few years ago, I wrote a simple Ruby program to do just this sort of filtering for me, for a different purpose. I told the dean that I would be happy to adapt it for use in his office to process data for all the departments in the college. My primary goal was to help the dean; my ulterior motive was self-improvement. On top of that, this was a chance to put my money where my mouth is. I keep telling people that a few simple programs can make our lives better, and now I could provide a concrete example.

Last week, I whipped up a new Python script. This week, I demoed it to the dean and an assistant dean. The dean's first response was, "Wow, this will help us a lot." The rest of the conversation focused on ways that the program could help them even more. Like all users, once they saw what was possible, they knew even better what they really wanted.

I'll make a few changes and deliver a more complete program soon. I'll also help the users as they put it to work and run into any bugs that remain. It's been fun. I hope that this humble script is an antidote, however small, to the common pain of software that is hard to use and not as helpful as it should be. Many simple problems can be solved by simple programs.

Brent Simmons has recently suggested that Swift would be better if it were more dynamic. Some readers have interpreted his comments as an unwillingness to learn new things. In Oldie Complains About the Old Old Ways, Simmons explains that new things don't bother him; he simply hopes that we don't lose access to what we learned in the previous generation of improvements. The entry is short and worth reading in its entirety, but the last sentence of this particular paragraph deserves to be etched in stone:

It seemed like magic, then. I later came to understand how it worked, and then it just seemed like brilliance. (Brilliance is better than magic, because you get to learn it.)

This gets to close to the heart of why I love being a computer scientist.

So many of the computer programs I use every day seem like magic. This might seem odd coming from a computer scientist, who has learned how to program and who knows many of the principles that make complex software possible. Yet that complexity takes many forms, and even a familiar program can seem like magic when I'm not thinking about the details under its hood.

As a computer scientist, I get to study the techniques that make these programs work. Sometimes, I even get to look inside the new program I am using, to see the algorithms and data structures that bring to life the experience that feels like magic.

Looking under the hood reminds me that it's not really magic. It isn't always brilliance either, though. Sometimes, it's a very cool idea I've never seen or thought about before. Other times, it's merely a bunch of regular ideas, competently executed, woven together in a way that give an illusion of magic. Regular ideas, competently executed, have their own kind of beauty.

After I study a program, I know the ideas and techniques that make it work. I can use them to make my own programs.

This fall, I will again teach a course in compiler construction. I will tell a group of juniors and seniors, in complete honesty, that every time I compile and execute a program, the compiler feels like magic to me. But I know it's not. By the end of the semester, they will know what I mean; it won't feel like magic to them any more, either. They will have learned how their compilers work. And that is even better than the magic, which will never go away completely.

After the course, they will be able to use the ideas and techniques they learn to write their own programs. Those programs will probably feel like magic to the people who use them, too.

The authors of Our Lives, Encoded found that they had lost access to much of their own digital history to the proprietary data formats of long-dead applications:

Simple text files have proven to be the only format interoperable through the decades. As a result, they are the only artifacts that remain from my own digital history.

I myself have lost access to many WordPerfect files from the '80s in their original form, though I have been migrating their content to other formats over the years. I was fortunate, though, to do most of my early work in VMS and Unix, so a surprising number of my programs and papers from that era are still readable as they were then. (Occasionally, this requires me to dust off troff to see what I intended for them to look like then.)

However, the world continues to conspire against us. Even when we are doing something that is fundamentally plain text, the creators of networks and apps build artificial barriers between their services.

One cannot simply transfer a Twitter profile over to Facebook, or message a Snapchat user with Apple's iMessage. In the sense that they are all built to transmit text and images, these platforms aren't particularly novel, they're just designed to be incompatible.

This is one more reason that you will continue to find me consorting in the ancient technology of email. Open protocol, plain text. Plenty of goodness there, even with the spam.

A local high student emailed me last week to say that he was writing a research paper about encryption and the current conversation going on regarding its role in privacy and law enforcement. He asked if I would be willing to answer a few interview questions, so that he could have a few expert quotes for his paper. I'm always glad when our local schools look to the university for expertise, and I love to help young people, so I said yes.

I have never written anything here about my take on encryption, Edward Snowden, or the FBI case against Apple, so I figured I'd post my answers. Keep in mind that my expertise is in computer science. I am not a lawyer, a political scientist, or a philosopher. But I am an informed citizen who knows a little about how computers work. What follows is a lightly edited version of the answers I sent the student.

1. Do you use encryption? If so, what do you use?

   Yes. I encrypt several disk images that hold sensitive financial data. I use encrypted files to hold passwords and links to sensitive data. My work laptop is encrypted to protect university-related data. And, like everyone else, I happily use https: when it encrypts data that travels between me and my bank and other financial institutions on the web.

2. In light of the recent news on groups like ISIS using encryption, and the Apple v. Department of Justice, do you support legislation that eliminates or weakens powerful encryption?

   I oppose any legislation that weakens strong encryption for ordinary citizens. Any effort to weaken encryption so that the government can access data in times of need weakens encryption for all people at all times and against all intruders.

3. Do you think the general good of encryption (protection of data and security of users) outweighs or justifies its usage when compared to the harmful aspects of it (being used by terrorists groups or criminals)?

   I do. Encryption is one of the great gifts that computer science has given humanity: the ability to be secure in one's own thoughts, possessions, and communication. Any tool as powerful as this one can be misused, or used for evil ends.

   Encryption doesn't protect us from only the U.S. government acting in good faith. It protects people from criminals who want to steal our identities and our possessions. It protects people from the U.S. government acting in bad faith. And it protects people from other governments, including governments that terrorize their own people. If I were a citizen of a repressive regime in the Middle East, Africa, Southeast Asia, or anywhere else, I would want the ability to communicate without intrusion from my government.

   Those of us who are lucky to live in safer, more secure circumstances owe this gift to the people who are not so lucky. And weakening it for anyone weakens it for everyone.

4. What is your response to someone who justifies government suppression of encryption with phrases like "What are you hiding?" or "I have nothing to hide."?

   I think that most people believe in privacy even when they have nothing to hide. As a nation, we do not allow police to enter our homes at any time for any reason. Most people lock their doors at night. Most people pull their window shades down when they are bathing or changing clothes. Most people do not have intimate relations in public view. We value privacy for many reasons, not just when we have something illegal to hide.

   We do allow the police to enter our homes when executing a search warrant, after the authorities have demonstrated a well-founded reason to believe it contains material evidence in an investigation. Why not allow the authorities to enter or digital devices under similar circumstances? There are two reasons.

   First, as I mentioned above, weakening encryption so that the government can access data in times of legitimate need weakens encryption for everyone all the time and makes them vulnerable against all intruders, including bad actors. It is simply not possible to create entry points only for legitimate government uses. If the government suppresses encryption in order to assist law enforcement, there will be disastrous unintended side effects to essential privacy of our data.

   Second, our digital devices are different than our homes and other personal property. We live in our homes and drive our cars, but our phones, laptops, and other digital devices contain fundamental elements of our identity. For many, they contain the entirety of our financial and personal information. They also contain programs that enact common behaviors and would enable law enforcement to recreate past activity not stored on the device. These devices play a much bigger role in our lives than a house.

5. In 2013 Edward Snowden leaked documents detailing surveillance programs that overstepped boundaries spying on citizens. Do you think Snowden became "a necessary evil" to protect citizens that were unaware of surveillance programs?

   Initially, I was unsympathetic to Snowden's attempt to evade detainment by the authorities. The more I learned about the programs that Snowden had uncovered, the more I came to see that his leak was an essential act of whistleblowing. The American people deserve to know what their government is doing. Indeed, citizens cannot direct their government if they do not know what their elected officials and government agencies are doing.

6. In 2013 to now, the number of users that are encrypting their data has significantly risen. Do you think that Snowden's whistleblowing was the action responsible for a massive rise in Americans using encryption?

   I don't know. I would need to see some data. Encryption is a default in more software and on more devices now. I also don't know what the trend line for user encryption looked like before his release of documents.

7. Despite recent revelations on surveillance, millions of users still don't voluntarily use encryption. Do you believe it is fear of being labeled a criminal or the idea that encryption is unpatriotic or makes them an evil person?

   I don't know. I expect that there are a number of bigger reasons, including apathy and ignorance.



8. Encryption defaults on devices like iPhones, where the device is encrypted while locked with a passcode is becoming a norm. Do you support the usage of default encryption and believe it protects users who aren't computer savvy?

   I like encryption by default on my devices. It comes with risks: if I lose my password, I lose access to my own data. I think that users should be informed that encryption is turned on by default, so that they can make informed choices.

9. Should default encryption become required by law or distributed by the government to protect citizens from foreign governments or hackers?

   I think that we should encourage people to encrypt their data. At this point, I am skeptical of laws that would require it. I am not a legal scholar and do not know that the government has the authority to require it. I also don't know if that is really what most Americans want. We need to have a public conversation about this.

10. Do you think other foreign countries are catching up or have caught up to the United States in terms of technical prowess? Should we be concerned?

    People in many countries have astonishing technical prowess. Certainly individual criminals and other governments are putting that prowess to use. I am concerned, which is one reason I encrypt my own data and encourage others to do so. I hope that the U.S. government and other American government agencies are using encryption in an effort to protect us. This is one reason I oppose the government mandating weakness in encryption mechanisms for its own purposes.

11. The United States government disclosed that it was hacked and millions of employees information was compromised. Target suffered a breach that resulted in credit card information being stolen. Should organizations and companies be legally responsible for breaches like these? What reparations should they make?

    I am not a lawyer, but... Corporations and government agencies should take all reasonable precautions to protect the sensitive data they store about their customers and citizens. I suspect that corporations are already subject to civil suit for damages caused by data breaches, but that places burdens on people to recover damages for losses due to breached data. This is another area where we as a people need to have a deeper conversation so that we can decide to what extent we want to institute safeguards into the law.

12. Should the US begin hacking into other countries infrastructures and businesses to potentially damage that country in the future or steal trade secrets similar to what China has done to us?

    I am not a lawyer or military expert, but... In general, I do not like the idea of our government conducting warfare on other peoples and other governments when we are not in a state of war. The U.S. should set a positive moral example of how a nation and a people should behave.

13. Should the US be allowed to force companies and corporations to create backdoors for the government? What do believe would be the fallout from such an event?

    No. See the third paragraph of my answer to #4.

As I re-read my answers, I realize that, even though I have thought a lot about some of these issues over the years, I have a lot more thinking to do. One of my takeaways from the interview is that the American people need to think about these issues and have public conversations in order to create good public policy and to elect officials who can effectively steward the government in a digital world. In order for this to happen, we need to teach everyone enough math and computer science that they can participate effectively in these discussions and in their own governance. This has big implications for our schools and science journalism.

When people ask Jean Yang this question, she reminds them that most of the features in mainstream languages follow decades of research:

Yes, Guido Van Rossum was a programmer and not a researcher before he became the Benevolent Dictator of Python. But Python's contribution is not in innovating in terms of any particular paradigm, but in combining well features like object orientation (Smalltalk, 1972, and Clu, 1975), anonymous lambda functions (the lambda calculus, 1937), and garbage collection (1959) with an interactive feel (1960s).

You find a similar story with Matz and Ruby. Many other languages were designed by people working in industry but drawing explicitly on things learned by academic research.

I don't know what percentage of mainstream languages were designed by people in industry rather than academia, but the particular number is beside the point. The same is true in other areas of research, such as databases and networks. We need some research to look years and decades into the future in order to figure what is and isn't possible. That research maps the terrain that makes more applied work, whether in industry or academia, possible.

Without academics betting years of their career on crazy ideas, we are doomed to incremental improvements of old ideas.

In her 1942 book Philosophy in a New Key, philosopher Susanne Langer wrote:

A question is really an ambiguous proposition; the answer is its determination.

This sounds like something a Prolog programmer might say in a philosophical moment. Langer even understood how tough it can be to write effective Prolog queries:

The way a question is asked limits and disposes the ways in which any answer to it -- right or wrong -- may be given.

Try sticking a cut somewhere and see what happens...

It wouldn't be too surprising if a logical philosopher reminded me of Prolog, but Langer's specialties were consciousness and aesthetics. Now that I think about it, though, this connection makes sense, too.

Prolog can be a lot of fun, though logic programming always felt more limiting to me than most other styles. I've been fiddling again with Joy, a language created by a philosopher, but every so often I think I should earmark some time to revisit Prolog someday.

So, a commodity chess program is now giving odds of a pawn and a move to a world top-ten player -- and winning?

The state of computer chess certainly has changed since the fall of 1979, when I borrowed Mike Jeffers's Chess Challenger 7 and played it over and over and over. I was a rank novice, really just getting my start as a player, yet after a week or so I was able to celebrate my first win over the machine, at level 3. You know what they say about practice...

My mom stopped by our study room several times during that week, trying to get me to stop playing. It turns out that she and my dad had bought me a Chess Challenger 7 for Christmas, and she didn't want me to tire of my present before I had even unwrapped it. She didn't know just how not tired I would get of that computer. I wore it out.

When I graduated with my Ph.D., my parents bought me Chess Champion 2150L, branded by in the name of world champion Garry Kasparov. The 2150 in the computer's name was a rough indication that it played expert-level chess, much better than my CC7 and much better than me. I could beat it occasionally in a slow game, but in speed chess it pounded me mercilessly. I no longer had the time or inclination to play all night, every night, in an effort to get better, so it forever remained my master.

Now US champ Hikaru Nakamura and world champ Magnus Carlsen know how I feel. The days of any human defeating even the programs you can buy at Radio Shack have long passed.

Two pawns and move odds against grandmasters, and a pawn and a move odds against the best players in the world? Times have changed.

Michael Fogus, in the latest issue of Read-Eval-Print-λove, writes:

The book in question was Thinking Forth by Leo Brodie (Brodie 1987) and upon reading it I immediately put it into my own "personal pantheon" of influential programming books (along with SICP, AMOP, Object-Oriented Software Construction, Smalltalk Best Practice Patterns, and Programmers Guide to the 1802).

Mr. Fogus has good taste. Programmers Guide to the 1802 is new to me. I guess I need to read it.

The other five books, though, are in my own pantheon influential programming books. Some readers may be unfamiliar with these books or the acronyms, or aware that so many of them are available free online. Here are a few links and details:

• Thinking Forth teaches us how to program in Forth, a concatenative language in which programs run against a global stack. As Fogus writes, though, Brodie teaches us so much more. He teaches a way to think about programs.


• SICP is Structure and Interpretation of Computer Programs, hailed by many as the greatest book on computer programming ever written. I am sympathetic to this claim.


• AMOP is The Art of the Metaobject Protocol, a gem of a book that far too few programmers know about. It presents a very different and more general kind of OOP than most people learn, the kind possible in a language like Common Lisp. I don't know of an authorized online version of this book, but there is an HTML copy available.


• Object-Oriented Software Construction is Bertrand Meyer's opus on OOP. It did not affect me as deeply as the other books on this list, but it presents the most complete, internally consistent software engineering philosophy of OOP that I know of. Again, there seems to be an unauthorized version online.


• I love Smalltalk Best Practice Patterns and have mentioned it a couple of times over the years [ 1 | 2 ]. Ounce for ounce, it contains more practical wisdom for programming in the trenches than any book I've read. Don't let "Smalltalk" in the title fool you; this book will help you become a better programmer in almost any language and any style. I have a PDF of a pre-production draft of SBPP, and Stephane Ducasse has posted a free online copy, with Kent's blessing.

There is one book on my own list that Fogus did not mention: Paradigms of Artificial Intelligence Programming, by Peter Norvig. It holds perhaps the top position in my personal pantheon. Subtitled "Case Studies in Common Lisp", this book teaches Common Lisp, AI programming, software engineering, and a host of other topics in a classical case studies fashion. When you finish working through this book, you are not only a better programmer; you also have working versions of a dozen classic AI programs and a couple of language interpreters.

Reading Fogus's paragraph of λove for Thinking Forth brought to mind how I felt when I discovered PAIP as a young assistant professor. I once wrote a short blog entry praising it. May these paragraphs stand as a greater testimony of my affection.

I've learned a lot from other books over the years, both books that would fit well on this list (in particular, A Programming Language by Kenneth Iverson) and others that belong on a different list (say, Gödel, Escher, Bach -- an almost incomparable book). But I treasure certain programming books in a very personal way.

I finally got around to preparing my federal tax return this weekend. As I wrote a decade ago, I'm one of those dinosaurs who still does taxes by hand, using pencil and paper. Most of this works involves gathering data from various sources and entering numbers on a two-page Form 1040. My family's finances are relatively simple, I'm reasonably well organized, and I still enjoy the annual ritual of filling out the forms.

For supporting forms such as Schedules A and B, which enumerate itemized deductions and interest and dividend income, I reach into my books. My current accounting system consists of a small set of Python programs that I've been developing over the last few years. I keep all data in plain text files. These files are amenable to grep and simple Python programs, which I use to create lists and tally numbers to enter into forms. I actually enjoy the process and, unlike some people, enjoy reflecting once each year about how I support "we, the people" in carrying out our business. I also reflect on the Rube Goldberg device that is US federal tax code.

However, every year there is one task that annoys me: computing the actual tax I owe. I don't mind paying the tax, or the amount I owe. But I always forget how annoying the Qualified Dividends and Capital Gain Tax Worksheet is. In case you've never seen it, or your mind has erased its pain from your memory in an act of self-defense, here it is:

It may not seem so bad at this moment, but look at that logic. It's a long sequence of "Enter the smaller of line X or line Y" and "Add lines Z and W" instructions, interrupted by an occasional reference to an entry on another form or a case statement to select a constant based on your filing status. By the time I get to this logic puzzle each year, I am starting to tire and just want to be done. So I plow through this mess by hand, and I start making mistakes.

This year I made a mistake in the middle of the form, comparing the wrong numbers when instructed to choose the smaller. I realized my mistake when I got to a line where the error resulted in a number that made no sense. (Fortunately, I was still alert enough to notice that much!) I started to go back and refigure from the line with the error, when suddenly sanity kicked it.

This worksheet is a program written in English, being executed by a tired, error-prone computer: me. I don't have to put up with this; I'm a programmer. So I turned the worksheet into a Python program.

This is what the Qualified Dividends and Capital Gain Tax Worksheet for Line 44 of Form 1040 (Page 44 of the 2015 instruction book) could be, if we weren't still distributing everything as dead PDF:

line   = [None] * 28

line[ 0] =      0.00                     # unused
line[ 1] =      XXXX                     # 1040 line 43
line[ 2] =      XXXX                     # 1040 line  9b
line[ 3] =      XXXX                     # 1040 line 13
line[ 4] = line[ 2] + line[ 3]
line[ 5] =      XXXX                     # 4952 line  4g
line[ 6] = line[ 4] - line[ 5]
line[ 7] = line[ 1] - line[ 6]
line[ 8] =      XXXX                     # from worksheet
line[ 9] = min(line[ 1],line[ 8])
line[10] = min(line[ 7],line[ 9])
line[11] = line[9] - line[10]
line[12] = min(line[ 1],line[ 6])
line[13] = line[11]
line[14] = line[12] - line[13]
line[15] =      XXXX                     # from worksheet
line[16] = min(line[ 1],line[15])
line[17] = line[ 7] + line[11]
line[18] = line[16] - line[17]
line[19] = min(line[14],line[18])
line[20] = 0.15 * line[19]
line[21] = line[11] + line[19]
line[22] = line[12] - line[21]
line[23] = 0.20 * line[22]
line[24] =      XXXX                     # from tax table
line[25] = line[20] + line[23] + line[24]
line[26] =      XXXX                     # from tax table
line[27] = min(line[25],line[26])

i = 0
for l in line:
    print('{:>2} {:10.2f}'.format(i, l))
    i += 1

This is a quick-and-dirty first cut, just good enough for what I needed this weekend. It requires some user input, as I have to manually enter values from other forms, from the case statements, and from the tax table. Several of these steps could be automated, with only a bit more effort or a couple of input statements. It's also not technically correct, because my smaller-of tests don't guard for a minimum of 0. Maybe I'll add those checks soon, or next year if I need them.

Wouldn't it be nice, though, if our tax code were written as computer code, or if we could at least download worksheets and various forms as simple programs? I know I can buy commercial software to do this, but I shouldn't have to. There is a bigger idea at play here, and a principle. Computers enable so much more than sharing PDF documents and images. They can change how we write many ideas down, and how we think. Most days, we barely scratch the surface of what is possible.

Ken Perlin's The Romantic View of Programming opens:

I was attending a panel recently on the topic of new media art that seemed to be culturally split. There were panelists who were talking about abstract concepts like "algorithm" as though these were arcane postmodern terms, full of mysterious power and potential menace.

I encounter this sort of thinking around my own campus. Not everyone needs to learn a lot about computer science, but it would be nice if we could at least alleviate this misunderstanding.

An algorithm is not a magic incantation, even when its implementation seems to perform magic. For most people, as for Perlin, an algorithm is ultimately a means toward a goal. The article closes with the most important implication of the author's romantic view of programming: "And if some software tool you need doesn't exist, you build it." That can be as true for a new media artist as it is for a run-of-the-mill programmer like me.

A colleague sent me the following exchange from his class, with the tag line "Best comments of the day." His students were working in groups to design a Java program for Conway's Game of Life.

Student 1: I can't comprehend what you are saying.

Student 2: The board doesn't have to be rectangular, does it?

Instructor: In Conway's design, it was. But abstractly, no.

Student 3: So you could have a board of different shapes, or you could even have a three-dimensional "board". Each cell knows its neighbors even if we can't easily display it to the user.

Instructor: Sure, "neighbor" is an abstract concept that you can implement differently depending on your need.

Student 2: I knew there was a reason I took linear algebra.

Student 1: Ok. So let's only allow rectangular boards.

Maybe Student 1 still can't comprehend what everyone is saying... or perhaps he or she understands perfectly well and is a pragmatist. YAGNI for the win!

It always makes me happy when a student encounters a situation in which linear algebra is useful and recognizes its applicability unprompted.

I salute all three of these students, and the instructor who is teaching the class. A good day.

Nature heralds AlphaGo's arrival courtesy of the American Go Association

In Why AlphaGo Matters, Ben Kamphaus writes:

AlphaGo recognises strong board positions by first recognizing visual features in the board. It's connecting movements to shapes it detects. Now, we can't see inside AlphaGo unless DeepMind decides they want to share some of the visualizations of its intermediate representations. I hope they do, as I bet they'd offer a lot of insight into both the game of Go and how AlphaGo specifically is reasoning about it.

I'm not sure seeing visualizations of AlphaGo's intermediate representations would offer much insight into either the game of Go or how AlphaGo reasons about it, but I would love to find out.

One of the things that drew me to AI when I was in high school and college was the idea that computer programs might be able to help us understand the world better. At the most prosaic level, I though this might happen in what we had to learn in order to write an intelligent program, and in how we structured the code that we wrote. At a more interesting level, I thought that we might have a new kind of intelligence with which to interact, and this interaction would help us to learn more about the domain of the program's expertise.

Alas, computer chess advanced mostly by making computers that were even faster at applying the sort of knowledge we already have. In other domains, neural networks and then statistical approaches led to machines capable of competent or expert performance, but their advances were opaque. The programs might shed light on how to engineer systems, but the systems themselves didn't have much to say to us about their domains of expertise or competence.

Intelligent programs, but no conversation. Even when we play thousands of games against a chess computer, the opponent seems otherworldly, with no new principles emerging. Perhaps new principles are there, but we cannot see them. Unfortunately, chess computers cannot explain their reasoning to us; they cannot teach us. The result is much less interesting to me than my original dreams for AI.

Perhaps we are reaching a point now where programs such as AlphaGo can display the sort of holistic, integrated intelligence that enables them to teach us something about the game -- even if only by playing games with us. If it turns out that neural nets, which are essentially black boxes to us, are the only way to achieve AI that can work with us at a cognitive level, I will be chagrined. And most pleasantly surprised.

I'm on a mailing list of sports fans who happen also to be geeks of various kinds, including programmers and puzzle nuts. Last Friday, one of my friends posted this link and puzzle to the list:

http://fivethirtyeight.com/features/can-you-win-this-hot-new-game-show/

Two players go on a hot new game show called "Higher Number Wins". The two go into separate booths, and each presses a button, and a random number between zero and one appears on a screen. (At this point, neither knows the other's number, but they do know the numbers are chosen from a standard uniform distribution.) They can choose to keep that first number, or to press the button again to discard the first number and get a second random number, which they must keep. Then, they come out of their booths and see the final number for each player on the wall. The lavish grand prize -- a case full of gold bouillon -- is awarded to the player who kept the higher number. Which number is the optimal cutoff for players to discard their first number and choose another? Put another way, within which range should they choose to keep the first number, and within which range should they reject it and try their luck with a second number?

From there, the conversation took off quiickly with a lot of intuition and analysis. There was immediate support for the intuitive threhold of 0.5, which a simple case analysis shows to give the maximum expected value for a player, 0.625. Some head-to-head analysis of various combinations, however, showed other values winning more often than 0.5, with values around 0.6 doing the best.

What was up? One of these analyses was wrong, but we weren't sure which. One list member, who had built a quick model in Excel, said,

I think the optimum may be somewhere around .61, but I'm damned if I know why.

Another said,

I can't help thinking we're ignoring something game theoretical. I'm expecting we've all arrived at the most common wrong answer.

To confirm his suspicions, this person went off and wrote a C program -- a "terrible, awful, ugly, horrible C program" -- to compute all the expected values for all possible head-to-head cases. He announced,

We have a winner, according to my program. ... 61 wins.

He posted a snippet of the output from his program, which showed a steady rise in the win percentages for cutoffs up to a threshold of 0.61, which beat the 99 other cutoffs, with a steady decline for cutoffs thereafter.

Before reading the conversation on the mailing list, I discussed the puzzle with my wife. We were partial to 0.5, too, but that seemed too simple... So I sat down and wrote a program of my own.

My statistics skills are not as strong as many of my friends, and for this reason I like to write programs that simulate the situation at hand. My Racket program creates players who use all possible thresholds, plays matches of 1,000,000 games between each pair, and tallies up the results. Like the C program written by my buddy, my program is quick and dirty; it replays all hundred combinations on each pass, without taking advantage of the fact that the tournament matrix is symmetric. It's slower than it needs to be, but it gets the job done.

Player 57 defeats 95 opponents.
Player 58 defeats 96 opponents.
Player 59 defeats 99 opponents.
Player 60 defeats 100 opponents.
Player 61 defeats 98 opponents.
Player 62 defeats 96 opponents.
Player 63 defeats 93 opponents.

The results of my simulation mirrored the results of the brute-force case analysis. In simulation, 0.6 won, with 0.59 and 0.61 close behind. The two approaches gave similar enough results that it's highly likely there are bugs in neither program -- or both!

Once my friends were confident that 0.5 was not the winner, they were able to diagnose the error in the reasoning that made us think it was the best we could do: Although a player's choice of strategies is independent of the other player's choice, we cannot treat the other player's value as a uniform distribution over [0..1]. That is true only when they choose a threshold of 0 or 1.

In retrospect, this seems obvious, and maybe it was obvious to my mathematician friends right off the bat. But none of us on the mailing list is a professional statistician. I'm proud that we all stuck with the problem until we understood what was going on.

I love how we can cross-check our intuitions about puzzles like this with analysis and simulation. There is a nice interplay between theory and empirical investigation here. A simple theory, even if incomplete or incorrect, gives us a first approximation. Then we run a test and use the results to go back and re-think our theory. The data helped us see the holes in our thinking. What works for puzzles also works for hairier problems out in the world, too.

And we created the data we ended by writing a computer program. You know how much I like to do that. This the sort of situation we see when writing chess-playing programs and machine learning programs: We can write programs that are smarter than we are by starting from much simpler principles that we know and understand.

This experience is also yet another reminder of why, if I ever go freelance as a consultant or developer, I plan to team up with someone who is a better mathematician than I am. Or at least find such a person to whom I can sub-contract a sanity check.

The quote of the day comes courtesy of the inimitable Matthias Felleisen, on Racket mailing list:

Computer science is the discipline of reinvention. Until everyone who knows how to write 10 lines of code has invented a programming language and solved the Halting Problem, nothing will be settled :-)

One of the great things about CS is that we can all invent whatever we want. One of the downsides is that we all do.

Sometimes, making something simply for the sake of making it is a wonderful thing, edifying and enjoyable. Other times, we should heed the advice carved above the entrance to the building that housed my first office as a young faculty member: Do not do what has already been done. Knowing when to follow each path is a sign of wisdom.

In Reversing the Tide of Declining Expectations Matthew Butterick exhorts designers to expect more from themselves, as well as from the tools they use. When people expect more, other people sometimes tell them to be patient. There is a problem with being patient:

[P]atience is just another word for "let's make it someone else's problem. ... Expectations count too. If you have patience, and no expectations, you get nothing.

But what if you find the available tools lacking and want something better?

Scientists often face this situation. My physicist friends seem always to be rigging up some new apparatus in order to run the experiments they want to run. For scientists and so many other people these days, though, if they want a new kind of tool, they have to write a computer program.

Butterick tells a story that shows designers can do the same:

Let's talk about type-design tools. If you've been at the conference [TYPO Berlin, 2012], maybe you saw Petr van Blokland and Frederick Berlaen talking about RoboFont yesterday. But that is the endpoint of a process that started about 15 years ago when Erik and Petr van Blokland, and Just van Rossum (later joined by many others) were dissatisfied with the commercial type-design tools. So they started building their own. And now, that's a whole ecosystem of software that includes code libraries, a new font-data format called UFO, and applications. And these are not hobbyist applications. These are serious pieces of software being used by professional type designers.

What makes all of this work so remarkable is that there are no professional software engineers here. There's no corporation behind it all. It's a group of type designers who saw what they needed, so they built it. They didn't rely on patience. They didn't wait for someone else to fix their problems. They relied on their expectations. The available tools weren't good enough. So they made better.

That is fifteen years of patience. But it is also patience and expectation in action.

To my mind, this is the real goal of teaching more people how to program: programmers don't have to settle. Authors and web designers create beautiful, functional works. They shouldn't have to settle for boring or cliché type on the web, in their ebooks, or anywhere else. They can make better. Butterick illustrates this approach to design himself with Pollen, his software for writing and publishing books. Pollen is a testimonial to the power of programming for authors (as well as a public tribute to the expressiveness of a programming language).

Empowering professionals to make better tools is a first step, but it isn't enough. Until programming as a skill becomes part of the culture of a discipline, better tools will not always be used to their full potential. Butterick gives an example:

... I was speaking to a recent design-school graduate. He said, "Hey, I design fonts." And I said, "Cool. What are you doing with RoboFab and UFO and Python?" And he said, "Well, I'm not really into programming." That strikes me as a really bad attitude for a recent graduate. Because if type designers won't use the tools that are out there and available, type design can't make any progress. It's as if we've built this great spaceship, but none of the astronauts want to go to Mars. "Well, Mars is cool, but I don't want to drive a spaceship. I like the helmet, though." Don't be that guy. Go the hell to Mars.

Don't be that person. Go to Mars. While you are at it, help the people you know to see how much fun programming can be and, more importantly, how it can help them make things better. They can expect more.

a linearly-polarized gravitational wave Wikimedia Commons (CC BY-SA 3.0 US)

This week the world is excitedly digesting news that the interferometer at LIGO has detected gravitational waves being emitted by the merger of two black holes. Gravitational waves were predicted by Einstein one hundred years ago in his theory of General Relativity. Over the course of the last century, physicists have amassed plenty of indirect evidence that such waves exist, but this is the first time they have detected them directly.

The physics world is understandably quite excited by this discovery. We all should be! This is another amazing moment in science: Build a model. Make a falsifiable prediction. Wait for 100 years to have the prediction confirmed. Wow.

We in computer science can be excited, too, for the role that computation played in the discovery. As physicist Sabine Hossenfelder writes in her explanation of the gravitational wave story:

Interestingly, even though it was long known that black hole mergers would emit gravitational waves, it wasn't until computing power had increased sufficiently that precise predictions became possible. ... General Relativity, though often praised for its beauty, does leave you with one nasty set of equations that in most cases cannot be solved analytically and computer simulations become necessary.

As with so many cool advances in the world these days, whether in the sciences or the social sciences, computational modeling and simulation were instrumental in helping to confirm the existence of Einstein's gravitational waves.

So, fellow computer scientists, celebrate a little. Then, help a young person you know to see why they might want to study CS, alone or in combination with some other discipline. Computing is one of the fundamental tools we need these days in order to contribute to the great tableau of human knowledge. Even Einstein can use a little computational help now and then.

a portion of my bookshelf (CC BY 3.0 US)

Marvin Minsky, one of the founders of AI, died this week. His book Semantic Information Processing made a big impression on me when I read it in grad school, and his paper Why Programming is a Good Medium for Expressing Poorly Understood and Sloppily-Formulated Ideas remains one of my favorite classic AI essays. The list of his students contains many of the great names from decades of computer science; several of them -- Daniel Bobrow, Bertram Raphael, Eugene Charniak, Patrick Henry Winston, Gerald Jay Sussman, Benjamin Kuipers, and Luc Steels -- influenced my work. Winston wrote one of my favorite AI textbooks ever, one that captured the spirit of Minsky's interest in cognitive AI.

It seems fitting that Minsky left us the same week that Google published the paper Mastering the Game of Go with Deep Neural Networks and Tree Search, which describes the work that led to AlphaGo, a program strong enough to beat an expert human Go player. ( This brief article describes the accomplishment and program at a higher level.) One of the key techniques at the heart of AlphaGo is neural networks, an area Minsky pioneered in his mid-1950s doctoral dissertation and continued to work in throughout his career.

In 1969, he and Seymour Papert published a book, Perceptrons, which showed the limitations of a very simple kind of neural network. Stories about the book's claims were quickly exaggerated as they spread to people who had never read the book, and the resulting pessimism stifled neural network research for more than a decade. It is a great irony that, in the week he died, one of the most startling applications of neural networks to AI was announced.

Researchers like Minsky amazed me when I was young, and I am more amazed by them and their lifelong accomplishments as I grow older. If you'd like to learn more, check out Stephen Wolfram's personal farewell to Minsky. It gives you a peek into the wide-ranging mind that made Minsky such a force in AI for so long.

Edge.org's 2016 question for sophisticated minds is, What do you consider the most interesting recent [scientific] news? What makes it important? Joscha Bach's answer is: Everything is computation. Read his essay, which contains some remarkable passages.

Computation changes our idea of knowledge: instead of treating it as justified true belief, knowledge describes a local minimum in capturing regularities between observables.

Epistemology was one of my two favorite courses in grad school (cognitive psych was the other), and "justified true belief" was the starting point for many interesting ideas of what constitutes knowledge. I don't see Bach's formulation as a replacement for justified true belief as a starting point, but rather as a specification of what beliefs are most justified in a given context. Still, Bach's way of using computation in such a concrete way to define "knowledge" is marvelous.

Knowledge is almost never static, but progressing on a gradient through a state space of possible world views. We will no longer aspire to teach our children the truth, because like us, they will never stop changing their minds. We will teach them how to productively change their minds, how to explore the never ending land of insight.

Knowledge is a never-ending process of refactoring. The phrase "how to productively change their minds" reminds me of Jon Udell's recent blog post on liminal thinking at scale. From the perspective that knowledge is a function, "changing one's mind intelligently" is the dynamic computational process that keeps the mind at a local minimum.

A growing number of physicists understand that the universe is not mathematical, but computational, and physics is in the business of finding an algorithm that can reproduce our observations. The switch from uncomputable, mathematical notions (such as continuous space) makes progress possible. Climate science, molecular genetics, and AI are computational sciences. Sociology, psychology, and neuroscience are not: they still seem to be confused by the apparent dichotomy between mechanism (rigid, moving parts) and the objects of their study. They are looking for social, behavioral, chemical, neural regularities, where they should be looking for computational ones.

This is a strong claim, and one I'm sympathetic with. However, I think that the apparent distinction between the computational sciences and the non-computational ones is a matter of time, not a difference in kind. It wasn't that long ago that most physicists thought of the universe in mathematical terms, not computational ones. I suspect that with a little more time, the orientation in other disciplines will begin to shift. Neuroscience and psychology are positioned well for such a phase shift.

In any case, Bach's response points our attention in a direction that has the potential to re-define every problem we try to solve. This may seem unthinkable to many, though perhaps not to computer scientists, especially those of us with an AI bent.

As computer scientists get older, we all find ourselves reminiscing about the computers we knew in the past. I sometimes tell my students about using 5.25" floppies with capacities listed in kilobytes, a unit for which they have no frame of reference. It always gets a laugh.

In a recent blog entry, Daniel Lemire reminisces about the Cray 2, "the most powerful computer that money could buy" when he was in high school. It was took up more space than an office desk (see some photos here), had 1 GB of memory, and provided a peak performance of 1.9 gigaflops. In contrast, a modern iPhone fits in a pocket, has 1 GB of memory, too, and contains a graphics processing unit that provides more gigaflops than the Cray 2.

I saw Lemire's post a day after someone tweeted this image of a 64 GB memory card from 2016 next to a 2 GB Western Digital hard drive from 1996:

The youngest students in my class this semester were born right around 1996. Showing them a 1996 hard drive is like my college professors showing me magnetic cores: ancient history.

This sort of story is old news, of course. Even so, I occasionally remember to be amazed by how quickly our hardware gets smaller and faster. I only wish I could improve my ability to make software just as fast. Alas, we programmers must deal with the constraints of human minds and human organizations. Hardware engineers do battle only with the laws of the physical universe.

Lemire goes a step beyond reminiscing to close his entry:

And what if, today, I were to tell you that in 40 years, we will be able to fit all the computational power of your phone into a nanobot that can live in your blood stream?

Imagine the problems we can solve and the beauty we can make with such hardware. The citizens of 2056 are counting on us.

Peter Naur died early this year at the age of 87. Many of you may know Naur as the "N" in BNF notation. His contributions to CS were much broader and deeper than BNF, though. He received the 2005 Turing Award in recognition of his contributions to programming language and compiler design, including his involvement in the definition of Algol 60. I have always been a huge fan of his essay Programming as Theory Building, which I share with anyone I think might enjoy it.

When Michael Caspersen sent a note to the SIGCSE mailing list, I learned something new about Naur: he coined the term datalogy for "the science of the nature and use of data" and suggested that it might be a suitable replacement for the term "computer science". I had to learn more...

It turns out that Naur coined this term in a letter to the Communications of the ACM, which ran in the July 1966 under the headline "The Science of Datalogy". This letter is available online through the ACM digital library. Unfortunately, this is behind a paywall for many of you who might be interested. For posterity, here is an excerpt from that page:

This is to advocate that the following new words, denoting various aspects of our subject, be considered for general adoption (the stress is shown by an accent):

• datálogy, the science of the nature and use of data,
• datamátics, that part of datalogy which deals with the processing of data by automatic means,
• datámaton, an automatic device for processing data.

In this terminology much of what is now referred to "data processing" would be datamatics. In many cases this will be a gain in clarity because the new word includes the important aspect of data representations, while the old one does not. Datalogy might be a suitable replacement for "computer science."

The objection that possibly one of these words has already been used as a proper name of some activity may be answered partly by saying that of course the subject of datamatics is written with a lower case d, partly by remembering that the word "electronics" is used doubly in this way without inconvenience.

What also speaks for these words is that they will transfer gracefully into many other languages. We have been using them extensively in my local environment for the last few months and have found them a great help.

Finally I wish to mention that datamatics and datamaton (Danish: datamatik and datamat) are due to Paul Lindgreen and Per Brinch Hansen, while datalogy (Danish: datalogi) is my own invention.

I also learned from Caspersen's email that Naur was named the first Professor in Datalogy in Denmark, and held that titled at the University of Copenhagen until he retired in 1998.

Naur was a pioneer of computing. We all benefit from his work every day.

Maciej Cegłowski is in fine form in his talk The Website Obesity Crisis. In it, he mentions recent projects from Facebook and Google to help people create web pages that load quickly, especially for users of mobile devices. Then he notes that their announcements do not practice what the projects preach:

These comically huge homepages for projects designed to make the web faster are the equivalent of watching a fitness video where the presenter is just standing there, eating pizza and cookies.

There is even more irony in creating special subsets of HTML "designed to be fast on mobile devices".

Why not just serve regular HTML without stuffing it full of useless crap?

Wikipedia photo (photographer not credited)

Indeed. Cegłowski offers a simple way to determine whether the non-text elements of your page are useless, which he dubs the Taft Test:

Does your page design improve when you replace every image with William Howard Taft?

(Taft was an American president and chief justice widely known for his girth.)

My blog is mostly text. I should probably use more images, to spice up the visual appearance and to augment what the text says, but doing so takes more time and skill than I often have at the ready. When I do use images, they tend to be small. I am almost certainly more parsimonious than I need to be for most Internet connections in the 2010s, even wifi.

You will notice that I never embed video, though. I dug into the documentation for HTML and found a handy alternative to use in its place: the web link. It is small and loads fast.

Jon Udell looks forward to a time when looking backward digitally requires faithful reanimation of born-digital artifacts:

Much of our culture heritage -- our words, our still and moving pictures, our sounds, our data -- is born digital. Soon almost everything will be. It won't be enough to archive our digital artifacts. We'll also need to archive the software that accesses and renders them. And we'll need systems that retrieve and animate that software so it, in turn, can retrieve and animate the data.

We already face this challenge. My hard drive is littered by files I have a hard time opening, if I am able to at all.

Tim Bray reminds us that many of those "born-digital" artifacts will probably live on someone else's computer, including ones owned by his employer, as computing moves to a utility model:

Yeah, computing is moving to a utility model. Yeah, you can do all sorts of things in a public cloud that are too hard or too expensive in your own computer room. Yeah, the public-cloud operators are going to provide way better up-time, security, and distribution than you can build yourself. And yeah, there was a Tuesday in last week.

I still prefer to have original versions of my documents live on my hardware, even when using a cloud service. Maybe one day I'll be less skeptical, when it really is as unremarkable as Tuesday next week. But then, plain text still seems to me to be the safest way to store most data, so what do I know?

A newcomer to the Racket users mailing list asked which was the better way to promote the language: start a discussion on the mailing list, or ask questions on Stack Overflow. After explaining that neither was likely to promote Racket, Matthew Butterick gave some excellent advice:

Here's one good way to promote the language:

1. Make something impressive with Racket.
2. When someone asks "how did you make that?", give Racket all the credit.

Don't cut corners in Step 1.

This technique applies to all programming languages.

Butterick has made something impressive with Racket: Practical Typography, an online book. He wrote the book using a publishing system named Pollen, which he created in Racket. It's a great book and a joy to read, even if typography is only a passing interest. Check it out. And he gives Racket and the Racket team a lot of credit.

Background

Whenever I teach my compiler course, it seems as if I run across a fun problem or two to implement in our source language. I'm not sure if that's because I'm looking or because I'm just lucky to read interesting blogs and Twitter feeds.

For example, during a previous offering, I read on John Cook's blog about Farey's algorithm for approximating real numbers with rational numbers. This was a perfect fit for the sort of small language that my students were writing a compiler for, so I took a stab at implementing it. Because our source language, Klein, was akin to an integer assembly language, I had to unravel the algorithm's loops and assignment statements into function calls and if statements. The result was a program that computed an interesting result and that tested my students' compilers in a meaningful way. The fact that I had great fun writing it was a bonus.

This Semester's Problem

Early this semester, I came across the concept of excellent numbers. A number m is "excellent" if, when you split the sequence of its digits into two halves, a and b, b² - a² equals n. 48 is the only two-digit excellent number (8² - 4² = 48), and 3468 is the only four-digit excellent number (68² - 34² = 3468). Working with excellent numbers requires only integers and arithmetic operations, which makes them a perfect domain for our programming language.

My first encounter with excellent numbers was Brian Foy's Computing Excellent Numbers, which discusses ways to generate numbers of this form efficiently in Perl. Foy uses some analysis by Mark Jason Dominus, written up in An Ounce of Theory Is Worth a Pound of Search, that drastically reduces the search space for candidate a's and b's. A commenter on the Programming Praxis article uses the same trick to write a short Python program to solve that challenge. Here is an adaptation of that program which prints all of the 10-digit excellent numbers:

    for a in range(10000, 100000):
        b = ((4*a**2+400000*a+1)**0.5+1) / 2.0
        if b == int(b):
           print( int(str(a)+str(int(b))) )

I can't rely on strings or real numbers to implement this in Klein, but I could see some alternatives... Challenge accepted!

My Standard Technique

We do not yet have a working Klein compiler in class yet, so I prefer not to write complex programs directly in the language. It's too hard to get subtle semantic issues correct without being able to execute the code. What I usually do is this:

• Write a solution in Python.
• Debug it until it is correct.
• Slowly refactor the program until it uses only a Klein-like subset of Python.

This produces what I hope is a semantically correct program, using only primitives available in Klein.

Finally, I translate the Python program into Klein and run it through my students' Klein front-ends. This parses the code to ensure that it is syntactically correct and type-checks the code to ensure that it satisfies Klein's type system. (Manifest types is the one feature Klein has that Python does not.)

As mentioned above, Klein is something like integer assembly language, so converting to a Klein-like subset of Python means giving up a lot of features. For example, I have to linearize each loop into a sequence of one or more function calls, recursing at some point back to the function that kicks off the loop. You can see this at play in my Farey's algorithm code from before.

I also have to eliminate all data types other than booleans and integers. For the program to generate excellent numbers, the most glaring hole is a lack of real numbers. The algorithm shown above depends on taking a square root, getting a real-valued result, and then coercing a real to an integer. What can I do instead?

Not to worry. sqrt is not a primitive operator in Klein, but we have a library function. My students and I implement useful utility functions whenever we encounter the need and add them to a file of definitions that we share. We then copy these utilities into our programs as needed.

sqrt was one of the first complex utilities we implemented, years ago. It uses Newton's method to find the roots of an integer. For perfect squares, it returns the argument's true square root. For all other integers, it returns the largest integer less than or equal to the true root.

With this answer in hand, we can change the Python code that checks whether a purported square root b is an integer using type coercion:

    b == int(b)

    isSquareRoot(r : integer, n : integer) : boolean
      n = r*r

(Klein is a pure functional language, so the return statement is implicit in the body of every function. Also, without assignment statements, Klein can use = as a boolean operator.)

Generating Excellent Numbers in Klein

I now have all the Klein tools I need to generate excellent numbers of any given length. Next, I needed to generalize the formula at the heart of the Python program to work for lengths other than 10.

For any given desired length, let n = length/2. We can write any excellent number m in two ways:

• a10ⁿ + b (which defines it as the concatenation of its front and back halves)
• b² - a² (which defines it as excellent)

If we set the two m's equal to one another and solve for b, we get:

        1
    b = -(1 + sqrt[4a2 + 4(10n)a + 1])
        2

Now, as in the algorithm above, we loop through all values for a with n digits and find the corresponding value for b. If b is an integer, we check to see if m = ab is excellent.

The Python loop shown above works plenty fast, but Klein doesn't have loops. So I refactored the program into one that uses recursion. This program is slower, but it works fine for numbers up to length 6:

    > python3.4 generateExcellent.py 6
    140400
    190476
    216513
    300625
    334668
    416768
    484848
    530901

    > python3.4 generateExcellent.py 8
    16604400
    33346668
    59809776
    Segmentation fault: 11