January 08, 2021 2:12 PM

My Experience Storing an Entire Course Directory in Git

Last summer, I tried something new: I stored the entire directory of materials for my database course in Git. This included all of my code, the course website, and everything else. It worked well.

The idea came from a post or tweet many years ago by Martin Fowler who, if I recall correctly, had put his entire home directory under version control. It sounded like the potential advantages might be worth the cost, so I made a note to try it myself sometime. I wasn't quite ready last summer to go all the way, so I took a baby step by creating my new course directory as a git repo and growing it file by file.

My context is pretty simple. I do almost all of my work on a personal MacBook Pro or a university iMac in my office. My main challenge is to keep my files in sync. When I make changes to a small number of files, or when the stakes of a missing file are low, copying files by hand works fine, with low overhead and no tooling necessary.

When I make a lot of changes in a short period of time, however, as I sometimes do when writing code or building my website, doing things by hand becomes more work. And the stakes of losing code or web pages are a lot higher than losing track of some planning notes or code I've been noodling with. To solve this problem, for many years I have been using rsync and a couple of simple shell scripts to manage code directories and my course web sites.

So, the primary goal for using Git in a new workflow was to replace rsync. Not being a Git guru, as many of you are, I figured that this would also force me to live with git more often and perhaps expand my tool set of handy commands.

My workflow for the semester was quite simple. When I worked in the office, there were four steps:

  1. git merge laptop
  2. [ do some work ]
  3. git commit
  4. git push

On my laptop, the opening and closing git commands changed:

  1. git pull origin main
  2. [ do some work ]
  3. git commit
  4. git push origin laptop

My work on a course is usually pretty straightforward. The most common task is to create files and record information with commit. Every once in a while, I had to back up a step with checkout.

You may say, "But you are not using git for version control!" You would be correct. The few times I checked out an older version of a file, it was usually to eliminate a spurious conflict, say, a .DS_Store file that was out of sync. Locally, I don't need a lot of version control, but using Git this way was a form of distributed version control, making sure that, wherever I was working, I had the latest version of every file.

I think this is a perfectly valid way to use Git. In some ways, Git is the new Unix. It provided me with a distributed filesystem and a file backup system all in one. The git commands ran effectively as fast as their Unix counterparts. My repo was not very much bigger than the directory would have been on its own, and I always had a personal copy of the entire repo with me wherever I went, even if I had to use another computer.

Before I started, several people reminded me that Git doesn't always work well with large images and binaries. That didn't turn out to be much of a problem for me. I had a couple of each in the repo, but they were not large and never changed. I never noticed a performance hit.

The most annoying hiccup all semester was working with OS X's .DS_Store files, which record screen layout information for OS X. I like to keep my windows looking neat and occasionally reorganize a directory layout to reflect what I'm doing. Unfortunately, OS X seems to update these files at odd times, after I've closed a window and pushed changes. Suddenly the two repos would be out of sync only because one or more .DS_Store files had changed after the fact. The momentary obstacle was quickly eliminated with a checkout or two before merging. Perhaps I should have left the .DS_Stores untracked...

All in all, I was pretty happy with the experience. I used more git, more often, than ever before and thus am now a bit more fluent than I was. (I still avoid the hairier corners of the tool, as all right-thinking people do whenever possible.) Even more, the repository contains a complete record of my work for the semester, false starts included, with occasional ruminations about troubles with code or lecture notes in my commit messages. I had a little fun after the semester ended looking back over some of those messages and making note of particular pain points.

The experiment went well enough that I plan to track my spring course in Git, too. This will be a bigger test. I've been teaching programming languages for many years and have a large directory of files, both current and archival. Not only are there more files, there are several binaries and a few larger images. I'm trying decide if I should put the entire folder into git all at once upfront or start with an empty folder a lá last semester and add files as I want or need them. The latter would be more work at early stages of development but might be a good way to clear out the clutter that has built up over twenty years.

If you have any advice on that choice, or any other, please let me know by email or on Twitter. You all have taught me a lot over the years. I appreciate it.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

January 05, 2021 3:33 PM

Today's Reading

Lot's of good stuff on the exercise bike this morning...

Henry Rollins on making things because he must:

I'm a shipbuilder. I don't want to sail in them. I want you to sail in them. I'm just happy that they leave the harbor so I can have an empty workplace.

Deirdre Connolly on the wonder of human achievement:

We, ridiculous apes with big brains and the ability to cooperate, can listen to the universe shake from billions of light years away, because we learned math and engineering. Wild.

Sonya Mann on our ultimate task:

Our labor is the same as it ever was. Your job is to pioneer a resilient node in the network of civilization -- to dodge the punches, roll with the ones that you can't, and live to fight another day. That's what our ancestors did for us and it's what we'll do for those who come next: hold the line, explore when there's surplus, stay steady, and go down swinging when we have to.

Henry Rollins also said:

"What would a writer do in this situation?" I don't know, man. Ask one. And don't tell me what he said; I'm busy.

Back to work.


Posted by Eugene Wallingford | Permalink | Categories: General

January 03, 2021 5:08 PM

On the Tenth Day of Christmas...

... my daughter gave to me:

Christmas gifts from Sarah!

We celebrated Part 2 of our Zoom Family Christmas this morning. A package from one of our daughters arrived in the mail after Part 1 on Christmas Day, so we reprised our celebration during today's weekly call.

My daughter does not read my blog, at least not regularly, but she did search around there for evidence that I might already own these titles. Finding none, she ventured the long-distance gift. It was received with much joy.

I've known about I Am a Strange Loop for over a decade but have never read it. Somehow, Surfaces and Essences flew under my radar entirely. A book that is new to me!

These books will provide me many hours of enjoyment. Like Hofstadter's other books, they will probably bend my brain a bit and perhaps spark some welcome new activity.

~~~~~

Hofstadter appears in this blog most prominently in a set of entries I wrote after he visited my university in 2012:

I did mention I Am a Strange Loop in a later entry after all, a reflection on Alan Turing, representation, and universal machines. I'm glad that entry did not undermine my daughter's gift!


Posted by Eugene Wallingford | Permalink | Categories: General, Personal

January 01, 2021 12:01 PM

This Version of the Facts

The physicist Leo Szilard once announced to his friend Hans Bethe that he was thinking of keeping a diary: "I don't intend to publish it; I am merely going to record the facts for the information of God." "Don't you think God knows the facts?" Bethe asked. "Yes," said Szilard. "He knows the facts, but He does not know this version of the facts."

I began 2021 by starting to read Disturbing the Universe, Freeman Dyson's autobiographical attempt to explain to people who are not scientists what the human situation looks like to someone who is a scientist. The above passage opens the author's preface.

Szilard's motive seems like a pretty good reason to write a blog: to record the one's own version of the facts, for oneself and for the information of God. Unlike Szilard, we have an alternative in between publishing and not publishing. A blog is available for anyone to read, at almost no cost, but ultimately it is for the author, and maybe for God.

I've been using the long break between fall and spring semesters to strengthen my blogging muscle and redevelop my blogging habit. I hope to continue to write more regularly again in the coming year.

Dyson's book is a departure from my recent reading. During the tough fall semester, I found myself drawn to fiction, reading Franny and Zooey by J. D. Salinger, The Bell Jar by Sylvia Plath, The Lucky Ones by Rachel Cusk, and The Great Gatsby by F. Scott Fitzgerald, with occasional pages from André Gide's diary in the downtime between books.

I've written about my interactions with Cusk before [ Outline, Transit, Kudos ], so one of her novels is no surprise here, but what's with those classics from sixty years ago or more? These stories, told by deft and observant writers, seemed to soothe me. They took the edge off of the long days. Perhaps I could have seen a run of classic books coming... In the frustrating summer run-up to fall, I read Thomas Mann's Death in Venice and Ursula Le Guin's The Lathe of Heaven.

For some reason, yesterday I felt the urge to finally pick up Dyson's autobiography, which had been on my shelf for a few months. A couple of years ago, I read most of Dyson's memoir, Maker of Patterns, and found him an amiable and thoughtful writer. I even wrote a short post on one of his stories, in which Thomas Mann plays a key role. At the time, I said, "I've never read The Magic Mountain, or any Mann, for that matter. I will correct that soon. However, Mann will have to wait until I finish Dyson...". 2020 may have been a challenge in many ways, but it gave me at least two things: I read my first Mann (Death in Venice is much more approachable than The Magic Mountain...), and it returned me to Dyson.

Let's see where 2021 takes us.


Posted by Eugene Wallingford | Permalink | Categories: General, Personal