May 31, 2010 8:48 PM

The End of the Day

... and a fine one, spent with family, enjoying the world. I recall a passage from Josef Albers:

calm down
what happens
happens mostly
without you

Thanks to David Schmüdde for reminding me of Albers's quiet aside.

Posted by Eugene Wallingford | Permalink | Categories: Personal

May 28, 2010 4:48 PM

Good Thing I Can Program...

A friend and colleague sent me this:

So I had a very strange dream last night. I almost never remember dreams, so this was worth bringing up.

You were in grad school ..., getting a second PhD (I think in psychology). I was there visiting you. You were single for some reason. Anyway, while I was there, you were accused of murdering another graduate student. The big evidence that they had of this was a video recording of you and your rock band (you were the lead singer) rehearsing for a gig. The other graduate student (Chinese girl) had been in the band, and suddenly she was totally missing from the video and your microphone was stained red. You and I had gotten a copy of the recording and were running from investigators. We finally got to a video editing lab on campus and were trying to figure out what was going on. We found a spot where the recording had clearly been edited. We were in the midst of finding the original recording when I woke up.

Boy would I like to have someone who interprets dreams take a whack at this one.

After I finish off my second Ph.D., I'm sure I'll be able to help with that.

It is perhaps sad that I am more interesting in my dreams than in real life, and sadder that I am more interesting in other people's dreams than in my own.

It's good to know that my ability to analyze video as data may well help me clear my good name!

Posted by Eugene Wallingford | Permalink | Categories: Personal

May 27, 2010 4:57 PM

Running Experiences, Old and New

Teaching class two-plus hours every day has been keeping me busy and, on some days, wearing me out, but I have maintained a decent running regimen nonetheless. Last weekend I ran my first 5K race in over a year. I have done no speed training since before my marathon last fall, and no training for a race this short in a couple of years. But I have been running approximately 30 miles a week and so had a decent aerobic base to work from. My splits were 7:17, 7:13, and 7:20, with a finish time of 22:26. I finished strong; the second half of the third mile was much faster than the first, which was much the slowest half-mile I ran that morning.

The race was put on by my daughter's high school student government as a fundraiser for a school in Cambodia. There were a lot of HS kids running the race, including several of their best cross-country athletes. Not too surprisingly, those guys smoked all of us duffers from the starting gun. Still, I did all right, quickly jumping to the top fifth of the pack and then picking off runners one by one over the next fifteen minutes. Conservatively, I estimate that I finished in the top fifteen of the 100+ runners. I wouldn't be surprised if I made the top ten. In any case, I cut over a minute off of the time I ran in my first 5K last year and felt good doing it.

The next week, though, I struggled with fatigue, which is pretty typical for the last few months. I seem just fine running 30 or so miles a week without much speed, but every few weeks I hit a trough. I am hoping that some slow and steady training for longer distances will yank me out of that pattern.

5Ks are old news for me. What's new? I was planning to run an 8-mile race the week before the 5K, but heavy rain and local flooding of our trails forced race organizers to postpone that event until late July. Eight miles would be something quite new for me; I've never even run a 10K. That new experience will have to wait.

For something really new, I am talking with a friend about putting together a six-man team to run the 192-mile Great River Relay, one of the races in the Ragnar Relay Series. The event is open to teams of six or twelve, but my buddy and I agree that it would be much cooler to run 50K or so each in twenty-four hours. (Not to mention how much easier it is to find four more crazy runners than to find ten!) I have certainly never run an "ulra" race distance, usually defined as anything more than the marathon's 26.2 miles. As a relay, this isn't really an ultra, because I would be able to rest for a couple of hours between legs. It would be a new experience, though!

Training for this will also be a new challenge. I am sure that it requires marathon-like mileage and workouts. But it probably also demands some days running two, three, or more times, to acclimate the body to running marginally fast for 4-8 miles six times in a day -- and overnight. I'm starting to get excited about this idea.

I am also thinking of a fall marathon. That goal, and how training for it fits with the relay, is currently TBA. One big challenge at a time...

Posted by Eugene Wallingford | Permalink | Categories: Running

May 25, 2010 4:16 PM

A Book on Alan Kay / Alan Kay on Science and Engineering

Everyone already seems to know about this, but I didn't until yesterday, when I saw a note from Hal Perkins...

In honor of Alan Kay's 70th birthday, a bunch of his colleagues and best friends have published Points of View, "a collection of previously-unpublished essays". The list of authors ranges from CS greats such as Ivan Sutherland, Butler Lampson, and Leonard Kleinrock, to luminaries from other human endeavors such as Betty Edwards and Quincy Jones. The list of essay titles makes me want to drop all of my work and read it from cover to cover right now. There may be a second printing of the book, but it is available in PDF from the website for free.

Very cool.

I've been meaning to post a short snippet from Kay for the last few weeks. He wrote a guest entry on Mark Guzdial's blog about the danger of trying to make learning 'too simple', a common theme in how Kay talks about understanding and building systems. The passage that really attracted me was on something else, though, in a comment responding to other commenters' comments. Considering that Kay's blog entry itself began as a comment intended for another entry, we have more than enough indirection to make us computer scientists happy for weeks!

In his comment, Kay describes some of his team's current work at Viewpoints, in which they extract from programs of millions of code an essential core that is executable, readable, and yet several orders of magnitude smaller than the original. He uses this description as a vehicle for explaining science and engineering:

In this direction -- where we have phenomena already, and our task is to make a much more compact, understandable and debuggable model -- we are doing actual science (this is what science does and is).

But we can also use the model idea to go in the other direction -- to come up with an idea, make a runnable debuggable model of it -- and then use this as a guide and reference ... for manifesting the ... artifact -- here, we are doing invention and engineering.

Computer scientists are often asked to explain why CS is a science. Kay has always been very clear that science is about method, not subject matter. His explanation of science and engineering nicely captures how CS can be science, how CS can be engineering, and how CS can be both. The last these is part of what distinguishes computer science from other disciplines, precisely because CS is about models of systems.

For me, this simple example was the big win from Kay's guest post on Mark's blog. It's also a fitting example of how Kay thinks about the world and why accomplished people would take the time to write essays to celebrate his birthday.

Posted by Eugene Wallingford | Permalink | Categories: Computing

May 24, 2010 8:31 PM

Teaching TDD and Incremental Design

Today in the lab while developing code, we encountered another example of stories colliding. We are early in the project, and most parts of the system are still inchoate. So collisions are to be expected.

Today, two pairs were working on stories involving Account objects, which at the start of the day knew only their account number, account name, and current balance. In both stories, the code would have to presume a more knowledgable object, one that knows something about the date of on which the balance is in effect and the various entries that have modified the balance over time.

One team asked, "How can we proceed without knowing the result of the other story?" More importantly, how can either team proceed without knowing how journal transactions will be recorded as entries to the accounts? Implicit in the question, sometimes, is a suggestion disguised as a question: Isn't this an example of where we should do a little up-front design?

In a professional setting, a little up-front design might be the right answer. But with newcomers to XP and TDD, I am trying to have us all think in as pure an XP way as possible. Finding the right point on the continuum between too little up-front design and too much is something better done once the developer has more experience with both ways of working.

This situation is actually a perfect place for us to reinforce the idea behind TDD and why it can help us write better software. Whatever the two pairs do right now, there will likely be some conflicts that need to be merged. Taking that as a given, how can the pairs proceed best? As they write their tests, each should ask itself,

What is the simplest interface we can possibly use to implement this story?

When we write a test, we design a little part of our system's internal interface. Students are used to knowing everything about an already-designed object when they write code to use the object. Programming test-first forces us to think about the interface first, without being privy to implementation. This is good, as it will encourage us to design components that are as loosely coupled as possible. Stories we implement later will impose more specific details on how the object behaves, and we can handle more detailed implementation issues then. This is good, as it encourages us (1) to write simple tests that do not presume any more about the object than is required, and (2) to do the simplest thing that could possibly work to implement the new behavior, because those later stories may well cause our implementation to be extended or changed altogether.

Our story collision is both an obstacle of sorts and an opportunity to let our tests drive us forward in small steps!

This collision also has another lesson in store for us. The whole team has been avoiding a couple of stories about closing journals at the end of the month. Implementing these stories will teach us a lot about what accounts know and look like. By avoiding them, the team has made implementing some of our simplest stories more contingent than they need to be.

Over the weekend, Kent Beck tweeted:

got stuck. wrote a test. unstuck.

A bit he later he followed up:

i get stuck trying to write all the logic at once. feels great to deliberately ignore cases that had me stumped. "that's another test"

This is a skill I hope my students can develop this month. In order for that to happen, I need to watch for opportunities to point them in the right direction. When a pair is at an impasse, unsure of what to do next, I need to suggest that they step back and try to take a smaller step. Write a test for that something smaller, and see where that leads them. The Account saga is a useful example for me to keep in mind.

If nothing else, teaching this course in real time -- in a lab with students testing, designing, and coding all the time -- makes clear something we all know. It is one thing to be able to do something, to react and act in response to the world. It is another thing all together to teach or coach others to do the same thing. I have to have ready at hand questions to ask and suggestions to make as students encounter situations that I handle subconsciously through ingrained experience. Teaching this course is fun on a lot of levels.

Posted by Eugene Wallingford | Permalink | Categories: Software Development, Teaching and Learning

May 19, 2010 4:39 PM

TDD Exploration on an Agile Student Project

Man, I having fun teaching my agile course. Writing code is fun, and talking design and technique with students in real time is fun. Other than being so intense as to tire me out every day, I think I could get used to this course-in-a-month model.

I had expected that this week would be our first iteration, but it became clear early on that the student team did not understand the domain of our project -- a simple home accounting system I might use -- well enough to begin a development iteration. Progress would have been too slow, and integration too halting, to make the time well-spent.

So, in agile fashion, we adapted. We began our time together yesterday by discussing the problem a bit more at the one-page story level. The team was having difficulty with idea of special journals and different kinds of transactions. We collectively decided to focus on the general journal for recording all transactions, and so adjusted our thinking and our stories.

Then we took inspiration from XP's practice of a spike solution. In XP, a spike is a simple program that helps a team to explore a thorny technical or design problem and learn enough to begin working on live code. As Ward Cunningham relates, a spike is the answer to the question, "What is the simplest thing we can program that will convince us we are on the right track?" For our team, the problem wasn't technical or design-related; it was a purely a matter of insufficient domain understanding.

We paired up, took a simple story, wrote a test, and wrote code. The story was:

Record a check written on May 15 for cash, in the amount of $100.

By writing even one test for this story, the team began to learn new things about the system to be built. For example,

  • A transaction is atomic. You can't add a debit to a journal independent of its balancing credit.

  • Recording a transaction does not update any account. That happens when the journal is closed at the end of the period.

  • There is a difference between the model, the core computation and data of a program, and the view, the way users see or experience the program's behavior. We can and usually should think about these parts of our program as independent.

The first two of these lessons are about the domain. The third is about software design. Both kinds of lesson are essential ones for a young team to to learn, or be reminded of, before building the system.

Some pairs explored this story and its implications for an hour or more. Others tried to forge ahead further, with a second story:

Record receipt of a $200 paycheck, with $100 going to my checking account, $20 to my prepaid medical expense account, and $80 to income tax withholding.

Again, these teams learned something: A transaction may consist of multiple debits or credits. This also means that a transaction must be able to record multiple amounts, unlike in the first story, because several debits may total up to the value of single credit. Finally, if there are multiple debits, the sum of their values must total exactly to the value of the single credit.

Each little bit of learning will help the team to begin to code productively and to be prepared to grow the design of the system.

The development team was not the only party who learned a lot with this spike. By watching the pairs implement a story or two, offering advice and answering questions, I did, too. I play two roles on this project, both as a teacher of sorts. I am the customer for the product and fully intend to use it when the course ends. This makes me a teacher of the domain, both specifically this program and generally Accounting 101. I am also the coach, which finds me helping to guide the XP process as well as teaching students a bit about Ruby, software design, and OO.

By collaborating with the development team as they wrote spike-like code, I learned a lot about how to write better stories. This is true of me as customer, who realized that my original stories lacked the concrete focus the team needed to be able to focus on essential features of the program. It is also true of me as coach, who realized that certain stories would be especially useful in helping the team arrive at a more valuable design more quickly.

It is more than okay for the coach and the customer to learn as much from working with the team as the team members themselves; it is expected. That's one of the great attractions and one of the great assets of agile software development. My students are getting to see that early in their experience with XP.

Tomorrow, I think we will shake of our experimental mindset and begin to write our program. I don't have a lot of experience with a team starting a brand-new system from Code Zero in XP; most of my greenfield development has been on projects where I am the only programmer or the only one writing the initial code base. I am considering trying out advice that @jamesshore tweeted last week:

When starting brand-new product, codebase too small for 8 programmers to work separately. Instead, use projector. 1 driver, 7 nav

This seems like a great way to build an initial code base around a set of commonly-understood objects and interfaces. If I try it out, I will need to avoid one temptation. I will surely want to drive, but I should let a student -- a member of the development team -- control the keyboard!

Posted by Eugene Wallingford | Permalink | Categories: Software Development, Teaching and Learning

May 17, 2010 5:19 PM

Course Notes from the Treadmill

Teaching a course two hours a day , especially a course that is essentially new in this incarnation, feels like running on ice. I'm enjoying every day, but tomorrow becomes today way too fast!

At the end of last week, I began to feel the effects of compressing a 3-credit course into four weeks. At the end of Week 1, we are a quarter of the way through the course. But one week is not really enough time for all these new ideas to soak into a student's brain or fingertips. TDD, refactoring, pairing, .... Ruby, an IDE, a VCS, ... Our brains take time to adjust. The students are doing remarkably well under the conditions, but some of them are feeling the rush of days, too.

I most noticed the compression in my conflicting desires to do stuff and to talk more about stuff before doing anything big. Most professors tend to err on the side of talking more, but that isn't the best way to learn most disciplines. I decided that we had seen enough background on XP and that students had practiced enough on small exercises such as the spreadsheet TDD challenge and refactoring Fowler's code, Ruby style. It was time to start building software, and learn as we go. So today we played the Planning Game and put ourselves in position to write Line 1 of code tomorrow.

It's been interesting talking to students about XP's practices. Pairing seemed odd to many of them at first, but they seem to have taken to it quickly. They are social beings. Refactoring seems like the Right Thing To Do to many of them, but in practice it is hard. Using a tool like Reek to identify some smells and an IDE like RubyMine to perform some of the refactoring will help, but RubyMine does not yet implement enough different refactorings to really dampen their fear of breaking code.

TDD is causing a couple of programmers fits, because it inverts how they think about coding. When it comes time to write tests for the app they are building -- no longer a small exercise in their minds -- I expect us to struggle as we think about simple design steps. I hope, though, that this practice will get them over the hump to see how writing tests early or first can really affect how we think about our code.

I am still surprised when developers bemoan their inability to deliver working code and then balk so mightily at a practice that could help them in a major way. But, as we all know, old habits die hard. When the mind is ready, change can happen. All we can hope for in a course is to try to be in position for change to occur whenever the mind becomes ready.

Posted by Eugene Wallingford | Permalink | Categories: Software Development, Teaching and Learning

May 14, 2010 9:15 PM

Greatness, Skill, and Metaprogramming

It's been a long week teaching and doing end-of-year reports for the department, not to mention putting out daily fires. I have a few things to say about the agile development course at the 1/4 mark, but another day.

While writing reports this evening, I listened to several talks and interviews. One was Giles Bowkett's talk on meta-programming at the 2008 Mountain West Ruby Conference. Actually, Bowkett objects to the the idea of meta-programming, as I discussed a few months ago. At one level, I agree with him; it's all just programming. In this talk, he elaborates on this position and does a little just-programming in Ruby to generate code.

The part of this talk that stood out for me this evening was the part of his conclusion in which he discusses Paul Graham's recent work. Bowkett summarizes most of Graham's writing about Lisp, programming, and meta-programming as:

Great programmers can write better programmers than they can hire.

He disagrees with this sentiment in only one word: 'great'. After comically mocking an undue focus on greatness that he attributes to most Harvard grads, he explains that he prefers the more straightforward 'skilled': Skilled programmers can write better programmers than they can hire.

I prefer 'skilled' to 'great' too, because 'great' intimidates too many people. They think other people are or can be great, but that they themselves can be merely ordinary. Maybe so, but ordinary programmers can improve their skills and learn new things. Most ordinary programmers can become skilled programmers, even in the dark art of metaprogramming. They, too, can learn to write better programmers than they can hire, or be.

Of course, this implies that we can help most of the programmers we want to hire be better programmers, by helping them to develop the skills that they need to be good.

If you watch the talk, watch out for a his egregious botching of Lisp syntax in the course of demeaning all those evil parentheses that Lisp foists on us. I would tell him the same thing I tell my students: the parentheses aren't nearly as bad -- or as numerous -- once you learn how to use them properly!

Posted by Eugene Wallingford | Permalink | Categories: Software Development

May 12, 2010 7:41 AM

A Morning in the Life

My mailbox this morning contained a long complaint from a student about a grade, a request to meet with a faculty member about an issue he didn't handle well this spring, a request for space for summer undergrad research, news that a student hold has been cleared so that a summer course can be created, a spreadsheet of my department's fiscal year-to-date spending for end-of-year planning, calls from upper administration for work on student outcomes assessment, merit pay evaluations, and year-end faculty evaluation letters, and several dozen miscellaneous messages. That doesn't count mailing-list mail, both professional and personal, which is procmailed into separate boxes. Such is the life of a department head.

I also received from a student a link to a negative review of git. I am considering using distributed VCS in my agile course, with git and Mercurial at the top of the list. Do you have a suggestion? Mercurial has been around longer, I think, and there are some good tutorials on it. My students need to get up to speed relatively quickly with the basic use cases of whatever tool we use, for a team of ten doing XP-style development.

After two days of class, I feel pretty good about the prospects of this group of ten students. They seem willing to collaborate and eager to learn. We are taking to heart the agile value "Responding to change over following a plan", with a change to Ruby as our working language and (git | Mercurial) as our VCS. This means I get to be agile, too, as I prepare some new course materials built around Ruby, Test::Unit, and so on. I'll be looking into RubyMine as an IDE with refactoring support. Do you have any experience using it? Let me know.

Posted by Eugene Wallingford | Permalink | Categories: Managing and Leading, Teaching and Learning

May 09, 2010 11:53 AM

Technical Debt as Consumption or Investment

A couple of days ago I retweeted a one-liner from @unclebobmartin:

All developers know that bad code slows them down; yet nearly all insist that writing bad code is faster.

This struck me as one of the themes I'd like for my agile software developments students to pick up on this month. It's tempting to write code quick and dirty so that you can feel as if you are ready to move on to the next task. But I as I mentioned here recently, "dirty remains long after quick has been forgotten". The feeling of completion is an illusion, one that we end up paying for later.

A recent student commented that this is the traditional trade-off between "pay now" versus "pay later":

I'd compare it to buying a car today using a loan versus paying outright next year. It's cheaper if you're willing to wait, but can you?

This is an apt analogy, because it allows us to peel off a layer and consider the decision at a deeper level. For example, if I can borrow money at a lower net interest rate than I can earn investing my money elsewhere, then it makes sense for me to borrow. This is a facet of financial borrowing that I think we can learn from in the context of software development. If we can borrow, in the form of technical debt, at a lower net cost than the value of the benefit we can accrue by putting our efforts elsewhere, then it makes sense for us to incur the technical debt.

Seth Godin wrote recently that consumer debt is not your friend. Thinking in this way about software development, we can contrast borrowing for consumption and borrowing for investment. Taking on "consumer" technical debt is a sucker's bet, a losing proposition. This is the sort of debt that agile developers rightly warn us about. Rushing through a story with inadequate testing or with inattention to the shape of the system after we add our code -- just so that we can make a tick on the burndown chart and get more stories done -- this is a habit that eventually buries a team in an avalanche of debt. Pretty soon, we can barely keep up with the minimum monthly payment, and we watch our total debt grow faster than we can add new value to the system. The result is inevitable: bankruptcy.

However, this analogy tells us that there may be a kind of debt that we should be willing to take on. As I described my own thinking above, if I can borrow money at a lower net cost than I can earn investing elsewhere, then it makes sense for me to borrow. Godin writes of borrowing in order to improve your productivity or to buy things that go up in value. In the software development world, this is what I call investment technical debt. If a team makes a thoughtful, conscious decision to let a part of the system fall out of compliance with its usual standards for coding, testing or design because doing so lets them create greater value in another way, then they will be better off in the future for taking on the debt. This is investment, and done well it can pay.

This is the sort of debt that Kent Beck has written about in recent months when he dared to say that it might be okay not to write a test. He has taken a lot of grief from some XP folks, who seem to fear that talking about not following agile practices to the letter will give other developers license to do the wrong thing under the mantle of Beck's advice. I feel for those folks. Investment can be dangerous. People lose money in the stock market all the time, and developers drown in technical debt all the time. It is important for beginners to learn solid fiscal habits, and good development practices, before venturing too far into the world of investing.

But Kent is speaking truth, the same truth my student alluded to when raising the pay-now versus pay-later trade-off. With experience and expertise, developers can begin to take on technical debt for the purpose of investment -- and not only survive, but thrive as a result.

Of course, it is essential that the investor be as honest as possible with herself about being able to repay the debt later. The team must be able to brings its test coverage back up to safe levels for the long term, and it must be able to refactor the system to bring its living design back up to a level that it supports ongoing development. Some teams like to pay down their debt in one or a few large focused episodes. This is akin selling a stock to paying off a note in whole, and I have done this in my own programming more than once.

When I can, though, I prefer to amortize the work of paying off technical debt over a more extended set of development tasks. I still think of this in terms of a relatively quick repayment, maybe a few iterations, because I don't want to the burden of the debt to affect the rhythm of development any longer than it must. (Similarly, I would never take out a 5-year loan to buy a car.) But I prefer the amortized approach because it fits better with how I want to think about my work: always conscious of the state of the system, always taking small steps to make my programs better. As a creature of habit, I like to work within a set of practices that keep me focused on continuous improvement.

That sort of development is not easy to do, or to learn. It's one of the challenges that newcomers to agile software development must overcome. I think back to another part of my student's comment: "It's cheaper if you're willing to wait, but can you?" I think this points out another common choice people face all the time: want versus need. The consumer debt situation in the U.S. is founded in large part on the fact that many consumers confuse wanting something with needing it. I imagine that some of the so-called software crisis has its roots in the same confusion. "We have to write subpar code in order to meet our goal..." -- only to find the team unable to meet its goal in party precisely because it cut corners earlier.

Godin's article closes with an exhortation to resist consumer debt in the face of temptation:

Stuff now is rarely better than stuff later, because stuff now costs you forever if you go into debt to purchase it. ... It takes discipline to forego pleasure now to avoid a lifetime of pain and fees.

Software developers are wise when they take this advice to heart, too. Investment debt is a good idea in certain circumstances, once you have the experience and expertise to take it on wisely, manage it, and pay it off promptly. Consumer debt is always a loser.

My summer agile course is less than 24 hours away. My mind is turning on all cylinders...

Posted by Eugene Wallingford | Permalink | Categories: Software Development