July 21, 2017 3:47 PM

Some Thoughts from a Corporate Visit: Agility and Curriculum

Last Thursday, I spent a day visiting a major IT employer in our state. Their summer interns, at least three of whom are students in my department, were presenting projects they had developed during a three-day codejam. The company invited educators from local universities to come in for the presentations, preceded by a tour of the corporate campus, a meeting with devs who had gone through the internship program in recent years, and a conversation about how the schools and company might collaborate more effectively. Here are a few of my impressions from the visit.

I saw and heard the word "agile" everywhere. The biggest effects of the agility company-wide seemed to be in setting priorities and in providing transparency. The vocabulary consisted mostly of terms from Scrum and kanban. I started to wonder how much the programming practices of XP or other agile methodologies had affected software development practices there. Eventually I heard about the importance of pair programming and unit testing and was happy to know that the developers hadn't been forgotten in the move to agile methods.

Several ideas came to mind during the visit of things we might incorporate into our programs or emphasize more. We do a pretty good job right now, I think. We currently introduce students to agile development extensively in our software engineering course, and we have a dedicated course on software verification and validation. I have even taught a dedicated course on agile software development several times before, most recently in 2014 and 2010. Things we might do better include:

  • having students work on virtual teams. Our students rarely, if ever, work on virtual teams in class, yet this is standard operating procedure even within individual companies these days.

  • having students connect their applications programs to front and back ends. Our students often solve interesting problems with programs, but they don't always have to connect their solution to front ends that engage real users or to back ends that ultimately provide source data. There is a lot to learn in having to address these details.

  • encouraging students to be more comfortable with failure on projects. Schools tends to produce graduates who are risk-averse, because failure on a project in the context of a semester-long course might mean failure in the course. But the simple fact is that some projects fail. Graduates need to be able to learn from failure and create successes out of it. They also need to be willing to take risks; projects with risk are also where big wins come from, not to mention new knowledge.

Over the course of the day, I heard about many of the attributes this company likes to see in candidates for internships and full-time positions, among them:

  • comfort speaking in public
  • ability to handle, accept, and learn from failure
  • curiosity
  • initiative
  • a willingness to work in a wide variety of roles: development, testing, management, etc.
Curiosity was easily the most-mentioned desirable attribute. On the matter of working in a wide variety of roles, even the people with "developer" in their job title reported spending only 30% of their time writing code. One sharp programmer said, "If you're spending 50% of your time writing code, you're doing something wrong."

The codejam presentations themselves were quite impressive. Teams of three to six college students can do some amazing things in three days when they are engaged and when they have the right tools available to them. One theme of the codejam was "platform as a service", and students used a slew of platforms, tools, and libraries to build their apps. Ones that stood out because they were new to me included IBM BlueMix (a l´ AWS and Azure), Twilio ("a cloud platform for building SMS, voice and messaging apps"), and Flask ("a micro web framework written in Python"). I also saw a lot of node.js and lots and lots of NoSQL. There was perhaps a bias toward NoSQL in the tools that the interns wanted to learn, but I wonder if students are losing appreciation for relational DBs and their value.

Each team gave itself a name. This was probably my favorite:

   int erns;
I am a programmer.

All in tools, the interns used too many different tools for me to take note of. That was an important reminder from the day for me. There are so many technologies to learn and know how to use effectively. Our courses can't possibly include them all. We need to help students learn how to approach a new library or framework and become effective users as quickly as possible. And we need to have them using source control all the time, as ingrained habit.

One last note, if only because it made me smile. Our conversation with some of the company's developers was really interesting. At the end of the session, one of the devs handed out his business card, in case we ever wanted to ask him questions after leaving. I looked down at the card and saw...

Alan Kay's business card, redacted

... Alan Kay. Who knew that Alan was moonlighting as an application developer for a major financial services company in the Midwest? I'm not sure whether sharing a name with a titan of computer science is a blessing or a curse, but for the first time in a long while I enjoyed tucking a business card into my pocket.


Posted by Eugene Wallingford | Permalink | Categories: Software Development, Teaching and Learning

July 17, 2017 2:35 PM

Getting Started With Unit Testing

Someone wrote on the Software Carpentry mailing list:

I'm using Python's unittest framework and everything is already set up. The specific problem I need help with is what tests to write...

That is, indeed, a problem. I have the tool. What now?

Rather than snark off-line, like me, Titus Brown wrote a helpful answer with generic advice for how to get started writing tests for code that is already written, aimed at scientists relatively new to software development. It boils down to four suggestions:

  • Write "smoke" tests that determine whether the program works as intended.
  • Write a series of basic tests for edge cases.
  • As you add new code to the program, write tests for it at the same time.
  • "Whenever you discover a bug, write a test against that bug before fixing it."

Brown says that the smoke tests "should be as dumb and robust as possible". They deliver a lot of value for the little effort. I would add that they also get you in the rhythm of writing tests without a huge amount of thought necessary. That rhythm is most helpful as you start to tackle the tougher cases.

Brown calls his last bullet "stupidity driven testing", which is a self-deprecating way to describe a locality phenomenon that many of us have observed in our programs: code in which we've found errors is often likely to contain other errors. Adding tests to this part of the program helps the test suite to evolve to protect a potentially weak part of the codebase.

He also recommends a simple practice for the third bullet that I have found helpful for both bullets three and four: When you write these tests, try to cover some of the existing, working functionality, too. Whenever I add a new test to the growing test base, I try to add one or two more tests not called for by the new code or the bug I'm fixing. I realize that this may distract a bit from the task at hand, but it's a low-cost way to grow the test suite without setting aside dedicated time. I try to keep these add-on tests reasonably close to the part of the program I'm adding or fixing. This allows me to benefit from the thinking I'm already doing about that part of the program.

At some point, it's good to take a look at code coverage to see if there are parts of the code that con't yet have tests written for them. Those parts of the program can be the focus of dedicated test-writing sessions as time permits.

Brown also gives a piece of advice that seasoned developers should already know: make the tests part of continuous integration. They should run every time we update the application. This keeps us honest and our code clean.

All in all, this is pretty good advice. That's not surprising. I usually learn something from Brown's writing.


Posted by Eugene Wallingford | Permalink | Categories: Software Development

July 11, 2017 3:17 PM

Blogging as "Loud Thinking"

This morning, I tweeted a quote from Sherry Turkle's Remembering Seymour Papert that struck a chord with a few people: "Seymour Papert saw that the computer would make it easier for thinking itself to become an object of thought." Here is another passage that struck a chord with me:

At the time of the juggling lesson, Seymour was deep in his experiments into what he called 'loud thinking'. It was what he was asking my grandfather to do. What are you trying? What are you feeling? What does it remind you of? If you want to think about thinking and the real process of learning, try to catch yourself in the act of learning. Say what comes to mind. And don't censor yourself. If this sounds like free association in psychoanalysis, it is. (When I met Seymour, he was in analysis with Greta Bibring.) And if it sounds like it could you get you into personal, uncharted, maybe scary terrain, it could. But anxiety and ambivalence are part of learning as well. If not voiced, they block learning.

It occurred to me that I blog as a form of "loud thinking". I don't write many formal essays or finished pieces for my blog these days. Mostly I share thoughts as they happen and think out loud about them in writing. Usually, it's just me trying to make sense of ideas that cross my path and see where they fit in with the other things I'm learning. I find that helpful, and readers sometimes help me by sharing their own thoughts and ideas.

When I first read the phrase "loud thinking", it felt awkward, but it's already growing on me. Maybe I'll try to get my compiler students to do some loud thinking this fall.

By the way, Turkle's entire piece is touching and insightful. I really liked the way she evoked Papert's belief that we "love the objects we think with" and "think with the objects we love". (And not just because I'm an old Smalltalk programmer!) I'll let you read the rest of the piece yourself to appreciate both the notion and Turkle's storytelling.

Now, for a closing confession: I have never read Mindstorms. I've read so much about Papert and his ideas over the years, but the book has never made it to the top of my stack. I pledge to correct this egregious personal shortcoming and read it as soon as I finish the novel on my nightstand. Maybe I'll think out loud about it here soon.


Posted by Eugene Wallingford | Permalink | Categories: Computing, Teaching and Learning

July 09, 2017 9:04 AM

What It's Like To Be A Scholar

I just finished reading Tyler Cowen's recent interview with historian Jill Lepore. When Cowen asks Lepore about E.B. White's classic Stuart Little, Lepore launches into a story that illustrates quite nicely what it's like to be a scholar.

First, she notes that she was writing a review of a history of children's literature and kept coming across throwaway lines of the sort "Stuart Little, published in 1945, was of course banned." This triggered the scholar's impulse:

And there's no footnote, no explanation, no nothing.
At the time, one of my kids was six, and he was reading Stuart Little, we were reading at night together, and I was like, "Wait, the story about the mouse who drives the little car and rides a sailboat across the pond in Central Park, that was a banned book? What do I not know about 1945 or this book? What am I missing?"

These last two sentences embody the scholar's orientation. "What don't I know about these two things I think I know well?"

And I was shocked. I really was shocked. And I was staggered that these histories of children's literature couldn't even identify the story. I got really interested in that question, and I did what I do when I get a little too curious about something, is I become obsessive about finding out everything that could possibly be found out.

Next comes obsession. Lepore then tells a short version of the story that became her investigative article for The New Yorker, which she wrote because sometimes I "just fall into a hole in the ground, and I can't get out until I have gotten to the very, very bottom of it."

Finally, three transcript pages later, Lepore says:

It was one of the most fun research benders I've ever been on.

It ends in fun.

You may be a scholar if you have this pattern. To me, one of the biggest downsides of becoming department head is having less time to fall down some unexpected hole and follow its questions until I reach the bottom. I miss that freedom.


Posted by Eugene Wallingford | Permalink | Categories: Patterns, Personal, Teaching and Learning

June 30, 2017 1:35 PM

Be Honest About Programming; The Stakes Are High

It's become commonplace of late to promote programming as fun! and something everyone will want to learn, if only they had a chance. Now, I love to program, but I've also been teaching long enough to know that not everyone takes naturally to programming. Sometimes, they warm up to it later in their careers, and sometimes, they never do.

This Quartz article takes the conventional wisdom to task as misleading:

Insisting on the glamour and fun of coding is the wrong way to acquaint kids with computer science. It insults their intelligence and plants the pernicious notion in their heads that you don't need discipline in order to progress. As anyone with even minimal exposure to making software knows, behind a minute of typing lies an hour of study.

But the author does think that people should understand code and what it means to program, but not because they necessarily will program very much themselves:

In just a few years, understanding programming will be an indispensable part of active citizenship.

This is why it's important for people to learn about programming, and why it's so important not to sell it in a way that ambushes students when they encounter it for the first time. Software development is both technically and ethically challenging. All citizens will be better equipped to participate in the world if they understand these challenges at some level. Selling the challenges short makes it harder to attract people who might be interested in the ethical challenges and harder to retain people turned off by technical challenges they weren't expecting.


Posted by Eugene Wallingford | Permalink | Categories: Computing

June 29, 2017 4:03 PM

A New Demo Compiler for My Course

a simple FizzBuzz program

Over the last couple of weeks, I have spent a few minutes most afternoons writing a little code. It's been the best part of my work day. The project was a little compiler.

One of the first things we do in my compiler course is to study a small compiler for a couple of the days. This is a nice way to introduce the stages of a compiler and to raise some of the questions that we'll be answering over the course of the semester. It also gives students a chance to see the insides of a working compiler before they write their own. I hope that this demystifies the process a little: "Hey, a compiler really is just a program. Maybe I can write one myself."

For the last decade or so, I have used a compiler called acdc for this demo, based on Chapter 2 of Crafting A Compiler by Fischer, Cytron, and LeBlanc. ac is a small arithmetic language with two types of numbers, sequences of assignment statements, and print statements. dc is a stack-based desk calculator that comes as part of many Unix installations. I whipped up a acdc compiler in Java about a decade ago and have used it ever since. Both languages have enough features to be useful as a demo but not enough to overwhelm. My hacked-up compiler is also open to improvements as we learn techniques throughout the course, giving us a chance to use them in the small before students applied them to their own project.

I've been growing dissatisfied with this demo for a while now. My Java program feels heavy, with too many small pieces to be simple enough for a quick read. It requires two full class sessions to really understand it well, and I've been hoping to shorten the intro to my course. ac is good, but it doesn't have any flow control other than sequencing, which means that it does not give us a way to look at assembly language generation with jumps and backpatching. On top of all that, I was bored with acdc; ten years is a long time to spend with one program.

This spring I stumbled on a possible replacement in The Fastest FizzBuzz in the West. It defines a simple source language for writing FizzBuzz programs declaratively. For example:

   1...150
   fizz=3
   buzz=5
   woof=7
produces the output of a much larger program in other languages. Imagine being able to pull this language during your next interview for a software dev position!

This language is really simple, which means that a compiler for it can be written in relatively few lines of code. However, it also requires generating code with a loop and if-statements, which requires thinking about branching patterns in assembly language.

The "Fastest FizzBuzz" article uses a Python parser generator to create its compiler. For my course, I want something that my students can read with only their knowledge coming into the course, and I want the program to be transparent enough so that they can see directly how each stage works and how it interacts with other parts of the compiler.

I was also itching to write a program, so I did.

I wrote my compiler in Python. It performs a simple scan of the source program, taking as much advantage of the small set of simple tokens as possible. The parser works by recursive descent, which also takes advantage of the language's simple structure. The type checker makes sure the numbers all make sense and that the words are unique. Finally, to make things even simpler, the code generator produces an executable Python 3.4 program.

I'm quite hopeful about this compiler's use as a class demo. It is simple enough to read in one sitting, even by students who enter the course with weaker programming skills. Even so, the language can also be used to demonstrate the more sophisticated techniques we learn throughout the course. Consider:

  • Adding comments to the source language overwhelms the ad hoc approach I use in the scanner, motivating the use of a state machine.
  • While the parser is easily handled by recursive descent, the language is quite amenable to a simple table-driven approach, too. The table-driven parser will be simple enough that students can get the hang of the technique with few unnecessary details.
  • The type checker demonstrates walking an abstract syntax tree without worrying about too many type rules. We can focus our attention on type systems when dealing with the more interesting source language of their project.
  • The code generator has to deal with flow of control, which enables us to learn assembly language generation on a smaller scale without fully implementing code to handle function calls.
So this compiler can be a demo in the first week of the course and also serve as a running example throughout.

We'll see how well this plays in class in a couple of months. In any case, I had great fun ending my days the last two weeks by firing up emacs or IDLE and writing code. As a bonus, I used this exercise to improve my git skills, taking them beyond the small set of commands I have used on my academic projects in the past. (git rebase -i is almost my friend now.) I also wrote more pyunit tests than I have written in a long, long time, which reminded me of some of the challenges students face when trying to test their code. That should serve me well in the fall, too.

I do like writing code.


Posted by Eugene Wallingford | Permalink | Categories: Software Development, Teaching and Learning

June 25, 2017 10:05 AM

Work on Cool Hard Problems; It Pays Off Eventually

In The Secret Origin Story of the iPhone, we hear about the day Steve Jobs told Bas Ording, one of Apple's UI "wizards", ...

... to make a demo of scrolling through a virtual address book with multitouch. "I was super-excited," Ording says. "I thought, Yeah, it seems kind of impossible, but it would be fun to just try it." He sat down, "moused off" a phone-size section of his Mac's screen, and used it to model the iPhone surface. He and a scant few other designers had spent years experimenting with touch-based user interfaces -- and those years in the touchscreen wilderness were paying off.

I'm guessing that a lot of programmers understand what Ording felt in that moment: "It seems kind of impossible, but it would be fun to just try it..." But he and his team were ready. Sometimes, you work in the wilderness a while, and suddenly it all pays off.


Posted by Eugene Wallingford | Permalink | Categories: Computing

June 23, 2017 2:04 PM

No Summer Job? Learn How to Program

The article Why Aren't American Teenagers Working Anymore? comments on a general trend I have observed locally over the last few years: most high school students don't have summer jobs any more. At first, you might think that rising college tuition would provide an incentive to work, but the effect is almost the opposite:

"Teen earnings are low and pay little toward the costs of college," the BLS noted this year. The federal minimum wage is $7.25 an hour. Elite private universities charge tuition of more than $50,000.

Even in-state tuition at a public universities has grown large enough to put it out of the reach of the typical summer jobs. Eventually, there is almost no point in working a low-paying job; you'll have to borrow significant amount anyway.

These days, students have another alternative that might pay off better in the long run anyway. With a little gumption and free resources available on the web, many students can learn to program, build websites, and make mobile apps. Time spent not working a job but developing skills that are in high demand and which pay well might be time spent well.

Even as a computer scientist, though, I'm traditional enough to be a little uneasy with this idea. Don't young people benefit from summer jobs in ways other than a paycheck? The authors of this article offer the conventional thinking:

A summer job can help teenagers grow up as it expands their experience beyond school and home. Working teens learn how to manage money, deal with bosses, and get along with co-workers of all ages.

You know what, though... A student working on an open-source project can learn also how to deal with people in positions of relative authority and learn how to get along with collaborators of all ages. They might even get to interact with people from other cultures and make a lasting contribution to something important.

Maybe instead of worrying about teenagers getting summer jobs we should introduce them to programming and open-source software.


Posted by Eugene Wallingford | Permalink | Categories: General, Software Development

June 15, 2017 2:14 PM

The Melancholy Department Head

In The Melancholy Dean, Matt Reed notes that, while most management books speak at the level of the CEO or a founder, most managers work further down the chain of command.

Most managers are closer to deans than to presidents. They're in the middle. ... it's not unusual that they find themselves tasked with carrying out policies with which they personally disagree. When success in a position relies largely on "soft power", having to carry out positions with which you personally disagree can be a real strain.
Obviously, if the disagreements become too large or frequent, the right move is to step out of the role. But that's the exception. More commonly, there's a vague sense of "I wouldn't have done it that way" that falls well short of a crisis of conscience, but can be enough to sap motivation. That's especially true when budgets are tightening and adverse decisions are made for you.

I have seen this happen to deans, but I also know the feeling myself. Here, department heads are administrators, and formally they depend upon the dean and provost for their positions. As public universities have to face falling state appropriations, increasing regulatory requirements, and increased competition for students, they often find themselves operating with more of a corporate mentality than the hallowed halls of academia we all dream of from yesteryear. Even with good and open leaders making decisions in upper administration (which I have been fortunate to have in my time as an administrator), more agency lives outside the department, more of the department head's time is spent carrying out activities defined elsewhere, and fewer strategic decisions are made by the head and faculty within the department.

It does wear on a person. These days, academic middle managers of all sorts have to cultivate the motivation they need to carry on. The good news is, through it all, we are helping students, and helping faculty help students. Knowing that, and doing at least a little programming every day, helps me relieve whatever strain I might feel. Even so, I could use more closure most days of the week.


Posted by Eugene Wallingford | Permalink | Categories: General, Managing and Leading

June 12, 2017 2:15 PM

Learn from the Bees

In The Sweet Bees [paywalled], Sue Hubbell writes:

Beekeepers are an opinionated lot, each sure that his methods, and his methods alone, are the proper ones. When I first began keeping bees, the diversity of passionately held opinion bewildered me, but now that I have hives in locations scattered over a thousand-square-mile area I think I understand it.... Frosts come earlier in some places than in others. Spring comes later. Rainfall is not the same. The soils, and the flowering plants they support, are unlike. Through the years, I have learned that as a result of all these variations I must keep the bees variously. Most people who keep bees have only a few hives, and have them all in one place. They find it difficult to understand why practices that have proved successful for them do not work for others. But I have learned that I must treat the bees in one yard quite differently from the way I do those even thirty miles away. The thing to do, I have discovered, is to learn from the bees themselves.

Even though I've taught at only two universities, I've learned this lesson over the years in many ways that don't require physical distance. Teaching novices in an intro course is different from teaching seniors. Teaching a programming course is different from teaching discrete structures or theory of computation. Teaching AI is different from teaching operating systems. I have learned that I must teach differently in different kinds of courses.

In an instructional setting, even more important are the bees themselves. I've been teaching Programming Languages every spring for the last years, and each group of students has been a little different. The course goes better when I have -- and take -- the time to make adjustments according to what I learn over the course of the semester about the particular set of students I have. This spring, I did not recognize the need to adapt quickly enough, and I feel like I let some of the students down.

You sometimes hear faculty talk about students "back in the old days". One thing is certain: the students we had then probably were different from the students we have now. But they were also different from the students that came before them. Each group is new, made up of individuals with their own backgrounds and their own goals.

It's nice when students are similar enough to what we expect that we can take advantage of what worked well last time. We just can't count on that happening all that often. Our job is to teach the students in class right now.

(I first encountered Hubbell's article in To Teach, by William Ayers. I gave a short review of it in yesterday's post.)


Posted by Eugene Wallingford | Permalink | Categories: Teaching and Learning