I can't let my previous post be my only comment on Software in Scientific Research. Hinsen's bigger point is worth a post of its own.
Software is a means of communication, just like papers or textbooks.
... much like the math that appears in a paper or a textbook -- except that, done properly, a computer program runs and provides a dynamic demonstration of an idea.
The main questions asked about scientific software [qua software] are "What does it do?" and "How efficient is it?" When considering software as a means of communication, we would ask questions such as "Is it well-written, clear, elegant?", "How general is the formulation?", or "Can I use it as the basis for developing new science?".
This shift requires a different level of understanding of programs and programming than many scientists (and other people who do not program for a living) have. But it is a shift that needs to take place, so we should so all we can to help scientists and others become more fluent. (Hey to Software Carpentry and like-minded efforts.)
We take for granted that all researchers are responsible for being able to produce and, more importantly, understand the other essential parts of scientific communication:
We actually accept as normal that the scientific contents of software, i.e., the models implemented by it, are understandable only to software specialists, meaning that for the majority of users, the software is just a black box. Could you imagine this for a paper? "This paper is very obscure, but the people who wrote it are very smart, so let's trust them and base our research on their conclusions." Did you ever hear such a claim? Not me.
This is a big part of the challenge we face in getting faculty across the university to see the vital role that computing should play in modern education -- as well as the roles it should not play. The same is true in the broader culture. We'll see if efforts such as code.org can make a dent in this challenge.
On the way to making a larger point about the role of software in scientific research, Konrad Hinsen writes these beautiful sentences:
Software is just data that can be interpreted as instructions for a computer. One could conceivably write some interpreter that turns previously generated data into software by executing it.
They express one side of one of the great ideas of computer science, the duality of program and data:
This is one of the reasons why it is so important for CS students to study the principles of programming languages, create languages, and build interpreters. These activities help bring this great idea to life and prepare those who understand it to solve problems in ways that are otherwise hard to imagine.
Besides, the duality is a thing of beauty. We don't have to use it as a tool in order to appreciate this sublime truth.
As Hinsen writes, few people outside of computer science (and, sadly, too many within CS) appreciate "the particular status of software as both tool an information carrier and a tool". The same might be said for our appreciation of data, and the role that language plays in bridging the gap between the two.
I would rather have too little architecture
than too much because that might interfere
with the truth of what I say.
-- Ivan Turgenev
How can agile approaches to software development create coherent programs? Don't we need a plan, a well-thought-out design to follow closely? Without one, won't we end up with a mess?
Let's turn again to Italo Calvino for some perspective. He is known for novels with intricate structure and acknowledges that, for at least a decade, the "architecture" of his books occupied more of his mind than it should have. Yet his novels never seemed to follow closely any of his advance plans:
I spend a lot of time constructing a book, making outlines that eventually prove to be of no use to me whatsoever. I throw them away. What determines the book is the writing, the material that's actually on the page.
The ultimate architecture of a book comes to life alongside the book itself, hand-in-hand with the writing.
And so it can be with software. Programmers can and should think about what they are building but, in the end, what determines the program is the programming, the material that's actually on the page or in the browser.
Again, we must careful not to take the analogy too far. Programs are often created for external clients with specific technical requirements. Programmers are not usually free, as novelists are, to change the product they are creating. Even so, design is how we make the product, not what it does. Whether it evolves during the course of programming or is specified up-front, the client can receive the product they asked us to make.
Ward Cunningham once gave what is, for me, still the best definition of design:
Design is the thinking you do when you make something.
The most important product of that thinking is not a design document or an architecture. It is the mind that is prepared to make the thing you need to make.
I wrote on Twitter Thursday [ 1 | 2 ] that I end up modifying my lecture notes every semester, no matter how well done they were the last time I taught the course. From one semester to the next, I find that I am more likely to change the introductions, transitions, and conclusions of a session than the body. The intros, transitions, and conclusions help to situate the material in a given place and time: the context of this semester and this set of students. The content, once refined, tends to stabilize, though occasionally I feel a need to present even it in a different way, to fit the current semester.
Novelist Italo Calvino knew this feeling as well, when he was preparing to be interviewed:
Rarely does an interviewer ask questions you did not expect. I have given a lot of interviews and I have concluded that the questions always look alike. I could always give the same answers. But I believe I have to change my answers because with each interview something has changed either inside myself or in the world. An answer that was right the first time may not be right again the second.
This echoes my experience preparing for lecture. The answer that was right the last time does not seem right again this time. Sometimes, I have changed. With any luck, I have learned new things since the last time I taught the course, and that makes for a better story. Sometimes, the world has changed: a new programming language such as Clojure or Scala has burst onto the scene, or a new trend in industry such as mobile app development has made a different set of issues relevant to the course. I need to tell a different story that acknowledges -- and takes advantage of -- these changes.
Something else always changes for a teacher, too: the students. It's certainly true the students in the class are different every time I teach a course. But sometimes, the group is so different from past groups that the old examples, stories, and answers just don't seem to work. Such has been the case for me this semester. I've had to work quite a bit to understand how my students think and incorporate that into my class sessions and homework assignments. This is part of the fun and challenge of being a teacher.
We have to be careful not to take this analogy too far. Teaching computer science is different from an author giving an interview about his or her life. For one thing, there is a more formal sense of objective truth in the content of, say, a programming language course. An object is still a closure; a closure is still an object that other code can interact with over time. These answers tend to stay the same over time. But even as a course communicates the same fundamental truths from semester to semester, the stories we need to tell about these truths will change.
Ever the fantastic writer, Calvino saw in his interview experience the shape of a new story, a meta-story of sorts:
This could be the basis of a book. I am given a list of questions, always the same; every chapter would contain the answers I would give at different times. ... The changes would then become the itinerary, the story that the protagonist lives. Perhaps in this way I could discover some truths about myself.
This is one of the things I like about teaching. I often discover truths about myself, and occasionally transform myself.
~~~~
The passages quote above come from The Art of Fiction No. 130, Italo Calvino in The Paris Review. It's not the usual Paris Review interview, as Calvino died before the interviewer was done. Instead, it is a pastiche of four different sources. It's a great read nonetheless.
Over the weekend, I read Hypothetical Reasoning and Failures of Disjunctive Inference, a well-sourced article on the problems people have making disjunctive inferences. It made me think about some of the challenges students have learning to program.
Disjunctive inference is reasoning that requires us to consider hypotheticals. A simple example from the article is "the married problem":
Jack is looking at Ann, but Ann is looking at George. Jack is married, but George is not. Is a married person looking at an unmarried person?
- Yes.
- No.
- Cannot be determined.
The answer is yes, of course, which is obvious if we consider the two possible cases for Ann. Most people, though, stop thinking as soon as they realize that the answer hinges on Ann's status. They don't know her status, so they can't know the answer to the question. Even so, most everyone understands the answer as soon as the reasoning is explained to them.
The reasons behind our difficulties handling disjunctive inferences are complex, including both general difficulties we have with hypotheticals and a cognitive bias sometimes called cognitive miserliness: we seek to apply the minimum amount of effort to solving problems and making decisions. This is a reasonable evolutionary bias in many circumstances, but here it is maladaptive.
The article is fascinating and well worth a full read. It points to a number of studies in cognitive psychology that seek to understand how humans behave in the face if disjunctive inferences, and why. It closes with some thoughts on improving disjunctive reasoning ability, though there are no quick fixes.
As I read the article, it occurred to me that learning to program places our students in a near-constant state of hypothetical reasoning and disjunctive inference. Tracing code that contains an if statement asks them to think alternative paths and alternative outcomes. To understand what is true after the if statement executes is disjunctive inference.
Something similar may be true for a for loop, which executes once each for multiple values of a counter, and a while loop, which runs an indeterminate number of times. These aren't disjunctive inferences, but they do require students to think hypothetically. I wonder if the trouble many of my intro CS students had last semester learning function calls involved failures of hypothetical reasoning as much as it involves difficulties with generalization.
And think about learning to debug a program.... How much of that process involves hypotheticals and even full-on disjunctive inference? If most people have trouble with this sort of reasoning even on simple tasks, imagine how much harder it must be for young people who are learning a programming language for the first time and trying to reason about programs that are much more complex than "the married problem"?
Thinking explicitly about this flaw in human thinking may help us teachers do a better job helping students to learn. In the short term, we can help them by giving more direct prompts for how to reason. Perhaps we can also help them learn to prompt themselves when faced with certain kinds of problems. In the longer term, we can perhaps help them to develop a process for solving problems that mitigates the bias. This is all about forming useful habits of thought.
If nothing else, reading this article will help me be slower to judge my students's work ethic. What looks like laziness is more likely a manifestation of a natural bias to exert the minimum amount of effort to solving problems. We are all cognitive misers to a certain extent, and that serves us well. But not always when we are writing and debugging programs.
Think differently, of course. But be humble. These attitudes go hand-in-hand.
To make money in the markets, you have to think independently and be humble. You have to be an independent thinker because you can't make money agreeing with the consensus view, which is already embedded in the price. Yet whenever you're betting against the consensus there's a significant probability you're going to be wrong, so you have to be humble.
This applies equally well to doing research. You can't make substantial progress with the conventional wisdom, because it defines and limits the scope of the solution. So think differently. But when you leave the safety of conventional wisdom, you find yourself working in an immense space of ideas. There is a significant chance that you will be wrong a lot. So be humble.
(The quote is from Learn or Die: Using Science to Build a Leading-Edge Learning Organization by Ray Dalio.)
After reading What Wittgenstein Learned from Teaching Elementary School in The Paris Review, I think that, while Wittgenstein seems to have had some ideas for making elementary education better, I probably wouldn't want to have him as my teacher. This passage, though, really stuck with me:
We all struggle to form a self. Great teaching, Wittgenstein reminds us, involves taking this struggle and engaging in it with others; his whole life was one great such struggle. In working with poor children, he wanted to transform himself, and them.
The experience of teaching grade school for six years seems to have changed Wittgenstein and how he worked. In his later work, he famously moved away from the idea that language could only function by picturing objects in the world. There is no direct evidence that working with children was the impetus for this shift, but "his later work is full of references to teaching and children". In particular, Philosophical Investigations begins its investigation of "the essence of language" by discussing how children learn language.
And Wittgenstein is sometimes explicit about the connection; he once said that in considering the meaning of a word, it's helpful to ask, "How would one set about teaching a child to use this word?"
We all know that teaching can change the student, but foremost it changes the teacher. Wittgenstein seems to have understood that this is a part of the universal task of forming one's self.