TITLE: OOPSLA Day 4: Mary Beth Rosson on the End of Users
AUTHOR: Eugene Wallingford
DATE: October 19, 2005 6:14 PM
DESC: *This* would have been a great keynote for the Educators' Symposium, given my aims for the day.
As you've read here in the past, I am one of a growing
number of CS folks who believe that we must expand the
purview of computer science education far beyond the
education of computer scientists and software developers.
Indeed, our most important task may well lie in the
education of the rest of the educated world -- the
biologists and sociologists, the economists and physicists,
the artists and chemists and political scientists whose
mode of work has been fundamentally altered by the
aggregation of huge stores of data and the creation of
tools for exploring data and building models. The
future of greatest interest belongs not to software
development shops but to the folks doing real work in
So you won't be surprised to know how excited I was to come
to Mary Beth Rosson's
Onward! keynote address
called "The End of Users".
has been an influential researcher across a broad spectrum
of problems in OOP, HCI, software design, and end-user
programming, all of which have had prominent places at
OOPSLA over the years. The common theme to her work is
how people relate to technology, and her methodology has
always had a strong empirical flavor -- watching "users"
of various stripes and learning from their practice how
to better support them.
In today's talk, Mary Beth argued that the relationship
between software developers and software users is changing.
In the old days, we talked about "end-user programming",
those programming-like activities done by those without
formal training in programming. In this paradigm, end
users identify requirements on programs and then developers
produce software to meet the need. This cycle occurs at
a relatively large granularity, over a relatively long
But the world is changing. We now find users operating in
increasingly complex contexts. In the course of doing their
work, they frequently run into ad hoc problems for their
software to solve. They want to integrate pieces of solution
across multiple tools, customize their applications for
specific scenarios, and appropriate data and techniques
from other tools. In this new world, developers must produce
components that can be used in an ad hoc fashion, integrated
across apps. Software developers must create knowledge bases
and construction kits that support an interconnected body of
problems. (Maybe the developers even strive to respond on
These are not users in the traditional sense. We might call
them "power users", but that phrase is already shop-worn.
Mary Beth is trying out a new label: use developers.
She isn't sure whether this label is the long-term solution,
but at least this name recognizes that users are playing an
increasingly sophisticated role that looks more and more like
What sorts of non-trivial tasks do use developers do?
An example scenario: a Word user redefining the 'normal'
document style. This is a powerful tool with big potential
costs if done wrong.
Another scenario: an Excel user creates a large spreadsheet
that embodies -- hides! -- a massive amount of computation.
(Mary Beth's specific example was a grades spreadsheet.
She is a CS professor after all!)
Yet another: an SPSS defines new variables and toggles
between textual programming and graphical programming.
And yet another: a FrontPage user does visual programming
of a web page, with full access to an Access database --
and designs a database!
Mary Beth summarized the characteristics of use developers as:
Creating use developers has potential economic benefits (more
and quicker cycles getting work done) and personal benefits
(more power, more versatility, higher degree of satisfaction).
But is the idea of a use developer good?
Mary Beth quoted an IEEE Software editor whose was
quite dismissive of end users. He warned that they do not
systematically test their work, that they don't know to think
about data security and maintainability, and -- when they do
know to think about these issues -- they don't know *how* to
think about them. Mary Beth thinks these concerns are
representative of what folks in the software world and that
we need to be attentive to them.
(Personally, I think that, while we certainly should be
concerned about the quality of the software produced by end
users, we also must keep in mind that software engineers have
a vested interested in protecting the notion that only
Software Engineers, properly trained and using Methods
Anointed From On High are capable of delivering software
of value. We all know of complaints from the traditional
software engineering community about agile software
development methods, even when the folks implementing and
using agile methods are trained in computing and are,
presumably, qualified to make important decisions about the
environment in which we make software.)
Mary Beth gave an example to illustrate the potential cost
inherent in the lack of dependability -- a Fannie Mae
spreadsheet that contained a $1.2B error.
As the base of potential use developers grows so do the
potential problems. Consider just the spreadsheet and
database markets... By 2012, the US Department of Labor
estimates that there will be 55M end users. 90% of all
spreadsheets contain errors. (Yes, but is that worse or
better than in programs written by professional
software developments?) The potential costs are not just
monetary; they can be related to the quality of life we
all experience. Such problems can be annoying and ubiquitous:
web input forms with browser incompatibilities; overactive
spam filters that lose our mail; Word styles that break
the other formatting in user documents; and policy decisions
based on research findings that themselves are based on
faulty analysis due to errors in spreadsheets and small
Who is responsible for addressing these issues? Both!
Certainly, end users must take on the responsibility of
developing new habits and learning the skills they need
to use their tools effectively and safely. But we in the
software world need to recognize our responsibilities:
- comfortable with a diverse array of software apps
and data sources
- work with them multiple apps in parallel and so want to
pick and choose among functionality at any time, hooking
components up and configuring custom solutions on demand.
- working collaboratively, with group behaviors emerging
- see the computer as a tool, not an end; it should not get
in their way
How do we build better tools?
Mary Beth called them smarter tools and pointed
to a couple of the challenges we must address. First,
much of the computation being done in tools is invisible,
that is, hidden by the user interface. Second, people do
not want to be interrupted while doing their work! (We
programmers don't want that; why should our users have
to put up with it?)
Two approaches that offer promise are interactive
visualization of data and minimalism. By minimalism, she
means not expanding the set of issues that the user has
concern herself with by, say, integrating testing and
debugging into the standard usage model.
The NSF is supporting a five-school consortium called
End Users Shaping Effective Software,
who are trying these ideas out in tool and experiment.
Some examples of their work:
- to build better tools, to build the scaffolding users
need to be effective and safe users. The
tools we build should offer the right amount of help
to users who are in the moment of doing their jobs.
- to promote a "quality assurance" culture among users.
We need to develop and implement new standards for
computing literacy courses.
How can we promote a culture of quality assurance? What is
the cost-benefit trade-off involved for the users? For
Mary Beth indicated three broad themes we can build on:
- CLICKS is a drag-and-drop, design-oriented web
is a help system integrated directly into Alice's user
environment. The help system monitors the state of the
user's program and maintains a dynamic menu of problems
they may run into.
is a JUnit-style interface for debugging spreadsheets,
in which the system keeps an eye on what cells have and
have not been verified with tests.
Some specific examples:
- K-12 education: making quality a part of schoolchildren's
culture of computer use
- universal access: creating tools aimed at specific
populations of users
- communities of practice: evolving reflective practices
within the social networks of users
The talk closed with a reminder that we are just beginning
the transition from thinking of "end users" to thinking of
"use developers", and one of our explicit goals should be
to try to maximize the upside, and minimize the downside,
of the world that will result.
For the first time in a long time, I got up to ask a
question after one of the big talks. Getting up to stand
in line at an aisle mic in a large lecture hall, to ask a
question in front of several hundred folks, seems a rather
presumptuous act. But my interest in this issue is growing
rapidly, and Mary Beth has struck on several issues close
to my current thinking.
My question was this: What should university
educators be thinking about with regard to this transition?
Mary Beth's answer went in a way I didn't anticipate: We
should be thinking about how to help users develop the
metacognitive skills that software developers learn within
our culture of practice. We should extend cultural literacy
curricula to focus on the sort of reflective habits and skills
that users need to have when building models. "Do I know
what's going on? What could be going wrong? What kinds of
errors should I be watching for? How can I squeeze errors
out of my program?"
After the talk, I spent a few minutes discussing curricula
issues more specifically. I told her about our interest
in reaching out to new populations of students, with the
particular example of a testing certificate that folks in
my department are beginning to work on. This certificate
will target non-CS students, the idea being that many non-CS
students end up working as testers in software development for
their domain, yet they don't understand software or testing
or much anything about computing very deeply. This certificate
is still aimed at traditional software development houses,
though I think it will bear the seeds of teaching non-programmers
to think about testing and software quality. If these folks
ever end up making a spreadsheet or customizing Word, the
skills they learn here will transfer directly.
Ultimately, I see some CS departments expanding their computer
literacy courses, and general education courses, to aim at
use developers. Our courses should treat them with the same
care and respect as we treat Programmers and Computer Scientists.
The tasks users do are important, and these folks deserve tools
of comparable quality.
Three major talks, three home runs. OOPSLA 2005 is hot.
- Youngsters who learn by debugging in Alice. This is
ongoing work by Mary Beth's group. Children play in
3D worlds that are broken, and as they play the child
users are invited to fix the system as they play. You
may recognize this as the
pedagogical pattern, but in a "non-programming"
- Debugging tools that appeal to women. Research shows
that women take debugging seriously, but they tend to
use strategies in their heads more than the tools
available in the typical spreadsheet and word
processing systems. How do we invite women with lower
self-confidence to avail themselves of system tools?
One experimental tool does this by letting users
indicate "not sure" when evaluating correctness of a
spreadsheet cell formula.
- Pair programming community simulations. One group has
has a Sim City-like world in which a senior citizen
"pair programs" with a child. Leaving the users
unconstrained led to degeneration, but casting the
elders as object designers and the children as builders
led to coherent creations.
- Sharing and reuse in a teacher community. The Teacher
Bridge project has created a collaborative software
construction tool to support an existing teacher
community. The tool has been used by several groups,
including the one that created
This tool combines a wiki model for its dynamic "web
editor" and more traditional model for its static design
tool (the "folder editor"). Underneath the service,
the system can track user activity in a variety of ways,
which allows us to explore the social connections that
develop within the user community over time.