Session 2

What is Artificial Intelligence?


Artificial Intelligence

On the Turing Test

Last time, I opened with a story, the beginning of Turing's seminal paper "Computing Machinery and Intelligence".

I propose to consider the question, "Can machines think?" ...

If you ever are looking for a way to kill a few minutes in an interesting way some Friday night, try playing this parlor game. Choose four participants: an interrogator, a man, woman, and a "runner" who acts as a teletype. (Better yet, try it using Unix 'talk'!) Your runner's responses should be 'practiced' enough so as not to give anything away. Play for five, ten, or fifteen minutes.

Of course, these days, many of you already have an experience akin to Turing's imitation game: meeting folks in chat rooms and carrying on extended discussions. And I am sure that many times you wonder, "Can this guy be for real??" Turing would be proud. :-)

Considering Turing's Anticipated Objections to the Turing Test

Before proceeding, spend some time answering the following questions:

Objections to the Turing Test

In his paper, Turing anticipates some objections to his game and to the proposition that machines can think if they can play the game as well as a man:

(I have omitted (9) The Argument from Extrasensory Perception. Is that fair? Is it reasonable?)

Most of these arguments are "Yes, but..." objections--they object to the very question "Can machines think?" Perhaps the strongest of these are the consolidated Arguments from Various Disabilities. People who adopt this stance identify a particular skill or behavior that humans exhibit and then claim that computers cannot exhibit the same. But one would have to justify why that is so, and most folks in trying to do so fall back on one of the other, less convincing objections. Turing proposes that people are really using an invalid form of induction when they draw this conclusion in the first place.

The The Argument from Consciousness comes closest to questioning the validity of the game itself. Consider what Turing himself says in reply to it:

This argument is very well expressed in Professor Jefferson's Lister Oration for 1949, from which I quote. "Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain--that is, not only write it but know that it had written it. No mechanism could feel (and not merely artificially signal, an easy contrivance) pleasure at its successes, grief when its valves fuse, be warmed by flattery, be made miserable by its mistakes, be charmed by sex, be angry or depressed when it cannot get what it wants."

This argument appears to be a denial of the validity of our test. According to the most extreme form of this view the only way by which one could be sure that machine thinks is to be the machine and to feel oneself thinking. One could then describe these feelings to the world, but of course no one would be justified in taking any notice. Likewise according to this view the only way to know that a man thinks is to be that particular man. It is in fact the solipsist point of view. It may be the most logical view to hold but it makes communication of ideas difficult. A is liable to believe "A thinks but B does not" whilst B believes "B thinks but A does not." instead of arguing continually over this point it is usual to have the polite convention that everyone thinks.

I am sure that Professor Jefferson does not wish to adopt the extreme and solipsist point of view. Probably he would be quite willing to accept the imitation game as a test. The game (with the player B omitted) is frequently used in practice under the name of viva voce to discover whether some one really understands something or has "learnt it parrot fashion." Let us listen in to a part of such a viva voce:

Interrogator: In the first line of your sonnet which reads "Shall I compare thee to a summer's day," would not "a spring day" do as well or better?

Witness: It wouldn't scan.

Interrogator: How about "a winter's day," That would scan all right.

Witness: Yes, but nobody wants to be compared to a winter's day.

Interrogator: Would you say Mr. Pickwick reminded you of Christmas?

Witness: In a way.

Interrogator: Yet Christmas is a winter's day, and I do not think Mr. Pickwick would mind the comparison.

Witness: I don't think you're serious. By a winter's day one means a typical winter's day, rather than a special one like Christmas.

And so on, What would Professor Jefferson say if the sonnet-writing machine was able to answer like this in the viva voce?

The sort of interrogation that Turing calls viva voce should sound familiar. Teachers do it all of the time, to assess whether a student understands something. The whole Oxonian tradition of tutorials is based on this sort of interaction. Considered this way, my reference to chat rooms above doesn't seem so silly after all!

If a program could carry out such a dialogue with an informed human in one or more areas, could we in any fairness say the program isn't intelligent?

I think that it is untenable to say that a program playing Turing's parlor game well--behaving in the spirit of the game--is not intelligent. Choosing to do so is, in my opinion, a choice to deny a possibility solely because we can't or don't want to imagine a world different than the one we are used to. That isn't a scientific attitude at all.

But what of the game itself?

Is the Turing Test a Good One?

Before proceeding, spend some time answering the following questions:

Challenging the Merits of the Turing Test

What might be wrong with the Turing Test as an evaluator of intelligence?

One of the first problems many people have with Turing's game is that it defines out of intelligence behaviors that aren't strictly cognitive. Is there a distinction between thinking and behavior?

What does Turing's game isolate, if not intelligence?

What is AI?

We still have a hard time defining intelligence--and thus artificial intelligence--today. People who present narrow definitions are usually taken to task, and most of the broader definitions (Gardner's and Root-Bernstein's, for instance) seem rather nebulous and untestable at this point.

Another complication is that the boundaries between what machines can and cannot do continuously move, so the research frontier does, too. (Remember Minsky's definition of AI...)

Since many other disciplines also concern themselves with intelligence, thought, and rationality, AI draws from a wide assortment of ideas: linguistics, psychology, philosophy, mathematics, physiology, neurology, prosthetics, and engineering--and we haven't even mentioned computer science yet!

The Turing Test evokes the dominant naive view of intelligence in the world, and thus of AI: the ability to act like a human. (Here, "naive" means uninformed or uninitiated in the ways of cognitive science.) It sidesteps the attempt to define intelligence and instead attempts to operationalize it.

The Turing Test seems to identify human-like behavior and human-like experience. To play the game well is to use language as we do, to exhibit strengths and weaknesses similar to humans, and to reproduce the peculiarities of human hardware.

For this reason, the authors of another popular AI textbook (Russell and Norvig) characterize the Turing Test as the "Acting Humanly" view of AI. They consider four possible views of AI, in two dimensions:

two dimensions

I called it "naive" view, in the sense that it corresponds to the view of intelligence held by most lay people. Turing's genius in this paper is that he gets to the heart of the naive view in such a way that a most fascinating academic and practical discipline emerges.

On Acting Humanly in the Turing Test Approach

Turing created an argument that most people could understand. In modern times, Hayes and Ford have distinguished between two styles of Turing Test: the gender test, following Turing's original formulation, and the species test, which captures Turing's intent but sidesteps spurious gender-related questions. We have had some fun in previous semesters with this sort of problem. (Besides, what does "successfully take the place of the man" mean anyway?? :-)

These days, the Turing Test is generally framed as a species test: A judge interacts with a human and a machine. The computer is successful if the judge cannot determine which is which after a protracted interaction. Sometimes this is a "mano a mano" game, and other times it occurs in more elaborate scenarios.

Modern Objections

If you care to see some of the other objections that Turing considers and rebuts, read the paper. The breadth of Turing's coverage is quite a work of mind! And it is readable, too.

However, one can raise some more technical objections against the Turing Test. Hayes and Ford suggest a couple:

In Closing

Despite all the objections one can raise to the Turing Test, there is something seductive about its allure (more evidence of Turing's genius). Even though I don't believe that the TT is ultimately the best test for research in AI, something tells me that if we are ever able to create a truly generally intelligent agent it will be able to pass the Turing Test in the way Turing imagined it.

Defining Artificial Intelligence

I have mostly defended the Turing Test so far in this discussion, but I don't subscribe to it as a way to define AI. Ultimately, I am unsatisfied for three reasons:

All that said, I love to re-read Turing's paper every fall and think about the issues he raises.

As I mentioned last time, this is the definition of AI I prefer:

Artificial intelligence is the computational study of how a system can perceive, reason, and act in complex environments.

To make this definition useful, we need to operationalize terms like "perceive", "reason", "act", "complex", and "environment". These are doable; for example, we will begin to characterize complexity next week when we investigate the topic of state-space search. This definition gives me a more explicit statement of what we in AI are doing. Perhaps the folks who like the Turing Test would find its explicitness limiting.

overhead slides in PDF format

Wrap Up

Eugene Wallingford ==== ==== August 30, 2001