Of course the idea here is to give a little background on AI and the various methods that have been used to try and produce intelligent behavior in computer programs.

There are two goals to AI, the biggest one is to produce an artificial system that is about as good as or better than a human being at dealing with the real world. The second goal is more modest: simply produce small programs that are more or less as good as human beings at doing small specialized tasks that require intelligence. To many AI researchers simply doing these tasks that in human beings require intelligence counts as Artificial Intelligence even if the program gets its results by some means that does not show any intelligence, thus much of AI can be regarded as "advanced programming techniques".

The characteristics of intelligence given here would be quite reasonable I think to most normal people however AI researchers and AI critics take various unusual positions on things and in the end everything gets quite murky. Some critics believe that intelligence requires thinking while others say it requires consciousness. Some AI researchers take the position that thinking equals computing while others don't.

One of the worst ideas to come up in these debates is the Turing test for thinking. People argue the validity of this test forever and ever. Every few months one of these discussions gets going in comp.ai.philosophy and then goes on for a month or so before it dies out. Notice that the goal of the Turing test is to fool naive questioners for a little while so in effect a major pre-occupation of AI has been to devise programs that fool the public and this is a very bad way go give the subject credibility. In fact one analog to the Turing test is the following test. I give you two test tubes, one has a tiny amount of fools gold at the bottom and the other has a tiny amount of real gold. I give you a few minutes to JUST look at them (no chemical or physical tests!) and then you must tell me which one has the real gold. Odds are that after I test a fair number of ordinary people half the people will be right and the other half will be wrong. Then since ordinary people after a short amount of time can't tell the difference between the two are they the same? Would you buy fools gold from me at the current price for gold (about $300 an ounce)? Yet this is exactly what the Turing test proposes for thinking. Maybe this IS one of Turing's jokes!

A much more meaningful method of determining whether or not a computer is thinking would be to find out exactly what people are doing and then if the artificial system is doing the same thing or so close to the same thing as a human being then it becomes fair to equate the two. Since writing the book I came up with a longer essay on this issue, see the page http://www.dontveter.com/caipfaq/testing.html. At any rate the only use I can see for the Turing test is in carnival side-shows.

One of the positions on intelligence that I mention in this section is that it requires consciousness and consciousness is produced by quantum mechanics. For those of you who have been denied a basic education in science by our schools quantum mechanics goes like this. By the beginning of the 20th Century physicists had discovered electrons, protons and various other very small particles were not obeying the known laws of Physics. After a while it became clear that particles also behave like waves. The most well-known formula they found to describe how the particles move around is the Schrodinger wave equation, a second order partial differential equation that uses complex numbers. Since then quantum formulas have been developed for electricity and magnetism as well. Although as far as anyone can tell the formulas get the right answers the interpretation of what's really going on is in dispute. The formulas and some experiments apparently show that at times information is moved around not just as the very slow speed of light but INSTANTLY. Results like this are very unsettling. One of the developers of QM, Neils Bohr once said: "Anyone who isn't confused by quantum mechanics doesn't really understand it." But, relax, there is virtually no QM in this book, certainly not the formulas and nothing to get confused about. On the other hand research into applying quantum mechanics to human thought may soon make it necessary to include QM in Psychology and AI books. For a sample of what may be coming see my The Quantum Basis of Natural Intelligence? WWW page.

Another embarrassing idea floating around AI is the idea that if a big enough computer executes a complex enough algorithm the computer will become conscious. Let me document this with a quote from a page by Anne Foerst, the Theological adviser to the Cog project at MIT. The page title is: ``Cog'', A Humanoid Robot and the Question of Imago Dei The context is that Dr. Foerst is arguing against the idea that there are things that computers cannot do (see the book by Stuart and Hubert Dreyfus, What Computers Can't Do: The Limits of Artificial Intelligence) and here she talks about consciousness:

The Cog builders' argument against this type of attack is twofold. First, they argue that phenomena like consciousness are an illusion; consciousness for instance arises because our brain is complex enough to abstract and categorize certain processes and analyze them [10, p.12] and emotions arise from chemical reactions in the body and their reflections and responses in the brain [12, pp.127]. Using this materialistic approach towards all complex phenomena which seem to be exclusive for humans, they can, secondly, argue that Cog is built accordingly to all these materialistic findings; as Cog becomes increasingly complex it might develop the same complexity and, hence, develop the same illusions.

Bottom lining it then, there are terrible disagreements between various camps in and out of AI as to what really constitutes intelligence and it will be a long time before it is sorted out.

Perhaps the most important principle in all of AI is found here, when one idea has been associated with another then when one comes up again the other will too. This was put very nicely by William James in his works on Psychology first published in 1890 (Yes, eighteen ninety!). There are several examples given. James' books are well worth reading if you ever have the time to spare. His other ideas come up in cognitive science fairly often.

When people first discovered that nerve cells pass around pulses of electricity many people ASSUMED that this activity was used to produce thought. Heck, what else was there, if you were a materialist? (The only alternative was the idea that there is a human soul and the materialists didn't want anything to do with that!) Now there is a new theory floating about because of recent discoveries about cells. Almost every cell has a cytoskeleton built out of hollow tubes of protein. The tubes are called microtubules and there is now a proposal that these structures are involved in quantum mechanical computing. If thinking is done this way then the human mind is a much more powerful computing device than anyone has suspected. I've seen estimates that the brain processes from 10^23 to 10^28 bits per second with this architecture, well beyond the 10^16 bits per second estimate with the neuron as a simple switch architecture. Another suggestion is that the microtubules could be set up to allow optical computing. For more on the microtubules see The Quantum Basis of Natural Intelligence? page and/or the article by Dimitri Nanopoulos available from Los Alamos National Laboratory

For the sake of trying to produce intelligent behaviour however really all that's being done is work with artificial neural networks where each cell is a very simple processor and the goal is to try and make them work together to solve some problem. That's all that gets covered in this book. Many people are skeptical that artificial neural networks can produce human levels of performance because they are so much simpler than the biological neural networks.

Symbol Processing has been the dominant theory of how intelligence is produced and its principles can be used to do certain useful tasks. Using this method you write programs that work with symbols rather than numbers. Symbols can be equal or not equal and that is the only relation defined between symbols, so you can't even ask if one is less than another much less do arithmetic with them. Of course in symbol processing programs the symbols do get represented by integers. These researchers have never been interested in how to implement symbol processing using neural networks. Besides the use of symbols the other key principle involved is that knowledge in the mind consists of a large number of rules. Symbol processing adherents make the bold claim that symbol processing has the necessary and sufficient properties to display intelligence and to account for human thought processes.

Even though they say that symbols can only be equal or not equal and there are no other relations defined for them quite often "symbolic" programs end up using integers or reals as part of the program and its called symbolic anyway even though by the strictest standard doing so no longer makes the program wholly symbolic, only mostly symbolic.

Relatively little has been released on the CYC project. One FAQ on CYChas been produced by David Whitten. The official CYC WWW site is Cycorp, Inc. My opinion is that CYC may be quite useful for certain applications but it still won't be operating like a human being or anywhere near as well as a human being so in those most important senses it will be a failure.

Another of the principles used in AI is heuristic search. In heuristic search (as opposed to blind search) you use some information about the problem to try and solve it quickly.

If you ask me and many others using just symbol processing and heuristic search has not been working out very well and new ideas have to be investigated. Its pretty straightforward to list most of them, you only have to take the assumptions of classical symbol processing AI and list the alternatives. First, symbols are not enough, real numbers are necessary. Second, symbols and structures of symbols are not enough, pictures are necessary (and of course representing them requires using numbers). Third, rules are not enough, actual cases are necessary. Another alternative is that quantum mechanics will be necessary.

MacLennan sees the use of real numbers and images (pictures) as the main characteristics of connectionist AI. Of course the symbol processing AI camp doesn't like connectionist AI and one of their big criticisms is that in connectionist AI there is no way to structure data with vectors of real numbers and so it can't possibly work. This is a good point, there is no ESTABLISHED GOOD WAY of structuring data in vectors but the research is moving along and some of this will come up in Chapter 6. MacLennan deals with this criticism from the symbol processing camp in a couple of very worthwhile papers that are fairly easy to read as such things go and I would love to see some commentary on them from people in the symbol processing camp. (I'll provide disk space here or link to a document on your system.) The two most worthwhile articles to check are:

"Continuous Symbol Systems: The Logic of Connectionism", from the Ohio State neuroprose archive and

"Characteristics of Connectionist Knowledge Representation", from the Ohio State neuroprose archive or from netlib.org wherever that is, Tennessee or somewhere thereabouts.

You may also want to browse his article on:

"Field Computation in the Brain", from the Ohio State neurprose archive.

The article by Vitiello is available online from Los Alamos National Laboratories. The article by Brian Josephson is also online from Cambridge University, United Kingdom. I cannot recommend the online article by Nanopoulos available from Los Alamos National Laboratory highly enough. Again, for more on the possible QM basis for human thought see my The Quantum Basis of Natural Intelligence? WWW page.

In this book I've alienated the classical symbol processing camp who believe that symbol processing is sufficient. Now let me make the even dumber mistake of alienating the other main camp, the connectionists. In my opinion for the most part so far connectionist AI is making the same glaring mistakes that symbol processing AI made. These mistakes are that picture processing is largely being ignored and case based/memory based methods are also largely being ignored. Of course there is no reason why connectionist AI can't adapt as time goes on.

This short section gives a brief outline of the rest of the book, the book starts at a low level with vision and then moves on to higher levels and ends with natural language.

A key concept in the book besides emphasizing new methods is the idea that pattern recognition forms the basis for AI programs. Pattern recognition can be done using various techniques, at one end of the spectrum you can use cases (memories), at the other end of the spectrum you can "compress" or "condense" the cases down to rules as symbol processing adherents want to do and now we have a method, neural networking that falls in between these two options. A lot of what goes on will be pattern classification, that is putting a particular instance of a pattern in a category but there is also pattern matching going on where the goal is to determine how similar a pair of patterns are.

Just one note here on the exercises. The "identify the newsgroup" or "choose the arithmetic operator" exercises might be handled quite easily with little AWK programs and some UNIX shell programming.

I thought my listing of characteristics of intelligence in the first section of this chapter was pretty reasonable, they were: some pattern recognition ability, some trial and error behavior and the ability to learn. I don't want to paint myself in a corner but I don't require thinking and consciousness for intelligence. If, as research goes on I have to change I consider myself free to do so. But now let me expand on the characteristics of intelligence and how it relates to the subject of Artificial Intelligence.

There were two aspects to AI stated at the very beginning of the book, first the larger goal is to try and find out how human beings do what they do in order to get programs that will do the same. This goal means trying to produce a whole system that behaves very much like a human being in that it has the same level of understanding of the world and the ability to improve its understanding of the world as people do. Of course a major way of finding this out in the AI perspective is to do experiments (write programs) that will help uncover how people work. To a certain extent you could consider AI a computer architecture problem, its goal is to find the human architecture or maybe even something better than the human one.

Actually meeting the above goal has proven to be difficult. There is no immediate prospect that it will be realized any time soon. Or to put it another way: everything tried so far has failed. Thus most of what has been going on in AI pertains to the secondary goal: trying to find ways to get programs to do very specialized tasks that people do and perhaps to do them better and cheaper than people. Here's where the problem comes up: most of these programs to do specialized tasks don't meet the very reasonable criteria for intelligence I listed and none of them is capable of dealing with the real world in the sort of way that human beings do. The point at which they typically fail is that they don't learn anything and they can't really learn anything because they don't really understand the subject matter. MOST PROGRAMS PRESENTED AS "AI" PROGRAMS DON'T HAVE ANY INTELLIGENCE AS THE WORD INTELLIGENCE IS DEFINED BY NORMAL PEOPLE (although they usually show SOME characteristics of intelligence). To get a keen appreciation of this you must go on through the book so it is difficult to give examples now, however consider the following. Some AI researchers contend that if a program simply does a task which if it were done by human beings would require intelligence then the program is showing intelligence. For example doing arithmetic by hand is one such task. When people do it, it is considered to be intelligent work. Of course it is awfully easy to program this task and to some AI researchers the program would be intelligent. The trouble is that it is awfully easy to let a bug into the "intelligent" program. Consider for instance the pentium division bug. This chip may be considered "intelligent" until you find the bug. But the chip can't fix this bug because it actually has no idea what it is doing. The "intelligent" chip is revealed to be very stupid, an electronic version of a dumb mechanical contraption. In my opinion it is not mechanically doing something that shows intelligence, it is noticing mistakes and understanding enough about the problem to be able to correct them that shows the intelligence. It is pushing on the boundary between the well known and the unknown that shows intelligence. Only a very few programs have done this and then only on very small tasks like game playing, not in dealing with the world in general.

A very serious consequence of the you might say "mechanical" definition of intelligence is that the "intelligent" programs, like the pentium chip, inevitable show their stupidity and then the general public can't take seriously the claims of AI researchers that they've created machine intelligence. It only produces one embarrassment after another like the Turing test where the goal is to fool the public. It would almost be better to rename AI as "advanced programming techniques".

Unfortunately then, much of what you find in "AI" isn't AI but a catalog of ideas that have mostly "failed". But on the bright side the "failures" can be extremely useful in certain applications. And there still is quite a legitimate GOAL of finding out how human intelligence operates so it can be duplicated in a program. Because so much "AI" does not show true intelligence and because I want to try and divert AI toward its original goal many wonderfully useful techniques will not be covered in this book. Thus many people especially from the classical symbol processing camp will consider the presentation here to be "unbalanced" and "biased". Sorry, I want to pursue the problems involved in the original goal of AI as much as possible.

I was very recently cheered to see that a well-known veteran AI researcher has expressed similar (but not identical) sentiments. In his article, "Eye on the Prize", (available in postscript from Stanford University) Nils Nilsson in the Summer 1995 edition of AI Magazine, pages 9-17 he wrote in the introduction:

In its early stages, the field of AI had as its main goal the invention of computer programs having the general problem-solving abilities of humans. Along the way, a major shift of emphasis developed from general purpose programs toward performance programs, ones whose competence was highly specialized and limited to particular areas of expertise. In this article, I claim that AI is now at the beginning of another transition, one that will reinvigorate efforts to build programs of general, humanlike competence. These programs will use specialized performance programs as tools, much like humans do.

High performance programs such as these are all very useful; they are important and worthy projects for AI, and undoubtedly, they have been excellent investments. Do they move AI closer to its original, main goal of developing a general, intelligent system? I think not.