In an earlier article, “AI: What’s Missing“, I listed several capabilities that are missing from current AI systems – capabilities that must somehow be provided before our systems will be able to exhibit anything approaching general, human-like intelligence. Perhaps the most important of these, because it interacts with several others, is “The ability to assimilate and store large amounts of symbolic knowledge and to make that knowledge effective.” In this article, we will explore what it means when we talk about “human-like” intelligence and “human-like” memory.
A key point: when I speak of human-like intelligence, I am talking about human-like capabilities, not necessarily human-like implementation. I want to focus on the breadth, flexibility, and the amazing search and inference capabilities of human memory, and not necessarily on how this is implemented in the marvelous pile of neuro-stuff that is the human brain. If we could understand the general principles and mechanisms of human memory, at some level of abstraction, that would guide us both in trying to understand what the structures of the brain are doing, and also in trying to implement similar capabilities using the very different electronic hardware that is available today – or it would tell us what kind of hardware we really need in order to do this job.
So I suggest that we – some of us, anyway – approach this problem not as neuroscientists or as cognitive psychologists, but as computer scientists: Here is what the system must do; we know that this is possible (humans can do it), so what algorithms, data structures, and organizing principles are required to do the job? Oh, and as computer scientists, we must understand this well enough to implement it – no hand-waving or vague metaphors about shadows on the wall of a cave are allowed in our final theory, though we might make good use of metaphorical reasoning along the way. That is the challenge.
Note that it’s possible – some would say probable, in this case – that we will come up with models that cannot be implemented, in any practical sense, on today’s computing hardware. That’s OK. But if we come up with the answer that the problem is computationally impossible or intractable in some fundamental way, we need to look again at how we are framing the problem, because at least one solution exists.
(Well, the brain could be using magic or some as-yet-undiscovered kind of physics, but we should only accept that conclusion if we’re absolutely certain, through proof or exhaustive search, that no solution exists in computation as we know it. And, today, we are very far from being able to make that claim, despite the ravings of some dilettante physicists and mathematicians who have wandered into the field.)
When I say that we are focused on the capabilities of human memory and not on implementation, I don’t mean to suggest that we should ignore the findings if neuroscientists and cognitive psychologists, just that we should not be constrained by them. There may be many possible ways to build a human-like memory, but at present we have only one example, so we should learn what we can from it. A few examples:
· The brain apparently does what it does using millisecond-speed components (neuron firings and protein bindings), but with a huge degree of parallelism. So that’s a clue.
· While access to the human memory is fast, creating new memories is much slower, at least in the long-term, symbolic part of memory – maybe one or a few new items can be permanently saved per second.
· In humans (especially in aging computer science professors) some memories just vanish, but for many more we seem to lose some of the access threads, or these become inactive until something “reminds” us of the item we are looking for; then we can easily repair the thread. I might be able to picture a certain actor who has appeared in recent pirate movies and remember a dozen other movies he has been in, but be unable to access his name until someone says, “Rhymes with ‘strep’.” (We might prefer to construct an artificial intelligence without this partial-forgetting phenomenon, making it super-human in that respect. That might work, or we might find that this decay of access paths is not a hardware limitation, but that it is somehow essential to the operation of a human-like memory.)
So what are the important aspects of human-like symbolic memory that I am so eager to capture? Let me try to illustrate the problem here with an example or two, and in a near-future article I will try to enumerate some particular abilities and features. Here is a description I’ve been using for a long time, in various forms. It first appeared in my 1977 Ph.D. thesis at the MIT AI Lab:
Suppose I tell you that a certain animal – let’s call him Clyde – is an elephant. You accept this simple assertion and file it away with no apparent display of mental effort. And yet, as a result of this transaction, you suddenly appear to know a great deal about Clyde. You can tell me, with a fair degree of certainty, how many legs he has, what color he is, and whether he would be a good pet in a small third-floor apartment. You know not only that he has eyes, but what they are used for, and what it implies if they are closed. If I try to tell you that Clyde builds his nest in a tree or that he amuses himself by hiding in a teacup, you will immediately begin to doubt my credibility. And you can do this very quickly and easily, with none of the sort of apparent mental effort that would accompany, say, adding two four-digit numbers. This effortlessness may be an illusion, but it is a compelling one.
“Elephant”, of course, is not the only concept that behaves in this way. The average person knows a huge number of concepts of comparable or greater complexity – the number is probably in the millions. Consider for a moment the layers of structure and meaning that are attached to concepts like lawsuit, birthday party, fire, mother, walrus, cabbage, or king. These are words we use casually in our daily lives, and yet each of them represents a very substantial package of information. In technical fields (except, perhaps, for the more austere parts of mathematics, the situation is the same. Consider how much you would have to tell someone in order to fully convey the meaning of concepts like meson, oscillator, hash-table, valence, ribosome, or leukemia. And yet, once these concepts are built up, they can be tossed around with abandon and can be used as the building blocks for concepts of even grater complexity.
The point is not just that we can handle large chunks of knowledge as though they were atoms; the important thing is that we can find our way through these complex, nested structures to whatever fact or relationship we might need at any given time, that we can do this in a very flexible way, and that we can somehow avoid having to look individually at each of the vast number of facts that could be – but are not – relevant to the problem at hand. If I tell you that a house burned down, and that the fire started at a child’s birthday party, you will think immediately of the candles on the cake and perhaps of paper decorations. You will not, in all probability, find yourself thinking about playing pin-the-tail-on-the-donkey, or about the color of the cake’s icing or about the fact that birthdays come once a year. These concepts are there when you need them, but they do not seem to slow down the search for a link between fires and birthday parties. If, hidden away somewhere, there is a sequential search for this connection, that search is remarkably quick and efficient, and it does not become noticeably slower as the knowledge base expands to its adult proportions.
So that’s the challenge, as I see it: To build a memory system or knowledge base that can support operations of this kind, on a very large scale, and that gives us access not only to the knowledge that is explicitly present, but the much larger body of knowledge that is implicitly present. This active knowledge base is not, by itself, an intelligent system, but it is the key missing piece – the piece that ties together our sensory systems, higher-level problem-solving modules, natural language processing, and all of our experiences: past, present, and imagined-future.
In future articles we will try to break down this problem and will begin to explore how we might implement a system of this kind.---------------------------
- I am not suggesting that symbolic memory – the things we humans can easily describe in natural language – is all that there is. I’ll have more to say about visual, sound, and motor memory in future articles. But, for now, I think that it’s best to focus on the most accessible and familiar part of our memory. [↩]
- Reprinted in book form by MIT Press: NETL: A System for Representing and Using Real-World Knowledge, Scott E. Fahlman, 1979. In addition, there’s a scanned version of the tech-report form of the thesis online at ftp://publications.ai.mit.edu/ai-publications/pdf/AITR-450.pdf. [↩]