The word “existence” can’t be learned by a classifier

The limits of learning by empirical association

This is the nineteenth post in a series on AGI. You can read the previous post here. You can also see a list of all posts here.

Learning by association is perhaps the most straightforward way to teach an AI practical language¹. In its simplest implementation — such as in a classifier — you can show an algorithm images of, say, a tiger, followed by the word “tiger”. With enough repetition the word becomes tightly correlated with the appropriate sensory inputs. This model for learning language has its roots in 17th century empiricism, and was introduced into cognitive psychology through theories like operant conditioning and Hebbian association. Today it is fair to say that conditional prediction is, in one way or another, used in nearly every ML algorithm that exists².

one thing seems clear to me about mental activity - that the purpose of much of it can be considered to be the making of predictions.— Richard Sutton, Mind Is About Conditional Predictions

Naive Association

Despite the popularity of this approach, most words in the dictionary are impossible to learn by correlating their appearance with a set of sensory events, or even with patterns of events. These include “existence”, “here”, “condition”, “self”, “reality”, “now”, “type”, “concept”, “mind”, and so on. The conditions for these words could be said to be present at all times and in all places. You are always experiencing “time”, “here”, and “existence”. This is in contrast to, say, a tiger, which is sometimes present, and at other times not, so the word “tiger” could feasibly be learned by associating it with the experience of one appearing. But with what experience, and at what time, would you associate the word “me”? You are always there, so it is impossible to learn “me” by correlating the word with the frequency of some event.

This presents a challenge to strictly statistical approaches to AGI. If the ability to learn a word depends on associating it with a strongly correlated experience, there appears to be no avenue to learn the words above. This is no small omission. The majority of words used in this post have no bounded or specific sensory inputs with which they can be correlated — e.g. “sense”, “the”, “this”, “for”, “or”, “e.g.” ‘it”, “with”, “have”, “identify”, “correlate”, “time” and so on. Many of these words could be more accurately viewed as subjective interpretations, and can be applied to any experience: you can always find something around you that you could say is “with” something else, even if it’s just you “with” the world. The word seems to depend more on your focus and mental state than on experiences themselves.

Despite all this, humans do learn these words, and they learn them through experience, which implies that there is another mechanism by which an AGI can acquire them. Let’s consider some options.

Innateness Theories

One possible explanation is that every mind comes pre-installed with a handful of innate, built-in ideas like self or time, and the word can be attached to these somehow. The nature of innate ideas has historically been hard to fully describe. But regardless of what they are, in order to be associated with words they must ultimately produce some sort of inner signal, like a new channel of information, one that is triggered in response to a pattern of events in the agent’s mind. This raises the obvious question: based on what criteria or mental pattern should each be triggered? What events should cause the brain turn the signal for “existence” on or off? Questions like these bring us right back to the original problem.

One could argue that perhaps innate signals don’t turn on and off, and the agent simply directs its attention to, say, the “me” signal when it pays attention to itself —similar to turning your eyes towards an object. There must, in such a case, be some reason it focused on “me” instead of something else like “the world”. That reason assumes the agent knows what it is looking for, or what it should pay attention to due to the situation it is in (e.g. “they are looking at me”). This makes the need for the signal superfluous — the situation that caused the change of focus could suffice as the signal.

The same could be said of Kant’s transcendental concepts, (and more recently Fodor’s Language of Thought). In his Critique, Kant made a compelling argument that concepts such as time and unity must be built-in functions of a mind, since they are necessary for shaping how the mind understands the world. Whether or not he was correct in that respect, difficulties still arise when it comes to associating these mental functions with words, as opposed to simply using them to apprehend reality. There is a difference between structuring one’s experiences through the form of space, and being able to name the concept of “space”; in the same way that you or I can use our brains to understand the world without being aware that neurons or even brains exist.

Repeated experiences of objects in space and time can’t lead one to associate those experiences with the words “space” and “time”, since the latter requires an associative mechanism that works outside of space and time — which is by definition impossible. The agent would have to step outside the machinery and look at it self-reflexively. It would no longer be associating experiences, it would be associating information about its modes of experience, which requires a new and different channel for learning.

Ultimately, all of the above solutions involving innate ideas end up begging the question. We are still left with the original problem: what is it that causes us to think of such words as “existence”? Without an answer, AGI research is in a tough bind. You and I know that such words can be learned, and even that they can be associated with sets of experiences. Yet we don’t have a practical method to recreate this critical capability. In a previous post in this series I argued that both innate and reductionist arguments fail when trying to acquire the simple concept of “more”. This is now the situation we find ourselves in with respect to a large chunk of the dictionary.

Useful definitions

Despite this, words are still social artifacts, so any agent (or human) must ultimately learn them through a series of social experiences, likely by hearing others say them at specific moments. It is a fact that there was once a time when you didn’t know the word “existence”. That was followed by one or more specific events during which you learned it. So if they’re not learned by correlating them with their referents, what options are left for teaching an AGI words like “existence”?

Maybe we should take a step back and reconsider what the mind actually learns during such moments. This time rather than looking for objective criteria for the word, let’s consider its subjective utility with respect to an agent’s needs. A word is ultimately a social tool for changing an agent’s environment. The purpose of teaching an agent to speak isn’t simply knowledge for its own sake, or as part of a passive model of the universe, but because it is useful —e.g. for getting something it wants. It is easy to imagine how an agent could learn to say “cookie!” if it really wanted to eat one; as easily as it could learn to turn a tap when it wants water. Uttering a word could be a simple conditioned action, a means to an end.

Those ends could vary widely, from wanting to impress a teacher during a vocabulary test, to trying to direct someone’s attention to an object. Besides naming it, there are other useful actions the agent can perform on seeing a cookie: eating it, reaching for it, inspecting its toppings, passing it to someone. Each may be appropriate depending on the problem it’s currently trying to solve.

This approach only applies to actions however; different problems arise when it comes to thinking of words, especially if one assumes that learning a word is an act of classification intended to predict the probability of a word given a set of inputs. As we saw, such an approach fails for words like “existence” which has no distinct priors. Yet if we ignore prediction for a moment and consider only usefulness then thinking of the word “cookie” ceases to be a fact and instead becomes a plan for what an agent should say given the problem context. The act of identification, i.e. predicting the object’s name, is therefore one of many plans for what to do when faced with the sight of a cookie.

An agent may even learn to think of the word “cookie” when it doesn’t see one, because it wants one to appear. Here we’ve reversed the normal direction of causation. Instead of the appearance of a cookie predicting the word “cookie”, now the word “cookie” — as a request for something useful — predicts the appearance of a cookie. Had it not learned the word or how to articulate it, it would be frustrated in its intentions. The word solves a problem.

Let’s apply this same approach for the word “me”. Since, as we said, the agent itself is always present, there are an infinite number of events that could precede the word “me”, with none exclusively predicting its appearance. However, there are a much smaller number of events where the word “me” would predict, or rather cause, something useful to the agent. For example:

Question: “Who should I give the cookie to?”

Agent: “Me”

The word “me”, like the word “cookie”, becomes the agent’s plan for what it should say to get something it wants. And the system reinforces this action if the word is effective. This inverts the normal language acquisition processes — learning how to use a word, or where it is best used, now precedes learning what it means. Does that sound familiar? It should — this is also how humans generally pick up new words during conversation.

The upshot of this is that a previously impossible problem now becomes tractable. We have a method for teaching the agent abstract terms that doesn’t require them to be based on equally abstract criteria, and so on, so that it’s “abstractions all the way down”. It also means the agent can apply the word “me” in unconventional ways, for example, to an avatar while playing a computer game. By equating the interactions of the avatar with its own goals, successes, and failures, it would label the avatar as “me” — e.g. “the knight give me the gold amulet” or “that dragon hit me”.

Strict factuality is no longer a requirement. This flexibility is useful for resolving ambiguous cases where the label depends on context; such as when the agent sees its reflection in a mirror. Should the agent call the entity in the mirror “me”? Only insofar as it is useful, e.g. when realizing that it (“me”) looks unkempt. Yet if someone throws a ball at the reflection, the agent would cease to call it “me”, since that word isn’t useful in such a scenario — the agent itself was not hurt by the impact.

Causes of a thought

There is still something missing from this approach to language acquisition. It would be silly to suggest that every time an AGI wanted to speak, it had to randomly guess at sounds until it found an effective word for what it wants to express. Indeed the term “express” suggests there is something internal and abstract that is being converted into words. This could be a concept, or perhaps a thought.

Intuitively, we know that words are not merely useful utterances. They are caused by entities in the mind (they are intentional). By this hypothesis, then, the AGI must first think of or imagine an underlying idea, and subsequently convert it into speech. And such an idea, we assume, must have been elicited by a learned set of features that are similar across many experiences. Seeing a cookie should lead the agent to think of that word. There must therefore be some idea or thought inside the agent that caused it to think of “me”.

And indeed there is. It is not just the end goal that drives the word, but the goal combined with the specific stimuli in the environment. There are two sides to an action or thought— the sensory inputs (including other thoughts), and the problem it is trying to solve within those circumstances. The same visual inputs can cause the agent to have different thoughts or actions depending on what it is trying to achieve. The only difference between concrete words like “cookie” and non-concrete ones like “now” is that the sensory inputs for “cookie” are generally easy to point to. Although a cookie can be variously coloured and decorated, there is some similarity of appearance across (many of) them which allows the word “cookie” to transfer easily between instances.

In cases where the inputs are too dissimilar, the word can transfer via an indirect connection, as long as the underlying problem is solved. For example, the agent is hungry, it sees an opaque jar, then cookies come out of it. It calls for a “cookie”, and its problem is solved. The jar itself now triggers the word “cookie” without the intermediary. Problem-solving acts as a glue that unites instances which otherwise have nothing in common.

In the case of the word “me”, the inputs that would cause the agent to think of the word are more diverse than those for concrete words. The term can be attached to a wide range of stimuli based on where it has been useful — including reflections in a mirror and social interactions. Thus transferring the word between instances must happen indirectly, using the same problem-solving time-hop as above³. For example:

Q: Who wants to play football?

AI: (thinks: football implies fun, and reinterprets question as “who wants fun?”)

AI: (thinks: Who wants to have fun?)

AI: “Me”

Generalizing across instances becomes significantly easier when you take into account the motive that is driving it. The problem context, i.e. the context of interpretation, is integral to picking the correct word. If someone were to point to a plastic toy cookie and ask “is that a cookie?” your first impulse would be to ask — “in what sense”? If they are hungry and plan on eating it, then no; if they are trying to teach someone what cookies look like, then yes. By including motivations (e.g. hunger) as the connective tissue, the distinction between these cases becomes much easier to define.

A word should not merely be pushed into an agent’s mind simply because it has been previously associated with certain inputs. Rather the word must connect the inputs with an intended goal. This implies that every time the agent wants to use a word in an unfamiliar situation, it must solve a problem. Learning must be an ongoing process, not easily divided into training and test phases. There are many complex aspects involved in the act of moment-to-moment problem-solving which are addressed in other posts in this series. Generally speaking, any word that has been useful in previous, similar circumstances takes a prominent position in the agent’s mind, and if effective again, it can be transferred to this new situation.

If an AGI is ever to learn practical language, it must be as a tool for social effectiveness. This requires us to move away from words as objective labels or associative predictions, towards a more utility-based notion of thinking. The only way to clarify the meaning of words — how they are used, what causes them to appear a mind — is through the contextual motives that underlie them.

In the next post we’ll dig further into identification as an act of fluid problem-solving.

Next post: Autonomous identification

¹ Practical, in the sense that it could use it to discuss its own experiences and interact with the world around it.

² Exceptions may include genetic algorithms and some forms of reinforcement learning.

³ In the first case, that of the cookie, the connection was made through real world experiences. In the latter it was made through connecting thoughts. Both are examples of logical reasoning in action. The example in the mind merely imitates the one in the real world.