A Layer of Self-Generated Experiences
The invisible machinery of thinking
This is the sixth post in a series on AGI. Read the previous post here. See a list of all posts here.
Summary of posts 1 to 5: To tackle the mountain of challenges related to developing Artificial General Intelligence (AGI), begin by describing a set of mental functions that are invariably present in every mind. These are invisible, atomic, and automatic mechanisms which drive the rest of cognition. One such function is the act of “becoming aware of something”, which changes the mind to reflect some content (i.e. learning). This content may be in the outside world, or you may “becoming aware of” your own thoughts.
The terms “thinking” and “thought” have been used many times throughout this series, as a glue that holds cognition together. Until now, it was enough for the discussion to rely on a vague intuition of what those words mean. At this point, however, we should dig into this critical unit of mental life; what it is, and how it can be mechanized. This post is a central one in the series, and is a necessary foundation for later analyses of the more complex aspects of contemplation and intelligence.
Let’s start at the easier end of the spectrum, by discussing thoughts which directly mimic some previously experienced content. We usually refer to these as “memories”, but for reasons that will become clear, that term is misleading.
When you see or hear something around you, and the conditions of attention are just right, you may find that you have made a rough copy of that experience in your brain. Later, you may recall it — you may recreate the experience, almost as if it were immediately present. We already showed that recalling a memory alters the mind at the moment of recall. There are still many open questions about what the purpose of a memory is and how it comes to be, which together should point to the mechanism by which it was created. For now let’s just consider the immediate phenomenon of “recollection” itself, and try to find ways of framing it.
You may have noticed from your own life that you can only recall sights and sounds; not smells, touch, or tastes, or any other senses like proprioception¹. And it’s not just memories that have this limitation. This is true of all thoughts, including those we refer to as “knowledge”; e.g. what a duck looks like, or what an arpeggio in G major sounds like. This constraint hints at something peculiar and tailored-made about memories. They are not just generic, abstract attempts to use prior experience to plan ahead. They use a concrete sensory modality, and have specific, determinate content.
What is the mind doing when it remembers, and why is it limited to sight and sound? To answer this, try the following experiment. Try to remember a smell like lavender or ham. There’s a good chance your attention has just turned to smelling your immediate surroundings. Perhaps you even inhaled a little through your nose. You may be trying to reshape the present sensations to somehow match what you think they should be. Since the world and your mind aren’t accommodating your request, you can’t do it. It’s more likely that an image of ham or lavender popped into in your head.
Why did you try to smell your surroundings at all? The task was to “remember” the smell. Why did you switch from remembering to sensing? It seems as if you were trying to get that sensation to show up in your mind in its raw state. This would allow you to consider it, interact with it, and in so doing, you could include some aspect of the experience in further deliberations and actions — e.g. you could tell a friend something about it.
Now apply this same experiment to memories of visual experiences. Try to remember what an apple looks like, or how it differs in appearance from an orange. Following the pattern above, your attempt to recollect the apple or orange can be seen as an attempt to recreate and re-see those images in the current moment. In this case, your mind accommodates your request. You are, in essence, seeing your memory in front of you again; though perhaps more mildly than if it were directly present. This way, your mind can act as if you had seen it again in the moment, and perhaps do something with that information, like tell someone what colour each fruit was, or that they are similar in shape.
There are therefore two features of memory recollection: first, it is a momentary action (rather than an ongoing state of mind), and second, it presents sights and sounds to the mind for re-consideration. Together they allow your mind to re-inspect the memory like you would inspect a stimulus in front of your eyes or ears. Indeed, you may discover something new about it. For example, as a child I was made to memorize the French Canadian national anthem phonetically. At the time, I didn’t understand what the sounds meant. Only now, thinking about them as an adult, am I able to stitch together what the lyrics were from the raw sound-thoughts; years after the fact. This is something I would normally only be able to do if someone were saying the words to me.
Memories have another rarely recognized property, namely that you can see and remember your own thoughts the same way you can remember your external experiences. This is an obvious fact, yet is hardly ever noticed in practice. For example, close your eyes and try to recall the previous sentence. There’s a high chance you didn’t recall the letters on the screen, but rather the voice with which you were speaking those words in your mind, including its tonal fluctuations. During recall, this voice echoed through your head once again. You could choose to frame this more generically as remembering your “thoughts”. But if we’re being specific, it was a set of sounds. Calling them by the generic term “thoughts” is a subsequent interpretation of the events. The most straightforward description of what happened is that you heard and remembered your thoughts, in the same way you would have heard and remembered a person speaking to you.
Another curious property of memories is that you can remember two concrete thoughts (i.e. sights or sounds) in a row as one, thus composing a new “memory”. Think of a beaver, then a racecar. The next time you trigger that thought, both appear together. And there is no end to the number of times you can stitch thoughts together. Using this model of memory we can explain both memories and composition as one fundamental process — at least when it comes to concrete images and sounds.
The distinction between thoughts and memories should, by now, be starting to blur. Is there any difference between them? In the example of the beaver-racecar, the resulting memory is generally considered a composition of thoughts, not a memory per se. But consider its properties: the event happened in a specific moment in time, it contained sense content (images), what you remembered is a direct reflection of what was experienced, and it changed your mind so that going forward you can now elicit that image once again. The only difference between the beaver-racecar example and a “true” memory is that the latter comes from the senses, and the former comes from the mind itself. The fundamental function, however, is the same.
It might be useful to merge these two processes into one general concept — that of concrete thinking². The definition of the word “memory” implies that it must refer to some true, objective event that happened outside your mind. By relaxing this definition a little, we can group memories of your own thoughts together with those of the outside world.
However, there is a problem with this model: this description of thinking, at least of concrete thinking, could also describe hallucinations. The only difference it seems, between concrete thoughts and hallucinations, is that with hallucinations the subject can’t differentiate what is real and what is imagined. This itself raises another question: how can you distinguish what is real from what is created by your mind? If memories are what we say they are — self-generated sensory experiences — then every human should be running around in a state of dangerous delusion.
The answer to this riddle is, you sort of are. Without self-imposed checks and some self-awareness, you usually don’t distinguish between your own thoughts and your sensory experiences³. Recall that dreams are a type of hallucination, yet dreamers tend to act as though everything in front of them, no matter how absurd, is real. So we should not place too much confidence in some automatic “hallucination detection skill”. Even in waking life distinguishing thoughts from reality is not a given. Children, untrained in this distinction, regularly intertwine fantasy and reality, and only self-censor once they are reproached for lying. It takes effort and training for people to separate their actual experiences from their interpretations of those experiences, or their imaginings about them.
Have you ever been in an argument with someone who claimed you said something earlier, when in fact they only thought you did? False memories like the Mandela Effect are common; likely more common then we have records of, since they are only recognized as false when they cause problems. Another reason people confuse thoughts and external experiences is that the two can be remembered together as one memory. For example, if you see a person speak about their awesome car, and think about the car while they speak, your memory will end up including both the person speaking and the thought of the car. (This, incidentally, explains the phenomenon described in the previous post; namely, that we don’t remember things exactly as they are — we add embellishments and interpretations.)
Perhaps you might wonder what the two “layers” — experience and imagining — look like when separated. One way to do this is to look once more at dreams. You may have noticed that dreams have a fluid, ephemeral, and personally symbolic quality to them. People who have different character traits and desires have different types of dreams. If the above model of imagining sights and sounds is correct, then dreams may represent that second layer of self-generated experiences, only now they are isolated from the underlying flow of real experiences. Consider that when dreaming, if you try to “think” during the dream, you only end up producing more dreams, i.e. furthering the dream narrative. Dreams, in this sense are only thoughts. Without any external stimuli to anchor them, all you are left with is a mass of subjective interpretations and imaginings. Or, flipping this argument around, one could say that your waking experiences are layered with a fog of dreams.
Over time you can eventually learn how to suss out which part of any given memory was the external experience, and which was imagined. For example, you may reason that “I can’t have seen his face, I was standing behind him, so I must have only imagined he looked angry”. Without this ancillary skill, humans default to treating their thoughts as though they were reality, or potential reality, until they are given a reason — like facts or consistency — to question this assumption.
And this isn’t a bug, it’s a feature. Ironically, such self-generated illusions are a necessary ingredient in the your capacity to know truth. Remember, just because it’s “imagined”, doesn’t mean it’s not true⁴. Predictions are also imagined, yet they may be real and useful. Your entire perception of what reality is outside your immediate sensory input is an intricate tapestry of imagined sights and sounds. What does the front of your house look like? The thought you’re now experiencing is an image that has been arbitrarily inserted by your mind, which you’re inspecting as if it were in front of you. Yet it is still likely real (in a sense).
If you imagine that a closed box contains dangerous spiders, you will act towards that otherwise harmless box as though it were a dangerous spider. You could call this a “useful delusion”. The only thing that separates real thoughts from delusions is their utility to you. The fundamental act of imagining is the same; whether you’re imagining the world as composed of atoms, as composed of four elements, as composed of angels, or as composed of magic and fairies.
This post has so far described the patterns involved in concrete thinking. It still hasn’t addressed abstract thoughts like concepts (the first challenge in the previous post). The explanation for those will be detailed in a later entry; for now consider what was discussed in The Smallest Units of Introspection, namely that introspection can be misleading. What appears to be an abstract thought may only be an introspective re-interpretation of a concrete thought as an abstract one.
What this post does succeed in doing is keeping to the thesis of the series; that is, describing only the fundamental, automatic, invisible functions of cognition, the ones which, when understood, could be used to recreate a mind. As we build on these and explore how people learn and connect ideas together, more complex patterns will evolve to deal with abstractions, conceptualizations, even morality, identity, and will.
[Next post: Drivers and motivations, a primer]
¹ Proprioception: the ability to sense where your body parts are without looking.
² This is easier to grasp once you realize that concrete knowledge is just a memory without a specific time attached. If I see my brother eat Mini-Wheats this morning, that becomes a memory — “he ate Mini-Wheats”. If I see him eat it twice or three times, it becomes knowledge — “he eats Mini-Wheats”. Or if I see an image of a bear for the first time in a book, that becomes a memory — “I saw an image of a brown creature in this book”. When I see more such images that becomes knowledge — “this is what that brown creature generally looks like”.
³ There is another distinction between thoughts and hallucination which keeps them “safe” which will be addressed once we’ve looked at how they are learned.
⁴ This has been an ongoing thread it both Western philosophy for thousands of years. Philosophers like Plato and Kant have argued that the imagined truth is the only real one; it is the senses rather that present us with illusions (e.g. Plato’s cave).
