The blinders of curiosity, and the predictability of novelty

How we misconstrue curiosity in cognitive science research

Curiosity seems to be a ubiquitous human drive. No member of our species is so cognitively bereft as to completely lack this characteristic trait. The desire to explore the world and plumb its unknown depths is arguably the foundation for human creative ingenuity. So it is tempting to assume that the human mind has a built-in urge to explore, an innate need to learn about the world for learning’s sake.

What drives this impulse, and how it manifests day-to-day, is not entirely clear. By far the most popular current theory across both Cognitive Science and AI research is rooted in rational choice theory. It is also the most intuitively appealing, and goes as follows:

1. Understanding the objective world is useful for making accurate predictions.
2. Predictability is sought out because it helps you plan future actions.
3. Where your predictions fail — when unforeseen experiences defy your expectations — curiosity takes over in order to fill in the gaps in your knowledge.
4. The result is an ever-growing knowledge-base of empirical beliefs that serves increasingly complicated goals.

The above model of curiosity is so widespread that providing supporting quotes is almost superfluous; it is the de facto standard theory in research, and has been since the time of Piaget. It forms the basis for the Free Energy Principle, Predictive Coding, and Model-Based Reinforcement Learning. It presents a mirror image of scientific discovery. Here it is being injected into our core mental functions, as a fundamental, intrinsic drive:

Improving the model components of a model-based agent so that they conform better with reality is almost always a good idea, regardless of the external performance standard. — AI: A Modern Approach, 4th Ed

Through exploration, the explorer acquires not just information about a specific task or environment in isolation, but information in relation to previous experiences, informing the development of more generally-useful behaviors — General intelligence requires rethinking exploration

The novelty of an image reflects how much it differs from what we expect to see based on what we have seen in the past. — Visual novelty, curiosity, and intrinsic reward in machine learning and the brain

Curiosity is the desire to create or discover more non-random, non-arbitrary, regular data that is novel and surprising […] in the sense that it allows for compression progress because its regularity was not yet known. — Driven by Compression Progress, Jurgen Schmidhuber

Indeed you’d be hard-pressed to find any research that holds an alternative view.

Experiments that investigate our tendency to engage with novelty always assume, of course, that the authors know what counts as novelty, e.g. a novel toy, a novel piece of art. In practical implementations, such as in novelty-driven Reinforcement Learning, there is always an algorithm that calculates the “novelty factor” of actions or experiential states, and uses it to determine what the agent should explore next.

Curiosity has structure

The standard approach, though superficially appealing, is also highly improbable, and runs into major obstacles as a result. Firstly, by its own assumptions, pure novelty — i.e. complete unpredictability — should be the most compelling and interesting of all stimuli; and yet experience shows this not to be the case. A textbook on advanced calculus, despite being unknown and novel to most people, is decidedly uninteresting to those who don’t understand basic math — and even to those who do. It is too obscure, and contains little to draw one’s attention. Yet if considered on the basis of sheer novelty, toddlers should be more curious about calculus books than books on construction vehicles, since they already know more about the latter.

The world is so diverse and unpredictable that pure novelty by itself is insufficient to draw your attention. Algorithms that calculate novelty from information density assume that your interest in an event is proportional to the degree of deviation from expectation. Yet often tiny changes, like a slightly different note in a song, stand out like a sore thumb. At other times you can easily tune out a cacophony of traffic outside your window. At the extreme end, pure randomness, like noise on a CR-TV, is as novel as it gets, and yet it fails to captivate anyone — a paradox that leads to the “noisy TV problem” in AI.

The crux of the issue is that any true novelty in sensory experiences would be inherently unrecognizable. It is equivalent to random noise that you cannot decipher, and exists so far outside your ability to ask questions about it that it is impossible to notice; like signage in a foreign country written in a language you don’t speak. Curiosity can only direct itself towards known things. For you to learn anything at all from an experience, you must be able to fit what curiosity delivers into a framework you already understand. It is not a haphazard reaction to novelty, it is a determinate act of asking a question and absorbing an answer. There is something directed, targeted, and specific about situations that trigger curiosity that distinguishes them from everyday random noise. An underlying tension is at work, searching for an answer; you want to find out something about the world.

Consider what happens when you watch a mystery movie. The suspense created by the setup is not simply a desire to know what comes next; that would be true for any movie you haven’t seen. It is the desire to know the answer to a specific question. The lack of an answer feels uncomfortable. Unsatisfied curiosity — not knowing the ending — is ultimately frustrating. An so curiosity has a negative side too, one which spurs your inquiry until you find what you need.

As discussed in other posts, the world will always behave unexpectedly to some degree, so you only pay attention or perk up when it goes against what you feel comfortable with. It is not just novelty, but uncomfortable novelty — novelty that undermines what you deem safe or secure, or which hints at possible problems — that drives your attention and curiosity¹. What you then learn is the resolution to that need.

And a murder-mystery would not be interesting if you didn’t have personal, emotional reasons to want to know who the perpetrator was. A simple multiplication of two numbers, e.g. 4238 x 1181, is also a mystery, and also has an answer, but fails to capture your attention, because the desire to know is absent. To captivate your interest, you must recognize something as a concern. You think: “there is a killer out there, roaming free — I wouldn’t want that to happen in my life, so what would I do?” This is a directed question, which means your mind has framed the precipitating experiences into a specific request — whodunit?

The resolution can’t just be whatever comes next in sequence. Not just any set of events will satisfy your curiosity, but one that resolves the underlying tension according to what you consider satisfactory. If a commercial break occurs, its contents do not count. Nor does a cliffhanger ending, nor the credits. Curiosity must be able to fit the answer into one of the boxes you are already familiar with and, in this context, are placated by — a resolution. The need to know reflects the presence of a situation that is potentially detrimental, but without the accompanying thoughts (plans/intents) that would put one at ease.

Every act of curiosity has this structure, and the mind, rather than being shaped by the world in its explorations, must determine beforehand what qualifies as a resolution. Although it need not foresee the precise answer, the answer must fit a category that is recognized as such². Even when curiosity seems purely reactive to unexpected events, e.g. “what was that sound?”, the answer must be something that, in your opinion, explains it — a toaster, or a car alarm, etc., and not a flower or a cloud.

There are many types of curiosity

Curiosity is not monolithic or homogeneous. The diversity of pursuits you develop over time each create their own modes of curiosity. As you mature and understand more about the world, the scope of your curiosity changes as well, because the content of your questions shifts to new domains. You may become curious about engineering, history, cosmology, religion, etc., each for their own reasons. These topics rarely elicit a toddler’s attention, despite being far more unknown to them. By adulthood you have built up a variety of concepts that trigger their own tensions and resolutions. Curiosity only appears singular or homogeneous from an outsider’s perspective, from the standpoint of someone who doesn’t realize what specifically is driving you to explore and learn in that case. From the inside, the experience is quite targeted.

For example, toddlers’ widespread curiosity about construction vehicles (diggers, cranes, trucks) is not incidental. At a time when they begin to engage their own ability to manipulate the world around them, they see in construction vehicles a powerful manifestation of the things they themselves wish to do: lifting heavy things, shovelling dirt, moving objects from place to place. Construction vehicles are a toddler power fantasy; like superheroes will become as they age up.

In adults, curiosity most often reaches for explicit, communicable facts, belying its roots in interpersonal motives³. At a time when you would hope to present an image of yourself as knowledgeable, not knowing some piece of trivia may cause you to look ignorant, leading to anxieties about your social image. Thus, when confronted with a potential factual question to which you do not have an answer (e.g. which African country is experiencing a famine in 2024?) there arises — in many people — a need to fill in the gaps with an answer they can speak to when the topic comes up, when all eyes are turned on them, anticipating their contribution. This is especially true if it is in your publicly declared, or privately believed, area of specialty. Such exploratory actions look like curiosity to someone who is unaware of your deeper motivations — and that may include yourself.

In other, more practical situations, such as in engineering, not understanding how to resolve a potential technical challenge may make you appear less competent in your field, especially if it is a challenge your peers have already resolved. So you are driven to study others’ solutions, which again looks like curiosity.

Although boredom — being the antithesis of novelty and interest — may appear to also drive curiosity, closer inspection reveals they are unrelated. For a start, boredom as a concept is difficult to define. The same experience and behaviour —i.e. a lack of variation in your environment combined with inaction — may often be termed comfort, ease, or placidity — imagine lounging on a beach. Boredom, rather, arises when a constraint is placed upon your actions, when you would rather be doing something else. Consider how, when bored, you may browse what’s available on TV with nothing much capturing your interest. Though you may not be aware of it, your heart yearns to be doing something else. You may perhaps be confused about what it is that you want. Either way a constraint is placed on you.

There are other behaviours may externally resemble curiosity, but are not directed towards gaining explicit knowledge, only competence. A child playing with blocks may look like they are trying to understand the laws of physics, but their goal is not to acquire explicit understanding. They desire control, the ability to make objects conform to their will. This is more like gaining proficiency in a sport than curiosity per se. More accurately, control and curiosity are on a spectrum; or rather curiosity is a type of control. Curiosity is “control” as learning something that may prove useful — filling in the uncomfortable gaps in one’s knowledge. It reflects cases when not knowing, in one sense or another, represents a problem for you.

Impartial curiosity is a fantasy

Standard theories of curiosity promote a model of impartial curiosity, as curiosity driven purely by sensory novelty. This is a fantasy. It reflects an idealized, virtue-laden image of humans as objective, scientifically curious explorers, unbiased by selfish motives. It is driven by the realization that only impartial curiosity can obtain to impartial truth. To admit that all acts of curiosity have “secret” personal motivations shaping their discoveries shatters that fantasy and that self-image. And so the hypothesis is widely discredited — at least among those rationalists for whom this fantasy has an appeal⁴.

Nevertheless, it is a necessary ego-sacrifice if we are to uncover the true roots of curiosity and resolve its inherent paradoxes. For example, one benefit of framing curiosity as a type of control is that we can more naturally balance its pursuit with the need to be cautious in dangerous situations. Curiosity killed the cat, after all, and your life would be very short indeed if you chased every will-o’-the-wisp that caught your eye. If, on the other hand, specific motivations drive acts of curiosity — that is, if you only pursue knowledge you deem to be necessary, when it is necessary — curiosity is now like any other attempt to control your situation towards your goals. You only explore as much as you feel you need to, driven by a desire which implicitly balances the cost or risk against what may be gained; a calculation which could not be made unless you knew what you were gaining. Thus curiosity and caution are no longer antitheses; in some cases it would be incautious not to be curious.

In the end, undirected curiosity is meaningless. To know that you don’t know something, you must know enough about it to recognize what your gaps look like, and what would fill them. Furthermore, of the innumerable things you could be curious about — and could waste your time on — that which captivates your attention must be guided in some way by an underlying personal motive.

True novelty entails a change of values

So where does this leave true novelty? According to the above analysis, teh results of your exploration of the world must necessarily fit your worldview. Your interest in movies is not just an urge for something novel but a search for what you want, an affirmation of your existing perspective. There can be no true exploration then; nor should there be, since the world is dangerous, and you should only take risks when there is a known or necessary benefit. The result is a rather blinkered and constrained view of the world, and doesn’t seem to leave any open space for real personal growth. Following your curiosity is more likely to entrench your existing predispositions by reconfirming what you already knew or wanted to see. This all seems wrong; and it is wrong, because there is more to the picture.

True novelty, when it arrives, always comes unannounced and unlooked-for. It imposes itself on you without your consent or inquiry. Curiosity plays no part in it. It defies your wishes, destroys old paradigms, and reconstitutes your mind. It is experienced in moments of profound despair, and even more profound revelation. These moments alter what you value and what you strive for going forward. Mundane curiosity and novelty, on the other hand, far from opening the doors to the infinite, serve to inform you about the boundaries of your thinking. Though you may not know exactly what you will discover, the answer must still fit a model of what you believe. Everything that you can expect is only a finite infinite. The infinite infinite rather rushes upon you unexpected, out of the corner of your eye.

In this new sense, curiosity and risk-taking are on orthogonal axes, unaffected by each other. You can only calculate risks and trade-offs when you know what you will gain or lose. Yet who, in the 16th century, could foresee the benefits of scientific exploration? Truly novel understanding is not a tangible benefit like cash or property, one that can be weighed and calculated against risks, but an opening of new doors of perception, a reshaping of the world into an unforeseen standpoint that recalibrates how you measure costs and risks.

Mundane curiosity is a guardrail that keeps you moving on the path you’ve already set for yourself. True novelty comes from outside and transforms your perspective; and true curiosity is the openness to such change.

¹ Novelty that draws your attention is that which is outside what you’d like to happen. The unknown is something partly known, but uncomfortable due to this incompleteness. Novelty is not defined with respect to what will happen, but only what you want to happen; it is that which doesn’t go according to your desires.

² “Fit a category” may seem ambiguous, and a more concrete breakdown is discussed in this post. In brief, what is noted and remembered must precede, in time, a recognized resolution.

³ One of the defining characteristics of what is commonly termed “curiosity” is that the topic is one of social import — a communicative, explicit fact, like who was the fourth king of Luxembourg, or what the difference is between a law and a tort. This is only a linguistic convention, though, and we need not follow it.

⁴ The illusion is supported by the belief that our image of reality already reflects reality — thus novelty in your experience of the world must echo some interesting novelty in reality itself. This, of course, ignores the subjective dimension in all experiences.