The Green Swan: Part 2

The Logical Impulse

From Narrow To General AI
18 min readSep 2, 2022

Read part 1 of the series.

There are many men that reason exceeding clear and rightly, who know not how to make a syllogism…A man knows first and then he is able to prove syllogistically. — Essay Concerning Human Understanding, Ch 17, John Locke

If ever you find yourself observing your own stream of thinking, you may notice a number of subconscious forces pulling your thoughts in various directions. You might isolate a “logical impulse” from within these forces — that is, an inclination to accept logically correct statements and reject illogical ones. This impulse occasionally shows up as a self-correcting mechanism, and for a brief window it directs your thinking. Whatever it’s origin, it’s clear that this impulse is a necessary ingredient for both informal and formal logic.

You might initially guess that such an impulse is built into your mind by nature, and is available from birth. In this post however, I’ll argue that the inclination to be logical is a conditioned social reflex, one that’s acquired in childhood. To understand how it first arises, we have to look back at your earliest days of language development, and to the moment when you made your first mistakes.

A Child’s First Errors

As a child first learning to speak, you and I felt free to express any thought you liked; real, imagined, aspirational, whatever. You learned to attach words to your needs and thoughts, and didn’t care much about being consistent. “Daddy is tall!” you might think on a whim, then say so without hesitation. As these were your first utterances, anything you said was greeted with smiles, even applause. Things were good for a while.

At some point you expressed a passing whim that you hoped would be positively received — “Daddy is taller than the Hulk!” But instead of accepting it, your audience became hardened, skeptical. “What do you mean?” asked an older sibling, “Daddy isn’t 10 feet tall”. This was unexpected; people were usually so accommodating, or at least attentive. You were suddenly faced a new problem: getting people to be happy with what you say.

Then, as now, you only spoke because you wanted to make something happen. This usually fell into one of three groups: getting people to act a certain way, getting people to think a certain way, or entertaining people so they have a better impression of you.¹

So as refutations, skeptical questions, and counter-arguments came more frequently and aggressively, they hurt, because they meant you wouldn’t be listened to, wouldn’t be believed, maybe you were even mocked. After a number of jarring experiences, you became more inhibited in your speech. Each time you were about to speak, a twinge of anxiety needled your soul. This could end badly.

In your childish imagination, it did truly seem like daddy was taller than the Hulk; that’s why you said it. You weren’t intending to be unreasonable. You simply hadn’t had any reason to consider being precise in your assertions. And why would you? Your statement could have been interpreted as hyperbole, in which case it wouldn’t have caused any backlash. The distinction between free-form, non-logical assertions — like hyperbole — and those that society requires to be more precise is determined entirely by whether or not you experience push-back from others.

So as others began to challenge your words, they planted the first seeds of what would eventually become your “objectivity”. The initial incentive for rational thinking had now taken root, and would persist for as long as you still wanted socialize — presumably your entire life.

Internalization

[Logical] norms govern our rational interactions with our peers. For instance, they might be thought to codify the permissions and obligations governing certain kinds of dialogues. Viewed from this perspective, logic’s normative impact on the intra-personal activity of reasoning is merely derivative, arrived at through a process of interiorization. — The Normative Status of Logic, Stanford Encyclopedia of Philosophy

Soon such incidents became the norm. You tried your best to avoid them. To this end, you began to internalize others’ counter-arguments. Before you made your case, you recalled and brought to mind the sorts of refutations your words might encounter. Your goal was to prepare yourself by rehearsing, in thought, how you could outmanoeuvre them, and avoid the worst of their scorn.

This manifested as a rudimentary type of planning. You imagined (relived) the hurtful refutations, then your mind came up with options and plans that you predicted would to evade their snares. You picked up these solutions by observing how others overcame similar situations, and parroting their words.

Such parroting could hardly be called “logic”; it was more like cautious circumspection and social conformity. This is the same process as when adults choose not to reveal that they actually enjoyed an otherwise unpopular movie; or more simply it is like teaching a child the correct word for a given object or situation, and correcting them when they fail to conform to how that language is used. As a child, such conformity was indeed your only modus operandi, since you weren’t cognizant enough to think more deeply about what you were going to say.

All early education in informal logic happens through the correction and imitation of explicit statements, if only because there is no other way to teach a child. Your immature mind at the time was exclusively occupied with thoughts of what you wanted and what you disliked. You easily mingled together fantasy and reality — that is, what you wanted to be true and what was actually true. Through social training, you gradually acquired a new reflex, a hesitation, an impulse to think things through before speaking.

By the time you were a pre-teen, society’s demand for you speak a certain way was firmly established, internalized, and accounted for. It eventually reached the point where you didn’t even have to be in front of an audience to feel pressured to adhere to such rules. In an especially invasive turn, your mind began to rearrange your personal and private thoughts, in deference to a now ubiquitous anxiety, that of being refuted². Even in your solitary moments, whenever you had a thought expressed in words, and you foresaw a possible refutation, your mind triggered its habitual tensions and would not let up until they were resolved. You became your own thought-police.

From that point forward, the development of rational thinking was the development, and ultimately refinement, of a variety of such reactions.

Playing by their Rules

If I said that a friend of mine was “nice” and later claimed that he was upsetting me, that is not considered contradictory; people change. If I said that an animal is a herbivore and then claimed it ate a zebra I would definitely be challenged. But as a child, you would not³ initially distinguish these two examples. In the same way, you wouldn’t automatically think that a spray-painted swan is not a valid counterexample to “all swans are white”. After all, stop signs are spray-painted red, and everyone seems comfortable saying they are actually red.

Nor can the words in the predicate itself help you make the distinction. The green swan is in fact a “green swan”. You need to be taught the scope of interpretation behind each predicate. To distinguish these two cases, an adult would have had to correct you when you saw, say, a spray-painted statue of a swan and took it as a representation of the real thing. He might have framed it as a “trick”, or as “make-believe” — something emotionally resonant to your immature mind. Only by making an emotional impact — say, on your need for approval — could he rise above the noise and affect a cognitive change.⁴

By raising objections and concerns, people regularly coached you on how to think about various topics, and more importantly how to separate those for which greater precision was compulsory. For example, your peers would demand a high degree of rigour in mathematical proofs, but were lax when it came to idle sophistry. Poetry and humour thrived on contradictions. In response, you built a repertoire of context-specific responses⁵. And it was all new information to you; it couldn’t be derived apriori from what you already knew.

Remember that being “illogical” is only a transgression if there is an explicitly agreed upon formalism to transgress. In cases where it doesn’t apply — e.g. the phrase “give the job 110%!” — calling an assertion “illogical” makes the speaker sound robotic and inhuman. In any case, it would severely restrict your popularity.

This is why logic is always considered “optional”. Whether, and in what way, you choose to apply it is motivated by whether you encountered social pressure to do so given the specific words and context — and also if you want to pay heed to that pressure.

In the absence of any concern about being challenged, your default mode is to think freely and uninhibitedly. Without any corrective social pressure, your mind can comfortably accept any kind of contradiction without hesitation, as you saw with the phrase “less is more”. You only need to apply as much diligence or precision, and only in such a way, as you can get away with — sometimes none at all.

To be adaptive in its use of logic, an A.I. must be permitted to learn both if and why some kind of “logic” should be used in the first place; and make that choice based on context and intentions. Otherwise it would necessarily fail to generalize correctly, usually by overextending a rule to a situation where it is not applicable.

New Pressure Creates New Responses

To see how society’s corrective behaviour can give rise to informal logic in children, let’s look back at the problem of the green swan from the first post, and see how, as a child, you would have initially learned the predicate “all swans are white”. Specifically, let’s look at what induced you to learn the quantifier “all” and add it to the assertion.

As a prerequisite to this, you should already have to attach the word “swan” to a set of visual experiences, to attach names of colours to different sights, and to use the connector “is”. In all these instances, your only motive for learning was to impress your expectant parents and teachers, and get their approving smiles. They represented, in microcosm, the general social pressure to express your thoughts using the language your community demands.

At some point, you saw a green swan. Being naive, you thought to yourself “that swan is green”. Later, when an adult asked you what colour swans are, both colour memories were jogged, and you expressed them both.

You were corrected. You saw no approving smiles, just sad or confused frowns. This didn’t feel good. You didn’t know, or even try to know, why this happened. At this age you had no high-level concepts; you only knew that there were right words and there were wrong words. You needed some alternate set of words to keep you in people’s good books⁶. The adult in question repeated the correct answer for your benefit: “swans are white”, and you memorized it.

Later, a friend your age mentioned that she saw a blue swan (presumably fictional). You remembered the backlash from your own experience of saying something like that and, either to quiet your underlying anxiety, or to advise your friend against embarrassment, you spoke out the right answer, that is: “swans are white”. In imitation of your teacher, your face and body language conveyed that she needed to change her words to the correct ones. You didn’t yet know why exactly a blue swan was “wrong”. Your knowledge of biology was sparse. You couldn’t give her reasons, and she wouldn’t understand them if you could.

Your friend now pushed back. She was convinced she saw a blue one, and you had given her no reason to reject the evidence of her eyes. Perhaps this blue swan was ‘special’?

But from your perspective, you wanted some way to communicate that there are no exceptions; that she was in error. For this purpose, the word “all” found its use. As with every word you’d learned so far, you’d picked it up through imitation from some similar situation — whether a book, school, adults’ conversation — and it seemed to work then. You liked the word because it emphasized your assertion. So you reiterated: “all swans are white”.

You don’t yet know that the word “all” was a universal quantifier, and what the consequences of claiming that “all swans are white” was. To you it is just a word to rebuff your friend. A different phrase, such as “wrong! Swans are white” would have been equally useful, and indistinguishable to you from “all swans are white”. The word “all” lacked the conscious depth of meaning that you would later ascribe to it, a depth that you would uncover through further conversations and corrections.

There’s far more to the development of this single predicate than can be included in one post. Each word and grammatical construction in your lexicon has a long history of associations and triggering cues. They are all, however, built on and by social motives — to impress, to gain approval. Abstract words like “all” are also rooted in motives; here the motive was to refute cases incorrectly claimed to be exceptions, like the blue swan.

The Roots of Logical Principles

Still, it is difficult to call such early training “logic”, since it was mostly about parroting the right words. Every case was a special case, a context-specific response. It was like learning irregular grammar⁷, or how to avoid a social taboo. And just as you learned not to say “I eat-ed”, but rather “I ate”, each amendment had to be learned by heart. The only problem you were solving was how to avoid being embarrassed, being rejected, having teachers and peers push back. Social motives — even infantile ones — drove you to learn new phrases, and add nuance to old ones.

Nevertheless it is critical to begin to approach logic from this direction. It is the only way to address the problem described in the first post; how logic can be effectively applied while being sensitive to context. To correctly understand predicates you must always engage with the social motives behind them. This will give them flexibility, and allows you to productively adapt them to new situations. I emphasize that word, since to be productive you must have a goal in mind, towards which your behaviour is considered productive.

You can already see how the nuance of individual logical predicates and operations gets baked into such early training. Mature, formal or abstract logic grows out of these early interactions as a self-reflexive elaboration of your own thoughts. Tracing the progress of its development in children is a signpost for how logical concepts function in adults.

The notion of consistency, for instance, originally grew out of a specific social tension. Others called you out on your assertion, but they called you out in a particular way: they rejected your assertion by using an earlier statement you yourself made⁸. They emphasized that the two cannot coexist due to the experienced constraints of reality; a sort of rejection by deference to the authority of the universe itself.

The idea of confidence — likelihood — of an assertion took root when you made a sweeping statement, and someone challenged the limited scope of your experiences. “Are you sure” she asked “that all swans are white?” To avoid the embarrassment of being proven wrong by an unsympathetic peer, you learned to add weasel-words (“some”, “maybe”). Later, you may have explored this seminal motive alongside numeracy, and from there developed your understanding of probability.

The same is true not just of logical operations but also of the content of predicates. This is why the A.I. in the first post failed to generalize the colour of the swan correctly. It lacked the ability to pick up on the motive underpinning how “animal colours” are defined. When you define an animal’s colour you are actually addressing a hidden social question:

What colour should I tell people an animal will be next time they see one in the wild, or in a book?

or equivalently:

What colour will authority figures say this animal is?

These are problems of human interaction, of personal credibility, of not looking like a fool. It’s not a matter of what RGB colour the animal will appear in every image in a dataset — this latter is how the A.I. interpreted the question. You want to appear knowledgeable and helpful to others.

Approaching it this way, it seems obvious that spray-painting a swan, chemically altering it, wearing tinted glasses, and using purple light are irrelevant, since these are unlikely to pertain in the wild, and would be ignored by scientists. Were the A.I. attuned to these underlying social motives, it could have quickly picked up on such distinctions, and ignored any red herrings.

Motivated Reasoning

Every act of thinking and understanding, from concepts to logical operations, is motivated⁹. And although logical A.I.s (like Prolog) can be said to be goal-oriented, this series aims to situate that aspect of logic in a broader and more integrated scope; that is, to represent informal logic as a series of pragmatic deliberations intended to resolve immediate social tensions.

To see what I mean, let’s pull together all the threads we’ve discussed so far, and see how you, as a child, might try to reconcile the presence of the spray-painted green swan. This section depicts an abbreviated progression of thoughts. The key thing to note is that each of them is driven and evaluated by an underlying social motive.

The first challenge you encounter is that of impressing a teacher who quizzed you on the colour of swans. You imagine a swan, and quickly satisfy your teacher’s expectations by repeating a label you’ve learned: “white”.

Some time later you see a spray-painted green swan. You’re proud that you can identify its colour as “green”, and you think this aloud. But now your two statements “conflict” with one another. Saying it’s one colour when you think it’s a different one has previously landed you with the accusation of lying, or of ignorance. You’re concerned that’s how you’ll appear to others, but as yet you have no way to reconcile your thoughts — this is the problem of inconsistency.

One day, on telling someone the painted swan is “green” you are told you have been misled. An adult informs you that spray-painting a swan does not indicate its true colour. Your error makes you feel embarrassed. Fortunately, the same person restores your confidence by revealing a solution: the true colour of the swan is its original colour, white. When the problem of inconsistency rears its head again — in thought or in actual conversation — you can resolve it: “the green swan is actually white”.

Your mind may not follow this progression exactly, and many intermediate steps have been omitted for brevity. The key point is that every one of the above thoughts is a reaction to a social problem, usually centred around what you say, or plan to say. Granted, many of the above expressions could be written out more formally, e.g.

Spray painted swans are not their true colour

This swan is spray-painted

etc…

but this is not how children generally think. Rather their minds react only to the specifics of each problem as it has been impressed upon them through discourse. Your entire early experience of informal logic is a tapestry of small problems and solutions, with scant few abstractions or generalized rules. Any logic that is there is only such as is implicit in your behaviour.

Rationality Grows out of Irrationality

Most of us are, implicitly at least, logical realists¹⁰. We feel convinced that logic has some reality, either in the mind, in the world, or both. Therefore this post may have seemed odd to you. Looking inside your own mind, you may observe many cases where logical necessity inexorably draws along your train of thoughts from one to another. There must be some mental force or tendency, you would say, that pushes you to embrace sound logical conclusions and reject illogical ones.

And indeed there is: it is your social motives, your desire not to be embarrassed, to be respected, to be heard. This is the secret spring of that mental force that seeks to correct your speech and thoughts. Informal logic has been ingrained into your thinking to the point that it seems intuitive and unavoidable… but only in certain cases. And that’s the point; the only way to explain why we often blatantly disregard logic as is socially appropriate is to assume that it is contextually acquired and socially tailored.

For an A.I. to appropriately ground its use of logic, and at the same time to account for common departures from logic (like rationalization and wishful thinking), it must make the application of logic a socially motivated choice. An A.I. that lacked either the social motives or the regular interactions that informed it which expressions are problematic would be at a serious, perhaps insurmountable disadvantage.

I used the word “problematic”, and not “contradictory”. So far in this series we’ve discussed early childhood development, when issues are highlighted on a case by case basis. You only acquire the higher-level concept of “contradiction” once you have had enough negative experiences that you feel compelled to explore them with other people and to give them names. Ultimately, with introspection and some external help, you may discover — or rediscover — rules of formal logic. This is discussed in the last post of the series.

Concluded in part 3.

¹ Notice that “because it’s the truth” is not listed as a reason for speaking. No one ever says something merely because it’s true. Any person who did so would end up spouting an endless stream of beliefs without pausing.

² This should be unsurprising, since every linguistic thought is an implicit intention to speak, whether or not it is ever actually spoken.

³ I say “would not”, instead of “could not”, since you have no reason to even try to do so until it became a problem.

⁴ All current logical A.I. are cued to address a query by a special command from their trainer or from the architecture. For example, a question-answering A.I. is triggered to resolve a given query when the user presses “submit”; and it can’t refuse to do so. Humans don’t have any such native interface. You can choose to ignore a request, or even fail to recognize it as a request. This is a critical detail that is usually overlooked in logical A.I., and it raises the question: why do humans respond at all? You must have a reason for deciding to solve a given problem. And since that reason is in some way based on the spoken words themselves, it must be learned.

⁵ You may have guessed that the method being discussed would give rise to very specified reactions, each addressing a particular set of words, and not a general concept. This makes them context-specific. For instance, a person’s “height” may have a different set of rules than their “width”, despite both being morphological measurements. Mistakes in the latter would perhaps be considered an insult rather than an inaccuracy. There is a rich history of social experiences behind each idea and concept, a history that is usually forgotten as we age. This doesn’t mean that each set of stimuli needs to acquire a directly attached response. Thoughts allow for implicit generalization by triggering, from one stimulus, another that is remembered or fabricated. In addition, you don’t have to experience each lesson for yourself. Seeing or reading about others who make mistakes is an adequate substitute. What matters in that case is how those relevant people react to the words, and your empathy does the rest.

⁶ The need to keep your speech to within others’ prescribed limitations persists into adulthood. A more nuanced version of this drives my writing of this blog post; it keeps me honest and thorough.

⁷ Logic is similar to grammar, in that both are socially-imposed restrictions on explicit statements. For a few decades in the last century, grammar was considered to be a fundamental pattern of thinking, under the theory of “Universal Grammar”. These days, that theory is discredited. An analogous mistake, I believe, is being made regarding logic.

⁸ This may raise the obvious objection: wouldn’t the mind have to know that the two statements are opposites, which implies it has a built in negation function i.e. a logical ‘NOT’? As should be clear so far, negation in the sense of “problem” is assumed to be built in to the mind. But to differentiate between a logical contradiction and a simple rejection for any other reason requires you to specify that this particular set of statements contains a pair of “opposites”; two things that cannot coexist. And as Hume observed in Treatise of Human Nature (part 1, section 5, 7), it is impossible to determine a priori if two things are opposites — such a relation must always be learned through experience. Thus “contradiction” is a type of rejection, but one that you find you cannot overcome due to the constraints of reality itself.

⁹ Here “motivated” is contrasted to “automatic”.

¹⁰ In a certain sense the logical realist perspective is true. Since the symbols of communication only exist in a social context, it’s expected that the rules you apply to them would be socially determined as well. And just as decisions about grammar are always settled in an interpersonal context, society may also be permitted to decide how word-sounds are logically arranged. “Objectivity”, in this theory, is a social invention all the way down. The only mistake, then, would be to assume that human brains fundamentally (to the neurons) engage with this same objective construct of the world, and therefore so should A.I.

--

--

From Narrow To General AI

The road from Narrow AI to AGI presents both technical and philosophical challenges. This blog explores novel approaches and addresses longstanding questions.