A Comprehensive List of the Obstacles on the Road to AGI
How social pressure, personal motives, and complexity hinder AGI development
AGI is not a prudent career choice. Compared to the development of narrow AI, which is driven by economic motives, the development of AGI only hints at possible financial benefits; and never guarantees them. But the fact that it may not pay as well doesn’t matter to its practitioners. AGI research is built on more introspective ambitions. Its true, unspoken, goal is to explore the grandest mystery of them all: the riddle of the human mind.
And this is where its problems begin.
The scope of what needs to be explained in the mind is vast. Every time you crest some boundary hill, it reveals an entirely new landscape you had not even considered. It is natural in such a situation to put up artificial walls where you are, and say there is nothing of importance outside them. A researcher may ignore for instance, artistic passion, or scientific rationality, or some philosophical school of thought like transcendental metaphysics, or religion and spirituality, or quirky social behaviours, or other fruits of mind, and call them inconsequential or delusional. Or he may argue that all of these can be explained by plausibly extrapolating from the currently held theory. Putting up such boundaries is necessary for researchers who want enough stability to get work done in their short lifetimes. Boundaries will help you sleep at night; though you may have uneasy dreams.
If you judge by their articles and writings, then few who have ventured into the field of AGI have maintained their original optimism for finding its philosopher’s stone. In each case they limit their work to some corner, some theory, and write the holistic enterprise off as chimerical. This is a necessary compromise when faced with the baffling number and type of questions that arise when attempting to explain the mind. In this post I will try to map out all the possible questions, issues, and domains that a true theory of AGI should reconcile. Hold onto your hats, it’s going to be rough.
It’s tempting to begin the study of mind via introspection. Introspection seems to be the first, most accessible, and most relevant source of information available on the mind. But you can’t always rely on the information you find by looking into yourself. There is little consensus as to what the mechanism is by which the mind forms beliefs about the world and itself, especially at higher levels of abstraction. If it has any biases, limitations, or inherent blind spots, they will be just as active when looking inside one’s own mind as when you study the natural world. All in all, tempering introspection with a bit of external critique would not go amiss.
Introspection can’t be ignored completely either. There are many mental events, such as daydreaming or religious contemplation, that have no obvious outward expression. Ultimately, what you’re looking for is a model that you can use to understand your own, and other’s minds, not just their external behaviour.
The resulting “model”, whatever its specific content, will inevitably be some type of “idea” or set of mental entities held in the mind. So you’re faced with a strange situation: you are building a set of mental constructs (the theory or model) to explain mental events (thoughts, feelings, concepts) in a way that is satisfying to your mind, i.e. another set of mental events (motives and goals), and which can be handled by that same mind. Any quirks or prejudices of human understanding must necessarily be present and active through the whole chain. If, as is often said, people believe what they want to believe, then your model of your own mind will be similarly bent towards what you like, or at the very least what you find useful.
It is even conceivable — in the extreme case — that you are weaving the entire tapestry of your thoughts and theories out of chaos. If the mind is itself a giant machine of self-delusion, every belief you have, no matter how convinced you are and no matter how you may point to pragmatic evidence from objective experiments, would in the end be a self-reinforcing delusion. You can see how paradox builds on paradox. Being comfortable with paradoxes may be a prerequisite for the enterprise of AGI.
The inevitability of such paradoxes extends outside your understanding of your introspective mental life and into the space of reality as well. You are confronted, at first glance, with a bewildering array of philosophical questions concerning the fundamentals of truth, life, and existence, and with apparently no concrete means of answering them. Exploring AGI may entail resolving contradictions in competing theories about free will vs determinism, materialism vs idealism, truth vs perception, subjectivity vs objectivity, fine art vs personal taste, the individual vs society, innate vs learned ideas, logic vs intuition, and the many faces of morality and virtue. Each presents of itself a whole continent of questions. To presume that a single theory — or person — can answer them all in one stroke of the brush seems to be the highest form of arrogance. And yet the expectation is that a person who proposed a theory of AGI would also have to understand and explain how their theory applies to every possible facet of human thinking, lest they be attacked from the side where they are weakest. Most people confronted with such a challenge give up at this point, for no other reason than being ashamed to publicly claim to have solved them all — even if, by chance they actually had.
Beyond such foundational questions is the entire realm of human discovery and knowledge. Fields as diverse as quantum physics and absurdist poetry can all take root and come to flower in one and the same mind. Any complete theory of AGI must be able to explain all of these, even in cases where they appear to contradict each other. The force of human creativity and ingenuity shows itself through many activities and manifestations, driven, it seems, by an indomitable will for more, and better. These forces of exploration and ambition throw the mind into its unlimited possibilities. The mind reaches out into cosmic theories, and world-changing sociological reforms. At times it climbs mountains, and at other times it wants only peace and habitual comforts. And these must all be encapsulated, in some way or another, in a theory of AGI.
Would that the problems ended there; however, it is not so. At a granular level, the individual mind, your mind, is an intricate latticework of interconnected thoughts, emotions, memories, and meanings. In any given five seconds, a dozen or so ideas and feelings may pass through it, many seemingly without cause or explanation. And yet they all, or almost all, appear to play a critical role in your subsequent thoughts, feelings, plans and actions; so they must be considered individually and also as a connected whole. What are these mental entities? By what specific process did each come to be there? Which of them was learned, which innate, and is that even a meaningful distinction? How can you introspectively perceive your thoughts and feelings at all? Are there some mental events that are invisible to the inner eye, and if so, why? Could you change your thoughts and feelings if you wanted to? How? Is each thought or feeling right or wrong, good or evil, true or false, and what does any of that mean? And finally, bringing all of these together is your consciousness itself, which is no small riddle — what is it, and how should you interpret it?
It is just at this juncture, unfortunately, that a more insidious hurdle in the path of AGI rears its ugly head; insidious, because it is self-imposed. I’ll describe this phenomenon as it manifests in two ways. In the first, we may not want to understand our minds fully, because doing so will not depict us in the way we wish to be represented. When we develop a theory of mind, we most often paint an image of an ideal intelligence — how we want to be, not how we actually are. The true mind itself is sloppy, chaotic, irrational, ignorant, lazy and selfish. It is rarely a diligent student of logic or morality. All cognitive projects you undertake to impose a structure on your mind — such as explaining logic or morality — are self-reflexive attempts to formalise and create order out of the mind’s inherent disorder. This is why they fall apart at the edges, and when you observe children’s amoral and uncritical behaviour. All this is unbecoming of “God’s greatest creation”.
Secondly, may people despise or fear determinism, that is, the discovery that all cognition is driven forward by unseen forces and is outside the control of some imagined, separate “soul” — the latter which may not even exist. Even hinting at this possibility can be both personally unpleasant and liable to social reproof. It suggests, at least in its simplest interpretation, that people cannot be held responsible for their actions, because it is difficult to moralize to a machine. Yet if one is to truly recreate the mind in some artificial medium, that assumes that the mind is to a large part deterministic or probabilistic. A theory of AGI must reject most definitions of the “free-will ghost in the machine”, that interferes with the mind’s natural behaviour from the outside. The same theory would still have to explain subjectivity, and the nature of qualia, only now as part of a deterministic machinery. The whole universe of abstract and self-reflexive thought must flow from the theory, and through it, as naturally and unstoppably as a river. All artistic pursuits, scientific formulas, social motives, emotional struggles, learning how to learn, self reflections and self-improvements, political ideologies, loves and hates, and more must be weaved into one inevitable and endlessly turning wheel.
Both the fear of determinism and the desire to view ourselves positively have the same inhibitory effect on theories of mind. Theories start to unconsciously carve out a space for freedom or morality. You can spot this tendency anytime you see a theory of mind or of cognition suggest that some mental behaviour is “better” than another. When this happens, a space is being created for an executive “ghost” in the machine, a magic “something” that has the ability to chose one path or the other. A true theory of AGI must have already incorporated that same executor in its formulas. An entity that observes itself, learns from others, and makes profound moral decisions is still one that is predestined to do so.
Congrats for reading this far. Before we close off this post, there is one last category of questions, which may seem trifling, yet may also be the most important. Spirituality, values, and personal growth present a special problem for any individual researcher. Your mind wants things; of this there is no doubt. What your mind wants and how those desires change with experience follows its own complex trajectory, and that demands its own exposition. I’m not referring here to the general question of morality, but specifically how it presents a unique problem for you and you alone. Though this is of seemingly small consequence compared to the big philosophical questions above, it is for you the most important issue. It points the way to the meaning of your life, and will shape all decisions you make from start to end. Questions like “what should I do in life?” define who you are, and also how you will be in the future. Were you to alter a few select aspects of your own values, you might give up the entire project of discovering AGI on a whim, and become a professional race car driver or high school teacher. No theory of AGI will be completely satisfying to you unless it answers one last, non-negotiable question: how can I become the kind of person that I want to be?
How can a single model ever hope to stitch together all these threads into a few diagrams and technical components? To call it a Titanic task is an understatement; the Titans appear minuscule by comparison. It is perhaps not to be faulted then, that many researchers and philosophers of mind give up on the whole project, and focus on one facet of the problem exclusively, making it the central argument of their life’s thesis.
What’s the alternative? A grand explanation of how humans have engaged in all fields of study from poetry to chemistry, a solution to many if not all philosophical riddles, a detailed explanation and provenance of the moment-to-moment inner life of any given human being, an etiology of psychological unease, and to boot, an answer to your own meaning of life. And all this must be presented in a way that is tolerable to the mind; you must be willing to accept the answer and not find it distasteful. If you’re not daunted by this prospect, then you haven’t been paying attention; or maybe you lack a little humility.
But being daunted is not the same as giving up. Humans are persistent and tenacious creatures. And given enough persistence, a few cracks may open up in the mountain of questions, enough to lever open a door. The first leverage point comes by asking one question: what is there that is not up for debate? What is the mind compelled to do in all cases, and seemingly without its say-so? This will be our starting point.
Read part 2: Guidelines for how to begin, getting comfortable with determinism
You may have noticed that I left out from the list “fear of AI apocalypse”. Maybe you consider this a hurdle, but I suggest it isn’t for two reasons. Firstly, because no one can see the future; secondly, such fears have never stopped human hubris from running the species off of existential cliffs before. I have confidence that financially-motivated humans will be able to easily overcome such trifling anxieties as catastrophic self-annihilation.
