Equivocate Though You May, AGI Will Never Exist

Why the star of AGI can, should, and likely will burn out.

Written by Brook Schaaf

---

OpenAI, the maker of ChatGPT, just raised $6.6 billion. It may be that the creation of artificial general intelligence (AGI) buoys the associated valuation of over $150 billion.

Sam Altman, as quoted by ChatGPT, said, “AGI is going to be the most important technological development in human history,” and “AGI will be the most dramatic economic transformation in the history of capitalism.” (Some of his detractors might say that the transformation would indeed be dramatic, because the very business is doomed.) 

While FMTC is a paid subscriber to ChatGPT’s GPT-4o mini, I would not advise anyone to bet on AGI. For my two cents, it cannot exist, at least not unless you change the definition of intelligence to exclude understanding or comprehension. An excellent case for this is philosopher John Searle’s famous Chinese Room Argument. Imagine a man sitting in a room receiving symbols in Chinese. He follows a book of instructions to send, in response, other symbols, also in Chinese. Even if the instructions are so good and the actions so fast as to pass the famous Turing test, it cannot be said that he understands Chinese. 

As Searle later wrote, “We cannot get from syntactical to the semantic just by having the syntactical operations and nothing else.” In a similar, though faraway, vein, theologian Reinhold Niebuhr wrote, “The self knows the world, insofar as it knows the world, because it stands outside both itself and the world.” 

No machine can reflect on itself, in part because reflection is inherently an emotional experience. If I reflect on my alma mater’s weekend football loss, I feel sad. If I think of seeing a little tuft of hair under a mess of bedclothes, I feel love and amusement. That which we call intelligence includes the vast subconscious, sometimes referred to as the soul, even by atheists. Plato said the soul consists of three parts: the calculating, the desiring, and the spirited. 

The calculating or reasoning part might be said to already exist. But a computer has no spirit and cannot desire. As self-reported, ChatGPT “doesn’t have consciousness, emotions, intentions, or self-awareness.” It will never feel happy or sad or, in a fit of pique, write an essay about the same.