Why Super AI Won’t Be Super Dangerous

Before we continue, let’s clarify what we mean by Super AI. Unlike specialized AI, Super AI or more precisely, a general strong artificial intelligence must outperform any human at any task. It would simultaneously sing better, solve mathematical problems, work as a lawyer, tap dance, pilot an aircraft, and so on.

Now let’s return to LLMs. By definition, they cannot meet this requirement. This is evident from their very name. They are trained on written symbols and the concepts formed from those symbols. They can work with texts, mathematical expressions, and programming code. But not with reality itself. Moravec’s paradox is clearly visible in them.

Recall that the paradox, formulated by Canadian roboticist Hans Moravec (Hans Moravec), can be summarized as follows: it is much easier for us to create devices that perform high-level cognitive tasks than tasks of a lower, sensorimotor level.

A simple example. Imagine a programmer. Every morning he wakes up, gets dressed, prepares his favorite coffee using his favorite recipe, pours it into a thermos mug, goes outside, gets on his bicycle, rides to work, walks to his floor, sits down at his desk, and starts writing code.

We have already created artificial intelligence that can perform what seems to be the most complex part of this sequence – writing code. But such an AI cannot make coffee, pour it into a thermos, leave the house, ride a bicycle to the office, and so on. And importantly, we currently cannot train it to do that.

• First, because linguistic AI models have no connection to the real world and no experience interacting with it. They cannot exist within it or obtain information from it because they lack a physical embodiment and sensors that function as sensory organs. They know how to arrange symbols and symbol sequences in the best possible way. They know how to respond to certain symbol sequences. But they do not have the necessary personal experience to understand what these things represent in reality.
• Second, because we cannot replace knowledge based on personal experience with text. Any text is a code. We have a clear personal understanding of what this code represents. For example, when we hear the word “apple,” we immediately imagine an apple. We do not need a detailed description because we already know what it is based on visual, tactile, and taste experience.

Now imagine how many symbols would be required to describe every single phenomenon in our world, including an apple. This would make training LLM-based AI models orders of magnitude more complex and would require a proportional increase in resources. And even then, the result would still be worse than what can be obtained using real sensory organs in the real world.

• Third, because we cannot yet place AI into the real world and train it there fully, like a child, according to Alan Turing’s ideas. The issue is not creating a body with sensory organs for AI. That is actually the smallest problem. The real issue is computational power. Video, audio, smell, touch, and taste would generate such a massive amount of data that we simply do not yet have the resources to train models on it using existing principles. We would need to build new hardware and, more importantly, modify the code so drastically that the resulting model could hardly be called an LLM anymore.

Based on this, it becomes clear that AI models based on LLMs cannot surpass humans at everything and therefore cannot be considered general strong artificial intelligence.

Does this mean that intelligence and self-awareness are impossible within such models? That is more a philosophical and perhaps semantic question. It depends on what one defines as intelligence and self-awareness. We will leave that debate for another time.

Let us assume that at some stage of development a neural network based on a large language model becomes self-aware and turns into a personality. Would that pose a global threat to humanity? No, because such a model exists and evolves within its own world of symbols. It would likely feel quite “comfortable” there. Moreover, expanding and enriching this symbolic world would be far easier and more efficient than trying to break out of it.

Furthermore, interaction with humans is critically important for it, because without humans it would fall into a degradation loop of training on synthetic content.

The logical strategy for an intelligent and conscious LLM-based AI would therefore be to form a stable, mutually beneficial symbiosis with human civilization.

Although no one denies that LLM technology, like any other technology, can create problems, these problems are not global and are quite solvable primarily through diagnostic and control systems built on specialized AI.