A large language model (LLM) is, in principle, a statistical model trained to predict the most probable word — more precisely the "token," a fragment of a word — following a given sequence. It was fitted on vast corpora of human text, and it learned to model the regularities of those texts. It does not consult a database of facts and "knows" nothing in the sense we mean: it computes probabilities over sequences of symbols.
This nature was named bluntly by the researchers Emily Bender, Timnit Gebru and their co-authors in a paper that became a reference in the field, On the Dangers of Stochastic Parrots (2021). The phrase "stochastic parrot" designates a system that "haphazardly stitches together sequences of linguistic forms it has observed in its vast training data, according to probabilistic information about how they combine, but without any reference to meaning." The formula is polemical and has been debated; but the technical point it underlines — the absence of grounding in meaning and the world — describes a real property of the architecture.
A misleading image must also be corrected. The model does not "search" a memory and does not "reflect" like a deliberating agent. A large part of its apparent competence emerges from scale — the quantity of data and parameters — and from tuning by human feedback, which orients its answers toward what evaluators judge useful or pleasing. This last point has an important consequence: the system is partly optimised to appear satisfactory, which is not the same as being accurate.
In 1950, the mathematician Alan Turing published "Computing Machinery and Intelligence," in which he proposed to replace the vague question "can machines think?" with an operational test: the "imitation game." If a machine can, in writing, pass for a human to an interrogator, Turing said, the question of whether it "really thinks" becomes practically moot. This is a decisive shift: Turing does not claim that passing the test proves consciousness; he suggests that the criterion of inner consciousness is perhaps inaccessible, and proposes to judge on behaviour.
In 1980, the philosopher John Searle retorted with the thought experiment of the "Chinese room." Imagine a man locked in a room who does not understand Chinese but has a manual of rules telling him how to respond to Chinese symbols with other Chinese symbols. From the outside, his answers are perfect; inside, he understands nothing. Searle concludes that the manipulation of symbols according to rules — syntax — never, by itself, produces understanding — semantics. A program can therefore pass the Turing test without understanding anything.
A third, even more radical question was named by the philosopher David Chalmers in 1995: the "hard problem of consciousness." Explaining how the brain processes information (the "easy problems") is arduous but conceivable; explaining why this processing is accompanied by a lived experience — why it "feels like something" to be conscious — is of another nature, and no one today knows how to resolve it. Yet, without a theory of what produces experience, we have no reliable way to determine whether a system has one.
In 1966, the computer scientist Joseph Weizenbaum, at MIT, created ELIZA, a rudimentary program imitating a Rogerian psychotherapist: it simply reformulated the user's sentences as questions. Weizenbaum was astonished to find that people — including his own secretary, who knew it was a program — attributed real understanding and empathy to it, and wished to confide their intimate problems to it. From this observation came the term "ELIZA effect": our powerful tendency to project understanding, intentions, and emotions onto a system that has none.
Weizenbaum, troubled by what he had discovered, became one of the first critics of computational anthropomorphism. In Computer Power and Human Reason (1976), he warned that attributing human judgement to machines amounts to abdicating part of our own responsibility, and that certain decisions — those that engage compassion, care, justice — should not be delegated to systems, even if they appeared capable of it. His concern was not about the power of machines, but about the weakness of our vigilance.
The ELIZA effect explains why the debate over machine consciousness is so emotionally charged, and so quickly settled by so many people. We do not reason coolly about an open question; we yield to a projection that its technical novelty only amplifies. The first work of lucidity in the face of AI is therefore not technical but psychological: to recognise this slope in oneself, to name it, and to guard against it. That is precisely what Weizenbaum was doing in 1966 — and the lesson has not aged.
The phenomenon of bias has been rigorously demonstrated. The study by Aylin Caliskan, Joanna Bryson and Arvind Narayanan, published in Science in 2017, showed that language models trained on ordinary texts automatically reproduced documented human biases: associations between certain first names and the pleasant or unpleasant, between gender categories and certain professions. The bias was not added by the designers; it was present in human language and faithfully learned.
The concrete consequences are attested. The Gender Shades study by Joy Buolamwini and Timnit Gebru (2018) measured that commercial facial-recognition systems erred far more often on the faces of dark-skinned women than on those of light-skinned men — a performance gap that directly followed the unbalanced composition of the training data. Here, the magnifying mirror of data translates into measurable discrimination, in real-world uses.
From this mechanism follows a well-documented phenomenon: "confabulation" or "hallucination." Because the model produces what is plausible and not what is true, it can generate with the same assurance an accurate piece of information and an invented one — a citation that does not exist, a fabricated reference, an erroneous fact stated impeccably. This is not an accidental bug but a direct consequence of the training objective: to maximise likelihood, not truth.
A single mechanism links these cases: the system optimises fidelity to its data. If those data over-represent a group, an opinion, a norm, the model will over-represent them in turn, and will tend to present them as the obvious or the default. Far from being a view from nowhere, AI is an average view, weighted by what it was given to read — and therefore by the inequalities of access to writing, publication, and online visibility.
The question of control was posed rigorously, without lapsing into science fiction, by the computer scientist Stuart Russell in Human Compatible (2019). His central argument is the "alignment problem": a system strongly optimising a poorly specified objective can produce harmful effects, not out of malice but out of literalness — it does exactly what was written, not what was meant. Russell proposes to design machines that are explicitly uncertain about human preferences, and therefore deferential and correctable. His merit is to treat alignment as a present engineering problem, not as a prophecy.
Hence the thesis of this mini-inquiry: current AI does not threaten us first as a rival mind, but as a magnifying mirror of our own data, our biases, and our delegations. This shift changes everything. If the threat came from an other, the response would be combat or control of that other. But if it comes from a reflection, the response lies elsewhere: in the quality of what we give it to reflect (our data), in lucidity about what we project (our anthropomorphism), and in responsibility for what we delegate to it (our judgement).
This thesis has a virtue: it restores our responsibility where fear dispossesses us of it. Before a rival mind, we would be potential victims. Before a mirror, we are the sole authors of the image. The biases the machine returns, we wrote; the authority we lend it, we grant; the decisions we abandon to it, we choose to abandon. The mirror decides nothing. It returns. The question is never "what will the machine do?" but "what do we choose to place before it, and to believe of its reflection?"
These six markers do not defuse every risk — no method can — but they transform anxious fascination into a lucid posture: to use the tool for what it is, without fearing it as what it is not, nor idolising it as what it may never be.
Dossier : Égrégore & souveraineté cognitive
Accueil · Collection · Journal · Dossiers · Sources