Talks like a human, must think like one

A biological virus is neither alive, nor sentient, it is inert by itself. Its chemical properties make it compatible with interfaces of cells and their subsystems, so it can plug into these processes and alter their course. Once a virus hijacks a cell, the cell may stop caring for itself and throws its resources at the virion reproduction, at its own expense.

A virus has no will or intentions, it cannot be evil. It just happens to mess with the natural processes in a living organism, because of how our interfaces work. Viruses also may play a positive role, speeding up evolution, and they kill bacteria – their impact on our life is complex and not necessarily a net negative.

We, humans, communicate verbally and non-verbally, but words are the cornerstone of the cultural part of our interfaces. Since the Internet has matured, a modern man gets to know his peers largely through text. You lot recognize and know me as a stream of text too.

Recently, this interface is being exploited, similarly to how a virus attaches to a cell. We are talking daily to LLM-based chatbots, just as mindless, unintentional and inert as a virus. Also, not malevolent.

Modern chatbots have become such convincing talkers that our minds start mistaking them for other living beings, anthropomorphizing them. Like a charismatic public speaker, they project a promise of competence, even if the speech, upon careful examination, turns out to be a human slop. As machines, they promise automation, efficiency, and scalability, as long as we do not see the unsubsidized bill for the compute. As a whole, this seems to be a solution to all our problems.

Since the times of primitive early chatbots like ELIZA (in 1966!), people have been reported to talk with them as if they were humans, expecting human qualities like empathy, treating them with politeness, making up their character traits. So, the phenomenon is not new, but the scale is. They did not make enough of a splash to warrant naming something "an ELIZA psychosis".

Now we charm ourselves, because a promise of a smooth talking demigod genius in a box has become too real and too attractive. Is there a better competitive advantage than a helping hand of the machine spirit? I'd speculate we are hardwired for that. A standard template constructor in everyone's pocket, for 200$ a month, please.

So is it surprising that we are throwing a large chunk of our resources into evolving AI, into constructing more datacenters to host AI, into feeding more compute and memory to AI – in short, on replicating and spreading it, while thinking it only serves our own best interests?