Baldur Bjarnason

'The LLMentalist Effect', and other links

– Baldur Bjarnason

“The LLMentalist Effect: how chat-based Large Language Models replicate the mechanisms of a psychic’s con” #

I’ve been working on this essay on and off for a few months now. I kept convincing myself that I had to be wrong, this is too dumb to be true, but then the research I did changed my mind again.

I’m now of the opinion that this is how chatbots have managed to convince so many that there’s an intelligence there when there isn’t. More specifically, it looks very likely that the illusion of intelligence is based on the same mechanism as that of a psychic’s con.

Essentially, every question or reply is just a statistically plausible guess and our minds do the work of making it look coherent because of a cognitive bias called subjective validation.

The end result is that we have chatbots with zero actual intelligence—about the same as that of a random number generator—that look to those who want to believe as intelligent agents. Who then get frightened by the “progress” and go on a press tour.

It’s all just a statistical illusion, which I’m calling “The LLMentalist Effect” simply because that’s the sort of silly joke that makes me laugh 😁

But the overall conclusion is not a joke and honestly makes me quite worried about the tech industry’s eager dive into pseudoscience and superstition.

Go read it over on the newsletter site.

The rest #

​