Why I wrote The Intelligence Illusion—a book on the risks of generative AI (available now)
Get The Intelligence Illusion in PDF and EPUB
I wasn’t planning on writing another book so soon after Out of the Software Crisis. But then “AI” started to happen.
I’m still very much a typical novelty-seeking software developer at heart. I love the new and the shiny, and I’ll try them out just because they’re new and shiny. When something novel takes off, like ChatGPT, my reaction is more likely to be “oh, shiny” than disdain. It doesn’t happen as often as you’d think, these days, because tech is really fond of recycling ideas and a lot of the crap that passes as new these days on its third go-around.
But, ChatGPT and language models? Definitely new, and that caught my interest.
What happens after I get interested is where I and most other people in tech tend to part ways.
I don’t know if it’s undiagnosed neuro-atypicality or practices ingrained during my PhD, but when I’m interested in something I research it. It doesn’t matter if it’s photography, programming, writing, note-taking, or 3D software, my first instinct when trying something new is to dig deep into its context and history.
I read academic papers; dig up books old and new; obsessively scour forums and Reddit for hints as to how people are actually using the thing; listen to interviews; watch old conference talks; analyse studies, experiments, and their methodologies; and generally take it as much apart as I can in order to understand it.
It’s fun! Easily the best part of discovering something new.
AI and language and diffusion models are a particularly deep vein to mine. Some of the issues in the field go back to its very beginning. Some of the papers I dug into were decades old and could have been written yesterday.
Hundreds of papers later and one thing became crystal clear.
Generative AI does not do what they say.
It isn’t the first spark of Artificial Generative Intelligence; it’s horrible for any sort of genuine information or knowledge management work; its image and text generation capabilities are incredibly flawed and obscenely oversold. They are amazing language engines, but currently much more limited than they’re made out to be.
Language models, for example, are an interesting—but, flawed—technology that are unsuitable for the task they’re being sold for.
It’s as if they were selling Austin Minis as great tools for digging drainage ditches. You could ‘pause’ development for a decade—two, even—if you wanted, but nothing would change the fact that a Mini is not good for digging ditches.
The more research I did, the clearer it became that Generative AI is just plain broken for many of tasks we’re expected to use it for. The sheer gall of the tech industry became mind-boggling.
As I wrote in the book’s afterword:
I’ve never before experienced such a disparity between how much potential I see in a technology and how little I trust those who are making it.
That disparity means you simply cannot rely on the statements of those selling these systems about what they are good for. You need to do your own research and discover the many practical business risks of the technology for yourself.
I hope my book will give you the head start you need.
What are the major business risks to avoid with generative AI? How do you avoid having it blow up in your face? Is that even possible?
The Intelligence Illusion is an exhaustively researched guide to the business risks of language and diffusion models.