The chatbot industrial complex
Every business consultant, every software vendor, every LinkedIn carousel in 2024 said the same thing: add a chatbot. Put it on your website. Let AI handle customer queries. Watch the magic happen.
We've been called in to fix three of these in the past year alone. The chatbots were live. They were confidently wrong. Customers hated them.
What went wrong
The chatbot wasn't the problem. The problem was that nobody asked what the chatbot was supposed to know. It was given a FAQ page and a product description. It was asked to handle returns, technical support, pricing questions, and complaints. It hallucinated policies that didn't exist. It promised refunds the business didn't offer. It apologised for problems that weren't problems.
A language model without grounded, structured, maintained knowledge is a confident guesser. Confidence without accuracy is worse than no answer at all.
A bad chatbot doesn't just fail to help. It actively damages trust.
What to do instead
Before any AI interface, build the knowledge layer. Know what the system needs to know. Define what it's allowed to say. Build escalation paths for what it doesn't know. Test it against real queries from real users before it goes live.
AI that knows its limits is far more valuable than AI that confidently crosses them.