When we look at why a service chatbot disappoints, the root cause is rarely the language model. It lies in an assumption that no one articulates during development: that the human on the other end will behave exactly as the dialogue design intends. They read the greeting. They select the correct category. They formulate their question just as the system expects it. In real interactions, they do none of these things.
This is not a marginal observation. As early as 2008, the Nielsen Norman Group measured across thousands of page views that on the web, people read an average of about 20 to 28 percent of a text. They scan, they skip, they search for the one clue that will move them forward. A chatbot that opens with a paragraph of explanations and options talks past this behavior. The user skims past the message and types their question, regardless of what was written above.
A chatbot is measured by what actually gets through to the user.
We see three misconceptions that can be found in almost every weak service chatbot. All three originate at the desk and during the conceptual design of the chatbot.
The mistake begins in the very first second
Many bots greet users with an instruction manual. What they can do, what they are not responsible for, how best to operate them. Well-intentioned, because it is supposed to manage expectations. Except nobody reads it. The user came with a question and they ask it immediately.
The same pattern repeats with every pre-selection. A bot that first asks for the location before answering loses those users who skip this selection. If they do not select the branch, they get a general response, even though the correct local answer is in the system. In one municipality, we observed exactly this: the location query was at the beginning, most callers bypassed it, and the generic answer left them puzzled. The bot had the correct information. It just tied it to a condition that the human did not fulfill.
A chatbot must be designed with scanning in mind. The first message must be short, the actual query first, and any pre-query only when it naturally arises from the conversation.
Customers speak in symptoms, chatbots listen in catalog language
The second misconception lies in the language. A customer describes their problem as they experience it. In mechanical engineering, they say: "The strip feed jams when starting up the machine." What they mean is documented under roller contact pressure, strip thickness, and torque referencing. Between what the human says and what the database knows, there is a translation gap.
Most chatbots do not bridge this gap. They search with the user's words in documents written in technical jargon, find no answer, and formulate an evasive response. The user feels misunderstood, and the data backs them up. In a Gartner survey of 5,728 customers, 43 percent stated they could not find content relevant to their issue in self-service, and 45 percent felt that the company did not understand their issue.
The task of a service chatbot is translation—from the customer's everyday language into the structure of corporate knowledge.
This translation is work that happens before the answer. A system must recognize the intent behind the phrasing and map it to the terms under which the knowledge is stored. Where this is missing, even the best product catalog remains silent.
A bot that keeps users busy for a long time is not yet a bot that helps
The third misconception lies in the metric. Many projects optimize for the containment rate—the share of conversations that the bot closes without handing over to a human. Mature systems reach 55 to 65 percent here. The number looks good, but hides what is happening beneath. The actual resolution rate, measured by satisfied customers with resolved queries, is closer to 25 to 40 percent according to industry benchmarks.
The gap between these two numbers is where trust is lost. A bot that always gives some kind of answer keeps the user in the channel but still sends them away unresolved. What follows is clearly demonstrated by a recent Gartner survey of 3,566 customers: for their service queries, people turn to general AI tools like ChatGPT about three times more often than to the provider's chatbot. Their own bot disappointed them once, and next time they ask elsewhere.
In this calculation, an honest handoff to a human seems like a flaw because it lowers the containment rate. Yet, it is the very reason why the customer stays. A bot that says "I cannot answer that, let me connect you" retains the customer, while a bot with a fabricated answer loses them.
What a service chatbot needs, designed with the user in mind
A design principle follows from these three misconceptions. A service chatbot becomes good when it takes the behavior of real humans as its foundation.
It opens short and sweet, but brings the user's request to the forefront. It translates between everyday and technical language instead of equating them. It anchors its answers to verified knowledge, so that it prefers to admit its limits rather than guess. And it measures its success by resolved queries, with a handoff to a human treated as an orderly exit and not as a defeat.
At Mercury.ai, we separate the tasks for this purpose. An orchestra of specialized models first recognizes the intent, finds and evaluates the matching source, and verifies it before any phrasing is formulated. The language model only verbalizes at the very end; it does not decide on the facts. This keeps the response tied to the company's knowledge, and the risk of fabricated answers drops significantly. How this source-bound architecture works in detail is described in the article on hallucinations in AI chatbots and on the Knowledge Hub page. For the moment when a human is needed to take over, the Agent Desk is there to assist.
The chatbot's second message must be earned
A service chatbot gets exactly one try with every user. The first response decides whether there will be a second sentence, or whether the human closes the window and asks elsewhere next time. This decision is not made in the language model. It is made in the question of whether someone took the behavior of real users seriously during development.
The technology for good service chatbots is available. What decides between success and disappointment is the willingness to design the bot starting from the human rather than the dialogue tree.
Want to see how a service chatbot is built with the user in mind? Speak with us or check out the customer service solution.
About the Author: Dr. Hendrik ter Horst is Chief Product Officer at Mercury.ai. He holds a PhD in Computer Science from the CITEC Institute at Bielefeld University, specializing in machine information extraction and data processing. For ten years, he has been working on dialogue-based AI systems, managing the design, implementation, and validation of AI-supported applications from initial concept to productive use.
Sources
Nielsen, J. (2008): How Little Do Users Read? Nielsen Norman Group. Evaluation of 45,237 page views, users read an average of 20 to 28 percent of a text.
Gartner (2024): Customer Service Self-Service Survey, survey of 5,728 customers. 14 percent of queries fully resolved in self-service, 43 percent find no relevant content, 45 percent do not feel understood.
Gartner (2026): Survey of 3,566 B2B and B2C customers. Customers use general AI tools about three times more often than the provider's chatbot for service queries.
