When we look at why a service chatbot disappoints, the cause is rarely the language model. It lies in an assumption that no one voices during construction: that the human on the other end behaves as the dialogue design intends. They read the greeting. They select the correct category. They formulate their question exactly as the system expects. In real-life interactions, they do none of these things.
This is not a marginal observation. As early as 2008, the Nielsen Norman Group measured across thousands of page views that people on the web read an average of only 20 to 28 percent of a text. They scan, they skip, they look for that one clue that gets them further. A chatbot that opens with a paragraph of explanations and options talks past this behavior. The user overlooks the message and types their question, regardless of what was written above.
A chatbot is measured by what actually registers with the user.
We see three misconceptions that can be found in almost every weak service chatbot. All three originate at the desk and during the conception phase of the chatbot.
The mistake begins in the first second
Many bots greet users with an instruction manual. What they can do, what they are not responsible for, how best to operate them. Well-intentioned, because it is meant to manage expectations. Except nobody reads it. The user came with a question and they ask it immediately.
The same pattern repeats with every preliminary gate. A bot that first asks for the location before answering loses those users who skip this selection. Those who do not select the branch receive a general response, even though the matching local answer is in the system. We observed exactly this in a municipality: the location query was placed at the beginning, most callers bypassed it, and the generic answer left them feeling helpless. The bot had the correct information. It just made it conditional on a requirement that the human did not fulfill.
A chatbot must be designed with scanning in mind. The first message belongs brief, the actual concern first, and any preliminary query only when it naturally arises from the conversation.
Customers speak in symptoms, chatbots listen in catalog language
The second misconception lies in the language. A customer describes their problem as they experience it. In mechanical engineering, they say: "The strip feed is jamming when the machine starts up." What they actually mean is documented under roller contact pressure, strip thickness, and torque referencing. Between what the human says and what the database knows, there is a translation gap.
Most chatbots do not bridge this gap. They search with the user's words in documents written in technical jargon, find no answer, and formulate an evasive response. The user feels misunderstood, and the numbers back them up. In a Gartner survey of 5,728 customers, 43 percent stated that they could not find content relevant to their issue in self-service, and 45 percent felt the company did not understand their concern.
The task of a service chatbot is translation, from the customer's everyday language into the structure of company knowledge.
This translation is work that happens before the response. A system must recognize the intent behind the phrasing and map it to the terms under which the knowledge is stored. Where this is missing, even the best product catalog remains silent.
A bot that keeps users engaged for a long time is not yet a bot that helps
The third misconception lies in the metric. Many projects optimize for the containment rate, i.e., the share of conversations that the bot resolves without handing over to a human. Mature systems reach 55 to 65 percent here. The number looks good and hides what is happening beneath. The actual resolution rate, measured by satisfied customers with resolved issues, is closer to 25 to 40 percent according to industry benchmarks.
The gap between these two numbers is where trust is lost. A bot that always gives some kind of answer keeps the user in the channel yet sends them away unresolved. What follows is clearly demonstrated by a recent Gartner survey of 3,566 customers: people use general AI tools like ChatGPT for their service inquiries about three times more often than the provider's chatbot. Their own bot disappointed them once, and next time they ask elsewhere.
An honest handover to a human looks like a flaw in this calculation because it lowers the containment rate. Yet, it is the very reason why the customer stays. A bot that says "I cannot answer that, I will connect you" retains the customer, while a bot with a fabricated response loses them.
What a service chatbot needs when designed from the user's perspective
A design principle follows from these three misconceptions. A service chatbot becomes good when it takes the behavior of real humans as its foundation.
It opens short and crisp, but brings the user's concern to the forefront. It translates between everyday and technical language instead of equating them. It binds its answers to verified knowledge so that it would rather admit its limit than guess. And it measures its success by resolved issues, with a handover to a human seen as an orderly exit and not as a defeat.
At Mercury.ai, we separate the tasks for this purpose. An orchestra of specialized models first identifies the intent, finds and evaluates the matching source, and verifies it before any phrasing is even formulated. The language model only verbalizes at the end; it does not decide on the facts. This keeps the information bound to the company's knowledge, and the risk of hallucinated answers drops significantly. How this source-bound architecture works in detail can be read in the article on hallucinations in AI chatbots and on the Knowledge Hub page. For the moment when a human needs to take over, the Agent Desk handles the transition.
The chatbot's second message must be earned
A service chatbot gets exactly one attempt with every user. The first response decides whether there is a second sentence or whether the human closes the window and asks elsewhere next time. This decision is not made in the language model. It is made in the question of whether someone took the behavior of real users seriously during construction.
The technology for good service chatbots is available. What decides success or disappointment is the willingness to think of the bot from the human perspective instead of the dialogue tree.
Want to see how a service chatbot is built from the user's perspective? Talk to us or take a look at the customer service solution.
About the author: Dr. Hendrik ter Horst is Chief Product Officer at Mercury.ai. He holds a PhD in Computer Science from the CITEC Institute at Bielefeld University, with a focus on machine information extraction and data processing. For ten years, he has been working on dialogue-based AI systems, responsible for the conception, implementation, and validation of AI-supported applications from initial idea to productive use.
Sources
Nielsen, J. (2008): How Little Do Users Read? Nielsen Norman Group. Evaluation of 45,237 page views, users read an average of 20 to 28 percent of a text.
Gartner (2024): Customer Service Self-Service Survey, survey of 5,728 customers. 14 percent of issues fully resolved in self-service, 43 percent find no matching content, 45 percent do not feel understood.
Gartner (2026): Survey of 3,566 B2B and B2C customers. Customers use general AI tools about three times more often than the provider's chatbot for service inquiries.
