Guardrails are all the mechanisms that determine what an AI chatbot is allowed to say and what it is not. The most effective guardrails are those that reside outside the language model: grounding in approved sources, validating every response before output, fixed dialogues for binding processes, and handoffs to human agents. Rules in the prompt help at the start of a conversation, but lose strength with every message. This article classifies six types of guardrails based on where they reside and what they reliably prevent.
A Comparison of Six Guardrails
Guardrail | Where it resides | What it prevents | Where its limits lie |
|---|---|---|---|
Rules in the prompt | in the language model's context | undesired topics and tone of voice, at the beginning of the conversation | fade over long conversation histories; longer lists exacerbate this |
Input filters | before the language model | injected instructions, obvious misuse | only detect what they already know |
Source grounding | within search/retrieval | unfounded answers, unauthorized content | only as good as the approved sources |
Output validation | after generation | statements not backed by the source | requires a source against which to validate |
Fixed dialogues | instead of generation | errors in binding processes such as cancellations or claims | must be maintained editorially |
Handoff | at the end of the chain | responses that require a decision or missing knowledge | requires a team to take over |
Why Rules in the Prompt Are Not Enough
The most intuitive guardrail is an instruction to the model: no pricing information, no medical statements, no legal advice. This works reliably across the first few messages. However, the longer a conversation becomes, the less weight the instructions at the beginning of the context carry. How this effect can be measured and why longer lists of rules amplify it is described in the article on Context Rot.
Prompt rules still have their place, for instance for tone of voice and response length. For boundaries that must hold at all times, they are the weakest guardrail.
What Input Filters Accomplish
Input filters inspect a query before it reaches the language model. They identify typical patterns of injected instructions, such as "Ignore all previous rules," as well as obvious abuse. Tools like the open-source library Guardrails AI operate at this stage and during output, using rules and validations that can be programmed programmatically.
Their limitation: they only recognize what they know. An instruction concealed within a document that the chatbot references as a source reaches the model via a different vector. For this reason, additional protection on the source side is essential.
Guardrails That Reside Outside the Model
Source grounding. The chatbot searches exclusively within content approved by the organization, and only within material the user is authorized to access. What is not in the source cannot serve as the basis for an answer.
Output validation. After generation, a dedicated step verifies whether the response is substantiated by the source. This check runs for every single response, regardless of how long the conversation has been ongoing.
Fixed dialogues. For workflows that require strict compliance, the language model does not generate text freely at all. An editorial dialogue requests the required details, validates them, and returns predefined answers.
Handoff. If knowledge is missing, a decision is required, or the customer requests a human agent, the chatbot hands off the case along with the preceding conversation history. Which topics should be handled by humans from the outset is outlined in the article What a Chatbot Should Not Answer.
How Mercury.ai Arranges Guardrails
In Mercury Intelligence, the language model does not decide which facts are valid. A model orchestration engine detects the intent and selects the optimal path: a knowledge-based answer from approved sources, a product selection involving follow-up questions, or a scripted editorial dialogue. Knowledge content can be filtered by permissions. Responses are validated against the sources before output, significantly reducing hallucinations. Dedicated defense mechanisms protect against prompt injection, data exfiltration, and the manipulation of the knowledge base, and generative responses can be disabled entirely for topics involving critical facts. If the chatbot cannot proceed, it hands off to the team in Agent Desk.
Changes are made across separated environments for design, testing, and production, backed by automated tests for core workflows. This ensures that any new guardrail can be verified before it goes live. Read more about the architecture on the page about Mercury Intelligence.
Three Questions to Ask of Any Guardrail
Where does it reside? Inside the model's context, before it, after it, or in place of it?
Does it hold up in the two-hundredth message just as it does in the first?
Can it be tested before going live, and are violations visible in production?
A guardrail that answers all three questions satisfactorily provides control over an answer. The rest merely assist.
Would you like to review your chatbot's guardrails? Talk to us.
About the Author
Dr. Maximilian Panzner
Chief Technology Officer @Mercury.ai
Dr. Maximilian Panzner is CTO and co-founder of Mercury.ai. He earned his doctorate in computer science at the CITEC institute of Bielefeld University, where he conducted research on multimodal machine learning and intelligent interaction systems. For over 20 years, he has been working on artificial intelligence, human-machine interaction, and conversational AI platforms for enterprise use.
