From Scripted Bots to Grounded Agents
For twenty years, “chatbot” meant a decision tree with a friendly face. Today the same interface retrieves from a curated knowledge base, keeps context across sessions, and increasingly takes action in the systems your teams already use. The change is not cosmetic — it is a change in what the software can actually be trusted to do.
The history matters because almost every failure mode people associate with conversational AI comes from an older generation of the technology. If a system answers “I don’t understand” to a lightly reworded question, that is a 2015-era architecture. If it invents a refund policy that does not exist, that is an ungrounded 2022-era architecture. Knowing which generation you are dealing with tells you what to expect before you deploy anything.
We build retrieval-grounded assistants and voice agents, and we spend a lot of time explaining the difference between them and the scripts they replaced. Here is the short version of that conversation.
Four generations, four sets of limits
Each generation solved the previous one’s headline problem and introduced a new one. None of them was useless — but only the last one can be trusted with a workflow.
1966 – 2010
Pattern matching
ELIZA, written by Joseph Weizenbaum in 1966, detected keywords, decomposed the sentence and returned a template. PARRY, a decade later, added an internal state so replies stayed consistent with a persona, but it was still rules, not reasoning. By the 1990s A.L.I.C.E. and the AIML markup made it possible to script thousands of canned responses — deterministic, brittle, and prone to what became known as the ELIZA effect, where users read comprehension into a system that only shuffles symbols.
2011 – 2021
Intent-based assistants
Cloud speech services and intent classifiers changed the unit of work from a keyword to an intent with entities. This is the era of early voice assistants and of dialog platforms where you authored training phrases by hand. It handled phrasing variation far better, and it could pull a balance or a booking through a narrow integration — but every new intent cost developer time, and anything off-script fell back to a blunt escalation.
2022 – 2024
Generative assistants
Large language models made responses fluent and open-ended, and the demo quality jumped overnight. The catch was grounding: a model answering from its own weights has no idea what your return window is. Confident, well-written answers to questions the business had never actually answered became the defining support headache of the period.
2025 – now
Grounded agents
The current generation pairs a model with retrieval, memory and tools. It looks up the answer in sources your business owns, keeps references for the claim, plans a short sequence of steps, calls into the CRM or ticketing system to act, and checks its own output before sending. Analysts expect roughly a third of new software features to ship with agentic capability by 2026 — and the reason is precisely this shift from talking to doing.
What retrieval actually changed
Retrieval-augmented generation sounds like a detail of model plumbing. In practice it moves three things that used to be engineering problems into the knowledge base, where the business can own them.
Currency without retraining. Instead of baking policies into a model, the assistant pulls the relevant passage at the moment of the question. Change the source document and the answer changes on the next conversation — no retraining cycle, no release window. For a support team whose policies move quarterly, that difference alone decides whether the assistant stays accurate past month three.
Answers that can be checked. Because the answer is traceable to a document, the assistant can surface where it got the claim, and an agent reviewing a ticket can verify it in seconds. Grounding does not make hallucination impossible, but it converts an unknown unknown into a visible gap — which is exactly what you want from a system that talks to customers.
Permissions that survive the conversation. Curated internal sources come with access rules already attached. A knowledge assistant answering employee questions can honour HR and IT permissions at the document level, so the same interface gives a new hire onboarding answers and a manager policy detail without leaking either to the wrong person.
Voice agents are the harder version of the same problem
On a phone call, a two-second pause feels like a dropped line. Voice agents inherit everything above — retrieval, memory, tools — and add real-time constraints: turn-taking, handling interruptions mid-sentence, recognition across accents and background noise, and a spoken style that does not read like a document aloud. The payoff is that the caller never left the channel. An order status, a rescheduled appointment or a technician dispatch can happen in one call, with the same grounding rules as the chat assistant and a clean handoff to a person when the request goes beyond the approved sources.
Two teams, two different payoffs
The technology is shared; the operational win looks different depending on who is running it.
For support teams
What changes
- Resolution quality replaces deflection rate as the metric worth optimising
- Scripts stop being maintained by hand; the knowledge base becomes the product
- Escalations arrive with a summary and the retrieval trail, not a raw transcript
- Voice and chat share one answer set instead of drifting apart
For sales teams
What changes
- Qualification happens before a rep spends time on the account
- Handoffs carry the full conversation, the objections raised and the next step
- Multilingual coverage without multiplying headcount
- Follow-up research runs on every inbound lead, not just the enterprise ones
What to insist on before you buy
The generational labels are marketing until you test them. Four questions separate a grounded agent from a well-dressed decision tree:
Where did that answer come from? Ask for the source passage on a live question. If nobody can show it, the system is guessing.
What happens when it does not know? A grounded assistant should say so and route, not improvise. Test it with a question your knowledge base deliberately does not cover.
Can it act, and can you audit that act? Writing to a CRM or issuing a refund is where value and risk both live. You want scoped permissions, a log of every action and an approval step on anything irreversible.
How much of your content needs curating? Grounding is only as good as its sources. Budget for the unglamorous work of cleaning policies and product docs — that effort is what makes the assistant improve over time rather than drift.
The arc from ELIZA to a retrieval-grounded voice agent took sixty years to close, and it closed on an unglamorous insight: the hard part was never making a machine talk. It was giving it something true to say, and somewhere safe to act on it. Teams that treat the knowledge base as the real product — and the chat window as one of many places it surfaces — are the ones getting compounding returns from conversational AI instead of another abandoned pilot.