What happens after an AI chatbot or agent is delivered

Once an AI chatbot is delivered, its maintenance begins: we watch it through traces and alerts, measure it every week against a golden dataset of real conversations and adjust it with the questions it couldn’t answer. It’s needed because your customers’ questions change, your rules change and the model provider ships new versions.

The question from whoever is buying is almost always the same: what if it gets something wrong in front of my customers? It’s a fair question. A language model can fail, and nobody serious should promise you otherwise. Here’s what we do to catch it in time and what your company gets.

How we evaluate a chatbot: the golden dataset

To know whether a chatbot is still working well, you need a reference. That reference is a golden dataset: real conversations from your business, each one with the correct answer written down and checked by a person. Ideally that person is someone on your team who knows the business.

The cases that work come from real conversations. We learned that with our own product. We wrote the first test cases by hand and they were useless, because they were never checked against the real system.

You also need a clear bar: how many of the cases the chatbot has to get right overall and for each type of request. In our appointment assistant, for example, it has to understand what the patient is asking for in at least 85% of all cases and in 70% of each type.

How we see what the chatbot does every day

Every time the chatbot works it leaves a trace in Arize Phoenix: what came in, what it did step by step, what it cost, how long it took and which model actually answered. Phoenix is a tool from Arize for seeing inside what an AI agent does. We use it on the chatbots and agents we maintain.

The traces don’t leave for a third party’s cloud. On custom projects, Phoenix is installed on your own server, as we explain in our guide to GDPR-compliant AI.

The traces show what the chatbot costs and whether something is slowing down. They also show whether the provider answered with a fallback model without telling anyone. In one of our chatbots, a check looks for exactly that and hasn’t found a single fallback model in 29,041 calls.

Tests before every change and a weekly review

The chatbot runs its tests again at two moments: before every change and once a week. Every change is a risk, however small it looks.

On every project, no change reaches production unless it passes the test suite. The database is backed up before the change is applied and, if the service stops responding afterwards, the system goes back to the previous version automatically. The suite grows over time. Our appointment assistant has nearly 5,000 automated tests by now.

On top of that, once a week the golden dataset runs against the real model. Those tests cost money on every run, so they don’t run around the clock. They run every week and whenever something important changes, and their results stay in Phoenix so one week can be compared with the next.

What we do with the questions it can’t answer

The questions the chatbot couldn’t answer are the best to-do list there is. We go through them with you and decide what each one needs: adding a term to the glossary, changing the model’s instructions (the prompt) or changing the chatbot’s design. Once it’s solved, the question joins the golden dataset and gets checked every week.

Mistakes go the same way. In our appointment assistant, if someone asked for an appointment on “January 11” in the middle of June, the chatbot read it as a January that had already passed. Since we caught it, the year is worked out by the code and not by the model, and that case is checked again with every change.

Alerts for the failures nobody sees

We also check that the alerts themselves work, because a failure that raises no alarm can go days without anyone noticing. Our appointment assistant, for example, has around a hundred checks inside its own code that fire as soon as something goes off script. One of them told us that appointments created by hand from the dashboard weren’t updating the customer record, and we added a test that covers it.

Alerts fail too. Once, an alert that fired too often used up the notification service’s quota, and 13 appointment reminders were lost without any warning. The clinic told us. Since then every alert has a daily cap, and a test checks that the cap holds.

When a new model comes out: switch or not?

A new or cheaper model doesn’t get in just because. First it runs the same golden dataset, and in Phoenix we compare the two versions case by case. If the new one gets fewer right, we don’t switch, even if it’s cheaper.

The model version is also pinned. Updating it is our decision and it goes through the tests like any other change, so a new version from the provider doesn’t reach production on its own.

What your company gets every week

Everything we see, you see too. We hand you:

• The Phoenix dashboard, with the traces, the spend and the comparison between versions.
• The results of every weekly run of the golden dataset.
• The questions the chatbot couldn’t answer, so we can decide together what to adjust.
• A weekly report that sums it up, if you need it.

On custom projects, the code and the infrastructure are in your name from day one. Maintenance is a monthly fee with no minimum term, and the prices are in our guide to how much an AI agent costs.

You won’t get a chatbot that never makes a mistake, because that doesn’t exist. You’ll get one that’s measured every week, tested before every change and improved with what it couldn’t answer.