Frequently asked questions about reliability, evaluation and observability

With real cases, edge cases, expected errors, ambiguous inputs and scenarios where it should refuse to act.

It’s the process of measuring whether the system answers, retrieves information or carries out actions with the quality required.

It’s the ability to analyze what happened inside the system: calls, tools, errors, costs, timings and results.

Yes. Before each change, every answer is compared with the correct one, recorded in the test suite. In production, the questions the system couldn’t answer are logged and reviewed every week.

Yes, but checking the error logs isn’t enough. The dangerous failures in an AI system don’t trigger an error: the system gives a wrong answer as if everything were normal.

That’s why every week we run a batch of test conversations on the live system and review the dashboard in Phoenix, a monitoring tool installed on a self-hosted server, along with the questions the agent couldn’t answer. Those three things are also what we deliver to the client.

Yes. That’s why there are limits, a maximum wait time, a set number of retries and an alternative for when none of that works, such as handing the case to a person.

It’s a control that limits or validates the system’s behavior before or after an action. In our systems those limits are enforced by code, not by an instruction to the model.

Yes. An agent should only have the tools it needs for its job.

Yes, if the system keeps enough traceability of states, calls and errors.

No, and we don’t promise it. What we do promise is to find out before the user does when something fails and to record in writing why it failed. In one of our systems we accept a ceiling of close to 92%, because chasing a hundred ends up tuning the tests instead of improving the system.

Yes. Human intervention can be part of the normal workflow.

With what we deliver to the client: the Phoenix dashboard, the results of each weekly golden dataset (test suite) run and the questions the agent couldn’t answer. That is what decides whether to add a glossary, change the prompt or change the architecture. If the client needs it, we also deliver a weekly report.

It’s a test on a set of known cases, each with its correct answer recorded, before a change reaches users. In our projects it’s the golden dataset (test suite), and no change reaches production without passing it.

It’s the measurement of the system’s behavior while it operates with real users and data. In our projects it includes a weekly test run against the system in production.

We don’t do it. We measure against a golden dataset (test suite) with the correct answer recorded, and the checks are done by code, not by another model. In one of our systems a second model reviewed the first and we removed it, because it also made mistakes.

It’s a set of representative cases, taken from real conversations, with the correct answer recorded, used to measure the system.

Yes, and in our systems it’s a rule, not a suggestion. No change to the prompt, model, retrieval, tools or rules reaches production without passing the test suite. Before it’s applied, the database is backed up, and if the service doesn’t respond afterward, it automatically rolls back to the previous version.

It’s a loss of quality in cases that used to work correctly, after changes are introduced.

Yes, if usage is logged per session, user or workflow.

Yes. You can log which functions the agent runs and with what parameters.

Yes. Alerts and metrics make it possible to detect changes in failure rates.

Yes. Evaluations can be run on the same cases to compare accuracy, cost and latency.

It doesn’t reach production. The model is pinned, and a model change passes the tests before it goes out. If something fails when any change is applied, it automatically rolls back to the previous version, because the database is backed up first.

Yes. The behavior of two configurations can be compared on controlled traffic.