Skip to content
Branemind
All insights
Use casePlatform··6 min read

What to instrument before your first agent goes live

Traces, evals and cost caps are not a phase two. Here is the minimum you need in place on the day you put an agent in front of a customer.

Issues caught pre-customer
83%
Mean time to diagnose
< 10 min
Cost variance
±6%

Measured against a baseline captured before the work started. One engagement, not a forecast for another business. See the methodology note below.

The failure mode of an unobserved agent is not a crash. It is a slow drift into being subtly, expensively wrong, which nobody notices until a customer escalates. By then you cannot reconstruct what happened, because you did not keep the trace.

The minimum viable instrumentation

  • Full step traces: every tool call, retrieval, prompt version and model, with inputs and outputs retained.
  • An offline eval suite that runs on every prompt change, including an adversarial set you keep adding to.
  • Online evals on a sampled slice of live traffic, scored daily by a human for the first month.
  • Per-request cost and latency caps, enforced at the runtime with a defined behaviour when they trip.
  • A replay button. If you cannot re-run last Tuesday's decision exactly, you cannot debug it.
Issues caught pre-customer
83%
Mean time to diagnose
< 10 min
Rollback time
seconds
Cost variance
±6%

Keep reading

All insights
What's new
Product·30 Jun 2026·3 min

BM Watch is generally available

Trace every step, eval continuously, cap the cost. The observability layer we had been running internally is now available to every customer.

BM Watch
Use case
Legal·14 Apr 2026·5 min

Contract intelligence that a lawyer will actually sign off on

Extraction is the easy half. The half that matters is a citation for every clause, a confidence you can act on, and a reviewer queue that gets shorter over time.

Document AI
Use case
Agritech · Energy·11 Feb 2026·4 min

Advisory agents for the field, where the signal drops

An agent that only works on good 4G is a demo. The interesting design constraint is what happens on two bars, in a local language, in the middle of a season.

Voice

Methodology and privacy

  • Clients are described rather than named. Where a quote appears it is attributed to a role and published with approval.
  • Figures come from a single engagement and are not a forecast of what a different business would see. Volume, data quality and process maturity move them more than the technology does.
  • Baselines were captured before the work started. The measurement window and sample for any figure on this page are available on request.
  • Where a number is illustrative rather than measured, it is labelled as such in the text.
If this looks like your loop

Review your path to production

A readiness read on evaluations, controls and ownership.

Review your path to productionOr book a 30-minute call