Two thirds of inbound calls were in a language the existing IVR did not speak. Callers were routed to a queue, waited, and hung up. The support metric that mattered was not CSAT. It was abandonment.
Latency is the whole product
In voice, anything past roughly 500ms of silence reads as the line going dead, and people start talking over the agent. Everything in the design is subordinate to holding p95 under that: streaming ASR, speculative first-token generation, a tool layer that returns partial results, and aggressive caching on the three intents that make up most of the volume.
- Code-mixing handled as the default case, not an edge case, because most real callers switch mid-sentence.
- Barge-in supported everywhere, so a caller can interrupt the way they would interrupt a person.
- A per-turn cost cap that routes to a smaller model when the conversation is clearly routine.
- Full transcripts, latencies and tool traces in BM Watch, sampled for human review daily.
- p95 latency
- 320ms
- Containment
- 63%
- Abandonment
- −54%
- Languages live
- 9
“The first week, callers kept asking which city the agent was sitting in. That was when we knew the latency work had paid off.”