Every loan file arrived as a pile: a bureau pull, a KYC packet, six months of bank statements, and a repayment history in a format nobody had modernised since 2016. An analyst spent between two and three days per file reconciling them by hand. The queue only ever grew.
The temptation with a problem like this is to build a scoring model and call it done. We didn't, for one reason: the team is regulated, and a number without a defensible reason behind it is worse than no number at all. What they needed was not a score. It was an argument.
What we built
Ingest and normalise
Each source lands in a typed record. Bank statements are parsed into cash-flow primitives, bureau data into obligations and delinquency events. Nothing reaches the model as raw PDF text.
Ground every claim
The agent retrieves against the normalised record with hybrid search plus a reranker. Every statement it makes carries the source row it came from, which is what makes the output auditable.
Argue, then tier
A planner-executor loop drafts the risk narrative, challenges its own weakest assumption, and only then proposes a tier and a limit. The disagreement step is where most of the accuracy came from.
Hand to a human
Anything outside the confidence band, or touching a policy exception, routes to an underwriter with the reasoning attached. They approve, adjust or reject in one screen.
What moved
- Median decision time
- 38s
- from 2.4 days
- Files auto-tiered
- 71%
- Analyst hours returned
- 310/mo
- Override rate
- 4.1%
The number the team cared about most was not speed. It was the override rate: how often an underwriter looked at the agent's reasoning and disagreed. It started at 19% in week one. Four eval cycles later it sat at 4.1%, and the remaining disagreements were genuinely interesting edge cases rather than sloppiness.
What we'd do again
- Normalise before you retrieve. Most 'the model hallucinated' bugs were actually parsing bugs.
- Make the agent argue against itself before it commits. It is the cheapest accuracy you will ever buy.
- Ship the override screen in week one. The disagreement log is your eval set.
- Instrument cost per decision from day one, not after finance asks.
“It didn't replace the underwriters. It gave each of them a first draft that was already 90% right, and gave us a record of why.”