Your Four Outcomes, Made Measurable
Each one needs a number attached or it stays an opinion
Every measure is paired with a guard
The Full Metric Set
All measured from Stage 0, reported monthly against the same definitions
| Measure | Definition | Good trajectory |
|---|---|---|
| Cycle time | Requirement accepted to UAT-ready, by kind of release | Falls. The headline number, and the one clients feel |
| Time per band | The seven-band split: spec, impact, code, test, review, regression, deploy | Shows where the gain came from, which tells you which agents earned their place |
| Review iterations per PR | Rounds of review before a change is merged | Flat or falling. Rising means drafts look right and are not |
| Drafted lines surviving to merge | How much generated code is used rather than rewritten | Rises as memory improves. Low means a context gap, not a model problem |
| Review agent catch rate | Findings caught before a human opens the PR | Rises. It should get stricter over time as review comments feed back into it |
| Escaped defects | Found after UAT sign-off, per release | Falls. The guard on every speed claim |
| Post-merge defect rate | Agent-touched code vs hand-written code | Equal or better. Worse means generation is not production-ready |
| Suite runtime and coverage | How long the gate takes, and coverage per zone | Coverage up, runtime bounded. A slow gate gets skipped |
| Manual regression hours | Human time spent re-running journeys per release | Falls sharply. This is the clearest QA readiness signal |
| Build to UAT time | Merge to a usable UAT environment, manual steps included | Falls, and becomes consistent rather than variable |
| Bus factor per module | Distinct engineers who shipped in that module in 12 months | Moves from one toward two or three |
| Regulatory lead time | Circular published to UAT-ready | Falls, and becomes predictable. Predictability is what compliance conversations need |
What We Deliberately Do Not Measure
Measuring these produces exactly the behaviour you do not want
Lines of code generated
Rewards verbosity. Creates an incentive to accept bloated drafts. More generated code is not the goal. Less human effort per delivered change is.
Per-engineer agent usage
Measure adoption per person and people adopt visibly while routing around quietly. The rejection log — your best data — stops being honest.
One org-wide coverage target
Spends the same effort on a settings screen as on a calculation engine, and produces tests written to satisfy a number. Coverage is targeted per zone.
Story points per sprint
Estimates inflate to match whatever the tooling makes possible. Cycle time is in days, which the people being measured cannot re-scale.
Where the Return Comes From
Ranked by expected contribution — and Stage 0 will tell us if this ranking is wrong for you
Why there are no percentage claims on this page
We could put improvement figures here. Every consultancy does. They are almost always from a different codebase, a different team and a different definition of the metric. Any number we quoted before measuring your Stage 0 baseline would be a guess dressed as a forecast. What we will commit to is procedural: baselines agreed in writing before we start, the same metrics monthly, and an honest verdict at Stage 4 including “this did not work” if that is what the data says.