Stage 3 — Weeks 7–11

Pilot

One module, real work with a real deadline. Measured against the Stage 0 baseline.

What the Pilot Is For

To find out whether this works on Nerve's code with Nerve's people

Everything before this is preparation. The pilot is the first real test of the claim. It runs on genuine client-requested work with a real commitment attached. A synthetic exercise tests the tooling and not the thing that matters: does the team's day change?

3.1 — Turn Agents On in Order

Not all fourteen at once. That makes failure impossible to attribute.

Five weeks, five steps

WEEK 8 WEEK 9 WEEK 10 WEEK 11 WEEK 12 Analysis only Spec · Architecture · Contract no code generation team works as normal; agents run alongside compare the outputs Test agents Characterization · Test & QA coverage rises before any feature code exists QA logs what it missed Build + review Backend · UI · Data · Review L1 suggest, then L2 draft steward reads every line review comments captured Full path requirement → UAT-ready + Regulatory, Triage, Verification, Release Pack, Knowledge Update every gate exercised measured most closely The two hard tests A. Non-owner change engineer who does not own it B. Restricted-zone change golden datasets in the loop the two claims worth proving Week 7 answers the most important question on its own Does the change map match what the engineers actually touch? If not, nothing downstream is trustworthy and we fix it before going further. EVERY WEEK: 30-MINUTE RETRO → AGENT INSTRUCTIONS UPDATED THE SAME WEEK The rejection log is the highest-value data the pilot produces. It only works if rejecting carries no cost.

3.2 — Measuring Honestly

Side by side, with the failure modes we are actively looking for

What we measureAgainstThe failure it detects
Cycle time, requirement to UAT-readyStage 0 baseline, same kind of changeThe headline claim being wrong
Time per bandThe seven-band split from Stage 0Whether the gain landed where we predicted, or somewhere else
Review iterations per PRBaseline rework rateDrafts that arrive fast and cost more in review than they saved. The most common quiet failure
Drafted lines surviving to merge—How much generated code is used rather than rewritten
Review agent catch rateWhat human reviewers found afterwardsWhether the review agent is earning its place
Defects escaping to UATBaseline escaped-defect rateSpeed bought with quality
Non-owner: where they got stuck— qualitative, cataloguedGaps in the knowledge base, precisely located
Engineer sentiment, weekly and candid—A workflow that looks good in metrics and is disliked will be dropped the moment we leave
The metric we watch hardest is rework, not speed
Faster first drafts are easy. Faster finished changes are the point.
Looks like success
  • First draft lands in 20 minutes instead of 3 hours
  • Everyone reports it feels faster
  • Volume of generated code goes up
?
Is actually success
  • Review iterations stay flat or fall
  • Escaped defects stay flat or fall
  • Cycle time to UAT-ready falls

If review iterations rise enough to cancel the drafting gain, the agent is producing work that looks right and is not. That is worse than not using it, because it consumes senior attention. If we see that, we say so in Stage 4.

3.3 — The Pilot Working Agreement

Written before week eight, so nobody improvises the rules under pressure

  • Every agent output is reviewed by a named human before it goes anywhere. No exceptions during the pilot
  • Any engineer may reject any agent output without justifying it. Forced adoption produces compliance, not evidence
  • Rejections get logged with a one-line reason. This is the best data the pilot produces
  • Restricted-zone changes go through the Domain Owner even for a one-line diff
  • If the pipeline is slowing real delivery in a given week, drop back to the old way for that work and record why
  • Weekly 30-minute retro. Agent instructions updated the same week
  • Either party can stop the pilot at any point. No penalty, no argument

Why the right to reject matters

The fastest way to get meaningless pilot data is to make people feel they are being assessed on how much they use the tool. They will use it, say it is fine, and quietly work around it. The rejection log only works if rejecting costs nothing — visibly, from week one.

Stage 3 deliverables

One complete change end to end through the fleet · one non-owner change with a list of what memory was missing · one restricted-zone change through the golden datasets · full metric set against the baseline · agent instructions updated from four retros · the rejection log.