If a harness doesn’t deliver reliable, high-quality, largely autonomous output from an AI agent, we may as well go back to hand-coding and save on tokens and data-centre cooling.
On a recent change to NetPace, a --profile CLI switch added to a published NuGet library, the work touched the public API, removed four interface methods as a breaking change, and introduced a new vocabulary that the project will reuse. Post-implementation rework was small enough to fit in a paragraph. No round-trips. No mid-flight redesigns. No surprises that pushed back the merge.
I used a custom agent harness built on top of spec-kit, an open-source toolchain for spec-driven development, with custom slash commands, skills and templates layered on top. It took six months to develop, test and tune. The whole thing, including the exact command sequence and how to run it yourself, is written up in Agentic Software Development Workflow.
The thesis
The AI’s job is to give me a wider option space at every decision point than I’d generate alone. Six constructor variants where I’d have proposed one. Twelve reviewer questions where I’d have noticed three. Five surprises in the build I couldn’t have predicted. Most of the time I’d reach the right answer anyway. But the option space matters at the edges, and that’s where defects live.
My job is to answer the questions that, from a deterministic process perspective, are arbitrary. The AI cannot tell me whether Profile should be an enum or a record. Whether an override should silently apply or reject. Whether to keep an integration test against Docker or treat it as an anti-pattern. These are judgement calls. There’s no rule that forces them.
A conversation recently gave me the language for this. It’s a mutual accountability loop. The AI is accountable for surfacing options and surprises. I’m accountable for binding decisions on the ambiguous parts. Both sides’ reasoning gets externalised on disk, where a third party (me tomorrow, a reviewer next week, a future maintainer) can inspect what was decided and why.
Without the loop you get one of two failure modes. Either the AI ships output that has to be reworked or distrusted, because no human judgement was applied to the arbitrary calls. Or the human hand-codes the change, because no AI option-generation was usable. Both are slower than running the loop.
The loop at four altitudes
The full walkthrough of this change is documented as a GitHub issue with every artefact preserved, fourteen stages from loose prompt to raised PR: issue #182. Four of those stages show the same loop firing at different scales.
Intent: clarification before drafting
The starting prompt was loose. “Help me ultrathink. I want a --profile switch so the user has an easy way to specify network payload. I’m thinking tiny/small/medium/large.” That would have been a poor brief for a junior developer and it was a poor brief for an agent. (The full clarification exchange is preserved as Stage 2 of the walkthrough)
What followed was a twenty-turn back-and-forth. The agent surfaced decision points I hadn’t named: profile budgets, scope of each profile, default behaviour, override semantics, where the type should live, provider-agnosticism. I rubber-stamped some, redirected others. The agent suggested a fifth Mega tier for inter-data-centre 10Gbps fibre. I accepted, with the qualification that the brittle undocumented payloads be quarantined to Mega only. The agent went through six rejected constructor shapes before we landed on two chained constructors with an inline switch.
Neither side could have produced this design alone. I would not have generated six constructor variants unprompted. The agent could not have decided which of the six fit this codebase’s conventions. The clarification conversation is the loop running at intent altitude.
Specification: the diff between drafts
The first complete spec was around 2,000 words. Enum shape, constructor pattern, profile-value table, CLI surface, acceptance criteria, open questions. Reviewable, but not yet ready to implement. (The draft, the review questions, and the final diff are Stage 6 of the walkthrough)
A reviewer agent, anchored differently, read the draft and surfaced twelve gaps. The change-intent-record (CIR for short, a one-page decision log filed in docs/) path didn’t match the existing repo convention. The size-cap parameter overloads would become a second source of truth after the refactor; what’s the deprecation strategy? Naming collision risk between DownloadSizeMb (a budget cap) and DownloadSizes (a pixel-dimension array). Default value of the cap when the settings record is hand-constructed. Override semantics when --profile tiny --downloadsize 5000 sets a cap orders of magnitude above the profile’s natural transfer total.
I answered all twelve. Nine were “agree with recommendation”: the reviewer had surfaced something already implicit in the draft, and my answer made it explicit. Two were reversals. I rejected the Docker integration test as an anti-pattern. I rejected the silent-apply override semantics in favour of strict profile adherence. One required the draft to be edited.
The diff between first draft and final spec was tiny. Two textual changes in a 2,000-word document. One paragraph rewritten to make a cap-as-backstop semantic explicit. One new Confirmed Decisions block capturing the twelve binding decisions verbatim.
That small diff is the signal of the loop working at specification altitude. The reviewer didn’t restructure the spec because the spec was directionally correct. What the review did was promote twelve decisions from implicit to explicit. Without the review, those decisions would have been made silently mid-implementation. The breaking-overload removal would have surfaced as a PR-review debate. The cap-as-backstop semantic would have surfaced months later as a confused user issue.
Implementation: surprises the plan couldn’t predict
The planning bundle expanded the spec into eleven cross-referenced artefacts and thirty-six numbered tasks. The agent ran /speckit-implement and worked the task list to first compile-clean, test-green. Thirty-two files modified, 2,658 insertions, 2,027 deletions, all 598 tests passing. (The full implementation account, including every surprise below, is Stage 10)
Five things the plan didn’t predict, all surfaced in the build:
Profilecollided withSpectre.Console.Profile. Spectre.Console is the terminal UI library the project uses. The Console project’s globalusing Spectre.Console;shadowed the new enum. A first-attempt project-wide alias broke the twoIAnsiConsoleimplementations that depend onSpectre.Console.Profilefor terminal capabilities.- Record equality on
int[]is reference-equal, not structural. A contract assertingnew OoklaSpeedtestSettings().Equals(new OoklaSpeedtestSettings(Profile.Medium))failed because both sides constructed a fresh array. Tests pivoted to field-by-field equality. - Verify (a snapshot-testing library) auto-launched vim on snapshot mismatch. First test run spawned a vim window per failure and left
.swpfiles behind. Resolution:DiffRunner.Disabled = true. - A pre-existing file had no namespace declaration. The planning contracts assumed the fully-qualified name; adding the namespace closed a latent bug as a side-effect.
- Dependency-injection threading of the built settings. Production wires the settings as a singleton, but the settings depend on parsed CLI args available only inside the command action, after DI was built. Solved with a mutable accessor populated by the action before the service is resolved.
None of these could have been in the spec. The type system, the test harness, the IDE, and the existing code’s quirks teach what they teach only when production code starts landing. The loop fires inside implementation too. The agent surfaces options for each surprise, I select the right fix, the chosen fix gets a one-line comment naming why. Each surprise was closed in quick succession. None emerged in PR review.
Review: five reviewers, anchored
Branch green, PR not yet open. Before a human reads the diff line by line, five specialist agents ran in parallel. Each one read a specific named document from the repo and judged the branch against it. The findings are defensible because the rule is on disk: either violated or not. (The five reviewers, their findings, and the action/defer calls are Stage 11)
- The code reviewer anchored to
CLAUDE.md, the constitution, and project memory. - The test analyser anchored to the
**Scenario:**labels inspec.mdandtest-plan.md. - The silent-failure hunter anchored to catch blocks and fallback paths.
- The type-design analyser anchored to NuGet-public types.
- The comment analyser anchored to XML docs and inline comments.
Each returned a severity-ranked report with file:line citations. Two critical findings, nine important, plus suggestions.
The critical findings were worth the whole stage. The first was a structural bug: default(Profile) == Profile.Tiny, because Tiny was declared first. A consumer doing Profile p = default; or deserialising a missing field would silently get the IoT preset. The type-design analyser caught it by reading the enum declaration. Fix: assign explicit values with Medium = 0, locked in with a regression test. The second was a UTF-8 BOM that had crept into a snapshot file.
Of the nine important findings, I actioned five and deferred four. The deferrals are themselves part of the loop. “Noted, not actioned” is a binding decision with reasoning, not a silent skip. A broad catch (Exception) was deferred because it flagged pre-existing scope, not a regression introduced by this change.
Five parallel agents are cheap to run. The human bottleneck moves from detection to curation. Because each agent is anchored to a documented rule rather than its general taste, deferring a finding is defensible: the rule was either violated or it wasn’t.
The principle
Four stages, one loop. At intent, the AI surfaces design options I wouldn’t generate alone; I make the binding calls. At specification, the AI surfaces ambiguities I’d otherwise leave implicit; I promote them to binding decisions on disk. At implementation, the AI surfaces surprises the spec couldn’t predict; I select fixes with reasoning preserved in the code. At review, the AI fans out across five anchored axes; I curate which findings matter.
The agent harness is the structure that makes each of these legible. The spec, the diff between drafts, the planning bundle, the multi-reviewer punch list: these aren’t documentation. They’re the loop made inspectable. Trust comes from the loop being visible, not from either side being trusted unconditionally. That same legible, on-disk record of who decided what is exactly what I look for when examining the source code behind a disputed project
If your harness ships rework, fix it or abandon it. The point is that the speed has to be real, and real speed only comes from real trust earned through credibility.
Try it yourself
If all of this sounds new and exciting, why not try it yourself?
NetPace is open source, and the whole thing is reproducible. The workflow doc has the exact command sequence and step-by-step setup. Issue #182 has every stage of the change, including the loose prompt that started it. Clone the repo (commit 48053a6), use the same starting prompt, run it through the same pipeline.
Either you reproduce the experience and your agent harness produces the same as mine, or it doesn’t, and the divergence is the interesting thing. Both outcomes are useful. If you find one worth talking about, the issue tracker is the place to go.