Mergepath

The operator was the review process

I was running Claude Code, Codex, and Cursor across a growing fleet of repositories. I had one rule I needed them to follow: every change goes through a pull request. I put it in the instruction files they read. An agent would still push straight to main, usually for a change small enough not to feel like it counted.

Then I became the review process. I read diffs after the fact, relayed feedback between sessions, and vetted output nobody had inspected. The agents produced more; my confidence did not keep up. That gap is the product problem.

I told the origin story in Agent Approval Workflow and the Genesis of Mergepath. This page covers the decisions I made after the problem had a name, and what each has cost. The record here runs through 2026-10-06; the dated outcomes below belong to that snapshot.

The authorities are deliberately separate

I split the work into six roles. As the operator, I own every repository, handle the settings automation cannot reach, and receive escalations. An authoring agent writes a change and opens its pull request under one shared author identity. What lands carries that name rather than the agent’s.

Two gates stand between the pull request and its merge. The reviewer agent behind the second gate has its own account, so approving and authoring cannot be the same hand. The CI service account runs checks and may never hold an approval. GitHub counts an approval from any account with write access; it cannot tell a robot from a person.

I also serve as the human tiebreaker. That is a narrower job than operating the fleet. A finding re-flagged after a rebuttal, or a round budget spent on findings that keep blocking, escalates automatically. Nobody has to decide whether the disagreement is worth raising. The diagram preserves the roles as they stood on its caption date; the later round-budget changes are in the decision below.

An authoring agent opens a pull request under one shared human identity, so what lands carries a single name. Two machine gates stand between that pull request and the merge: branch rules with CI, which fail closed only where a repository actually lists them as required, and a reviewer agent on its own account that can approve but never merge. The CI service account runs those checks and may never hold an approval at all. A finding re-flagged after a rebuttal, or a round counter past its budget, routes to the human tiebreaker rather than back into the automation. Only the author identity merges.

The decisions

I could describe Mergepath by counting checks: seven fail-closed CI checks in April 2026, twenty-seven by mid-May, seventy-two as of 2026-08-28, with seventy-one wired into the lint workflow across eighty invocations. That inventory says little about judgment. Most of those mechanisms exist because something failed while the fleet was live.

One principle runs through most of them: clearance belongs to the exact commit reviewed. A push that changes the content voids it. These are five decisions I could reasonably have made differently.

1. EnforcementMixed

   Enforce at boundaries that can refuse

   What I encountered

   I put the no-direct-push rule in the instruction files the agents read. They followed it when it was convenient and skipped it when it wasn't. Every lapse landed on me.

   Over

   I could have written the rule more forcefully, in more files, more often. That was free and an obvious first move. I had already given it every chance to work.

   What I decided

   I moved enforcement into places that could refuse: a server-side branch rule that stops a push, a local hook that refuses a malformed pull request before the API call, and CI that fails closed. I documented what each layer binds, including where it offers no coverage.

   Why

   A written rule has to be remembered when following it is least convenient. A refusal does not. I found agents, like people, more reliable when a mechanism enforced the rule.

   Cost

   A one-line fix still needs a pull request, self-review, an identity switch, review, and gates. I pay the same overhead for it as for a rewrite. A gate can refuse a formality as readily as a real defect.

   What it changed

   Four months later, a pull request was refused because its body put the required header in bold. The local and CI layers held. The fleet's own audit found the larger hole: most consumers weren't requiring the checks that ran on them. The server layer, which could bind everyone, was largely unconfigured.

2. IndependenceMixed

   Make independence an account, not a prompt

   What I encountered

   Asking an agent to review its own work in the same conversation kept running into the same blind spots. An assumption that had felt right while writing the code still felt right while reviewing it.

   Over

   I could have kept a review checklist in the same session. That would have saved three GitHub accounts and their credential plumbing. Even crude same-chat review finds real bugs.

   What I decided

   I gave every agent one shared human author identity and its own machine reviewer account. No pull request can be approved by the account that authored it, and every review has a byline I can audit.

   Why

   Across three agent platforms, I kept seeing the separate reviewer identity find things the authoring session had missed. Separate accounts made the identity rule mechanical. It no longer depended on an agent remembering to switch roles.

   Cost

   I now had one GitHub account per agent, each with a token, signing key, and credential plumbing. The plumbing broke. In August 2026, a 1Password item repurposed from one identity to another silently sent three agent reviews out under the CI robot's byline on this site's repository, `nathanpaynedotcom#668`.

   What it changed

   The reviewer-identity effect is a repeated observation across three platforms. I have no comparison arm or defect ledger, and I cannot explain the mechanism. The byline failure did lead to a concrete change: a per-repo deny-list of accounts that may never hold an approval.

3. PropagationMixed

   Mirror byte-for-byte, and declare every divergence

   What I encountered

   I had one standard and seven consumer repositories. Each repo's copy could drift until a weekly audit caught it. One operating-rules file had already become nine mutually incompatible copies across the fleet.

   Over

   I could have maintained each copy by hand, or shipped a shared dependency for consumers to install. A package would version cleanly. It would also let a consumer pin an old version and quietly stop receiving the standard.

   What I decided

   I gave propagation three declared modes: canonical paths mirrored byte-for-byte (137 as of 2026-08-28), kit directories that require the template files but allow local additions (five), and templated paths rendered from each consumer's declared facts (two). Any intentional divergence belongs in that repo's registry, with a written reason.

   Why

   A standard that seven repositories apply differently is seven standards. One canonical copy made drift a diff I could review. A registry gave intentional divergence a name and a reason.

   Cost

   The mechanism I built to stop drift caused it. On 2026-08-28, a propagation wave copied the hub's weaker pull-request gate over the stronger one this site's repository had built. Only a consumer test kept outside the mirror caught the loss. The hub could not see the consumer's own hardening.

   What it changed

   `mergepath#1132` was opened and fixed the same UTC day. Getting consistent copies was easier than making room for intentional differences. As of 2026-10-06, all seven consumer registries are empty: six have no override file, and the seventh declares only a schema version. Consistency still rests on nobody having diverged, rather than on a record of the differences.

4. ConvergenceRevised

   Route non-convergence to a person, not to more automation

   What I encountered

   I had set the external-review budget at two rounds, assuming a review would converge in a round or so. The automated external leg could post an approval on its own.

   Over

   I could have kept the budget at two, or given a runaway review to the automated external reviewer for a fresh verdict. Changing reviewers was a defensible response to a stuck loop.

   What I decided

   I raised the budget to ten and made Codex the primary reviewer on every pull request. CodeRabbit would review changes the classifier marked complex, plus anything it could not read. Reviews that did not converge would come to me as the human tiebreaker, rather than go to the automated leg.

   Why

   The automated leg can approve. Sending a disagreement there could let a change merge without anyone settling it. Settling that disagreement was my job as the human tiebreaker.

   Cost

   A review could then run to ten rounds before I saw it. The budget rose to twenty on 2026-09-28. Since 2026-10-01, spending it alone no longer summons a person; continuing disagreement does. My read is that a review still running at round ten is evidence that the pull request was too broad. The budget does not say that.

   What it changed

   One pull request drew four review rounds. Rounds two and three each found a real defect introduced by the previous round's fix. Escalating at round three would have stopped a review that was still finding problems. That was why I moved the budget from two to ten.

5. EvaluationPending

   Replay the policy before you trust it

   What I encountered

   A review threshold or protected path changes how pull requests are routed. Editing the configuration meant learning its effects one pull request at a time, after the policy was already live.

   Over

   I could have edited the YAML and waited for the next pull request. That works, but I would learn each mistake in production.

   What I decided

   I built a single-file page, shown in the screenshot above, that replays real pull requests against a draft policy. I can see what a threshold or path pattern would route before committing the YAML.

   Why

   A policy file is a product decision wearing configuration syntax. I wanted to see what a rule did to real traffic before shipping it. That instinct came straight from product work.

   Cost

   The page can tell me which pull requests would escalate under a draft policy. It cannot tell me whether the resulting review is any good. It simulates the routing, not the reviewers.

   Validation boundary

   As of 2026-10-06, no shipped policy change traces to a playground run. The documented retunes came from measured review latency and review-round experience. I would need one configuration change made because a replay showed the old routing was wrong before I could claim the playground had changed a decision.

mergepath#1080 was the pull request that led me to raise the round budget. Four rounds were requested between 06:15 and 07:12, and it merged at 07:53. The diagram shows only the two rounds that returned findings. Each found at least one defect in code the previous round’s fix had just introduced.

A change pushed at 06:12 on 2026-08-22 drew four requested review rounds before it merged an hour and a half later. This shows the two that returned findings; the other two returned none on the record and are omitted. The review at 06:46 carried two blocking findings, one of them on the 06:32 fix; the one at 07:04 carried one blocking finding on the original change and a lesser one on the 06:53 fix—so each round found a defect the previous fix had introduced, and a fix followed each before the pull request merged at 07:53.

What the record shows, and what it does not

I can measure adoption. I cannot measure impact. Keeping those separate limits what I can honestly claim.

• 8 reposthe hub plus seven consumers, as of 2026-10-06
• 609 PRsmerged on the hub by 2026-10-06, not proof each ran the loop
• 0 forkspublic and forkable since at least 2026-04-16
• No baselineI never timed the manual path, so I cannot claim time saved

The fleet reached eight repositories by shrinking. One consumer was archived and retired from the manifest in August 2026; another went dormant and was dropped in September. The July 2026 stress test produced 134 review-finding threads across a 24-hour, eleven-PR record, which I examined in A Perfect Score on the Wrong Axis.

The sharpest result cut against the hub. Propagation sent files the template’s own review had already cleared into two unfamiliar codebases. Seven external-review rounds per pull request kept finding bugs in them. Review found the least where the files were most familiar.

The propagation audit also reports its own state. On 2026-08-28, a read-only run reported the same eighteen manifest paths adrift on all eight consumers then enrolled. That identical set was the clue: every consumer had been synced to the hub’s main that afternoon. The eighteen paths were the previous evening’s merges that my local hub checkout had not pulled. A directory entry such as scripts/ci/ counts as one path, however many files inside it moved:

$ scripts/sync-to-downstream.sh --audit
nathanpaynedotcom (nathanjohnpayne/nathanpaynedotcom)
  baseline: main@eafa46b
  ✗ .github/workflows/agent-review.yml         drift: 1751 diff line(s)
  ✗ tests/test_approval_merge_continuation.sh  drift: 996 diff line(s)
  ✗ scripts/merge-clearance-gate.sh            drift: 78 diff line(s)
  ✗ scripts/ci/                                drift: 3 file(s) drifted
  … 14 more paths, and the same eighteen on all seven other consumers

The audit still gave me a diff I could review, whichever side of the comparison had moved. A bulk sync can close a gap, and several in this repository’s history have. This particular snapshot caught my stale checkout.

I have no manual baseline for what the gates save. I never timed the work before adding them, so there is nothing to subtract from. The repository’s one savings figure multiplies auto-approved pull requests by a hardcoded thirty-to-180-minute constant nobody measured. The two real measurement systems both measure cost: external-review latency and the latency budget the CI suite has to stay inside.

The operator gets a console

Until October 2026, I assembled the fleet’s picture by hand. I used gh pr checks for merge gates, gh run view for failing steps, the Phase 4b ledgers for review-agent spend, and the sync audit above for drift. One command at a time, per repository.

Mergepath Cockpit puts that picture on one live page. It is a local server written with the Python standard library and no third-party packages. It shells out to the hub’s installed yq, jq, gh and git, binds to the loopback address, polls GitHub every fifteen seconds to two minutes, and pushes changes to the page over server-sent events.

The resident server holds only the cached reviewer token; the author token is stripped from its environment. It can watch the fleet and cannot merge into it. Its one write is a fleet sync, which starts as a preview I read and then confirm.

I held the dashboard to the same discipline as the gates. Every panel starts unavailable. A failed read keeps the last good observation and shows its age. Partial coverage cannot be reported as a clear fleet.

One gap remains in this record. The account billing endpoint returned 404 to the reviewer credential, so Actions spend reads unavailable. I do not know whether the token or the endpoint caused it. The epic opened on 2026-10-02 and closed on 2026-10-04. By this page’s snapshot, nobody had run the fleet through the console long enough to say what it changed.

The premise the fleet did not meet

This is the finding I would want a reader to see. A GitHub job can gate a merge only when branch protection lists it as a required check. Otherwise it can run, go red, and watch the pull request merge anyway.

On 2026-07-28, the fleet’s own audit found eight of the ten repositories then enrolled enforcing zero of five canonical checks. Three had no protection on main at all. The hub itself enforced two.

The gap subsequently closed. The five checks have been required on every repository the weekly audit could read since at least 2026-08-19. As of this page’s snapshot, its one remaining finding was a single consumer’s admin bypass under the 2026-09-06 staged decision.

Until that audit, red CI could look like enforcement while acting as advice. I had built a standard around enforcement at boundaries that could refuse, and most of the fleet wasn’t requiring its checks.

1. Expected

   I expected closing the administrator escape hatch to harden the fleet. A gate I can walk around as an administrator is a speed bump. The hub's enforcement record already showed merges reaching `main` without the approval the policy required.

   Observed

   With that escape closed on the hub, `mergepath#1121` sat blocked on 2026-08-27 Pacific time. All thirty-six review threads were resolved, feedback accounting was clear, and a cross-agent approval was in place. A rate limit made a gate exit with a usage error instead of a verdict. Its own log read `unresolved: 0`. Nobody had the authority to act on what it said.

   Response

   On 2026-08-28 UTC, I declined that hardening and disabled admin enforcement on the hub. That made the fleet uniform at the time. I judged a deadlock with no recourse worse than an auditable bypass, and the audit stopped reporting the bypass as drift that day. On 2026-09-06, I reversed the consumer half of that decision: admin enforcement became the target for every consumer, earned per repository by a passing canary. As of 2026-10-06, one consumer was staged and had not yet flipped. The hub kept its escape.

What it bought the other projects

Five of the seven projects on this site are consumers of the standard. A sixth is Mergepath itself. The seventh left the fleet in September 2026 when its repository went dormant.

The standard lets those pages stay mostly about their products. Features shipped mid-sailing on Five Across landed as reviewed pull requests. Override’s engine was built a month before the standard existed, and every subsequent change to its product code has run through the fleet’s review path. Thirty governance, docs, and sync commits went straight to its main, the last on 2026-08-28. Those pages spend a line on governance and the rest on the product. This page is where that line gets audited.

The pull request that ships this page will go through the system it describes. That can prove the choreography runs. It proves nothing else. A system approving its own description is exactly the evidence this standard was built to distrust.