Six PRs, One Bug: What AI Agents Actually Get Wrong

Friends & Family Billing exists to send one kind of email: an invoice asking my friends and family for money. In early April 2026 it could not be trusted to send it. The template editor showed one version of the message, Preview a second, and the email that arrived a third—text turning bold that nobody had bolded, spacing no two surfaces agreed on. Issue #159 names bold and spacing as the two regressions, and the product consequence outranks both: a billing tool that cannot show you what it is about to send is untrustworthy at exactly the moment it asks someone to pay.

The first version of this post told the story in the order I remembered it: I filed the issue, then one agent spent roughly twenty hours opening six pull requests that failed to resolve it. The timestamps say the opposite. Every one of the six PRs was opened before issue #159 existed. The last closed thirty-six minutes before the issue was filed. Only the fix came after.

The chronology, corrected

That inversion is the story. First PR to merged fix is twenty-two hours and six minutes—“roughly twenty hours” was a fair round number and exactly wrong about the order. Issue #159 was not the starting gun for six failed attempts. It was the concession the six forced: the moment a stream of symptoms got named as one problem, with a definition of correct attached to the work. The definition was a day old by then, sitting in the design spec. It had just never been attached to anything anyone was reviewing.

“Six PRs” needs an inclusion rule, which the first version never stated. Six pull requests in one session on this surface: one originating implementation (PR #144), which introduced the architecture the bug lived in; three attempts at the parity bug (PR #146, PR #153, PR #158); and two real but orthogonal fixes (PR #154, PR #155). “Six failed attempts” is wrong twice over—#144 predates the bug it caused, and #154 and #155 were never aimed at it.

What parity has to mean

A trustworthy template editor makes the same document mean the same thing everywhere it appears. The first version stated this as Editor = Preview = Sent email, which reads as pixel equality and overclaims. The invariant is semantic parity. The editor renders its own DOM; the sent email is HTML built for mail clients, plus a plain-text part from a separate builder. Those outputs legitimately differ in appearance and encoding. Meaning must not, and issue #159 put that in one sentence: “Text that is not bold in the editor must not become bold in Preview or sent email.”

After PR #144, the architecture could not honor that:

Three paths, three outputs. The editor rendered structured content directly. Preview and send first flattened the document into markdown-like plain text via docToPlainTextWithTokens(), then reparsed that text through two different HTML pipelines. Once the same content was no longer guaranteed the same semantics, “WYSIWYG” stopped being true in any sense a user could rely on.

What the screenshots show

The screenshots in issue #159 are three layers of one system interpreting the same document differently.

In Edit mode the content is still structured TipTap JSON and nothing has been lost: body text, then the billing link, payment block, divider, and signature in order.

Preview flattened that document to markdown-like text and reparsed it with CommonMark, so separator lines and list content picked up different interpretations and default margins—identical content renders heavier and more spread out.

The sent email took the same flattened text through a separate regex renderer in the Cloud Function: a third interpretation of one document, and the only one that lands in an inbox.

The issue also attached a known-good sent email. That mattered: the job was never “make Preview look nicer” but “restore parity with a real output that previously existed”—a target, not a taste.

What I said to the agent

The prompts that drove the six PRs come from my unpublished session log. Unlike the timestamps and review counts elsewhere in this post, these excerpts are author records a reader cannot check. I quote them anyway, because for most of those twenty-two hours they were the only statement of the problem attached to the work—the design spec had described correct output all along, but nothing being built or reviewed pointed at it.

Prompt 6 (first report, two screenshots):

The text shows bold in preview, but not in the editor.

Prompt 7 (second report, two screenshots):

The bold issue is still there. It does not get fixed by bolding and unbolding, it doesn’t work at all. To add insult to injury, the app is now loading slowly or failing to reload, even after restarting the browser. Look hard this time, you keep missing something.

Prompt 9 was prompt 7 again, verbatim—I had run out of new ways to describe the problem. Only the evidence changed: two screenshots became five, showing a full reload cycle to prove the bug survived a hard refresh. The agent had changed code between those prompts. The output had not.

Prompt 11:

Given this is a simply single email template, maybe it is okay to sacrifice to get it right?

By prompt 11 I was offering to throw the template away rather than keep watching the loop. The arc is recognizable to anyone who has run an agent for more than ten minutes: describe the bug, escalate to “you keep missing something,” repeat yourself verbatim, end by questioning your own requirements. Notice what none of those prompts contain—not “Preview and email must use the same rendering path,” not “the markdown bridge is the wrong architecture,” not “audit your previous fixes first.” I described symptoms with rising urgency and expected a structural diagnosis to emerge. Handed a symptom each time, the agent reasonably patched the nearest plausible code path.

Six pull requests, each locally reasonable

Claude Code authored all six; the fix came later under the Codex identity—the rotation is visible in PR #161’s title prefix. But the per-PR record is not a story about agent incompetence. Each PR is competent inside its frame; what no PR could supply was the frame.

PR #144: the implementation that created the bridge

PR #144 introduced the TipTap WYSIWYG editor—the feature this arc is about—twenty-one hours before the issue existed. My kickoff prompt was one line: “Read invoicing-tab-redesign.md and implement the plan.” The design spec it pointed at was good. It chose TipTap JSON as canonical storage and described the derived output model plainly: “HTML is generated from JSON for Preview rendering. Email-safe HTML is generated from JSON for final outbound email rendering.” That sentence is essentially the invariant. But the same spec also required backward compatibility: buildInvoiceBody had to handle both legacy plain-text templates and the new TipTap JSON format.

The agent satisfied both the simplest way available: flatten the new format into the old one via docToPlainTextWithTokens() and reuse the existing pipeline. That honors the spec’s letter and defeats its intent—and it is not an unreasonable reading. The compatibility constraint had a named function and checkable behavior; the architectural intent was a sentence. When a spec carries both, the constraint that can be verified wins. The structured document became a temporary format on its way back to plaintext: the bug, one day early.

PR #146: reviewed, approved, and aimed at the wrong layer

PR #146 fixed bold-token round-tripping: the serializer began emitting bold-marked tokens as **%token%**, and the token regex learned to match both forms. A perfectly reasonable patch, if the bridge is the right architecture and merely lossy.

The first version of this post claimed my automated reviewer flagged the round-trip problem on this PR and the agent patched around the warning. The review record says the opposite, and the correction matters more than the original claim did. The nathanpayne-codex review of this PR, in full:

External re-review: APPROVED. I re-reviewed the invoice.js fix for the two issue #145 findings. The balanced regex now leaves one-sided ** as literal text, and docToPlainTextWithTokens() preserves bold-marked tokens as **%token%**, so the legacy plaintext fallback round-trips correctly. Verification in a clean worktree: exact round-trip repro cases, npm ci, npm --prefix functions ci, npm test, and npm run build.

Zero blocking reviews, zero inline comments, from either reviewer identity; nathanpayne-claude approved as well. Neither review mentions the invariant, because the PR did not attach it. The invariant existed—in a design document neither reviewer had reason to open, because nothing in this PR referenced it. The reviewers verified that the patch did exactly what it claimed, and what it claimed was never the question. Review confirms a diff against whatever standard the PR puts in front of it; when no standard is attached, it confirms the diff against itself.

PR #153: one part semantic patch, one part visual patch

PR #153 bundled fixes for five tracked InvoicingTab bugs, and two of them show the trap in miniature. On the serialization side it wrapped inline marks back into markdown:

if (marks.some(m => m.type === 'bold')) result = '**' + result + '**';
if (marks.some(m => m.type === 'italic')) result = '*' + result + '*';

On the presentation side it collapsed preview spacing with CSS:

.invoice-preview-message li p { margin-bottom: 0; }
.invoice-preview-message li p + p { margin-top: 2px; }

One fix assumes the problem is markdown fidelity; the other, preview styling. Both are locally valid and probably improved things. Neither touches the reason formatting could drift between surfaces at all.

PR #154 and PR #155: real fixes, orthogonal to the bug

PR #154 fixed useEditor recreating the editor on every keystroke. PR #155 converted bold, italic, and links in legacy templates so they stopped rendering literally in the editor. Real fixes, both—and neither aimed at parity, which is why the inclusion rule matters: counting them as failed parity attempts, as the first version did, inflates the drama and blurs the record.

They stay in the story for two reasons. Adjacent wins make progress feel like it is happening while the invariant stays broken. And PR #155 is where the reviewers actually pushed back—the beat this post previously misattached to PR #146. nathanpayne-codex filed three CHANGES_REQUESTED reviews on #155 before approving, each flagging a round-trip safety failure in the serialization layer, each answered with a scoped fix. Three blocking rounds pointing at the same lossy boundary, three patches to the boundary—and nothing in front of anyone licensing the question “should this boundary exist?” The spec had answered it a day earlier, in a document this PR did not cite.

PR #158: the bridge got cleaner

PR #158 is where sunk cost shows. The serializer moved into its own template-doc.js module—better boundaries, better round-tripping, and two more blocking review rounds before it merged. Competent work in service of the wrong system, now with cleaner internals and more invested effort standing between anyone and the uncomfortable question. It closed at 16:16 UTC. Thirty-six minutes later, the question got asked.

The review record

Across the six PRs, the public record holds seven blocking review rounds:

The first version said nine. Counting blocking rounds gives seven; counting every review submission gives nineteen; no counting rule I can reconstruct gives nine. Seven, with the rule stated, is the number this post now carries.

My session log adds figures the repository cannot—like the prompt excerpts above, author records with no public artifact behind them: eighteen user prompts across the arc, and three automated stop-hook interventions between prompts, one of which flagged that the plaintext fallback was derived from editor.getText() rather than from the renderer—the divergent-path problem, stated by a machine. Everything else in this post traces to public timestamps and review states.

The moment the problem got a name

Thirty-six minutes after PR #158 closed, I filed issue #159. It did three things nothing in the prior twenty-one hours had done: it treated editor, preview, and sent email as one problem instead of a stream of symptoms; it attached a known-good email as a concrete target; and it turned the invariant from a line in a design document into a requirement later work could be checked against. The issue stayed open under ninety minutes; the fix that closed it merged about sixty-five minutes after the filing.

The brief I gave the second agent

The prompt that produced PR #161 was not a bug report. It was a task document—titled “Codex Task—Investigate Failed Fixes (Issue #159)“—and like the session prompts above, my artifact, quoted rather than linkable. It opened:

Multiple fixes have already been attempted by Claude Code, and did not resolve the issue. You MUST treat this as a failed-fix investigation, not a greenfield implementation.

It listed the prior PRs and required an audit of each before any code: of its eight steps, the first three were pure reading—understand the issue, audit the failed fixes, identify the root cause. It stated the invariant as a requirement:

There must be one canonical rendering pipeline. At minimum, Preview and Sent Email must be generated from the exact same rendering path.

And it banned, by name, the moves already tried:

• Do NOT add another layer of transformation
• Do NOT “fix” by overriding CSS only
• Do NOT leave multiple rendering paths in place
• Do NOT rely on regex to fix formatting
• Do NOT optimize for minimal diff over correctness

Every line maps to the record—the CSS patch in #153, the regex work in #146 and #155, the extra transformation layer in #158, the minimal-diff instinct throughout. Not a best-practices list: the six PRs, inverted into constraints. It also required two deliverables beyond the code: an audit explaining which prior assumptions were wrong, and regression tests proving the Preview and email outputs structurally equivalent.

What the fix changed

Lined up side by side, the prior attempts shared one assumption: every one preserved the markdown bridge. The fix removed it from the HTML body of the preview and the test email. Not from the plain-text part, which that same payload still builds through the bridge, and not from the path the recipient’s invoice takes—the subject of the correction below. The core addition is a canonical renderer for invoice templates:

export function buildInvoiceTemplateEmailPayload(ctx, shareUrl) {
    return {
        html: renderInvoiceTemplate(ctx, shareUrl),
        text: buildInvoiceBody(ctx, 'text-only', shareUrl, 'email'),
    };
}

Both surfaces now consume it. The preview call site:

const previewEmailPayload = buildInvoiceTemplateEmailPayload(ctx, previewShareUrl);
const previewBodyHTML =
    previewEmailPayload.html || renderInvoiceTemplate(ctx, previewShareUrl);

And the test-email send in the Invoicing tab, which builds its own payload from the same function. Note which send this is—the button beside the preview, whose subject the same handler prefixes with [Test]:

const payload = buildInvoiceTemplateEmailPayload(ctx, shareUrl);

await queueEmail({
    to,
    subject,
    body: payload.text,
    html: payload.html,
    uid
});

The Cloud Function now sends trusted app-generated HTML when provided instead of re-parsing markdown. The winning change did not get more sophisticated about formatting; it got simpler about boundaries.

“When provided” is the whole hinge, and it is worth following where it leads. An email carries canonical HTML only if its producer supplies an html field, and in the entire application exactly one producer does: the [Test] send above. The settlement board’s per-member “Email Invoice” action passes { to, subject, body, uid } and no html, so the invoice a household member actually receives still renders through the markdown bridge—docToPlainTextWithTokens into simpleMarkdownToHtml, the function this whole arc exists to have displaced. PR #161 never touched that file; git show --stat lists eight, and EmailInvoiceDialog.jsx is not among them.

There is a sharper way to say it. Before #161 the test email and the invoice email rendered identically—both sent body with no html. #161 gave the test email canonical HTML and left the invoice email where it was. The fix closed the gap on the surface where the bug was observed and opened a new one between the test email and the real invoice, and that divergence has stood since 2026-04-04. I did not find this while writing the post; a later evidence audit did, which is the uncomfortable part—the post shipped asserting a parity its own screenshots could not have shown.

“One rendering path” needs its boundary named, because stated baldly it overclaims. The editor still renders its own DOM directly from the document. The plain-text part of the email payload has its own builder, by design. The preview keeps a fallback—previewEmailPayload.html || renderInvoiceTemplate(...)—a second call site into the renderer, not a second renderer. The claim the merged code supports is narrower still: the template body in the preview and in the test email is produced by renderInvoiceTemplate, so text that is not bold in the editor cannot become bold in either place. Two surfaces, not three, and neither of them is the recipient’s invoice. That email wraps its body in envelope HTML the preview does not show—a branded header, a container, a “Sent via Friends & Family Billing” footer—visible in the screenshots above and outside the renderer by design. That is the semantic parity issue #159 asked for, on the two surfaces it reached, and it is much narrower than “everything renders identically.” Evidence and commands: plans/759/project-pages-ledger.md §C40.

What actually varied

It would be easy to read this arc as “Codex is better than Claude Code at architecture.” The first version asserted the opposite—“the difference was not the model”—and the record supports neither claim. Between the failed sequence and the fix, everything changed at once: the model, the tooling, the accumulated session context, the visibility of six prior PRs to audit, and the framing of the task. No run isolated framing as the only variable, so there is no clean causal claim to make about it.

What the record does support is narrower. The framing was the variable I controlled. After the kickoff prompt, every prompt in that session described a symptom, and each time the agent did what a competent developer does when someone leans over the desk and says “this looks wrong”: it found the nearest code path that could explain the symptom and patched it. The brief described a pattern of failures, demanded an explanation of them before any code, stated the invariant, and banned the patch classes already tried. Forty-nine minutes after the issue was filed, the PR that ends this story was open; sixteen minutes later it was merged. Whether another agent—or the same one, re-briefed—would have converged the same way, one run cannot establish. I did not rerun the experiment; I shipped the fix.

The rules I kept, and what they cost

After the merge I turned the arc into standing rules, each earning its keep against a cost. Two of the four are written into this repository’s policy and bind every agent working in it; the other two are practice I follow, unenforced. The distinction matters: a rule with a home in a policy file is checkable in exactly the way the design spec’s invariant was not, which is the whole subject of this post.

Two failed fixes change the task. (Repository policy: Two-strike audit rule.) If two fix attempts on the same problem have failed, the third must begin with an audit of the prior PRs: what each assumed, and why the assumption was wrong, before any new code. The cost: on genuinely shallow bugs the audit is pure overhead, and the rule cannot tell you in advance which kind you have.

The serialization checklist. (Repository policy: Serialization layer review requirement.) When a change crosses a format boundary, review asks three questions: is the round-trip lossless, do all consumers of the format produce equivalent output, and is the intermediate format necessary at all. The questions are designed to surface exactly what stayed invisible here. Whether they would have changed the outcome at PR #144 is a counterfactual this record cannot test; they are standing practice now. The cost: a checklist only binds when the reviewer runs it against the architecture, and this arc shows reviewers faithfully verifying diffs while the architecture drifted.

Constraint-driven prompts for cross-layer bugs. (Personal practice, not adopted policy.) When a bug touches more than one layer, the prompt carries an explicit list of banned approaches derived from prior failures in this codebase—not abstract best practices. The cost: the extraction is operator work—the banned list here took reading six PRs—and an over-broad ban can exclude the right fix.

Invariants outrank backward compatibility. (Personal practice, not adopted policy.) When a spec carries both a new architecture and a compatibility requirement, it now states which wins: the new rendering path is canonical, and legacy format support is a migration concern, not an architectural peer. The cost: the compatibility work gets more expensive and more explicit up front—which is the point, because implicit is how the bridge got built.

The bug was fixed about sixty-five minutes after it was named—on the surface where it was reported. The preview and the test email now agree with what the editor means, and the brief’s regression tests were aimed at keeping the whole class of defect closed rather than patching one instance. What a recipient sees still does not: the invoice email was never on the canonical path, and a later evidence audit is what established that, not this fix. The expensive part was the twenty-one hours before the name existed, in which six pull requests of locally reasonable, individually reviewed work shipped against a correctness standard nobody was checking. And the standard was not missing. It was in the design spec from the start, one sentence describing exactly the output model the bug violated. What it never was, until issue #159, was a requirement attached to any piece of work anyone reviewed. That is the process failure, and it is harder than “write it down”: prose in a design document loses to a named function with checkable behavior, and no amount of louder symptom reporting closes the gap.