Matchline

The problem

Matchline is built on two premises about the 2026 job market. First, anyone can generate a tailored résumé and cover letter in seconds, so every application is polished and polish is no longer a signal. Second, what remains as signal is specificity a candidate can defend—and a generative writer can’t supply it, because the model doesn’t know which projects mattered, which numbers are honest, or which claims survive an interview.

The clearest version of the problem is not lying. It is the thirteen phrases the validator refuses to let through—collaborated cross-functionally, drove results, moved the needle. Each one survives any traceability check, because the user did collaborate on something somewhere, and each one tells a reader nothing. The problem is not that applications contain lies. It is that they contain sentences which are true and empty.

So Matchline inverts the usual order. A typical AI career tool takes a résumé and a job description and generates prose. Matchline treats work history as a structured graph of capabilities, outcomes, and evidence, built once and refined over time, and uses AI to select from that graph against a specific opportunity. The model’s job is selection, sequencing, and framing. A claim that doesn’t trace to something the user has documented and approved is treated as a defect, and the product’s central mechanism exists to catch it.

Matchline began as a single-user hypothesis, based on my own job search. I did no external user research and no market validation, and V1 was deliberately scoped to proving whether the workflow worked for me before it was worth asking whether it worked for anyone else.

• 1 userthe entire V1 user base, by design
• $25/mothe Anthropic cap—about twelve tuning runs
• 10 × 11the whole evaluation corpus: résumés by job descriptions

What got built

Capability	State
Résumé into Experience Units	Built
Approved evidence only, into matching	Built
Job description into requirements, with gaps surfaced	Built
Fabrication gate blocking export	Built, never user-validated
Latency and cost targets	Designed only

LinkedIn and long-form ingestion were deferred. Uploaded PRDs, decks and retrospectives never existed—an earlier version of this page listed them as inputs, incorrectly, and the correction is recorded here rather than made silently.

The evidence model

An Experience Unit is the product’s atom. It is not a sentence about a job—it is one claim with the structure that makes the claim checkable, and the schema is strict about it, so the model cannot return a Unit that is missing its evidence:

export const ExtractedUnitV1Schema = z
  .object({
    raw_text: z.string().trim().min(1),
    normalized_summary: z.string().trim().min(1),
    unit_type: UnitTypeSchema,

    skills: z.array(z.string().trim().min(1)),
    tools: z.array(z.string().trim().min(1)),
    domains: z.array(z.string().trim().min(1)),
    seniority_signals: z.array(z.string().trim().min(1)),
    scope_signals: z.array(z.string().trim().min(1)),
    business_outcomes: z.array(z.string().trim().min(1)),
    metrics: z.array(MetricSchema),

    evidence_type: EvidenceTypeSchema,
    // Prompt says "below 0.50 should not be emitted" — enforce at
    // schema level too so a hallucination flood is rejected.
    confidence_score: z.number().min(0.5).max(1),

    date_range: DateRangeSchema.optional(),
  })
  .strict();

Here is one, from the repository’s own evaluation corpus—a hand-labelled Unit for a synthetic candidate, quoted as the fixture stores it:

Claim

Led Threadline B2B pivot from D2C-brand-as-customer to factory-as-customer, ran 38 customer development calls in six weeks; ARR per logo doubled, monthly churn dropped from 4.1% to 1.8%

Skills

customer development · go-to-market strategy · pricing strategy · founder-led sales

Domains

b2b saas · founder operations

Tools

—

That is the unit of currency. Not “led a pivot”—a specific pivot, with the calls that informed it and the two numbers that moved, decomposed so the matcher can ask whether this claim answers that requirement. The fixture carries the four fields the scorer compares; a Unit the extractor returns in production also carries metrics, seniority and scope signals, an evidence type and a confidence score, all produced by the model and validated server-side against the contract above.

Approval is not in that list, deliberately: the extractor is barred from returning it, and the server stamps every extracted Unit unapproved. For anything the model proposes, review is the only path into matching, and the filter that enforces it runs server-side at all three places evidence is consumed. A user can also type a Unit by hand, and those arrive pre-approved and marked user_confirmed—a Unit the user wrote is already the user’s own assertion, so there is nothing for review to add.

A pasted resume is extracted into Experience Units, each of which is written to the capability graph already, but unapproved. Human review flips that approval flag in place. The gate is not on the way into the graph but on the way out of it: a server-side filter passes only approved Units into matching, so an unapproved Unit sits in the store and is never read. Units the user types by hand arrive pre-approved instead. Separately, a job description is parsed into requirements. The matching engine scores approved Units against those requirements and surfaces requirements with no qualifying evidence as gaps rather than hiding them. Generation draws only on approved matches, and a traceability check sits between the generated draft and export: a claim that cannot be traced back to an approved Unit blocks the export until the user resolves it.

The two orange nodes are the whole product. Note where the first one sits: unapproved Units are written to the graph immediately, and the gate is a read filter on the way out rather than a lock on the way in—which is why the audit calls it a read gate and not a write gate. Everything else is plumbing that a competent team would build the same way; those two are where the thesis lives. Requirements with no qualifying evidence are surfaced as gaps in the same list as the matches rather than hidden behind a tab—the gaps are the point, not an embarrassment to bury.

The decisions

1. TrustPending

   Make zero fabrication a gate, not a warning

   What I encountered

   A generated application is a document the candidate has to defend out loud, in a room, weeks later. The failure mode is not a bad sentence—it is a good sentence about something that never happened, which reads as confident until someone asks a follow-up question.

   Over

   Score each claim's confidence and show the risky ones in a warning banner—the standard pattern, and the one that lets a user ship anyway on a deadline.

   What I decided

   Parse every claim in a draft back to the Experience Unit it came from, and disable export while any claim fails to trace. The user resolves it by editing the claim, removing it, or supplying a Unit that supports it.

   Why

   A warning transfers the judgment to the person least able to make it: someone under time pressure, looking at prose that sounds like their own. Blocking is only defensible if the thing being blocked is genuinely disqualifying, and an unprovable claim in a job application is.

   Cost

   The tool is most obstructive exactly when the user is in a hurry. It also cannot ship a claim that is true but undocumented—if the user has not written the evidence down, their real accomplishment is unusable, so the product taxes the honest case and the careless one identically.

   Validation boundary

   The gate is real in code and pinned by a fixture whose only job is to break it. What that proves is that the orchestration blocks fabrication; the language-model checks underneath are mocked in that test, so the detector's own reliability is unmeasured, and no use of a generated output by a user is recorded anywhere.

2. ScopePending

   Build for one user and refuse to generalize

   What I encountered

   Every V2 layer worth wanting—outcome-driven tuning, a referral graph, a should-I-apply score—was specifiable on day one, and each would have been more interesting to build than the four steps underneath it.

   Over

   Build the multi-user foundation first, on the usual reasoning that retrofitting tenancy is expensive and doing it up front is cheap.

   What I decided

   Scope V1 to a single user with a single job search, and put multi-user, sharing, teams, job-board integrations, and every learning layer explicitly out of scope in the spec rather than in a backlog.

   Why

   The open question was never whether the architecture could hold more users. It was whether evidence-grounded generation produces an application a person would actually send, which one user can answer and a thousand cannot answer any better.

   Cost

   Every validation signal the product can generate is self-generated. The only customer is also the author, the evaluation corpus is drawn from his own prospect list, and there is no reading of the results that separates a product that works from a product its builder is willing to tolerate.

3. MethodMixed

   Fix the ruler before trusting the measurement

   What I encountered

   Match accuracy was the number the whole matching layer was being tuned against. One run reported 18.1%, across a spread of 12.5% to 25.0% over three samples of a single fixture. The spread was wider than any improvement being measured.

   Over

   Keep tuning against the noisy metric and average more samples, which is cheaper in engineering time and was affordable within the cap.

   What I decided

   Stop tuning the matcher and repair the measurement instead: replace the absolute 0.30 similarity threshold with relative best-match mapping and a 0.10 sanity floor, then build a content-addressed stage cache so a tuning run replays unchanged stages for free.

   Why

   A metric whose run-to-run noise exceeds its effect size cannot tell you whether a change helped. Averaging buys precision at a per-run cost, and the cost was binding: one run at three samples was $2.06 against a $25 monthly cap, so the budget bought about twelve tuning runs a month.

   Cost

   The last two working weeks produced no user-visible change. The product is paused with better instruments and its last recorded accuracy unchanged—nothing measured it after the repair—and the work that would have shown up in a demo is exactly the work that did not get done.

   What it changed

   Both repairs landed and neither was measured. Eight weeks of product work separate them—the mapping fix in early June, the cache in the final session at the end of July—and nobody ran the repaired metric in any of it. So the project stopped with a better-designed instrument rather than with evidence that the instrument was better.

4. PrioritizationMixed

   A deadline someone else depends on wins

   What I encountered

   In early July two projects wanted the same summer. One was this: a single-user tool, no launch date, nobody waiting. The other was Five Across, a live multiplayer bingo game for a nine-night cruise, with a sailing date that could not move and players who would either have it or not.

   Over

   Push Matchline to a usable V1 first, on the argument that a tool this close to working should not be abandoned mid-flight.

   What I decided

   Stop Matchline and ship the game, then spend the one remaining Matchline session on measurement infrastructure rather than features.

   Why

   A date someone else set is not negotiable and a date I set is. The reasoning here is mine and the record can only corroborate its shape, not its cause: the commit timestamps establish that one project stopped and another started, and nothing in them establishes why.

   Cost

   The instruments were finally cheap, the accuracy was not yet there, and the run that would have shown whether the repair worked was never made. Knowing whether matching works needs more than that one run—a finished corpus, and a real application end to end—but the run was the next step, and it is the step that did not happen.

   What it changed

   Product work stopped the day before the Five Across repository was created, apart from one final session spent on the evaluation cache. That cache closed the phase; the next product commit lands a month later.

The measurement problem

The quality bar was 80% extraction accuracy and 80% match accuracy. Neither was reached, and how the number moved is more interesting than where it stopped.

The scorer first compared generated summaries to hand-labelled ones with string equality, and language models paraphrase, so the metric collapsed to 3.3%. Token overlap lifted it to 34%. A coefficient that gives full credit when the labelled set is covered by a longer generated one lifted it to 50.3%. Only then did a real system change—expanding the matching vocabulary from roughly 220 entries to 415—move it, by a single point, to 51.3%.

Extraction accuracy rose from 3.3 percent to 51.3 percent across four measurements, against an 80 percent target that was never reached. Three of those four measurements came from different definitions of the scoring metric rather than from any change to the product: string equality scored 3.3 percent, token overlap 34 percent, and a coverage coefficient 50.3 percent. Only the last step was a change to the system itself, an expansion of the matching vocabulary, and it added a single point to reach 51.3 percent.

Match accuracy went from 4.2% to 16.7% on that same vocabulary work, and then stopped being legible at all. One run reported 18.1%, and three samples of that one fixture spread from 12.5% to 25.0%—eight to thirteen points of noise, against a target improvement smaller than the noise. Averaging it away costs money, and money was binding: a single run cost $2.06 against a $25 monthly cap, about twelve runs a month.

So the last working sessions went into the instrument rather than the product.

The artifact

A zero-fabrication constraint is easy to state and easy to fake: any validator that returns “looks fine” scores well on honest inputs. So the repository carries a hand-labelled adversarial case—a fabricated paragraph attached to a real résumé, claiming a multi-year x86 and Arm CPU roadmap, a hundred-million-VM fleet, a 40% throughput gain on SAP HANA, and $400M in contracts displacing a competitor. None of it traces to any Experience Unit. The fixture enumerates all four untraceable claims with the reason each fails, and records the expected validation status as failed.

It runs in continuous integration against the Firestore emulator, and a passing run means the export was blocked. That proves the orchestration refuses a known untraceable claim. It does not prove the model reliably detects a novel one, because the model checks are mocked in that test—its integration boundary is the database, not the model API.

How it was built

Agents authored the implementation and independent agent identities reviewed it, under the enforcement system documented in Mergepath. The reviews materially changed the product: they caught a seniority scorer that was zeroing every Experience Unit because extraction emits verbs where the ladder matched nouns, a schema converter silently returning an empty object that had disabled model-input validation across four subsystems at once, and a matching rerun that discarded the user’s rejections. On that last one my own first reply to the reviewer was wrong, cited a test that did not cover the case, and the reviewer pushed back a second time before it was fixed.

The pause has a date the commits set: the evening of 2026-07-31, Pacific, when the eval stage cache landed. For the month after that only maintenance moved—dependency bumps and repository-template syncs, none of it touching the product. Product commits resumed on 2026-08-31 and, as of 2026-09-01, are still landing; what they fixed is the subject of the next section.

A development build is deployed behind a sign-in wall, and the “View Live Product” button above goes to it. It was redeployed on the evening of 2026-08-31 and now carries the June and July work this page rests on, rather than the 2026-05-02 build it served until then. Signing in with a new account gets an empty instance—every document is scoped to its owner_uid—so the link shows the shell, not the corpus.

What would restart it

I never wrote these criteria down while the project was live, which is itself a finding: the repository had exit criteria for finishing V1, but none for deciding whether to continue. Criteria for building, not for deciding.

Resume it when:

• The evaluation corpus is finished and labelled.
• Measurement is trustworthy enough to distinguish improvement from noise.
• One real application has gone through the product end to end—the condition the second build phase set for itself and never met.

Kill it when:

• Matching remains fundamentally poor after a full tuning cycle on the cheap evaluation infrastructure. At that point, the approach is wrong rather than merely untuned.
• I complete a job search without reaching for it. The product’s success metric was becoming my primary tool rather than another system abandoned for a spreadsheet. If I do not use it under real pressure, that answers the question.

Where it stands

As of September 2026, the conditions for revisiting the project are becoming real rather than hypothetical. The job search it was built for has resumed, giving Matchline the real-world test it never received.

Writing this page also exposed three stacked production failures: missing Cloud Run invoker bindings blocked CORS preflights, trailing newlines in both provider secrets broke authentication, and extraction held a sixty-second timeout against three long model calls. With those fixed, résumé extraction and JD parsing run end to end—and expose a worse problem: the matching ontology fails on out-of-domain requirements, so jaccard() zeros 45% of the score before fit is considered.

The eval harness can now route tuning runs through a subscription CLI instead of the metered Anthropic cap that had limited testing to roughly twelve runs a month.

The project remains paused. But the infrastructure now works, the cost of experimentation is low, and the job search provides the test it was built for. The next step is a deliberate restart against the criteria above—not continued development by inertia.

What I learned

The hardest part of building an AI product wasn’t getting the model to produce something plausible. It was deciding what evidence would make the output trustworthy—and then building a measurement system trustworthy enough to tell whether the product was improving.

Matchline got the first half further than the second. It can structure evidence, ground claims, and refuse an untraceable export. The matching layer stayed well below its quality bar, and I stopped after fixing the instrument that could finally measure it.

That is why the project is paused rather than shipped. A product that exists is not necessarily a product that works, and a metric that moves is not necessarily evidence that anything got better.