Frozen source
The repository and PR cutoff is origin/master at SHA 2cdd7383, merged PR #1101, on August 14, 2026. The analysis joins Git history, private PR descriptions and file lists, and orchestrator state copied from two execution hosts.
The corpus contains 1,077 first-parent commits, 1,016 merged PRs, and 711 deduplicated run records. Of those runs, 614 reached implementation, 599 passed, and 478 map to merged PRs. The complete quantitative method is in the analysis report, including notes on the essay’s figures and a correction to the original report’s 1,072-commit count.
Intervention
Intervention means the operator had to return to the keyboard after implementation started. The repository has no perfect “key pressed” event, so the primary direct proxy counts:
- An explicit operator response recorded during a run.
- A retry cap raised beyond the initial cap after the run crossed that initial cap.
- A
failedstate, which cannot retry without a person restarting or replacing the run.
The broader sensitivity proxy adds explicit kill markers and durable phase gaps of at least two hours. A long gap is not proof of human action, so it is not part of the primary rate. Superseded ancestor runs are included when a run maps to a merged PR.
Defects
Every merged PR was classified from its title, description, and changed files. Repair PRs were clustered when several repaired one underlying problem and split when one PR repaired independent problems. Product/customer defects exclude test-system, CI, observability, and internal developer-tool failures. Strict bugs are the subset involving incorrect calculation, behavior, persistence, authorization, or contract results; incomplete features and UX defects are tracked separately.
Origin attribution uses explicit PR references first, then changed-line blame and prior-touch evidence. The published escaped-defect example has medium-confidence origin attribution: the repaired lines trace to PR #834, but the private history is not an independently audited issue tracker. See the classification rubric.
The ratios of 0.55 strict and 0.97 inclusive known repaired product defects per current non-test KLOC divide historical discoveries by the current tree. Deleted and replaced code makes that denominator imperfect. The product is young, so additional latent defects may be found later.
“Clean” cases
The two clean cases passed with no direct or broad intervention signal, no gate failure, no rejected review, and no reviewer finding. No later repair in the classified history was attributed to either origin by the cutoff.
That definition is deliberately narrower than “bug-free.” It cannot see an undiscovered bug, a defect fixed after the cutoff, or a problem that never produced a repair PR.
Selection
The examples are curator-selected for legibility and contrast, not sampled for inference:
- The full spec is a substantive backend change with focused tests and a complete one-pass artifact trail.
- The task is a product change with 239 lines of churn, close to the all-history task median of 242.
- The quality-gate case preserves exact failed-test and reviewer-finding text and required no operator action.
- The escaped-defect case preserves pre-merge catches, a real intervention, origin evidence, and the later repair.
- The essay’s additional examples preserve selected database-lifetime, consent-ordering and idempotency findings; the overlap-test instruction and implementation; and the currency-error withdrawal and repairs. They were selected for concrete, traceable behavior rather than to estimate a catch rate.
This selection is disclosed in machine-readable form in the selection manifest. The examples illustrate mechanisms; aggregate claims come from the full corpus.
Redaction and integrity
The publication replaces local paths, temporary paths, database URLs, and private GitHub links. It exports selected run fields and focused diffs rather than whole logs or a source snapshot. Credentials, customer records, production payloads, and the full PR/defect ledgers are excluded.
The merge SHAs, reviewed base/head SHAs, timestamps, gate histories, and finding text are retained. A SHA-256 checksum ledger covers every raw artifact, allowing the published set to be frozen and compared over time.
Comparison limits
The archive makes Retirement Lab more inspectable; it does not make a private repository independently reproducible. Comparisons with published defect/KLOC studies are directionally useful, but reporting channels, product age, language mix, release definitions, and denominators differ. The honest conclusion is that the repository does not look implausibly defect-heavy and offers cause for optimism, not that it is proven several times better than a conventional team.