We Put Everything Behind Owner Approval and Delivery Stopped
Tomohiro Iida · Published August 25, 2026 · Updated August 25, 2026
The more implementation work an AI agent takes on, the more the operating model turns into a single question: where does a human approve? Until 2026-08-21, Netsujo required an explicit exact-SHA Owner authorization on every pull request before it could reach main, regardless of branch name. It was meant to be the safe choice. What actually happened was that low-risk changes stopped moving even after CI was green, head identity matched, the freeze lease was clear, and no file collision existed.
Key takeaways
- Adding approval points does not only add safety. It adds waiting time and it thins out the attention available for each approval.
- LOW and MEDIUM changes now integrate autonomously through the existing machine gates and the Integration Controller.
- HIGH changes still require an exact-SHA Owner authorization plus an independent FINAL_QC review.
- UNKNOWN is fail-closed. A change that could not be classified is never treated as low risk.
- Owner judgement belongs at boundaries that are irreversible or whose loss ceiling cannot be read.
Incident Card
| Field | Value |
|---|---|
| Incident | Every pull request required an explicit exact-SHA Owner authorization before merging to main. |
| Symptom | Low-risk changes waited on the Owner even after CI, head identity, freeze, and collision checks all passed. |
| False assumption | More approval points always increase safety and never degrade the quality of the review itself. |
| Root cause | Risk classification and merge authority were built separately, so one approval contract applied uniformly to every diff. |
| Immediate fix | Keep the exact-SHA authorization contract and narrow its scope to high-risk pull requests. |
| System fix | Classify every diff as LOW, MEDIUM, HIGH, or UNKNOWN and vary the integration path and required evidence per class. |
| Remaining risk | Missing classification patterns, repository layout drift, and an approval backlog that nobody can count. |
The Owner sat on the critical path for every change
The observation was simple. Until the Owner wrote an authorization string, no pull request advanced. The authorization itself was a marker in the pull request body naming the target pull request number and a 40-character commit SHA. The contract has a genuinely useful property. A pull request number alone goes stale as soon as another commit lands, because the number still points at a moving target. Binding the authorization to a 40-character SHA means the approval expires automatically the moment the head changes. The problem was scope, not the contract. Requiring that marker on every pull request put the following kinds of change into the same queue as a database migration.
- A typo fix in documentation.
- An added test.
- A wording change inside an article body.
- A README update.
Machine checks can decide whether those changes are correct, and a failure has no customer impact in production. They still waited for a human to type a string. The implementation comment in the repository records the state directly: low and medium changes that had already passed CI, exact identity, freeze, and collision checks put the Owner on the critical path every time, and integration work stopped in bulk.
The false assumption: approval only adds safety
Adding an approval point looks close to free at the moment you add it. One line of configuration appears to remove one possible incident. In practice an approval point creates two costs at once.
- Waiting time: everything downstream stops until the approver acts.
- Diluted attention: as the number of approvals grows, the time available for each one shrinks.
The second cost is the dangerous one. If thirty approval requests arrive in a day and twenty-eight of them are typo fixes, the two genuinely risky changes receive the same glance as the rest. An approval that has become a formality can be worse than no approval, because it leaves behind a record saying the change was reviewed.
Calling a design “the safe choice” is only accurate when the cost sits on one side. Approval design does not meet that condition.
Root cause: classification was never wired to authority
The cause was not a slow Owner and not an untrustworthy agent. A mechanism that judged the risk of a change and a mechanism that decided who authorizes what existed side by side without being connected. A risk classifier was already in the repository, yet the merge decision read only the single approval contract. Classification produced reporting output and never reached authority. The same shape appears without AI agents. Applying a code-owner rule to every directory in a repository produces the same queue.
Immediate fix: keep the contract, narrow the scope
The first change was not to weaken the authorization contract. The contract stayed exactly as it was, and only its scope moved to high risk.
| Before | After |
|---|---|
| Exact-SHA authorization required on every pull request. | Exact-SHA authorization required on high-risk pull requests only. |
| Low and medium changes waited for human input. | Low and medium changes integrate through the existing machine gates. |
| Unclassifiable diffs had no defined handling. | Unclassifiable diffs fall to the high-risk side. |
In the current implementation the requirement is expressed as a negation: Owner FINAL_QC is required unless the risk is low or medium. The negative form matters. When a new class appears, it lands by default on the side that requires Owner approval. Writing “required only when high” would let an unknown value through without one.
System fix: four classes, four contracts
| Class | Integration path | Required evidence |
|---|---|---|
| LOW | Autonomous integration by the Controller. | Green CI, head identity, freeze clear, no collision. |
| MEDIUM | Autonomous integration by the Controller. | The same machine gate set as LOW. |
| HIGH | Excluded from autonomous integration. | Exact-SHA Owner authorization plus an independent FINAL_QC review. |
| UNKNOWN | Not integrated. | Nothing proceeds until classification succeeds. |
HIGH is decided mechanically from file paths. The implementation enumerates workflow definitions, agent operating contracts, governance files, quality gate configuration, package manifests, deployment configuration, database migrations, and the authentication, authorization, billing, session, and customer-data-access modules. LOW enumerates the opposite: documentation, README files, markdown, and test files, where a change does not alter customer-visible production behaviour. Anything matching neither list becomes MEDIUM, so an unclassifiable diff is never demoted to LOW.
UNKNOWN is fail-closed
The most important part of the design is the default when observation fails. The current implementation treats all of the following as high risk: an empty changed-file list, a malformed API response, an invalid pull request number or repository name, and an exception during classification. Each returns a failure code and marks Owner FINAL_QC as required.
Not having observed something is not the same as having observed that it is safe.
The quality gate configuration states the same principle explicitly: neither a classification result nor a machine-check pass authorizes execution, and telemetry that cannot be observed is recorded as unknown rather than filled in by inference.
A HIGH approval cannot be self-issued
A change classified as HIGH is not integrated on the strength of an Owner authorization alone. An independent FINAL_QC review is also required, and that review must carry a PASS status line together with a marker naming a 40-character SHA. The check compares that SHA against the immutable commit identifier GitHub records for the review, so the value is not a string the implementer can choose freely. Pushing another commit makes the previous authorization stale on its own. Netsujo also reports implementation, merge, deployment, and production verification as separate states after that check passes. A successful deployment on its own is not reported as completion.
Make the approval inventory visible
Splitting by risk class does not drive the HIGH count to zero. It concentrates genuinely dangerous changes in the Owner queue, which makes the size of that queue the number worth watching. Today the Controller integrates at most one pull request per run and acts as a single writer on GitHub, starting only on events that change integration feasibility, with a six-hourly schedule reserved for recovery from missed events. A dedicated inventory of pending approvals is not implemented. What the operation can read today is the Controller run log and the pull request list. The following items are the proposed output.
- The number of HIGH pull requests awaiting approval and how long each has waited.
- The number of authorizations that expired because the head moved after approval.
- The number of items stopped as UNKNOWN, with the classification failure code.
- The number of LOW and MEDIUM changes that integrated autonomously.
Without that inventory, “the Owner has become the bottleneck” can only be detected by feel, and there is no way to evaluate whether the class thresholds are set correctly.
A reusable checklist
- If this change fails, is the failure irreversible? If it can be undone, prior human approval is not mandatory.
- How large is the loss if it fails? Customer data, billing, and authentication have no readable ceiling.
- Can the judgement be made deterministically by a machine? If so, route it to a check rather than a person.
- When the judgement itself fails, which way does the default fall? If it does not fall to the safe side, the design is incomplete.
- Is the number of approvals small enough that the approver can actually read them?
- Is the approval bound to the state at the moment of approval? An approval not bound to a SHA drifts off target.
- Can the pending count and waiting time be counted after the fact?
Evidence
| Item | Class | Basis |
|---|---|---|
| Requiring exact-SHA authorization on every pull request stopped low and medium integration in bulk. | OBSERVED | Implementation comment in scripts/agent-os/integration-controller.mjs |
| The exact-SHA authorization scope was narrowed to high risk. | IMPLEMENTED | ownerMergeAuthorization in scripts/agent-os/integration-controller.mjs |
| Owner FINAL_QC is required for everything other than low and medium. | IMPLEMENTED | scripts/agent-os/owner-final-qc-policy.mjs |
| Classification or observation failure is treated as high risk and fails closed. | IMPLEMENTED | Failure codes in scripts/agent-os/owner-final-qc-policy.mjs |
| LOW, MEDIUM, and HIGH are decided mechanically from file paths. | IMPLEMENTED | classifyRisk in scripts/agent-os/integration-controller.mjs |
| Neither classification nor a machine-check pass authorizes execution. | IMPLEMENTED | executionAuthorization in .quality/codex-review-policy.json |
| A diff matching no rule becomes MEDIUM rather than LOW. | IMPLEMENTED | scripts/quality/classify-change-risk.mjs |
| FINAL_QC authorization is matched against the GitHub review commit identifier, so it cannot be self-issued. | IMPLEMENTED | scripts/agent-os/final-qc-authorization.mjs |
| The Controller integrates at most one pull request per run. | IMPLEMENTED | .github/workflows/agent-integration-controller.yml |
| Workers do not merge; integration is concentrated in the Controller. | IMPLEMENTED | AGENTS.md in the repository |
| Production release requires a manual dispatch with a confirmation string. | IMPLEMENTED | .github/workflows/deploy-signal-production.yml |
| Git-triggered automatic deployment is disabled on every branch. | IMPLEMENTED | git.deploymentEnabled in vercel.json |
| An inventory listing pending approval counts and waiting times. | PROPOSED | Not implemented |
| The lead-time reduction produced by the class split. | INFERRED | Continuous waiting-time measurement is not in place |
Limits of this design
- The diff must be observable by file path, and paths must correlate with risk to a usable degree.
- CI, type checking, and tests must have real detection power over the LOW and MEDIUM classes.
- A single writer must exist so that integration can be serialized.
- The approver must be able to read and judge the content of HIGH changes.
Where paths correlate weakly with risk, enumerating classifier patterns does not work, and the range pushed to MEDIUM or above has to widen. The HIGH path list Netsujo uses depends on the layout of our own repository. It is our operating guideline, not a universal answer. The change in integration lead time before and after the split has not been measured; the stalled state clearly cleared, but the size of the reduction is unmeasured.
Remaining risk
- A new directory is added and high-risk changes land without matching any HIGH pattern.
- Code that does change behaviour is placed under a path enumerated as LOW.
- A change to the classifier patterns themselves lands without review.
- The HIGH count grows until Owner review becomes a formality again.
- A commit lands after an approved SHA and the expiry goes unnoticed.
- An autonomously integrated change carries a defect that CI cannot detect.
The last item matters most. Autonomous integration of LOW and MEDIUM changes trusts the detection power of the machine checks. If that power drops, the safety of the class split drops with it. Whether a test actually detects what its name claims is confirmed by mutating the code and checking that the test fails, not by counting tests.
Conclusion
Owner approval is an instrument for making development safe, not an instrument for stopping it. Decide the three routing rules first: LOW and MEDIUM go to machine gates plus autonomous integration, HIGH goes to exact-SHA Owner authorization plus an independent FINAL_QC, and UNKNOWN does not integrate at all. Only then enumerate which paths belong to which class. Reversing that order adds approval points without improving the quality of the judgement.
Frequently asked questions
- Does splitting by risk class make audits harder to satisfy?
- Defining the path and required evidence per class and holding the judgement in code makes it possible to reproduce mechanically which change took which route. That can be easier to explain than a record showing a human approved everything.
- Do LOW changes still need human review?
- That depends on the goal. At Netsujo, LOW and MEDIUM changes still pass CI, type checking, tests, and the project checks. Human review is removed from the mandatory integration conditions, not prohibited.
- Does binding approval to a SHA force re-approval after every fix?
- Yes, and that is the intended behaviour. The approved code and the integrated code must be the same, so an approval expires when the head moves. To reduce the frequency, split HIGH changes into smaller pieces.
- Does a fail-closed UNKNOWN stop work?
- It does. The count and the reason are recorded, so the stoppage can be fixed as a gap in the classification patterns. Sending UNKNOWN to LOW avoids the stop but removes any way to identify later what passed through.
- Does this apply to development without AI agents?
- The structure transfers. Removing machine-decidable judgements from human approval and concentrating people on irreversible, high-loss boundaries does not depend on the kind of team. The HIGH path list has to be rebuilt per repository.
Episode 10: a successful deployment is not a production checkThe previous episode covers provenance, the production SHA, authenticated paths, and browser evidence.
AI agent development and operations incident logThe series hub lists every incident in this record and the order to read them in.
A repeated instruction is a system bugThe separate article covers moving repeated human attention into a Controller, Evidence Ledger, and Gates.
We design risk classification, integration authority, CI, quality gates, and production evidence as one operating model.
Talk to Netsujo about AI implementation