CLOSED Is Not MERGED

Tomohiro Iida · Published August 25, 2026 · Updated August 25, 2026

When AI agents do the implementation work, progress tends to be read off the pull request list. Closed pull requests accumulate and open issues appear to shrink. But a pull request being CLOSED on GitHub does not mean the change reached main. A pull request that was merged and one that was abandoned both appear in the list as a closed pull request. Measured in the Netsujo repository on 2026-08-25, 15 of the 57 closed pull requests numbered #1423 through #1550 had state CLOSED with a null mergedAt, meaning nothing from those pull requests themselves reached main. The population is stated as a fixed number range rather than "the most recent 60" because a sliding window does not reproduce: running the same query twice on the same day shifted the window as new merges landed, and the set changed.

Incident card

One word, five states

The sentence "that defect is fixed" does not say which of the following it means: a fix was written and a pull request opened, the pull request was closed, the pull request was merged, another pull request absorbed the same content, the change exists on current main, or the behaviour is actually corrected in production. Read as prose those six look like one thing. In GitHub and Git they succeed or fail independently.

All four rows were measured in the repository on 2026-08-25 with gh pr view --json state,mergedAt,title. Looking only at #1425 it reads as handled because it is closed, yet not one line reached main from it. The change was delivered by #1437. Conversely #1173 and #1207 were simply closed, and no replacement by another pull request is automatically guaranteed for them.

The false assumption: a closed pull request was taken in

• Close: abandoned without merging because of a change of direction, a dropped requirement, or an exhausted budget.
• Superseded: the same intent was handed to another pull request and the original was closed.
• Consolidation: several pull requests were combined into one and the individual ones were closed.
• Manual porting: the change was cherry-picked or rewritten locally and the original was closed.

The first outcome puts nothing on main. The other three put something on main, but what landed is not necessarily identical to the original pull request. The scope can be narrower, the conditions can differ, or the tests can have been dropped. A single closed bit cannot separate these four.

Root cause: conflating the pull request lifecycle with the content of main

The cause is the state model, not the judgement of the AI. A pull request represents the lifecycle of a container of work. Main represents the content of the code. Pushing both into a single word called complete makes it impossible for anyone to say which sense of complete is meant. AI agents surface this structural problem faster, because conversation is natural language, reports are summarised, and often only the pull request number is carried forward.

Immediate fix: read state and mergedAt separately

gh pr view <N> --json state,mergedAt,title — state CLOSED with mergedAt null means PR_NOT_DELIVERED; state MERGED with a non-null mergedAt allows the next layer.

Reading state alone does not decide anything, because a MERGED pull request is also a closed pull request in GitHub terms. The values that decide are mergedAt and the merge commit.

System fix: define the evidence ladder as five layers

In the Netsujo repository this decision lives in the Delivery Truth Gate at scripts/agent-os/delivery-truth.mjs. The layers rise in the order below, and a higher layer is never claimed by skipping a lower one.

• CLOSED_NOT_MERGED: closed but not delivered, with prDelivered false.
• MERGED_NOT_TARGET_BASE: merged, but the target branch was not main.
• CANDIDATE_UNVERIFIED: merged, but not confirmed identical to the authorized candidate.
• MAIN_CHANGE_UNVERIFIED: whether the change exists on main is unverified.
• EQUIVALENT_CHANGE_ON_MAIN: an equivalent change arrived on main by another route.

Do not rewrite superseded as merged

The last code is the important one. Even when the intent of a closed pull request is satisfied by another pull request, prDelivered stays false and changeSource becomes SEMANTIC_EQUIVALENT. Equivalence never holds automatically: it passes only when equivalentEvidenceVerified is supplied as separate verified evidence. The moment this is summarised as effectively merged, which diff from which pull request actually landed becomes untraceable.

Merged does not mean present on current main

The third layer is not automatic either. A merge into the base of a stacked pull request did not target main. A merge that was later reverted still leaves a merge commit in history while the content is gone. The post-merge check therefore confirms separately that the merge commit is reachable from current main. The same idea appears in rollback checking: scripts/deploy-policy/verify-rollback-target.mjs runs git merge-base --is-ancestor against the rollback target and fails when the commit is not in the history of origin/main.

A reusable confirmation procedure

• Run gh pr view <N> --json state,mergedAt,mergeCommit,baseRefName. A null mergedAt stops here: nothing arrived from this pull request.
• Confirm baseRefName is main. If not, stop: the target base is different.
• Run git merge-base --is-ancestor <mergeCommit> origin/main. A non-zero exit stops here: it is not in current main history.
• Re-fetch the diff and confirm the intended change or its equivalent is on current main. Equivalence requires verified evidence.
• Confirm the required behaviour through a test or a real observation.

Evidence

Limitations

• This covers GitHub pull requests. On other hosts, the equivalent of mergedAt and a way to check history reachability have to be established separately.
• Semantic inclusion at layer four cannot be decided automatically. Our operating rule passes it only when a person supplies the reasoning explicitly, and that is our baseline rather than a universal standard.
• Behaviour confirmation in production at layer five is covered by a later part of this series. This article mainly stops at whether the change exists on main.
• The measured pull request states are from 2026-08-25. Pull requests can be reopened later, so live state is re-fetched for every decision.

Remaining risk

• Even with a mergedAt value, a later revert means the change is no longer on current main. Layers three and four cannot be skipped.
• Inclusion through another pull request can be judged equivalent while the scope has narrowed. The equivalence reasoning itself has to be reviewed.
• Manual porting rarely records whether the original tests came along, so the gap first appears at layer five.
• State written in a pull request body or an old conversation is not a substitute for live state. Re-fetch it every time.

Frequently asked questions

If a pull request is CLOSED, how can I tell whether it was merged?

Read mergedAt with gh pr view <N> --json state,mergedAt. A MERGED pull request is also a closed pull request internally, so state alone cannot separate them. A null mergedAt means nothing reached main from that pull request.

If mergedAt is set, can I call it fixed?

Not yet. The merge target may not have been main, and the merge may have been reverted afterwards. Confirm that the merge commit is reachable from current main, for example with git merge-base --is-ancestor, before moving to the next layer.

If another pull request contains the same fix, can the original be treated as merged?

No. Our implementation records that situation as EQUIVALENT_CHANGE_ON_MAIN and leaves prDelivered false on the original. Judging the change equivalent requires separately verified evidence.

What should I check when an AI reports that something is fixed?

Ask which layer was confirmed. Opening a pull request, confirming a merge, re-fetching the diff on current main, and executing the behaviour all mean different things. A completion report that does not name a layer is not treated as evidence.

Do all five layers have to be checked every time?

It depends on the risk of the change. We require all five for changes touching authentication, billing, data deletion, or production configuration, and operate up to layer three for small-impact changes. When the classification is unclear, we take the higher side.

Continue in this series

Previous: Trust the exact SHA, not the pull request number

Next: Do not rerun CI the moment it fails

Back to the twelve-part AI agent incident record