Trust the Exact SHA, Not the Pull Request Number
Tomohiro Iida · Published August 25, 2026 · Updated August 25, 2026
When AI agents implement code, their reports arrive in natural language: CI is green, the change is reviewed, it is on main. On the GitHub side, however, there are seven states that can each be a different commit: the pull request head, the commit CI actually tested, the commit a review read, the commit an owner authorized, the commit created on main by the merge, the commit a deployment built, and the commit production is currently serving. What looks like one thing in a conversation is seven things in the system.
Key takeaways
- In AI agent operations, the identifier for the object being operated on is the 40-character commit SHA, not the pull request number.
- Give separate names to seven SHAs: PR_HEAD_SHA, CI_TESTED_SHA, REVIEWED_SHA, OWNER_AUTHORIZED_SHA, MERGED_SHA, DEPLOYED_SHA, and PRODUCTION_SHA.
- Fetch the pull request head from two independent paths and compare them. Treat UNKNOWN as DENY, never as ALLOW.
- A green CI run only proves that some commit passed. It does not prove which commit, nor that the head has not moved since.
- The commit produced by a squash merge is not the pull request head, so exact-head CI from the pull request cannot be reused for it.
Incident card
| Field | Value |
|---|---|
| Incident | The remote branch carried newer commits, but the GitHub pull request API kept returning an older head, and the Controller merged that older head. |
| Symptom | A state that dropped three verified fixes, including two known high-risk findings, reached main. |
| False assumption | If the pull request number and the green CI badge line up, the merge target is the same code that was verified. |
| Root cause | No step existed that fetched the verified SHA and the operated SHA separately and compared them, so a single displayed head was trusted. |
| Immediate fix | Re-fetch the remote branch ref through an independent path immediately before merging and refuse to merge on mismatch. |
| System fix | Give seven SHAs distinct names and deny whenever the identity equation breaks or a value cannot be read. |
| Remaining risk | A squash merge commit differs from the pull request head, so the exact-head CI evidence cannot be carried over to it. |
A pull request number is a container whose contents change
A pull request number identifies a unit of work. The commits inside it change through pushes, rebases, force pushes, and base branch advances. The number does not change. That is why none of the following sentences identifies a verification subject on its own: PR #1234 is green, PR #1234 is reviewed, PR #1234 was merged. Each one omits which version of #1234 is meant.
On 2026-08-17 this happened in the Netsujo repository. Three commits were already pushed to the remote branch, but the pull request API kept returning an older head. The Controller merged that head, and a state that dropped three verified fixes reached main. The merge API sha precondition did not help: it compares the head that GitHub itself holds, not the actual remote ref, so a stale head simply matches another stale value.
The false assumption: green CI means the merge target is verified
A green badge shows only that some commit passed some checks. It does not show which commit was tested, whether that commit is still the head, whether the job was a required check, whether checks were skipped because of the change set, or whether the head advanced after the run finished.
The same gap appears on reruns. A manual rerun in GitHub Actions replays the original event payload, so a green rerun reports on the SHA from the original event, not the SHA at rerun time. The repository closes this with a first step in ci.yml called Reject stale PR head rerun, which compares the event head SHA against the live branch ref over the Git protocol and fails with STALE_PR_HEAD when they differ. A separate workflow, pr-exact-head-ci.yml, cross-checks the local checkout, the event head, and the remote branch ref, and fails if any of the three is missing or disagrees.
Root cause: identity checking was not implemented anywhere
The cause was not the judgement of the AI or the wording of its report. No step on the path mechanically compared the SHA that was verified with the SHA that was operated on. Human teams hit the same structure, but running several agents in parallel raises both how often main moves and how many reports arrive, and visual checking stops keeping up.
Immediate fix: re-fetch the head through an independent path
The first fix was simple. Just before merging, fetch the remote branch ref from a source other than the pull request API and compare. The source of truth for this decision is evaluateHeadIdentity in scripts/agent-os/integration-controller.mjs, which requires the following equation to hold.
PR_API_HEAD_SHA = REMOTE_BRANCH_REF_SHA = REQUIRED_CI_TARGET_SHA = MERGE_EXPECTED_HEAD_SHA
| State | Code | Decision |
|---|---|---|
| Pull request head unavailable | PR_HEAD_UNKNOWN | DENY |
| Remote ref unavailable | REMOTE_REF_UNKNOWN | DENY |
| Pull request head differs from remote ref | STALE_PR_HEAD | DENY |
| CI target SHA differs | CI_TARGET_MISMATCH | DENY |
| Expected merge head differs | MERGE_HEAD_MISMATCH | DENY |
The important part is that a value which could not be fetched also results in DENY. Turning UNKNOWN into ALLOW converts a missing check into an automatic approval.
System fix: give seven SHAs seven names
| Name | What it points at | Where it comes from |
|---|---|---|
| PR_HEAD_SHA | The commit the pull request head points at | Both the pull request API and the remote branch ref |
| CI_TESTED_SHA | The commit CI actually checked out and tested | The ref a workflow run targeted |
| REVIEWED_SHA | The commit whose diff a review read | The head at review submission time |
| OWNER_AUTHORIZED_SHA | The commit an owner authorized for merge | The owner marker in the pull request body |
| MERGED_SHA | The commit created on main by the merge | The resulting merge commit |
| DEPLOYED_SHA | The commit a deployment built | The execution subject of the deploy workflow |
| PRODUCTION_SHA | The commit production is currently serving | Metadata of the deployment behind the alias |
- OWNER_AUTHORIZED_SHA is expressed as an authorization line in the pull request body that pins both the pull request number and a 40-character SHA. A different number yields DIFFERENT_PR_AUTHORIZATION and a different SHA yields AUTHORIZATION_SHA_MISMATCH, so authorization applies to a commit, not to a pull request.
- MERGED_SHA is a different object from the pull request head. The workflow candidate-exact-sha-gate.yml checks out the post-merge candidate SHA and reruns lint, typecheck, tests, and the production build, then records a commit status on that SHA.
- PRODUCTION_SHA is resolved by scripts/deploy-policy/resolve-production-sha.mjs, which walks from the production alias to the deployment metadata. The expected value comes from the workflow execution subject rather than from human input, and a fetch failure, an empty record, or a mismatch all fail closed.
Separate the right to verify from the right to record
In candidate-exact-sha-gate.yml the job that executes candidate code has no statuses write permission, and the job that writes the status executes none of the candidate code. Verification code and its dependencies can run arbitrary commands, so placing a write token there would let the verdict itself be forged. Identity design is not finished when values are compared; it also has to answer who is allowed to write those values.
A reusable exact-SHA checklist
- Require a 40-character SHA, not only a pull request number, as a mandatory field in every report.
- Fetch the pull request head independently from the pull request API and the remote branch ref.
- Record the SHA a CI run targeted and compare it with the current head.
- Check that a green rerun is not a result for the stale SHA of the original event.
- Record authorization per SHA rather than per pull request.
- Run a separate verification against the post-merge SHA instead of reusing the pull request run.
- Compare the deployment subject SHA with the SHA production is currently serving.
- Treat any value that cannot be fetched as DENY.
Avoid abbreviated SHAs. Every decision runs on 40 lowercase hexadecimal characters, and a value that fails the format check fails before anything else happens.
Evidence
| Statement | Class | Source |
|---|---|---|
| An older pull request head was merged and three verified fixes were dropped on 2026-08-17. | OBSERVED | Recorded as incident context in scripts/agent-os/integration-controller.mjs |
| Merge identity is decided by a four-value equation and UNKNOWN results in DENY. | IMPLEMENTED | scripts/agent-os/integration-controller.mjs |
| Authorization is validated as a pair of pull request number and 40-character SHA. | IMPLEMENTED | ownerMergeAuthorization in scripts/agent-os/integration-controller.mjs |
| Reruns compare the event head SHA against the live branch ref and fail on mismatch. | IMPLEMENTED | Reject stale PR head rerun in .github/workflows/ci.yml |
| The pull request head is confirmed from local checkout, event payload, and remote ref. | IMPLEMENTED | .github/workflows/pr-exact-head-ci.yml |
| The post-merge candidate SHA is checked out again for a full gate and a recorded status. | IMPLEMENTED | .github/workflows/candidate-exact-sha-gate.yml |
| The job executing candidate code holds no status write permission. | IMPLEMENTED | .github/workflows/candidate-exact-sha-gate.yml |
| The commit production serves is resolved from the alias and mismatches fail closed. | IMPLEMENTED | scripts/deploy-policy/resolve-production-sha.mjs |
| Machine-enforced SHA identity makes mistaken targets surface earlier. | INFERRED | The size of the reduction is not measured. |
Limitations
- This record covers operations on GitHub and GitHub Actions. On other hosts, the two independent sources for each value have to be designed separately.
- Within the twelve-part incident series, this article stops at identity around merge. Production verification itself is covered by a later part.
- Comparing all seven SHAs mechanically is a Netsujo operating baseline for environments where AI agents merge in parallel. In low-frequency projects the bookkeeping can cost more than it returns.
Remaining risk
- A squash merge commit differs from the pull request head, so the post-merge SHA needs its own gate.
- A successful commit status cannot be treated as a pass unless the workflow run that wrote it can be identified.
- A workflow_dispatch run uses the definition on the selected ref, so in-workflow guards do not defend against an actor who can rewrite the definition.
- Aligned SHAs say nothing about the depth of the checks. Identity is a precondition for quality, not quality itself.
Frequently asked questions
- Is using pull request numbers itself the problem?
- No. A pull request number is a convenient human reference. What to avoid is deciding whether something is verified based on the number alone. Converse in pull request numbers and decide on 40-character SHAs.
- If CI is green, can the change be merged as is?
- Green is a result for one commit. Only after confirming that the commit is still the head, that the job was a required check, and that no check was skipped for the change set can the merge target be called verified.
- Does CI need to run again after a squash merge?
- The pull request head and the post-merge SHA are different objects, so the earlier result cannot be reused. Netsujo checks out the post-merge candidate SHA, reruns lint, typecheck, tests, and the build, and records a status on that SHA.
- Can abbreviated SHAs be used in operations?
- Not recommended. Our operating rule fixes every decision value at 40 lowercase characters and fails as soon as the format differs. Abbreviations are limited to human-readable display.
- If an AI reports that the SHAs match, can that be trusted?
- Self-reporting is not evidence. Check the path the value was fetched from, the run it belongs to, and the decision code in the record. The right to decide and the right to write the result also need to be held by different jobs.
Continue in this series
Previous: Stopping the chat does not stop the work
Back to the twelve-part AI agent incident record
If pull requests, CI, approvals, merges, deployments, and production have drifted into separate states after adopting AI agents, we help design the identity checks and evidence that tie them back together.
Discuss AI implementation and development operations design