Splitting Design, Implementation, and Audit Across Three AIs

Tomohiro Iida · Published August 5, 2026 · Updated August 23, 2026

For part of the implementation work I lead personally at Netsujo, the process runs like this: ChatGPT organises the purpose, requirements, constraints, and acceptance criteria; a Markdown instruction document is produced for Claude Code; Claude Code inspects the real repository and does the implementation and testing; Codex inspects the near-final diff; and a human decides which findings to accept and whether to publish. The goal is not to use a lot of AI. It is to bring the separation between designer, implementer, and auditor into work that one person owns end to end.

Why three AIs touch one implementation

In a human development team, a planner organises the purpose, an engineer implements, and a different engineer reviews the code. When one person or a very small team owns the work, that separation weakens. The person who defined the requirement also directs the implementation and judges completion, so the original assumption can survive untested all the way to the end.

Different products do not create independence on their own. Independence comes from giving the implementer and the auditor different roles, withholding edit authority from the auditor, passing only the latest diff, writing down the evaluation criteria, refusing findings without a basis, not reviewing the same diff repeatedly, and fixing the exit condition in advance. The point of three AIs is not three virtual engineers. It is one deliberate refutation step inside a single judgement path.

Why ChatGPT handles the design

My work starts from business development, so the first thing I think about is not code but customers, revenue, and operations. A request such as “let customers enter their goal on the analysis screen” hides several questions: who enters it, for what purpose, one goal or several, free text or a selection, how the input affects the analysis logic, which parts of the existing screen stay untouched, what is explicitly out of scope this time, and what counts as done. Left unstated, an AI fills the missing assumptions by inference. The faster the implementation, the faster a vague assumption becomes code. So before any code, the request is converted into purpose, current state, the problem to solve, target users, mandatory requirements, conditions that must not change, non-goals, acceptance criteria, expected risks, verification method, and open questions.

The quality of a design document is not decided by its length. It is decided by whether the purpose, constraints, acceptance criteria, and non-goals are correct. Specifying file names and implementation details before the repository has been inspected only freezes a wrong assumption in place.

Why Claude Code handles the implementation

A correct design document still meets an existing repository with its own conventions, dependencies, uncommitted changes, tests, CI, and past design decisions. So Claude Code does not execute the instruction document directly. It first inspects the current branch and working tree state, the existing implementation of the target feature, the related types, APIs, and data models, the existing lint, type check, test, and build commands, the checks GitHub Actions actually runs, the difference between pre-existing failures and new ones, and whether the instruction document contradicts the repository. Where there is a contradiction, it is reported before implementation: what the document assumes, where the code differs, the available options with their blast radius, and the recommended option given the goal. That prevents the failure mode where a wrong instruction is implemented faithfully. Claude Code’s job is to connect a conversational proposal to real code, commands, and test results.

Why Codex audits the near-final diff

Codex does not implement in place of Claude Code. It answers a fixed set of questions: does this meet the user requirement, is there any change not in the instruction document, does it harm authentication, authorization, billing, or customer data, does it break API or data-structure compatibility, is there a logical defect the passing tests miss, are error handling, boundary conditions, and operational failure paths considered, and is the implementation larger than the change requires. Asking Codex to “look at everything and fix what needs fixing” collapses the separation. If the reviewer also edits, it stops being clear who owns the implementation, and out-of-diff improvements, incidental refactoring, and repeat reviews of the same commit consume tokens and time. The auditor stays a read-only finder of serious defects.

What the split delivered

• Vague requirements surface before implementation, while the cost of fixing them is still small.
• What was asked for, which assumption changed during implementation, and which checks ran all stay in Markdown and the Git diff rather than in someone’s memory.
• Self-approval is harder to reach. Design, implementation, and evaluation in one continuous conversation tend to evaluate against the AI’s own chosen approach.
• A human can manage quality through acceptance criteria, forbidden changes, risk, and accept-or-reject decisions without hand-writing every line.
• A good instruction document becomes an asset. Failure causes, correct constraints, verification commands, and preventive measures move into project rules, CI, and quality gates.
• There is one deliberate pause before release, which keeps “it works” separate from “it may be published”.

What it cost

The serial arrangement carries a handoff tax. ChatGPT, Claude Code, and Codex each re-read the same background, and the longer the design document grows, the more effort the implementer and the auditor spend locating what they need. Using several AIs is not by itself an efficiency gain.

• A ChatGPT design can drift from the repository. Specifying file layout and function names without having inspected the code pulls Claude Code toward a wrong design.
• Introducing independent review only just before completion means architecture and data-model problems are found last, when the fix is widest.
• Sending problems that format, lint, type checking, unit tests, or a build can decide to an LLM raises cost and dilutes attention on the serious findings.
• A false sense of safety appears. Several AIs reaching the same conclusion does not make it correct; given the same vague requirement, different AIs can err in the same direction.
• When both Claude Code and Codex edit the same branch, ownership of a change becomes ambiguous, and fixes get reverted, unrelated diffs get mixed in, and the same problem is investigated twice.
• AI keeps producing improvement candidates, so “just one more fix” blurs the line between a release blocker and a post-release improvement.

Review reliability is decided by clear ground truth, the latest diff, findings with a stated basis, separated authority, and an exit condition. It is not decided by how many AIs looked at it.

What actually happened at Netsujo

A request to add a customer goal field to an existing analysis screen grew, through successive improvement instructions, into a redesign of the whole screen. The cause was less the capability of the AI than our failure to write down that the existing screen structure was to be preserved and that only the input field and its connection to the analysis were in scope. A vague request was implemented at high speed. Non-goals and invariants became mandatory fields in the Task Contract after that. Separately, letting both Claude Code and Codex make changes on GitHub made the state of the main branch and the uncommitted changes hard to track, because the owned area, working branch, source of truth, and integration owner were undefined. The standard now is one task per branch, no direct changes to main, Codex read-only as a rule, implementation fixes returned to Claude Code, merge, push, and deploy only on an explicit human decision, and no double review of the same head SHA.

The defect that passed every check

The customer-facing analysis screen for Netsujo SIGNAL lets a user pick a period and a comparison condition, including “no comparison”. The expected behaviour is unambiguous: when “no comparison” is selected, no section uses previous-period data. After the implementation, comparing the Task Contract against the diff with Codex surfaced the real state. Some sections honoured the setting. Others kept receiving the previous-period specification. The screen rendered, the type check passed, the existing tests passed, and the build succeeded. The code was not broken. The meaning of some results no longer matched the condition the user had chosen. This class of defect is hard to catch with type checking: if the previous-period value has the right type and the function accepts it, TypeScript has nothing to say. If the existing test only asserts that data can be fetched, a contract violation like “pass no previous period in any section when comparison is off” goes straight through. What a diff review examines is not syntax but the flow of data. The fix did not stop at the missing application sites. We added a regression test for the no-comparison case, a script that checks for unapplied analysis conditions, and wiring for both into the package scripts and GitHub Actions CI. What mattered was not that Codex found the problem once. It was converting the finding into a check that stops the same defect mechanically next time. Fixing only what an AI review comment mentions lets the same defect reappear in another section.

A check you wrote has the holes you cannot imagine

Moving a finding into a check is not the end. After adding the check script above, a later diff review reported a defect in the check itself: the comment-stripping logic treated the double slash inside a URL in a string literal as the start of a comment and silently skipped the rest of that line. The check appeared to work and quietly missed lines written a particular way, and the miss never appeared in its output. Whoever writes a check is poorly placed to imagine what it cannot catch, because the tests and the check script both come from the same person, and no check gets written where the blind spot is. So check scripts are review targets too. A green CI run is not evidence that a check worked as intended. Every time a check is added, a second pair of eyes has to establish what it fails to catch.

Three AIs are not needed every time

The reasonable order is: a human defines the purpose and the risk; a Task Contract is written; Claude Code inspects the repository; contradictions are reported before implementation; work proceeds one task per branch; machine checks run, meaning format, lint, type checking, related tests, a full test run only when needed, and a build; the change risk is classified; Claude Code fixes; machine checks run again; a single focused re-review happens only if required; and the human decides on release.

This allocates reasoning budget by risk rather than by the number of AIs. A very large change is not handed to Codex in one piece; the pull request or the change unit is split first.

From a detailed order to a verifiable contract

Non-goals, invariants, and stop conditions matter most. If adding a goal field must not replace the existing analysis screen, write that down. If three rounds of fixes produce the same failure, hand it back to a human instead of continuing. Weakening tests, type settings, or security settings in order to obtain a PASS is prohibited. Raising an AI’s self-correction ability depends more on clear pass and stop conditions than on more iterations.

From full re-scan to diff-limited review

Codex code review on GitHub targets the pull request diff and can read repository-specific review rules from the agent configuration file. Using that, what we hand over is limited to the Task Contract, the base and head SHAs, the target diff, the minimum related code, a summary of the machine checks, the risks to focus on, and the instruction to return serious findings only.

• No unbounded re-exploration of the whole repository.
• No long successful logs.
• No aimless full text of lockfiles or generated output.
• No refactoring proposals outside the diff.
• No duplicate review of the same head SHA.
• No automatic commit, push, merge, or deploy by the reviewer.

The response format is specified too: per finding, the severity, the file and location, the contract that was broken, the impact on users or operations, the basis, the minimal fix, and the missing regression test. Where nothing applies, the reviewer writes “no applicable findings” and is explicitly not allowed to write “this is safe”. Those are not the same statement. Nothing was found within the scope reviewed, the evidence supplied, and the criteria used. Release still needs machine checks, authority design, operational confirmation, and human approval. AI review is one layer of quality assurance, not a replacement for tests, CI, branch protection, or human approval.

Separate the roles inside Claude Code

Calling an external reviewer every time is not required. Project rules, skills, subagents, and hooks can hold the separation inside Claude Code: a Builder that implements and fixes, a Deterministic Judge that runs lint, type checking, tests, and builds, an Independent Judge that compares the Task Contract against the diff read-only, and a Manager that owns the iteration count, the failure signature, and the stop conditions. Low and medium risk changes finish in that internal loop, and the external audit is reserved for authentication, billing, customer data, and important diagnosis logic. Separating authority and evaluation criteria inside one environment first, and adding an external audit only where it is needed, balances cost and reproducibility better than keeping different models permanently running.

Measure whether it actually works

A feeling that three AIs raise quality does not justify the operating model. At minimum we record lead time from implementation to release, first-pass success rate, the average number of fix rounds, the number of Codex invocations, input, output, and total tokens per review, duplicate reviews of the same SHA, serious defects found, how many of those moved into regression tests or CI, incorrect findings, problems the machine checks caught first, defects discovered after release, rollbacks, and the human time spent on integration decisions. Those are compared over 30-day windows, separating changes that used Codex from those that did not. If the serious-defect detection rate is unchanged and only the cost is large, the review conditions tighten further. If post-release defects are being prevented in high-risk areas, the reasoning budget concentrates there. External material is also useful: OpenAI published a Datadog case study in January 2026 in which past pull requests linked to incidents were replayed for Codex review, and the engineers who had handled those incidents judged that the finding would have made a difference in more than ten cases, roughly 22 percent of those examined. That is a single company case study published by OpenAI, not a general detection rate, but it does support using an additional review layer for system-wide interactions that deterministic rules struggle to catch. Without measurement, splitting work across AIs stops being a development process and becomes a belief.

More AI does not distribute human responsibility

Lining up ChatGPT, Claude Code, and Codex does not produce a team of three engineers. Each is a probabilistic system that carries no responsibility. Given a wrong goal, it will refine the wrong goal. Given weak acceptance criteria, it will call a presentable but unfinished state a PASS. With nobody to stop the release, it will keep producing improvement candidates. What stays with the human is deciding what to build, what not to change, which risks to accept, whether a finding is a specification change or a defect, where to stop fixing, and who answers for it after release. A development lead in the agent era writes less code and designs more roles, authority, evaluation criteria, and stop conditions.

Conclusion

• Structure the instruction document as a Task Contract.
• Have Claude Code confirm the reality of the repository before implementing.
• Run machine checks before any LLM.
• Limit Codex to read-only audit of high-risk diffs.
• Keep integration responsibility and the stop decision with the human.

Products and models will change. What remains is the operating design: which role receives what, what it is not given, on what basis something passes, and where it stops.

Frequently asked questions

Why separate ChatGPT from Claude Code?

ChatGPT concentrates on business requirements: customer value, purpose, constraints, and acceptance criteria. Claude Code implements while checking the real repository, dependencies, tests, and CI. Separating them makes the requirement and the result easier to compare.

Should every change go through a Codex review?

Not recommended. Low-risk changes such as wording, articles, and limited CSS are often adequately covered by machine checks. Concentrate the review on authentication, billing, customer data, databases, public APIs, and important diagnosis logic.

Does using different AIs make the review independent?

Different products do not guarantee independence. It takes a separate context, explicit evaluation criteria, read-only authority, the latest diff, findings with a stated basis, and a defined stop condition.

Should a human still read the code?

The need to read every line by hand goes down, but purpose, risk, acceptance criteria, accepting or rejecting findings, and the release decision stay with the human. For high-risk changes, a human also reads the important diffs and test results.

Does letting AIs auto-fix each other raise quality?

Increasing iterations alone raises cost, wrong fixes, and infinite loops. Define the maximum number of fix rounds, the stop rule for a repeating failure, the budget ceiling, and the escalation condition first.

AI agent development and operations incident logThe series hub lists every incident in this record and the order to read them in.

A repeated instruction is a system bugThe separate article covers moving repeated human attention into a Controller, Evidence Ledger, and Gates.

The same work item, different chat namesThe separate article covers how work is identified when it moves between tools.

Continue the series

Next: #01 Give AI review a budget and an exit condition

Back to all 12 episodes of the AI Agent Development Incident Log