How much should an AI agent be allowed to do? Four controls for consequential actions
Netsujo Inc. ·
Imagine approving version 3 of a campaign announcement. The agent later consults an old list and selects another page with the same name. An operations lead needs answers to two questions. Can the agent explain the rule? What will actually happen when the publishing service receives the request? The second answer depends on which targets the service accepts and how the team verifies the result.
How a decision error turns into a business loss
A request to change only what is necessary leaves room for interpretation. A deadline can compete with a restriction on production changes. A long task can resume with its goal intact while losing a budget cap or an approved revision from the summary. External text can influence the next action when the agent reads it as part of its research.
These are four ways a proposed action can diverge from the intent. Excessive authority is a fifth concern that determines whether the action can cause an effect. If an agent assigned to gather information can also send messages, an instruction embedded in research material can lead to an external send. OpenAI's instruction-hierarchy research studies how models can prioritize instructions. OWASP covers prompt injection and excessive agency as distinct risks. Use these five questions to trace possible failures in a workflow. OpenAI · OWASP Prompt Injection · OWASP Excessive Agency
Give each control a clear owner
| Control | Owner | What to check for a publication |
|---|---|---|
| Contract | Business owner and approval record | Pin the site, page ID, revision, action, and approval expiry |
| Runtime enforcement | Publishing service | Compare the actual target and revision with the approval, and reject requests that cannot be validated |
| Independent verification | Reviewer using a separate read path | Read the public URL and confirm that the approved content is live |
| Stop and recovery | Execution control and operations | Query the original operation ID after a lost response and hold resends while the result is unknown |
Suppose the service records a scheduled publication and then loses the response. Sending the same copy again can create another reservation. If readback shows a single matching reservation, the team can proceed using it. While the outcome is uncertain, hold that operation and continue independent work such as copyediting. Message delivery history or a billing provider's transaction record can serve the same role for other actions.
Separate subscription allowance from metered charges when defining spend authority. For approved metered use, identify the billing account, currency, maximum cost, validity period, and stop condition. Use a route that can enforce the cap and reserve capacity before sending. During verification, distinguish the reservation from the charge recorded by the billing provider and account for concurrent work while the reservation is held. Hold the paid action when the cap cannot be established.
What the tests showed
In a synthetic check of published ARC 1.0.1, three changes to declared inputs were rejected against the earlier binding. Changing only a fictional tool definition left the declarations consistent and passed the declaration check (G1) with execution_authorized:false. ARC checks the consistency of supplied declarations. The system carrying out the action is responsible for authorizing it at execution time. ARC reproduction
In a separate SQLite fixture, a changed target reached a protected record when the runtime check was disabled. With the check enabled, zero effects were recorded. After a simulated lost response, a separate read connection found one write. The application held the operation when its read path was unavailable. The fixture used fictional data in one process. Production acceptance needs the real destination, its durable records, and appropriately separated credentials.
The reproducible Python script (fixed commit from merged ARC PR #118) and six-case result TSV are publicly readable. The fixture uses only the Python standard library. The application and test oracle share one process and read function, while reader unavailability is simulated by skipping the application readback. Network=0 and resend=0 are constants by construction. Real services, credential separation, crash recovery, and duplicate-operation branches were not tested. ARC PR #118 has been merged; this is not independent production acceptance.
What to decide first
Start with one delegated action. Record what was approved, where it is checked, where the result is observed, and what stops if the result cannot be established. Run a permitted operation and an operation whose target changes after approval. Record both the response and the actual destination state.