A problem for people who use AI
What do you check AI output and completion reports against?
Development practitioners, individuals who leave AI settings to it, and general users of generative AI
Matching done; reduction in burden unconfirmed
A practice on October 22 and a retrospective from February to October. The developer survey is June–August, and the general-user burden survey is an observation of February.
Editorial check: 2026-10-23
What people are paying attention to
A developer survey listed easy verification as a condition for trusting AI. An October personal record also has an example of checking reports and explanations of reasons.
What is hard
Reports of nonexistent commits and passing tests, and mismatches between the reasons for settings and operation records, have been made. A background survey of general users also has answers about the checking burden.
Workarounds being tried
There are examples of matching AI reports against deliverables, tests, design documents and operation history. A pre-save check of the reasons for settings is a plan for next time; what has been done is the check after saving.
What the sources confirm
In personal records they found missing implementation and inconsistent explanations of reasons. How much these steps reduce checking time or misses has not been measured.
What is still unclear
How checking time, misses and rework change when verification steps are added is unconfirmed. Records compared under the same conditions are needed.
Workarounds and results in more detail
Match deliverables against completion reports
Reported as done
People found missing implementation and similar problems. Comparison results for defect rate and total working time are not published.
Scope and limits: One development practitioner. Checks files, tests and the design document, and limits which agent writes to Git.
Match the reasons for settings against operation history
Done after saving
Reported finding two inconsistent explanations of reasons. The general rate of occurrence is unknown.
Scope and limits: A self-report from one Windows PC and one run. It is not the result of independently verifying the original logs.
Check the reasons before saving
Plan for next time
No result of carrying it out has been shown yet.
Scope and limits: Stated by the person who rebuilt the settings as a step they want to do next time.
Read with care
- Developers' trust, the burden on general users in Japan, and individual failure records differ in subjects, questions and timing. They cannot be summed or compared as occurrence rates.
- In the background survey of general users, the burden of checking and choosing is not the majority experience. The relationship with checking frequency is also a correlation.
- Trust in being able to verify is self-reported and is not a result measuring the AI's actual accuracy.
What to check next
Records comparing, under the same working conditions, checking time, misses and rework before and after introducing a checking step.
Decide the working method and the point where a person finally checks
A development practitioner who uses several AIs changes the approach for features with settled specifications, designs that need running to check, and small bug fixes. For small fixes, they skip the design document but keep tests and review by a separate model. After implementation they match the design and the deliverable item by item, and check the behavior themselves in the development environment.
In another job, they look back that the burden up to finishing the prototype and the later fixes were large, so the speed of the implementation part did not translate into a shorter whole. Editorially, this can be read as a case that matches the type of change, the conditions to check and the checker in advance. It is the person's own operations record, and the effect compared with other methods under the same conditions is unconfirmed.
Questions to think with
- For a recent change, can you match the requested conditions to the implementation and execution results item by item?
- Looking at the whole time including prototyping, checking and fixing, which step needs to be lightened?
Editing that checked the primary sources. The questions to check in your own work are proposals for verification.