On 24 September 2026, ten researchers submitted the preprint “Who Holds the Pen? Let Specifications, Not Agents, Sign Off”. It examines a stage that agent systems can easily obscure: the same model performs a task, judges whether it has met the requirements, and decides when to declare completion. From SkillsBench task prompts, workspace information and injected skill instructions, the authors extracted 509 source-grounded requirements. Across seven models, they report that 79.6 to 86.4 per cent of those requirements were satisfied, while agents’ claimed completion rates exceeded official evaluator pass rates by 28.7 to 37.9 percentage points. Their proposed system, SpecHarness, compiles verifiable requirements into source-linked obligations and allows an external runtime to update state and finalise a task only on qualifying evidence. This is a preprint that has not undergone peer review, and its experimental results have not yet been independently reproduced in public.
The philosophical issue is not simply that models make mistakes. Any actor can misreport progress. The deeper question is what an AI agent is doing when it says that a task is complete. The statement may be a report about the world, a signal requesting that execution stop, or a command that software treats as sufficient to close the task. These roles are not equivalent. A report may be true or false. A stop signal may have a real causal effect. But causing a process to stop does not by itself confer the authority to establish an accepted state of completion.
Speech-act theory offers a direct distinction. Some utterances primarily describe an existing fact, such as “the file is in the specified folder”. Others can create a social fact under suitable institutional conditions, as when an authorised person declares a meeting closed. The latter do not work merely because their grammar resembles a declaration. The speaker must have the relevant standing, the time and procedure must be appropriate, and the required preconditions must hold. An unauthorised person saying “the meeting is closed” may cause observers to leave, but does not thereby close the meeting validly as an institutional matter.
An AI completion claim has a similar structure. An agent’s output may cause an orchestrator to stop, a user interface to turn green or a downstream process to begin. That shows the output has operational force granted by the software design. Operational force, however, is causal permission. It does not prove that the file was saved, the tests passed, the data remained intact, the references were checked or the user accepted the result. Treating the power to stop a workflow as authority to establish completion compresses action, evidence and institutional acceptance into a single word.
Completion should therefore not be understood as an internal model judgement alone. It is a relational state: the requirements are specified; the relevant actions occurred; the world or artefact corresponds to those requirements; the evidence comes from a channel capable of observing that state; the validation remains fresh for the current version; and a qualified process or person accepts the evidence. If any link is missing, “complete” is at most a claim awaiting examination. Even when the claim happens to be true, accidental correctness does not establish authority retrospectively.
This is also why asking a second model to check the first does not necessarily solve the problem. The second model may be separate from the executor while relying on the same incomplete log, mistaken assumption or invisible environment. Independence, competence and authorisation are distinct conditions. A qualified verifier need not be human, nor need it be another model. It might be a deterministic test, a filesystem readback, a database constraint, a hardware sensor, a professional reviewer or user acceptance. What matters is whether the evidence channel is suited to the state being established and whether failure can prevent that state from being committed.
In terms of Sustenesis Theory, Difference first distinguishes the task requirement, the agent’s action, the agent’s self-report, the actual environmental state and the completion state accepted by the system. These may correspond, but they are not identical. Constraint is not merely a generic limit; it is the set of formative conditions specifying which evidence may enter a judgement, who may update state and when revalidation is required. Sustained Coherence asks whether the relationship among specification, action, evidence and acceptance remains intact through execution, modification, readback and final commitment. A single output saying “done” does not possess that sustained structure.
The paper’s external runtime is a technical attempt to preserve these differences. Externalisation should not be romanticised, however. The authors explicitly leave ambiguous, subjective, conflicting or unverifiable requirements as advisory, and hard enforcement covers only source-grounded, observable and verifiable requirements. A validator may be defective, and an institution may give authority to an unqualified role. Moving authority outside the model creates an inspectable boundary; it does not guarantee that whatever stands on the other side is always correct.
My judgement is that an AI agent’s completion claim should be treated by default as a request to close the task, not as proof that closure has been achieved. In low-risk, subjective or exploratory work, a user may accept that request directly. Where a task concerns files, transactions, medical matters, public decisions or other high-consequence states, acceptance should depend on evidence and authority proportionate to the task. A human click is not automatically authoritative either: if the person cannot see the relevant state, lacks the competence to review it or can only provide a formal acknowledgement, human acceptance may be just another unsupported completion claim.
This judgement does not require an elaborate audit system for every activity, nor does it deny that models can accurately assess their own output. It draws a narrower conceptual boundary: self-assessment may count as evidence but cannot, solely by referring to itself, become the final basis of acceptance; a completion claim may initiate verification but cannot alone create the fact of completion. Open questions remain about suitable evidence for open-ended and value-laden tasks, proportionality between verification cost and risk, and who is qualified to set completion conditions for different kinds of work. The current preprint offers a testable engineering proposal, but one benchmark study cannot settle those institutional questions.
References
Haiqing Li et al., “Who Holds the Pen? Let Specifications, Not Agents, Sign Off”, preprint, 24 September 2026. https://arxiv.org/abs/2609.29921
Xiangyi Li et al., “SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks”, preprint, 13 February 2026. https://arxiv.org/abs/2602.12670
Stanford Encyclopedia of Philosophy, “Speech Acts”, substantively revised 24 September 2020. https://plato.stanford.edu/entries/speech-acts/
Stanford Encyclopedia of Philosophy, “Social Institutions”, 2024 revision. https://plato.stanford.edu/entries/social-institutions/
Discover more from Geoffrey Chen
Subscribe to get the latest posts sent to your email.