When Generation Becomes Cheap, Why Does Verification Become the Bottleneck?

Geoffrey Chen

After AI Enters the Workflow · Season One: “From Answering Questions to Participating in Work” · Article 6

A team once produced ten substantive reports each month. Drafting was slow, so review capacity roughly matched production. After adopting generative AI, the team can prepare thirty plausible drafts. Management initially describes this as a threefold productivity gain.

The reviewers do not experience a threefold gain. They now face more factual claims, more citations, more charts and more recommendations. Each document looks polished enough to deserve attention, but its claims still have to be checked against source material, calculations rerun, legal implications reviewed and final recommendations approved. A queue forms. Some drafts are reviewed superficially; others wait. Eventually, people begin to assume that fluency is a proxy for correctness because there is no longer enough time to inspect everything.

Generation became cheap. Verified completion did not.

This is one of the most important changes AI introduces into work. When the cost of producing candidate outputs collapses, the scarce resource moves downstream. The bottleneck becomes the ability to establish that an output is correct, appropriate, authorised and safe to adopt.

Generation and verification are not symmetrical forms of labour

To generate a statement, a model needs to produce one plausible continuation. To verify the statement, a reviewer may need to identify the relevant claim, locate an authoritative source, confirm the applicable date and jurisdiction, inspect qualifications, reproduce a calculation and consider contrary evidence.

A line of code can be generated in seconds. Verifying it may require understanding the requirement, constructing tests, examining side effects, checking security and observing behaviour in an environment the model never saw. A contract summary can be produced from a document in one pass; confirming that it has not omitted an exception may require reading the entire agreement and related schedules.

The asymmetry becomes sharper when output is open-ended. There may be many acceptable ways to draft a paragraph, but a single factual assertion can be wrong in numerous subtle ways. Verification has to look beyond the answer’s surface and ask which failure modes matter in the particular context.

That is why an increase in generation capacity does not automatically produce an equal increase in completed work. The organisation must decide what counts as proof of completion and fund the processes that obtain it.

Why people overestimate work that looks finished

AI output often arrives with the visual signals of completion: headings, balanced paragraphs, professional tone, tables and a confident conclusion. Traditional unfinished work announces itself through notes, blanks and rough language. Generated work can conceal its provisional status.

This creates two cognitive and organisational pressures. First, reviewers may anchor on the proposed answer and search mainly for obvious errors rather than reconstruct the problem independently. Second, managers may count documents produced rather than documents validated, making careful review appear to be the source of delay.

The “jagged frontier” research is relevant here. In a field experiment involving consultants, AI improved performance on tasks within its capability boundary but could reduce accuracy on a task outside that boundary. The tasks appeared related, which made the boundary difficult for users to recognise. Harvard Business School, “Navigating the Jagged Technological Frontier”

The lesson is not simply to distrust AI. It is to avoid using polished form as evidence that the task fell within the system’s reliable range.

Generation speed creates verification debt

Software teams speak of technical debt: shortcuts that make delivery faster now but create future maintenance costs. AI-enabled organisations can accumulate verification debt in a similar way.

Verification debt arises when candidate outputs enter circulation faster than their claims, permissions and consequences can be checked. The debt may appear as unreviewed documents, code merged with weak tests, duplicated records, undocumented assumptions, provisional recommendations treated as decisions or communications sent before approval.

The debt has compounding effects. Later work cites earlier unverified work. A generated summary becomes context for another model. An incorrect field is copied into a database and then retrieved as an authoritative fact. Once the original source is separated from the derived outputs, correction becomes more expensive.

A work system should therefore make provisional status visible. Drafts generated by AI should not silently become records of decision. Important claims should retain provenance. Review queues should be measured, and high-risk outputs should not be produced at a rate that makes meaningful inspection impossible.

AI can participate in verification, but it must not monopolise it

AI can also reduce verification costs. A second pass can extract claims, compare versions, run tests, identify missing citations, calculate totals or ask questions designed to expose inconsistency. Chain-of-Verification research found that having a model formulate and answer separate verification questions before revising an answer reduced hallucination in evaluated tasks. Dhuliawala et al., “Chain-of-Verification Reduces Hallucination in Large Language Models”

Agent evaluation methods can test whether a system reached a target state, complied with constraints or used tools correctly. Anthropic’s guidance on agent evaluations emphasises building tasks, graders and traces that reveal process failures rather than relying only on the final response. Anthropic, “Demystifying evals for AI agents”

These methods are valuable, but they should introduce different evidence or failure detection—not merely ask the same generative system whether it agrees with itself. Reliable verification combines techniques with different weaknesses: deterministic checks, source inspection, tests, independent models where useful, sampling, expert review and post-deployment monitoring.

No single method proves everything. Unit tests do not prove that a requirement was correct. Citations do not prove faithful interpretation. Human review does not work if the reviewer lacks time, knowledge or power to reject. Verification is a system of converging checks.

Verification effort should follow risk

Not every output deserves the same burden. Asking AI to reformat private notes is different from using it to recommend medical treatment, approve a payment or alter production infrastructure.

A practical risk classification considers at least:

  • the magnitude of harm if the result is wrong;
  • how difficult the error is to detect;
  • whether the action is reversible;
  • how many people are affected;
  • whether rights, money, safety or public commitments are involved;
  • whether the output will become input to further automated work.

Low-risk, reversible tasks may be checked by automated rules and occasional sampling. Medium-risk work may require named review before adoption. High-risk or rights-affecting decisions need qualified human oversight, documented reasons, access to evidence and a route to stop or reverse the action.

Article 14 of the EU AI Act sets human-oversight requirements for high-risk AI systems, including the ability of overseers to understand relevant capacities and limitations, detect anomalies, avoid over-reliance, interpret outputs and disregard, override or interrupt the system. EU AI Act, Article 14

The provision is legally specific, but its operational insight is broader: human oversight is a capability and a position in the workflow, not the mere presence of a person.

Redesign throughput around accepted outcomes

If verification is the bottleneck, organisations have several choices other than asking reviewers to work faster.

They can reduce generation volume by requiring a clear purpose before a draft is created. They can make outputs more verifiable by requiring structured evidence, citations, tests and confidence labels. They can route low-risk and high-risk work differently. They can allocate expert review to the claims that matter most and automate checks with objective criteria. They can also improve source systems so reviewers do not repeatedly reconstruct the same factual basis.

Most importantly, they can change the metric. Instead of counting prompts, drafts or lines of generated code, count outputs that passed the relevant checks and entered use without creating unacceptable downstream correction work.

NIST’s AI Risk Management Framework and Generative AI Profile organise practice around governing, mapping, measuring and managing risk. The approach helps prevent “verification” from becoming an improvised final glance by requiring organisations to consider context, testing, monitoring and accountability across the lifecycle. NIST AI Resource Center and NIST Generative AI Profile

Conclusion: productivity must be measured in verified outcomes

Generative AI can dramatically reduce the cost of producing candidate work. That is a real productivity opportunity. But a candidate is not yet an adopted result.

When verification capacity remains fixed, faster generation can reduce quality, enlarge backlogs and spread untested claims into later systems. The apparent gain is then partly a transfer of labour and risk from authors to reviewers, users and future maintainers.

The governing measure should be:

Productivity is not the amount of material AI can generate. It is the amount of useful work that can pass proportionate verification and be responsibly adopted.

This shifts attention from spectacular output volume to work-system capacity. It asks whether claims can be traced, tests can be run, reviewers can exercise real judgement, risky actions can be stopped and errors can be corrected. When those capacities grow alongside generation, AI can improve throughput. When they do not, verification becomes the bottleneck—and the queue of polished drafts becomes a new form of organisational debt.

Primary sources and further reading

Continue reading: Explore the After AI Enters the Workflow series.

Discover more from Geoffrey Chen

Subscribe to get the latest posts sent to your email.