Why Is Using the Most Powerful Model for Every Task Bad Organisational Design?

Geoffrey Chen

After AI Enters the Workflow · Season Two: “From Personal Tool to Organisational Capability” · Article 7

An organisation creates a single AI platform. Translation of headings, email classification, spreadsheet checking, contract drafting and analysis of complex complaints all go to the most capable available model. The design appears simple: one contract, one interface, one training program and no argument about which model to choose.

Months later, low-risk tasks consume a growing budget, simple classifications wait for relatively slow responses, and sensitive information travels through a general path that did not need to see it. When the service fails, every AI-assisted activity stops. The organisation still adds human review to high-risk tasks, so the strongest model has not removed the differences among workflows.

“Use the strongest model by default” treats capability as the only objective. Work is also constrained by cost, latency, data, verifiability and consequence.

A task needs sufficient and appropriate capability, not maximum capability

Sorting customer messages into three explicit classes and interpreting a complex contract with conflicting provisions do not require the same system. Rules, a conventional classifier or a smaller model may handle the first reliably. The second may require stronger reasoning, more context and professional review.

Using a general large model for a simple task pays for unnecessary capability and may increase output variation and review demand. Using an inadequate system for a consequential task to save money can make the error cost exceed the saving.

“Sufficient” must be established through use-case evaluation, not inferred from model size. A small model can be stable on a narrow task. A large model will still fail when its records are wrong.

Compare at least five dimensions

The first is quality: does the configuration meet the standard on the organisation’s real samples and failure conditions?

The second is latency: does user waiting time interrupt work, or can a batch process tolerate a slower and more thorough response?

The third is cost: not only model calls, but retrieval, tools, review, monitoring and repair.

The fourth is data and deployment: where information is processed, how long it is retained and whether a local or dedicated environment is required.

The fifth is controllability: can output be structured, tested, constrained and rolled back?

Google Cloud’s official guidance on agentic design recommends selecting architecture according to task openness, latency and performance, cost, and human involvement. It also notes that predictable or structured work may not need an agentic system at all. Google Cloud, “Choose a design pattern for your agentic AI system”

The same principle applies to model selection. Complexity should follow the task rather than platform preference.

Tiered routing resembles real work more closely

An organisation can build tiers. Deterministic rules handle what can be calculated explicitly. Smaller or faster models perform low-risk classification and format conversion. Stronger models address complex, open-ended analysis. Steps affecting rights, major funds or public commitments go to qualified people.

The system can also escalate uncertainty. Incomplete input, unresolved conflict, a failed confidence condition or a high-risk category should route to a stronger model or a human instead of forcing completion on the original path.

OECD analysis of AI markets distinguishes model segments with different capability and price characteristics and emphasises switching as a condition of competition. OECD, “Developments in Artificial Intelligence Markets”

Those differences become practical choice only when the organisation can route tasks. One universal default creates another single point of dependency.

Multiple models introduce their own complexity

Tiering is not free. Models differ in outputs, limits, updates and data terms. The router can misclassify work. An attacker may manipulate a low-risk classification so sensitive content reaches a weaker path.

An organisation should not operate a dozen models merely because it can. Every added component needs a purpose, tests, monitoring, supplier management and an exit route.

A practical starting point may contain only three layers: non-generative tools, one tested general model and one enhanced path for complex work. Further division should follow evidence.

The NIST AI RMF applies risk management according to context rather than assigning identical controls to every AI use. NIST, “AI RMF 1.0” Model allocation should follow the same proportional principle.

Treat the strongest model as scarce specialist capacity

When every small task consumes the strongest model, the organisation may encounter cost, quota or latency pressure at the moment it genuinely needs higher capability. Placing it on an escalation path concentrates the resource on difficult exceptions, consequential synthesis and work that is hard to verify.

This resembles expert allocation inside an organisation. The most senior professional does not complete every form, but the process knows when escalation is mandatory. Routing is not only about reducing the price of each call. It protects scarce capability and makes exceptions visible.

The UK Government AI Playbook includes “use the right tool for the job” among its principles, together with understanding limitations, lifecycle management and meaningful human control. UK Government, “AI Playbook”

Evaluate total cost per accepted outcome

A cheap model that creates more errors and human rework may not be cheap. A frontier model that improves accuracy on a low-risk task from 98 to 98.5 per cent while multiplying cost and latency may not be worthwhile.

The comparison unit should be a verified work outcome. How much model, infrastructure, human review and error correction does each accepted result require? What is the likelihood of a serious failure and the time needed to repair it?

These measures should be recorded by task tier. The organisation can then see where to upgrade, where to downgrade and where a generative model should not be used.

Routing decisions must be explainable and auditable

When software chooses which model receives a task, it allocates cost, data and risk. The organisation needs to know the basis of the route, how misclassification will be detected and whether a user can request escalation.

Classification should rely where possible on observable conditions—information sensitivity, monetary value, external communication and effects on individual rights—rather than another opaque model alone. Sampling outcomes from every tier can reveal complex work left on a weak path and low-risk work escalated without need.

The activity record should retain the routing rule and model versions in force. After an incident, the organisation must inspect not only the answer but why the task entered that path. Without this layer, a multi-model platform turns the opacity of one model into an invisible relay among several.

The router can also fail and should not operate as an invisible traffic officer. The organisation should record why a task was sent to a model, test boundary inputs and let users escalate when risk has been underestimated. For consequential work, an intelligible conservative rule may be safer than allowing another opaque model to decide silently which system should judge. Model allocation reduces risk only when routing errors are observable, reviewable and rapidly correctable.

Conclusion: model allocation should follow the division of work

The strongest model can be an important organisational resource. Making it the default for everything raises cost, concentrates failure and hides the different controls tasks require.

A better rule is:

Use the least complex system that reliably meets each task’s risk, quality, latency, data and cost requirements, with a clear escalation path for uncertainty and consequence.

This is not “small models first” as an ideology. It rejects a capability ranking as a substitute for work design. A mature AI platform knows when no AI is needed, when an ordinary model is sufficient, and when a stronger model and human judgement are indispensable.

Primary sources and further reading

Continue reading: Explore the After AI Enters the Workflow series.

Discover more from Geoffrey Chen

Subscribe to get the latest posts sent to your email.