Short answer
AI cannot become the party that absorbs responsibility for an organisation. Allocate responsibility by real control: the person or institution choosing the use remains accountable for the service and remedy; the provider for its product, claims and defects; the deployment team for data, permissions, testing and monitoring; the professional decision-maker for judgement that cannot be delegated; and the operator only for matters they had the competence, information and authority to control. An affected person should not have to identify which model in a complicated supply chain failed before receiving correction.
“The AI did it” is not an account of responsibility
When credit is wrongly denied, a message discloses data or money goes to the wrong account, “the algorithm decided” does not identify who chose the system, configured permissions, approved the use, ignored warnings or can provide remedy. A technical description cannot substitute for organisational responsibility.
AI has no assets from which to compensate, cannot be professionally disciplined, and cannot be required to explain and repair harm for a person. Legal liability varies with jurisdiction, contract, industry and fact; this article is not legal advice on a specific case. A sound governance default remains possible: the organisation delivering a service or decision cannot transfer its fundamental responsibility to a model.
Australian Government material on the legal landscape explains that existing privacy, consumer, discrimination, corporations, intellectual-property and sector laws can apply to AI activities. Adoption does not create a zone without existing duties. Australian Government: The legal landscape for AI in Australia
Separate the different meanings of responsibility
“Who is responsible?” contains at least five questions. Who must prevent the problem? Who can control operation? Who investigates cause? Who corrects or compensates the affected person? Who bears internal discipline, contractual cost or legal liability? These need not have the same answer.
Procurement chooses a provider, engineers configure the system, a business manager approves automation, staff handle exceptions, and a provider runs the model. An incident can involve failure at several control points. Responsibility needs allocation, not a convenient scapegoat.
Also distinguish responsibility to an affected person from internal allocation. A customer should face one clear service owner. That organisation can later seek contractual contribution from a provider. The customer should not first have to understand the whole model value chain.
The adopting institution owns the use and external outcome
An organisation decides the context, data, permissions, accepted risk and delivery of results to customers or staff. It is normally closest to the affected person and best able to pause use, correct a record and provide remedy. Its external responsibility cannot be removed casually because a third-party model generated content.
This does not mean the organisation is solely responsible for every hidden provider defect. It means the institution must perform risk-proportionate due diligence, testing, human control, monitoring, challenge and incident response, and remain a visible accountability entry point.
If an organisation cannot understand the most basic data handling and limitations of a product but deploys it for high-impact decisions, that lack of understanding is itself a governance problem. A provider that cannot be evaluated may be unsuitable for the use.
Providers are responsible for product capabilities, limits and claims
A model or application provider controls training, system design, updates, security mechanisms, logs and terms. It should accurately state intended use and known limitations, provide necessary documentation and incident support, and not use a generic “AI may make mistakes” statement to excuse foreseeable defects or misleading marketing.
Procurement terms should address data use, retention, subprocessors, security, versions, service levels, incident notice, audit information, intellectual property, indemnity, termination and data return. Liability caps and exclusions need to fit the worst intended use and be reviewed by appropriately qualified people.
An organisation cannot purchase a general tool and treat marketing material as local validation. The provider controls the product layer; the adopter controls configuration and use. Both can bear responsibility.
Deployment and engineering teams own technical controls
The people connecting a model to real data and tools control permissions, retrieval, instructions, validation, versions, monitoring and stops. If an agent can delete every file through a shared administrator account, the problem is not only misunderstanding by a model; it is an architecture failure. If untested updates change behaviour and cause loss, change management is relevant.
Technical owners should document risk assumptions, test coverage, known failure modes and deployment approval. During an incident, they preserve evidence, contain impact and reconstruct the trajectory. “Models are unpredictable” must not become an excuse for ignoring deterministic controls.
Engineers should not individually own the organisation's decision to pursue a high-risk use. Business and governance owners approve objective, loss budget and automation boundary.
Business and professional decision-makers retain non-delegable judgement
Doctors, lawyers, finance leaders, safety engineers, hiring managers and public decision-makers using AI remain subject to applicable professional and organisational duties. AI may prepare material, but generating advice does not transfer legal authority, professional obligations or the responsibility to provide a reasoned decision about a person.
Real oversight requires competence, evidence, time and veto. If an organisation expects one person to approve thousands of cases and shows only an AI summary, it cannot fairly allocate every later error to the last click. The design removed effective control.
Responsibility should follow control. A person should answer for matters they could reasonably detect and change; the organisation must supply training, workload and escalation.
Front-line staff should not become the default scapegoat
An employee can enter incorrect data, ignore a clear warning or use a tool without authority, and these actions may require response. But if policy is vague, access is open by default, performance pressure rewards rapid acceptance and evidence is hidden, an individual mistake also reflects system conditions.
An incident inquiry should ask why controls allowed an error to reach the outside, not only who clicked. A just culture helps distinguish honest mistake, training gap, process design and deliberate violation, encouraging people to report near misses.
Punishing the last operator after every event drives problems underground and leaves architecture unchanged. Removing every individual duty would also make rules meaningless. Clear, achievable responsibilities and proportionate consequences provide the balance.
Affected people need a simple route to challenge and remedy
A person should not have to prove an internal model mechanism before correcting a mistake. A notice should explain whether AI materially influenced an important result, which relevant data were used, how facts can be corrected, who can reconsider and the time for challenge.
Remedy may include pausing action, restoring service, correcting a record, fresh human evaluation, notifying third parties, refund or compensation. Restore the affected person first; allocate responsibility between organisations afterwards. Otherwise supply-chain complexity becomes delay.
OAIC's guidance for commercial AI products emphasises continuing organisational responsibility for personal-information handling, including accuracy, transparency, security and appropriate human oversight of AI-assisted decisions. OAIC: Guidance on privacy and commercially available AI products
Contract cannot resolve all responsibility
A provider and customer can allocate cost and duties contractually, but their agreement may not bind affected consumers, employees or the public, and cannot remove obligations that law makes non-excludable. An internal notice saying “users are responsible for all AI output” does not automatically cure organisational design and supervision.
Contract still matters: who provides logs, how quickly a vulnerability is reported, how model change is communicated, who assists investigation and how exit works. If a provider refuses information necessary for a high-risk use, the adopter should narrow that use or select another solution.
An individual using a free service normally lacks bargaining and audit rights, making it especially unsuitable as a component in important automatic decisions. A low subscription price may leave risk and control asymmetrically with the user.
Open models do not make responsibility disappear
Open access can improve inspection and control while giving the deployer more choices. Model publisher, fine-tuner, host, application developer and adopting organisation each control different components. Licence terms are a starting point, not a substitute for safety testing, data governance and external accountability.
Self-hosting also adds infrastructure, patching, access, logging and model-update responsibility. Absence of a hosted vendor does not mean absence of a supply chain; training data, libraries, weights, hardware and integrations still have origins.
Map responsibility by control point rather than an open-versus-closed label.
Add control, evidence and remedy to a RACI
A conventional RACI identifies responsible, accountable, consulted and informed roles. An AI workflow also needs three fields for each party: what it can control, what evidence proves the control worked, and what remedy it can provide.
For a payment agent, a business owner approves purpose and limit; finance approves payment rules; engineering implements permissions and deduplication; security monitors accounts; the provider supplies the model and incident assistance; operators manage exceptions. List logs, tests, approval records, stopping authority and recovery actions for each.
Avoid assigning “responsible” to many roles without a final owner. Each risk needs one clear owner, and every external decision needs one institution visible to the affected person.
Investigate incidents in time order
Protect people and systems first: stop expansion, preserve necessary evidence, revoke dangerous access and correct urgent results. Then reconstruct the trajectory from input to action, including source, model and prompt versions, tool parameters, approvals and receipts.
Analyse direct cause, contributing conditions and governance cause. The direct cause may be a model misreading an amount; a contributing condition is the absence of a deterministic cap; the governance cause is that a demonstration led to deployment with no approved loss budget. Editing the prompt alone leaves deeper gaps.
Finally assign remedy, system repair, notice and contractual recovery, and feed findings into tests and use rules. Focus on repeatable protection rather than inventing a psychological motive for a model.
How the NIST framework helps allocate work
The NIST AI Risk Management Framework places governance across mapping, measurement and management, emphasising accountability structures, documentation and continuing treatment. It is not an arbiter of legal liability, but it helps an organisation specify who should do what, how operation is evidenced and when to stop. NIST AI Risk Management Framework
The generative AI value chain is particularly important: foundation model, data, plugin, application and adopting institution are interdependent. Each party should disclose information proportionate to its control, and the adopter translates that into use-specific safeguards.
My assessment: match responsibility to control and centralise remedy for the affected person
Internal responsibility may be shared and complicated; external accountability should not become a vacuum. The party delivering the service or decision supplies an accessible correction route first, then allocates cost according to control and contract.
Blaming the final operator ignores design. Blaming only the provider ignores chosen use. Blaming AI puts a tool that cannot offer remedy between the institution and the person. Mature governance recognises multiple causes while preserving one clear external owner.
Responsibility checklist
- Who chose AI for this specific use and risk level?
- Who controls data, model version, instructions, tool permissions, tests and monitoring?
- Who owns the final business or professional decision, with a genuine veto?
- Do provider terms cover data, change, incident, logs, audit and exit?
- Do front-line duties match training, evidence, time and authority?
- Can affected people learn AI's role, correct data, obtain human review and receive remedy?
- Does every critical control have evidence, an owner, a stop and recovery?
- After an incident, can direct, contributing and governance causes be distinguished?
Conclusion
When AI causes loss, responsibility cannot remain with AI. The adopting institution owns use, external outcome and remedy; providers, engineers, business decision-makers and operators bear duties proportionate to their control. Internal allocation may be detailed, while the external route must remain simple. Matching responsibility to control and putting remedy before blame keeps a complex AI supply chain from becoming an accountability vacuum.
Related questions
- What Are Humans Still Responsible for After AI Gives an Answer?
- How Can You Write Practical AI Use Rules for Yourself or a Small Team?
Continue reading: All articles in How Far Should You Trust AI?
Discover more from Geoffrey Chen
Subscribe to get the latest posts sent to your email.