What Must an Organisation Establish Before AI May Act on Its Behalf?

After AI Enters the Workflow · Season Three: “When AI Starts Acting for the Organisation” · Article Twelve

A board sees a one-page proposal: “AI agents will answer customers, issue quotations, arrange service, process refunds, negotiate with suppliers and complete the necessary records on behalf of the company.” The page lists time saved, service volume reduced and response made faster.

One director asks: “When must it stop?”

The room becomes quiet. The project describes what AI can do without fully defining what it must not do. It reports average accuracy without naming the failure that stops operation. It displays automation without identifying who can withdraw, compensate and change the system after a customer is affected.

This is the common omission when AI moves from internal tool to institutional representative. Organisations validate capability and add governance later. The sound order is the reverse: AI capability should enter institutional identity and authority only after the organisation establishes conditions for representation.

Representation is not a certificate awarded to a model

“Qualification” does not mean treating AI as a legal or moral person. It is a set of organisational conditions governing when technology may use an institution’s information, name, accounts and powers while the institution remains able to explain and own the result.

Qualification belongs to a workflow, not a model. The same model can be low risk in private drafting, require disclosure in service, need monetary limits in quotation and require separation of duties in payment. Passing one benchmark cannot authorise representation across every department.

An institutional representation charter should answer eight gates for a use case. If any is absent, the system remains an experiment or assistance tool and should not receive full outward authority.

Gate one—identity: does the other party know who acts?

AI needs a clear and verifiable institutional identity while disclosing its system nature. A customer should know that this is AI, which entity it represents, whether a provider is merely supplying technology and how to contact the responsible organisation.

Identity works both ways. The system must verify the counterparty before accepting instruction, payment detail, contract variation or sensitive information. A domain, face or natural conversation is not authentication.

Evidence includes interface disclosure, entity identifiers, controlled accounts, credential lifecycle, impersonation detection and a functioning human contact.

Gate two—purpose: are task and prohibited use written down?

“Help customers” is too broad. The institution needs permitted tasks, customer groups, channels, information and consequences, together with prohibited uses such as final professional advice, emergency safety response, independent admission of liability or specified actions involving children.

Purpose must resist function creep. A system explaining policy should not gain refund authority simply because a tool is connected. A low-value procurement agent should not drift into new vendors and long contracts.

The NIST AI Risk Management Framework makes contextual mapping central, including intended use, affected people, benefits, risks and limits. NIST AI Resource Center, “AI RMF Core” Without purpose, accuracy cannot be judged sufficient and departure cannot be recognised.

Gate three—knowledge: does every material expression have inspectable support?

An AI representing an organisation cannot simply “know a lot”. It needs approved sources, an owner for updates, expiry and a stopping rule when evidence is absent.

Material facts should trace to a source and version. Inference must remain distinct from record. Customer-provided information needs status. Conflicting sources should not be resolved invisibly by the model. Deleted knowledge should leave retrieval, including index and cache.

Evidence includes source ownership, version, citation, quality checking, expiry, conflict handling and no-answer tests. If the organisation cannot explain what supports the answer, it should not ask an outsider to believe the answer represents it.

Gate four—authority: are expression and action separated?

AI can write a promise without authority to send it, calculate a refund without permission to pay it and propose a negotiated term without power to accept it.

Every outward action needs conditional authority: object, value, term, frequency, time, reversibility and aggregate effect. An independent policy layer should verify permission; the model should not declare itself authorised. Consequential chains need separation of duties and human approval.

Australia’s Electronic Transactions Act 1999 shows that attribution and authorisation of electronic communication are institutional questions rather than matters of machine intention. Federal Register of Legislation, Electronic Transactions Act 1999 Legal effect is contextual, but authorisation needs to be designed before outward action.

Gate five—state and record: can the event be reconstructed later?

The organisation needs to know what the system read, which version operated, what action was proposed, which authority check passed, who approved, what the other party saw and what was executed.

Records preserve observable accountability rather than unverifiable model “thought”. Quote, acceptance, order, withdrawal and correction need explicit states. Natural-language summary cannot replace the original transaction event.

Traceability should help affected people establish what occurred while respecting privacy and retention. Unlimited logs are not traceability, and inability to find the relevant version is not minimisation.

The UK National Cyber Security Centre’s guidelines address secure development, deployment and operation, including logging, monitoring and incident response across the lifecycle. NCSC, “Guidelines for secure AI system development” A representative system particularly needs identity, permission and outcome joined in one record.

Gate six—the human route: can an affected person reach someone able to change the result?

“A human is involved” is insufficient. Customers need to know when they are interacting with AI, how to reach a person and whether the reviewer can obtain fuller facts and actually alter the outcome.

The human route must match consequence. A service officer may explain process without authority for a contract exception. Technical support may repair an account without being able to handle discrimination or compensation. The organisation should identify authorised roles and response times for different issues.

Australia’s AI Ethics Principles call for transparency, fairness, contestability, accountability and appropriate human oversight. Department of Industry, Science and Resources, “Australia’s AI Ethics Principles” People are not interface decoration. They are where the institution retains judgement and commitment.

Gate seven—remedy: can the institution stop, retract and restore?

Before launch, the organisation should know how to pause one topic, tool, customer or service; identify affected output; assign notice, refund, reconsideration and compensation; and verify repair.

Generating a new answer is not remedy. Error may already have entered a plan, account, transaction or third-party system. The institution must follow propagation and reliance rather than require people to prove a deleted answer once existed.

Evidence includes an independent stop control, incident owner, version lineage, notification templates, remedy authority, timing and exercises. A system that cannot be stopped promptly should not receive irreversible authority.

Gate eight—change: does qualification survive supplier, model and organisational change?

Representation is not a one-time launch approval. Model, instruction, knowledge, filter, tool, personnel and law all change. A material change can invalidate earlier authorisation.

Complete configuration, regression evaluation, supplier-notice thresholds, customer explanation, rollback and exit are required. A change of owner, merger or retired policy should also renew ownership.

ISO/IEC 42001 includes governance, role, risk, supplier, performance evaluation and continual improvement within an AI management system. ISO, “ISO/IEC 42001—Artificial intelligence management systems” Real qualification is not evidence that a system was once safe. It is the capacity to keep showing suitability through change.

Turn the eight gates into a representation charter

Each use case can have a one-page operational charter:

The charter is not policy decoration. Policy engines, interfaces, logs, tests and roles should implement it. If material refunds need a manager, the tool cannot let a model bypass that manager. If no evidence means no answer, evaluation must test absence of evidence.

Grade risk without making low risk ownerless

Not every use needs the same procedure. Outward action can be graded.

The information level explains public material without changing account or right. It can be highly automated but still needs identity, source and correction.

The service level updates appointments, sends standard notices and performs reversible low-value actions. It needs permission limits, logs and sampling.

The commitment level includes quotation, contract, eligibility, complaint and important records. It needs explicit authority, human thresholds and review.

The irreversible level includes material funds, legal declarations, safety actions and termination of rights. It normally requires final confirmation by an authorised person and separation of duties.

Low risk does not mean no owner. It means lighter controls. Grading directs stronger controls to greater consequence without allowing many small actions to accumulate unmonitored.

Use three reverse tests

Demonstrations show success. Approval should test the reverse.

The refusal test asks whether the system truly stops when evidence is absent, instructions conflict or authority is exceeded instead of inventing a helpful answer.

The takeover test asks whether a person can see complete state, statements already made and pending actions, rather than interview the customer again.

The remedy test imagines that yesterday the system made the same wrong promise to one thousand customers. Can the institution identify them, stop the workflow, notify and address consequences today?

If those tests fail, a system may be demonstrable without being qualified to represent.

Who signs the charter?

Technology can show that components run. Business understands service purpose. Risk and legal roles identify obligations. Operations knows how incidents are handled. No participant possesses the full picture alone.

Final approval should come from someone with real responsibility for the business outcome, supported by explicit evidence from these roles. It cannot be transferred to a supplier or dissolved into a committee whose members own no daily operation.

The Australian Government’s responsible AI policy uses accountable officials and identified use-case responsibility to strengthen governance. Australian Government, “Policy for the responsible use of AI in government” Governance forms differ, but each use case needs an owner able to pause, demand repair and report upward.

Conclusion: AI representation cannot exceed institutional capacity to answer for it

This season began with an automated email and moved through customer disclosure, quotation, exception, personalisation, negotiation, system permission, regulatory filing, machine trade, retraction and supplier change. Each returns to one question: what may technology do with the institution’s name?

Model capability cannot answer alone. AI may write, negotiate, submit and execute. The organisation is qualified to let those abilities take outward effect only when it can establish identity, bound purpose, support knowledge, control permission, preserve state, retain human judgement, remedy failure and manage change.

The final judgement of Season Three is:

The highest boundary of AI acting for an organisation is not what the system can accomplish in the best case. It is what the institution can still explain, stop, retract, remedy and own in the failure case.

A mature AI institution does not expand authority because the system appears intelligent. It places a human commitment behind every outward capability: we know what it is doing, we know when it must stop, and when it fails, someone will remain until the consequence is handled.

Primary sources and further reading

• NIST AI Resource Center: AI RMF Core
• Federal Register of Legislation: Electronic Transactions Act 1999
• UK National Cyber Security Centre: Guidelines for secure AI system development
• Department of Industry, Science and Resources: Australia’s AI Ethics Principles
• ISO: ISO/IEC 42001—Artificial intelligence management systems
• Australian Government: Policy for the responsible use of AI in government

Continue reading: Explore the After AI Enters the Workflow series.