An Agent Demo Has No Organizational Chart
Ask an AI agent to search a policy library and summarize a document, and the boundaries are relatively visible. Ask it to create a purchase order, update a customer record or trigger a maintenance task, and a new question appears: who authorized this action, under which conditions, and who answers for its consequences?
Autonomy changes the nature of the system. It is no longer just producing information. It is participating in work that has an owner, a cost, a time constraint and an audit trail.
That is why the rush to build agent catalogs can be misleading. Twenty functioning demos do not constitute an enterprise operating model.
Deloitte's 2026 State of AI in the Enterprise found that nearly three-quarters of surveyed companies planned to deploy agentic AI within two years, while only 21% reported mature governance for agents. Those figures describe survey responses, not a prediction of how every industry will adopt agents. They do describe a practical gap between ambition and control.
The Missing Layer Is Authority
Most architecture diagrams show models, tools, retrieval and orchestration. They often omit the more important business contracts:
- Identity: On whose behalf is the agent operating?
- Scope: Which records and actions can it access?
- Policy: What must it never do, even when the user asks?
- Evidence: What information supported its decision?
- Escalation: Which exceptions require a person?
- Ownership: Who investigates a failure and can suspend the workflow?
These are not optional administrative details. They determine whether an organization can safely delegate work.
The same person who may approve a transaction as an employee should not automatically grant every tool invoked by an agent that same authority. Permission boundaries need to follow the action and the data, not merely the model's apparent intentions.
An Agent's Three Levels of Authority
A practical operating model starts with the risk of the action rather than the sophistication of the model.
| Level | Permitted behavior | Typical controls |
|---|---|---|
| Observe | Retrieve, classify, summarize, recommend | Source traceability, access checks, quality tests |
| Prepare | Draft a change, stage a transaction, propose a decision | Validation, diff review, explicit confirmation |
| Act | Execute a bounded action in an enterprise system | Scoped identity, policy enforcement, limits, logging, rollback or compensation |
Not every action can be rolled back. A customer notification, physical movement or external payment may require compensating actions rather than technical reversal. This should be determined during workflow design.
Moving an agent from prepare to act should be an explicit business approval, not an incidental change to the prompt.
Evaluation Must Follow the Whole Workflow
Testing whether a model gives a good answer is insufficient for an agent that calls tools.
Consider an illustrative procurement assistant. It interprets a request, looks up a supplier, selects an item, checks a price, creates a draft and submits it. The model can sound completely reasonable and still choose the wrong supplier entity or quantity.
A meaningful evaluation suite must ask:
- Did the agent select the correct workflow and permitted tool?
- Did it retrieve information the user was authorized to access?
- Did it correctly handle missing, contradictory or outdated information?
- Did it ask for approval when the transaction crossed a defined threshold?
- Could an operator reconstruct what happened afterward?
Add tests for retries, duplicate submissions, revoked permissions, malicious content in retrieved documents and partial system outages.
An agent that succeeds 95% of the time on a short isolated task can still be unsuitable for an irreversible action. Acceptable error rates depend on the harm and reversibility of the action, not a universal benchmark.
The Operating Model Needs Four Named Owners
The business owner defines desired outcomes, acceptable risk and what is allowed to happen without approval.
The technology owner is accountable for engineering reliability, integrations, observability, model changes and release management.
The security and risk function defines controls proportionate to the workflow, including identities, data handling, threat models and response procedures.
The human operator handles exceptions, exercises judgment and gives feedback that improves the workflow.
These responsibilities may be combined, but none should disappear into "the agent team."
This is consistent with the control-oriented thinking in the NIST Generative AI Profile. OWASP's guidance on agent and MCP security likewise stresses the danger of combining untrusted content, private data and powerful tools under broad privileges.
A Release Gate for Agentic Workflows
Before an agent moves into real operations, I would want a one-page answer to the following:
- Business contract: What outcome does the agent own, and what remains human-owned?
- Authorized actions: Which tools, data, roles and limits are explicitly allowed?
- Evaluation: What representative and adversarial tests passed? What is the known failure envelope?
- Recovery: Can actions be canceled, reversed or compensated? Who responds when they cannot?
- Monitoring: Which outcome, quality, latency, security and cost signals trigger intervention?
- Change control: Who can approve a new model, prompt, integration or level of autonomy?
If the answers are unclear, the next investment should be in the operating model, not in more agents.
The Real Moat Is Institutional Learning
McKinsey's August 2026 State of AI survey reported that 40% of respondents at organizations with more than $1 billion in annual revenue said they were scaling AI agents, versus 22% at smaller organizations. But scale alone does not prove durable value.
An advantage emerges when the organization learns from exceptions: which task was misclassified, which approval took too long, which source was stale, which boundary prevented harm. That knowledge becomes reusable policy, tests, process design and better data.
The agent is replaceable. The organization's validated understanding of its own work is much harder to copy.
The next question for enterprise leaders is not how many agents they have. It is how much work they can safely delegate, with evidence that the delegation improves the business.
Evidence and Further Reading
- Deloitte, State of AI in the Enterprise 2026. Agent deployment intentions, process redesign and governance maturity.
- McKinsey, The State of AI in 2026: On the Road to ROI. Survey evidence on scaling agents and enterprise value.
- NIST, AI RMF Generative AI Profile. Cross-sector risk management guidance.
- OWASP, AI Agent and MCP Security. Threats and least-agency controls.
- Related framework: The Eval-First Principle.