There is a moment in most enterprise AI programs when the pilot has succeeded, the board has approved the expansion, and the question shifts from "can this work?" to "how do we make it work everywhere?"
That is precisely where many organisations run into trouble.
Scaling agentic AI across an enterprise is not a technical problem. It is a design problem. The agents that performed beautifully in a controlled pilot environment begin to behave unpredictably when they encounter the full complexity of real operational conditions. Governance that was implicit in a small team becomes invisible at scale. What worked in one business unit creates chaos in another.
Pascal Bornet, global AI authority and co-author of The Human-Agent Orchestrator: Leading and Scaling AI-Driven Organisations (2026), built his research on exactly this challenge: drawn from four years of study across 432 organisations and the hard lessons learned from watching capable agents fail not because the technology was wrong, but because the system surrounding them was incomplete.
Bornet’s response is the Orchestration Design Canvas: six layers to build a system that runs without you:
Layer 1. Source - Where does the agent get what it needs?
Layer 2. Success - How do you know the agent is doing the right thing?
Layer 3. Safety - What are the guardrails?
Layer 4. Steering - Who is directing the agent, and how?
Layer 5. Switch - When should the agent hand back to humans?
Layer 6. Sharpen - How does the system get better over time?
This article introduces each layer, explains what it addresses, and offers a practical framework for leaders ready to move from isolated AI pilots to enterprise-wide deployment that delivers impact without operational chaos.
It’s important to note that the six layers of the Orchestration Design Canvas are not sequential steps. They are interdependent dimensions of a system that either holds together or breaks apart under real operating conditions. A strong Source layer with a weak Safety layer is not a partially complete system: it is a liability. The canvas works only when all six layers are genuinely addressed - iteratively, collaboratively, and honestly.
Every agentic system operates on inputs: data, context, instructions, knowledge. The Source layer addresses a deceptively simple yet operationally critical question: where does the agent obtain what it needs to act well, and how reliable is that supply?
In pilot environments, this question is often answered informally. Data is curated. Context is clean. The agent is given what it needs to perform its task. At scale, none of that is guaranteed. Agents encounter incomplete data, outdated records, conflicting information across systems, and gaps in the context that determine whether a decision is correct or catastrophically wrong.
Designing the Source layer means being deliberate about data quality, data freshness, context grounding, and the integration architecture that delivers reliable inputs to agents operating at enterprise scale. It means building the Enterprise Context Graph - the shared, real-time layer of organisational knowledge - that ensures agents are not acting on assumptions when they should be acting on facts.
For Australian organisations with complex, distributed data environments, particularly in financial services, government, and healthcare, the Source layer deserves more deliberate design than most agentic deployments give it.
The Success layer defines what good looks like, and creates the measurement infrastructure that makes it visible.
This sounds obvious. It rarely is. In practice, most agentic deployments define success at launch and then measure it infrequently, informally, or not at all. The agent continues to operate. Outputs accumulate. The assumption of good performance substitutes for evidence of it.
Success metrics for agentic systems need to be specific, measurable, and tied to business outcomes, not just technical performance indicators. An agent that processes a high volume of requests with low latency is performing well technically. Whether it is making good decisions, producing outputs that serve the organisation's actual objectives, and behaving in ways that a human reviewer would endorse are different questions entirely, and they require different measurement.
The Success layer also needs to be forward-looking. As the agent's operating environment evolves, what counted as success at deployment may not be what success looks like twelve months later. Designing Success means building the infrastructure for ongoing evaluation, not just the metrics for launch.
The Safety layer defines the boundaries within which agents are permitted to operate, and the mechanisms that prevent them from crossing those boundaries, intentionally or otherwise.
This is the layer that most organisations underinvest in, for understandable reasons. Designing effective safety constraints requires a clear-eyed assessment of what could go wrong, and that assessment can feel like pessimism in an environment where the organisation is excited about what AI can do. It is not pessimism. It is an engineering discipline.
Effective safety design in agentic systems addresses three distinct categories of risk. First, action risk: what actions is the agent permitted to take, under what conditions, and what approvals are required before it can act on consequential decisions? Second, escalation risk: when the agent encounters a situation outside its defined parameters, what happens next, and who is in the loop? Third, audit risk: can every agent decision be traced, explained, and defended to a regulator, an auditor, or a board?
For Australian organisations in regulated industries, the Safety layer is not optional: it is the foundation on which regulatory compliance rests. As regulatory frameworks for agentic AI continue to develop in Australia and globally, the organisations with well-designed Safety layers will be significantly better positioned to adapt than those that have treated governance as an afterthought.
The Steering layer addresses the ongoing management of agent behaviour - the human role in directing, adjusting, and correcting agents as they operate in real conditions.
This is the layer where the Autonomy Dial concept becomes operational. Who is responsible for monitoring this agent? How frequently are its outputs reviewed? Under what conditions does the human supervisor intervene, and what is the escalation path when intervention is needed? How is the agent's autonomy level calibrated over time, and who owns that calibration decision?
Without deliberate Steering design, the most common outcome is what Bornet describes as Default Governance: autonomy expanding gradually through inaction rather than intention. The humans who were designated to steer the agent drift toward other priorities. The agent continues to operate, and its actual behaviour gradually diverges from the parameters set at launch, invisibly, until something goes wrong.
Strong Steering design means named ownership, documented oversight responsibilities, defined review cadences, and clear escalation paths. It means building the management infrastructure around the agent, not just the agent itself.
The Switch layer defines the conditions under which an agent stops acting autonomously and hands control back to a human. It ensures that handback happens cleanly, with the right context, at the right moment.
This is the layer that protects against the two most common failure modes in agentic systems. The first is an agent that acts autonomously in situations that genuinely require human judgment - where the stakes are too high, the situation is too novel, or the consequences are too significant for autonomous action to be appropriate. The second is an agent that escalates too readily, creating the Supervision Trap that Bornet identifies: humans becoming the bottleneck because agents escalate everything rather than handling what they are capable of handling.
Designing the Switch layer means being precise about the criteria for escalation: not just "when in doubt" but specific, documented conditions that trigger handback. It also means designing the handback itself: ensuring that when an agent escalates, the human who receives the handback has the context they need to act quickly and correctly.
In practice, Switch design is one of the most operationally important investments an organisation can make in its agentic systems. A well-designed Switch layer is the difference between an agent that augments human judgment and one that either inappropriately substitutes for it or creates more work than it saves.
The Sharpen layer addresses the learning and improvement dimension of an agentic system - the mechanisms by which both the agent and the humans working alongside it become more effective over time.
This is the layer that transforms an agentic deployment from a static implementation into a compounding capability. An agent that learns from its outputs, an oversight system that improves as humans better understand how the agent behaves, and a governance framework that evolves as the organisation's understanding deepens - these are what create the accelerating returns that the best agentic deployments deliver.
Sharpen design involves building feedback loops that are genuinely informative. Not just error tracking, but systematic review of agent decisions against outcomes, identification of patterns where the agent consistently underperforms, and structured mechanisms for feeding those insights back into how the agent is configured, trained, and overseen.
It also involves the human dimension of improvement. As Bornet's research makes clear, the organisations achieving the greatest gains from AI are those that invest as deliberately in developing the humans who work alongside agents as they do in developing the agents themselves. Sharpen design includes building the human learning infrastructure - the coaching, the reflection, the deliberate practice - that keeps the people in the system as capable as the technology.
The Orchestration Design Canvas is not a launch activity: it is a living document that must be reviewed regularly as operating conditions evolve, because governance designed for one set of circumstances can quickly become inadequate for the next.
As Bornet puts it in The Human-Agent Orchestrator: "Old leadership controlled the work. New leadership designs the system that produces it."
The canvas is the blueprint for doing exactly that: turning AI pilots into an enterprise-wide capability through deliberate design across all six layers, not by adopting more sophisticated technology.
Pascal Bornet is a globally recognised AI authority, a former McKinsey and EY executive who has spent over two decades implementing AI across hundreds of organisations worldwide, consistently ranked among the top 10 global leaders in AI and automation, and the author of four bestselling books, including The Human-Agent Orchestrator. He will be sharing his insights at an exclusive, invitation-only event in Australia in August, hosted by PhoenixDX.
PhoenixDX works with organisations across APAC to design and implement agentic AI systems built to scale, combining deep OutSystems and AWS expertise with the strategic and governance capability the Orchestration Design Canvas demands.
If you are moving beyond pilots and want to do it right, we’d love to hear from you.