The agentic AI questions IT leaders are actually asking

Give a room full of CIOs, CTOs and transformation leaders live Q&A time with an AI authority, and the questions stop being about chatbots and start being about accountability, trust and organisational design. 

That is exactly what happened at The Last Competitive Advantage: Leading Agentic AI Before It Leads You, a closed-door executive event PhoenixDX ran with Pascal Bornet across Sydney, Melbourne and Singapore in August. Seventy-five leaders. Forty-seven submitted questions. Twenty minutes of live Q&A per city, nowhere near enough to get through them all.

What happens in the room stays in the room. That is how the event was framed, and it is why nothing below names an attendee. But the questions themselves are too good, and too common, to keep private. Strip away the names and the companies, and the same handful of concerns showed up in insurance, health, government, logistics and consumer goods alike. Here are nine of them, generalised enough to protect the room and specific enough to be useful to any IT leader asking the same thing.

The nine questions fall into three natural groups. The first is about definitions and ownership: what actually counts as an "agent", and who should be accountable for one. The second is about trust and guardrails: how much you can rely on an agent, and what keeps it safe as it scales. The third is about people and structure: whether this needs a new org chart, how you train managers for it, and what you tell a workforce that is nervous about what comes next.

The throughline: Pascal Bornet's 6S Design Canvas

Pascal Bornet needs little introduction to anyone tracking enterprise AI: a former McKinsey partner, author of four best-selling books on AI and automation, and a researcher whose latest work draws on 432 organisations navigating agentic deployment. What makes his answers useful beyond the room is that they do not float free of a framework: almost every response below maps onto specific fields in his 6S Design Canvas: a Named Owner, a Decision Rights Map, an Access Scope, an Exception Table, a Human Layer Index. It is a structured set of design decisions every agent deployment is expected to make explicit, rather than a general philosophy.

That is the throughline for what follows, across the three clusters below.

Cluster 1: What exactly are you governing?

What actually counts as an "agent"?

"What actually counts as an 'agent', as opposed to a tool?"

Bornet's answer is behavioural, not technical: a tool executes the same instruction the same way every time; an agent exercises delegated judgement inside a decision space someone has defined for it, and can act without a human present. The shift he tracks in his research is a manager waking up to dozens of agents reporting in alongside a handful of humans: a promotion from configuring tools to being accountable for delegated judgment. PhoenixDX's own recommendation starts from the same place: a register of AI use cases, agents and suppliers, mapped to each other and to the business outcomes they exist to serve, so the definitions, and the accountabilities that follow from them, stay clear as the fleet grows.

Should the named owner be a business person, or IT?

"Should the named owner of an agent be a business person, not IT?"

Yes, and the Canvas is explicit about it: the named owner has to be able to explain the agent's decision logic to a client or a regulator in plain language, which is naturally a business-side capability. PhoenixDX's view lines up: the agent owner is the person accountable for the business outcome the agent exists to solve, with a separate technical owner handling security, access and other engineering concerns at each stage of the agent's life. One name, one business outcome, one person a regulator can actually ask.

Is deploying an agent basically like hiring an employee?

"Is deploying an agent essentially like hiring an employee, complete with a job description, decision authority and KPIs?"

Closely, yes. In Canvas terms, the Intent and Outcome Statements are the job description, the Decision Rights Map is the decision authority, and a Business Outcome Target (a specific metric, moved by a specific amount, over a defined period) is the KPI. Where the analogy breaks is retirement: agents get retired against pre-defined criteria rather than performance-managed, which is a cleaner, less ambiguous process than most employee offboarding ever is.

Cluster 2: How much do you trust it, and how do you keep it safe?

Why do we hold agents to a higher trust bar than humans?

"Why do we hold agents to a higher trust threshold than we hold humans, and how should that be addressed?"

Bornet's answer separates two things people conflate: ability transfers to agents easily (it is testable in advance), but benevolence does not transfer at all, because agents have no interests and cannot care about you. Much of the "higher bar" is really a mismatch of expecting a kind of trust agents were never going to earn. His five conditions for human-agent trust (predictability, explainability, calibrated autonomy, transparency about limits and meaning alignment) are the actual levers, and predictability in particular requires leaders to actively narrate an agent's performance record; trust does not get inferred from good outputs alone, it has to be told. PhoenixDX scores these five dimensions explicitly in its AI Readiness Assessment, so a client can see which lever is genuinely the gap rather than treating "low trust" as one undifferentiated problem.

Are agents creating a new data problem?

"Is the sheer volume of data agents now produce becoming a governance problem in its own right?"

Yes, but it is a governance problem, not a volume problem. The Canvas specifies exactly what to capture per agent action: inputs, output, timestamp, confidence and reviewer, plus five metrics (output volume, error rate, exception rate, override rate and a Human Layer Index) reviewed on a fixed cadence rather than mined ad hoc. The discipline is deciding what you will monitor and what you will do when it moves, before agents start producing the data, or you end up with dashboards nobody acts on. PhoenixDX builds this into a Register of AI Agents aligned to ISO/IEC 42001, capturing those elements alongside who reviews them and how.

What are the baseline guardrails, and do they move as the agent learns?

"What baseline guardrails should every AI deployment have, and do they need to adjust as the agent learns?"

The baseline is an Exception Table (threshold, category and absence triggers, each specific enough that a new team member could enforce it), paired with a defined default state (pause, continue with a flag, or partial delivery) while a case waits for escalation. An alarm with no defined default state is one of the most common design failures. Guardrails can move as an agent proves itself, but only deliberately: one autonomy cell at a time, evidence documented, revisited on a fixed cadence. They should never loosen quietly on their own. PhoenixDX builds agent registers that record risk and autonomy level per agent, so what "adequate guardrails" means is decided up front and revisited on schedule, not renegotiated in the moment.

Cluster 3: How do you scale it across people and the organisation?

Does this require an organisational redesign, or just changed roles?

"Does scaling agentic AI require organisational redesign, or can it be done through role and responsibility changes?"

It does not require restructuring to start. A Decision Rights Map and a Named Owner can usually be layered onto an existing structure as role and responsibility changes, tracked through a RACI. The signal you have outgrown that approach is structural, not a feeling: the same person ends up named owner across too many agents spanning multiple functions, or an "hourglass" shape (broad agent execution at the base, a concentrated layer of human governance at the top) starts emerging on its own. PhoenixDX's advice matches: start with role changes via the Canvas, and let recurring boundary failures in post-incident reviews tell you when to restructure, rather than restructuring pre-emptively.

Where do you send people to actually learn this, vendor-agnostically?

"Where can leaders send people to upskill as AI agent managers without tying themselves to one vendor's platform?"

Bornet's own Canvas, worked through rather than read passively, is designed as vendor-agnostic curriculum, tracing a concrete first-year arc for building this capability in a manager. PhoenixDX runs a live, guided version of the same journey: a structured interview through the Canvas that scores current maturity by dimension and hands back a scorecard managers can act on. It also runs training courses on AI fundamentals and hands-on agent-building sprints, deliberately independent of any specific platform.

What KPI actually proves this is working, and what do you tell an anxious team?

"What KPI should leadership actually track, and what should you tell employees who are anxious about what AI means for their role?"

The common mistake is a KPI focused on AI adoption: the percentage of processes automated. A better one measures the degree to which humans have been freed up for higher-value work, paired with a check on whether people are doing more complex, judgment - intensive work than they were 90 days ago. That pairing is what proves a transformation is elevating people rather than just replacing them, and it is also the honest answer to the second question. The script is not "don't worry", it is a genuine repositioning: from being the expert who owns the answer, to using judgment to know which answer fits. PhoenixDX runs this as a structured, honest conversation with the workforce itself, not just with leadership, before any change is rolled out broadly.

How PhoenixDX can help

Nine questions from three cities point to the same underlying job: turning a philosophy of governing agents responsibly into fields someone can actually fill in: a named owner, a decision rights map, an exception table, a metric that proves the work is elevating people rather than just replacing them.

That is the work our Agentic AI Readiness Assessment does first: a structured interview against Pascal Bornet's Design Canvas that scores your current maturity by dimension and hands back a scorecard your managers can act on. From there, our ISO/IEC 42001 advisory journey (Exploration & Strategy, Design & Governance, Risk Management, and Audit & Certification) builds the governance model out properly, and our AI empowerment and training service takes the same honesty into the rooms where employees are asking what this means for them.

If any of these nine questions sounded like the one your own leadership team keeps circling back to, let's talk.