At a recent DX Connect roundtable, one attendee put the question bluntly: if a decision now emerges from several AI agents acting in sequence, who signs their name to it? It's not a hypothetical. Agentic AI systems are already approving claims, allocating resources and making judgment calls that used to sit with a named person, and most organisations haven't decided who owns the outcome when one of them gets it wrong.
That dilemma is quickly becoming the single biggest governance issue in enterprise AI. Here's what the evidence and the people closest to the problem are telling us about how to solve it.
Pascal Bornet, global AI and automation authority and co-author of Agentic Artificial Intelligence, has spent the past year auditing exactly this failure mode inside large organisations. In a recent piece, Who Owns It When the Agent Gets It Wrong?, he describes reviewing an agent's "owner" field and finding it listed as "compliance function": 400 people, none of whom had personally signed off on anything. His conclusion is stark:
"Accountability that cannot be traced to a single name is not accountability. It is diffusion."
When ownership belongs to a function rather than a person, Bornet argues, accountability effectively belongs to no one. Organisations usually only discover this the day something goes wrong: there's no decision log, and the team responsible "could describe what the agent was designed to do. It could not reconstruct what it had actually done that day."
Bornet has distilled the fix for this dilemma into a test that any leader can run today, with more details below.
This isn't just a best-practice question in regulated industries; it's becoming a compliance one. Under Australia's CPS 230 Operational Risk Management standard, APRA now captures AI "by function, not by name": any AI system that supports a critical business operation or creates material operational risk falls within scope, regardless of its internal name.
APRA's April 2026 supervisory letter made it explicit that entities can no longer rely on "point-in-time and sample-based assurance" for systems that learn and adapt, and that continuous validation is now the expectation, not periodic sign-off. The regulator has also flagged manipulation of autonomous agents as one of three named control priorities, alongside prompt injection and data leakage. Critically, compliance accountability remains with the regulated entity even when an AI vendor is doing the work, which means boards are now expected to have sufficient AI literacy to "provide effective challenge," not simply take a vendor's word for it.
For any organisation that treats agent governance as a technology problem to be solved later, CPS 230 is a reminder that regulators already treat it as today’s problem.
None of this is an argument against agentic AI; it's an argument for designing it deliberately. Tom Davenport, one of the most cited voices in enterprise AI research, makes the case plainly in his essay We're Screwed If We Don't Stay in the Agentic AI Loop: agents "will still make mistakes and won't be able to anticipate every eventuality," and organisations that "inject smart, creative humans into your agentic loops" consistently get "more accurate, flexible, and innovative outcomes" than those chasing full automation.
That's not a case for slowing down. It's a case for building accountability and human judgment into the architecture from day one, rather than retrofitting it after an agent has already made a decision nobody can explain.
Pulling these threads together, a practical governance model for agentic AI needs to answer four questions before an agent is ever put into production, a structure echoed in recent enterprise governance guidance from AvePoint:
Identity and ownership: a single named business owner and technical owner, assigned before the agent acts, not after an incident forces the question.
Access and data governance: least-privilege access mapped to data classification, so an agent's reach matches its actual job, not the entire environment it happens to sit in.
Behavioural guardrails: defined human-in-the-loop checkpoints for high-risk actions, calibrated to impact and reversibility rather than applied uniformly.
Audit and observability: a centralised, exportable decision log across every platform the agent touches, so "what did it actually do today" is always answerable.
Ownership and enforcement also need to be deliberately split: IT holding the technical inventory and access controls; compliance owning risk assessment and regulatory mapping; and security governing behavioural guardrails and incident response. Sharing responsibility this way avoids the two most common failure patterns: IT-only programs that underweight regulatory risk and compliance-only programs that write policies that nobody has the technical means to enforce.
Pascal Bornet's audit test is the most useful place for any organisation to start: pull up the owner field on your highest-risk AI agents today. If it names a function, a committee, or a team rather than one person, you've found your next incident before it happens — and you now have the chance to fix it before it does.
How PhoenixDX can help
Running that test is one thing; closing the gap it reveals is another. At PhoenixDX, we apply Bornet's Human-Agent Orchestration approach to help organisations build the governance layer their agents are missing — from naming accountable owners to designing the escalation and audit trails regulators now expect. Our ISO/IEC 42001-certified consultants work alongside your team to translate that framework into a certifiable, board-ready AI Management System, so your governance is documented, owned, and built to scale with you.
If you'd like to see what that looks like in practice, reach out to our team for a walkthrough of our approach.