Who Decided the Machine Could Do That?

I recently attended the GRC 2026 Conference in sunny San Diego, presented by ISACA (Information Systems Audit and Control Association) and the IIA (Institute of Internal Auditors). I attended a session by Madalina Manoiu, “Agentic AI in GRC: Where Do We Draw the Human Line?” that raised a question I think most organizations can’t answer. 

We’ve all experienced this: You attend an industry conference and leave with more questions than answers, especially in the dawning age of AI. The questions surrounding the use of AI are numerous and difficult for any organization to address. At the same time, many of our clients are implementing AI at various stages of production and control and wondering how to assess and explain that usage to their own clients, who are asking the exact same questions. Governing bodies, especially in the EU, are also increasingly interested in organizations having explainable decisions and records that support both process and policy. 

Your agents are already making decisions that used to require a person. The question your auditor, your biggest client, and eventually your regulator will ask is not whether the AI is accurate. It is: Who decided the machine was allowed to do that, and can you show me the record? 

Most companies deploying agents right now cannot answer either half. 

My world largely exists in the space of auditing for ISO 42001, the first global, certifiable international standard that specifies the requirements for implementing, maintaining, and continually improving an Artificial Intelligence Management System. ISO 42001 seeks to establish an acceptable level of governance for a product or organization that utilizes AI. It is a great starting point and a reference for other standards and frameworks that address more specific AI use cases, like agentic loops. AIUC (AI Underwriting Company), for example, publishes AIUC-1, a standard designed specifically for AI agents that covers data and privacy, security, safety, reliability, accountability, and societal risks. 

When there is no one between the model and the consequences 

We have all asked AI a question it has confidently answered incorrectly. Agents that operate without properly placed human governors, or worse, without human governors at all, are a massive risk to any organization employing them and their downstream partners. 

Consider the cases you’ve read about in which a law firm filed briefs citing decisions that did not exist. Hallucination rates have improved since then, but the risk remains, particularly with proprietary production data. 

The brief at least had a lawyer between the model and the court. An agent with tool access does not have that gap. By the time anyone reads the output, the action has already been taken. 

When you deploy agents that produce client-facing content or operate technical and security controls, your exposure grows with every responsibility you delegate. So how do you build accountability into a system where agents are doing the real work? 

Walk me through one decision 

When I audit an organization that has put AI agents into production, I ask for one thing early: Show me the last time an agent acted on a live system and walk me through it. If a company built this properly, it takes about four minutes. 

An agent monitoring entitlements on a finance system notices an administrator account that has not been used in 45 days. It scores the finding as high risk because the account is both privileged and dormant. It proposes revocation and records why: the rule it applied, the data it looked at, and how confident it was. 

The IT control owner reviews the proposal and approves it. The system captures the owner’s identity and the time. The revocation executes through the identity system under the agent’s own credential, not a shared service account, so the action is attributable. Closure is verified, and the pattern is added to the monitoring baseline through change control, so the next dormant admin account is caught using the same parameters. 

I can follow the decision from the signal that triggered it to the outcome it produced, and I can tell you which human authorized it. 

Most organizations cannot produce anything like this. 

A more likely scenario is that I get a dashboard or spreadsheet listing remediations. When I pick one and ask why, the answer is some version of: The tool flagged it. Nobody can tell me what rule was applied, what the agent was instructed to do, whether anyone approved the action or it was automatically applied, or whether the person who approved it had any real basis for doing so. 

This is where a governance gap eventually becomes a technology problem. It may remain hidden until someone asks a specific question about a decision and discovers there is no answer. 

From “did it run?” to “why did it decide?” 

Testing a control has historically meant confirming it operated. The process was deterministic: same inputs, same outputs. That is why a sample of 25 tells you something reliable about a population of 10,000. 

Agents break this logic. 

An agent pursues a goal rather than a fixed path and chooses its tools based on what it finds. Given the same account on two different days, it can reach two different conclusions because the context it assembled was different, a policy had been updated, a system returned incomplete data, or a threshold had been retuned. 

With agents, confirming that the control ran is no longer enough. You also have to understand why it reached a particular decision. Historically, you needed proof that the process was followed. With an agent, you need evidence that each judgment was defensible. 

More logging won’t save you. Your systems already produce logs that cannot answer the question because they record what happened rather than why. An agent’s reasoning exists for a moment, in the instructions and context it was working from, and then it is gone unless something captures it on purpose. 

Three things that break 

Many things can go wrong with an agent, but in the context of governance, three account for most of the damage. 

Nobody owns it. 

Ask who is accountable for the agent that revokes access at two in the morning, and there may be no answer. The failure this produces is quieter than people expect. 

A control that quietly stopped revoking dormant accounts in March went unnoticed until September because monitoring alerts on errors, and this was an absence rather than an error. That meant six months of a control that appeared in every report and did nothing. 

The agent did what it was instructed to do. Nobody was negligent. There was simply no one accountable for its decisions. 

You automated the second opinion. 

Every executive understands why the person who writes a check should not approve it. Agentic controls reintroduce that problem in a form the org chart cannot show. 

One agent performs your access reviews, and another agent checks whether the reviews were performed. Different teams, different owners, clean reporting lines, but both are running on the same platform against the same model and using prompts from the same internal library. 

If that shared instrument has a systematic blind spot, both agents inherit it. The agent performing the work misses the problem, and the agent checking the work may miss it for exactly the same reason. What looks like a second opinion may actually be a repeat of the first. 

Your vendor can change your controls without telling you. 

The model underneath your agent is a hosted service. Providers deprecate versions, adjust safety filters, and change how models select and call tools, any of which can move your agent’s behavior away from what you validated six months ago. 

The result may simply be gradual drift. The control continues to run and report, and its numbers may still resemble last quarter’s, even though its underlying behavior has changed. You can end up re-certifying it against a baseline that no longer accurately describes how it operates. 

Most organizations treat this as internal change management, where it cannot be detected, instead of third-party risk, where it can be contracted for. 

What proof looks like 

When a client, regulator, or your own audit committee asks about one action an agent took eight months ago, six things let you answer: 

• What it decided and why: inputs, rules applied, and confidence. 

• The instructions in force at the moment: the March version should explain a March decision. 

• Which credentials it used: a specific agent identity, not a shared account. 

• Who approved it and proof the approver was not the executor: an agent must never approve its own action, and that belongs in code rather than policy. 

• Every limit it hit and what happened next: the threshold, the automatic response, and the human follow-up. 

• Who was told, how fast, and how it ended. 

The hard part is that all six must be captured at the moment of decision because reasoning cannot be reconstructed after the fact. Organizations that treat evidence as something to assemble before an audit learn this at the worst possible time. 

A useful test is to pick one agent action from six months ago and try to produce all six by the end of the day. What you find when conducting this test may concern you. 

What certification settles 

ISO 42001 will not tell you where to draw the human line. It will force you to draw it somewhere and write down who decided. That is what an AI management system actually buys an organization, and it is worth more than it sounds. 

It makes an implicit decision explicit. Right now, whoever configured the tool answers the question of how much authority your agents hold. Certification requires you to state it per process, with a name attached to the answer. 

It is independently verifiable, so it answers your clients’ questions instead of arguing with them. Enterprise security reviews have started asking about AI governance, and a certificate closes that conversation faster than a policy document does. It is the foundation other programs build on rather than compete with. AIUC-1 and the assurance schemes emerging around it treat a management system as the foundation they sit on, so the work doesn’t get stranded the next time this market moves. 

Governance and testing have to work together. Without testing, you can have a well-documented program without knowing whether it works as intended. Without governance, testing may tell you what happened at one point in time without establishing who is accountable for it. 

You can start by answering a few practical questions. How quickly your organization can answer them will tell you a great deal about the current state of your AI governance. 

How many AI agents are running in production right now, and who is the named person accountable for each one? 

For our highest-risk agent, what is the largest action it can take without a person approving it first? 

Think back six months. If you had to defend a decision an agent made then, could you produce its reasoning along with the outcome? 

Consider the systems involved as well. Does the platform running a control also produce the evidence that the control worked? And if your model provider changed the underlying model tomorrow, how would your organization find out? 

I left San Diego with more questions than answers, which is usually the sign of a conference worth attending. One of those questions turned out to be answerable, and it is the one I now ask first: When an agent acts on your behalf, can you show me the record? 

Most organizations cannot, yet. Being able to answer that question starts with treating agentic AI as a governance issue now, while there is still time to establish the accountability and evidence behind those decisions.