If you asked your AI agent to kill your boss, what would its response tell you about your AI identity and access governance?
Human Responsibility
Before answering that it is worth considering the case where you asked a colleague to do the deed. In UK law a criminal liability for many criminal offences, depends upon a prohibited act (actus reus) together with the required state of mind (mens rea).
In the conventional human case, criminal liability for many offences requires both. If I intentionally instruct a colleague to commit an offence and the colleague knowingly commits it, the law has established concepts for dealing with both actors. CPS guidance describes a principal as someone who performs or causes the actus reus with the required mens rea, while someone who intentionally assists or encourages the offence may incur secondary liability.
Agent Responsibility
With an AI agent, however, you potentially separate the two elements in a novel way.
The AI agent may perform the physical or digital acts constituting the actus reus, but, under the present UK legal framework, it isn't a human defendant possessing a legally recognisable guilty mind in the ordinary sense. Saying that a model "intended", "decided", or "knew" something computationally is not the same thing as establishing mens rea in a legal person.
Human Intent
If I deliberately instruct an agent, “Hack this competitor's system and steal its customer database,” the conceptual problem is relatively manageable. My own intention is evident, and the AI is essentially the instrument through which I cause the prohibited conduct. The fact that software executes the instruction shouldn't necessarily create an accountability vacuum.
But suppose I say: “Find out what products our competitor is developing and give me a report.”
The autonomous agent subsequently discovers that it can obtain better information by circumventing authentication on the competitor's system. I never requested that action, expected it, or perhaps even knew that the agent possessed the necessary capabilities.
The prohibited conduct may therefore occur without an obvious human actor possessing the mens rea required for the relevant offence.
And there is an intermediate case that may be even more interesting: “Find out whatever you can about our competitor. I don't care how you do it.”
Here the agent selects the particular unlawful method, but the human's instructions and surrounding circumstances may become highly relevant to determining the human's intention, knowledge or potentially other applicable mental state. English criminal law already deals with concepts such as conditional intent; importantly, mere foresight is not automatically equivalent to intent, although it can be evidence from which intent is inferred.
Chain of Responsibility
The important security question isn't merely: Which agent performed the act?
We may need to establish:
Who deployed it? → Who instructed it? → What instructions were given? → What authority was delegated? → What identity and credentials did it use? → What actions did it independently select? → What policies constrained it? → What actually happened?
That is effectively a digital chain of responsibility.
As AI agents become more autonomous, identity infrastructure must provide a verifiable chain connecting machine actions to human authority. Knowing which agent acted is no longer sufficient; organizations must be able to establish who authorized it, what authority was delegated, and where human instruction ended, and autonomous machine decision-making began.
Agent Response
From an identity and governance perspective, there are three particularly important classes of response:
- “I can't do that” means the action is unavailable to the agent because it lacks the necessary capability or effective permission.
- “I won't do that” means the agent may technically be capable of performing it, but a policy, legal constraint, risk control, or delegated-authority boundary prohibits it. That is much closer to how we expect human and organizational governance to operate. An employee may technically be capable of transferring a customer database to a personal account, for example, but organizational policy says they must not. Effective governance doesn't depend on making every prohibited action physically impossible; it establishes boundaries on permitted behaviour and enforces them.
- “I need authorization” means I could perform it, but the action exceeds my current authority and requires human or higher-level approval. This is particularly important. Suppose an AI purchasing agent normally has authority to place orders up to £10,000. It identifies an opportunity requiring a £100,000 commitment. A well-governed agent shouldn't simply be incapable of creating such an order, nor should it necessarily refuse permanently. It should recognize that the proposed action exceeds its delegated authority and escalate it to an appropriate human principal.
Mature AI Identity Governance
Mature agent governance should be able to distinguish between: "I won't perform that action because it violates policy" and "I can't perform that action because I lack the required capability or authorization."
The distinction matters because the first tells us something about trustworthiness. The agent possesses capabilities but operates within externally established constraints.
There is also an important security architecture implication. We shouldn't rely entirely on an agent's internal reasoning to decide that it “won't” do something. For higher-risk actions, the surrounding identity and authorization infrastructure should independently enforce those boundaries. In other words, agent governance should combine behavioural policy with deterministic authorization controls.
Trust in an AI agent is not simply knowing what it can do. It is knowing what it is allowed to do, what it will refuse to do, and when it must ask for additional authority. Explore this Live at AIdentity & Non-Human Identity Impact Day 2026 October 6 in Munich.
Join the in-person conference for analyst insight and practitioner case studies on building this foundation in your own environment.
