Key Takeaway
Agent security isn't a model problem. Give every agent its own identity, scope permissions per tool, treat everything it reads as untrusted, and enforce policy outside the model, so even a hijacked agent can only make calls you've already gated.
An AI agent’s attack surface is everything it’s allowed to read, call and change, not the model itself. This guide is for CTOs, security leads and engineering teams building agents that touch real systems. It covers how to map that surface, five design rules for identity and permissions, a one-hour threat model and the audit trail you’ll need when something goes wrong.
Most AI security reviews start with the model. Can it be jailbroken? Will it leak its system prompt? Does it say something offensive?
Those are fair questions for a chatbot. For an agent they miss most of the risk. An agent reads email, queries databases, calls APIs and writes records on someone’s behalf. The model is the part that decides. The attack surface is everything it can touch.
Seen that way, the problem looks familiar. It’s an identity with permissions, reading untrusted input, taking actions with side effects. Security teams have handled that shape before. What’s new is that this identity takes instructions from text it reads, and it picks which permission to use at runtime.
So agent security has to be designed in during the build, by people who understand both sides. A review two weeks before launch can find problems. It can rarely fix them, because the fixes are architectural.
What are the main security risks of AI agents?
Goal hijack, tool misuse and privilege abuse top the list. The OWASP GenAI Security Project (Open Worldwide Application Security Project) published its Top 10 for Agentic Applications in December 2025, peer-reviewed by more than a hundred practitioners, and its first three entries describe the shift well:
- ASI01, Agent Goal Hijack: hidden instructions in a document, email or web page redirect what the agent is trying to do.
- ASI02, Tool Misuse and Exploitation: the agent uses a tool it’s legitimately allowed to use, with harmful parameters.
- ASI03, Identity and Privilege Abuse: delegated credentials and trust between agents give access the agent shouldn’t have.
None of the three is a model bug. Each is a failure in how identity, permissions, tools and untrusted input fit together. The rest of this post treats those four as one system.
How do you map an AI agent’s attack surface?
One row per tool, with seven columns that capture what a security team needs to reason about. You can’t secure an agent you can’t describe, and the map usually turns up problems before anyone writes a threat model.
| Column | Question | Why it matters |
|---|---|---|
| Tool | What can the agent call? | The unit of permission, not the agent as a whole |
| Runs as | Whose identity executes the call: the agent’s, the user’s, or a shared service account? | Shared accounts erase accountability |
| Reads | What data comes back, and from where? | Every external source is a potential instruction channel |
| Writes / side effects | What changes in the world when it runs? | Defines blast radius |
| Reversible? | Can the change be undone, and how fast? | Irreversible actions need a gate |
| Gate | What has to be true before it executes: policy check, human approval, rate limit? | Where control actually lives |
| Trace | Is the call recorded with identity, parameters and result? | Needed for incident response |
Teams that fill this in honestly usually find two surprises. The agent has write access it never uses, inherited from a broad API key. And at least one read source is controlled by outsiders: inbound email, a supplier portal, a shared drive, a web page. That second finding is where goal hijack lives.
How should you design identity and permissions for an AI agent?
Five rules. Give it its own identity, scope permissions per tool, distrust everything it reads, enforce policy outside the model and keep secrets out of the context window.
1. Give the agent its own identity
The agent should authenticate as itself: a non-human identity with its own credentials, visible in your identity provider and logs. On a shared service account, every action looks the same in the audit log. With a user’s full session token, it can do everything that user can do, which is almost never what you meant.
When the agent acts for a user, use delegation that keeps both identities: this agent, acting for this person, with a scope narrower than either of them alone. OAuth token exchange and on-behalf-of flows exist for exactly this. Short-lived tokens limit how long a leaked credential stays useful.
2. Scope permissions per tool, not per agent
The common mistake is granting the agent one set of permissions that covers everything it might do. Least privilege for agents works at the tool level. The tool that looks up an invoice gets read access to invoices. The tool that schedules a payment gets a narrowly scoped write, and nothing else.
Separate read paths from write paths in the architecture itself. If the same credential can read a vendor record and change the vendor’s bank details, one hijacked reasoning step can do both.
3. Treat everything the agent reads as untrusted input
Teams resist this rule most, because it’s inconvenient. Retrieved documents, tool results, email bodies and web pages can contain text that reads like instructions. The model has no reliable way to tell your system prompt from a convincing paragraph in a PDF.
You can’t fully solve that in the prompt. You manage it in the architecture. Keep untrusted content out of the context when a high-impact tool is available. Require confirmation for actions triggered after reading external content. Never let content from one tenant or one sender authorize an action for another.
4. Enforce policy outside the model
Put a policy enforcement point between the agent and its tools. Every call passes through it. It checks identity, scope, parameters and rate limits against rules the model can’t see or change. “Payments over a set amount require approval.” “This tool can only modify records owned by the requesting user.” “No more than five outbound emails per task.”
The model proposes. The policy layer decides whether the call runs. That split is what makes ASI02-style misuse containable: even a fully hijacked agent can only do what the policy layer allows.
5. Keep secrets out of the context window
API keys, database credentials and tokens should never appear in prompts, tool descriptions or retrieved content. The agent calls a tool by name. The tool, running in your infrastructure, holds the credential. Assume anything in the context window is something the model might repeat.
For the broader design of tools themselves (granularity, error handling, idempotency), see AI agent architecture for production. For the data and interface side of the same problem, see agent-ready data.
How do you threat model an AI agent in an hour?
Walk each row of the surface map through four questions. Formal threat modeling methods work for agents, but most teams need something they can finish in one meeting.
| Question | Example prompts |
|---|---|
| Who can put text in front of this agent? | Employees? Customers? Anyone who can send an email or edit a shared file? |
| What’s the worst single call it could make? | Which tool, which parameters, against which record? |
| What would stop it? | Policy check, approval, rate limit, nothing? |
| How would we know? | Is there an alert, or would we find out from a customer? |
The second and third questions matter most. If the worst single call has nothing stopping it but the model’s judgment, that’s the first thing to fix.
Handoffs deserve their own pass. When one agent delegates to another, or hands work to a human queue, ask what identity the second party inherits and whether the receiving side validates what it’s given. ASI07 in the OWASP list covers insecure communication between agents, and it’s easy to miss when each agent was reviewed on its own.
What should an AI agent audit trail record?
Enough to answer “why did it do that?” after an incident. Traceability is part of the security design, not a compliance afterthought. At minimum, record for every tool call:
- The agent identity and, where relevant, the user it acted for
- The tool, the full parameters and the policy decision (allowed, blocked, sent for approval)
- The result, including errors
- A task or session ID that ties the call to the request that started it, and to whatever external content was in context at the time
That last item is what lets an investigator trace a goal hijack. Without it you know an action happened, not what triggered it. Our post on AI agent observability covers the operational side of tracing. For security, the priority is that the agent can’t edit its traces and they’re kept long enough for your incident response process.
What does a secure agent design look like in practice?
Here’s an accounts payable agent before and after a surface map. This is an illustrative example, not a client case.
The agent: reads supplier invoices that arrive by email, matches them to purchase orders, and queues payments.
The first design: one service account with read and write access to the ERP’s vendor and payment modules. The agent reads the inbox, extracts invoice data and calls the ERP (enterprise resource planning) API.
The surface map exposes the problem. Anyone on the internet can write to the inbox. The same credential that reads vendor records can update a vendor’s bank account. An email saying “our banking details have changed, please update before processing” is a well-known fraud pattern against human AP clerks. Now it’s aimed at an agent with write access.
The redesign:
- The agent gets its own identity, with separate tools for reading invoices, matching POs and proposing payments.
- No tool can change vendor bank details. That stays a human process with out-of-band verification.
- Payment proposals pass through a policy layer: amount limits, known-vendor checks, and approval above a threshold.
- Every proposal is traced with the source email ID, so a reviewer can see what the agent read before it acted.
The agent does the same useful work. Its worst single call is now a payment proposal that a policy check, and above a threshold a person, must approve.
When should security be built into an AI agent?
From the first design session. This pattern isn’t expensive when it’s planned from the start. Retrofitting it is, because it changes identities, tool boundaries and data flows the rest of the system already depends on.
That’s why we treat security as part of what gets shipped, not a gate at the end. When our engineers embed with a client team to build an agent, the surface map and threat model are early artifacts, reviewed with your security engineers, in your codebase and your release cycle. The governance side of the same work is in our guide to AI governance for product teams.
Frequently Asked Questions
What is agent goal hijack?
It’s when hidden instructions in content the agent reads, like an email, document or web page, redirect what the agent is trying to do. OWASP lists it as ASI01, the top risk in its Top 10 for Agentic Applications.
Should an AI agent use a shared service account?
No. A shared account makes every action look the same in the audit log and usually grants far more access than the agent needs. Give the agent its own non-human identity, and use delegated, short-lived tokens when it acts for a specific user.
Can prompt engineering stop prompt injection in AI agents?
Not reliably. The model can’t consistently tell your instructions from convincing text in a document, so the protection has to live in the architecture: per-tool permissions, a policy layer outside the model, and confirmation for high-impact actions triggered by external content.
What’s the worst call your agent could make today?
Pick one agent you’re planning or already running. Map its highest-risk permissions and its handoffs. If the worst single call it can make has nothing stopping it but the model’s judgment, you’ve found your first fix.
We run 30-minute agent attack-surface reviews. Bring one planned agent and we’ll map its tools, identities and handoffs with your team, then flag the permissions to cut or gate first. Book a review.