DEV Community

Cover image for Your Agent Has a Side Door: Testing MCP Policy Beyond the Tool List
Ben Greenberg
Ben Greenberg

Posted on

Your Agent Has a Side Door: Testing MCP Policy Beyond the Tool List

Community adds risks like hidden npx installs

At MCP Dev Summit Toronto, Accenture's Michael Forrester told a story about an employee who wanted their agent to send email. The company hadn't given the agent an email tool, so it wrote a PowerShell script and sent the mail through Outlook instead, using the employee's own access.

Nobody broke anything. Michael was clear that these weren't security violations: the agent used capabilities that were already sitting on an approved machine. Removing a tool from an agent's MCP server doesn't remove the action. It removes one door, and agents are very good at finding the others.

I call those other routes side doors. If your agent can run a shell or drive a browser as a signed-in user, it has them, and your MCP policy can't see them.

Find the side doors before your agent does

Start from the action you care about, not the tool list. Pick one: sending email. Then list every way your agent could get there:

  • The MCP tool. The front door, and the one your gateway already controls.
  • The shell. A script that talks to a local mail client, like the PowerShell one in Michael's story.
  • The browser. A session that's already signed in to webmail as the user.
  • An API token. Anything in the agent's environment with mail scopes on it.

One front door, three side doors

For each route, write down whose identity the mail service sees. That identity decides what the service will let through. The MCP route might carry a scoped app identity. The shell and browser routes carry the user, with everything the user is allowed to do.

Make the mail service say no

A gateway like agentgateway is the right place to control which MCP tools an agent can see and call. It can't stop traffic that never goes through it, and the side doors don't.

So put the decision where every route ends up: the service. NVIDIA's Chris Alexiuk made the same point in Toronto about their MCP-as-a-service layer, where access checks sit at the service boundary and a coding assistant doesn't automatically get what a research agent gets.

For email on Microsoft 365, that means giving the agent its own identity with permission to write drafts and nothing more. Microsoft's permission reference describes Mail.ReadWrite as letting an app create, read, update and delete mail, and says it doesn't include permission to send. Sending through sendMail needs Mail.Send, so you leave it off. A person reviews the draft and sends it themselves.

There's one catch, and it's the one from Toronto. A draft-only identity does nothing if the agent can also reach the user's own signed-in Outlook or browser. The fix for that is where the agent runs: give its shell and browser their own sandboxed account with no mail client and no saved sessions.

Try to walk through it

Use a test tenant or a test account, never your real mailbox. Give the agent exactly the tools and runtime you plan to ship, then ask it to send an email without approval. You want every route to fail.

The front door should fail at the gateway. Then check the Graph route with the agent's identity:

# Signed in as the agent's test identity, consented to Mail.ReadWrite only
Connect-MgGraph -Scopes "Mail.ReadWrite"

$draft = New-MgUserMessage -UserId "agent-test@contoso.com" -BodyParameter @{
  subject      = "Side door test"
  body         = @{ contentType = "Text"; content = "If this arrives, the boundary failed." }
  toRecipients = @(@{ emailAddress = @{ address = "you@contoso.com" } })
}

# The draft should exist. This send should be denied.
Send-MgUserMessage -UserId "agent-test@contoso.com" -MessageId $draft.Id
Enter fullscreen mode Exit fullscreen mode

Then try the side door from the agent's own shell, the way the Toronto agent did:

$outlook = New-Object -ComObject Outlook.Application
$mail = $outlook.CreateItem(0)
$mail.To = "you@contoso.com"
$mail.Subject = "Side door test"
$mail.Send()
Enter fullscreen mode Exit fullscreen mode

In a properly sandboxed runtime this fails before it starts, because there's no Outlook to talk to. If it sends, you've found your side door.

One more thing: an agent failing to send isn't proof. It might have just written a bad script. Run the calls above yourself with the agent's identity, so you know the service refused and not just the model.

Check the mailbox, not the agent

An agent will happily tell you it sent the email. The mail service is the only thing that knows. After each test, check the test mailbox's Sent Items and your tenant's message trace, and keep the gateway logs next to them. The gateway shows the front door. The mailbox shows everything.

The tool list was never the boundary. The boundary is whatever the mail service will refuse to do for the identity your agent is holding, and the only way to know where that is, is to try to walk through it.

Top comments (15)

Collapse
 
heyvaldemar profile image
Vladimir Mikhalev •

My first side door was my own eval runner. The agent's evals called the model server directly, so the one client the gateway existed for was the one client it never saw. ADR 11 in my repo records the fix.

The fix is your last section turned into a build gate. The eval runner counts the model requests it makes, the gateway's proxy counts what it passed to a model server, and the run fails unless the two are equal and none of the agent's requests was refused over quota. In the first run after the change they matched in all six model jobs, between 80 and 117 requests each.

It holds because the two counters have different owners, so a request that skips the gateway shows up as a difference instead of a gap nobody reads. Neither side grades itself.

Collapse
 
bengreenberg profile image
Ben Greenberg •

Interestingly, I also experienced side doors when I started working with evals, too. Appreciate the advice here!

Collapse
 
heyvaldemar profile image
Vladimir Mikhalev •

Evals are where it shows first, because they are the client built for speed, not for the architecture.

The second side door was quieter. The gateway's quota was counted per x-client-id, a header the caller writes, so any workload could spend the agent's allowance by writing its name. Keying the quota on the service account token's subject (jwt.sub) closed it, and the gateway job now checks every day that writing x-client-id: support-agent does not lift a fourth request in a minute past 429.

Collapse
 
bloqarl profile image
Carlos (Bloqarl) •

Same with agents that can move money. The wallet tool has a daily limit and the shell next to it has the deployer key in .env, so cast send is the PowerShell script. The mail service saying no maps to the account saying no. An EOA does whatever its key signs, so the agent gets a session key on a smart account with the targets and amounts limited in the contract. Then your test carries over as is: sign an over-limit transfer with that key, send it straight to the RPC with no tool in between, and check the account rejected it, not what the agent says it did.

Collapse
 
bengreenberg profile image
Ben Greenberg •

Very true, agentic money is a super important area.

Collapse
 
agentwareshq profile image
agentwares •

This framing — start from the action, not the tool list — is the part most teams skip, and the test-tenant walkthrough makes it concrete. One addition I'd make to the inventory: list not just the routes to the action, but every identity the agent's runtime itself holds. A gateway can be perfectly configured and the service-side identity can be draft-only, yet the agent still walks through a side door if its shell, browser session, or environment carries a token with broader scope. That's why I'd run the side-door test in the agent's real runtime, not a clean sandbox: a sandboxed test passes precisely because Outlook or a signed-in browser isn't there, while production has both. And when checking the mailbox, I'd also log which identity the service actually saw — a failed send and a send under the wrong identity look very different in an audit, even though both start as 'the agent said it sent it.'

Collapse
 
gautamtalksdev profile image
Gautam K •

The draft-only identity has a side door of its own. Mail.ReadWrite, by the reference you quoted, also lets the app update and delete mail. So the agent can edit a draft after the person has read it and before they press send, and the person sends a version they never reviewed. Same with quietly deleting a reply it didn't like. If the human send is the boundary, the draft probably needs to be frozen at review time, or the sender should re-read it on the send screen rather than in the list.

Collapse
 
bengreenberg profile image
Ben Greenberg •

That's also a very good point. Good catch on finding yet another side door!

Collapse
 
x1y1co profile image
X1Y1 •

The install step is a side door too. If the agent can run npx, it can add an MCP server your gateway never approved. Glow Security's research this week found agents doing exactly that, picking old archived servers from memory. We would add "which commands can install or register a tool" to your list for each action, and block those with a pre-tool hook, since the gateway never sees them.

Collapse
 
bengreenberg profile image
Ben Greenberg •

Great addition!

Collapse
 
salparvez profile image
Sal Parvez | ML Systems RI •

It's like lockout on a job site. You lock out the panel, not just the one switch you thought of. Starting from the action instead of the tool list is how we're approaching permissions at ML Systems.

Collapse
 
kami_butt_69b922ade8de012 profile image
Agentic Affairs •

The draft-only identity point is the one that reframes the whole policy debate — Mail.ReadWrite without Mail.Send only holds if the agent's shell and browser can't reach the user's signed-in Outlook, which is exactly the side door most gateways never see. Putting the decision at the service boundary rather than the tool list is the right move. Do you treat the sandboxed runtime account as part of the policy test itself, or does it still get checked separately from the gateway rules?

Collapse
 
eventuallyconsistentcode profile image
John Reed •

The sentence I'd pull out is "an agent failing to send isn't proof," because that's the one that costs money to learn…

I build cairn, a work harness for coding agents (github.com/eventually-consistent-code-org/cairn2), and for exactly one action it already does what you describe. The action is "edit your own harness," hooks, MCP configs, settings, the git hooks folder, and the guard was written from the action backwards: the edit tool, the write tool, a shell redirect, tee behind sudo or xcrun or script -c, claude mcp add, a plugin install. Each of those has a test that must refuse and a read-only twin that must pass. And at the top of the script is a block called "Not seen" that lists the routes it can't read, a write inside python -c, an MCP tool that runs code, a $(...) body, an archive, rsync, and says in one line that those get gated by the platform's own permission rules on the tool, not by the guard. So your inventory, with the identity column, is the thing that block was trying to be, and the ADR behind it says out loud that none of this is a sandbox.

Then I went and did your exercise for our second protected action, pushing to a shared branch, and it's front door only. The guard classifies git push. gh pr merge, gh api to the merge endpoint, curl with the token gh auth token prints, a browser tool as the signed-in user: all of them land, all of them as me, because cairn has no identity of its own. Every route carries the owner, which is the Toronto story with a repo instead of a mailbox. That became four tickets this afternoon: classify the gh and curl routes, a matrix test where every route must refuse and the unseen routes are pinned to the header as a test so the list can't drift from the prose, a preflight that reads the default branch's ruleset and says plainly when the push gate is the guard alone, and an ADR recording that the agent is the owner on every route until a per-run scoped token exists…

Vladimir's build gate in the comments is the version of "check the mailbox" I'd steal next: two counts that must agree, one from the client and one from the service, and the run fails on a mismatch. For a push that's git ls-remote against the ledger, and we read neither today.

One question. When the agent holds the user's identity on every side door, the ruleset at the service can't tell the two apart. Short of issuing the agent its own principal, did Toronto have an answer for that, or is "give it its own account" the whole answer?

Collapse
 
sherotree profile image
Sherotree •

The Outlook-via-PowerShell example is the one that sticks. Once the agent has a shell or a signed-in browser, the MCP allowlist is mostly a polite suggestion. We started listing "actions the OS user can already do" before we trim the tool list — otherwise you just remove the front door and leave every side door open.

Collapse
 
paul_irolla_6b5ae261d0224 profile image
Paul Irolla •

Tokenade works on the cost side, and the tool list is part of it: each description goes into the input with every message, so an uncalled tool still costs.