At MCP Dev Summit Toronto, Accenture's Michael Forrester told a story about an employee who wanted their agent to send email. The company hadn't gi...
For further actions, you may consider blocking this person and/or reporting abuse
My first side door was my own eval runner. The agent's evals called the model server directly, so the one client the gateway existed for was the one client it never saw. ADR 11 in my repo records the fix.
The fix is your last section turned into a build gate. The eval runner counts the model requests it makes, the gateway's proxy counts what it passed to a model server, and the run fails unless the two are equal and none of the agent's requests was refused over quota. In the first run after the change they matched in all six model jobs, between 80 and 117 requests each.
It holds because the two counters have different owners, so a request that skips the gateway shows up as a difference instead of a gap nobody reads. Neither side grades itself.
Interestingly, I also experienced side doors when I started working with evals, too. Appreciate the advice here!
Evals are where it shows first, because they are the client built for speed, not for the architecture.
The second side door was quieter. The gateway's quota was counted per
x-client-id, a header the caller writes, so any workload could spend the agent's allowance by writing its name. Keying the quota on the service account token's subject (jwt.sub) closed it, and the gateway job now checks every day that writingx-client-id: support-agentdoes not lift a fourth request in a minute past 429.Same with agents that can move money. The wallet tool has a daily limit and the shell next to it has the deployer key in .env, so
cast sendis the PowerShell script. The mail service saying no maps to the account saying no. An EOA does whatever its key signs, so the agent gets a session key on a smart account with the targets and amounts limited in the contract. Then your test carries over as is: sign an over-limit transfer with that key, send it straight to the RPC with no tool in between, and check the account rejected it, not what the agent says it did.Very true, agentic money is a super important area.
The draft-only identity has a side door of its own. Mail.ReadWrite, by the reference you quoted, also lets the app update and delete mail. So the agent can edit a draft after the person has read it and before they press send, and the person sends a version they never reviewed. Same with quietly deleting a reply it didn't like. If the human send is the boundary, the draft probably needs to be frozen at review time, or the sender should re-read it on the send screen rather than in the list.
That's also a very good point. Good catch on finding yet another side door!
This framing — start from the action, not the tool list — is the part most teams skip, and the test-tenant walkthrough makes it concrete. One addition I'd make to the inventory: list not just the routes to the action, but every identity the agent's runtime itself holds. A gateway can be perfectly configured and the service-side identity can be draft-only, yet the agent still walks through a side door if its shell, browser session, or environment carries a token with broader scope. That's why I'd run the side-door test in the agent's real runtime, not a clean sandbox: a sandboxed test passes precisely because Outlook or a signed-in browser isn't there, while production has both. And when checking the mailbox, I'd also log which identity the service actually saw — a failed send and a send under the wrong identity look very different in an audit, even though both start as 'the agent said it sent it.'
The install step is a side door too. If the agent can run npx, it can add an MCP server your gateway never approved. Glow Security's research this week found agents doing exactly that, picking old archived servers from memory. We would add "which commands can install or register a tool" to your list for each action, and block those with a pre-tool hook, since the gateway never sees them.
Great addition!
It's like lockout on a job site. You lock out the panel, not just the one switch you thought of. Starting from the action instead of the tool list is how we're approaching permissions at ML Systems.
The tool list is only the surface. The policy has to cover the transport too, because a relay or a proxy on the same host can call a tool the agent never declared. Testing the side door means denying by default at the gateway instead of only filtering what the model sees.
The sentence I'd pull out is "an agent failing to send isn't proof," because that's the one that costs money to learn…
I build cairn, a work harness for coding agents (github.com/eventually-consistent-code-org/cairn2), and for exactly one action it already does what you describe. The action is "edit your own harness," hooks, MCP configs, settings, the git hooks folder, and the guard was written from the action backwards: the edit tool, the write tool, a shell redirect, tee behind sudo or xcrun or script -c, claude mcp add, a plugin install. Each of those has a test that must refuse and a read-only twin that must pass. And at the top of the script is a block called "Not seen" that lists the routes it can't read, a write inside python -c, an MCP tool that runs code, a $(...) body, an archive, rsync, and says in one line that those get gated by the platform's own permission rules on the tool, not by the guard. So your inventory, with the identity column, is the thing that block was trying to be, and the ADR behind it says out loud that none of this is a sandbox.
Then I went and did your exercise for our second protected action, pushing to a shared branch, and it's front door only. The guard classifies git push. gh pr merge, gh api to the merge endpoint, curl with the token gh auth token prints, a browser tool as the signed-in user: all of them land, all of them as me, because cairn has no identity of its own. Every route carries the owner, which is the Toronto story with a repo instead of a mailbox. That became four tickets this afternoon: classify the gh and curl routes, a matrix test where every route must refuse and the unseen routes are pinned to the header as a test so the list can't drift from the prose, a preflight that reads the default branch's ruleset and says plainly when the push gate is the guard alone, and an ADR recording that the agent is the owner on every route until a per-run scoped token exists…
Vladimir's build gate in the comments is the version of "check the mailbox" I'd steal next: two counts that must agree, one from the client and one from the service, and the run fails on a mismatch. For a push that's git ls-remote against the ledger, and we read neither today.
One question. When the agent holds the user's identity on every side door, the ruleset at the service can't tell the two apart. Short of issuing the agent its own principal, did Toronto have an answer for that, or is "give it its own account" the whole answer?
The Outlook-via-PowerShell example is the one that sticks. Once the agent has a shell or a signed-in browser, the MCP allowlist is mostly a polite suggestion. We started listing "actions the OS user can already do" before we trim the tool list — otherwise you just remove the front door and leave every side door open.
The draft-only identity point is the one that reframes the whole policy debate — Mail.ReadWrite without Mail.Send only holds if the agent's shell and browser can't reach the user's signed-in Outlook, which is exactly the side door most gateways never see. Putting the decision at the service boundary rather than the tool list is the right move. Do you treat the sandboxed runtime account as part of the policy test itself, or does it still get checked separately from the gateway rules?
Tokenade works on the cost side, and the tool list is part of it: each description goes into the input with every message, so an uncalled tool still costs.