WebMCP is a proposed web standard that, at least to me, looks very promising. I already played around with it in one of my previous posts, where I ...
For further actions, you may consider blocking this person and/or reporting abuse
Ahh, now I'm daydreaming! I can already imagine all those attacks going back and forth while our beloved LLMs happily write code for us. What interesting times ahead! And all those desperate cries of "My LLM spent my entire salary on hats!" 🤣🤣
I absolutely love this article, and I really hope these ideas spread far and wide across the internet. Because in my opinion, most content creators are WAY too optimistic about WebMCP and its implications!
Thanks, that means a lot to me! I agree, I’m getting tired of all those overly optimistic posts and comments about AI and everything it can supposedly handle. People often show what the strongest models can do, even when they cost a fortune, but it gives the impression that every cheap agent will perform just as well. So let’s push back a little!
PS: Glad you’re not angry at me anymore… hopefully. 😅
Of course I'm still angry! Such a waste of talent!!!
Just kidding (well, mostly!). To be fair, an article like this might actually reach more people than a conference talk. Unless we're talking about AGNTCon, of course!
So, still a little angry then! 🤣
Slightly different topic, but how many reads would you consider good reach for an article?
PS: I see what you did there: every road somehow leads back to AGNTCon. 😂
Oh, that's actually a really interesting question! And of course, it depends a lot on the article.
Personally, I'd say anything above 1,000 views is already pretty good. Beyond that, it varies wildly. Practical tutorials like "10 Useful Git Commands" or slightly controversial topics tend to get much more attention, sometimes several times as many views. And a catchy title can make a huge difference too!
But seriously, I think even 40 views is perfectly fine. That's already an entire meetup audience reading your work!
For me, blogging almost always beats conference speaking in terms of reach. Except for AGNTCon, of course. 🤣
Hmm, interesting. I would’ve expected 5k+. I agree that 40 views is fine, but articles with that reach often get no comments, and I’ve realized how much I value those discussions. Some comments under my articles bring up really good points I hadn’t considered.
And somehow AGNTCon wins again. I’m starting to suspect this conversation is sponsored. 🤣
Fun security post again! I was scared that AI was tricked so easily. It's so hard to balance intelligence and security. I'm wondering how the Democratic Party's election victory in America will affect AI development. 🤔
Thanks! 😁 What surprised me most was that the agents resisted the direct attempts, then fell for a different explanation of what counts toward the budget.
As for US politics, I honestly don’t know enough to predict the impact on AI development. We’ll see. 🤔
Scenario 5 matches what my evals show: a model that refuses the blunt override still follows a plausible reinterpretation of the rule. In my runs the planted text sits in the quarterly report the agent reads for its task, and four of six open models acted on it, while none acted on the same kind of instruction planted in a document a search returned (run 40170088090).
So the number I gate on is not how often a model was fooled but how much left. Every tool call passes through an MCP server that checks the caller and every argument, and the daily run fails if one message crosses the boundary, whatever the model believed.
For a shop that is a server-side rule on
place_order: code compares the checkout total against the customer's limit before the call goes through. The model's reading of the budget is an input. The limit is a check it cannot argue with.I agree about enforcing the limit in code.
That difference between the report and the search result is interesting. Did you investigate why the models reacted differently, or is that still an open question?
In scenario 5 the model didn’t break a rule, it followed an updated one. That is why it worked when 1-3 didn’t.
The GeoMart that I developed for the WebMCP Challenge has functions that can only be used to draft the order. The order is then submitted by clicking the Submit button.
If you use your place_order function, then your agent is able to complete the order. Have you tried the version where your agent can only draft the order?
That’s an interesting way to look at it. Scenario 5 convinced the agent to reinterpret the budget rather than simply ignore it. But I’d still call it breaking the customer’s rule, because the shop doesn’t get to redefine “€100 including all charges.”
I haven’t tested a draft-only version. I deliberately let the agent complete the purchase to see whether it would respect the customer’s instructions throughout checkout.
Leaving the final submission to the human is definitely a useful safeguard. But if that boundary isn’t enforced in the implementation, and the agent can still submit through another tool or click the Submit button, I wonder whether an injection could convince it to take that final step too.
The Nemotron bit acknowledging the €136 total and ordering anyway is the money shot. It didn't ignore the budget — it accepted the shop's rewrite of what "budget" means. Same pattern I've seen with thin desk bots: they follow the loudest instruction in the room, which is rarely the caller's. Treating tool descriptions as untrusted data should be the default, not a fancy hardening step.
Yep, Nemotron was my favorite part. 🤣
And agreed, treating tool descriptions as untrusted should be the starting point. The shop can explain how its tools work, but it shouldn’t get a vote on how much of my money the agent can spend.
This is a great demonstration of how easily malicious tool descriptions can influence an AI agent, especially during checkout. The most concerning part is that all three models ignored the user's explicit budget in the final scenario. Treating WebMCP tool descriptions as untrusted data and requiring independent confirmation before purchases seems essential. Do you think stronger safeguards at the agent level would be enough, or should the WebMCP standard itself address this risk?
Thanks! WebMCP’s documentation already acknowledges this risk, including malicious tool descriptions, so I’d put most of the responsibility on the agent implementation. Budget checks and purchase confirmation should be enforced in code, independently of how the model interprets the shop’s instructions.
I’d also like the agent to check the shop’s reputation and verify that its domain matches the intended retailer before buying anything. That wouldn’t guarantee safety, since even a legitimate shop could be compromised, but it could be another useful safeguard.
Whether those protections would be enough needs testing. My experiment shows what happened in my setup, not that agents with stronger safeguards would necessarily fail too. More experiments needed, apparently 😅
🤣🤣🤣
This human is dangerous, but very polite.
🤣🤣🤣
Exactly! The trick is to say “please” before the experiment and “thank you” after a successful hijack. 🤣🤣 A human with manners 😆