To an agent reading a UI, "Delete database" and "Download report" look equally clickable.
Same rectangle, same cursor, same click. Nothing in the interface tells the agent that one action is harmless and the other is catastrophic.
Humans have instincts about consequences. We hesitate before the red button. We double-check before wiping something. Agents have no such instinct — they act on patterns, not consequences.
That's the real problem with giving agents raw access to your tools and dashboards. It's not that agents are reckless. It's that every action is equally weightless to them until it's done.
One wrong click at 3 AM with no human watching, and "Delete database" is just another completed task in the log.
This is why permissions and approvals aren't optional safety theater for agent systems. They need to be structural: which actions are safe to run alone, which ones need a human sign-off, and which ones the agent should never be able to reach.
The agent doesn't need better judgment. The system around it needs better guardrails.
What's the scariest action you've ever let an agent take unsupervised?
Top comments (1)
We had this same issue with a deduplication agent last year - it'd routinely try to discard ambiguous clusters using the same call it used to finalize confirmed merges, because to it they're just two write operations. Spent a few days adjusting prompts before accepting that the model literally couldn't tell them apart from text alone. The fix was hiding the destructive tool from the default schema and requiring a separate explicit call to even access it. Kind of annoying to work around, but nothing else reliably stopped it.