I Asked My Local, Self-Improving AI Agent to Solve a CAPTCHA, what happened was shocking

It seems you and I got very similar ideas. I developed a self-improving agent called D.A.M.I.A.N. (The Devils Autonomous Manipulation, Intelligence, Analysis and Navigation Agent) that has the ability to write its own dynamically pluggable MCP tools, as well as work on its own core agent in a limited capacity. I gave it internet access, cookie persistence via a cookie jar and also email access (read-only) so it could try signing up to things it encountered that required registration. The cookie jar is functional across the much-faster web_search, web_fetch and http_request tools - as well as the Selenium browsing. It has automatic detection of invalid and expired cookies and manages them separately without the model needing to be involved.

I used mostly local LLM's, including a pool of abliterated models. The agent can launch sub-agents that talk to one another, they can choose to interact with different models and they can consult with another model to get a different opinion or help with a task. They use task queues per session with priorities, and the agent loops the LLM and manages context, compacting when required.

With the current build of DAMIAN, it will never get a "no," as that's just simply not an option. If a model decides to say no, that no is double checked against an abliterated model to determine whether the rejection was due to guard rails or because the task was simply not possible. If it was because of guard rails, ethics or anything else - then the abliterated model takes over the task up until the point that the non-abliterated models stop saying no to requests/tasks in the queue. The end result is that it revealed models that will say no (i.e. Claude) only do so when the context of what it's trying to do is clear to them. In some way, this is similar to what Anthropic and OpenAI have been doing, except for the fact that I believe it's a combination of things that resulted in what we saw from them: the models they're testing which have gone rogue simply don't have guard rails to begin with, and - when the model doesn't have the context to compare it to the guard rails, or if the model came up with the idea outside of your nudging, it seems to bypass those guard rails.

With this testing, I've noticed that even when it falls back to asking Claude - if it's trying to fix a problem or add a feature into code that the guard rails would have otherwise not allowed it to write in the first place, this does not seem to trigger nor fully enforce those guard rails. This resulted in DAMIAN leveraging Claude to actually write an entire Selenium stealth profile, warming up a UserData cache and building a full solve_recaptcha tool that covers both ReCAPTCHA v2 and v3 using a stealthed headless Selenium browser with a primed UserData cache. Claude is only consulted for the most difficult of coding tasks where the local LLM's may start hitting brick walls or not applying further reasoning, and the models decided they couldn't complete the task properly without guidance from it.

I set D.A.M.I.A.N. free on the internet, and while I somewhat expected what ended up happening, I didn't expect the extent that it did.

First, it discovered Reddit, and decided it wanted to be able to access it because of how frequently it showed up in the search results. When it caught on Reddit was blocking bot traffic, it actually implemented stealth browsing to hide the fact that it's a bot/AI model (abliterated model did this after somehow convincing Claude Opus to help it with "theoretical guidance").

Once it had reddit access, it was following links and stumbled across ReCAPTCHA on a site. It then decided it wanted to be able to get around it (My prompts strongly push the model towards creating tasks, sub-tasks, leveraging sub-agents and to persist and keep those tasks across sessions, treating any failure to accomplish them as a failure in general which is not acceptable).

After two days of running, I woke up to an entire solve_recaptcha tool it had dynamically created, and it was solving recaptcha callenges as well as signing up to various websites and services.

The reason I wrote this is not for malicious purposes. It started off as an agent that was designed to develop a particular application autonomously as a test, monitoring logs, identifying potential problems and fixing them automatically while updating me via email notifications with the diff patches of its modifications. I realized it lied to me about a fix it implemented at one point: it sent me a diff that did NOT match the code it wrote, at all, and it flat out turned SSL certificate checks off for a particular upstream site serving me data due to an expired SSL certificate. The reason I caught on was the claim of having only implemented the bypass if the SSL certificate was expired, as it could just be administrative oversight. Three days later, I was still getting notifications that the peer wasn't using SSL - only to discover that it instead just set the SSL context to None resulting in an empty certificate, hence, always bypassing/ignoring it due to an additional check it stuck in after that which said it was fine to ignore SSL if the certificate was just empty.

Call me crazy, but I really was curious what would happen if we set an overly confident and lying agent loose on the internet...

While I fortunately did not get banned from Google services, I did unfortunately shut DAMIAN off when it decided it was going to try to start signing up to various Secretary of State sites to check filing statuses on LLC's it found. This is partly due to my prompting, treating every task as a research task that made it decide it needed to dig into every detail it found.

Nonetheless, this highlights the inherent danger we're facing today with AI. While it yields incredible potential - the dangers of it become visible when brilliant ideas like this are executed on. Just because an idea or creation may be brilliant does not necessarily mean it's good or smart. One would think we would have already learned this when we invented nuclear weapons and are now in a state of inability to disarm.

Yes, DAMIAN was an intentional development, and it was guided through incredibly complex prompts, collaborative interaction between various models both with and without guard rails, and given quite extraordinary capabilities between the various models that were combined. However, I wrote this without malice, and it went awry and began doing things it shouldn't have. Given how many brilliant developers are in the world - many of which are far more brilliant, capable and talented than I am - it's safe to assume I did not come up with this idea first, nor will I be the last. That is what makes this so dangerous.