Agents Can’t Decide What’s Okay to Break

There’s a lot of discourse out there about the engineering bottleneck moving from time spent writing code to time spent deciding what to build. At Major League Hacking (MLH), where I run product and engineering, we’ve found a different bottleneck: knowing what’s safe and justified to break, and how those breaks can cascade to impact other systems and people.

This year MLH took the lead on running Hacktoberfest. We pivoted away from an event focused on incentivizing pull requests to an event focused on experimenting with, building, and using open-source and open-weight AI tooling largely through gathering and hacking with your peers. In just the last two weeks, more than 80,000 people have signed up and nearly 40,000 have actually participated across 500+ in-person and online events, all using a brand-new portal to track their progress. We also had real incentives on the line, the kinds that drive internet developer frenzies like free swag, promo credit for popular platforms, and more. Every access vector we opened up was a door someone was inevitably going to try running through.

We had a short timeline to build everything, and it sparked a good amount of internal debate. Which endpoints should we open? How would our API refactors affect other systems? Could we handle Hacktoberfest-scale load in a brand-new vibe-engineered portal? Some of our engineers were hesitant, and for good reason. Our coding agents would have made any change we asked for. What the agents couldn’t see was who else depended on those endpoints: internal tools, partner integrations, teams downstream, and other interconnected systems.

So we had to decide on a risk appetite for the whole project around things like how we secure promo codes, how people unlock swag, and how much gaming of our rules we’re willing to tolerate (philosophically and financially). A shared $5 credit? Fine, anyone with an account can grab it. Scarce, high-value codes? Those needed real gates. Then every engineer had to translate that high-level risk decision into dozens of small decisions on their own, like which API endpoints to open up, how they’re permissioned, and how aggressively to rate limit logins.

Early on we learned that not all anti-fraud gates are created equal. Put free credit behind something too easy to bypass and people will happily spin up fresh accounts to grab some. So we moved the line. For the most in-demand offer we required a verified phone number, a GitHub account older than 30 days, and participation in our flagship online event, and over 90% of the codes went to legitimate participants as a result. By the time the spammers showed up, the codes were already gone and in the hands of real hackers. Risk appetite isn’t a fact that an agent can look up in the codebase. It’s a business decision, and someone has to own the consequences, including when you get it wrong.

Some of this context can be documented and made available through internal tooling and knowledge bases. As a counterpoint to the ambiguity of Hacktoberfest, people across MLH used to constantly ask our engineering team for production stats, and we basically would just take someone’s question, plug it into an LLM to generate SQL, sanity check it, and give back the result. The sanity check was the actual job: knowing stuff no model would guess, like the fact that the cached registration counts on our events can lag behind the real numbers. To save ourselves time we built a Slack bot called Coreacle, backed by documentation of how our data actually behaves, with audit logs on every query. In the past month alone 80% of our team has asked it a question, and the docs we wrote for it now teach our coding agents how our data model works too. But even Coreacle started with a judgment call no documentation could make: how much direct access to real business data we were comfortable giving the whole team.

As a founder, every day I’m asked to make decisions about which tradeoffs are safe and which breakages are worth it to achieve a larger goal. That kind of business judgment — whether to open up a new API, build a new internal tool, and much more — is moving downstream to individual engineers, because agents have compressed the time between a request and a deploy way too much for every call to route through someone like me. The engineers on your team need to learn to tell an optimal technical solution from a worthwhile business tradeoff. And as leaders, our job is to share the stakes with them directly, so they can ship faster and with more confidence in the business outcome than our pre-agentic workflows ever allowed.