The More Powerful the AI, the More the Architecture Matters

The boundaries I designed, the gaps I haven't solved, and why the difference matters

Part 14 findings of an experiment: building an LLM-powered support agent with deterministic boundaries. The companion repo contains the full code.

If someone asked me over a coffee what I've been building, the short version is this.

A support agent that can work out whether you're owed a refund, and cannot give you one.

The AI reads the message and finds the documents. Software decides eligibility, owns the risk tiers, scopes every lookup to whoever is actually logged in, and parks anything expensive in a queue for a human. Fourteen posts, one repo, and the whole thing runs with no API key.

flowchart LR
    M["Customer message"] --> AI["AI: intent, retrieval"]
    AI --> SW["Software: scoping,<br/>eligibility, risk tier"]
    SW --> G{"Gate"}
    G -->|"LOW"| X["Runs"]
    G -->|"MEDIUM to HIGH"| H["Human approves"]
    G -->|"VERY HIGH"| P["Proposed only,<br/>a person executes"]
    classDef step fill:#eef2f6,stroke:#8fa3b8,color:#24313f
    classDef decision fill:#f7f4ec,stroke:#b3a988,color:#24313f
    classDef good fill:#ecf2ed,stroke:#93b39d,color:#3d5344
    class M,AI,X step
    class SW,H,P good
    class G decision

The bit I'd defend hardest

Every guarantee in there has a test that fails when it stops being true — the gate, the scoping, the risk tiers. The one I'd argue about longest is the dullest of them:

/** The only packages a domain class may reference: itself and the JDK. */
private static final String[] DOMAIN_ALLOWLIST = {
        "dev.tonal.support.domain..", "java..", "javax.."
};

@Test
void domainDependsOnlyOnItselfAndTheJdk() {
    classes().that().resideInAPackage("dev.tonal.support.domain..")
            .should().onlyDependOnClassesThat()
            .resideInAnyPackage(DOMAIN_ALLOWLIST)
            .check(production);
}

An allowlist, not a blacklist: the business rules may reference themselves and the JDK, and nothing else. No enumerating of forbidden layers, which would need updating every time I add a package. Add a JSON library to a policy class and the build goes red — I checked, by breaking it on purpose and watching it fail.

It exists because this kind of architecture decays quietly. Nobody announces the afternoon they import a framework into the business rules. It just happens, and a few months later "the AI doesn't decide business questions" isn't true any more — and nobody noticed the day it stopped being true.

A sentence in a README can't catch that. A failing build can.

What I left out

Quite a lot. The interesting question is which parts were decisions.

Multi-tenancy was. Per-tenant risk policies mean the tier lookup becomes runtime configuration, and "who gets to decide" is the last thing I want configurable by a form. I'd want evidence that a second tenant genuinely needs different tiers before I'd trade that away.

Real persistence was too, though it sounds lazier. The approval queue and the audit trail are ports with in-memory adapters behind them, and the ports already spell out what a real implementation owes you: durability, notification, permanence. Writing that adapter is an afternoon. Deciding what it must guarantee was the part worth the thinking.

And no LLM-as-judge in the eval suite, which I keep being tempted by. A model grading a model has its own variance and its own bad days, so it would need its own eval before I'd believe a word of its verdicts. That's a project, not a checkbox.

What's just missing

There's a difference between "I decided not to" and "I haven't got to it".

Saying which is which out loud is the only thing stopping every gap from becoming retroactive wisdom.

Nothing rate-limits proposals, so somebody patient could bury a reviewer in plausible-looking refund requests until approving becomes reflex. An approved refund executed twice after a retry isn't prevented. And nothing notices if the knowledge base quietly gets worse while every test stays green. Those sit in the failure-mode catalogue marked Planned, which is a better place for them than my head.

What none of it proves

Here's the part that would make me sceptical if I were reading someone else's series.

It has never served a real customer. Every scenario in the eval suite is one I invented, which makes it good at catching regressions and useless against the thing I didn't think of. Real users are more imaginative than I am, and they aren't trying to be fair.

The suite also grades deterministic classifiers, so anyone can clone the repo and get my exact numbers. That's a genuine win for reproducibility and a genuine limit on what I've shown: the version swap I graded was one component against another, not one provider against another. And I've never measured end-to-end latency with a hosted model in the loop.

So: a design I believe in, tests for the parts that are testable, and no war stories. If it ever does meet production traffic, the thing I most want to know is whether the audit trail catches what the evals missed.

What's the gap in your system that everyone calls deliberate?