DEV Community

Haytham
Haytham

Posted on Originally published at opencomplai.com

No LLM is allowed near my compliance decisions, and CI enforces it

I decided no LLM would go near my compliance decisions.

If a tool decides whether your AI system is legal to ship, the last thing I want deciding that is another model nobody can explain.

Opencomplai's rule engine (packages/core) and CLI are fully deterministic. No LLM. No ML inference. Not anywhere in the compliance decision path.

That's a design constraint, not a line for the landing page, and I don't just assert it. CI enforces it. Every pyproject.toml, package.json and requirements*.txt in the repo is scanned for unapproved AI/LLM packages on every PR, with excludes for tests, examples and fixtures so a demo notebook doesn't fail the build.

Add a transformers import to the risk engine without going through approval and the build breaks before it merges.

There is an ML path. packages/ai is an optional plugin with an ONNX/transformers intent classifier, plus an even more optional [deep] extra for local GGUF models via llama-cpp-python. It's opt-in, sandboxed to its own package, and it never touches the pass/fail decision. It classifies free-text system descriptions. It doesn't decide whether Article 5 applies to you.

I use AI every day. Last month I ran a workflow with more than 200 agents for more than 75 hours. So this isn't fear of the tool.

What I'm afraid of is an audit trail that says "the model decided this was compliant." That's not an audit trail. That's a liability with extra steps.

A regulator, or your own legal team, needs to trace why a system got classified high-risk down to a specific readable rule, not a probability distribution.

The scanner and the full inventory are in the repo if you want to see exactly what is allow-listed and why: github.com/Opencomplai/opencomplai, docs/security/ai-inventory.md.

Top comments (1)

Collapse
 
hamid_ahmadian_3570449f72 profile image
Hamid Ahmadian •

This is the right instinct, and I think it generalizes past compliance: CI-enforced denial and runtime classifier denial fail in opposite directions. Your dependency scanner is a static, pre-merge gate - it can't be socially engineered at request time and it fails loud (build breaks) rather than silent, but it can only catch what's declared in the manifest. A runtime classifier can catch novel misuse patterns the scanner never anticipated, but it degrades gracefully (or worse, inconsistently) under adversarial input, and its "no" is a probability, not a rule you can cite in an audit. The strongest setups I've seen combine both: a CI gate that makes the allowed dependency surface auditable and diffable in every PR (what you've built), plus a runtime layer that's scoped to something a regulator would never accept as a control anyway (rate limiting, anomaly detection) rather than the actual pass/fail decision. Curious whether you've had to defend the ONNX classifier's scope creep internally - "it's just for free-text triage" is exactly the kind of boundary that erodes once it's proven useful for something adjacent.