Picture a release with a perfect paper trail. You have the reviewed commit, the resolved dependencies, the builder identity, the artifact digest, and a green record of every check that ran. All of it is signed.
It can still contain a bug that lets anyone mark an invoice as paid.
Provenance is worth having, and AI-assisted development makes it more useful, because teams increasingly need to reconstruct how a change came together. The trouble starts when a complete record begins to feel like evidence that the code is good.
The audit trail vs. the actual behavior
Provenance answers questions like these:
- Which commit is in this artifact?
- Who approved it?
- Which dependency versions were resolved?
- Which build produced it, and which checks ran?
Those answers are valuable during an incident, and they help you reproduce a release. None of them tells you whether the code does the right thing.
The scenarios below are illustrative, not real incidents.
Failure mode 1: faithfully recording the wrong code
An assistant helps write a payment webhook:
app.post("/webhooks/payments", (req, res) => {
const sig = req.get("X-Signature");
if (sig && !verifySignature(req.rawBody, sig)) {
return res.sendStatus(401);
}
markInvoicePaid(req.body.invoiceId);
res.sendStatus(200);
});
A bad signature gets rejected. A request with no signature at all skips the check and marks the invoice paid.
The build can be hermetic and the attestation can verify cleanly. The record is accurate: this exact code, built this exact way, reached production. Build integrity tells you the artifact wasn't tampered with on the way there, and frameworks like SLSA are aimed at that problem. Whether the source was right to begin with is a separate question.
Failure mode 2: "checks ran" gets read as "checks were enough"
The tests for that webhook might cover a valid signature and an invalid one. Both pass, so the record says unit-tests: pass, and that's true.
The missing case is the one that matters:
test("rejects a webhook with no signature", async () => {
const res = await request(app)
.post("/webhooks/payments")
.send({ invoiceId: "inv_123" });
expect(res.status).toBe(401);
});
A provenance record can show that a job ran and passed. It can't show what the job never tried.
Failure mode 3: a record of review isn't a record of scrutiny
approved_by: reviewer-a is a fact about a click. It doesn't tell you whether the reviewer traced the unsigned path, or spent four minutes on a large AI-assisted diff near the end of the day.
Keep the approval record. Just don't let it stand in for the review itself. On sensitive changes, the reviewer should be able to answer a specific question: what happens when this header is missing?
Failure mode 4: a clean scan is a snapshot
A dependency scan can pass on release day, and a vulnerability in one of those packages can be published a week later. The record of that clean scan stays true while the risk changes.
Here, provenance earns its keep. If you recorded exactly what shipped, you can answer "are we affected?" in minutes instead of guessing. Just treat the old scan result as a dated observation, and keep scanning what's already deployed.
Failure mode 5: the audit log becomes the leak
It's tempting to capture everything an AI tool touched, including full prompts. But prompts pick up whatever a developer pasted in: a stack trace with a customer's email address, a production log, a connection string.
Now that material sits in a long-lived record that may be widely readable, kept in the name of traceability.
Most incident questions don't need the raw prompt. Something like this usually goes a long way:
{
"artifact_digest": "sha256:...",
"commit": "9f3c1ab",
"pull_request": 4821,
"approved_by": ["reviewer-a"],
"lockfile_sha256": "...",
"builder": "ci/release.yml@refs/tags/v2.14.0",
"checks": [
{ "name": "unit-tests", "result": "pass" },
{ "name": "static-analysis", "tool_version": "...", "result": "pass" }
],
"ai_assisted": true,
"ai_tool": "assistant-name@version"
}
That shape is only an example. If you want a standard format, look at SLSA provenance and in-toto attestations. If your team does need to keep prompts, redact them, limit who can read them, and give them a retention period.
What goes in the record, and what still has to happen
Record what helps you explain and reproduce a release:
- Commit, PR, and approvers
- Resolved dependencies
- Builder identity and artifact digest
- Which checks ran, with tool versions and results
- Whether AI tools were involved, and which ones
Then do the work the record can't do for you:
- Code review: is this the right behavior?
- Testing: does it do what the requirement says, including what it has to refuse?
- Security analysis: are there risky patterns, data flows, vulnerable dependencies, or committed secrets?
Provenance is the part that lets you trace and reproduce a release. Those three are how you find out whether it should have shipped.
A signed record tells you where the software came from. It takes a different kind of work to know whether it's safe.
What does your team actually record for AI-assisted changes, and has any of it helped you during a real incident? Do you keep prompts, or did you decide against it?
Top comments (3)
Spot on, Dimitris! This is a crucial distinction that often gets blurred in DevSecOps discussions. Build attestation proves provenance (who built it, from what source, with what tools), but it says absolutely nothing about the inherent safety of the code itself.
In the Web3 space, we see this exact gap. A team might have a perfectly attested, OIDC-secured CI/CD pipeline (proving the chain of custody is intact), but a developer could still accidentally commit a hardcoded test key or introduce a subtle logic flaw. The attestation will happily sign off on that vulnerable artifact.
This is why attestation must be paired with automated, context-aware security gates in the pipeline. In my work, I build custom Slither detectors that run as a hard fail in CI/CD. The attestation proves the detector ran in a trusted environment, but the detector is what actually validates the safety of the smart contract.
Curious to hear your take: As attestation frameworks (like SLSA) become more standard, do you see security teams struggling with 'scanner fatigue' from false positives, or is the real bottleneck still just getting teams to adopt the attestation infrastructure in the first place?
I think the bigger bottleneck is still getting teams to adopt the infrastructure and, more importantly, understand what each layer actually guarantees. Scanner fatigue is definitely real, though. If teams get flooded with false positives, they eventually start treating security gates as another checkbox rather than something worth investigating.
I like your Slither example because it highlights the distinction nicely. The attestation gives you confidence that the checks ran in a trusted environment, while the detector provides a separate layer of evidence about the code. Neither replaces the other. I think the real challenge is making these layers work together without turning the pipeline into noise.
The missing-signature webhook is the perfect example — the attestation is accurate, the artifact is exactly what was built, and the invoice still gets marked paid. Failure mode 3 is the one I'd put on a wall: approved_by is a fact about a click, not a record of scrutiny. Do you find teams actually change anything once they see that distinction, or does the green checkmark keep winning the argument?