My drift detector quarantined a healthy agent. The agent hadn't gotten worse — the benchmark had gotten bigger. The detector compared failure counts across corpus versions, so growing the benchmark from 10 tasks to 20 made a 10/10 agent look like it "failed" half the suite. Corpus growth looked like decay.
I built the drift detector for HivePlane v0.2.0 to catch the failure mode I'd written about in v0.1.0 — "the dangerous agent usually drifted, not got edited." The harder problem turned out not to be detecting drift. It was not detecting drift that wasn't there. A detector that cries wolf trains operators to ignore the signal, and an ignored gate is no gate.
The detector that ate its own benchmark
| Finding | What the detector did | Why it was wrong |
|---|---|---|
| #530 | Compared failure counts across corpus versions | Growing the benchmark auto-quarantined a healthy agent |
| F-13 | Reinstatement required status == QUARANTINED
|
Auto-quarantine resets to UNCERTIFIED — the gate refused its own quarantine |
| #609 / S24 | The probe runner only checked admission | A probe could never flag behavioral decay — only a refused run could fail one |
The scariest line in the release wasn't a missed drift. It was a healthy agent quarantined because the detector compared it to a benchmark that no longer existed.
The benchmark that looked like decay
530 was the one that changed how I think about drift baselines. The detector compared failure counts across corpus versions. If the benchmark grew from 10 tasks to 20 — because the learning loop added a reviewed case — a healthy agent that passed 10/10 now "failed" 10 of 20 by the old accounting. Corpus growth looked like decay.
The agent hadn't gotten worse. The benchmark had gotten bigger. The detector couldn't tell the difference, so it quarantined a perfectly good agent.
The fix: drift is measured against the agent's own baseline on comparable corpora. You compare the same tasks across re-evaluations, not different counts. A corpus version change establishes a new baseline, not a drift signal. Growing the benchmark is growth, not regression.
tip: If your drift metric is a count, a benchmark change will look like behavior change. Compare the agent to itself on the same tasks, or any corpus edit becomes a false quarantine.
The reinstatement that refused its own quarantine
F-13 was the symmetry that broke. Drift auto-quarantine resets the certification status to UNCERTIFIED — not QUARANTINED. Reinstatement required status == QUARANTINED. So the gate that created the quarantine refused to lift it. A workload could be auto-quarantined and then never reinstated, because the precondition described a state the detector never set.
The fix was removing the stale precondition. An active quarantine record is reinstatable — the workload is re-certified and admission is re-checked. The status field is not the source of truth; the quarantine record is. The gate keyed off the wrong one.
The field test caught this in scenario S5, where reinstatement after S4's auto-quarantine failed in sequence. Two scenarios, one bug, and the lesson: the source of truth for "is this quarantined" is the quarantine record, not the certification status enum.
The probe that couldn't smell decay
609 / S24 was the most subtle. The synthetic probe runner ignored Spec.expected and only checked admission. So a probe could only fail if the run was refused — it could never flag that the agent ran and produced the wrong answer. The probe was a liveness check pretending to be a readiness check.
A probe's job is to catch decay before the drift threshold trips. If the probe only checks "did it run," it can't catch "did it run correctly." The fix: the runner starts the run, waits for terminal, and compares the output to expected. A mismatch fails the probe with a probe.decay warning. Now a probe can smell decay the drift detector hasn't measured yet.
What I learned
False-positive controls are the product, not a nicety: a single exceeding run is a warning, and quarantine requires N consecutive failures or a strong signal — a stable agent is never quarantined on noise. Measure drift against comparable baselines: the same tasks, the same corpus version; a corpus bump is a new baseline, not a regression. The quarantine record is the source of truth, not the status enum — reinstatement keys off the record, or it refuses the very quarantines it should lift. And a probe must assert the expected output, not just that the run was admitted — liveness is not readiness.
The v0.2.0 field test demonstrates the full loop — drift due/assess/probe, auto-quarantine, reinstatement after a fresh certification — as scenarios S4, S5, S8, and S24. The synthetic probe flags decay before the drift threshold in S24. Full evidence in the field-test report; the certification and learning-loop designs are in the certification v2 design and the learning loop design.
References
-
HivePlane —
pip install hiveplane==0.2.0 - Field test report (36/36)
- Certification v2 design · Learning loop design
- Operator runbook — drift & quarantine
When was the last time your detector quarantined something that was fine? If your operators have learned to reinstate without reading, your detector isn't a control — it's a checkbox.
Top comments (8)
The count-versus-rate trap is what happens whenever an evaluation harness treats benchmark size as a static constant rather than an expanding sample. In sequential testing, if the task corpus grows while the failure threshold stays fixed as an integer count, the probability of false quarantine approaches one purely as a function of sample size, regardless of agent quality.
The deeper operational failure is the erosion of the loss matrix. The moment a detector throws false alarms, operators stop pricing the risk of a real regression. Reinstatement becomes a reflexive zero-click habit because the immediate cost of a halted pipeline is tangible, while the expected damage of a silent failure remains a distant tail risk. Once operators stop reading the quarantine log, the detector has not just failed as a gate, it has subsidized the exact behavior it was designed to prevent.
In production monitoring, moving to sequential probability ratio tests or CUSUM tracking fixes the denominator drift, but the gate only holds if the quarantine threshold is explicitly calibrated against the cost asymmetry between false stops and leaked defects.
Agreed on every point. The fix moved drift off counts:
new_failuresnow counts only tasks that were in the baseline corpus and regressed (current_failed ∩ previously_passing), so growing the benchmark can't look like decay. The threshold (10% pass-rate drop, max 2 new failures, 2 consecutive evaluations) is the cost-asymmetry knob you're describing. And your point about the eroded loss matrix is exactly why false positives are the product — an ignored gate is no gate.I'd add one regression test where the corpus grows and the agent gets worse on an old task in the same release. Resetting the baseline mustn't absorb that failure. Run both agent versions on the shared tasks, then report the added tasks separately.
That's the exact case the per-task fix targets: because
new_failuresonly counts previously-passing tasks that now fail, a regression on a shared task still trips the gate even when the corpus grew — the reset can't absorb it. Reporting the added tasks separately from the shared-task delta is a good report improvement; adding it.The corpus-growth failure is a denominator bug: the detector compared a count across a baseline that kept moving. I got burned by the same shape from the other side.
I tokenized 10,000 real prompts to test the "4 characters per token" rule. Across the batch the estimate was off by 5.08%. Nothing about the denominator was wrong there - it was fixed at 10,000 - and the per-item number still lied: 40.5% of individual prompts were off by more than 20%, spread from -24% to +47%.
So I think your fix needs a second clause. Measuring against the agent's own baseline on comparable tasks kills the count-masquerading-as-rate bug. It doesn't kill rate-masquerading-as-per-item. Two fields I'd add to every alert payload: the denominator, and its previous value. "12 failures" is noise; "12 of 20, was 12 of 10" tells the operator the suite grew and the agent didn't move. Most drift alerts I've seen get the first half right and leave the operator to reconstruct the second.
That's the right payload. The assessment already carries
pass_rate_before/afterandfailed_before/after; adding the task-set size and its previous value makes the corpus-growth story explicit instead of leaving the operator to reconstruct it. "Rate masquerading as per-item" is a real second clause — thanks for the sharp example from the token-count side.Some comments may only be visible to logged-in visitors. Sign in to view all comments.