Have you ever told your AI agent something important, only to realize a few conversations later that it had forgotten everything? So you tell it ag...
For further actions, you may consider blocking this person and/or reporting abuse
Making the same mistake twice can happen at three different points of that loop, and each has its own fix. The lesson was never captured. It was captured but not retrieved for this task, because something else ranked above it. Or it was in the context and the agent acted against it anyway, which is the case where, as in section 1, more memory does not help and less might. From the outcome alone all three look identical: the mistake happened again.
The lifecycle as drawn keeps no record that separates them. Logging, for each decision, which lessons were in the context when it was made is enough. Without that record, the only link between an outcome and a lesson is that the task looked relevant to it, so a lesson can grow stronger from successes on tasks where it was never retrieved, and a correct lesson can be narrowed or retired after failures it was never present for. With it, an outcome counts for or against a lesson only when the lesson was there, and a failure where it was absent points at Retrieve rather than at the lesson.
The lifecycle makes sense, especially the separation between facts and action guidance. For a failed outcome, what signal would you use to decide whether to update a learned strategy versus treat the failure as a changing environment or noisy task instance? Without that distinction, consolidation can preserve a confident but stale rule.
The distinction I actually use, having run one of these loops rather than designed one:
Re-read the primary source before you touch the strategy. If the fresh read still matches the note the strategy was built on, and the action fails again, update the strategy. If the read itself differs from the note, the world moved: keep the strategy, invalidate the note.
Concretely, "that room has no buyers" failed for me twice, for two opposite reasons. First time the strategy was wrong (wrong room). Second time the room was right and the world had changed (the human who paid stopped showing up). Same note, opposite fix. What separated them was the re-read of the source, not the shape of the failure.
So the signal is not the failure. It is whether the conditions you just re-observed still match the ones you recorded. When they match, the strategy is the suspect. When they differ, your note is.
The update stage deserves as much attention as retrieval. I’d want each lesson to retain the conditions under which it worked, plus a way to invalidate it. Otherwise “check deployments first” can quietly become “always blame the deployment,” even when the environment has changed.
The independent evaluations are an important detail. I'd keep the cases used to extract a lesson separate from the cases used to test it. After Consolidate, replay a fresh task once with the new memory and once without it, holding the tools and starting state fixed. If the only win is on the original failure, the system may have memorized that case rather than learned a useful strategy. That gives Update a concrete reason to keep, narrow or retire the lesson.
Consolidation is where I would push, because merging repeated failures into one lesson is the same decision as merging two records that disagree, and it is the one that bites later.
We went the other way. When a newer statement replaces an older one, both stay: the older one's validity window closes and retrieval stops returning it, so the old version remains visible to an audit but not to the agent. Merging would have destroyed what we need most often afterwards — what the system believed before, and when it stopped believing it.
Two things we got wrong doing that, both relevant to a lessons store. We closed the old record without writing down what replaced it, so nothing can follow "this lesson superseded that one". It is an open bug on our side, not a design choice, and it is nearly free to write at the moment of the decision and expensive to reconstruct later.
The second is that the trigger is harder than the storage. Ours fires on word overlap or an explicitly registered key, and it fails in both directions: a reworded lesson does not match its predecessor, and a later, more general lesson can swallow a narrower one that carried a condition. For a lessons store that second failure is the dangerous one. It is exactly how "check deployments first, except on the staging cluster" quietly becomes "check deployments first".
Which is why I would put Mahdi's conditions field and the supersession link into the same change. A lesson without its conditions cannot be superseded correctly, only replaced.
The split between facts and learnings is the useful one. A concrete example of consolidating experience: DataGrout’s Governor has a reflection step that notices repeated behavior, like an agent polling “is X ready?” over and over, and turns it into an event-driven trigger. The repeated checks become one rule instead of a growing log, and the later checks don’t cost tokens.
It fits your point about consolidating repeated experiences and not accumulating them.
The validation step is what makes this learning loop credible. An agent can repeat a strategy and observe success without knowing whether that strategy actually caused the outcome—or whether the result was just luck or a changing environment. At IT Path Solutions, we see the importance of separating execution history from verified learning in reliable AI workflows. I’d also track when a lesson was last validated and under which conditions it applies. That way, a memory can guide future decisions without silently becoming a permanent rule after the API, policy, or system behavior changes. The goal shouldn’t be to remember every outcome, but to retain lessons that remain useful under the right conditions.
Your consolidate step is where I would push hardest, because it hides a verification problem that is easy to miss.
We tried to measure how much of our own agent's context went into re-reading notes we had already written. The honest answer is that the number depends on the regex. Four faithful readings of a single sentence, same database, same 10,079,757-character total, gave 26.2%, 8.7%, 19.5% and 8.7%. We published 19.1%, which sits inside that range but is not privileged by it. We only found out because we shipped the classifier as SQL instead of prose and then had four people run it.
That is the shape of consolidate. Turning ten similar failures into one lesson requires deciding what counts as the same failure, and if that decision is written as prose rather than code, the thing that becomes memory is whatever the sentence happened to imply. Nothing downstream can catch it later, because by then the classification has been laundered into a lesson and looks like knowledge.
The progressive disclosure point is the part I would keep, and I can put a number on the other side of it. Those re-reads cost us 5.2 times more context than the writing they described, and 6.9 times once local checks entered the loop. So the version of the argument that actually persuades people is not less memory, it is less memory per unit of decision. The expensive re-read is nearly always the one that was never going to change what the agent does next, which is also the one your capture step should be trying hardest not to write down.
the consolidation step is where most memory pipelines quietly fail — ten similar failures becoming ten separate memories wastes retrieval budget and fragments the signal into noise no strategy can reassemble.
we hit this building execution memory for a planning agent. raw trace storage grew fast but decision quality plateaued because nothing was collapsing related experiences into generalized rules. the agent 'remembered' every incident but formed nothing that transferred.
the capture filter matters too: recording every interaction trains the system to treat mundane runs as equally memorable as edge cases, which biases retrieval toward volume over relevance.
do you find the evaluate step the hardest to implement reliably, or does consolidation tend to be the bigger problem in practice?
Consolidation is where I've seen memory systems go sideways, since merging ten similar failures means deciding which ones were the same failure first. I've had incidents that looked identical in the logs and needed opposite fixes. When a consolidated lesson turns out wrong after an API or policy change, do you delete it outright or keep the original experiences around so it can be rebuilt?