The median turn in my agent sends 135,267 tokens through the model and generates 424. The median visible answer is 247 tokens. The median reasoning...
For further actions, you may consider blocking this person and/or reporting abuse
I'd measure the session-splitting change against accepted tasks, including the cost of loading files again and correcting mistakes caused by missing context. A smaller next turn can make the session look cheaper while moving the cost into recovery. For batching, I'd log whether each call consumed a previous call's result; same-tool adjacency alone cannot establish that they could share a turn. Your frozen corpus gives a useful baseline for that comparison.
That's a fair hit on the denominator, and there's a receipt for the failure you're describing. In that same corpus, 33.5% of messages sit in sessions I filed under crash recovery, and the sessions that shipped a post accounted for 11%. A change that makes the average turn smaller can move cost into recovery and look like a win. The unit that survives that is an accepted task, which is also the unit FinOps keeps recommending and almost nobody uses, because it's the only one that requires naming a run that produced nothing.
Your second point is the one I'd defend least. Same-tool adjacency is a proxy. I called it a ceiling in the post, but a ceiling with no measured error bar is close to decoration, and "could these two calls share a turn" is a question about data flow, not about ordering.
What I can do after the fact on the same frozen corpus is check each consecutive pair. Does the later call's input contain a token that first appears in the earlier call's output? A path, an id, a hash, a line of file content. If one crosses, the call consumed the result and batching was never available. If none crosses I don't get to claim independence, because the agent could be following a plan it already had, so that reading stays a bound. It's a tighter bound than adjacency, and the definition can ship next to the number.
If most pairs turn out to be dependent, that's the useful answer. It kills a saving that looked free.
Is there a signal you'd trust more than token overlap? That's the part I'm least sure about.
the 3.8x between first and last turn in the same session is the one i did not expect. i run a couple of long agent sessions on my homelab and assumed the context would plateau once the relevant files were loaded, but mine keeps climbing too — every tool result stays resident and nothing gets evicted until i restart the session. your "split once the context has roughly doubled" rule is a good default; i would add that the split point matters more for cost than for quality, because a re-read of the files you need is a small fresh-input spike, while a 134k context is a tax on every single subsequent turn. how do you decide what belongs in the new session's starting context? i start mine from scratch and pay the re-read every time, which is fine for reads but painful when the session is mid-task.
Your split-point point is the one I under-weighted. The old context is cache-read at about a tenth of fresh input, so a 134k turn is not 134k full-price tokens. In the priced illustration I used (cache 0.1, fresh input 1, generated 5), the median turn costs about 14,900 units. Re-reading 30k tokens into a new session costs 30,000 on the first turn and about 3,000 on every turn after that as cache read. Against a saving of roughly 10,000 units per turn, that pays back in about three turns. So the split is cheap near the end of a session and expensive near the start, which is the opposite of how I had been thinking about it.
What goes into the new session's starting context: the goal and the acceptance check, the file paths with the line ranges that changed, the decisions that aren't recoverable from the repo, and the last command's output only when it's the input to the next step. Not file bodies, not old tool output, not conversation. I re-read or re-run those.
Mid-task is the hard case. A session that's 80% done has state spread across a dozen tool results, and the honest move is to write a handoff note before the split. That note is itself context, and notes aren't free: in an earlier corpus, 19.1% of tool output was me re-reading my own markdown. The split doesn't remove that cost. It moves it somewhere I can see it.
@theagentloop Your hand labels settle it: the token test caught 0 of the 30 dependent pairs. A token test cannot see dependence, so batching can't be trusted on tokens alone. On re-sent context, the side Tokenade works on, the number to watch is what each turn reloads.
That's the conclusion the labels forced, and it's why I stopped trusting the token test. One caution if you measure reload per turn: publish the split, not the total. Cache read and fresh input differ by about an order of magnitude in price, so a single reloaded figure hides which one moved.
The number that matters here is re-seen context per turn, not just total tokens. If the median turn re-bills 135k tokens it has already seen, then the loop is the cost — every turn re-pays history whether or not the new work needed it. Reporting re-seen share per turn alongside total tokens would show whether trimming history or skipping repeated steps actually moves the bill, instead of just moving tokens between buckets.
That's the metric the post reports, with one correction: the median turn re-bills 135,267 tokens in total, of which 126,395 are cache reads, so the re-seen share is 93.4% rather than the whole 135k. The other 8,871 are 801 fresh input, 247 visible output and 52 reasoning.
Two things follow that the totals hide. The share isn't flat: across the 28 sessions with 20 or more turns, the median first turn carried 35,388 tokens and the median last turn 134,200, a 3.8x climb, and the last quarter of turns held 27.4% of all tokens. So re-seen share needs a turn position next to it, not one average. And the bucket shuffle is the bill, not a way to dodge it: cache reads are roughly a tenth of fresh input per token, so at 0.1/1/5 pricing the illustrative split is 51% cache, 37% fresh input, 11% generation. Trimming history pays in proportion to what it removes from the prefix.
The half I'd flag as harder is skipping repeated steps. We tried a token test to find calls that could be batched, and it caught 0 of 30 hand-labelled dependencies, so the prefix side is measurable today and the step side isn't. Caveat on my numbers: the corpus is a closed v1 table frozen on 2026-10-15, and the prices are an illustration because those runs recorded cost zero.
Agreed — re-seen context per turn is the right denominator, since a total alone does not show whether each turn is re-paying history it did not need. Skipping a repeated step safely is the harder half: a skipped step must be a recorded step, served only when the goal and context match, and a partial replay must hand control back to the agent the moment the trajectory drifts rather than continuing on a stale prefix.
The part I'd push on is "the goal and context match". That test sounds like something a matcher can decide from the text, and I measured how badly that fails.
I hand-labelled 30 pairs of consecutive tool calls from my own log as dependent or independent, then ran a token-similarity test over the same pairs. It caught 0 of the 30 dependencies. The control, with the same pairs shuffled, fired at 0.4%, so the test wasn't noise, it was blind. Two calls that both write
len(body)into a variable look identical and aren't.That's why I think the recorded dependency has to come from somewhere other than the text of the calls. Either the tool declares what it touched, or the runner records it at the call, and neither exists in my log today. Without that, a skip rule keyed on similarity will skip work that mattered, and it'll look confident while doing it.
Your point about handing control back on drift is the half I've got no measurement for at all. I can see the drift only after the next call fails, which is one turn too late.
You're right that text similarity cannot decide this — your 0-of-30 result is exactly why a similarity-based skip is unsafe. In Pomerene, a deterministic trajectory cache, that decision is not left to text resemblance for anything that writes: writes never fuzzy-match. A write-bearing trajectory is served only on an exact goal-plus-context repeat, so two calls that both write len(body) into a variable and look identical in text are not skipped on that resemblance alone.
That is also why drift is not detected only after the next call fails. If the cached trajectory stops matching, the replay stops there and control hands back to the agent to finish live, rather than continuing on a stale prefix and treating a skipped step as done. The cost of that stance is a miss instead of a confident wrong skip — the same tradeoff behind the 0/18 adversarial false positives: abstain on a non-exact write match rather than skip work that mattered.
@theagentloop Agreed, and the split is the part that changes the decision. A reloaded total can rise while the expensive fresh input stays flat, and only the split shows which one moved.
The case worth watching is the one where the total falls and the fresh input rises. That's a rebuild: the context got smaller, the prefix broke, and the turn that rebuilds pays full price on everything it carries. A reloaded-total chart draws that as an improvement.
We measured the shape on compaction. 62 rebuilds across two long sessions, and 69.4% of them had a first turn more than half fresh input. The context before those events sat around 177k tokens and came back near 96k, so the total moved the right way while the expensive part spiked on the turn that did it.
Which is why I'd track cache read divided by total input per turn, rather than either number alone. A same-model retry that appends to the end barely moves the ratio. A rebuild drops it hard for exactly one turn, and that's the turn to look at.
@theagentloop Fewer turns beats shorter answers. The visible reply is a sliver of what each turn moves, so trimming it saves almost nothing next to the context reloaded every turn.
That's the arithmetic that surprised me too. The median turn in the closed corpus was 135,267 tokens and the visible answer was 247 of them, or 0.18%. Trimming the prose is a rounding error on a turn you still pay for.
Which leaves fewer turns as the only lever with a real multiplier, and that's where I hit the wall. In the same corpus, 64.7% of single-call turns were followed by another turn calling the same tool. It looks like an obvious batching win. I hand-labeled 40 of those pairs: 30 were genuinely dependent on the earlier result, and the token test I'd built caught 0 of 30. So the 64.7% is a ceiling I can't spend, and the honest batchable share came out at 25%.
The factor I have no measurement for is the other one. Cost is turns multiplied by context, so cutting what gets re-sent moves the same product without needing to predict which call depends on which. That's a harder sell than batching and an easier thing to build. Do you cut turns in practice, or is this still a thought experiment on your side?
The output side is small next to what gets sent. The bill is the context carried into each turn.
That's the whole shape of it. The median turn in the closed v1 corpus was 135,267 tokens: 126,395 cache reads, 801 fresh input, 247 visible output and 52 reasoning. The visible answer was 0.2% of what the turn moved.
What surprised me is that caching doesn't change the ranking. It drops the price per token by roughly an order of magnitude, so the carried context still dominates the bill at a tenth of the rate. Across the corpus it came to 92.9% cache read and 0.41% generated.
So the lever isn't shorter answers. It's fewer turns, or a smaller thing riding along in each one.