TL;DR: Claude Opus 5.5 came out yesterday with cache reads at $0.20 per million tokens, 60% cheaper than Opus 5. I redid the cost math for my text-to-SQL CRM assistant. For one design choice, the cost gap between "retrieve only the relevant schema" and "send the whole schema every time, cached" shrank from 2.1x to 1.2x. At that point the choice stops being about cost and becomes about accuracy. The migration guide also has two breaking changes that hit text-to-SQL setups directly.
Some context: how Aria picks the schema to send
A few months ago I wrote about Aria, an AI assistant that lets CRM agents ask questions in plain English and get answers from live SQL.
The core trick is RAG over the schema, not over the data. A Python pipeline writes a plain-English description of every table, column and enum value in the CRM (~90 docs in total). When an agent asks a question, I embed the question, run a pgvector search over those docs, and send only the top 5 to the model along with the question.
Here's why I built it that way:
- Cost. I didn't want to pay for the whole semantic layer on every request.
- Focus. I reasoned that a model shown 5 relevant tables writes better SQL than one shown all 15.
Opus 5.5's pricing mostly takes away reason #1. That leaves reason #2, which I never actually measured.
The pricing change that matters here
| Opus 5 | Opus 5.5 | |
|---|---|---|
| Input | $5 / MTok | $4 / MTok |
| Output | $25 / MTok | $20 / MTok |
| 5-min cache write | $6.25 / MTok | $5 / MTok |
| Cache read | $0.50 / MTok | $0.20 / MTok |
Most of the headlines are about the 20% cut on input and output. For a schema-heavy workload, though, cache reads matter most. They used to cost 0.1x the input price and now cost 0.05x.
The napkin math
These are my assumptions. Plug in your own numbers:
- 800 questions/day (40 agents × ~20 questions, which is roughly the load I described in my last post)
- Full semantic layer: ~20k tokens (~90 docs × ~220 tokens)
- Top-5 retrieval: ~1.1k tokens per question
- ~10 cache writes/day. The default cache lives 5 minutes, and every hit refreshes it for free. 800 questions over a 9-hour workday is about 1.5 per minute, so the cache stays warm most of the day and only goes cold at the start of the day or after lunch.
I'm only counting the schema part of the prompt, because that's the only part that differs between the two designs.
Option A: retrieve top 5, no caching (the retrieved docs change on every question, so there's no stable prefix to cache)
Option B: send the full semantic layer on every request as a cached prefix
| Per day | Option A: retrieve | Option B: full + cache | B vs A |
|---|---|---|---|
| Opus 5 | $4.40 | $9.25 | 2.1x |
| Opus 5.5 | $3.52 | $4.20 | 1.2x |
Per question on Opus 5.5, that's $0.0044 vs $0.0053. Over a 22-working-day month, it's about $77 vs $92.
On Opus 5, "just give the model everything" cost twice as much, which made the pgvector step easy to justify. On Opus 5.5, it costs about $15 a month more across the whole team.
At that price, cost no longer decides the question. Accuracy does.
What Option B would get me
- No embedding call or vector search before each question. That's one less network hop and one less thing that can fail.
-
No retrieval misses. With retrieval, if the question says "stale leads" and the embedding doesn't pull up the doc that explains
last_activity_at, the SQL is wrong before the model even starts. With the full layer, the model can always see every table. - A static prompt prefix, which matters for the next section.
What it wouldn't replace: my intent examples, the question→SQL pairs that get promoted from thumbs-up feedback. That set grows over time, so it still belongs in a retrieval step. The likely end state is a hybrid: a static cached schema plus retrieved examples.
Two breaking changes that hit text-to-SQL setups
I read the Opus 5.5 migration guide with Aria in mind. Two changes stood out.
1. Forced tool use now returns a 400
tool_choice of type tool or any is rejected. Many text-to-SQL setups force the model to call their run_sql tool so it can't answer from memory. On Opus 5.5, the fix is tool_choice: auto, marking the tool as strict, and saying clearly in the prompt when the tool must be used.
My SQL validator doesn't change. That step never trusted the model anyway: SELECT-only checks, agent-ID injection from the JWT, and a read-only Postgres role. Moving to a better model shouldn't move your trust boundary.
2. Conversations should be append-only
Thinking is always on in Opus 5.5 (you control it with effort instead of turning it off). The guide says to keep conversations append-only, with no edits to system, tools or earlier messages mid-conversation. For newer accounts, replaying a thinking block after such an edit returns a 400 by default.
That's a problem if I move my current design over to Opus 5.5 unchanged. I rebuild the system prompt on every turn with a new set of retrieved schema docs. That's an edit to system in the middle of a conversation.
So there are two ways to comply:
- move the retrieved docs out of
systemand into each new user turn, or - make
systemstatic, which is Option B again.
Both the pricing and the API now push in the same direction: put the large, rarely changing context in a static prefix, cache it, and add new context only by appending.
Here's roughly what the request would look like:
{
"model": "claude-opus-5-5",
"max_tokens": 1024,
"output_config": { "effort": "low" },
"system": [
{ "type": "text", "text": "You write read-only PostgreSQL for a student-housing CRM. Always answer by calling query_crm_database." },
{ "type": "text", "text": "<full semantic layer: every table, column, enum>",
"cache_control": { "type": "ephemeral" } }
],
"tools": [{ "name": "query_crm_database", "strict": true, "...": "..." }],
"tool_choice": { "type": "auto" },
"messages": [
{ "role": "user", "content": "<retrieved intent examples>\n\nWhich of my leads haven't been contacted in 3 days?" }
]
}
(I'd use effort: low for the formatting pass and try medium, the new default, for SQL generation.)
What I'm going to measure
I haven't switched Aria over yet. This is napkin math and a close read of the docs, not a benchmark. The plan:
- Take the 30 question→SQL examples my schema pipeline already generates and use them as an eval set.
- Run both options: top-5 retrieval vs full cached layer.
- Compare how often the generated SQL returns the same rows as the reference SQL, plus latency and actual cost from the
usagefields (cache_read_input_tokenstells you whether caching actually happened).
If the full-context version is at least as accurate, I'm removing the pgvector step from the question path. I'll post the numbers when I have them.
Question for you: if you're doing text-to-SQL, do you retrieve schema per question or send all of it? And has cheaper caching made you rethink the pieces around RAG? I'd love to hear in the comments.
Top comments (4)
The join-table retrieval miss is what killed dynamic schema retrieval for us long before cache pricing dropped. A vector search on 'stale leads' pulls the leads table and maybe activities, but it almost always drops the join table or audit enum that actually links the two. The model gets a hallucinated join condition because the glue schema was ranked 7th instead of top 5.
At 20k tokens, putting the full schema into a static cached prefix completely sidesteps that failure mode. The only thing I would watch in production with the 5-minute ephemeral cache is latency jitter after brief idle gaps. When an agent pauses for six minutes between calls, that first turn takes the full cache-write recomputation latency before it warms up again. Keeping the examples in the append-only user turn and the DDL pinned in the system block is the cleanest separation for that.
Thanks Reid, the join-table miss is a great example and it's exactly the failure I was worried about. Top-5 retrieval ranks each doc on its own, but a query needs a connected set of tables. The linking table rarely looks similar to the question, so it's the first thing to fall out of the top 5.
Good point on the idle gap. One detail helps here: if the cached block holds only the shared schema and the agent-specific context comes after it, all 40 agents hit the same cache entry. Then the cache only goes cold when the whole team is quiet for 5+ minutes, not whenever one agent pauses.
For those gaps I see two options: pay for the 1-hour cache ($8/MTok to write, so ~$0.16 per write for 20k tokens), or send a small keep-warm request every ~4 minutes during work hours (roughly half a dollar a day at $0.20/MTok cache reads). I'll measure first-request latency after an idle gap as part of the eval.
The reframing here is the valuable part: cheaper cache reads don't just shave a bill, they dissolve the original justification for retrieval and force you to defend it on accuracy alone â which, as you note, is reason #2 that you "never actually measured." I'd push on that measurement before migrating, because the two options aren't symmetric on failure. Top-5 retrieval fails loudly and early (wrong table retrieved â obviously wrong SQL, easy to catch in eval). Whole-schema-cached fails quietly: the model has all 15 tables and picks a plausible-but-wrong join, and those errors are much harder to detect because the SQL runs. So even at 1.2x I'd want an eval set that specifically probes near-miss table selection before trusting the fat prompt. One caution on the cache napkin math: the 5-min TTL and "1.5 questions/min keeps it warm" assumption is fragile at the start of day and post-lunch cold windows you mention â those cold reads pay the full write price, so the effective multiplier is worse than steady-state. Have you logged your actual cache hit rate vs the assumed one? That single number decides whether the 1.2x is real.
Thanks Max, the asymmetry point is a good one. A missing table breaks the SQL in an obvious way. A wrong join across tables that are all present runs fine and returns believable numbers.
That's part of why the eval compares the rows each query returns against the reference SQL, not just whether it runs. A plausible-but-wrong join usually returns different rows, so it should show up there. But you're right that my 30 examples come from the schema pipeline and probably don't include enough near-miss cases. I'll add questions where two tables look similar or two join paths both seem reasonable.
On the cache hit rate: no, I haven't logged it yet. Aria isn't on Opus 5.5, so ~10 writes/day is an assumption. The usage fields make it easy to measure (cache_read_input_tokens vs cache_creation_input_tokens), so I'll log it during the eval. To show how sensitive the number is: each extra cold start costs about $0.10 at 20k tokens, so even 20 writes/day puts Option B around $5.20/day, about 1.5x instead of 1.2x. It also helps that the cached prefix is shared across all 40 agents, so it only goes cold when the whole team is idle.