Here's a line from YouTube's auto-captions on a lecture about attention in neural networks:
consider heart attention looking at the architecture ...
For further actions, you may consider blocking this person and/or reporting abuse
Real-word errors outnumbering the misspellings is the part that maps onto ticker tagging. Our check that an LLM-suggested ticker really belongs to an article accepted any word of the company's name longer than two letters. That's how Voyager Therapeutics ended up on a story about Viking Therapeutics, matched on "therapeutics". T1 Energy got onto a TotalEnergies story the same way. The fix asks for a word that isn't in the dictionary, and that check already existed in our code but only ran on crypto names. Since your term list leans toward AI names, have the earnings calls shown you company names that look like ordinary words?
Yes, the earnings calls are actually full of them, and in both directions too.
Some company names are ordinary words (Spire, Mosaic, Travelers, Yeti), so a dictionary check passes them without question. More often, the ASR (which is from 2021 so it's admittedly outdated) turns an unusual name into an ordinary word: Culp became "cult" and "coal", Spire became "fire", and Danaher's Cytiva and Beckman came out as "sativa" and "Batman".
Those are the real-word errors a misspelling check can't catch, and that's why the call's company name goes in as a Priming term for added context. Your fix, requiring a match on a word that isn't in the dictionary is kinda like the same lesson from the other side. Only a distinctive word in a name is evidence.
Cytiva and Beckman coming out as sativa and Batman made my morning. Priming with the call's own company name is a neat, cheap fix.
Nice breakdown of the two error classes. For the real-word kind, one cheap layer between spellcheck and the LLM: run a domain lexicon with phonetic/Levenshtein matching over a sliding window (tanh vs tange, hard vs heart) so the model only adjudicates flagged spans instead of re-reading every line - that's what keeps the $5/month budget honest as uploads grow. And your accept/reject UI is quietly collecting labeled data: after a few hundred decisions you can fit a small classifier on (context, caption word, suggested fix) triples and use it to pre-rank the LLM's suggestions.
Thanks! There actually is a cheap phonetic layer: one detector matches against a domain lexicon, and another catches a term spelled differently elsewhere in the same transcript. It deliberately skips common English words, though, because "heart" vs "hard" is a perfectly good word on both sides.
Unfortunately checking every ordinary word against the lexicon buried the real errors in false alarms. I actually started with the design you described, where the LLM only judged flagged spans. Letting it read everything, with the local flags as hints, it found several times more real-word errors at higher precision, for about 5–6¢ per hour of audio. So for this kind of error, reading everything turned out to be what makes it worth the money.
On the labeled data I do think that it's a good idea but right now uploads and decisions are deleted after 24 hours. So learning from reviews is something I'd revisit once people are actually using it, and doing it right means changing that retention promise transparently.
The false-alarm floor from lexicon matching is a known trap - once ordinary words start firing, nobody trusts the flags and the real errors get missed in the noise. The 5-6c/hour number is what makes full-context judging viable at all; if that cost holds on longer transcripts it changes the economics of caption QA.
The full-context-beats-flags result at 5-6c/hour is worth publishing as an ablation, because that cost figure is what lets others find the threshold where reading everything starts to pay. On retention: a 24h delete window is precisely what makes the labeled-data idea viable later - (context, word, fix) triples can be aggregated into identity-free counts, so revisiting it would not have to change the privacy promise.
Splitting the error cause into
misheardversusgrammarorstyleis the constraint that keeps transcript pipelines from turning into uninvited copy editors. When I ran LLM cleanup passes over Whisper transcripts without a cause tag, the model quietly rewrote spoken hesitation, dropped filler phrases, and drifted the word counts enough that timestamp alignment broke downstream. On the 400-word chunk boundary, overlapping the last two sentences as read-only context helps with real-word homophones that depend on a term introduced right before the cut. A chunk that starts cold on "heart attention" has to guess until the next mention, whereas carrying the prior sentence's "soft attention" across the boundary resolves the contrast on line one without adding to the output token budget.Yup splitting the error is definitely what limits the LLM's "corrections" from being it's own idealised paraphrasing!
The drift problem is also handled structurally: the model never returns text, only word-index spans plus a replacement. Each fix is spliced into its caption at the exact character offset, and nothing outside it changes, so a chatty model can't shift word counts or timings.
Regarding boundaries each chunk does carry about 40 words of read-only context on each side. I use a word count instead of sentences because YouTube auto-captions sometimes have no punctuation to find sentences by (the example video's transcript for example has none). Chunks are meant to close at a sentence end after ~400 words, but on unpunctuated captions they run to an 800-word cap.
Fun fact: the first mention of "heart attention" is about 100 words into its chunk, with "soft attention" in the same chunk, so the model had the contrast in view. Your cold-start case is real, though, and I haven't measured how often it happens.