Yesterday, we released new Gemini Live models in the Gemini API and Google AI Studio, expanding our developer suite for building real-time, voice-first product experiences:
Gemini 3.8 Live and 3.8 Live Extended Thinking: Gemini 3.8 Live brings a step change to our native speech-to-speech models, capable of performing tasks while maintaining dialogue. For complex requests, 3.8 Live Extended Thinking delivers deeper reasoning, ranking #1 on Artificial Analysis’ Speech-to-Speech leaderboard.
Gemini 3.5 Transcribe: Our dedicated speech-to-text model brings highly precise transcription across 85+ languages. Released last month, it achieved an average Word Error Rate (WER) of 4.0% (streaming) and 2.6% (non-streaming).
Gemini 3.8 Live & 3.8 Live Extended Thinking: Build more intelligent conversational agents
Our new models, Gemini 3.8 Live and 3.8 Live Extended Thinking enable developers to build voice agents that can reason and execute tasks while maintaining the flow of conversations. Key capabilities include:
- Asynchronous function calling: Execute API and tool calls in the background while continuing to stream audio responses to the user
- Visual context: Ground dialogue in live visual inputs to help enable agents that can understand what users say and see
- Alphanumeric precision: Accurately parse confirmation codes, claim numbers, and technical data
- Multilingual support: Reach global audiences with coverage for 97+ languages and accent consistency
- Incremental content updates: Seamlessly merge real-time audio with structured data to return context-aware responses
3.8 Live Extended Thinking also supports configurable thinking to help handle complex, multi-step reasoning in the background, while responding or narrating its progress in the main conversation. These models represent a step-change from our previous live models and provide a more streamlined alternative to cascaded architectures.
Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking are available via the Live API. Competitively priced at $0.005/min for audio input and $0.018/min* for audio output, they allow developers to scale voice applications with industry-leading performance.
Developers can also access the models through Agora, Fishjam, LangChain, LiveKit, Pipecat, Vercel, and Vision Agents, our Live API integration partners that handle media streaming infrastructure for real-world deployment.
Gemini 3.5 Transcribe: Convert streamed speech to text
Real-time speech understanding is critical for voice-first interfaces. Last month, we released Gemini 3.5 Transcribe for low-latency transcription with high precision, achieving a 4.0% WER, and useful features:
- Automatic code-switching: Handle intra-sentence and inter-sentential code- and language-switching without manual configuration
-
Custom vocabulary biasing: Steer speech recognition toward domain-specific terms, uncommon jargon, company names, and proper nouns by passing a
custom_vocabularylist of up to 1,000 terms - Smart transcription mode: Deliver polished, reader-ready transcripts with structured formatting, self-corrections, and disfluency removal that eliminates filler words
3.5 Transcribe supports 85+ languages and provides a strong listening engine for voice experiences and stateless tasks like sub-second captioning, call center agents, and real-time audio analytics. You can also access the model via the Interactions API to transcribe audio files up to 1 hour long with structured timestamps and speaker labeling. Read our developer guide to learn more.
Our complete audio suite for developers
To get started, try out the models in ai.studio/live, clone example apps from GitHub, or equip your agent with our live api skill.
You can also create audio experiences with our speech and music generation models, all available in the Gemini API:
- Gemini 3.5 Live Translate: Speech-to-speech translation across more than 70 languages
- Gemini 3.1 Flash TTS: Highly configurable speech generation (with more updates coming soon)
- Lyria 3.5: Production-grade music generation
The mic is yours, and we can’t wait to hear what you build!
Top comments (13)
Love the async. live API 🥰
Questions:
The asynchronous tool-call detail is the part I’d pressure-test first. Continuing the audio stream while a tool runs is great UX, but it makes delivery semantics explicit: what should the user hear if the tool times out, returns partial data, or completes after the conversation has moved on? I’d model each call with an idempotency key and a cancellable state machine, then surface queued, running, committed, and failed states instead of letting the voice layer imply success. The 85+ language coverage and custom vocabulary are useful, but production quality will depend just as much on those boundaries as on WER.
The asynchronous function calling is the part that stands out to me. I run a voice interface on a low-power RISC-V board (MaixCAM), where replies have to be short enough to hold an 8-year-old's attention — and the bottleneck was never the model, it was the pipeline: transcription, reasoning, TTS synthesis, transfer, playback. Every hop added latency, and kids notice even ~2 seconds of silence. Running tool calls in the background while speech keeps streaming attacks exactly the right part of that problem. One thing I'd love to see documented more: how these models handle turn-taking with real-world noise — my percept stream tracks pause counts and volume trends, because barge-in and half-finished sentences are the norm with a child, not the exception.
The async function calling while streaming audio is the part that actually matters — every voice agent I've built deadlocks the moment you need to call an API mid-conversation. But that 4.0% streaming WER across 85+ languages is suspicious. I've tested transcription on mixed-language calls (Spanish-English code-switching) and the WER jumps to 15-20% easy. Would love to see the breakdown per language instead of an average that smooths over the hard cases.
Real-time bidirectional voice applications introduce a fascinating latency budget constraint for persistent context.
In standard conversational chatbots, a 1-2 second retrieval latency for user history or external context is acceptable. In live voice interactions, however, human conversational turn-taking demands a sub-500ms round-trip (speech-to-text -> memory retrieval -> LLM first token -> text-to-speech) to avoid jarring pauses or speech overlap.
If session state or caller memory takes >200ms to fetch, it breaks the illusion of natural conversation. Keeping state retrieval deterministic and strictly under 50ms is crucial when orchestrating memory alongside streaming audio feeds. Exciting to see the Live API expanding!
Really interesting approach to building real-time voice applications with Gemini Live. The combination of low-latency interaction and transcription opens up some exciting possibilities beyond traditional voice assistants.
I especially like the idea of using this for creative workflows, where users can speak naturally, capture their thoughts, and turn them into structured content without interrupting their flow. Curious to see how developers handle latency, context management, and long conversations in production.
Great write-up and useful practical examples!
Sneak peek: the Live API's 100 ms latency budget is a beast to hit—watch out for the VAD warm‑up causing cold starts that can OOM your container. Have you tamed that latency wall yet?
Some comments may only be visible to logged-in visitors. Sign in to view all comments.