How we built a realtime system for responsive voice AI in six months
Quick Answer
OpenAI's GPT-Live voice AI system eliminates turn detectors, enabling full-duplex communication and real-time responsiveness.
Quick Take
By optimizing architecture for low latency, it achieves sub-second responsiveness, enhancing user interaction without interruptions.
Key Points
- GPT-Live allows simultaneous listening and speaking, enhancing conversational flow.
- The new architecture streams audio directly, minimizing latency and interruptions.
- Stateful inference maintains context across long voice sessions, improving user experience.
- WebRTC ensures low-latency media transport, adapting to network conditions.
- The transition from Python to Go improved frame delivery performance significantly.
DeepSignal Analysis
What happened
OpenAI developed the GPT-Live voice AI system, which eliminates the need for turn detectors, allowing for full-duplex communication. This system achieves sub-second responsiveness by optimizing its architecture for low latency, enabling real-time interactions without interruptions. The architecture separates media flow from application logic, enhancing customization without affecting responsiveness.
Key evidence
- GPT-Live removes turn detectors from the audio path, allowing simultaneous listening and speaking, which enhances the natural flow of conversation.
- The system architecture was optimized for low latency, achieving sub-second responsiveness, which is crucial for effective human-like interactions.
- The media frontend and inference logic were rewritten in Go, improving frame delivery smoothness, with the new system's p95 matching the previous system's p50.
Why it matters
The advancements in GPT-Live represent a significant shift in voice AI technology, moving from turn-based interactions to a more fluid, real-time communication model. This change not only improves user experience but also opens up new possibilities for applications that require immediate feedback and interaction. The ability to customize application behavior without compromising responsiveness is particularly valuable for developers looking to create more engaging voice-driven applications.
Source Excerpt
GPT-Live enables continuous voice interaction with AI, using a turnless speech model and low-latency architecture for faster, more natural conversations.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from OpenAI Blog
See more →Scientific computing in the age of agentic AI
AI agents are transforming scientific computing by streamlining software development, enabling researchers to focus on discovery. Projects using Codex and Claude Code report accelerated development and improved maintenance, though challenges in validating AI outputs remain. Long-term stewardship of research software is crucial to ensure reliability and reproducibility.