Why WebRTC Is a Poor Fit for AI Voice: The Fatal Flaw of Latency-First Design

The Controversy Over OpenAI's Choice of WebRTC

OpenAI recently published a technical article on how to deliver low-latency voice AI at scale, choosing WebRTC as the transport protocol for its real-time voice API. However, Luke Curley, a developer of the Media over QUIC (MoQ) protocol, offered sharp criticism—arguing that WebRTC's core design philosophy fundamentally conflicts with the requirements of AI voice interaction.

This isn't a simple matter of technical preference. It involves a deep-seated contradiction between a protocol's underlying architecture and its application scenario.

WebRTC's Design Philosophy: Built for Video Conferencing

WebRTC (Web Real-Time Communication) is a set of real-time communication standards open-sourced by Google in 2011 and later standardized by the W3C and IETF. Its underlying transport uses the UDP protocol with SRTP (Secure Real-time Transport Protocol) for media transmission, and handles NAT traversal through the ICE (Interactive Connectivity Establishment) framework. WebRTC's congestion control algorithms (such as Google Congestion Control, GCC) dynamically adjust bitrate based on network conditions, instructing the encoder to reduce quality or outright drop packets when necessary—rather than waiting for retransmission like TCP.

WebRTC was originally designed to serve real-time video conferencing scenarios. In video calls, fast back-and-forth conversation is critical, and any pause waiting for audio data is unacceptable. Accordingly, WebRTC adopts an aggressive strategy: proactively dropping audio packets to maintain low latency when network conditions deteriorate.

To understand why WebRTC drops packets, we need to revisit the fundamental differences at the transport layer. TCP guarantees every packet arrives through acknowledgment mechanisms and retransmission, but at the cost of introducing unpredictable latency—a single lost packet can block all subsequent packets (known as Head-of-Line Blocking). UDP completely abandons reliability guarantees; once a packet is sent, there's no concern about whether it arrives. WebRTC chose UDP as its underlying transport precisely because in video conferencing, a stale video frame or outdated audio segment has no playback value—retransmitting it would only worsen latency.

If you've ever heard distorted audio in a video conference, that's WebRTC at work—it would rather let you hear broken sound than make you wait for complete data packets to arrive.

For real-time conversations between humans, this trade-off makes perfect sense. The human brain excels at using context to fill in missing information; even if you miss a few words, you can still understand what the other person means. But AI doesn't possess this kind of fault tolerance.

Why WebRTC Is Catastrophic for AI Voice

Luke Curley hits the nail on the head: when a user sends a voice prompt to an LLM, WebRTC will degrade or even discard portions of the user's prompt during network fluctuations.

The working mechanism of Large Language Models (LLMs) makes them extremely sensitive to input completeness. LLMs are based on the Transformer architecture and use attention mechanisms to process relationships between every token in the input sequence. When the speech-to-text (ASR) stage produces errors or gaps due to lost audio packets, these errors are treated by the LLM as genuine input for inference, creating a classic "garbage in, garbage out" effect. Unlike humans, LLMs lack true commonsense reasoning ability to fill in obvious transmission errors—they will faithfully generate responses based on the (potentially truncated) text they receive. Worse still, users cannot determine whether a low-quality AI response is due to model limitations or truncated input.

The logical conflict here is glaringly obvious:

What Users Actually Need

• Users are paying real money for every AI call (Curley's words: "paying good money to boil the ocean")
• A truncated prompt directly translates to a low-quality or useless AI response
• LLM inference itself isn't exactly fast—users can absolutely accept waiting an extra 200 milliseconds for accurate, complete input

What WebRTC Forces Instead

• The protocol hard-codes a real-time latency-first strategy at the protocol level, giving developers no choice
• In browser environments, it's impossible to retransmit a dropped WebRTC audio packet
• Curley mentions that they tried to implement audio packet retransmission at Discord, and the final conclusion was: it's technically impossible

Discord, as one of the world's largest real-time voice communication platforms with hundreds of millions of users and extremely rich real-time audio engineering experience, had their engineering team deeply investigate WebRTC's audio transport stack, attempting to implement custom audio packet retransmission mechanisms on the browser side. Their final conclusion was that WebRTC's browser implementation (primarily libwebrtc in Chromium) encapsulates packet-dropping decisions at a level unreachable by the application layer—the JavaScript API exposes absolutely no interfaces for controlling packet-drop behavior or triggering retransmissions. This isn't a limitation that can be bypassed through "hacks"; it's a fundamental architectural constraint.

Put simply, users pay money for AI to answer their questions, but WebRTC drops the critical parts of those questions during transmission. It's as absurd as calling customer service and having the phone line automatically censor your keywords.

Technically Irreconcilable: Hard-Coded Latency Priority

Curley's criticism goes beyond theory. He explicitly points out that this isn't a problem solvable by tweaking configuration parameters. WebRTC's implementation is "hard-coded for real-time latency or else"—latency priority is baked into the protocol with no room for negotiation.

This means that even when developers clearly know that data integrity should take priority over minimal latency in AI voice scenarios, WebRTC doesn't offer that option. You cannot tell WebRTC: "For this transmission, please ensure every audio packet is delivered—I'm willing to wait a few hundred extra milliseconds."

This reveals a deeper architectural problem: forcibly applying a protocol designed for one use case (real-time human-to-human calls) to a fundamentally different scenario (human-to-AI voice interaction) inevitably creates a mismatch.

The two scenarios define "real-time" in completely different ways:

• Real-time in video conferencing: Drop data rather than break conversational rhythm
• Real-time in AI voice interaction: Wait a moment longer rather than compromise input completeness

MoQ and Other Alternatives: More Flexible Protocol Options

Luke Curley himself is a driving force behind the MoQ (Media over QUIC) protocol. MoQ is built on the QUIC transport layer and provides more flexible reliability and latency trade-off options.

QUIC (Quick UDP Internet Connections) is a transport-layer protocol designed by Google and later standardized by the IETF, now serving as the underlying transport for HTTP/3. QUIC runs on top of UDP but implements TCP-like reliability guarantees while solving TCP's head-of-line blocking problem—it supports multiplexing, where packet loss on a single stream doesn't block other streams. MoQ leverages this QUIC feature, allowing different media streams to be assigned different reliability levels. For example, a developer could set the user's voice input stream to "fully reliable" (ensuring every packet is delivered) while setting the AI's voice output stream to "partially reliable" (allowing moderate packet loss to reduce latency). This kind of fine-grained control simply doesn't exist in WebRTC's architecture.

For AI voice interaction scenarios, the ideal transport protocol should allow developers to flexibly choose between "guaranteeing data completeness" and "minimizing latency" based on specific needs. MoQ's design moves in exactly this direction—it doesn't presuppose a fixed trade-off strategy but instead hands the choice to application-layer developers.

Of course, MoQ is still in its development phase and is far from matching WebRTC in ecosystem maturity and browser support. But from a technical architecture perspective, this flexible and configurable design philosophy is clearly better suited for emerging scenarios like AI voice than WebRTC's one-size-fits-all approach.

OpenAI's Pragmatic Considerations and the Core Lesson in Technology Selection

Despite the technical mismatch, OpenAI's choice of WebRTC has pragmatic business logic behind it. WebRTC is currently the only real-time media transport solution natively supported in all major browsers, requiring no plugins or additional software. Its ecosystem, developed over more than a decade, boasts mature TURN/STUN server infrastructure, comprehensive NAT traversal solutions, and numerous production-proven open-source implementations (such as libwebrtc, Pion, mediasoup, etc.). In contrast, MoQ has yet to gain native support from any major browser—developers must use it indirectly through the WebTransport API or WebAssembly, significantly increasing integration complexity and deployment barriers.

The core lesson this discussion brings to developers and architects is: technology selection cannot be based solely on surface-level feature matching.

WebRTC can indeed transmit real-time audio, but its definition of "real-time" and the requirements of AI voice interaction for "real-time" are fundamentally different. When OpenAI chose WebRTC, it gained a mature ecosystem and broad browser support, but also inherited underlying assumptions optimized for the wrong scenario.

In today's rapidly evolving landscape of AI voice applications, the choice of transport protocol will directly impact user experience and AI response quality. Understanding WebRTC's design limitations and following the development of emerging protocols like MoQ is crucial for building the next generation of AI voice products.

Key Takeaways

• WebRTC proactively drops audio packets to maintain low latency during poor network conditions, which is catastrophic for AI voice prompt completeness
• Retransmitting WebRTC audio packets in browsers is impossible—this limitation is hard-coded, as Discord's team has verified
• Users would rather wait an extra 200ms for an accurate AI response than receive garbage answers due to truncated prompts
• WebRTC's assumptions, designed for real-time human-to-human calls, fundamentally conflict with the requirements of human-to-AI voice interaction
• The QUIC-based MoQ protocol offers fine-grained reliability control, allowing different streams to have different transport strategies—potentially a better alternative for AI voice scenarios
• OpenAI's pragmatic reason for choosing WebRTC lies in its unmatched native browser support and mature ecosystem, but this is a pragmatic compromise at the cost of architectural fitness