One Line of Synchronous Code Cripples asyncio: Voice AI p95 Latency Spikes to 11.4 Seconds

A voice interview service responded in 0.9 seconds with a single user, but with 8 concurrent users, the last user waited 11.4 seconds—it sounded broken. The model, network, and voice provider were all fine. The problem was one line of code: calling the synchronous client = Anthropic() inside an async def.

The author of this bug is a developer at Preterview, a voice interview platform. He spent two full days investigating in the wrong direction before finally pinpointing the root cause through event loop lag monitoring. The entire process has complete data records and is valuable reference for any developer writing async Python.

Why Single-User Testing Doesn't Reveal It

A blocking event loop costs nothing when no other coroutines are waiting. In a single-user scenario, a 2.3-second blocking call looks just like a 2.3-second LLM call. There are no other sessions queued behind it—everything appears normal.

The author's local tests, staging environment, and demo calls all had only one user—himself. All measurements showed 0.9-second time-to-first-audio. So he shipped it.

Behavior in Production

The initial complaints were "the interviewer froze for a few seconds" and "the audio was choppy." It didn't happen in every session—only during certain times of day.

The author investigated three directions in turn: the speech-to-text provider was slowing down during peak hours (switched regions, no change); TTS chunks were too small (made them larger, got worse); the LLM was slowing down (but Claude's per-call latency was completely stable from 3 AM to 8 PM).

The third clue was the key. Model latency hadn't changed, but user wait times increased significantly during peak hours. The extra time was being consumed inside his own process.

How to Detect a Blocked Event Loop

Enable asyncio debug mode and lower the slow callback threshold—the event loop will log every step that exceeds the threshold along with its corresponding task:

import asyncio
import logging

logging.basicConfig(level=logging.WARNING)

async def on_startup():
    loop = asyncio.get_running_loop()
    loop.set_debug(True)
    loop.slow_callback_duration = 0.1

In a load replay test on staging, the logs were filled with entries like Task-412 coro=<session_turn() running at app/session.py:88>> took 2.312 seconds.

Debug mode has overhead and isn't suitable for production. The author switched to a 10-line lag monitor—sleep for 100 milliseconds, then check how late it woke up. If the event loop is idle, it wakes up on time; if it's busy, the wake-up delay reflects the blocking duration.

async def loop_lag_monitor(interval: float = 0.1, warn_ms: float = 50):
    loop = asyncio.get_running_loop()
    while True:
        start = loop.time()
        await asyncio.sleep(interval)
        lag_ms = (loop.time() - start - interval) * 1000
        if lag_ms > warn_ms:
            log.warning("event loop lag %.0f ms", lag_ms)

Over a week of production logs, this monitor recorded 1,412 warnings exceeding 100ms. After matching against staging debug traces, the source distribution was as follows:

Blocking Source	Warning Count	Percentage
Synchronous Anthropic client calls	868	61%
pypdf resume parsing	325	23%
Synchronous requests.post webhook + time.sleep retry	219	16%

The longest single block was 7.9 seconds: while pypdf was processing a 14-page resume, every in-progress interview on the server went silent.

The Difference Between Slow Callback Tracing and Loop Lag Monitoring

Asyncio's debug mode records timestamps before and after each event loop step (including a task's execution step, timer callbacks, and socket readiness handling). If the duration exceeds loop.slow_callback_duration, it outputs a warning with the running coroutine name and source location—this is direct evidence for locating the blocking point.

The lag monitor works on a different principle: it doesn't intrude into the event loop internals. Instead, it uses a periodic asyncio.sleep task as a heartbeat. If the loop is idle, the coroutine wakes up on time after 100 milliseconds; if a synchronous task is occupying the loop, the heartbeat is delayed. The wake-up time difference minus the expected interval equals the blocking duration.

The two approaches are complementary. Debug mode gives you the exact call stack of who is blocking, but has higher overhead—only suitable for staging. The lag monitor gives you production-grade metrics on how long blocking occurred, with near-zero overhead—suitable for long-term operation. This article used debug mode first to identify the source distribution on staging, then used the lag monitor to continuously count occurrences in production. The two datasets corroborated each other, forming a complete investigation chain.

Why Synchronous Clients Block Everything

async def only yields control at await points. Synchronous network calls have no await, so the event loop stays stuck until the socket read returns. The author's code looked fine:

client = Anthropic()  # sync client

async def session_turn(ws, history):
    with client.messages.stream(model=MODEL, max_tokens=400, messages=history) as stream:
        for text in stream.text_stream:      # blocks the loop waiting for each chunk
            await ws.send_text(text)         # yields, but only for a moment

await ws.send_text makes it look asynchronous. But for text in stream.text_stream is a synchronous iterator doing blocking socket reads between each chunk. In reality, the event loop is frozen throughout the entire generation process, except for a few microseconds of yielding on each send.

Audio frames, TTS chunks, and heartbeats from all other sessions are queued up waiting. With 8 people answering simultaneously, the last person has to wait for the previous 7 generations to complete before starting their own.

Event Loop and Cooperative Concurrency

Python's asyncio uses a single-threaded cooperative concurrency model. All coroutines share a single event loop on one thread, and only one coroutine can run at any given moment. A coroutine voluntarily yields control back to the event loop only when it encounters an await expression, which then schedules the next ready task.

The key point is that await is just a yield point—it can't make blocking code asynchronous by itself. If the underlying operation being awaited is still synchronously blocking—for example, a synchronous socket read—the event loop can't run any other tasks until it returns. Many developers mistakenly believe that writing async def automatically grants concurrency capabilities. In reality, async def only defines a coroutine factory, and synchronous code inside the function body still executes line by line, monopolizing the thread. This is why synchronous socket reads, requests calls, and time.sleep become killers in async programs: they don't yield control, causing all queued coroutines—audio frames, heartbeats, TTS chunks—to freeze simultaneously. Understanding this is essential to grasping why one line of synchronous client code causes latency to compound across all users on the server.

The Degradation Curve Under Concurrency

The author wrote a load test that replays recorded answers and measures time-to-first-audio (TTFA)—from when the user stops speaking to when the first TTS byte is sent. Here is the p95 data for 200 conversation rounds per level:

Concurrent Sessions	Sync Client	Sync Client in to_thread	AsyncAnthropic
1	0.9s	0.9s	0.9s
4	5.2s	1.0s	0.9s
8	11.4s	3.4s	1.0s

Is asyncio.to_thread Good Enough?

Only up to a point. asyncio.to_thread runs calls on the event loop's default ThreadPoolExecutor. Since Python 3.8, the default number of worker threads is min(32, os.cpu_count() + 4). On a 2-vCPU machine, that's 6 threads.

The author's first instinct was to wrap the synchronous stream in to_thread, and it seemed to solve the problem at 4 sessions. But at 8 sessions, two conversations were waiting for idle threads in the executor queue, and p95 rose to 3.4 seconds. The event loop was no longer frozen, but a new bottleneck appeared: the thread pool.

You can expand the thread pool, but doing so means every in-progress LLM call occupies a thread—just to keep a client that wasn't designed for your runtime working. The SDK already provides an async client; just use it.

Thread Pools, the GIL, and Concurrency Bottlenecks

asyncio.to_thread throws synchronous functions onto the default ThreadPoolExecutor for execution, thereby moving blocking off the event loop thread and restoring loop responsiveness. But Python's Global Interpreter Lock (GIL) means only one thread can execute Python bytecode at a time. For network I/O operations that release the GIL while waiting, multithreading can indeed achieve concurrent waiting. However, for CPU-intensive pure Python tasks like pypdf parsing, threads will repeatedly contend for the GIL, yielding little concurrency benefit—and possibly even slowing down due to thread switching.

Additionally, the default thread count is min(32, os.cpu_count() + 4), which is only 6 on a 2-vCPU machine. Once concurrency exceeds the thread count, new tasks wait for idle threads in the executor queue. So the event loop is no longer frozen, but tasks start queuing, and latency follows a new degradation curve. The correct decision is to categorize: I/O-bound synchronous calls should preferentially use official async clients; CPU-intensive tasks that truly can't be made async should go into ProcessPoolExecutor to bypass the GIL; only a small number of occasional synchronous blocks are suitable for to_thread as a transitional solution.

The Correct Fix

Use AsyncAnthropic and iterate the stream with async for, so that each wait for the next chunk is a genuine await:

from anthropic import AsyncAnthropic

client = AsyncAnthropic()

async def session_turn(ws, history):
    async with client.messages.stream(model=MODEL, max_tokens=400, messages=history) as stream:
        async for text in stream.text_stream:
            await ws.send_text(text)

A three-word change brought the p95 at 8 sessions from 11.4 seconds down to 1.0 second.

The other 39% of warnings needed different fixes. pypdf resume parsing is CPU-intensive pure Python code. Moving it to to_thread keeps the event loop mostly responsive but still competes with the loop thread for the GIL. The author ultimately moved it to ProcessPoolExecutor(max_workers=2) with loop.run_in_executor. requests.post was replaced with httpx.AsyncClient. The time.sleep(2) in the retry helper was changed to await asyncio.sleep(2)—it was hidden in a "utils" file, freezing the entire server for two full seconds on every retry.

In the week after the fix, the lag monitor recorded only 9 warnings exceeding 100ms—all under 180ms, all from garbage collection pauses. From 1,412 down to 9.

Preventing Recurrence

The author summarized three measures:

Keep lag monitoring in production and alert when p99 loop lag exceeds 100ms. The code is only 10 lines with near-zero overhead.

Load tests should test concurrency, not just latency. Single-session tests simply can't catch this type of bug. The author's CI now runs replay tests at 1, 4, and 8 sessions, failing if p95 at 8 sessions exceeds 1.5 times that of 1 session.

Search async code paths for common suspects: Anthropic(, OpenAI(, requests., time.sleep, and synchronous file or PDF parsing. One lint rule that flags synchronous clients in modules containing many async def caught two more issues before they shipped.

The author concludes: In async Python, the slowest blocking call determines the latency for every user on the machine—not just the user who triggered it.