Cloudflare AI Cache Leak: Why Opening a New Chat Isn't Enough

Cloudflare published an incident report on October 26 regarding the GLM 5.3 deployment on Workers AI. The report describes a historical event that has already been fixed, not an ongoing leak. According to analysis by a YouTube channel, the core issue in this incident was that when two users sent requests in succession, the second reply carried fragments of data from the first request. Cross-verification by independent sources shows that the root cause was not contamination of the model's training data, but rather a failure of isolation in the inference engine's shared memory management.

Incident Recap: A Chain Reaction Triggered by a "Long Prompt"

Cloudflare precisely scoped the impact to 155 affected responses, all of which came from prompts longer than 681,280 tokens. This number is not a percentage of all chat requests, but the number of samples confirmed to be affected in this incident. The exposure window lasted from the model's launch on August 27 to September 9; on September 8, a researcher reported the issue, and Cloudflare took the model offline during the investigation.

The incident report specifically emphasizes that this was a problem within a single named deployment and does not mean all AI services are leaking. However, the design flaw it exposes carries broadly applicable lessons.

Key Distinction: KV Cache Is Not Model Weights

To understand this incident, one must first clarify a concept that is often confused. A running model continuously maintains a temporary numerical representation of the text it is currently processing, commonly referred to in the industry as the key-value cache. This working state is entirely different from modifying the model's training weights. The appearance of data in a particular response does not in itself prove that the model was trained on that data.

Cloudflare's deployment used a disaggregated architecture: one set of GPUs was responsible for preparing prompts, and another for generating responses. The prepared cache needed to be transferred between the two sets of processors. This can be understood through the analogy of numbered trays: a request reserves a number of trays and releases them after use. Reusing memory itself is not a leak; the key is that the system must ensure each request only reads state belonging to itself. Ownership and complete writes are the real boundaries.

Technical Nature of KV Cache: When a Transformer model performs autoregressive generation, each new token generated requires recomputing attention over all previous tokens. To avoid this quadratic repeated computation, the inference engine caches the Key and Value projection matrices of already-processed tokens in each layer—this is the KV cache. Its size is proportional to sequence length, number of layers, number of attention heads, and precision (FP16/FP8), so in long-context scenarios the KV cache can easily reach tens of GB and must be managed via paging. In a disaggregated architecture, the KV cache produced during the prefill stage must be transferred to the decode node via high-speed interconnect (such as NVLink or RDMA); this transfer and memory reuse are precisely the source of isolation risk.

Root Cause Analysis: Capacity Mismatch and Failure of the Rejection Path

The direct trigger of the incident was a mismatch between the context window and the actual capacity of the preparation pool. The context window allowed 1,048,576 tokens, while the preparation pool held only 681,280. A request could satisfy the publicly advertised context window while exceeding the actual upper limit of the preparation pool. Cloudflare originally expected the engine to return an error, but after the oversized request was internally rejected, it was replaced with a single-token placeholder.

The problem lay in the handoff step of the disaggregated deployment: the rejection did not prevent the request from being passed on. The preparation side transmitted the small cache for the placeholder, but the generation side treated it as a complete transfer—even though it had already reserved space for the original long request. Most of the reserved pages were not overwritten by the new request and still retained early data from the previous request. The generation stage read these stale pages, causing another request's data to be mixed into the response.

Here is an easily overlooked lesson: a coherent answer and a successful web response do not prove that isolation is in effect. Early testing only checked response codes and performance, without verifying the ownership of the output content.

Paged Attention and Memory Pools: Modern inference engines (such as vLLM's PagedAttention) divide the KV cache into fixed-size blocks (block/page), allocated on demand from a unified memory pool, analogous to virtual memory paging in operating systems. Pages from different requests may be physically adjacent or reused, so the engine must maintain a "page table" to record which blocks each request owns. If the page table is not properly cleaned up after a request is rejected, or if a new request does not fully overwrite reserved pages, subsequent reads will hit stale pages belonging to other requests. This is precisely the mechanism-level explanation for "reserved pages not overwritten" in this incident—the isolation failure occurred in page table and ownership management, not in the computation itself.

Remediation: Expanding Capacity and Closing the Rejection Path

Cloudflare's immediate fix had two parts. First, expand the cache pool to accommodate the full context. Second, patch the rejection path so that rejected requests are stopped before allocation and transfer.

According to Cloudflare's statement, both fixes passed reproduction tests and marker tests before the model was restored on September 9. Further hardening was subsequently carried out. The investigation concluded that, after excluding its own tests and the researcher's requests, no evidence of deliberate exploitation was found—this is an investigative finding, not an assertion that "exposure never occurred."

Reproduction Tests and Marker Tests: Reproduction testing means reconstructing the incident path with a known triggering long prompt to confirm that cross-request data mixing no longer occurs after the fix; marker testing (canary/marker test) involves implanting a uniquely identifiable sentinel string into a request and then checking whether that marker appears in responses to other requests. Such tests target "data ownership" rather than "whether the request succeeded," and can catch subtle failures where the response code is normal but the content crosses boundaries. They are an essential means of isolation verification under shared inference memory architectures.

Implications for Operators and Users

For those operating AI services, context limits must match actual capacity; failure paths also require isolation testing. A successful request must be tested not only for whether it completed, but also for whose data it contained.

For ordinary users, opening a new chat window does not verify server-side internal isolation. Sensitive materials should be handled with caution; no setting toggle can fix a server-side memory problem.

The real question this incident leaves for the industry is not "can the model answer," but "did this answer use only the data this request was permitted to read." In architectures where shared inference memory is the norm, the latter is the starting point for security assessment.

One-line Summary

Cloudflare incident reveals inference engine KV cache isolation failure leading to cross-request data leakage.

Paragraph Summary

Cloudflare published an incident report on the GLM 5.3 deployment on Workers AI, disclosing a fixed historical event: 155 responses were mixed with data fragments from other requests due to a failure of shared memory isolation in the inference engine. The root cause was a mismatch between the context window (1,048,576 tokens) and the preparation pool (681,280 tokens); oversized requests were internally rejected and replaced with a single-token placeholder, but the rejection failed to block cache transfer in the disaggregated architecture, and the generation side read stale pages that had not been overwritten. The report emphasizes that the KV cache is temporary working state unrelated to model training weights. Remediation included expanding the cache pool and terminating rejected requests before allocation, verified through reproduction and marker tests. The lesson: a successful response does not mean isolation is in effect; security assessment should focus on whose data the response used.