GLM-5.3 on Workers AI Leaked User Prompts: SGLang Fix List

Cloudflare's GLM-5.3 Workers AI endpoint was found to return other customers' data in some long-prompt responses. According to the original report, a total of 155 long-prompt responses contained cross-user content leakage. This incident not only exposed isolation flaws in the inference service stack, but also sounded an alarm for all providers and developers calling large models through APIs: when models are hosted on shared inference infrastructure, session isolation and cache management must be treated as first-class citizens.

Incident Review: Cross-User Leakage Triggered by Long Prompts

According to the original material, the problem occurred on the GLM-5.3 model endpoint running on the Cloudflare Workers AI platform. When requested prompts were long, some responses were mixed with other customers' content. The figure of 155 affected responses comes from the original report, indicating that the leakage was not an isolated case but a systemic issue reproducible under specific load patterns.

The reason long prompts become a trigger condition is usually related to how the inference engine manages context length, batching strategy, and KV cache. When multiple requests are dynamically batched into the same inference batch, if session boundary markers or cache key design has flaws, one request's context may be incorrectly associated with another request's output.

For API users, the danger of this type of problem is that it does not appear as an error. What users see is a grammatically complete, seemingly reasonable response—except the content belongs to someone else. This means conventional health checks and error-rate monitoring cannot detect the leakage; dedicated content isolation audits are required.

Where the SGLang Inference Stack May Go Wrong

SGLang is a widely adopted high-performance large model inference framework, and its core advantages lie in RadixAttention prefix caching and efficient batching scheduling. While these optimizations improve throughput, they also introduce the complexity of shared state.

From an architectural perspective, the potentially problematic areas are concentrated in several places. The first is the key design of prefix caching: if the cache key does not fully include tenant identifiers or session identifiers, identical prefixes from different users may hit the same cache entry. The second is request lifecycle management in batching scheduling: when long prompts cause requests to remain in the queue longer, if the scheduler's slot reuse logic has race conditions, residual context may be passed to the next request. The third is incomplete buffer cleanup on the streaming response path.

The original material uses "SGLang Fix List" as a clue, pointing precisely to this type of inference-layer defect. It should be noted that the original material does not provide a complete technical root-cause analysis of which specific code path caused this leakage. Therefore, the discussion here is a reasonable inference about high-risk areas in such systems, not a restatement of official conclusions.

RadixAttention prefix caching is SGLang's core optimization technique. Its principle is to build multiple requests' prompts into a radix tree, compute the KV cache only once for identical prefixes, and allow subsequent requests to reuse it directly. In multi-tenant scenarios, this sharing mechanism requires strict cache key design: the key must include a unique tenant or session identifier; otherwise, identical prefixes from different users will hit the same cache entry, causing context crosstalk. The KV cache itself is the intermediate computation result of key-value pairs in the attention mechanism and consumes a large amount of GPU memory, so inference engines tend to reuse it as much as possible rather than discard it—which also amplifies the consequences of isolation flaws.

Continuous batching refers to the inference engine dynamically concatenating tokens from different requests into batches at the iteration level to maximize GPU utilization. The batching scheduler maintains each request's lifecycle and slot allocation. When long prompts extend request residency time, if slot reuse lacks strict session boundary cleanup, residual KV state may be read by the next request. Buffers on the streaming response path also need to be isolated per request to avoid incomplete reset after abnormal interruption or timeout.

Upgrade and Configuration Checklist

For such issues, operations and platform teams should establish an executable verification process. The following checklist combines general practices for inference service security hardening and is consistent with the upgrade and configuration directions emphasized in the original material.

Upgrade Level

Following up on the latest versions of SGLang and Workers AI-related components as soon as possible is the most direct way to eliminate known defects. When upgrading, pay attention to release notes related to cache isolation, batching scheduling, and multi-tenant support, rather than only looking at performance numbers. For teams running self-hosted inference clusters, it is recommended to first run regression tests in a canary environment using long prompts and multi-tenant concurrent scenarios, and then roll out fully after confirming that isolation behavior meets expectations.

Configuration Level

Configuration checks should cover several key points. Confirm whether cache keys include the tenant dimension in multi-tenant deployments; confirm that the maximum sequence length and concurrency settings for batching do not cause excessive request concatenation; confirm whether the streaming interface correctly resets session state after abnormal interruption; confirm whether logs record sufficient request identifiers for post-incident auditing. Individually, these configuration items are performance tuning parameters, but in shared inference scenarios they are also security boundaries.

Monitoring Level

Conventional latency and error-rate metrics are insufficient to detect content leakage. It is recommended to add sampling comparisons based on content fingerprints, such as checking consistency between response content and request context, or injecting markers into requests from different tenants that should belong only to that tenant and regularly verifying that the markers do not appear in others' responses.

Content fingerprinting is a lightweight method for content consistency verification, usually generating a unique identifier by hashing text or extracting n-gram features. In inference service monitoring, the fingerprint of the request context can be compared with suspected context fragments in the response; if a fingerprint marker that does not belong to the current tenant appears in the response, an alert is triggered. Another practice is Canary Token: inject a random string belonging only to that tenant into the request; if the string appears in another tenant's response, it can be confirmed that the isolation boundary has been breached. Such methods do not rely on semantic understanding of model output, are low-cost and automatable, and are suitable as routine auditing measures for multi-tenant inference services.

What API Users Should Ask Providers

For developers relying on hosted APIs, this incident raises several questions that must be clarified. Does the provider run long-context requests from different customers on the same inference instance? How are cache and batching isolation mechanisms designed? When a similar incident occurs, is the provider able to locate the affected request scope and proactively notify customers? Are the detection and response processes for content leakage publicly available?

These questions may sound compliance-oriented, but they directly determine the data risk of downstream applications. Teams handling sensitive business data in particular should include inference-layer isolation capabilities in vendor evaluation checklists, rather than only comparing model quality and price.

Implications for the Industry

Large model inference is moving from single-tenant experiments to multi-tenant large-scale services, and the tension between performance optimization and security isolation will become increasingly obvious. Technologies that improve efficiency—prefix caching, dynamic batching, and streaming output—essentially all involve shared state, and shared state is the source of isolation risk.

The value of this leakage incident is not to blame a particular framework, but to remind the entire ecosystem: security testing of the inference stack needs dedicated design and cannot rely only on functional tests and performance stress tests. For framework maintainers, this means incorporating multi-tenant isolation into continuous integration; for platform providers, this means making content isolation auditing a launch gate; for application developers, this means maintaining reasonable skepticism about the isolation capabilities of hosted services and implementing proper data classification.

Multi-tenant isolation in inference services involves multiple layers: network and process isolation, GPU memory and KV cache isolation, scheduling queue isolation, and output buffer isolation. Unlike traditional SaaS multi-tenancy, large model inference has finer sharing granularity—it shares not only compute resources but also intermediate computation results (KV cache) and batching batches. Therefore, isolation design needs to run through the entire data flow of the inference engine, not just the deployment architecture. Isolation testing in continuous integration should include multi-tenant concurrent stress testing, interleaved long-context requests, abnormal interruption recovery, and other scenarios, and should treat content leakage detection as an assertion condition rather than only verifying functional correctness and throughput metrics.

One-line Summary

Cloudflare's GLM-5.3 endpoint leaked other customers' data in long prompts, exposing multi-tenant isolation flaws in the inference stack.

Paragraph Summary

The GLM-5.3 endpoint on Cloudflare Workers AI returned other customers' data in 155 long-prompt responses, constituting cross-user content leakage. The analysis points out that the leakage is related to high-risk areas of shared-state mechanisms in the inference stack, including SGLang's prefix cache key design, request lifecycle management in dynamic batching, and streaming response buffer cleanup. Because leaked responses are grammatically complete and do not trigger error-rate alerts, conventional monitoring struggles to detect them. The article proposes a remediation checklist including upgrading components, checking multi-tenant cache keys and batching configurations, and adding content-level isolation audits, and suggests that API users ask providers about tenant co-location policies and incident notification capabilities, calling for multi-tenant isolation to be incorporated into continuous integration and launch gates.