# Cloudflare's Reported KV Cache Exposure Across Requests: Why Did SGLang Handoff a Rejected Request?

> Canonical: https://vxi.me/en/posts/glm53-kv-cache-cross-request-exposure/
> Locale: en-us
> Published: 2026-10-28
> Updated: 2026-10-29
> Markdown: https://vxi.me/en/posts/glm53-kv-cache-cross-request-exposure.md

<figure class="glm53-kv-rejection-probe not-prose" lang="en-us">
<picture><source media="(prefers-reduced-motion: reduce) and (max-width: 640px)" srcset="/images/posts/glm53-kv-cache-cross-request-exposure/glm53-kv-rejection-probe.mobile.still.en-us.svg" type="image/svg+xml"><source media="(prefers-reduced-motion: reduce)" srcset="/images/posts/glm53-kv-cache-cross-request-exposure/glm53-kv-rejection-probe.still.en-us.svg" type="image/svg+xml"><source media="(max-width: 640px)" srcset="/images/posts/glm53-kv-cache-cross-request-exposure/glm53-kv-rejection-probe.mobile.en-us.svg" type="image/svg+xml"><img src="/images/posts/glm53-kv-cache-cross-request-exposure/glm53-kv-rejection-probe.en-us.svg" alt="Counterexample probe: HTTP 200 and transfer-complete do not prove complete KV pages or ownership by the current request. The incident path comes from Cloudflare's report; the pinned CPU diff does not reproduce GPU/RDMA or a real cross-tenant leak." width="1200" height="700" loading="eager" fetchpriority="high" decoding="async" style="display:block;width:100%;height:auto"></picture>
<figcaption class="mt-2 text-xs leading-relaxed text-slate-500">Counterexample probe · Separates the reported incident path from the pinned CPU diff. No real cross-tenant leak was reproduced.</figcaption>
</figure>
In October 2026, Cloudflare disclosed cross-request KV-cache exposure affecting GLM-5.3 on Workers AI. Its report says 155 responses were affected between August 27 and September 9, and the relevant prompts exceeded 681,280 tokens. It did not provide the total number of requests in that period, so an exposure rate cannot be calculated from the disclosure. [Cloudflare incident report](https://blog.cloudflare.com/kv-cache-exposure-remediation/) · [SGLang coordinated disclosure](https://www.lmsys.org/blog/2026-10-26-cf-security).

This was not a general risk affecting every AI API. Cloudflare said other Workers AI models were unaffected and GLM-5.3 was fixed and restored on September 9. The disclosed impact concerned GLM-5.3 requests above 681,280 input tokens during the incident window. API users should first check whether their use matches that scenario; operators running their own SGLang prefill/decode-disaggregated clusters should use the four acceptance gates below.

The useful question is narrower than “how can a model leak?”: **why could a request already marked for rejection still enter the prefill/decode handoff with a one-token placeholder?** Two conditions combined: the declared context limit exceeded the capacity of the actual KV pool, and the rejection state did not stop the request early enough at the disaggregated-inference entry point. The first triggered rejection; the second let the rejected request continue down a path it should not have taken.

**The seven-step production path described by Cloudflare** (PR #36988 had merged on September 6 but had not yet reached Cloudflare production when the September 9 engine fix was deployed):

1. **Bootstrap:** the router tells decode the original prompt length; decode reserves the corresponding KV pages.
2. **Prefill admission:** admission checks `req.finished()`. The abort reason is recorded in `to_finish`, not `finished_reason`, so the request can still look unfinished and be admitted.
3. **Prefill forward:** the prefill input is replaced by a one-token placeholder.
4. **KV transfer:** prefill sends only the page corresponding to that placeholder but signals `transfer-complete`.
5. **Decode admission:** the same finish-state check on decode accepts the handoff.
6. **Decode attention:** decode reads pages up to the reserved length, although only the first page was written by the current request; other pages may still contain data left by an earlier request.
7. **Response:** Cloudflare reported an HTTP 200 instead of the expected HTTP 400, with content from another user's session.

The individual steps can look normal in isolation. Together, they show the mismatch: rejection had not taken effect, yet later stages continued to process the request normally. See the [Cloudflare report](https://blog.cloudflare.com/kv-cache-exposure-remediation/), [SGLang issue #30233](https://github.com/sgl-project/sglang/issues/30233), and [PR #38935](https://github.com/sgl-project/sglang/pull/38935).

## 681,280 and 1,048,576 describe different capacities

Cloudflare configured the model context length as 1,048,576 tokens, while the prefill GPU's KV-pool capacity at the time was 681,280 tokens. It later increased the pool to about 2.04 million tokens and added an engine-side rejection safeguard. The two occurrences of 681,280 in the report refer to different things: the affected-prompt threshold and the actual pool capacity. The number is the same; the roles are not.

Context length is the configured upper bound for a request. The KV pool is the runtime memory pool that can actually allocate and retain key/value state for tokens. Increasing the configured context does not automatically expand the pool. A request can therefore remain below the declared context limit while exceeding what the current worker can hold. Insufficient pool capacity should cause rejection; the critical failure was that the rejection state did not stop the later handoff.

## A recorded abort does not mean admission has retired the request

In the relevant SGLang path, `Req.set_finish_with_abort()` records the rejection reason in `to_finish` and replaces the prompt with a one-token placeholder. But the old `finished()` check reads `finished_reason`. The abort reason is present on the request object, while the predicate used to decide whether it has finished can still treat it as an ordinarily schedulable request. See the root-cause discussion in [SGLang PR #38935](https://github.com/sgl-project/sglang/pull/38935).

The state mismatch is easier to verify directly in the source. These are **non-contiguous excerpts** from the pinned commit; omitted lines are marked. See the full context for [`set_finish_with_abort()` (L2053–2072)](https://github.com/sgl-project/sglang/blob/8f3d6944b76c428d884582cb6c82a39db1465360/python/sglang/srt/managers/schedule_batch.py#L2053-L2072), [`finished()` (L1479–1481)](https://github.com/sgl-project/sglang/blob/8f3d6944b76c428d884582cb6c82a39db1465360/python/sglang/srt/managers/schedule_batch.py#L1479-L1481), and [`update_finish_state()` (L1842–1848)](https://github.com/sgl-project/sglang/blob/8f3d6944b76c428d884582cb6c82a39db1465360/python/sglang/srt/managers/schedule_batch.py#L1842-L1848):

```python
# Req.set_finish_with_abort() — excerpt
self.origin_input_ids = array(
    "q", [0]
)  # set it to one token to skip the long prefill
# ... omitted ...
self.to_finish = FINISH_ABORT(error_msg, status_code, err_type)

# Req.finished()
return self.finished_reason is not None

# Req.update_finish_state()
if self.to_finish:
    self.finished_reason = self.to_finish
    self.to_finish = None
```

This does not mean `to_finish` is never applied: `update_finish_state()` later moves it into `finished_reason`. The failure is that admission checks `finished()` before that transition occurs.

Before Cloudflare deployed the engine fix on September 9, this state mismatch could propagate through prefill/decode disaggregation. Admission treated the request as eligible; decode reserved enough KV pages; prefill processed only the placeholder and transferred insufficient KV state. Decode could then read pages not overwritten by the current request. Cloudflare attributed the actual exposure to KV data left by earlier requests. This is the vendor's incident account, not a leak reproduced by the tests described here.

Three signals are easy to confuse:

- **HTTP 200** means the client received a successful HTTP status. It does not prove semantic correctness or page ownership.
- **Prefill completion / `transfer-complete`** means a processing or handoff step emitted its completion signal. It does not prove that the transferred tokens cover all pages reserved by decode.
- **Decode reading complete, valid KV pages** is closer to the data-level check that matters. Without checking the actual write range and page ownership, a completed transfer can be mistaken for complete, request-owned data.

A completed flow is not necessarily correct data.

## #36988, #38935, and #38704 were not one fix

| Date / record | Status | What it addressed |
| --- | --- | --- |
| 2026-07-06, [issue #30233](https://github.com/sgl-project/sglang/issues/30233) | Closed | Earlier report of a similar long-prompt, one-token handoff problem in PD disaggregation. It is not an independent reproduction of the full Cloudflare incident. |
| 2026-09-06, [PR #36988](https://github.com/sgl-project/sglang/pull/36988) | Merged; Cloudflare associated it with v0.5.20 | Correctness and cleanup for aborted prefill results: clear pending chunks, avoid re-queuing abort results as unfinished, release metadata/KV/HiCache/Mamba resources, and ignore delayed results for retired requests. |
| 2026-09-16, [PR #38935](https://github.com/sgl-project/sglang/pull/38935) | Merged; Cloudflare associated it with v0.5.21 | Retire rejected requests earlier at prefill and decode admission, avoiding unnecessary bootstrap, metadata buffers, sender/receiver initialization, and forward passes. Its author described this as a waste-and-latency fix, not a second independent correctness fix after #36988. |
| Created 2026-09-09, closed 2026-10-10, [PR #38704](https://github.com/sgl-project/sglang/pull/38704) | Closed, not merged; no release | An alternative safeguard proposed by Cloudflare; it must not be described as part of the upstream mainline. |

PR #38935 provides a useful timing anchor: on a post-#36988 main, decode no longer submitted the insufficient handoff. Its upstream 1P1D (2×H200) before/after comparison showed decode changing from a 300-second timeout to an immediate 400 response after #38935; bootstrap and metadata buffers were no longer created, and the stub batch went from 1 to 0. This is an upstream comparison, not a benchmark run for this article.

The guard cannot simply say “retire every request whose `to_finish` is non-empty.” `is_unadmitted_reject` distinguishes a rejected request that has not entered and acquired resource ownership. A re-entering or preempted request may already own KV or metadata resources and must not be dropped solely because it carries an abort marker. Decode also needs to account for re-entry because some resource state may not remain on the `Req` object. Retiring too early could skip cleanup owned by the normal scheduling path, such as detaching a streaming session's active slot.

Other safeguards address related but different failure modes. [PR #39500](https://github.com/sgl-project/sglang/pull/39500) added an optional handoff KV checksum. Cloudflare also described KV canaries, which detect and record potential corruption, and read-time page tagging. Page tagging was a prototype tested in summer 2026, tried on GLM-5.2 and later disabled for reliability reasons; it is not a general currently deployed safeguard. Cloudflare said checksum and canary support did not yet cover all model architectures, including GLM-5.3. A handoff checksum checks transfer from prefill to decode, not every later decode read.

[PR #40645](https://github.com/sgl-project/sglang/pull/40645) and [PR #41402](https://github.com/sgl-project/sglang/pull/41402) jointly address drain-acknowledgement gaps or missing coverage of decode peers. [PR #41404](https://github.com/sgl-project/sglang/pull/41404) extends deferred release to transfer failures beyond decode-initiated aborts, so pages are not released or reused while in-flight RDMA writes may still touch them. These KV-page lifetime races are distinct from the deterministic rejected-request handoff path.

## Pinned CPU validation checks code paths, not a tenant leak

The evidence here has three separate layers:

| Evidence | Observation | What it supports |
| --- | --- | --- |
| Synthetic Python state machines | Handoff model 6/6; admission/re-entry model 9/9 | The guards in these simplified models behave as expected, not that SGLang runtime behavior was reproduced. |
| Pinned upstream SGLang CPU unit tests (P1) | 19 passed across two test files; the admission test also passed 12 subtests | Real upstream mixin, queue, and retirement code ran. Some transport/cache/scheduler collaborators were mocks; no live cluster was used. |
| Before/after probes (P2) | Both the #38935 admission diff and #36988 prefill-cleanup diff were observed | Fixed inputs exercised different code paths across the two versions; these are CPU code/probe results only. |

P1 is pinned to SGLang commit **8f3d6944b76c428d884582cb6c82a39db1465360** ([source](https://github.com/sgl-project/sglang/commit/8f3d6944b76c428d884582cb6c82a39db1465360)). The tests were `test_prefill_abort_result_cleanup.py` (11 passed) and `test_admission_abort_not_enqueued.py` (8 passed, plus 12 subtests), 19 passed in total. The environment was a Linux CPU-only VM with Python 3.12.3, pytest 9.2.0, and CPU torch. No GPU, model weights, RDMA, or tenant data were used.

P2 isolates each PR's base and head rather than mixing fixes:

- **Experiment A — #38935 admission (base [17ba2c2e](https://github.com/sgl-project/sglang/commit/17ba2c2e7c7b81f31a8a9e693e7435ab262c16b4) → head [8f3d6944](https://github.com/sgl-project/sglang/commit/8f3d6944b76c428d884582cb6c82a39db1465360)).** Before the fix, rejected requests entered the queue and triggered sender/receiver creation. Afterward, they were retired at admission without creating those handoff objects. Valid and re-entering requests with metadata buffers were preserved in both trees.
- **Experiment B — #36988 prefill cleanup (base [9c254df8](https://github.com/sgl-project/sglang/commit/9c254df8cdbe99ef5f58e7fcd932da052f258538) → head [2ee5742a](https://github.com/sgl-project/sglang/commit/2ee5742af92db2727236460473554c2ee8c4ddaa)).** The probe observed that, before the fix, a final aborted result could be cached as unfinished and re-queued; a delayed result for an already retired request could still send a KV chunk. After the fix, cleanup completed as expected and delayed results no longer triggered those side effects.

P2's 7/7 and P1's 11 passed come from different versions. P2 uses the seven cases present at the #36988 head, `2ee5742a`. The later pinned `8f3d6944` added sampling-mask parameterized tests, expanding that file to 11 cases. Thus **0/7 → 7/7** is a before/after comparison of the same seven cases, not something to add to P1's 11. The seven pre-fix test failures were six `AttributeError` exceptions and one `ValueError`, caused by the code falling through into a normal transfer path not modeled by the test stub. They were not seven failed business assertions. The more useful evidence is the observed re-queueing, cleanup, and delayed-send behavior.

### Public evidence and reproduction notes

I published the two differential probes, four raw JSON outputs, the P2 receipt, and a SHA-256 checksum list in the [public evidence bundle](/evidence/glm53-kv-cache-cross-request-exposure/README.md). It pins both base/head pairs and documents the Mock boundaries and CPU-only scope. The pre-fix 0/7 result is stub fall-through (six `AttributeError` and one `ValueError`), not seven assertion failures. These artifacts support the observed code-path differences on pinned versions; they do not reproduce a GPU/RDMA issue or a real cross-tenant leak.

## Four gates operators should verify

For a multi-tenant inference service or an SGLang prefill/decode-disaggregated deployment, “upgrade to v0.5.22, send one long prompt, see HTTP 400, and call it done” is not enough. Configuration and resource-lifecycle paths can still undermine isolation. I would separate acceptance into four independent gates:

1. **Pool / context — does configuration match runtime capacity?** After a worker starts, compare the KV-pool capacity printed in startup logs with the configured context limit. Recheck after changes to configuration, GPU partitioning, or concurrency. Include a negative case where input is within the context limit but exceeds the actual pool capacity, and confirm explicit rejection rather than continued handoff.
2. **Reject / admission — does rejection take effect at the entrance?** Verify that a rejected request is retired before bootstrap, queue admission, metadata-buffer allocation, or sender/receiver initialization. Test new and valid requests, re-entry, preemption, and requests that still own KV, Mamba, or metadata resources. Do not discard every request carrying an abort state; cleanup ownership still matters.
3. **Transfer / read — does the valid transferred range cover the range read?** Compare reserved token/page counts with the actual written and transferred range. HTTP 200 and `transfer-complete` cannot replace length and ownership checks. If considering `--disaggregation-enable-kv-checksum` or `--kv-canary`, first confirm support for the current model and backend, then inspect logs to verify that validation actually ran. Cloudflare said these mechanisms did not cover all architectures, and a handoff checksum does not cover every decode read. Do not count an unsupported option as a working safeguard.
4. **Lifetime / ownership — when may pages be released or reused after failure?** Add negative cases for aborts, delayed results, and in-flight RDMA writes. Ensure unfinished writes cannot corrupt reallocated pages, delayed results from retired requests cannot send more chunks, and resources are released only once. Confirm deferred KV release on both prefill and decode. For Mooncake, check that transfer-batch limits and release timeouts are compatible, so a timeout does not leave a long tail of in-flight writes. Versions including [PR #41023](https://github.com/sgl-project/sglang/pull/41023) enable deferred release by default for supported backends, but verify the actual version and configuration. #40645 and #41402 address drain acknowledgement; #41404 handles deferred release for other transfer failures. Test these lifecycle risks separately from the original deterministic handoff defect.

SGLang [v0.5.22](https://github.com/sgl-project/sglang/releases/tag/v0.5.22) was released on October 19, 2026. Cloudflare recommended v0.5.22 or newer, or backporting the relevant fixes as needed. An upgrade alone is not proof that a deployment passes isolation acceptance: verify the actual runtime version, pool/context configuration, negative admission cases, and page lifecycle. If checksum support is not enabled, do not count it as an active defense.

The key lesson is not that one Boolean was wrong. Rejection state, data integrity, and resource ownership are separate boundaries. A reason recorded on a request does not mean admission sees it; a completed transfer does not mean every page read by decode was written by the current request; an abort signal does not mean all in-flight writes have ended. Verify each boundary separately before treating a successful-looking request path as evidence of data isolation.

*Tests performed: October 28, 2026.*

## Verification context

- Level: source-inspected
- Verified: 2026-10-28T00:00:00.000Z
- Environment: Linux; AMD EPYC 9D25; 2 vCPU; 7 GB RAM
- Environment: Python 3.12.3; pytest 9.2.0; torch 2.14.1+cpu
- Evidence mode: mixed
- Public repository: https://github.com/sgl-project/sglang
- Evidence commit: 8f3d6944b76c428d884582cb6c82a39db1465360
- Evidence receipt: https://vxi.me/evidence/glm53-kv-cache-cross-request-exposure.json