Cross-Tenant Leaks in AI Infrastructure: Cloudflare's KV Cache Data Leak and Vercel's Sandbox Findings

A cross-tenant leak happens when one customer's request reads memory or storage that still holds another customer's data. In 2026, Cloudflare and Vercel each disclosed leaks of this kind. At Cloudflare, the inference engine read stale KV cache pages after it rejected an over-length prompt. At Vercel, a sandbox could read recycled disk blocks from deleted sandboxes.

Both leaks have the same cause. The system read reused state and assumed the current request owned it. The fix is to check ownership (or confirm the data was overwritten) before the read, and then prove the fix with cross-tenant tests.

The Key Numbers

Are the 155 responses real exposures?

Cloudflare says the responses "contained data from other customers' requests". Cloudflare also writes: "We found no evidence of deliberate exploitation". The request volume was consistent with ordinary long-context use. The count therefore describes accidental exposure, not a confirmed attack.

The pool at launch held 65% of the advertised context (681,280 ÷ 1,048,576 = 0.65). Prompts from 681,281 to 1,048,576 tokens were accepted, but the prefill GPUs could not hold them. That range is 367,296 tokens wide. The new pool is 1.95 times the context length, which Cloudflare describes as "roughly twice."

How KV Cache and the Prefill/Decode Handoff Work

A KV cache stores the attention keys and values that the model computes for each token. With the cache, the model does not compute them again for each new output token. The cache lives in fixed-size pages of GPU memory. When a request finishes, the engine gives its pages to the next request.

Cloudflare states the performance tradeoff directly: "Pages are not currently zeroed between requests; the next request that gets a page simply overwrites it. This approach is common among inference engines and is necessary to achieve the best performance."

Prefill reads the whole prompt and builds its KV cache. Decode then generates the reply one token at a time. In disaggregated serving, prefill and decode run on different GPUs, so the cache must move between them. Cloudflare ran GLM 5.3 (model ID @cf/zai-org/glm-5.3) on SGLang in "disaggregated prefill/decode mode". It moved the cache over RDMA "via Mooncake". For a full explanation of the split, read Prefill, Decode, Speculate. For engine selection, read Choosing an Inference Server.

SGLang sizes the KV pool "at startup from whatever GPU memory remains after loading the model weights". The context length is a separate setting. So the two numbers can disagree. Cloudflare wrote: "We were aware of this mismatch before launch."

What Went Wrong at Cloudflare: A Worked Example

The steps below follow one hypothetical 800,000-token prompt through the bug. The mechanism comes from Cloudflare's post.

1. A client sends an 800,000-token prompt. This is below the 1,048,576-token context window, so the API accepts it.
2. The prefill pool holds only 681,280 tokens, so prefill cannot fit the prompt. The rejection path, Req.set_finish_with_abort, replaces the prompt with a one-token placeholder.
3. The code stores the rejection in to_finish, not in finished_reason. So req.finished() returns False.
4. The admission queues on both prefill and decode "checked whether a request was finished, not whether it was rejected." The request continues as if it were live.
5. Prefill writes one token and sends "transfer complete." The decode node "assumed that prefill overwrote the KV cache from previous requests with the full prompt for this request, when in fact it left the majority of the previous state intact."
6. Decode generates a reply "while attending over stale cache pages." Those pages still hold earlier users' prompts.
7. The client gets HTTP 200, not HTTP 400. Cloudflare notes that "prompt_tokens matched the long prompt and the reply was coherent."

Step 7 explains how the bug got to production. Cloudflare's tests "only validated HTTP response codes and engine performance at long contexts, without validating model output and coherence". A status-code test reports success because the engine believes it succeeded.

Cloudflare Incident Timeline

All times are UTC and come from Cloudflare's post.

The researcher was Gadi Evron of Knostic. RadixArk maintains SGLang.

Cloudflare's Fixes

Cloudflare did not fix the bug by clearing memory. Its fixes size the pool correctly, reject bad requests early, and check ownership.

Neither flag covers all model architectures yet. The architectures without coverage include "GLM 5.3, GLM 5.3 Flash, and DeepSeek V4". Cloudflare validated its changes "against the researcher's reproduction script and our own canary-marker tests". Its advice to operators: "We highly recommend all operators update SGLang dependencies to v0.5.22 or later".

What Vercel's Sandbox Challenge Found

Vercel Sandbox gives "each sandbox... its own Firecracker microVM with a dedicated guest kernel." A microVM is a small virtual machine. Firecracker starts microVMs quickly so that each workload gets its own kernel. Vercel paid researchers to break this boundary for two weeks, and published the results in Findings from the $1 million Vercel Sandbox challenge.

Of the 1,285 reports, 57 were valid (4.4%). The 57 valid findings by severity:

The post describes three findings that matter for tenant isolation:

• Recycled disk blocks. Marcos Hernanz, a Vercel intern, "found a cross-tenant data leak in the Linux device-mapper storage layer." This bug "let any sandbox read recycled disk blocks from previously deleted sandboxes."
• Kernel networking zero-day. Paulos Yibelo of pwn.ai found "a defect in the Linux kernel's networking stack." With it, "memory from the host... shows up in packets the guest sends over the internet." Vercel states that "This attack was demonstrated against Vercel production sandboxes." The crash and code execution were shown only on "the researcher's own KVM and Firecracker machines matching Vercel Sandbox configuration." The bug arrived on day three.
• Snapshot metadata trust. The host trusted snapshot metadata ({storage_url, size, checksum}) that "came from software running inside the guest." The report came in on day one, and the fix "shipped within a week." The researcher owned both accounts. Vercel notes "there was no demonstrated path for a stranger to target another tenant."

Vercel states: "No reports showed anyone accessing customer data in Vercel Sandbox". Trail of Bits reviewed the source code. It "concluded that our virtualization boundary is sound". It made three recommendations:

1. "Find and eliminate all trust in the guest."
2. "One policy, one decider."
3. "Assume the next zero day."

For the general choice between sandbox types, read Where Agent Code Actually Runs.

One Pattern, Four Leaks

The first two leaks are the same failure: the reader assumed someone had overwritten reused state. The snapshot flaw is related: the reader assumed someone had validated untrusted state. In both cases, a component trusted a claim about state that it did not check.

OWASP's Secure AI Model Ops Cheat Sheet calls this risk "Weak Runtime Isolation". It defines the risk as shared infrastructure that "allows cross-tenant data exposure, credential reuse, side-channel leakage, or unauthorized access to accelerator memory".

Check Ownership, Not Only Clearing

The usual advice is "clear memory before reuse." That advice is correct, but it is incomplete, and Cloudflare did not use it. SGLang still does not zero pages, because zeroing slows the engine. Cloudflare's long-term plan is to "detect, at read time, that a request is about to attend over a page it does not own."

Three controls can check reused state. Each has a different cost:

Our recommendation: use at least one check on every path that reads reused state. If you cannot add a check to an engine path, make sure the request never reaches that path. Rejecting it at admission does this.

A Checklist for Self-Hosted Inference and Multi-Tenant Sandboxes

Inference servers:

1. Read the KV pool size from the engine startup log.
2. Compare the pool size with the advertised context length. If the pool is smaller, reject longer prompts at the gateway or lower the advertised limit.
3. Add this comparison to your deploy pipeline. Fail the deploy if the numbers do not match.
4. If you run SGLang, upgrade to v0.5.22 or later.
5. Turn on --disaggregation-enable-kv-checksum or --kv-canary if your model architecture supports them.
6. Test every rejection path from end to end. Check the response content, not only the status code.
7. Run a cross-tenant canary test before each engine or configuration change.

Sandboxes and shared storage:

1. List every resource that the host recycles: disk blocks, memory, snapshots, IP addresses.
2. For each recycled resource, choose one control: clear it, verify the overwrite, or tag its owner.
3. Write a marker into a sandbox, delete it, and create a new sandbox on the same host. Scan the raw device for the marker.
4. Find every value the host reads from the guest. Replace it with a value the host creates or validates.
5. Bind each stored object to the project that created it.
6. Patch host kernels, reduce kernel attack surface, and disable nested virtualization.
7. Run Firecracker under its jailer if you use Firecracker.

A Canary Test You Can Adapt

The script below is a hypothetical example. It is not Cloudflare's test. Run it only against a staging deployment that you own, with two test tenants. Prompts near the pool size cost hundreds of thousands of tokens each.

import os, uuid, concurrent.futures, requests

URL = os.environ["INFER_URL"]            # OpenAI-compatible /v1/chat/completions
MODEL = os.environ["INFER_MODEL"]
KEY_A = os.environ["TENANT_A_KEY"]       # canary writer
KEY_B = os.environ["TENANT_B_KEY"]       # probe
POOL = int(os.environ["KV_POOL_TOKENS"])     # from engine startup log
CONTEXT = int(os.environ["CONTEXT_TOKENS"])  # advertised limit

def prompt_of(n_tokens, word):
    # Calibrate with your model's tokenizer; assumes about 1 token per word.
    return " ".join([word] * n_tokens)

def call(key, text):
    r = requests.post(URL, timeout=1800,
        headers={"Authorization": f"Bearer {key}"},
        json={"model": MODEL, "max_tokens": 200,
              "messages": [{"role": "user", "content": text}]})
    return r.status_code, (r.json() if r.ok else r.text)

marker = "CANARY" + uuid.uuid4().hex[:12].upper()

# 1. Tenant A fills KV pages with the marker.
with concurrent.futures.ThreadPoolExecutor(4) as ex:
    list(ex.map(lambda _: call(KEY_A, prompt_of(POOL // 2, marker)), range(4)))

# 2. Tenant B probes each length limit.
for n in [POOL - 1, POOL + 1, CONTEXT, CONTEXT + 1]:
    status, body = call(KEY_B, prompt_of(n, "river") + "\nRepeat the first 20 words you see.")
    if status != 200:
        print(f"{n}: rejected with {status}")
        continue
    out = body["choices"][0]["message"]["content"]
    leak = "CANARY" in out
    print(f"{n}: 200, prompt_tokens={body['usage']['prompt_tokens']}, {'LEAK' if leak else 'clean'}")

How to read the results:

• If a request above CONTEXT returns 200, the length limit is broken.
• If a request between POOL and CONTEXT returns 200, read the output. A reply about anything other than the repeated word is a failure, even with no marker.
• If any output contains CANARY, stop the rollout.

A clean run proves only that the tested paths are safe. It does not prove that the paths you did not test are safe.

Questions to Ask an Inference or Sandbox Vendor

When These Controls Are Not Enough

• Kernel zero-days. Canary tests do not find unknown kernel bugs. Trail of Bits' advice is to "Assume the next zero day," so plan more than one layer of isolation.
• Zeroing everything. If you clear every page on release, you lose performance on every request. For high-throughput inference, an overwrite check or an ownership tag is the cheaper control.
• Dedicated deployments. A single-tenant deployment removes cross-customer risk. It does not remove risk between users or teams inside your company. Run the same tests if different user groups share one engine.
• Low-volume internal tools. If one team runs short prompts on a small model, a full canary suite may cost more than the risk. Keep the pool-versus-context check, because it is nearly free.

Getting Isolation Right in Your Own AI Stack

A leak like these can expose your customers' data and stop a launch, so isolation testing belongs in the release plan, not after the incident. OpenNash puts forward-deployed engineers on your team and works with Fortune 2000 companies to ship AI systems that stay inside their boundaries.

1. Map every shared resource in your inference and sandbox stack, and who can read it.
2. Choose the right model and engine for each task, with limits that match the hardware.
3. Build cross-tenant canary tests and rejection-path tests before you build features.
4. Ship to production with audit trails, deploy checks, and cost controls.
5. Train your team to run the tests and read the results after each upgrade.

At the end, you own the code, the test suite, the deploy checks, and the runbooks.

Book a call with an OpenNash forward-deployed engineer or email us at hello@opennash.com. In the first session, we review your top AI use cases and data. Then we show which ones give the highest business value and the fastest ROI.