Cloudflare KV Cache Leak: Questions for Your AI Host

Cloudflare has disclosed that 155 responses from its hosted GLM-5.3 model contained data from other customers' requests. The affected window ran from 27 August to 9 September 2026, and every case involved a prompt longer than 681,280 tokens (Cloudflare). Shared inference is multi-tenant infrastructure, and multi-tenant infrastructure has isolation bugs.

TL;DR

• Cloudflare says 155 responses on the GLM-5.3 Workers AI model contained other customers' data.
• The cause was a context window larger than the KV cache pool, plus a bug in SGLang's disaggregated prefill and decode mode.
• Shared inference is multi-tenant infrastructure; isolation failures are a vendor risk you cannot patch yourself.
• Cut secrets and personal data from prompts, cap prompt length and ask your provider precise questions.

Disclosure: ZeroRaised publishes this analysis and is a launch site for bootstrapped products. Facts come from the linked sources as checked on 28 Oct 2026; ZeroRaised has no commercial relationship with the companies named.

A KV cache is the model's short-term working memory of the prompt it is processing. On a shared GPU it is also memory that must be isolated between customers.

What exactly happened at Cloudflare?#

Between 27 August and 9 September 2026, a small number of customers using the GLM-5.3 Workers AI model (@cf/zai-org/glm-5.3) received responses containing data from other customers' requests made to the same model shortly before theirs (Cloudflare). Cloudflare counts 155 affected responses, all to prompts longer than 681,280 tokens (Cloudflare).

Security researcher Gadi Evron of Knostic reported the issue on 8 September through Cloudflare's HackerOne programme. Cloudflare reproduced it the next morning, took the model offline within 40 minutes of declaring an incident and restored it on 9 September (Cloudflare). No other Workers AI model was affected (Cloudflare).

That is a fast, specific disclosure, and credit is due: the token threshold, the timeline and the root cause let customers check whether they were in scope.

Why did the cache return another customer's data?#

The root cause was two overlapping issues, not one bad line of code. Cloudflare deployed GLM 5.3 with a 1,048,576-token context window, but the KV cache pool on the prefill GPUs had room for only 681,280 tokens (Cloudflare). Cloudflare knew about the mismatch before launch and expected the engine to reject over-length requests with an error (Cloudflare).

Instead, a bug in SGLang's disaggregated prefill and decode mode let the rejected over-length requests proceed. The decode node then generated a response while attending over stale cache pages from earlier requests (Cloudflare, SGLang and LMSYS). The check existed; the pipeline ignored it.

Cloudflare found no evidence of deliberate exploitation. It resized the pool to hold the full context and patched the engine (Cloudflare). This is the shape of most infrastructure incidents: a known limit, an assumption that the framework enforces it, and a gap between the two.

Does this make hosted inference too risky?#

Not on its own. Self-hosting an open-weight model moves the risk onto you without removing it: you still need isolation, patching, monitoring and an incident process, and many small teams will struggle to match a large provider on all four.

The honest reading is that this is a low-base-rate tail risk: it needs a very long prompt, a specific model, a small cache pool and a framework bug at once. But tail risks produce the email nobody wants to write.

So treat it as a vendor question, not a vendor verdict. The question is not whether hosted AI is safe, but what you would have known, and when, if your provider had this incident. The guide to choosing an LLM API covers the wider comparison; add isolation and disclosure to that list.

What should change in your prompts?#

Data minimisation is the cheapest control you own. If a field is not needed for the task, do not send it.

• Do not paste secrets. API keys, passwords and connection strings should live in your code, never in model context. Give the model a tool that performs the action instead. The API key rotation guide covers recovery.
• Redact before you send. Send a customer ID or a masked record, not the full row with names, addresses and payment hints.
• Cap the prompt length. A 681,280-token prompt is not a normal product request. Set a hard limit well below your provider's maximum and reject anything larger.
• Trim retrieved context. Retrieval pipelines are the usual source of giant prompts. Rank, filter and cut before the call; the LLM cost guide treats the same habit as a margin lever.

Data minimisation does not fix a cache bug. It limits what a cache bug can hand to the next customer.

What should you ask your AI provider?#

Ask in writing, and keep the answer. Good questions:

1. Is the KV cache isolated per customer, per request or per model instance, and how is that enforced?
2. Does the advertised context window fit inside the cache pool, and what happens when a request exceeds it?
3. What is the incident disclosure commitment, and where is the post-mortem published?
4. Are prompts retained, for how long, and are they used for training?
5. What configuration checks run before a model is promoted to general availability?

A provider that answers these plainly is easier to trust than one that only publishes a status page. If you weigh open weights against hosted APIs, the open-weight versus closed models comparison covers the trade-offs.

What should you tell your own customers?#

If a vendor incident could have touched their data, say so before they read it elsewhere. A short note works: what happened, what you send to the vendor, whether their data was in scope, what you changed, and when you will update them.

Do not claim zero impact unless you can support it, and do not borrow the vendor's transparency as your own. Your customers bought your product, not Cloudflare's. If you have a security page, add the vendor dependency to it; the vibe-coded app security checklist is a reasonable place to start.

What this means for bootstrapped founders#

You cannot audit a GPU cache pool. What you can do is keep a short vendor risk list, send less sensitive data, set length limits, ask questions in writing and keep one fallback model ready.

The commercial angle matters too. Enterprise buyers can ask about AI subprocessors, and "we minimise prompt data, we cap length and we can switch models" is a sales asset, not just hygiene. If hosted inference is core, decide how you would absorb a week of downtime.

Where ZeroRaised fits#

ZeroRaised is a launch and discovery site for products built without institutional venture capital or accelerator equity. If you have a product people can try, submit it here; the pricing page explains the current options. A launch page is also a good place to state how your product handles data and which AI providers it uses.

For more analysis like this, join the ZeroRaised newsletter. No promises of traffic or results, just the useful parts.

Sources#

• Cloudflare: KV cache exposure remediation
• SGLang and LMSYS: Cloudflare security incident analysis