PagedAttention

PagedAttention is the inference-memory idea that 148. 对游凯超3小时访谈：开源Infra、和模型Co-design 、“如果vLLM失败，我们会后悔一辈子” identifies as the technical origin of [[VLLM|vLLM]]. [[YuKaichao|游凯超]] says the idea itself was not necessarily complicated in hindsight, but it arrived early, was experimentally solid, and became valuable because it was turned into a usable open-source inference engine.

The source’s durable lesson is that an inference optimization becomes important only when it survives contact with production. PagedAttention matters less as an isolated paper result than as the first proof that attention state, memory layout, request scheduling, and user-facing serving needs belong in one systems problem.

Key Claims

• PagedAttention helped make vLLM possible by treating attention-state management as an inference-system problem.
• Its impact came from implementation timing, experiments, and open-source packaging, not only from algorithmic novelty.
• It links memory management to AI Inference Cost Structure because inefficient state handling raises serving cost and lowers throughput.
• It is an early example of Model-Infra Co-Design because model attention behavior and inference runtime behavior cannot be fully separated.

Connections

• [[VLLM|vLLM]] — project that grew from the PagedAttention work.
• Continuous Batching, High-Throughput Inference Batching, and Prefix Caching — adjacent serving-efficiency concepts.
• AI Inference Cost Structure, Inference Acceleration Stack, and Model-Infra Co-Design — broader system context.