Radix Attention

Radix Attention is the SGLang mechanism [[ShengYing|盛颖]] explains in E247｜对话盛颖：xAI，Infra的浪漫，SGLang，开源，平权与“甄嬛传”. It uses a radix-tree structure to manage shared prompt prefixes and reuse already computed KV cache, reducing repeated computation in workloads with overlapping context.

The episode frames the mechanism as especially relevant to multi-turn dialogue and agents. In those settings, many requests share instructions, history, tool context, or retrieved material, so Prefix Caching becomes a core part of Agent Inference Workload rather than a small serving optimization.

Key Claims

• Shared prefixes should be treated as cacheable structure, not as unrelated text repeated request by request.
• Prefix reuse matters more when agent or dialogue workloads repeatedly expand from the same context.
• Serving engines need data structures that match model and workflow shape, which links Radix Attention to Model-Infra Co-Design.
• Architecture churn can still force new adaptation work, so Radix Attention sits alongside Day-Zero Model Support rather than replacing it.

Connections

• SGLang and [[ShengYing|盛颖 / Sheng Ying]] - source mechanism and explainer.
• Prefix Caching, Agent Inference Workload, and Inference Acceleration Stack - serving optimization context.
• Model-Infra Co-Design, AI Inference Cost Structure, and AI Infrastructure As Product - broader infrastructure economics and product frame.
• [[RadixARC|Redix ARK]] - company context around the SGLang work.