Long-Horizon AI

Long-horizon AI is Yao Shunyu / 姚顺宇’s frame in 140. 对姚顺宇的4小时访谈：请允许我小疯一下！在Anthropic和Gemini训模型、技术预测、英雄主义已过去 for models that can complete longer task chains than ordinary context windows and benchmark episodes expose. His shorthand is “train with finite context, use as infinite context”: train models under bounded context, but deploy them in interactive systems where they can gather information, retrieve, summarize, forget, and keep working over a much longer horizon.

138. 对罗福莉3.5小时访谈：AI范式已然巨变！OpenClaw、Agent范式很吃后训练、卡的分配、组织平权 adds an architecture and agent-data version through Luo Fuli / 罗福莉 and Memo VR. The source treats code, long context, cross-session memory, plan compression, and agent framework traces as training signals, while Agent-Optimized Model Architecture tries to make long-context behavior fast and cheap enough for real agent work.

Episode 17: 向量模型工程师：AI 的隐藏瓶颈与新时代的信息迷宫 adds the retrieval-boundary version. N 同学 / N Student argues that longer context helps some single-document reading tasks, but it does not replace Retrieval-Augmented Generation over very large document collections, and long conversations can still suffer Context Decay when earlier information becomes vague or stale.

Key Claims

• Long-horizon capability is not only a larger context-window number; it is the ability to decide what information remains relevant during extended work.
• Context Engineering becomes part of model training and post-training, not only a user habit, because the model must learn when to compress, retrieve, ignore, or ask for new context.
• Yao treats continual learning and long-horizon work as closely related: a model’s active context and KV cache can be viewed as a temporary form of weight-like state.
• The technical routes include pretraining-side changes such as sparse attention and post-training-side changes such as context management; Yao says his own focus is more on post-training.
• The concept is especially important for agentic work because tasks such as research, software projects, and personal assistance fail when models lose earlier decisions or optimize only the next short step.
• Evaluation must be scientific and comparative; a long-horizon technique matters only if it improves real extended tasks rather than merely making prompts longer.
• Long-horizon ability can be trained indirectly through code and agent workflows because both require dependencies, recovery, planning, and verification over many steps.
• Long context is not a universal substitute for retrieval; large libraries still need Vector Model Engineering, chunking, and ranking.

Connections

• Yao Shunyu / 姚顺宇 — source speaker and practitioner.
• ML Coding — current work area where long-horizon ability matters for experiments, code, analysis, and hypothesis loops.
• Context Engineering, Persistent Agent Memory, and Agent Harness — adjacent ways of handling state, memory, and environment outside raw model weights.
• AI Coding Verification and Agentic Workflow — software-agent cases where extended task chains need review and regression control.
• World Models and Proactive Agents — related directions where models must carry state across actions rather than answer isolated prompts.
• Luo Fuli / 罗福莉, Memo VR, Agent-Optimized Model Architecture, and Agent Post-Training — long-context architecture and agent-data branch added by episode 138.
• Retrieval-Augmented Generation, Context Decay, and Vector Model Engineering — retrieval-boundary branch added by the Fuyou Tiandi vector-model episode.