Meta-Model Training Curve Prediction

Meta-model training curve prediction is [[LiuZiming|Liu Ziming]]’s proposed “next curve” task in 149. 亲历中美 New Labs 资本狂潮，和清华刘子鸣聊：AI for AI、机制可解释性和 Max Tegmark. The input is a candidate model plus conditions such as dataset and optimizer; the output is a predicted training curve. Liu contrasts this with next-token prediction in language models and next-state prediction in World Models.

The source says the idea came partly from Liu training himself: he repeatedly predicted training curves before running experiments and improved over dozens of days. The model version would scale that judgment by training on many diverse small models and their curves, then using the learned predictor to rank architecture ideas before spending real compute.

Key Claims

• The task is to predict how a model will train, not to generate code or text directly.
• If accurate enough, the meta-model can cheaply triage many architecture hypotheses.
• The data requirement is many model/curve pairs, which may favor diverse smaller experiments over a single giant cluster run.
• Curve prediction is part of “smarter” AI For AI because it selects fewer better experiments rather than only running many trials.
• The concept depends on Physics Of AI because predictions need stable structure across architectures, data, and optimization conditions.

Connections

• [[LiuZiming|Liu Ziming]] — source proposer.
• AI For AI, Auto Research, and OPHIS Research Workflow — research-automation context.
• Physics Of AI, Research Taste, and Training Compute Allocation — theory, judgment, and compute-triage context.
• Training Autopilot and Vibe Training — product horizons that could use meta-model predictions.
• World Models — contrast with next-state prediction.