AI Verification

AI verification is the broader problem of checking whether an AI-generated answer, hypothesis, tool action, training example, or self-improvement step is correct enough to use. E242｜最快半年AI跑通自进化？与陈天桥首席科学家聊聊硅谷模型必争之地 makes verification the central constraint on Recursive Self-Improvement and Discovery Model work.

The source separates easy-to-check domains from judgment-heavy domains. Code and math can use execution, tests, and formal proof tools, but even code can fail when tests are too broad, too narrow, or written to reward the wrong behavior. For open-ended scientific and research problems, Apodex uses agent teams: one agent or group proposes, another verifies, redundant agents compare answers, and the system learns which information sources deserve trust.

137. 对洪乐潼的4小时访谈：AI for Math、把数学变成Lean、数学天书中的证明、直觉、被创造与被发现的 adds the formal-math version through Hong Letong / 洪乐潼 and Axiom. In this source, Lean Theorem Prover, Mathlib, and Interactive Theorem Proving provide a stronger verifier than ordinary tests because proofs become machine-checkable artifacts. The limitation shifts toward Auto-Formalization and Formal Specification: the system can verify a proof only after the mathematical or software target has been stated precisely.

Key Claims

• Verification errors can compound across recursive self-improvement loops.
• Code and math are attractive early domains because they have stronger external checkers than ordinary prose.
• Tests are not automatically reliable; a model can pass weak tests while still solving the wrong problem.
• Multi-agent review can reduce single-agent drift, but it still needs source-quality judgment and human oversight.
• Reward hacking is a verification failure: the model optimizes the proxy rather than the human need.
• Scientific discovery needs verification and taste together, because a true but trivial result may still be the wrong target.
• Formal proof can give AI systems a stronger correctness signal than prose, but only when the target statement is correctly formalized.
• AI For Math is attractive because mathematics provides a cleaner digital sandbox for verification than many physical science domains.

Connections

• AI Coding Verification — software-specific verification branch already tracked in the wiki.
• Multi-Agent Collaboration — agent-team checking pattern used in the source.
• Recursive Self-Improvement and Discovery Model — high-stakes loops that depend on verification.
• Research Taste, Human Judgment Under AI, and Domain Expert Alignment — human standards that keep verification grounded.
• AI For Math, Axiom Prover, Auto-Formalization, and Formal Specification — formal-math verifier branch added by episode 137.