Video-MME

Overview

Video-MME is an AI video-understanding benchmark cited by 李磊 as an example of evaluation that extends the measured capability from very short clips toward videos lasting more than an hour.

Current Profile

Within the bounded episode, Video-MME matters less as a leaderboard than as an example of Benchmark Adoption Design / Benchmark 采用设计: it proposes a consequential new evaluation dimension and is meant to be reproducible, easy to run, and compatible with common evaluation ecosystems.

Key Characteristics

• Targets long-duration video understanding rather than only short-clip performance.
• Is presented as measuring a new or materially extended model capability.
• Illustrates the importance of realistic tasks rather than obviously artificial test prompts.
• Connects scientific contribution to reproducibility, usability, and ecosystem integration.

Evidence

• Capability evidence: AI 时代，PhD 路上的 10 个关键问题｜对谈李磊：港大 PhD、小米 MiMo 研究员 says the benchmark extends video evaluation from roughly six seconds to more than an hour.
• Adoption evidence: AI 时代，PhD 路上的 10 个关键问题｜对谈李磊：港大 PhD、小米 MiMo 研究员 uses it while arguing that benchmarks need one-command execution, reproducibility, and compatibility with mainstream tools.

Qualifications

The current wiki evidence is one guest’s retrospective description. It does not independently verify dataset composition, duration distribution, contamination controls, scoring validity, model rankings, adoption, or whether long-video scores predict real user outcomes.

What Changed

• Added Video-MME as a bounded project profile and benchmark-design example.

Relationships

• 李磊 - interview guest who uses the benchmark as his design example.
• Benchmark Adoption Design / Benchmark 采用设计 - framework describing why a scientifically meaningful benchmark also needs low-friction adoption.
• Agent Evaluation Benchmarks - adjacent evaluation branch with different task and safety concerns.
• AI Research Feedback Compression - evaluation can shorten experiment cycles when results are reproducible and operationally easy to obtain.