Ethan Mollick warns that complex AI benchmarks are losing crucial human baseline comparisons
English summary
Ethan Mollick highlights that as frontier AI benchmarks grow more complex, they increasingly lack human baseline comparisons, which are vital for validated evaluation. He emphasizes that proper benchmarks should include baselines from multiple humans, though this is becoming harder and more expensive. Without these comparisons, the ability to meaningfully measure AI performance against human capability is diminished.
Chinese summary
伊桑·莫里克指出，随着前沿AI基准测试日益复杂，越来越多测试缺失了人类基线对比，而这对于经验证的评估至关重要。他强调，合格的基准应包含多个人类测试者的基线结果，尽管这在当下已变得愈加困难和昂贵。缺少此类对比，将削弱衡量AI性能相对于人类能力的意义。
Key points
Frontier AI benchmarks are growing more complex and losing human baseline comparisons.
前沿AI基准测试日趋复杂，并丧失人类基线对比。
Validated benchmarks require human baselines, ideally from multiple individuals.
经验证的基准需要人类基线，理想情况下来自多个个体。
Obtaining human baselines is increasingly difficult and costly but remains essential.
获取人类基线正变得更加困难和昂贵，但仍然至关重要。