Get the latest tech news
Will Smith eating spaghetti and other weird AI benchmarks that took off in 2024
Weird AI benchmarks like Will Smith eating spaghetti, Pictionary, and Minecraft blew up in 2024. But why, exactly?
Image Credits: LMSYSEthan Mollick, a professor of management at Wharton, recently pointed out in a post on X another problem with many AI industry benchmarks: they don’t compare a system’s performance to that of the average person. “The fact that there are not 30 different benchmarks from different organizations in medicine, in law, in advice quality, and so on is a real shame, as people are using systems for these things, regardless,” Mollick wrote. And as my colleague Max Zeff wrote about recently, the industry continues to grapple with distilling a technology as complex as AI into digestible marketing.
Or read this on TechCrunch
