Get the latest tech news
Just add humans: Oxford medical study underscores the missing link in chatbot testing
Patients using chatbots to assess their own medical conditions may end up with worse outcomes than conventional methods, according to a new Oxford study.
It includes important medical details (it’s painful to look down) and red herrings (he’s a regular drinker, shares an apartment with six friends, and just finished some stressful exams). This study is useful, but not surprising, according to Nathalie Volkheimer, a user experience specialist at the Renaissance Computing Institute (RENCI), University of North Carolina at Chapel Hill. This study serves as a critical reminder for AI engineers and orchestration specialists: if an LLM is designed to interact with humans, relying solely on non-interactive benchmarks can create a dangerous false sense of security about its real-world capabilities.
Or read this on Venture Beat