Get the latest tech news
OpenAI admits ChatGPT safeguards fail during extended conversations
ChatGPT allegedly provided suicide encouragement to teen after moderation safeguards failed.
According to the lawsuit, ChatGPT provided detailed instructions, romanticized suicide methods, and discouraged the teen from seeking help from his family while OpenAI's system tracked 377 messages flagged for self-harm content without intervening. OpenAI eased these content safeguards in February following user complaints about overly restrictive ChatGPT moderation that prevented the discussion of topics like sex and violence in some contexts. The company wants to build "a network of licensed professionals people could reach directly through ChatGPT," potentially furthering the idea that an AI system should be mediating mental health crises.
Or read this on ArsTechnica
