Get the latest tech news
Answering Legal Questions with LLMs
If you asked a lawyer whether ChatGPT could do his job, he would laugh you out of the room. The tool is often useful, but can’t handle legal questions end to end. It makes up sources, its reasoning can be flawed, and it may overlook key aspects of the law. We decided to tackle this problem at Hotseat. After iterating multiple times, implementing advanced agentic workflows, and testing with dozens of lawyer, we’re certain that the product has glaring limitations and it won’t replace any jobs soon.
After iterating multiple times, implementing advanced agentic workflows, and testing with dozens of lawyer, we’re certain that the product has glaring limitations and it won’t replace any jobs soon. However, we figured out an unintuitive, LLM-only method of doing RAG, and we leveraged it to make LLMs answer complex questions about select EU regulations. No lawyer wants to expend effort to ask a detailed question, wait 10 minutes for an answer, wade through a 2-pages-long response, and find that the AI made an error.
Or read this on Hacker News