Get the latest tech news
LLMs struggle to explain themselves
can sometimes recognize number patterns, but can they explain their reasoning? See for yourself! The interactive demo below generates a random program and uses that to compute three number sequences. The LLM is given two of those as examples and asked to pick the third out of a lineup.
You can change a lot through the settings: not just the prompts, but also the HTTP requests the demo makes (by default it’s configured to use Claude 3.5 Sonnet by Anthropic). The values on the stack are actually rational numbers $p/q$, but I figured this would make things too complicated for the LLM, so the problems that I generate only involve integers. Lastly, I’m writing this blog post on the second anniversary of the unexpected death of my dad in the year he was planning to retire.
Or read this on Hacker News
