Get the latest tech news
Is chain-of-thought AI reasoning a mirage?
Reading research papers and articles about chain-of-thought reasoning makes me frustrated. There are many interesting questions to ask about chain-of-thought…
There are many interesting questions to ask about chain-of-thought: how accurately it reflects the actual process going on, why training it “from scratch” often produces chains that switch fluidly between multiple languages, and so on. You couldn’t have built a reasoning model on top of GPT-3.5 - there’s just not enough raw brainpower there to perform the relevant operations, like holding multiple possible solutions “in memory” at the same time. LLMs construct superficial chains of logic based on learned token associations, often failing on tasks that deviate from commonsense heuristics or familiar templates Models often incorporate … irrelevant details into their reasoning, revealing a lack of sensitivity to salient information models may overthink easy problems and give up on harder ones Together, these findings suggest that LLMs are not principled reasoners but rather sophisticated simulators of reasoning-like text
Or read this on Hacker News
