Get the latest tech news
What can LLMs never do?
On goal drift and lower reliability. Or, why can't LLMs play Conway's Game Of Life?
These are extraordinary models, capable of answering esoteric questions about economics and quantum mechanics, of helping you code, paint, make music or videos, create entire applications, even play chess at a high level. In some ways this makes sense, because an auto regressive model can only do one forward pass at a time, which means it can at best use it existing token repository and output as a scratch pad to keep thinking out loud, but it loses track so so fast. While doing this Wordle evaluation experiment I read Wolfram again and started thinking about Conway’s Game of Life, and I wondered if we would be able to teach transformers to be able to successfully learn to reproduce the outputs from running these automata for a few generations.
Or read this on Hacker News
