Get the latest tech news

You Exist in the Long Context

Thoughts on the quiet revolution of long-context AI models, from NotebookLM's Editorial Director Steven Johnson.

This wasn’t commented on enough at the time in the popular press, but in a very real sense the explosion of interest in AI after the launch of ChatGPT in December of 2022 was more a product of the context window expanding than it was some advance in the model’s “general” intelligence. In early 2024, when I first got access to an initial Gemini million-token context model, one of the first tests I ran was uploading the full text of The Infernal Machine, which at that point had not yet been published, and asking questions about the plot of the book. That is, in fact, how I got to the story of Henry Molaison that I began with; I was mulling over the themes of short- and long-term memory in the context of AI, and asked the Everything notebook if it had anything to contribute, and the model reminded me of the tragic tale of patient H. M. that I had first read about in the 1990s.

Get the Android app