Get the latest tech news
When ChatGPT broke the field of NLP: An oral history
Researchers in “natural language processing” tried to tame human language. Then came the transformer.
To put it another way: In 2019, Quanta reported on a then-groundbreaking NLP system called BERT without once using the phrase “large language model.” A mere five and a half years later, LLMs are everywhere, igniting discovery, disruption and debate in whatever scientific community they touch. Soon after it was introduced in October 2018, Google’s open-source transformer BERT (and a lesser-known model from OpenAI named GPT) began shattering the performance records set by previous neural networks on many language-processing tasks. In an attempt to repair it, the NLP research community surveyed itself in the summer of 2022 on “30 potentially controversial positions” — including “Linguistic structure is necessary,” “Scaling solves practically any important problem” and “AI could soon lead to revolutionary societal change.”
Or read this on Hacker News
