Get the latest tech news
AI founders will learn the bitter lesson
Historically, general approaches always win in AI. Founders in AI application space now repeat the mistakes of AI researchers.
From an AI research perspective, the Bitter Lesson deals with clear definitions of “better.” In computer chess, it’s your win rate; in speech recognition, it’s word accuracy. “It seems plausible that the schlep will take longer than the unhobbling, that is, by the time the drop-in remote worker is able to automate a large number of jobs, intermediate models won’t yet have been fully harnessed and integrated” Instead, they constrain it to just analyze financial statements, hardcoding specific metrics and validation rules.2) This always helps in the short term and is personally satisfying to the researcherThe developer finds that this increases reliability.The developer finds that specialization improves accuracy since the model only needs to handle a narrow set of documents and metrics.3) In the long run, it plateaus and even inhibits further progressThe constrained workflow sometimes does not give the correct output when faced with novel situations that weren’t considered in the hardcoded steps.The specialized system can’t handle related tasks like analyzing merger documents or earnings calls, requiring separate specialized systems for each type of analysis.4) Breakthrough progress eventually arrives by an opposing approach based on scaling computationNew model releases enable reliable agents that can figure out the right approach dynamically, backtracking and correcting mistakes as needed.New model releases can understand any business document holistically, extracting relevant information regardless of format or type, making specialized systems unnecessary.For problems with unclear solution paths, products with more autonomy will achieve better performance.
Or read this on Hacker News
