Get the latest tech news
ARC-AGI without pretraining
By Isaac Liao and Albert Gu In this blog post, we aim to answer a simple yet fundamental question: Can lossless information compression by itself produce intelligent behavior? The idea that efficient compression by itself lies at the heart of intelligence is not new (see, e.g., Hernández-Orallo & Minaya-Collado, 1998; Mahoney, 1999; Hutter, 2005; Legg & Hutter, 2007). Rather than revisiting those theoretical discussions, we make a practical demonstration instead.
We challenge the conventional reliance on extensive pretraining and data, and propose a future where tailored compressive objectives and efficient inference-time computation work together to extract deep intelligence from minimal input. Likewise, if we port the rest of what we described above (plus modifications regarding equivariances and inter-puzzle independence, and ignoring regularization) into typical machine learning lingo, we get the above description of CompressARC. Top methods rely heavily on data augmentation and larger alternative datasets, and sometimes perform autoregressive training on the target puzzle during inference time.
Or read this on Hacker News
