Get the latest tech news
Continuous Thought Machines
Introducing Continuous Thought Machines: a new kind of neural network model that unfolds and uses neural dynamics as a powerful representation for thought.
Unlike conventional sequential models -- such as RNNs or Transformers -- that process inputs step-by-step according to the sequence inherent in the data (e.g., words in a sentence or frames in a video), the CTM operates along a self-generated timeline of internal thought steps. The CTM can dynamically build representations over time for tasks like image classification, form rich internal maps to attend to specific input data without positional embeddings, and naturally exhibit adaptive computation. It is crucial to note that our approach advocates for borrowing concepts from biology rather than insisting on strict, literal plausibility; real neurons may not access their activation history as modeled in the CTM, yet emergent phenomena like traveling waves still manifest.
Or read this on Hacker News