Get the latest tech news

Computing Inside an AI

Since the launch of ChatGPT, explorations in two directions kicked into high gear. The first direction is about technical capabilities.

I propose a new mode of interaction, where models play the role of computer (e.g. phone) applications: providing a graphical interface, interpreting user inputs, and updating their state. When we want precision, we turn to tools instead, using direct manipulation and high-bandwidth visual interfaces to make diagrams and write code and design CAD models. In place of the charmingly teletype linear stream of text provided by ChatGPT, a model-as-computer system will generate something which resembles the interface of a modern application: buttons, sliders, tabs, images, plots, and all the rest.

Get the Android app