Get the latest tech news
World Emulation via Neural Network
I turned a forest trail near my apartment into a playable neural world. You can explore that world in your web browser by clicking right here: By "neural world", I mean that the entire thing is a neural network generating new images based on previous images + controls.
Dataset: 22,814 frames (30FPS SDR video, timestamped ARKit poses) captured at Marymoor Park Audobon Bird Loop with iPhone 13 Pro. Inference: Control-conditioned sequential autoregression with 60FPS cap, preprocessing in JS, network in ONNX Runtime Web's WebGL backend. Compared to serious "Foundation World Models" with billions of parameters, the GAN-based WM featured in this post is a toy (and a fairly brittle one at that).
Or read this on Hacker News