Get the latest tech news
Real-time action chunking with large models
Physical Intelligence is bringing general-purpose AI into the physical world.
These results are obtained by attempting to do real-time inference with temporal ensembling to smooth over the discontinuities, a popular prior method introduced in ACT(Zhou et al., 2023). Without a specialized algorithm, the new chunk might be totally incompatible with the robot's current trajectory — the model might be reacting to new information, or it might just be sampling a different “strategy” from its learned distribution of behaviors. A good real-time algorithm should produce a new chunk that is consistent with these overlapping actions while still preserving the model's reactivity to new information and ability to make intelligent decisions.
Or read this on Hacker News