Get the latest tech news

Diffusion transformers are the key behind OpenAI’s Sora — and they’re set to upend GenAI

At the heart of Sora and Stable Diffusion 3.0 is a type of AI model architecture called the diffusion transformer.

OpenAI’s Sora, which can generate videos and interactive 3D environments on the fly, is a remarkable demonstration of the cutting edge in GenAI — a bonafide milestone. Most modern AI-powered media generators, including OpenAI’s DALL-E 3, rely on a process called diffusion to output images, videos, speech, music, 3D meshes, artwork and more. The current process of training diffusion transformers potentially introduces some inefficiencies and performance loss, but Xie believes this can be addressed over the long horizon.

Get the Android app