Get the latest tech news
Diffusion Models
Notes on the theory behind models like Stable Diffusion and their applications. I spent 2022 learning to draw and was blindsided by the rise of AI art It's been two years, and image generation with diffusion is better than ever.
Since we have a closed form solution for \(q(\mathbf{x}_{t-1} | \mathbf{x}_t, \mathbf{x}_0)\), if we could use the entire dataset at generation time, we could use the law of total probability to compute \(q(\mathbf{x}_{t-1} | \mathbf{x}_t)\) as a mixture of gaussians, but we can't (billions of images!) Second, it turns out that the forward diffusion process can be described by something called a stochastic differential equation (SDE) which tells us how the data distribution evolves over time as we add noise to it. Subsequent checkpoints of Stable Diffusion 1 are fine-tuned on subsets of LAION-5B selected for “aesthetics” From the LAION-aesthetic readme, as automatically labeled by a linear regression on CLIP trained on 4000 hand-labelled examples.. See this blog post for a look inside.
Or read this on Hacker News