Get the latest tech news
Tiny-diffusion: A minimal implementation of probabilistic diffusion models
A minimal PyTorch implementation of probabilistic diffusion models for 2D datasets. - tanelp/tiny-diffusion
The current model configuration doesn't work well on the line dataset, which I consider the most basic among them. The capacity of the model doesn't seem to be a bottleneck, as similar results are obtained across various hidden layer sizes. The use of sinusoidal embeddings for the inputs helps with learning high-frequency functions in low-dimensional problem domains, such as mapping each (x, y) pixel coordinate to (r, g, b) color, as demonstrated in this study.
Or read this on Hacker News