Get the latest tech news

AuraFlow v0.1: a open source alternative to Stable Diffusion 3

Open-source AI is in jeopardy. As community interest in AI models skyrocketed over the past year, we noticed that development of new open-source foundational models came to a halt. Some even boldly announced that open-source AI is dead. Not so fast! We are excited to present you the first release

Fortunately, we noticed MMDiT architectures were also zero-shot LR transferred with maximal-update-parameterization was utilized.Compared to SP, muP was clearly the winner in terms of predictability of learning rate at scale. Utilizing the findings from above, we were able to train a text-to-image model from scratch in our largest possible settings for 4 week of compute time, including 256x256, 512x512, 1024x1024 pre-training and aspect ratio fine-tuning. Some of this expertise was directly transferable from production grade inference/fine-tuning systems, where we were able to use open source projects like JuiceFS and some were more novel challenges like how do you stream massive amounts of data in and out of multiple nodes while leveraging local NVME space as a staging ground to not to reduce the MFU.

Get the Android app