Get the latest tech news
DualPipe: Bidirectional pipeline parallelism algorithm
Contribute to deepseek-ai/DualPipe development by creating an account on GitHub.
DualPipe is an innovative bidirectional pipeline parallism algorithm introduced in the DeepSeek-V3 Technical Report. It achieves full overlap of forward and backward computation-communication phases, also reducing pipeline bubbles. DualPipe was created and developed by Jiashi Li and Chengqi Deng and Wenfeng Liang.
Or read this on Hacker News
