Skip to content

Scale ZeRO-3 __avg_scatter_grads by sequence_parallel_size - #8854

Open
deep1blue wants to merge 1 commit into
deepspeedai:masterfrom
deep1blue:fix-zero3-sp-avg-scatter-grads
Open

deep1blue wants to merge 1 commit into
deepspeedai:masterfrom
deep1blue:fix-zero3-sp-avg-scatter-grads

Commits

  1. Commits on Oct 18, 2026