[WIP] Feat: Context Parallel - #181

Open

Chamberlain0w0 wants to merge 7 commits into

feat/context_parallel

Commits on Jul 3, 2026

Commits on Jul 6, 2026

fix: fix nan and test case
Chamberlain0w0
committed

Commits on Jul 7, 2026

fix: fix precision and p2p hang under bf16 autocast cases
Chamberlain0w0
committed

Commits on Jul 10, 2026

fix: support compute-comm overlap in cp attn
Chamberlain0w0
committed

Commits on Jul 16, 2026

feat: support CP compute-compute overlap
Chamberlain0w0
committed