nemo-mbridge-perf-expert-parallel-overlap

This NVIDIA skill is an operational guide for validating and using Mixture-of-Experts (MoE) expert-parallel (EP) communication overlap in Megatron-Bridge. It addresses a specific performance-tuning problem: the token dispatch/combine all-to-all communication in EP training can dominate step time, and this skill explains how to hide that cost by overlapping it with expert feed-forward compute. It centers on the overlap_moe_expert_parallel_comm and delay_wgrad_compute config knobs and the flex dispatcher backends DeepEP and HybridEP.

The guide provides a quick decision framework (when to enable EP overlap, when to prefer the standard alltoall dispatcher versus flex, and when to avoid overlap entirely, such as with full activation recompute or PyTorch below 2.6.0). It documents a correctness-first isolation benchmark measured on Qwen3 MoE 30B-A3B SFT across 16 H100 GPUs, including exact override flags and steady-window timings showing EP overlap producing roughly a 1.3x speedup on that inter-node all-to-all shape, while noting delayed wgrad did not show an independent win there. It gives concrete Python enablement snippets for both the alltoall and flex dispatcher paths, plus compatibility constraints on expert model parallel size, expert count, and dispatcher type. It is Apache-2.0 licensed and shipped with published evaluation results.

It targets ML engineers and performance specialists training large MoE models who are tuning for throughput or tracing a throughput regression back to an EP overlap config change. Use cases include enabling EP overlap on a communication-bound run, choosing a dispatcher backend for a given GPU generation, and diagnosing why a config change slowed training. It produces configuration guidance, shell/Python snippets, and analysis; it does not itself perform destructive or privileged operations.