nemo-mbridge-perf-moe-optimization-workflow

This NVIDIA skill provides a systematic workflow for optimizing Mixture-of-Experts (MoE) training throughput in Megatron Bridge, grounded in the Megatron-Core MoE paper. It solves the sprawling, iterative problem of MoE performance tuning by organizing it around the paper's Three Walls framework (memory wall, communication wall, compute and host-overhead wall) and a repeatable fit-first, scale-second, profile-third, retune loop, so engineers attack bottlenecks in a principled order instead of trial and error.

The workflow is presented as ordered phases with a first-answer checklist. Phase 1 makes a run memory-feasible using the smallest model parallelism that fits, selective recompute before full recompute, offloading only as a last resort, and a fake-init flag to sanity-check large layouts on one GPU. Phase 2 chooses parallelism for scale by maximizing data parallelism, keeping hot communication on the fast interconnect, using pipeline plus virtual pipeline parallelism, preferring expert parallelism over extra tensor parallelism, and adding context parallelism for long sequences, all expressed through the Parallel Folding meshes that decouple attention and MoE parallelism. Phase 3 profiles the dominant bottleneck, and retuning then adjusts dispatcher, overlap, FP8 mode, CUDA graphs, or recompute. It also documents default mappings such as alltoall for bring-up and flex plus deepep or hybridep by platform.

It targets performance engineers and researchers running full MoE tuning sweeps or diagnosing throughput regressions after a commit or config change. The content is analytical guidance referencing configuration flags and profiling, with no destructive or system-modifying operations, and its published benchmark reports a passing evaluation with a 100% security dimension.