H200 MoE Grouped GEMM profiling and tuning

Mixing T × top-k expanded rows with DeepEP compact rows overstated work by roughly 4–6×. The corrected compact-row count changed both the benchmark distribution and the optimization target.

Steady-state forward ranges were collected, filtered, and replayed locally. This preserved representative shapes while making kernel configuration, warmup, and repeated A/B measurement controllable.

Candidate SM90 configurations varied tile shape, cluster shape, swizzle, and expert distribution. A selected target shape improved by about 5.3%, while the full shape/distribution matrix improved by about 1.84% in aggregate.

Pure GEMM MFU and local latency do not equal end-to-end MoE or model throughput. Dispatch, communication, imbalance, launch overhead, and synchronization remain outside that metric.