Boosting MoE Training Throughput with Advanced Fusion Kernels
Mixture-of-experts (MoE) models enable larger AI model capacity by activating only a subset of parameters per token. NVIDIA explores advanced fusion kernels to improve MoE training throughput for large-scale AI systems.