github.com2025-02-24authorityHN front pageabout the hopper familyHN 441 pts · 108 comments2,705 words
FlashMLA is DeepSeek's library of optimized attention kernels powering inference of the DeepSeek-V4.1 model on NVIDIA GPUs and Huawei Ascend NPUs. The 2026.09.30 release removes support for the Hopper architecture and earlier models including DeepSeek V3 / V3.2 / V4.0 while changing the FP8 / FP4 KV cache format. On the Huawei Ascend 950 NPU, sparse attention kernels achieve up to 410 TFlops during prefill and 360 TFlops during decoding, while on B200 with CUDA 13.3, sparse decoding reaches up to 1024 TFlops and fused kernels hit up to 1460 TFlops during prefill.written, reviewed, cited
- The fused norm-RoPE-attn-RoPE-cast kernel requires pre-permuting Q_b and Wv weights to achieve higher TFlops.
- CUDA 13.1 decoding kernels fall back to slower widening through FP32 instead of using native bf16 conversions.