pub fn load_fp8_block_scaled_as_fp8weight(
store: &WeightStore,
prefix: &str,
gpu: &dyn GpuBackend,
) -> Result<Fp8Weight>Expand description
Load an FP8 E4M3 block-scaled checkpoint weight as a native Fp8Weight.
The FP8 checkpoint stores:
{prefix}.weight: FP8E4M3 tensor [N, K]{prefix}.weight_scale_inv: BF16 (Qwen/DeepSeek) or FP32 (MiniMax) tensor [N/block, K/block]
The w8a16_gemv kernel uses 2D block scales directly:
dequant[i,j] = E4M3_LUT[fp8[i,j]] * block_scale[i/BS, j/BS]
No per-row max reduction needed — the kernel loads the correct block
scale for each 128-element K chunk.
Scale precision (block-FP8 numerics): the block scale is widened to a
genuine FP32 device buffer here, once, so it is applied in full FP32 in the
W8A8/W8A16 GEMM epilogues — matching vLLM / DeepGEMM / HF block-FP8 (which
also accumulate the scale in FP32). The checkpoint may store the scale as
BF16 (lossless widen), FP32 (straight copy), or F8_E8M0 (exact power-of-two
widen); in every case row_scale ends up an FP32 [N/BS, K/BS] buffer.
Every FP8 block-scale kernel reads
const float* — see kernels/gb10/common/w8a16_gemv.cu et al.