nvfp4_mmq_gemm

spark_model::layers::ops

Function nvfp4_mmq_gemm 

Source 
pub fn nvfp4_mmq_gemm(
    gpu: &dyn GpuBackend,
    kernel_nc: KernelHandle,
    kernel_wc: KernelHandle,
    a_fp4: DevicePtr,
    w_nvfp4: DevicePtr,
    out_bf16: DevicePtr,
    m: u32,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

NVFP4 W4A4 MMQ GEMM: C[m,n] (bf16, missing ×scale2) = A_fp4[m,k] x W_nvfp4[n,k].