cublas_bf16_proj

Function cublas_bf16_proj 

Source
pub fn cublas_bf16_proj(
    gpu: &dyn GpuBackend,
    derived: &DerivedWeights,
    act: DevicePtr,
    fp8w: &Fp8Weight,
    out: DevicePtr,
    m: u32,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

Route a projection out[M,N] = act[M,K] @ weightᵀ through cuBLASLt BF16. The FP8 weight is dequantized to BF16 once (cached); W16A16 here is strictly more accurate than the blockscaled W8A8 path it replaces.