dequant_q2_0_gn_to_bf16

Function dequant_q2_0_gn_to_bf16 

Source
pub fn dequant_q2_0_gn_to_bf16(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    blocks: DevicePtr,
    out: DevicePtr,
    n: u32,
    k: u32,
    group: u32,
    stream: u64,
) -> Result<()>
Expand description

Dequant a packed Q2_0 weight [N, K] (contiguous block_q2_0 blocks) into a pre-allocated BF16 scratch buffer [N, K] on stream, IN PLACE (no alloc, no host sync). Reuses the load-time dequant_q2_0_gn_to_bf16 kernel (dequant_gguf_bf16 module). Used by packed-Q2 PREFILL: dequant → transient BF16 → normal BF16 GEMM → free scratch (the resident weight stays 2-bit).

n_blocks = n * (k / group); each block is 2 + group/4 bytes and expands to group BF16 elements. Kernel: grid (n_blocks,1,1) block (256,1,1).