forward_dense

Function forward_dense 

Source
pub fn forward_dense(
    gpu: &dyn GpuBackend,
    k: &Glm5NextMlpKernels,
    cfg: &Glm5NextMlpConfig,
    w: &Glm5NextDenseMlpWeights,
    inter: usize,
    x: DevicePtr,
    out: DevicePtr,
    m: usize,
    ws: &Glm5NextMlpWorkspace,
    stream: u64,
) -> Result<()>
Expand description

A BF16 SwiGLU MLP of width inter: down(clamped_swiglu(gate(x), up(x))).

Used for both the dense layers and the shared expert — identical math, different widths. With tp_world_size > 1 the result is a partial sum; the caller reduces.