layer_norm

Function layer_norm 

Source
pub fn layer_norm(
    x: &[f32],
    w: &[f32],
    b: &[f32],
    d: usize,
    eps: f32,
) -> Vec<f32>
Expand description

nn.LayerNorm over the trailing d: mean-subtract, variance-normalise, then w * x + b.

🪤 Not an RMSNorm. The mean subtraction and the bias are both real, and both are invisible from tensor shapes — indexer.k_norm.bias existing in the checkpoint is the only tell.