Function ple_add_highway 

pub fn ple_add_highway(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    ple_out: DevicePtr,
    hidden: DevicePtr,
    n: u32,
    stream: u64,
) -> Result<()>

highway += ple_out, in FP32. The reference adds PLE’s output to the residual before that layer’s attention hyper-connection.