Module state

Module state 

Source
Expand description

One GLM-5.3 decoder layer’s per-sequence state.

A GLM layer is one of two mixers and the two need different kinds of state: KDA carries a recurrent hidden state plus a causal-conv window and touches no KV cache at all, while DSA carries an indexer key cache alongside paged KV blocks. Each mixer therefore returns its own concrete LayerState and crate::layers::glm5next_layer::Glm5NextLayer downcasts to the one its mixer expects.

🔴 KDA’s state is the pool’s SsmLayerState, not a GLM-private type. rollback_ssm_states_dispatch walks every LayerType::LinearAttention layer and downcasts to exactly that type to rewind a rejected speculative draft. GLM’s KDA blocks are linear_attention in layer_types, so a GLM-private state means the first rejected draft is a hard error — and the shapes line up byte-for-byte anyway:

pool (config fields, TP-local)GLM (Glm5NextKdaConfig)
hnv · vd · kd · 4heads · head_dim² · 4
conv(nk · kd · 2 + nv · vd) · d_conv · 43 · heads · head_dim · conv_kernel · 4

The parser fills linear_num_{key,value}_heads / linear_{key,value}_head_dim / linear_conv_kernel_dim from linear_attn_config, already divided by TP, so ModelConfig::ssm_h_state_bytes() and ssm_conv_state_bytes() return GLM’s own numbers.

🪤 The two state kinds are NOT interchangeable and admission needs both kinds satisfied — a KDA slot is not a KV block. That is crate::layers::glm5next_skeleton::StateKind, made real.

Functions§

alloc_kda_ssm_state
Allocate and zero a KDA layer’s recurrent + conv state, pool-free.