Expand description
GDN / SSM decode-path flags, resolved ONCE from the serve command line.
These three select KERNELS on the GDN decode path, and they are coupled:
the FP16 h-state twins only exist on the fused-norm arm, so h_f16 without
fused_norm reaches an FP32-only kernel that would read the FP16 pool as
FP32 — plausible numbers, silent garbage. That coupling is checked at serve
time by spark-server’s arg validation, not discovered at the first decode
step.
§Why these are set, not read
They were three independent std::env::var reads scattered across six call
sites, each with its own convention (ATLAS_SSM_H_FP16 presence-gated —
where =0 meant ON — and the other two == "1"). That is how the same
flag came to be decoded two different ways in one binary. They are now ONE
cell, written once from set_from_cli before any model is built.
The environment variables remain honoured when the setter never runs (a test, a microbenchmark example, an older script), so nothing that worked before stops working; the CLI wins when both are present.
Follow-up: this is process-scoped, so a hot-swap to a model with a
different recipe keeps the first model’s kernel selection. The proper home
is ModelLevers, which is carried per model — deferred because the h-state
dtype is read from SsmLayerState construction sites that have no
ForwardContext.
Structs§
Constants§
- MAX_
F16_ TWIN_ DFLASH_ GAMMA - The largest
--dflash-gammawhose verify width still has an FP16 twin. - MAX_
F16_ TWIN_ K - Widest chain-verify K with an FP16 h-state twin
(
gated_delta_rule_wy{5..16}_f16).
Functions§
- default_
dflash_ gamma - The served DFlash gamma for a drafter of this trained block size, when
no
--dflash-gammawas given. - flags
- The resolved flags, falling back to the environment on first touch.
- gdn_
fused_ norm_ enabled --gdn-fused-norm(legacyATLAS_GDN_FUSED_NORM=1).- set_
from_ cli - Publish the command line’s resolution. Call once, before the model builds.
- ssm_
batched_ recurrent_ enabled --ssm-batched-recurrent(legacyATLAS_SSM_BATCHED_RECURRENT=1).- ssm_
h_ dtype_ bits - SSOT decode of
--ssm-h-dtypeinto the two h-state bits it publishes:(h_f16, h_f16_pool). - ssm_
h_ f16_ pool_ enabled - Stage 3 of the f16 h-state: h pools SIZED at 2 bytes/element
(
--ssm-h-dtype f16-pool). Impliesssm_h_fp16_enabled— a narrow pool holding FP32 would be an OOB write, not a mode — whichssm_h_dtype_bitsguarantees at the one place the value is decoded. - ssm_
h_ fp16_ enabled --ssm-h-dtype f16(legacyATLAS_SSM_H_FP16).- verify_
exact_ enabled --exact-verifygiven (and h-state is FP32): the MTP-verify pass runs the sequential-decode-exact chain. FALSE by default — without the flag the verify pass runs the WY/chunkwise arms and #435’s spec-on/spec-off output divergence remains. SeeGdnFlags::verify_exact_active.