pub const CANONICAL_KEY_MIN_WIDTH: usize = 8;Expand description
Batch WIDTH (sequences) at or above which the canonical depth→slot
assignment is applied. Below it verify_batch_order /
verify_batch_permutation take their canonical = false arm, which is
the pre-canonical (pre-PR-#552) behaviour byte for byte: each sequence
keeps its own confidence-chosen depth and the batch sorts deepest-first,
ssm-slot second, ties on input index (a stable
sort_by_key(|(a, k)| (Reverse(k), slot)), exactly what both call sites
used before).
Default 8, from a same-binary same-session A/B on dgx2 (ladder-38
round 7, tip e0b845f11) with ONE variable — the kill switch
ATLAS_NO_CANONICAL_VERIFY_KEY=1. tok/s, higher is better:
C | canonical ON | canonical OFF | verdict
----+-------------------------+------------------------+------------------
2 | 30.09 | 30.83 | costs -2.4%
4 | 64.00 (round 7: 65.56) | 68.11 (round 6, no it) | costs ~-3.7%
8 | 110.63 | 106.48 | GAINS +3.9%
16 | 203.50 | 203.44 | no effect
32 | 291.50 | 291.52 | no effect
64 | 387.62 | 386.99 | no effect
128 | 477.55 | 477.69 | no effectThe shape of that table follows the key counts (module docs): the
collapse pays exactly where the arrangement space is large. Above width 8
the gate is inert in either direction — D-Cut is off there
(dcut_width_cap), ks is uniform, and both arms reduce to “sort by
slot” (pinned by uniform_depths_are_identical_under_both_arms), which
is why the C>=16 rungs move by <= 0.2% either way.
Hypothesis for the cost below 8, recorded but NOT load-bearing for this
threshold (the threshold is the measurement, not the mechanism): forcing
the assignment makes the two-launch batched GDN conv+WY fast path decline
more often, i.e. n*(2k-1) launches per layer instead of 2 — 768 vs 96
per step at n=2, k=4 over 48 GDN layers. PR #553’s rate telemetry under
ATLAS_MTP_ACCEPT_DEBUG reports that decline rate directly.