Module expert_peer

Structs§

RdmaConfig: RDMA device selection for the verbs (MODE_VERBS) transport. Ignored for TCP clients. One (device, gid_idx) per CX7 adapter: a dual-rail client requests N rails and the peer registers each layer mmap on every rail so the client can stripe expert fetches across both adapters. The default is a SINGLE rail (roceP2p1s0f1, RoCEv2 GID index 3) — the pre-dual-rail behavior, unchanged for verify_verbs.sh; add a second --rail to serve dual-rail clients.
VerbsClientParams: The client’s half: just its QP identity (its arena MR is local-only).
VerbsServerParams: The server’s half of the verbs handshake (RO dialect: expert / weight / LoRA tiers): its QP identity plus, per MoE layer (or per shard), the base virtual address + rkey of that entry’s registered MR. remote_addr(layer, expert) = layers[layer].0 + expert * record_stride.

MODE_TCP: Two-sided TCP record streaming (the expert record path).
MODE_VERBS: One-sided RDMA READ over verbs: the server publishes its store’s MRs and the client READs records directly into its arena.
SHUTDOWN_MARKER: Sentinel request that asks the server to close the connection.
STATUS_ERR
STATUS_OK

decode_request: Parse a request buffer.
encode_request: Serialize a request: (layer, expert).
read_manifest: Read the length-prefixed manifest from a freshly-connected stream and parse it. Shared by the client (expert_tier_rdma).
read_server_rails: Read want per-rail VerbsServerParams framed by a leading [u8 n_rails]. Bails if the framed count is zero, absurd (> 8), or != want — the client already negotiated want rails, so any other count is a protocol error.
serve: Serve records from dir on addr until interrupted. One thread per connection. Blocking; intended to run as its own process (atlas-expert-peer).
write_server_rails: Frame N per-rail VerbsServerParams for the dual-rail RO tiers: a leading [u8 n_rails] count followed by each rail’s params — exactly how the RW blade frames its per-rail CacheServerParams. Single-rail (n == 1) is the default, byte-for-byte the pre-dual-rail path plus the one-byte count prefix.