Workspace Layout

Atlas is a nineteen-member Cargo workspace plus a build-time kernel tree (count them in the root Cargo.toml members list). This chapter maps every top-level directory to its role, and the crates to the axes of variation they each insulate.

Repository tree (top level)

atlas/
├── README.md                     headline, benchmarks, porting guides
├── QUICKSTART.md                 per-model Docker recipes
├── CONTRIBUTING.md, AGENTS.md    contributor workflow
├── SECURITY.md                   disclosure
├── CLA.md                        contributor license agreement
├── LICENSE                       AGPL-3.0-only
├── Cargo.toml                    workspace root (12 members)
├── Cargo.lock
├── rust-toolchain.toml           pins stable
├── deny.toml                     cargo-deny allow/deny lists
├── crates/                       Rust source for every crate
├── kernels/                      CUDA source, organized as (hw, model, quant)
├── docker/                       per-hardware Dockerfiles
├── scripts/                      bench, model-sweep, release helpers
├── tests/                        cross-crate integration tests (run_all_models.py lives here)
├── docs/                         design notes, history, release notes
├── paper/                        LaTeX paper (ArXiv)
├── jinja-templates/              chat templates for models that need custom ones
├── bench/                        stable benchmark harness outputs (tracked)
├── book/                         this book (mdBook source)
└── vendor/                       vendored deps (e.g. xgrammar-rs)

The workspace members

Cargo.toml lists:

members = [
    "crates/atlas-core",
    "crates/atlas-kernels",
    "crates/atlas-plugin",
    "crates/atlas-tier",
    "crates/atlas-rdma",
    "crates/spark-runtime",
    "crates/spark-comm",
    "crates/spark-model",
    "crates/spark-nllb",
    "crates/spark-server",
    "crates/spark-storage",
    "crates/atlas-spark-bench",
    "crates/cufile-sys",
    "crates/xgrammar",
]

Each is its own crate with its own Cargo.toml, its own unit tests, and its own responsibility:

CrateRoleConsumed by
atlas-coreTraits & types used by every crate below: ComputeTarget (build-time compiler abstraction), KernelTarget (runtime dispatch key), Vendor, Dtype, Tensor, ModelConfig parsing, and the host-side numerics (numeric: FP8 E4M3 LUT, f32 → BF16 RNE cast) every weight loader shareseveryone
atlas-kernelsAuto-generated Rust glue over compiled PTX. build.rs enumerates kernels/<hw>/<model>/<quant>/*.cu, compiles each through the matching ComputeTarget, emits one target_ptx.rs that include!()s back into this cratespark-runtime
spark-runtimeGpuBackend trait (27 methods) + CUDA impl (cuda_backend.rs). KV cache, prefix cache (radix tree), paged FP8 cache, buffer arena, sampler, WeightStore (O_DIRECT + pipelined safetensors loader). Everything that touches the GPU goes through here.spark-model, spark-server
spark-commCommBackend trait (collective ops) + NCCL impl. SingleGpuBackend is the no-op impl for single-GPU runs.spark-model, spark-server
spark-modelModel assembly: layers (Qwen3Attention, Qwen3Ssm, NemotronMamba2, MoeLayer, VisionEncoder), per-family weight loaders, TransformerLayer trait, the inference engine (engine.rs), speculative decoding, vision preprocessingspark-server
spark-serverBinary. HTTP server (axum), OpenAI + Anthropic compatible endpoints, tool-call parsing (Hermes / Qwen3-coder / Mistral / XGrammar), tokenizer wrapper, rate limiter, CLIn/a — the deliverable
atlas-spark-benchCriterion benchmark client. Targets a live server, records per-endpoint throughput + TTFT. The numbers in bench/ come from here.bench runs only

The dependency graph runs strictly downward in the table above — atlas-core has no internal deps, every crate above it builds on crates below. There are no cycles.

The kernel tree

kernels/
└── gb10/                                        # One directory per hardware target
    ├── HARDWARE.toml                            # vendor, arch, memory specs
    ├── qwen3-next-80b-a3b/                      # One directory per model target
    │   ├── MODEL.toml                           # layer counts, sampling presets, behavior
    │   └── nvfp4/                               # One directory per quantization target
    │       ├── KERNEL.toml                      # compile flags, module name overrides
    │       └── *.cu                             # ~35 hand-written CUDA kernels
    ├── qwen3.5-35b-a3b/
    │   └── nvfp4/
    │       └── *.cu
    ├── qwen3.6-35b-a3b/
    │   └── fp8/
    ├── nemotron-3-nano-30b-a3b/
    │   └── nvfp4/
    ├── mistral-small-4-119b/
    │   └── nvfp4/
    ├── minimax-m2-229b/
    │   └── nvfp4/
    └── ... (one leaf per (model, quant) target — 22 under kernels/gb10/ today)

Every leaf directory is a fully self-contained (gb10, model, quant) target. The kernels inside a leaf can use any tile shape, any register budget, any shared-memory layout — they are physically incapable of regressing a different target.

This is the mechanism that makes kernels/ a scalable structure. Adding a new GPU is kernels/<new-hw>/. Adding a new model is kernels/<hw>/<new-model>/. Adding a new quantization is kernels/<hw>/<model>/<new-quant>/. Nothing else moves.

Docker layout

docker/
├── gb10/
│   ├── Dockerfile                         multi-model image — compiles every target
│   ├── qwen3-next-80b-a3b/nvfp4/          per-model slim image
│   ├── qwen3.5-35b-a3b/nvfp4/
│   └── ... (one slim Dockerfile per supported model)
└── docker-guide.md                        build + run instructions

The multi-model Dockerfile at docker/gb10/Dockerfile is what ships as avarok/atlas-gb10:latest. Per-model Dockerfiles exist for operators who want a smaller image containing only one target — the kernel registry still uses KernelTarget at runtime, but only one target set is baked in.

Docs, design records, history, releases

Inside docs/:

  • adr/ — architecture decision records (licensing, pure-Rust, hybrid SSM/attention, NVFP4/FP8 quantization, TP/EP composition, EP batched decode, etc.). Treat these as the long-form rationale behind code changes; commit messages are deliberately terse and point here. Top-level notes like ARCHITECTURE.md, ATLAS_KERNELS.md, and HARDWARE.md sit alongside them.
  • ATLAS_SPARK_JOURNEY.md — benchmark journey and retrospective across the Spark line. Useful context, but not a contract.
  • releases/ — human-readable release notes keyed by release (README.md plus per-release files).

The book you're reading in book/ synthesises all of this into a single narrative — it is not a canonical rewrite of those documents. The design records in docs/adr/ remain the authoritative reference and the book links to them directly from the deep-dive chapters.

What changes when you add a…

You addedYou touched
A new quantization (e.g. MXFP4)kernels/<hw>/<model>/<scheme>/*.cu, a format module under spark-model/src/quant_format/, the loader arms in spark-model/src/weight_map/, and any new host-side conversion in atlas-core/src/numeric.rs
A new model family (e.g. Phi-4)spark-model/src/weight_loader/<family>.rs, one arm in spark-model/src/factory.rs, kernels/<hw>/<family>/<quant>/MODEL.toml, optional jinja-templates/<family>.j2
A new hardware vendor (e.g. MI300X)atlas-core/src/compute.rs (new ComputeTarget impl), atlas-kernels/build.rs::resolve_compute_target() arm, spark-runtime/src/<vendor>_backend.rs (new GpuBackend impl), spark-comm/src/<vendor>_backend.rs if the vendor needs its own collective impl, kernels/<hw>/HARDWARE.toml, kernel source under kernels/<hw>/<model>/<quant>/
A new CLI flagspark-server/src/cli.rs, plumbing wherever it lands
A new tool-call formatspark-server/src/tool_parser.rs

Each row touches a small, bounded set of files. That bounded-ness is the architectural payoff of the workspace being split along axes of variation. Read Kernel Dispatch Pipeline next to see the runtime side, or SBIO to see how the trait layering makes the whole thing testable without a GPU.