docs: update DeepSeek-V4 agentic PD recipe

[ZhangLirong-amd]

Update the DeepSeek-V4 PD server recipe for the 0813 checkpoint, dspark with three speculative tokens and synthetic acceptance length 3.01, FP8 KV, and FP4 index cache. The synthetic acceptance setting is documented as benchmark-only.

The TP 1–32, DP attention 64–128, and DP attention with CPU offload 256 sections each provide complete prefill and decode commands. All six server blocks define their own environment, fixed role-specific port, and JSON written directly in --kv-transfer-config; no shared setup or inherited exports are required. The model path must include matching tokenizer files because the server loads its tokenizer from --model; the router and client explicitly set an accessible copy of that same tokenizer in their own shells.

TP decode requests dense graph sizes through min(64, CONC * 2) to respect the existing sequence limit; DP attention decode requests every per-rank batch size through CONC / 4, giving 1–16 at 64c, 1–32 at 128c, and 1–64 at 256c. All prefill commands keep default graph sizes. DP attention prefill uses TBO with GPU_MAX_HW_QUEUES=5; decode keeps TBO off. The DP-only prefill delayer setting is omitted from TP setup. The server commands retain unbuffered Python output; a fixed Python hash seed is scoped to the LMCache offload producer, and profiling/optional fault-handler overrides are omitted.

Only recipes/DeepSeek-V4-Agentic-PD-Max.md changes. Cache-aware routing and matched-rail support come from the already merged #2276. CPU offload guidance consistently requires at least 1,280 GiB of available host memory: 1,024 GiB for eight worker caches plus 256 GiB of headroom. GPU memory settings, CONC * 2 sequence expressions, AIPerf commands, and historical measurement values are preserved by this update. The historical C128 throughput figures explicitly identify the TBO-disabled prefill configuration.

Validation: all 11 Bash blocks pass syntax checks. Fourteen command simulations run the six standalone server blocks from empty environments across representative concurrency values for TP, DP attention, and CPU offload. They check ports, tokenizer paths containing spaces, inline KV JSON, graph bounds, TBO, queue settings, and the role-specific Python environment; six invalid-concurrency setups fail before launch. Five router/client command simulations also verify tokenizer paths in fresh shells. Documentation links, environment readers, unchanged client commands, GPU memory settings, and historical measurements, and git diff --check pass. No model or GPU workload was started.

[github-actions]

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

• ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

• ✅ Run after the PR is approved and Pre Checkin passes
• ✅ Run immediately when an approval review is submitted
• ✅ Can be requested before approval with labels

Label	Tests
ci:full	Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom	Run native ATOM model accuracy tests
ci:vllm	Run ATOM vLLM OOT model accuracy tests
ci:sglang	Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 2277 --add-label <label>

[Copilot]

🟡 Changes recommended

The new router and client commands depend on variables exported only in a separate server shell, producing invalid URLs or paths when copied independently.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Updates the DeepSeek-V4 agentic PD recipe with the current C128 cache-aware configuration while preserving historical profiles and measurements.

Changes:

• Adds matched-rail Mooncake, FP4 index cache, DSpark, and cache-aware router commands.
• Clarifies benchmark scoring, estimates, historical results, and memory headroom.
• Documents dependency on PR #2276.

File summaries

File	Description
recipes/DeepSeek-V4-Agentic-PD-Max.md	Updates C128 deployment instructions and historical benchmark documentation.

Review details

Suppressed comments (1)

recipes/DeepSeek-V4-Agentic-PD-Max.md:201

• The client block has the same shell-boundary problem: it relies on MODEL_PATH and PREFILL_IP from the server block, so copying it to the router/client shell leaves the tokenizer path and --server-metrics URL empty. Initialize or validate these variables in the client block so the documented command is runnable on its own.

aiperf profile --scenario inferencex-agentx-mvp \
  --url http://localhost:8000 --endpoint /v1/chat/completions \
  --endpoint-type chat --streaming \
  --model deepseek-ai/DeepSeek-V4-Pro \
  --tokenizer "$MODEL_PATH" --tokenizer-trust-remote-code \

• Files reviewed: 1/1 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The documented FP4 PD commands are rejected by the current DeepSeek-V4 region-map gate, including the C256 hybrid connector example.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (1)

recipes/DeepSeek-V4-Agentic-PD-Max.md:423

• Even if plain Mooncake FP4 support is added, this C256 example still combines FP4 with a multi connector whose children are Mooncake and lmcache_offload with offload_layout=hybrid. The connector factory treats any region-reading child (including hybrid offload) as a block-region topology, so the FP4 gate rejects this startup; only dense offload bypasses that gate. The C256 profile is therefore not runnable as documented; use FP8 for this hybrid offload path or change the implementation/configuration.

  --kv-transfer-config '{"kv_connector":"multi","connectors":[{"kv_role":"kv_producer","kv_connector":"mooncake","proxy_ip":"10.0.0.1","handshake_port":6301,"protocol":"rdma"},{"kv_connector":"lmcache_offload","kv_role":"offload","offload_layout":"hybrid","max_pending_saves":8,"slot_sidecar_staging_slots":4,"lmcache.local_cpu":true,"lmcache.max_local_cpu_size":128,"lmcache.local_disk":null,"lmcache.max_local_disk_size":0,"lmcache.remote_url":null,"lmcache.chunk_size":256,"lmcache.cache_policy":"LRU","lmcache.lookup_server_worker_ids":[],"lmcache.store_location":"LocalCPUBackend","lmcache.retrieve_locations":["LocalCPUBackend"]}]}'

• Files reviewed: 1/1 changed files
• Comments generated: 1
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

The recipe documents unsupported environment variables, and its matched-rail setting is not consumed by the current Mooncake connector.

Get a fresh assessment by requesting another Copilot review.

Review details

Suppressed comments (4)

recipes/DeepSeek-V4-Agentic-PD-Max.md:199

• ATOM_PD_LOAD_PAIRING has no definition or read site in the current ATOMesh/ATOM tree, so exporting it cannot select independent P/D pairs. The effective setting here is the explicit --atom-pd-rank-mapping-policy none flag below; keeping this unsupported export makes the recipe suggest a control that does nothing. Remove the export or implement it in atomesh.

export ATOM_PD_LOAD_PAIRING=independent

recipes/DeepSeek-V4-Agentic-PD-Max.md:236

• ATOM_MOONCAKE_MATCHED_RAILS is not consumed by the current Mooncake connector; the only environment override it reads is ATOM_MOONCAKE_IB_DEVICE (atom/kv_transfer/disaggregation/mooncake/mooncake_connector.py:615-639), otherwise it selects the GPU-local HCA. With rank mapping set to none, P/D ranks may differ, so this line does not establish the promised matched-rail reachability and RDMA transfers can fail. Use the supported per-connector HCA/alternate-registration settings or add a reader for this variable before relying on it.

export ATOM_MOONCAKE_MATCHED_RAILS=ionic_0,ionic_1,ionic_2,ionic_3,ionic_4,ionic_5,ionic_6,ionic_7

recipes/DeepSeek-V4-Agentic-PD-Max.md:243

• ATOM_DP_LOAD_SNAPSHOT has no definition or read site in the current ATOM tree, so it cannot enable the claimed prefill load reporting; setting it is a no-op. Remove this export or add the corresponding server/metrics implementation before documenting it as required.

export ATOM_DP_LOAD_SNAPSHOT=1

recipes/DeepSeek-V4-Agentic-PD-Max.md:298

• ATOM_PD_LOAD_PAIRING has no definition or read site in the current ATOMesh/ATOM tree, so exporting it cannot select independent P/D pairs. The effective setting here is the explicit --atom-pd-rank-mapping-policy none flag below; keeping this unsupported export makes the recipe suggest a control that does nothing. Remove the export or implement it in atomesh.

export ATOM_PD_LOAD_PAIRING=independent

• Files reviewed: 1/1 changed files
• Comments generated: 3
• Review effort level: Lite

[Copilot]

🟡 Changes recommended

The recipe needs corrections for matched-rail naming, synthetic acceptance disclosure, and low-concurrency graph sizing.

Get a fresh assessment by requesting another Copilot review.

Review details

• Files reviewed: 1/1 changed files
• Comments generated: 3
• Review effort level: Lite

[Copilot]

Copilot review overview

🟡 Changes recommended

The documented separate tokenizer override is not honored by ATOM server launches.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2

Open (2)

• TOKENIZER_PATH override does not affect OpenAI server workers · New
• ATOM_MOONCAKE_MATCHED_RAILS is not consumed by the current Mooncake connector; the only…

Resolved since last review (4)

• This server command cannot start with the current DeepSeek-V4 PD implementation. When…
• [verified] The TP section covers concurrency 1–32, but hard-coding --max-num-seqs 256 makes ATOM…
• [verified] Matched-rail selection looks up the consumer-advertised HCA name in the producer's…
• [verified] --spec-decode-acceptance-length uses a forced synthetic schedule that stops comparing…

[Copilot]

Copilot review overview

🟡 Changes recommended

The generic commands rely on undefined variables and do not actually retain the documented 1–64 decode graph ladder.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 4 · 1

Open (5)

• PORT is unset before conditional server command · New
• KV_TRANSFER is undefined when starting PD server · New
• TOKENIZER_PATH override does not affect OpenAI server workers
• ATOM_MOONCAKE_MATCHED_RAILS is not consumed by the current Mooncake connector; the only…
• CUDAGraph ladder exceeds schedulable decode batch · New

[Copilot]

Copilot review overview

🟡 Changes recommended

The reported throughput used prefill TBO disabled, while the documented commands enable prefill TBO, so the measurement is not reproducible as written.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

• Benchmark result mismatches documented prefill TBO configuration · New

Resolved since last review (2)

• FP4 DeepSeek-V4 indexer rejects prefill TBO
• Router and AIPerf lack tokenizer path configuration

[Copilot]

Copilot review overview

🟡 Changes recommended

The CPU-offload section’s embedded client command remains fixed at concurrency 256, so advertised higher-concurrency configurations are not actually benchmarked.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

• Client concurrency remains hard-coded at 256 · New

Resolved since last review (1)

• Benchmark result mismatches documented prefill TBO configuration

[Copilot]

Copilot review overview

🟢 Approval recommended

The documentation is consistent with the current V4, Mooncake, routing, and speculative-decoding interfaces, with no blocking issues found.

Review effort: Lite
Findings: None

Resolved since last review (1)

• Client concurrency remains hard-coded at 256

[Copilot]

Copilot review overview

🔵 Needs a closer look

The recipe combines multi-node RDMA, FP4 transfer, DSpark, TBO, and CPU offload without a full GPU workload validation.

Review effort: Lite
Findings: None

[Copilot]

Copilot review overview

🟡 Changes recommended

The offload memory guidance inconsistently specifies 1,280 GiB versus 1,024 GiB as the startup requirement.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1

Open (1)

• Align documented host-memory thresholds to prevent startup OOM · New

[Copilot]

Copilot review overview

🔵 Needs a closer look

Distributed GPU, RDMA, offload, and graph settings were not validated with a live model workload.

Review effort: Lite
Findings: 1

Open (1)

• Align documented host-memory thresholds to prevent startup OOM

[Copilot]

Copilot review overview

🔵 Needs a closer look

The recipe changes multiple distributed GPU configurations and transport/cache behavior without live workload validation.

Review effort: Lite
Findings: None

Resolved since last review (1)

• Align documented host-memory thresholds to prevent startup OOM