Skip to content

ROCm: add DeepSeek V4.1 Flash support for Strix Halo (gfx1151) - #1036

Open
kyuz0 wants to merge 2 commits into
antirez:mainfrom
kyuz0:feat/rocm-deepseek41-halo
Open

ROCm: add DeepSeek V4.1 Flash support for Strix Halo (gfx1151)#1036
kyuz0 wants to merge 2 commits into
antirez:mainfrom
kyuz0:feat/rocm-deepseek41-halo

Conversation

@kyuz0

@kyuz0 kyuz0 commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Enable DwarfStar DeepSeek V4.1 Flash Q2 text and vision on AMD Strix Halo (gfx1151), previously restricted to Metal. Adds ROCm graph operators, resident and SSD expert loading, bounded parallel Engram reads, optimized prefill kernels and complete session-state handling. Fixes resident vision admission and lets SSD cache allocation use the admitted memory budget.

Tested with ROCm 10.0 on a single 128 GB Strix Halo and a 2 TB Kingston FURY Renegade SSD (PCIe 4.0 ×4). An 8 GiB direct model-file read measured 5.16 GB/s, verified with device counters. Native 8K inference measured 214.15 prefill / 5.47 decode tokens/s with an 89 GiB cache and 16 greedy outputs. A 94 GiB cache also passes 16K/512-output and image/state checks, filling all 9,372 slots while keeping at least 11.48 GiB RAM available during the continuation, with no swap-out or OOM. Separate scoring/agent runs recorded 2.62 MiB of host zram swap-out, with no sampled model swap or OOM. These are individual runs; startup is excluded and image prefill was not timed.

Quality evidence:

  • Official 100-case panel, 2,994 targets: NLL 0.366245, versus published same-Q2 Metal 0.365681 resident / 0.364576 scalar-SSD (+0.154% / +0.458%). This is an aggregate comparison, not a fresh paired Metal run.
  • Another 12 batched cases and selected 8K/16K/64K cases exercise changed arithmetic. NLL rises 0.639% / 4.327% / 1.126% / 4.436% against earlier ROCm controls; detailed deltas are included in QA.
  • Both modes allocate 256K and complete a real 64K prompt + 128 outputs, with all 129,280 logits and the printed continuation identical. SSD uses 92 GiB cache; 94 GiB was refused at that allocation. Populated 256K and retrieval quality were not tested; small host zram paging is disclosed in QA.
  • 44/44 greedy and 28/28 sampled practical checks, six image fact cases in both modes, native image-driven code edits with an independent oracle, state replay/cancellation, kernel references and five binary help/link checks pass. Matching resident/SSD schedules preserve the tested logits, states and continuation; differing short-prefill schedules can produce different output.

Resident text and vision were also tested on upcoming 192 GB hardware; performance results will be released soon. Engram remains disk-backed in both modes. The original 512-token thinking truncation completes with a documented 4,096-token allowance. Physical CUDA/Metal and multi-device execution were not validated.

Implement V4.1 graph operators, resident and SSD expert loading, bounded parallel Engram reads, gfx1151 prefill kernels and vision projection support. Preserve complete session state across append, restoration and cancellation.

Use explicit ROCm memory admission for resident vision and SSD cache growth. Add production-shape kernel, cache-span and memory-pressure regressions, setup guidance and qualification results, including Metal Q2 comparisons, numerical drift and storage/state checks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant