metal: add opt-in slab residency for DeepSeek V4.1 Flash

[Dango233]

DeepSeek V4.1 Flash SSD decode runs at about 0.25 tokens/s on this M2 Ultra when using the default expert-cache slabs. Attaching those slabs to a Metal queue residency set raises throughput to 9.57–9.67 tokens/s over eight generated tokens, or 13.71–13.83 tokens/s after the first token.

Cause and change

A model-free reproducer fills and locks a 104 GiB buffer pool, then reads small ranges through GPU addresses. Before queue attachment, each large-buffer submission takes about 250.5 ms: Metal reports 249.7 ms in its kernel/driver interval and 0.013 ms of GPU execution. Queue attachment reduces submission time to 0.180 ms. A 32 GiB pool does not show the delay. An unattached residency set, additional CPU mlock, and warmup alone did not resolve the model-level slowdown.

These results point to large-allocation submission/residency overhead. The reproducer checks every GPU checksum and opens no model file. Its retained set stays warm after queue detachment, so the model benchmarks below use fresh processes for each run.

DS4_METAL_STREAMING_SLAB_RESIDENCY=1 enables the change on macOS 15+. It is off by default. The set contains only owned expert-cache slabs and uses the existing queue. Cache teardown and successful mlock-margin relief remove and release it; after relief, residency stays disabled until the cache is rebuilt.

Performance

M2 Ultra, 192 GiB, macOS 15.7.4, Metal; calibrated DeepSeek V4.1 Flash IQ2_XXS/Q2_K. Base: bd66c402070042bf0a79ad6ece8242de4c93680c. Each run uses the same binary, 2,048-token prompt, 8,257 allocated context, and automatic expert cache. The OS/file cache is not flushed.

Run	Residency	Prefill t/s	Decode t/s, 8 tokens
1	off	76.15	0.25
2	on	70.95	9.57
3	on	69.16	9.67
4	off	60.11	0.25

All four outputs match. The eight-token run keeps the slow control manageable; first-token latency has a substantial effect on its average. Prefill varied across runs.

Tests

• Clean Metal build and CPU compilation.
• Metal API validation: slab relief, rebuild and deallocation; SSD expert and MoE prefill tests, including table-admission and address modes; real-weight V4.1 sessions, snapshots and cancellation.
• Frontend, Engram, GGUF and quality-tool unit tests.

The build retains two existing SDK 15 warnings about unused Metal 4 symbols. The real-model tests use matching V4.1 fixtures; the full legacy make test suite was not run. Hardware validation covers this M2 Ultra.

Commands, model hash, raw CSVs and detailed results.