Skip to content

The q8 mul_mm double-buffer stamps derive from the tensor template; a census cell pins each stamp's stage width - #4018

Merged
borisbat merged 3 commits into
masterfrom
bbatkin/q8-mulmm-fold
Sep 13, 2026
Merged

The q8 mul_mm double-buffer stamps derive from the tensor template; a census cell pins each stamp's stage width#4018
borisbat merged 3 commits into
masterfrom
bbatkin/q8-mulmm-fold

Conversation

@borisbat

Copy link
Copy Markdown
Collaborator

Why. The q8 mul_mm tensor and double-buffer kernel classes carried the same eight-line body, one stage width apart, so every fix had to land twice.

What changes.

  • The double-buffer template is gone: the tensor template takes the K chunk width and the staging tile as constants, and the two Db stamps override them, keeping their dispatch names and PSO globals.
  • A device-free cell pins the width each stamp reads through the emitter's census string; the double-buffer gate sets the threadgroup length from the stamp it dispatches on both arms.
  • Six checklists fix what applying them to this diff surfaced: the template-constant rule names its runtime channels, a named ledger item counts as a row, an emission-identical kernel-body change needs neither board rows nor a parity run, the exit rule reaches the occupancy report, a gate reading another stamp's threadgroup global updates with it, the per-step chunk is a shape constant, and the emitter routing fires on class-level stamp edits.

Observable behavior. None - refactor only; emitted kernels are byte-identical.

Where to look. The two override BK / override TWB stamps in dasllama_metal_prefill.das and the new width cell in test_metal_gemm_kernels.das.

Validation, claims, ledger

Validation

  • Kernel-emission compare: every emitted Metal kernel's MSL (299) compiled to AIR before and after the fold, disassembled with value names discarded and diffed: 299 identical, 0 changed, 0 added, 0 removed. The emitted _tgmem companions read 0x4800 on both Db stamps and 0x3000 on the tensor stamps after the fold, the values the deleted template carried.
  • run.das -- --suite kernels --arm width,gemm: green; the gemm corpus cell dispatches both Db stamps (kdim 256, 288 and 512, the 288 arm exercising the tail chunk).
  • run.das -- --changed (llm, infra): 97 files, green but for test_vulkan_dec_tail and test_vulkan_tier, the M5's standing Vulkan class-rail declines (32 KB shared-memory cap, same 2+3 errors on master).
  • Negative controls: override TWB = 6144 on the 128-row Db stamp reds the gemm corpus cell; dropping override BK = 128u reds the new width cell and nothing else.
  • Full preflight ran once; its lint gate was red on the Linux rail (the census globals do not exist without dasMetal) and passed after the cell took the file's static_if guard; the skipped lanes (docs, cpp, interp, jit, aot) ran green once each.
  • REVIEW_GPU.md's alignment finding on these stamps (no requires contract for kdim % 64) is a false positive: the tensor form handles a partial last chunk, and the gemm cell's kdim 288 arm proves it against the fp64 oracle.
  • benchmarks/matmul/occupancy_report.das has no test reaching it; it compiles clean with the renamed globals.

Not done

  • The Metal ledger's fold census still lists the K45/K6 double-buffer pair, the next fold of this shape.
  • Two lint candidates from the checklist reviews: a REVIEW.das gate for a C++ module's Module::require matching its CMake links, and the kernels-suite pinned-gate registry moving out of tests/REVIEW.md into a cited section.
  • modules/ has no ARCHITECTURE.md for its checklist's opening slot; whether to add one is an owner's call.

borisbat and others added 2 commits September 12, 2026 20:10
…the K chunk width and the staging tile are its constants, the two Db stamps override them and keep their dispatch names, and the emitted globals unify on the tensor kernel's name; the AIR of all 299 Metal kernels is identical before and after, and ledger row 143 goes

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rough the emitter's census, the double-buffer gate sets the threadgroup length from the stamp it dispatches, the Metal ledger's fold census drops the landed item, and five checklists fix what applying them surfaced: a template constant fed to a helper is the sanctioned shape, a named ledger item counts as a row, an emission-identical in-body change needs no board rows and no parity run, the exit rule reaches non-timing reports, a gate reading another stamp's threadgroup global updates with it, and the per-step chunk is a shape constant rather than the reduction width

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 13, 2026 03:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

It introduces a non-attaching mid-file //! doc comment that violates the repo’s comment-hygiene convention and should be changed to //.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Refactors the dasLLAMA Metal q8 mul_mm double-buffer stamps to reuse the tensor template (parameterized by K chunk width and staging tile size), and adds a device-free test cell that pins each stamp’s stage width via the emitter census string. The PR also updates relevant review/checklist docs to reflect stamp-level threadgroup-memory coupling and emitter routing rules.

Changes:

  • Folded the q8 mul_mm double-buffer template into MetalQ8MulMmTensorT via new @template_constant parameters (BK, TWB), and updated dispatch/pso wiring to the new emitted _t_msl globals.
  • Updated the GEMM gate to set threadgroup-memory length from the specific stamp being dispatched, and added a census-based test to assert each stamp’s stage width.
  • Refined multiple module checklists/ledgers/architecture notes to codify the refactor’s invariants and review expectations.
File summaries
File Description
modules/REVIEW.md Broadens shader-emitter review routing to include kernel-generating classes and member declarations.
modules/REVIEW_SHADER_EMITTERS.md Clarifies runtime reduction width vs per-step chunk as a shape constant for tmm2d_*.
modules/dasLLAMA/tests/test_metal_gemm_kernels.das Updates q8 double-buffer gate dispatch globals/tgmem selection; adds census-based width test.
modules/dasLLAMA/tests/REVIEW_KERNEL_CELLS.md Adds rule for stamp-specific @workgroup size changes affecting gates that read another stamp’s *_tgmem.
modules/dasLLAMA/REVIEW.md Tightens wording and clarifies ledger-row deletion/editing rule; minor checklist phrasing tweaks.
modules/dasLLAMA/REVIEW_GPU.md Clarifies GPU checklist routing and defines when parity evidence is required for emission-identical kernel-body refactors.
modules/dasLLAMA/followup_metal.md Removes the now-addressed q8 mul_mm double-buffer fold item reference.
modules/dasLLAMA/followup_general.md Deletes the completed follow-up ledger row describing the q8 mul_mm template fold.
modules/dasLLAMA/dasllama/dasllama_metal_prefill.das Introduces BK/TWB constants in tensor template; rehomes Db stamps onto it; updates PSO compilation globals.
modules/dasLLAMA/benchmarks/REVIEW.md Clarifies which changes require benchmark record updates and strengthens “no results => non-zero exit” rule.
modules/dasLLAMA/benchmarks/matmul/occupancy_report.das Updates the q8 dbuf stamp to use the new emitted _t_msl globals.
modules/dasLLAMA/ARCHITECTURE_GPU.md Updates architecture text to reflect Db forms as stamps (BK=128) rather than a separate staging template.
Review details
  • Files reviewed: 12/12 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread modules/dasLLAMA/tests/test_metal_gemm_kernels.das Outdated
… does not attach

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings September 13, 2026 03:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It changes low-level GPU kernel stamp/template structure and associated dispatch/gate wiring, which is high-impact even with strong evidence of byte-identical emission.

Review details
  • Files reviewed: 12/12 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

@borisbat
borisbat merged commit 3cb7bcf into master Sep 13, 2026
31 checks passed
@borisbat
borisbat deleted the bbatkin/q8-mulmm-fold branch September 13, 2026 04:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants