Skip to content

perf(lazy_turtle): targeted ancestor+subtree map in materialize_submodule - #2989

Merged
Qubitium merged 1 commit into
mainfrom
devin/port-lazy-turtle-opt
Jul 30, 2026
Merged

perf(lazy_turtle): targeted ancestor+subtree map in materialize_submodule#2989
Qubitium merged 1 commit into
mainfrom
devin/port-lazy-turtle-opt

Conversation

@Qubitium

@Qubitium Qubitium commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

LazyTurtle.materialize_submodule was building dict(target_model.named_modules()) on every call, which is O(total_modules × modules_per_layer) and dominated per-module load time for large MoE models. This change replaces that full-model scan with a targeted map that contains only the target submodule, its ancestors, and (when recurse=True) its descendants, which is exactly what _resolve_prefer_transposed_hint needs.

# before
modules_by_name = dict(target_model.named_modules())

# after
modules_by_name = {"": target_model}
for i in range(len(parts)):
    modules_by_name[prefix] = target_model.get_submodule(prefix)
modules_by_name[module_path] = target_submodule
if recurse:
    for subname, submod in target_submodule.named_modules():
        modules_by_name[...] = submod
  • materialize_submodule now accepts optional module_path, recurse, tie_weights, and show_progress kwargs, builds the targeted map, and passes it to _copy_checkpoint_tensors_into_submodule.
  • _copy_checkpoint_tensors_into_submodule now accepts an optional modules_by_name map and falls back to a full named_modules() scan only when none is provided.
  • Added tests/test_lazy_turtle_materialize_map.py with regression tests that monkeypatch nn.Module.named_modules to raise if a full model scan is used, while still verifying is_transposed hints on ancestors and descendants and numerical parity of the loaded weights.
  • Added scripts/benchmark_lazy_turtle_materialize_map.py and scripts/profile_lazy_turtle_lifecycle.py for micro-benchmarking map construction and measuring the full turtle -> shell -> disk lifecycle on a synthetic MoE model.

Validation:

  • ruff check ... clean
  • pytest -q tests/test_structure.py tests/test_lazy_turtle_materialize_map.py — 5 passed
  • python scripts/benchmark_lazy_turtle_materialize_map.py and LAZY_LIFECYCLE_LAYERS=2 python scripts/profile_lazy_turtle_lifecycle.py run successfully on CPU

…dule

Avoid scanning the entire shell model with target_model.named_modules() on every
materialize_submodule call. Instead build only the target submodule's ancestor
chain plus its descendant subtree when recurse=True. This removes an
O(total_modules * modules_per_layer) overhead that dominated per-module load time
for large MoE models.

- materialize_submodule now accepts optional module_path, recurse, tie_weights,
  and show_progress kwargs and passes a targeted modules_by_name map into
  _copy_checkpoint_tensors_into_submodule.
- _copy_checkpoint_tensors_into_submodule accepts an optional modules_by_name
  map and falls back to a full named_modules() scan only when none is provided,
  preserving backwards compatibility.
- Add regression tests ensuring the targeted map is used and transpose hints are
  still resolved for ancestors and descendants.
- Add benchmark and lifecycle profiler scripts for measuring the map
  construction and turtle -> shell -> disk overhead.
@Qubitium Qubitium self-assigned this Jul 30, 2026
@devin-ai-integration

Copy link
Copy Markdown

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@Qubitium
Qubitium merged commit 984a220 into main Jul 30, 2026
6 checks passed
@Qubitium
Qubitium deleted the devin/port-lazy-turtle-opt branch July 30, 2026 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant