Skip to content

A walk following a call finds its callee's values without scanning the program - #7404

Merged
matz merged 1 commit into
matz:masterfrom
makenowjust:MakeNowJust/method-values-index
Oct 5, 2026
Merged

matz merged 1 commit into
matz:masterfrom
makenowjust:MakeNowJust/method-values-index

Conversation

@makenowjust

Copy link
Copy Markdown
Contributor

Fixes #7403

What changes

  • method_value_leaves takes a method's returns from a chain per scope, comp_sret_first / comp_sret_next in src/compiler.c, the same shape as the existing CallNode chain comp_scall_first. The chain is built in node order and rebuilt when the node table's version or the scope count moves. It is used only while the scope index is frozen (the inference fixpoint and after), since before that a node's scope can still move; otherwise the scan runs as before. The returns, their order and the cut at the caller's capacity are the ones the scan gave.
  • method_has_other_body remembers its answer per scope while the scope index is frozen, keyed by the scope-index epoch and the scope and class counts (the inputs of its scan: the scopes' names, kinds and classes, and the class hierarchy).

A similar per-scope grouping of ReturnNodes is added inside promote_shared_stored_strings by #7386 (an_returns_by_scope, rebuilt per run); the two could be merged later.

Why the results do not change

spinel -c was run on master d38099f and on this change (based on it) from the same worktree, with the same input paths, in both overflow modes, on every test/*.rb (5,791 files × raise/promote = 11,582 compilations): the generated C is identical except in the 4 tests that embed the compiler's own revision in RUBY_DESCRIPTION (frozen_chilled_builtin_strings, object_scoped_ruby_constants, ruby_description_shape, symbol_id2name_ruby_desc_minmax, both modes), and the refusals are the same.

Before / after

spinel -c --int-overflow=promote on master ab9b925 and on this change, one after the other, counting the calls of the two helpers and the iterations of their scans (a local counter, not part of this change):

program calls (each helper) ReturnNodes visited scopes compared time
18k-line machine-generated 7,484 1,835,605 → 8,921 2,196,246 → 31,352 9.3 s → 9.5 s
53k-line machine-generated 225,538 85,415,533 → 384,751 83,366,902 → 81,552 37.8 s → 36.7 s

Each iteration of the scans is cheap, so at these sizes the time is within noise. The scans grow with (calls followed × program size), and on a 289k-line program (about 3,000 returns and 2,000 methods) they were 918 of the 2,300 samples that widen_boxed_array_sources took at 180 s into the compilation (with #7377, #7378 and #7384), the walk itself being about 18% of all samples at that point.

(Timings are from a shared machine in low-power mode; the two runs of each program were taken back to back.)

Testing

make gate:

scale-test: instance_eval forwarding work at 2x the wrappers is 1.71x (limit 2.50)
scale-test: work at 4x the program is 4.73x (linear 4.00, limit 5.20)
scale-test: work at 4x the program, compiled to C, is 6.08x (limit 6.90)
scale-test: call-shape work at 4x the units, compiled to C, is 4.22x (linear 4.00, limit 4.50)
Tests: 5928 pass, 0 fail, 0 error
gate: stamp for tree 30c5c214706d on master ab9b925aa333; git commit --amend --no-edit adds the Gate: trailer
gate: ALL GREEN

The stamp line above names the origin/master the clone had fetched when the gate finished (ab9b925); the tree the gate ran was based on 92510d6, as described below.

SPINEL_INT_OVERFLOW=promote make test: 5909 pass, 15 fail, 13 error. Every failure is one master e5e8f79 already had; none is new.

The gate ran on a branch based on 92510d6 that combines this change with the other compile-time fixes from the same investigation (#7377, #7378, #7384, #7375, #7386, #7388, #7390, #7392, #7396, #7398, #7400, #7402). This branch was then rebased onto ab9b925 without conflicts.

…e program

widen_boxed_array_sources and array_src_walk follow a call into the
value of the method it binds to: method_value_leaves collects the
method's returns, and method_has_other_body asks whether a subclass or
a second definition can answer instead. The first scanned every
ReturnNode of the program and the second every scope, once per call the
walk follows. On a large machine-generated program (289k lines, about
3,000 returns and 2,000 methods) the walks follow hundreds of thousands
of calls per round, and these two scans were most of their time.

The returns now come from a chain per scope (comp_sret_first, beside
the CallNode chain comp_scall_first), built in node order, so the
values and their order, and the cut at the caller's capacity, are the
ones the scan gave. method_has_other_body remembers its answer per
scope while scope shape is fixed (the scope-index epoch and the scope
and class counts). Both fall back to the scans while the scope index is
not frozen. The generated C is unchanged.
@coderabbitai

coderabbitai Bot commented Oct 5, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 11 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 8 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e770cff2-fd53-4623-babc-2f912f21a604
📥 Commits

Reviewing files that changed from the base of the PR and between ab9b925 and 29c120e.

📒 Files selected for processing (3)
  • src/analyze_pass.c
  • src/compiler.c
  • src/compiler.h
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@makenowjust

Copy link
Copy Markdown
Contributor Author

Counts on the 289k-line program the issue describes, to add to the description. spinel -c --int-overflow=promote, master ab9b925 against this change on top of it, run back to back on a machine in low-power mode, each stopped after 120 s (neither finishes within 300 s there; other passes dominate). Both had called each of the two helpers 294,661 times by then, so they had reached the same point. Counted with a local counter that is not part of this change:

master this change
ReturnNodes visited by method_value_leaves 876,189,594 441,438
scopes compared by method_has_other_body 627,038,608 1,440,656

@matz
matz merged commit 91b0c80 into matz:master Oct 5, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Following a call into its callee's value scans every ReturnNode and every scope of the program

2 participants