Repository navigation
Plumbing for the TDFA rewrite: Copy op, inline records, accept field, parallel moves - #203
Merged
Merged
Conversation
Copy (dst, src) copies the raw contents of one memory cell into another, preserving the position/value encoding. The Laurikari TDFA construction needs it to preserve a recorded position when a parallel NFA path is about to overwrite the source register. Not emitted by the compiler yet, so generated code is unchanged. (cherry picked from commit 8ab62944 without its oracle hunk) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Named fields instead of positional arguments: [Set_position {dst}],
[Set_value {dst; value}], [Copy {dst; src}]. The determinization rewrite
manipulates these in several places (allocation, register moves,
renaming), where [dst] and [src] read better than tuple positions.
No behavior change.
(cherry picked from commit abf8efa, redone on the pre-rewrite code)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
hhugo
force-pushed
the
pr4a-tdfa-prep
branch
from
September 18, 2026 11:50
b2dec3a to
7a30d66
Compare
hhugo
force-pushed
the
pr4a-tdfa-prep
branch
from
September 18, 2026 16:01
7a30d66 to
f73e1db
Compare
hhugo
marked this pull request as ready for review
September 18, 2026 16:04
Member
|
Very cool! |
hhugo
force-pushed
the
pr4a-tdfa-prep
branch
from
September 19, 2026 06:45
f73e1db to
8279a8b
Compare
A DFA state's [finals : bool array] becomes [accept : accept option],
with [accept = { rule; final_ops }]. For an accepting state it holds
the lowest-numbered accepting rule, which the PPX used to recompute
from [finals] in [best_final], and the tag operations to run on entry
just before [Sedlexing.mark]. The TDFA construction will use those to
copy the accepting path's working registers into the canonical cells
read by the binding extraction code; the compiler sets the list to []
for now, so generated code is unchanged.
Nothing driving the automaton needs the other accepting rules of a
state, so [finals] goes away; [accepting_rule] finds the winning rule
directly from the rules' final nodes. The DOT printer labels an
accepting state with that rule only. No expect output listed more
than one rule, so the graphs are unchanged.
[call_state] moves below [gen_tag_ops], which it now uses, and the DOT
printer shows final ops on accepting-state labels when there are any,
with [tag_op_to_string] hoisted to top level.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The TDFA construction emits register moves whose Copy operations must all read the pre-transition contents: a transition may both copy cell A into B and overwrite A. [gen_tag_ops] now let-binds every Copy source that the same list also writes, via the new raw accessors [__private__mem_get] / [__private__mem_set], and emits the writes in list order. [op_dest] is exported for it. No Copy is emitted yet, so generated code is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
hhugo
force-pushed
the
pr4a-tdfa-prep
branch
from
September 19, 2026 19:09
8279a8b to
988e77d
Compare
This was referenced Sep 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Preparation for the tagged-DFA rewrite that fixes #199 (series: #200, #201, #202). Four commits that add everything the rewrite needs outside the compiler's determinization, without the compiler producing any of it yet. Generated code and every expect output are byte-identical (
dune runtestprints no diff after each commit); the review check is that nothing new is emitted.Commits:
Copytag operation. Copies the raw contents of one memory cell into another, preserving the position/value encoding. Runtime__private__copy_mem, PPX generation, DOT labeltN<-tM. Not emitted yet.tag_op.Set_position {dst},Set_value {dst; value},Copy {dst; src}: the rewrite manipulates these in allocation, register moves and renaming, where named fields read better than tuple positions.accept : accept option, withaccept = { rule; final_ops }, replacesfinals : bool arrayon DFA states. Rule priority is resolved in the compiler (accepting_rule) instead of the PPX'sbest_final, and the accepting state carries the operations to run on entry just beforeSedlexing.mark. The rewrite will use them to copy the accepting path's working registers into the canonical cells read by binding extraction; the list is[]for now, sogen_tag_opswraps nothing. Nothing driving the automaton needs a state's other accepting rules, sofinalsgoes away and the DOT printer labels an accepting state with the winning rule only; no expect output listed more than one rule, so the graphs are unchanged.gen_tag_ops. A transition's operations form a parallel move: aCopyreads its source as it was before the list ran. Sources the list also writes are saved in let-bound locals through the new raw accessors__private__mem_get/__private__mem_set;op_destis exported for it. NoCopyis emitted yet.With these in, the rewrite's diff outside
src/compiler/sedlex.mlis confined to expect outputs (plus the matching one-line changes in the #202 interpreter).🤖 Generated with Claude Code