Skip to content

Plumbing for the TDFA rewrite: Copy op, inline records, accept field, parallel moves - #203

Merged
hhugo merged 4 commits into
ocaml-community:masterfrom
hhugo:pr4a-tdfa-prep
Sep 20, 2026
Merged

hhugo merged 4 commits into
ocaml-community:masterfrom
hhugo:pr4a-tdfa-prep

Conversation

@hhugo

@hhugo hhugo commented Sep 18, 2026 •

Copy link
Copy Markdown
Collaborator

Preparation for the tagged-DFA rewrite that fixes #199 (series: #200, #201, #202). Four commits that add everything the rewrite needs outside the compiler's determinization, without the compiler producing any of it yet. Generated code and every expect output are byte-identical (dune runtest prints no diff after each commit); the review check is that nothing new is emitted.

Commits:

  1. Copy tag operation. Copies the raw contents of one memory cell into another, preserving the position/value encoding. Runtime __private__copy_mem, PPX generation, DOT label tN<-tM. Not emitted yet.
  2. Inline records for tag_op. Set_position {dst}, Set_value {dst; value}, Copy {dst; src}: the rewrite manipulates these in allocation, register moves and renaming, where named fields read better than tuple positions.
  3. accept : accept option, with accept = { rule; final_ops }, replaces finals : bool array on DFA states. Rule priority is resolved in the compiler (accepting_rule) instead of the PPX's best_final, and the accepting state carries the operations to run on entry just before Sedlexing.mark. The rewrite will use them to copy the accepting path's working registers into the canonical cells read by binding extraction; the list is [] for now, so gen_tag_ops wraps nothing. Nothing driving the automaton needs a state's other accepting rules, so finals goes away and the DOT printer labels an accepting state with the winning rule only; no expect output listed more than one rule, so the graphs are unchanged.
  4. Parallel-move gen_tag_ops. A transition's operations form a parallel move: a Copy reads its source as it was before the list ran. Sources the list also writes are saved in let-bound locals through the new raw accessors __private__mem_get / __private__mem_set; op_dest is exported for it. No Copy is emitted yet.

With these in, the rewrite's diff outside src/compiler/sedlex.ml is confined to expect outputs (plus the matching one-line changes in the #202 interpreter).

🤖 Generated with Claude Code

hhugo and others added 2 commits September 18, 2026 11:21
Copy (dst, src) copies the raw contents of one memory cell into
another, preserving the position/value encoding. The Laurikari TDFA
construction needs it to preserve a recorded position when a parallel
NFA path is about to overwrite the source register. Not emitted by the
compiler yet, so generated code is unchanged.

(cherry picked from commit 8ab62944 without its oracle hunk)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Named fields instead of positional arguments: [Set_position {dst}],
[Set_value {dst; value}], [Copy {dst; src}]. The determinization rewrite
manipulates these in several places (allocation, register moves,
renaming), where [dst] and [src] read better than tuple positions.
No behavior change.

(cherry picked from commit abf8efa, redone on the pre-rewrite code)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@hhugo hhugo changed the title Add Copy tag op and final_ops plumbing ahead of the TDFA rewrite Plumbing for the TDFA rewrite: Copy op, inline records, accept field, parallel moves Sep 18, 2026
@hhugo
hhugo marked this pull request as ready for review September 18, 2026 16:04
@pmetzger

Copy link
Copy Markdown
Member

Very cool!

hhugo and others added 2 commits September 19, 2026 21:04
A DFA state's [finals : bool array] becomes [accept : accept option],
with [accept = { rule; final_ops }]. For an accepting state it holds
the lowest-numbered accepting rule, which the PPX used to recompute
from [finals] in [best_final], and the tag operations to run on entry
just before [Sedlexing.mark]. The TDFA construction will use those to
copy the accepting path's working registers into the canonical cells
read by the binding extraction code; the compiler sets the list to []
for now, so generated code is unchanged.

Nothing driving the automaton needs the other accepting rules of a
state, so [finals] goes away; [accepting_rule] finds the winning rule
directly from the rules' final nodes. The DOT printer labels an
accepting state with that rule only. No expect output listed more
than one rule, so the graphs are unchanged.

[call_state] moves below [gen_tag_ops], which it now uses, and the DOT
printer shows final ops on accepting-state labels when there are any,
with [tag_op_to_string] hoisted to top level.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The TDFA construction emits register moves whose Copy operations must
all read the pre-transition contents: a transition may both copy cell
A into B and overwrite A. [gen_tag_ops] now let-binds every Copy source
that the same list also writes, via the new raw accessors
[__private__mem_get] / [__private__mem_set], and emits the writes in
list order. [op_dest] is exported for it. No Copy is emitted yet, so
generated code is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants