Skip to content

Add an oracle test harness: reference matcher vs compiled tagged DFA - #202

Open
hhugo wants to merge 2 commits into
ocaml-community:masterfrom
hhugo:pr3-oracle
Open

hhugo wants to merge 2 commits into
ocaml-community:masterfrom
hhugo:pr3-oracle

Conversation

@hhugo

@hhugo hhugo commented Sep 18, 2026 •

Copy link
Copy Markdown
Collaborator

Tests only: a differential test harness for the compiler, in a new test/oracle library that uses sedlex.compiler directly and involves no PPX.

Two independent implementations of sedlex's matching semantics run on the same (rules, input) pairs and their results are compared; a disagreement is a compiler bug.

  • Reference matcher (ref_match in test/oracle/oracle.ml): a brute-force backtracking matcher over Ir.t. It enumerates every parse in leftmost-greedy priority order (alternation prefers the left branch, Star/Plus/Rep prefer one more iteration) and keeps the first parse among those of maximal length; across rules the longest lexeme wins and the lowest-numbered rule breaks ties. End of input is a marker after the input, read at most once and adding no length (Read end of input once, and let priority decide its ties #211). This is the specification of capture semantics and the part to review. It is deliberately naive so that it is correct by inspection.
  • DFA interpreter (dfa_match): runs the output of Sedlex.compile_ir the way PPX-generated code does: memory cells start at -1, a transition's operations record the position before the code point that selected it (TDFA(1): delay the tag writes of a closure to the outgoing transitions #209), the final operations run on accept before the mark (Stop copying the capture cells in Sedlexing.mark and backtrack #208), end of input is fed once as the code point -1 without advancing, and on a dead end the last mark wins. Bindings are extracted as gen_binding_code does, discriminator cells included.

Entry points: oracle prints one line per input, prefixed with ERROR and both results on disagreement; qcheck is a deterministic random sweep (fixed seed, no shrinking) over sequences of up to four elements on the alphabet a..d plus eof, with at least one capture, plus or-pattern rules binding the same name in both branches, on inputs of length 0 to 5. Failing cases print through oracle with the pattern, so an empty expect block means full agreement.

The hand-written cases cover the basic capture shapes, or-patterns, greedy disambiguation, the loop-before-capture patterns of #199 and end of input (capture before eof, eof adding no length so rule priority decides, a tie within a rule, Plus eof / Star ('a' | eof) / eof, eof). Every expect block is clean on master; the harness is what was used to check #204, #207, #208, #209, #210 and #211 as they were written (millions of random cases locally, the sweeps here are sized for CI: about 1.5 s).

The interpreter is only as faithful as its imitation of the PPX glue, so a bug in that glue rather than in the compiler can escape it.

Commits:

  1. qcheck-core declared as a with-test dependency in dune-project and sedlex.opam.
  2. The harness (oracle.ml), the expect tests (test_oracle.ml), and the dune library, gated on OCaml >= 4.14 like the other expect tests.

🤖 Generated with Claude Code

hhugo and others added 2 commits October 1, 2026 12:28
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two independent implementations of sedlex's matching semantics are
compared on the same (rules, input) pairs; a disagreement is a
compiler bug.

- A brute-force backtracking matcher over Ir.t enumerates every parse
  in leftmost-greedy priority order (alternation prefers the left
  branch, repetition prefers one more iteration) and keeps the first
  parse among those of maximal length; across rules the longest lexeme
  wins and the lowest-numbered rule breaks ties. End of input is a
  marker after the input, read at most once and adding no length. This
  is the specification of capture semantics.

- An interpreter for Sedlex.compile_ir output runs it the way the
  generated code does: transition operations record the position
  before the code point that selected them, final operations run on
  accept before the mark, end of input is fed once, and bindings are
  extracted through the discriminator cells as gen_binding_code does.

Hand-written cases cover the capture shapes, or-patterns, greedy
disambiguation, the loop-before-capture patterns of ocaml-community#199 and end of
input. Two deterministic random sweeps over the alphabet a..d plus
eof print nothing when every case agrees.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant