diff --git a/changes/checker-rewrite-r3-skeleton-6.internal.md b/changes/checker-rewrite-r3-skeleton-6.internal.md new file mode 100644 index 0000000000..eeb9733a39 --- /dev/null +++ b/changes/checker-rewrite-r3-skeleton-6.internal.md @@ -0,0 +1 @@ +- Checker rewrite R3, sixth design turn: the record of the reviews of universal variables and packages, three corrections of built code with their tests (what two variant fields gain beside types that hold a universal variable, the walk for free universal variables below a polymorphic type without bound variables, the arguments of a row name that is no longer the innermost row's), and the design of pattern mode and its equations in two slices, with a harness, acceptance tests and build prompts. diff --git a/compiler-rust/docs/CHECKER_REWRITE.md b/compiler-rust/docs/CHECKER_REWRITE.md index 85deda8e05..77aeb7a262 100644 --- a/compiler-rust/docs/CHECKER_REWRITE.md +++ b/compiler-rust/docs/CHECKER_REWRITE.md @@ -2178,7 +2178,9 @@ it stops and reports a reproduction. | R3n | The references of a copied variant row: shared when the row variable is, new otherwise (8.6, decision 34) | `rows.rs`, `expand.rs` | R3e, R3f | R3g | 150 | | R3g | Universal variables and polymorphic types with variables: the scopes, `unify_univar`, `enter_poly`, the escape test, `occur_univar`, PR#6744 in `unify_row_field` (8.6, decisions 49 to 51; 8.8) | `rows.rs` | R3e, R3f, R3n | R3p | 600 | | R3p | Packages: the merge of two name lists, completed through `PackageDecls` (8.6, decision 52; 8.8) | `rows.rs` | R3e | R3g | 150 | -| R3h | Pattern mode, equations, `reify`, `mcomp` | `rows.rs`, `unify.rs` | R3n, R3g | R3i, R3j | 600 | +| R3h | Pattern mode: state, equations, `reify`, the arms without a backtrack (8.9) | `rows.rs`, `unify.rs`, `expand.rs` | R3n, R3g | R3i, R3j, R3q | 450 | +| R3r | Pattern mode: `mcomp`, the two arms that try and fall back (8.9) | `rows.rs`, `unify.rs` | R3h | R3i, R3j, R3q | 450 | +| R3q | `instance_poly` and `instance_label`: the copy that goes through free universal variables without sharing (`copy_sep`), and the fixed-row rule of `copy_row` only it reaches (8.6, decision 47; 8.8, open question 1) | `rows.rs`, `expand.rs` | R3n, R3g | R3h, R3i, R3j | 300 | | R3i | `moregeneral`, `equal`, `matches` | `relate.rs` | R3n, R3g | R3h, R3j | 600 | | R3j | Subtyping without the declaration-directed coercions | `relate.rs` | R3n, R3g | R3h, R3i | 600 | | K2 | The hot paths of unification, no behaviour change (8.7) | `unify.rs` | R3c | every slice above | 150 | @@ -2253,6 +2255,24 @@ Packages are R3p and not part of R3g: no generated case can judge them arm that is already wired, and they ask the declarations three questions that R5a answers. R3h was not started in this turn. +State after the sixth skeleton turn (trunk `61f7f41292`). Merged: every +slice above R3h in the table, R3g and R3p included (built as designed; +what their reviews put on record is decisions 53 to 57 of 8.6), and K1 to +K4. `cargo test --lib tc::` on that trunk: 265 passed, 0 ignored. At the end +of the sixth turn: 281 passed, 15 ignored (ten for R3h, five for R3r). Without a design +before this turn: R3h, the `instance_poly` slice (R3q: a slice name has +three characters), R3i and R3j. What the sixth turn did is listed in 8.6 +("Decided in the sixth skeleton turn"). + +| Slice | Decisions | Tests | Run in this turn | +|---|---|---|---| +| R3h | 61 to 67, 69, 71 | ten ignored in `r3_tests.rs`: nine hand-built and the step test `pattern_mode_at_ten_thousand` | none: no prototype was made; they compile against the skeleton and the check of the ignored tests passes. The harness and the old checker's side (decisions 61, 62) are nine tests that run today | +| R3r | 63, 68, 70 | five ignored: three hand-built and `mcomp_at_ten_thousand` in `r3_tests.rs`, the fixed cases and 12,000 generated cases in `differential.rs` (`pattern_plan_agrees_on_generated_cases`) | none, as above | + +After the sixth turn: designed and not built, R3h and R3r (8.9); without +a design, R3q, R3i, R3j and the ambiguity levels of GADT instances +(decision 62). + The ignored tests are checked mechanically (`differential::tests::ignored_tests_name_the_slices_they_reach`): each names the slices it reaches, reaches its own, and reaches none that is @@ -2556,6 +2576,9 @@ End state: no `todo!("R3p")`; the three ignored tests pass. One commit. Failure: a frame or a saved descriptor left after a clash in the merge; a supplier not put back; a `Vec` of pairs. +**R3h, R3r.** Functions, end state, commits and gates: section 8.9 +("The parts") and the two prompts of 8.4. + **R3h to R3j.** Functions: every signature of `rows.rs` and `relate.rs` carrying the slice's name. End state for each: no `todo!` with its name; its generator extension and old-side vacuity test (first commit); the @@ -3004,9 +3027,136 @@ for two equal paths; a change in unify.rs or outside rows.rs and the three ignore lines. At most 150 lines. ``` +Not issued as it stands: the review of this turn found corrections the +design owes first (8.9, "Corrections owed after review"). The prompt is +issued when they are in the document and the tests. + +```text +R3h. Read section 8.9 whole and section 8.6, decisions 47, 60, 61 and +62; compiler/ml/ctype.ml 253-273, 1478-1535, 1802-1881, 2085-2104, +2229-2440 and 2739-2750; in src/tc/unify.rs the doc comments of +PairSet, FrameKind::Flags, Checker::{unify_pattern, cut_equations} and +the bodies of unify_pair, unify_expanded, unify3 and run_frames; in +src/tc/rows.rs the doc comments of Equation, LocalDecls and +Checker::{reify, add_gadt_equation, non_recursive_abbrev, duplicate, +find_newtype_level, drop_name_children, copy_refs}; in src/tc/expand.rs +TypeTable::{instantiable, non_aliasable, injective}, LOCAL, and the body +of Checker::subst. R3g, R3n and R3p are merged. + +Commits, in this order, each pushed with + cargo test --manifest-path compiler-rust/Cargo.toml --lib tc:: +seen to pass: +1. The fixed-row rule of Checker::copy_refs (decision 47, last + paragraph), with the hand-built row of that decision as a running + test in src/tc/r3_tests.rs. +2. Checker::{find_newtype_level, non_recursive_abbrev, duplicate, + add_gadt_equation} in rows.rs, Checker::cut_equations in unify.rs, + and the equation arm at the top of Checker::subst in expand.rs + (decision 67). +3. Checker::reify in rows.rs (section 8.9, "reify"). +4. Checker::unify_pattern; `pattern` loaded once in unify_expanded and + passed to unify3, with the lookup of the table before unify3; the + arms of decision 64 marked 1, each `todo!("R3h")` of unify.rs + replaced; the Flags arm of run_frames; the arm of decision 65 in + unify_pair. The three places marked 2 in the table of decision 64 + become `todo!("R3r")`. Then the `#[ignore = "R3h build"]` lines of + src/tc/r3_tests.rs removed and nothing else in those tests changed. +End your turn after the fourth commit is pushed and no test is ignored +for "R3h build". + +Steps: one `#[cfg(test)] super::steps::tick()` per node `reify`, +`non_recursive_abbrev` and `duplicate` visit, and per entry +`cut_equations` pops; unify_pair's own tick counts the pairs. + +Relations to hold: every tc:: test that passed before passes +unchanged; pattern_mode_at_ten_thousand as written; after +unify_pattern, on either result, what +r3_tests::assert_pattern_left asserts. + +Report, before and after, the instructions of the tc_kernel bench +operations unify_vars, unify_ground1, unify_ground3, unify_datatype2 +and unify_abbrev2 (compiler-rust/docs/TC_KERNEL_R3.md, "Machine and +method"). No target is set (decision 46). Report also what +Checker::generalize does to a memo or to anything that is not a node's +level when `duplicate` calls it: if it writes something +`Graph::restore` does not put back, stop and report it. + +If a test of yours fails where your code does what the ctype.ml lines +say, stop and report the test with the lines: do not change the test. + +Shortcuts that count as failure: linking two structures in pattern +mode; an occurrence test for a structure pair in pattern mode; reading +`pairs` or `umode` in unify_pair or anywhere in expression mode beyond +the one load of unify_expanded; a recursive walk (use g.work and +Checker::marks with a new generation); a hash set or a `Vec` allocated +per call; cutting equations in Graph::restore or on a failed +unification; a body of an equation that shares a node with its +destination; writing a TypeTable row of an equation anywhere but +add_gadt_equation and cut_equations; a change in graph.rs, level.rs or +template.rs, or under src/types. At most 450 lines. +``` + +Not issued as it stands: the review of this turn found corrections the +design owes first (8.9, "Corrections owed after review"). The prompt is +issued when they are in the document and the tests. + +```text +R3r. R3h is merged. Read section 8.9, decisions 63, 64 (the rows +marked 2), 68 and 70 and "No supplier"; compiler/ml/ctype.ml 1886-2080, +2359-2371, 2400-2408 and 2416-2421; in src/tc/unify.rs the doc comments +of Try, FrameKind::Try and Checker::recover, and the body of +run_frames; in src/tc/rows.rs the doc comments of LocalDecls, +DeclMatch, Checker::{mcomp, push_compatible, enter_poly, unify_univar, +unify_row, unify_fields, flatten_row, merge_rows, unify_package} and +the body of Checker::leave_scopes in unify.rs. + +Commits, in this order, each pushed with the tc:: tests seen to pass: +1. Checker::{mcomp, push_compatible} in rows.rs (decision 70), and the + call of mcomp in the constructor arm of unify3 that R3h left + `todo!("R3r")`. +2. Checker::recover and the Try arm of run_frames in unify.rs: every + place where run_frames returns a clash first asks `recover(base)` + and goes on with the loop when it answers `Some(Ok(()))`; a + `Some(Err(_))` is a clash again and asks again. Then the argument + arm and the variant arm of unify3 (decision 64, the rows marked 2) + and the package arm of decision 63. +3. The `#[ignore = "R3r build"]` lines of src/tc/r3_tests.rs and + src/tc/differential.rs removed, nothing else in those tests changed. +End your turn after the third commit is pushed and no test is ignored +for "R3r build". + +Steps: one tick per pair `mcomp` takes off its stack. + +Relations to hold: every tc:: test that passed before passes +unchanged; mcomp_at_ten_thousand as written; `Checker::compatible` and +`Checker::tries` are empty when unify_pattern and mcomp return; +`g.speculation_depth` is what it was. + +If pattern_plan_agrees_on_generated_cases reports a seed, read the +ctype.ml lines of the arm first. If your code does what they say, stop +and report the seed with the lines: do not copy the old checker, and do +not pin the seed yourself. The old checker is not a judge of a pattern +unification that succeeds on two structures (decision 62), and +`differential::old_judges` already ends a case before one; a seed that +differs there means old_judges missed a link: report it. + +Report the same five bench operations as R3h, before and after. + +Shortcuts that count as failure: a nested call of Checker::unify for +an attempt (the attempt is frames above a Try frame); cutting +`equations` in recover; a second table of pairs for unification (the +table of mcomp is `compatible`, the table of unification `pairs`); +comparing declarations in the kernel (LocalDecls::compare is the only +reader of a declaration); answering compatible for two data types +without a supplier; calling leave_scopes anywhere but in recover and +mcomp's marker; a change in graph.rs, level.rs, template.rs or +expand.rs, or under src/types. At most 450 lines. +``` + The prompt of K2 is in section 8.7. The last prompt below is superseded by decision 5 of 8.6: no builder starts a slice from it. R3e, R3f, R3n, R3g -and R3p have their own prompts above it; R3h to R3j have none yet. +and R3p, R3h and R3r have their own prompts above it; R3q, R3i and R3j +have none yet. ```text R3e..R3j. First commit, before any body: extend @@ -4286,6 +4436,347 @@ unless it says "ignored". `Completion::Missing` are separate answers so that R3j can skip them without a second trait. +On record from the reviews of the fifth skeleton turn, of R3g and of R3p +(trunk `61f7f41292`). They bind the sixth skeleton turn and R5c. + +53. **`split_univars` puts the join off and runs it again.** OCaml + unifies the gained types that hold a universal variable and then + builds the two fields from the lists as they were before those + unifications (`List.iter (unify env tu1) (tlu1 @ tlu2)`, then `tl1'` + and `tl2'` as already computed, ctype.ml:2693-2706). The kernel + returns `Ok(None)`, lets the frame stack unify the holders, and runs + `join_either` from its start: the two type lists are read again and + compared again. The field holds the same SET of nodes either way. The + two differ in one corner: a gained type WITHOUT a universal variable + that those unifications make the same node as a type the gaining side + already has is not gained by the second run, and is kept, as a + duplicate entry of the list, by OCaml. Accepted as built. It is + reopened by a case in which a printed type or a later join shows the + duplicate; the sixth turn built the corner by hand on the old side + (decision 58). +54. **`join_either` lowers the two gained ranges each on its own.** With + `passive_variants` set and holders on both sides, `split_univars` + returns two ranges with a gap between them (what its compaction left + behind: holders, which OCaml does not lower, ctype.ml:2699-2703). The + caller lowered everything from the start of the first range to the end + of the second; corrected in the review of R3g. Nothing in `tc` sets + `passive_variants`, so no test reached the correction until the sixth + turn wrote one by setting the field (decision 59). +55. **`package_end` reads the right node's descriptor without following + a link.** For the right node to be linked away between the two frames + it would have to be nested in an argument of one of the two packages, + and every such shape ends in an occurrence clash before the + `PackageEnd` frame is popped. Whether pattern mode changes that is + answered by the design of R3h (decision 63: it does not, and why). +56. **A mark of `free_univar` is live while its context is open, and a + polymorphic type WITHOUT bound variables opens a context.** A node + first visited inside such a type is visited again after the type is + left, where OCaml's `visited` map, which compares the sets of bound + variables, does not visit it again (`bound' ⊆ bound` holds for two + equal sets, ctype.ml:1572-1581). The answer is the same; the visits + are more. The count is stated and asserted by the sixth turn's step + tests (decision 59). +57. **The bench record of R3g, for R5c.** The five unification operations + of `tc_kernel` at the builder's start and at the merged head: + `unify_vars` 118 and 122, `unify_ground1` 120 and 120, `unify_ground3` + 807 and 807, `unify_datatype2` 815 and 815, `unify_abbrev2` 2885.06 + and 2886.06. Tried and reverted, each measured: `#[inline(never)]` on + the five bodies (no figure moved) and `#[cold]` on the seven rare + functions (every operation dearer than the merged head). What moved + the figures between the two heads was a merge of trunk that changed + none of these bodies: code placement under one codegen unit. For + R3p, `#[cold] #[inline(never)]` on its two bodies put all five + figures back at their base, and is kept. Consequence, binding every + later prompt: no slice has a target for a `tc_kernel` figure + (decision 46), and **no prompt prescribes a remedy for a figure that + moves**; a build prompt asks for the five operations before and + after, as a report. + +Decided in the sixth skeleton turn (trunk `61f7f41292`), each with what +was run. Decisions 58 to 60 are corrections and tests of BUILT code that +the reviews above left owed; every test named runs today. + +58. **`split_univars` keeps what OCaml keeps: decision 53 is reopened + and closed.** The corner was built by hand on the old checker + (`differential::tests::old_checker_gains_a_type_the_holders_made_a_ + duplicate`): `poly([< #A((u, x)) & x]; u)` against `poly([< #A((u', + x')) & x']; u')`. The two tuples hold a universal variable and are + unified, which makes `x'` and `x` one node; each field then holds + THREE types, the last two one node (`#A?^0((!, ') @4 @4)`). The OCaml + compiler was run on a source program that reaches the corner: + + ```rescript + let f = ( + o: {"m": 'u. [< #A(('u, 'a)) & ('a)] => unit}, + p: {"m": 'u. [< #A(('u, 'b)) & ('b)] => unit}, + ) => (if true { o } else { p }, o, p) + ``` + + With a counter added to `unify_row_field` (a local build, not + committed) the join runs once with two types a side, the split keeps + one and holds one a side, and after `set_row_field` both fields read + three types. So OCaml has the duplicate; the kernel, which compared + the two lists again after the holders were unified, had two types. + Can it show? A printed type lists every type of a `Reither` field + (`tree_of_row_field`, printtyp.ml:730-737), and a later join reads + the length of the list: the arm of PR#7496 needs `List.length tl1 = + List.length tl2` (ctype.ml:2652-2655) and `rigid_variants` reads + `List.length tl1 = 1` (ctype.ml:2667-2668), so a field of three + types and a field of two can take different arms against the same + third field. That is enough to correct it. (The diagnostic of the + program above, with `let g: int = f` appended, prints two types for + the tag in OCaml although the copies it is made from hold three; what + drops the third between the copy and the printed text was not + established, and is not needed for the decision.) + The correction, in `rows.rs`: the arm of `split_univars` with holders + on both sides writes the types each side KEEPS after the pair's + record in `saved` (one entry with the two counts, one entry a type; + 16 bytes each, only in this arm), and the second run of + `join_either` reads them back instead of comparing the two lists + again. Test: `r3_tests::a_field_gains_what_it_lacked_before_the_ + holders_were_unified` (both modes; three types, the last two one + node; it fails without the edit, observed: two types). +59. **`free_univar` opens no context for a polymorphic type without + bound variables; the step tests of the two walks; the passive + split.** + - *A fault of built code, found by the step test decision 56 asked + for.* Every object field is a `Tpoly (t, [])`. With a context per + such type, a node first met below one was visited again after it + was left, and the second visit met the types below it in closed + contexts again: on `t(i + 1) = (poly(t(i);), t(i))` the walk took + `2^(n + 1) - 1` visits (observed without the edit: 2,047, 4,095 + and 2,097,151 for n = 10, 11, 20). OCaml visits each node once: + such a type adds nothing to `bound` (ctype.ml:1587-1589), and a + node visited with a `bound` is not visited again with the same one + (ctype.ml:1569-1581). A type of that shape is what a value used + twice gives (`(x, {"m": x})`), and the walk runs at every binding + of a variable to a type with children once the unit has ANY + universal variable. Corrected: the `Poly` arm of + `push_free_univar_children` pushes the body of a type of arity 1 + in the current context. The extra visits are now ZERO, stated and + asserted: exactly `2n + 1` visits at n = 1,000 and n = 10,000 + (`r3_tests::the_walks_for_universal_variables_at_ten_thousand`, + which also has the linear relations of `free_univar` over a wide + type and of `univars_escape` with a known binder). + - *Left as it is, and why.* A polymorphic type WITH bound variables + still opens a context, and a node shared between the inside of two + sibling types with binders, or between the inside and the outside + of one, is visited once per context it is met in. OCaml visits + such a node again too, once per strictly smaller `bound` + (`Type_set.inter bound bound'`, ctype.ml:1574-1576: "that's way + too expensive", ctype.ml:1562). On a chain of n nested types WITH + binders that share their bodies with the outside the kernel's + visits double per type where OCaml's grow with the number of + distinct `bound` sets. No program has shown it: a polymorphic + method whose type shares a node with a type outside it needs a + value that is not generalized inside an annotated method type. + Open question 3 of this turn. + - *A row walked by `free_univar`*: `r3_tests::a_free_universal_ + variable_is_found_in_a_row_in_order` (fields, then the name's + arguments, then the row variable; an opened `Reither` field through + its chain; nothing below a reference resolved to absent). + - *The passive split*: `r3_tests::passive_variants_keep_the_levels_ + of_the_types_that_hold_a_universal_variable` sets + `Extras::passive_variants` by hand, with holders on both sides and + the holder AFTER the kept type in each list: the holders stay two + nodes at their level, the kept types are lowered to the merged row + variable's level. With the caller's loop as it was before decision + 54 the test fails (observed: a holder at level 1). +60. **The arguments of a row's name are children only of the innermost + row, in every walk.** OCaml's `iter_row` visits them only when + `row_more` is not itself a variant (btype.ml:229-241), every walk of + a type goes through `iter_type_expr` (btype.ml:263-265), and a copy + takes the name of the innermost row (`row_repr`, btype.ml:126-135). + The kernel's walks push every slot of a node. Before this turn: + + | Walk | Kernel | Visited them for a row whose `more` is a variant | OCaml | + |---|---|---|---| + | lowering | `Graph::lower`, `Checker::lower_scoped` | yes | no: `update_level`, ctype.ml:625-626 | + | generalization | `Graph::generalize`, `generalize_structure` | yes | no: ctype.ml:509-516, 524-540 | + | occurrence of a structure pair, of a binding | `Checker::reaches`, `occur_expanding`, the scan of `binding_scan` | not entered: a row stops the walk | not entered: `occur_rec`, ctype.ml:1403 | + | deep occurrence | `Checker::reaches` with `deep`, the routed `binding_scan` | yes | no: `deep_occur`, ctype.ml:1735-1782 | + | the lowering part of a binding | `binding_scan`, `LEVEL_ONLY` entries below a row | yes | no: `update_level` | + | free universal variable | `Checker::free_univar` | yes | no: `occur_univar`, ctype.ml:1601 | + | escape of a universal variable | `Checker::univars_escape` | yes | no: ctype.ml:1648 | + | closedness of an expansion | `Checker::closed_over` | yes | no: `closed_parameterized_type`, ctype.ml:441-451 | + | copy, instance | `Graph::copy`, `Graph::instance` | copied as slots of the copy | not copied: `copy_row` of `row_repr`, ctype.ml:800-868 | + + Observed on the old checker with `[> #B]` unified with `[> #A] as + p`, `y` in no field (`differential::tests::old_checker_does_not_ + reach_the_arguments_of_a_stale_row_name`): a lowering, a + generalization, `deep_occur`, a copy and a binding all leave `y` + alone. What can show it: the level of a type that is the argument of + a phantom parameter of a named open row, after the row gained a tag. + Corrected on the write side, so that no walk pays a descriptor load + per variant node: when `unify_row` has run both `set_more`, each row + that is not fixed has a variant node as its `more` (ctype.ml:2616), + and when such a row has a name with arguments its home node is + written with a new list in which the slots of those arguments hold + what the `more` slot holds (`Checker::drop_name_children`; in the + explaining mode a `Fields` frame puts the descriptor back on a + failure, as it puts back the rest whose link made the name stale). + Nothing reads the name of a row that is not the innermost. Cost: two + compares per unified pair of rows; for a named row with arguments + that gains a tail, one list of the node's slots. Test: + `r3_tests::the_arguments_of_a_stale_row_name_are_not_children`, one + block per walk of the table, in both modes (without the edit the + lowering already reaches `y`, observed). Two other places link a row + variable to a variant node, and each calls `drop_name_children` for + the row when it is built: `reify` (ctype.ml:1848-1852, R3h; 8.9) and + `rigidify_rec` (ctype.ml:3139-3147, R3i). + Found on the way and NOT corrected: `update_level` drops the name of + a row whose path would escape (`Some (p, _tl) when level < get_level + env p -> ... row_name = None`, ctype.ml:618-623), and + `Checker::lower_scoped` has no such arm. It needs a named row whose + type has a scope, which no generated program has. Open question 4 of + this turn. + +61. **The harness of pattern mode: the plan bit `PATTERN`.** + `differential::PATTERN` (bit 9 of a plan) makes a case a case of + pattern mode. What the bit adds: + - *the program*: `type n0`, `type n1`, `type k<'a>` and + `type d<'a> = D('a)`, and after the generated values four values + whose parameters are the pairs R3h is about + (`differential::PATTERN_VALUES`): a locally abstract type against a + structure and against the other one; constructors of one path with + arguments, abstract (`k`) and a data type (`d`), over a locally + abstract type, over `int` and over a variable; arrows over them. + With `VARIANTS` two more (`PATTERN_ROWS`): rows that clash and are + not compatible (a tag the other closed row lacks, a constant tag + against one with an argument) and rows that unify under an + equation; a row that clashes and is compatible comes from `reify` + (a fixed row against an open row with one more tag). There is one + data type, so two different data types are never compared and the + supplier of declarations answers `Clash` (8.9); + - *both sides*: `n0` and `n1` are locally abstract types of two + newtype levels, the level the case starts at and one above it. The + old side binds them again as local identifiers whose declarations + have `type_newtype_level`; the new side sets `NEWTYPE` and a + `Newtype` row on their `TypeTable` rows. The old side sees the unit + from outside, as `Case`: its types are paths through a module and + only the predefined types are bare identifiers + (`in_current_module`). The new side says the same with the `LOCAL` + flag, set on the predefined types only; + - *the operation* `Op::Gadt(left, right, flags)`: one top-level + unification in pattern mode with the current level as the newtype + level; bit 0 of `flags` is `generate_equations`, bit 1 + `assume_injective`. `unify_gadt` itself is 3; 1, 0 and 2 are the + modes unify3 installs below it (ctype.ml:2352, 2368) and that no + top-level call of OCaml asks for: `Checker::unify_pattern` takes + the two flags so that a case reaches each. Half of the pattern + unifications of a generated case are over two parameters of a new + instance of a pattern value, the rest over any two operands; + - *the form*: both sides end it with the equations in force, one + line a type that has one, sorted by the type's name: `= name := + body`, the bodies numbered together. The old side prints the + manifests of the local constraints of the environment the last + successful pattern unification left; the new side instantiates the + manifest of every entry of `Checker::equations` at the generic + level. A type `reify` made prints by its name (`$0`, `$1`), which + the new side takes from the supplier of the harness + (`differential::FreshNames`, the rule of `get_new_abstract_name`, + ctype.ml:908-922). + - *the deliberately wrong side* `Fault::ExpressionMode`: a pattern + unification runs as `Checker::unify`. + + Running today: `the_two_sides_agree_on_pattern_programs_without_a_ + pattern_unification` (the self-test: the fixed program and 300 + generated cases with the pattern unifications taken out agree), + `old_side_of_the_pattern_plan_is_not_vacuous` (observed, 1,200 + cases: 950 pattern unifications that succeed and 493 that fail + among the judged operations; 130 cases end with an equation, 3 with + two, 183 with a reified type) and + `generated_cases_find_a_pattern_unification_in_expression_mode` (the + wrong side needs nothing of R3h: of 228 cases with a pattern + unification it is reported in 21 that have an equation on the old + side and in 58 where only it fails). + +62. **What the old checker does in pattern mode, and where it is a + judge.** Observed on `differential::tests::PATTERN_PROGRAM`; each + item names its test in tc/differential.rs. + - *Which pairs get an equation, and its direction* (a judge): + `old_checker_gives_the_equation_to_the_newer_locally_abstract_type`. + A locally abstract type against a type that is not one gets the + equation on either side (ctype.ml:2385-2392) and the two nodes + stay two nodes. Of two locally abstract types the one of the + higher newtype level gets it (ctype.ml:2373-2384). Without + `generate_equations`: none, and a success (`reify`, no `mcomp`). + `int` against a tuple with equations: a clash (`mcomp`). After + `n0 := int`, `n1` against `n0` gives `n1 := int`: the body is the + expanded head. + - *The arguments of one path* (a judge): + `old_checker_refines_the_arguments_of_a_constructor_by_its_ + injectivity`. With `assume_injective` the arguments are unified + with equations; below that an abstract type that is not of the + unit gets none and what does not unify is reified, and a data type + gets them. A later expression unification of two `k` nodes that + pattern mode assumed equal fails: nothing of the assumption is + left after the top-level call. + - *What `reify` makes* (a judge, also of the names of variables + without one): `old_checker_reifies_a_variable_and_a_row_variable`. + A variable at the newtype level is a link to a new abstract type + `$0` at the variable's level; a row variable is a link to a fixed + row without tags over `$0`; the second is `$1`. A variable with a + name (`$'a`) is not reachable from an operation: tested by hand. + - *What a failure leaves* (a judge of the graph): + `old_checker_keeps_what_reify_linked_when_a_pattern_unification_ + fails`. `reify` links before it refuses a variable below the + newtype level, and the row a clash reified stays fixed when + `mcomp` refuses the pair: unify3's handler puts back only the left + node's description (ctype.ml:2437-2439). A failure after an + equation was added leaves the graph as it was. The caller that + goes on restores a snapshot, the levels and the environment it + kept (`set_state`, compiler/ml/typecore.ml:1299-1305); the kernel's + caller restores a mark and cuts the equations + (`Checker::cut_equations`, 8.9). The harness drops the old side's + environment of a failed pattern unification, so it does not judge + the equations a failed one added. + - *Rows* (a judge): + `old_checker_accepts_rows_that_clash_and_are_compatible`. + - *Two structures unified in pattern mode are ONE node on the old + side* (NOT a judge): + `old_checker_links_the_structures_a_pattern_unification_unifies`. + OCaml records the pair as assumed equal and links neither + (`Pattern -> add_type_equality t1' t2'`, ctype.ml:2327-2331). The + old checker links them: two tuples of two instances are one node + afterwards, and of `[#A(n0) | #B]` against `[#A(int) | #B]` the + left row prints `int`: a type equal only under the equations of + one case has become the other for good. Three source programs + were tried (a constructor whose result names a tuple, the same + through a tuple pattern, the same through an abbreviation) and + both compilers accept all three: no source program is known that + shows it. `differential::old_judges` gives the number of + operations of a case before the first pattern unification that + succeeds and leaves a structure reachable from an operand a link; + `compare` runs both sides on that prefix under `PATTERN` (observed: + 64 of 1,200 cases are cut). So the generated cases judge failures + of structure pairs in pattern mode and never their success; the + hand-built tests of 8.9 do, from the OCaml lines. + - *Ambiguity is not checked on the old side* (NOT a judge, and not + in R3h): OCaml refuses + `type rec t<_> = Int: t` / + `let f = (type a, x: t, y: a) => switch x { | Int => if true { y } else { 0 } }` + with "This instance of int is ambiguous: it would escape the scope + of its equation" (`trace_gadt_instances`, `Env.gadt_instance_level`, + `add_gadt_instance_chain`, ctype.ml:176-201, 1220-1227, 2289-2298; + the report is typecore's). The Rust compiler accepts it (observed + with `compile_source_in_process`): the old checker has the tracing + flag and no instance levels. The kernel has neither after R3h; it + is a slice of its own after R3h (open question 1 of this turn). + - Not observed to differ, and not judged: the arm of `unify` for two + locally abstract types under local constraints, which expands the + newer one once (ctype.ml:2258-2270), is missing on the old side; + it expands both heads, and the equations that result were the + same on every case tried. + +63 to 71. **The design of R3h** is section 8.9, where each is written + with its reasons: 63 `package_end` in pattern mode; 64 what replaces + the link, per arm of unify3; 65 the arm of `unify` for two locally + abstract types; 66 the declaration facts; 67 an equation, its + storage and its cut; 68 the two arms that try, as frames; 69 `occur` + in pattern mode; 70 `mcomp` and the answer without a supplier; 71 + what a unit that never types a pattern pays. + ### 8.7 The hot paths of unification (K2) @@ -4595,6 +5086,438 @@ it follows and names its witness. binder nested in a binder is a method inside a polymorphic field, and the depth is the nesting of the source. +### 8.9 Pattern mode, equations, `reify` and `mcomp` (R3h) + +What 8.6 decided for this slice: the harness and where the old checker +is a judge (decisions 61 and 62). This section is the design; decisions +63 to 71 of 8.6 are its numbered decisions, written here so that they +are read with their reasons. The work is cut in two slices, **R3h** +(state, equations, `reify`, the arms that need no backtrack) and +**R3r** (`mcomp`, the two arms that try and fall back; a slice name +has one letter, which the check of the ignored tests reads), built in +that order, each under 450 lines. + +**State.** Five things, all on `Checker`. + +| state | where | written by | read by | +|---|---|---|---| +| `umode` | `Checker::umode` | `unify_pattern` (entry, exit) | `unify_expanded`, once a call | +| `generate_equations`, `assume_injective` | two bytes of `Checker` | `unify_pattern`; the `Flags` frame and the `Try` record put back what they found | unify3's pattern arms | +| newtype level | `Checker::newtype_level`, 0 outside | `unify_pattern` | `reify`, `add_gadt_equation` | +| assumed equalities | `Checker::pairs` (`PairSet`) | unify3 in pattern mode | `unify_expanded` in pattern mode | +| equations | `Checker::equations`, and the `TypeTable` row of each | `add_gadt_equation`, `cut_equations` | the row: `expand_once`; the length: `unify_pair`'s arm for two newtypes | + +*The table of assumed equalities* (`unify_eq_set`, +compiler/ml/ctype.ml:2102-2104). Key: the two node indices of the pair, +the smaller first, in one `u64` (`order_type_pair`, ctype.ml:2104; the +kernel's index is OCaml's `id` for this purpose: a total order on +nodes). It is a hash set: pairs of nodes have no dense index. Size: 8 +bytes a pair plus the set's control bytes; empty and unallocated for a +unit that never types a pattern. Cleared when `unify_pattern` returns, +on either result (ctype.ml:2746, 2749): `if !keys.is_empty() { +keys.clear() }`, which is one byte a bucket of the capacity the largest +pattern unification of the unit left. A pattern unification of 10,000 +structure pairs costs 10,000 inserts and at most 20,000 lookups; the +next one, of one pair, costs one clear of that capacity (16 KiB of +control bytes written), and that is the only cost that outlives the +call. Never read or written in expression mode. + +**63. `package_end` reads the right node as it does today.** In pattern +mode no structure node is linked, so the right node of a `PackageEnd` +frame is the node the frame was pushed with, and decision 55's argument +is not needed. In expression mode nothing changed. The one difference +of pattern mode for packages is the arm of `Not_found` +(ctype.ml:2417-2421): where `unify_package` answers a clash because a +completion was not `Completion::Type`, pattern mode reifies the types of +both lists, left list first, and succeeds (R3r). + +**64. What replaces the link, per arm of unify3.** `unify_expanded` +loads `umode` once and passes `pattern: bool` to unify3 (the load unify3 +makes today moves; the count of loads of a unification in expression +mode does not change). In pattern mode `unify_expanded` answers `Ok` +before unify3 when the pair of the two expanded heads is in the table +(`unify_eq`, ctype.ml:2233, 2286); the two tests `unify_eq t1 t1'` and +`unify_eq t2 t2'` that choose the swapped call (ctype.ml:2299) stay +tests of identity: a pair in the table is two heads that were distinct +when unify3 took them, and a node and its own expanded head are never +that (an expansion is made from the node, after it stopped being a +head). The first three arms of unify3 (a variable on either side, two +universal variables, two fields) are as in expression mode, the +occurrence test of a variable included (`occur !env t1' t2`, +ctype.ml:2316-2323). Then, with `pattern`: + +1. the pair `(t1', t2')` goes into the table (`add_type_equality`, + ctype.ml:2329). No occurrence test (decision 69), no link, no + `Linked` frame, nothing in `saved`: a failure has nothing of this + node to put back (`t1'.desc <- d1` writes what is there); +2. `create_recursion` is computed and its `Recursion` frame pushed as + today (ctype.ml:2308, 2428-2436: the repair is outside the match on + `umode`); +3. the arm, by the two heads, first that applies: + +| heads | `generate` | what runs | part | +|---|---|---|---| +| arrow, arrow of one length | any | labels match when equal or when neither is optional (ctype.ml:2337-2343); the pairs of the parameters, then of the results | 1 | +| tuple, tuple | any | the pairs of the elements | 1 | +| constructor, constructor, one type | off | the pairs of the arguments | 1 | +| the same | on, `assume_injective` on | a `Flags` frame, the flags become (on, off), the pairs of the arguments above it (ctype.ml:2352-2354) | 1 | +| the same | on, injective off, the type's row has `LOCAL` or `DATATYPE` | the pairs of the arguments (ctype.ml:2355-2358; decision 66) | 1 | +| the same | on, otherwise | per argument, last first: when `TypeTable::injective` the pair; otherwise a `Try` frame (flags (off, off), fallback: reify both) and the pair above it (ctype.ml:2359-2371) | 2 | +| nullary constructors of two types, both `instantiable` | on | `add_gadt_equation`: the source is the left type when its newtype level is greater (`find_newtype_level`, compared as pairs), otherwise the right one; the destination is the other side's head (ctype.ml:2373-2384) | 1 | +| nullary `instantiable` constructor, anything | on | `reify` of the right head, then `add_gadt_equation(left type, right head)` (ctype.ml:2385-2388) | 1 | +| anything, nullary `instantiable` constructor | on | `reify` of the left head, then `add_gadt_equation(right type, left head)` (ctype.ml:2389-2392) | 1 | +| a constructor on either side | any | `reify` of the left head, of the right head; with `generate` on, `mcomp` of the two (ctype.ml:2393-2397) | 1; `mcomp` 2 | +| object, object | any | `unify_fields` | 1 | +| variant, variant | any | `open_row` of both, a `Try` frame (flags unchanged, fallback: reify both, and `mcomp` with `generate` on), `unify_row` above it (ctype.ml:2400-2408; decision 68) | 2 | +| `nil`, `nil` | any | nothing | 1 | +| polymorphic types | any | as in expression mode | 1 | +| packages | any | `unify_package`; decision 63 | 1; the fallback 2 | +| anything else | any | a clash | 1 | + +A `Flags` frame (`FrameKind::Flags`) holds in `extra` the two flags as +they were (bit 0 `generate_equations`, bit 1 `assume_injective`) and +puts them back when it is popped; a clash leaves them to the `Try` +record or to `unify_pattern`, which put back their own. + +The rows are in OCaml's order; "left" and "right" are unify3's own +`t1'` and `t2'`, after the swap of unify2, so an equation's direction +does not depend on which side was expanded. Part 1 leaves +`todo!("R3r")` in the three places marked 2 and in `mcomp`. + +**65. The arm of `unify` for two locally abstract types** +(ctype.ml:2258-2270) is in `unify_pair`, after the arm for two nullary +constructors of one type, in every mode: when both heads are nullary +constructors (in registers), `equations` is not empty (one load, made +only for such a pair) and both rows are locally abstract types +(`flags & (NEWTYPE | DATATYPE | PRIVATE) == NEWTYPE`: `is_newtype`, +ctype.ml:1862-1868): `expand_once(·, ExpandMode::Unif)` of the right +node when the left type's newtype level is lower, otherwise of the left +node; when it answers `NONE` (`Cannot_expand`) the pair goes to +`unify_expanded` as it is; otherwise the pair of the unexpanded side and +the expansion is pushed with `push_pair` and the arm returns `Ok`: the +pushed pair is the nested `unify` of OCaml, with its own entry in the +trace. Cost for a unit without equations: one load for a pair of two +nullary constructors of different types, nothing for any other pair. + +**66. Declaration facts.** Held today and read by this slice: +`NEWTYPE` and the `Newtype` row (`type_newtype_level`), `DATATYPE` +(`is_datatype`: a record, a variant or an open type), `ABBREV`, +`PRIVATE`, `ALIAS` (a manifest), `scope` (`Path.binding_time` for a +type that is not a newtype: `get_level`, ctype.ml:562-569, and +`find_newtype_level`, ctype.ml:2085-2091, read the same thing). Added: + +- `LOCAL` (bit 7 of `TypeInfo::flags`): `in_current_module`. Set by + the caller that owns the table (`TypeTable::set_local`; R5a), not by + `from_segment`: a segment does not say whether it is the unit being + checked. `TypeTable::non_aliasable` and `TypeTable::instantiable` are + built and read it. +- `TypeTable::injective_bits: Vec`, indexed by type index, bit + `i` the parameter `i` (`Variance.(mem Inj)` of `type_variance`). + Empty, or shorter than the index, reads as not injective, which is + what an abstract type without an annotation is (observed: `k`, + decision 62). Read only by the last constructor arm, for a type that + is neither `LOCAL` nor `DATATYPE`. A parameter past the 32nd reads as + not injective. R5a fills it; the tests of R3h write it by hand. +- `expands_to_datatype` of `t1'`, `t1` and `t2` (ctype.ml:2356) is the + `DATATYPE` flag of the row of the two heads' type: the heads are of + one type, and `t1` and `t2` expand to them, so the three chains end + at that row; a type on the way that is a data type with a manifest + (`type t = M.t = A | B`) names the same constructors as its head. + +`find_newtype_level(type_index)`: the `Newtype` row's `(level, +expansion_level)` with `NEWTYPE`, otherwise `(scope, scope)`. + +**67. An equation.** `add_gadt_equation(source, destination)` +(ctype.ml:2093-2100): + +1. `non_recursive_abbrev(source, destination)` + (`local_non_recursive_abbrev`, ctype.ml:1478-1535: the type does not + name `source` through its children or through the expansions of the + constructors it names). False: nothing happens, no clash. A walk + with marks and `g.work`, expanding with `expand_once(·, + ExpandMode::Opt)` (`try_expand_once_opt`), O(nodes reached); +2. `duplicate(destination)`: the generic copy (`duplicate_type`). Made + from parts that exist: `g.speculate()`; with `g.level` 0, + `generalize(destination)`, which makes every node of it generic; + with `g.level` generic, `copy(destination)`; `g.level` put back; + `g.restore(token)`. The copy's nodes are new and are not on the + trail, so they stay; the destination has its levels again. Three + passes over the destination; +3. a `Manifest` row `{ template: an empty template, params: 0, body: + the copy }`. A body that is a node of the graph and not a + `RELATIVE` reference marks an equation: `subst` gets one arm at its + top for it, which copies the body at the node's level + (`Checker::copy` with `g.level` the node's), memorizes the copy and + returns it (one compare of a loaded field for every other + expansion); +4. `equations.push(Equation { type_index: source, previous: the row + })`, then the row gets `ABBREV` and the manifest. Its `NEWTYPE` and + `Newtype` row stay (`new_declaration (Some source_lev) (Some + destination)`), so the type is still a locally abstract type for + decision 65 and no longer `instantiable`; +5. `cleanup_abbrev()`. + +`cut_equations(len)` pops the entries above `len`, last first, writing +each `previous` row back, and calls `cleanup_abbrev()` when it popped +one. It is what a caller does where OCaml drops the environment of a +case (`set_state`, typecore.ml:1303-1305, and the end of a case). The +`Manifest` rows and the copies are not reclaimed: 20 bytes and the +body's nodes for every equation a unit ever adds. Nothing of an +equation is on the graph's trail: `Graph::restore` does not cut +equations, and a caller that restores a mark also cuts. Decision 68 +says why the kernel's own backtrack must not. + +**`reify(ty) -> Result<(), Clash>`** (ctype.ml:1813-1859). The +signature changes: it can fail. A walk with a new mark generation and +`g.work`, children pushed so that they pop in `iter_type_expr`'s order +(the fresh names are numbered in visit order). Per representative not +yet marked: + +- a variable: `fresh = locals.fresh_abstract(name)`; the `TypeTable` + row of `fresh` is pushed (`rows.len()` must equal `fresh`: the + supplier counts with the table) as abstract with `NEWTYPE` and a + `Newtype` row of `(newtype_level, newtype_level)`, scope + `newtype_level`; a constructor node of it at the variable's level; + `g.link(variable, node)`; then, when the variable's level was below + `newtype_level`, a clash whose trace pair in `Mode::Explain` is the + new node and a new variable at the variable's level + (ctype.ml:1837-1839). The link stays (decision 62); +- a variant: unless the row is static, for a fixed row the walk goes + on into `more`; otherwise `more` is a variable (anything else is an + invariant failure) and becomes, through `g.link`, a variant node at + its level over a new `VariantRow` without tags, `fixed`, not closed, + whose `more` is the fresh constructor made as above; then + `drop_name_children` for the row (decision 60) and the same level + test. Then the row's children in `iter_row`'s order; +- a constructor whose path is an object type (`is_object_type`): no + kernel type has such a path; not built; +- anything else: its children. + +No supplier (`locals` is `None`) and a variable to reify: a panic that +names `Checker::locals`; a caller that types a pattern installs one. + +**68. The two arms that try** (the argument of a constructor that is +not injective, ctype.ml:2362-2371; two variants, ctype.ml:2400-2408) +are frames, not recursion. A `Try` record in `Checker::tries` (40 +bytes; empty outside pattern mode) holds the index of its frame, the +`Speculation`, the lengths of `trace`, `saved` and `scopes`, the two +flags as they were, and whether `mcomp` follows the reification; its +frame `FrameKind::Try` holds the two types and the record's index. What +is tried is pushed above the frame. + +- The frame is popped (everything above it succeeded): `g.keep`, the + flags put back, the record popped. +- `run_frames` gets a clash from any callee: `recover(base)`. No + record whose frame is at or above `base`: the clash is the caller's, + as today. Otherwise the last record is taken: `frames` is cut to its + frame, `Checker::restore` of its speculation, `leave_scopes` to its + scopes, `trace` and `saved` cut to its lengths, the flags put back; + then `reify(left)`, `reify(right)` and, when the record says so and + `generate_equations` is now on, `mcomp(left, right)`. A clash of + these three is a new clash, handled the same way; otherwise the loop + goes on. + +In both modes of the graph: `Mode::Success` logs to the trail while a +speculation is open and stops when the last one is kept; `Mode::Explain` +has its own speculation below. The descriptors a `Linked` or `Fields` +frame above the `Try` frame would put back are put back by the restore. +`leave_scopes` is the one place that cuts scopes (decision 51): the +record keeps the length it found and `recover` calls it. +What the backtrack does NOT undo, as in OCaml, where `env` is a +reference that `backtrack` does not know: an equation added inside the +attempt of two variants stays in `equations`, and so does a type +`reify` made. Only the graph goes back. + +**69. `occur` in pattern mode.** Only the two variable arms of unify3 +test occurrence (ctype.ml:2316-2323), and they are shared. A structure +pair is not tested (`occur` is in the `Expression` branch, +ctype.ml:2327-2329): nothing is linked, so no cycle is made. +`add_gadt_equation` has its own test (step 1 of decision 67), and a +type that fails it gets no equation without a clash. + +**70. `mcomp(left, right)`** (ctype.ml:1905-1984). Never links, never +lowers; it expands (`expand_head(·, ExpandMode::Opt)`), which +memorizes. Its own table `Checker::compatible` (a second `PairSet`; +ORDERED pairs, `Type_pairs.find type_pairs (t1', t2')`), emptied when +it returns; its stack is `g.work`, two entries a pair. Per pair, after +`repr`: one node, a variable on either side, or two nullary +constructors of one type: compatible. Otherwise both heads expanded; +one node or a pair in the table: compatible; else the pair is added +and, by the heads: + +- arrows of one length whose labels match as in decision 64: the + parameters and the results; tuples of one length: the elements; +- two constructors (`mcomp_type_decl`, ctype.ml:2011-2039): a type + without a row: compatible. One type: the pairs of the arguments + whose parameter is `TypeTable::injective`. Two types, both + `non_aliasable`: a clash. Otherwise by the two rows: both + `DATATYPE`: the pairs of the arguments (a clash when the lists + differ in length), then `LocalDecls::compare`, which pushes the + pairs of the member types with `push_compatible` or answers + `DeclMatch::Clash`; neither `DATATYPE`: compatible; one `DATATYPE` + and the other not `non_aliasable`: compatible; else a clash; +- a constructor against anything else: a clash when its row is + `non_aliasable` or `DATATYPE`, else compatible; +- packages, `nil`: compatible. Objects: `mcomp_fields` + (ctype.ml:1969-1981) over `flatten_fields` and the merge + `unify_fields` uses. Variants: `mcomp_row` (ctype.ml:1985-2009) over + `flatten_row` and `merge_rows`, the table of ctype.ml:1996-2008; +- polymorphic types without variables: the bodies. With variables: + `enter_poly(left, right)`, which pushes a `LeavePoly` frame and the + pair of the bodies on `frames`; `mcomp` pops both and pushes on its + own stack a marker with the frame's scope length and the pair, and a + popped marker calls `leave_scopes`. Two universal variables: + `unify_univar`; +- anything else: a clash. A clash has no trace pair (`Unify []`). + +**No supplier.** `Checker::locals` is `None` and two different data +types meet: a clash, as decision 52 answers for packages: the kernel +has no declaration to compare and must not accept a pattern on a +guess. R5a installs a supplier before it types a pattern, so the +answer is reached only by a test. + +**71. What a unit that never types a pattern pays.** Reads added to +expression mode: none in `unify_expanded` and unify3 (the load of +`umode` moves from one to the other); in `unify_pair` one load of +`equations.len()` for a pair of two nullary constructors of different +types; in `subst` one compare of a field it has loaded; in +`run_frames` nothing on the success path (`recover` is called from the +failure path, where it loads `tries.len()`). Memory: three empty `Vec`s +and two empty sets on `Checker`. The benchmark operations of +TC_KERNEL_R3.md are reported before and after by each part; no target +is set (decision 46). + +**Step tests** (relations, 1,000 against 10,000; tc/r3_tests.rs): +`pattern_mode_at_ten_thousand`. (a) a chain of 10,000 nested tuples +against its copy in pattern mode: the steps are linear in the pairs +(ten times the steps of 1,000, within one part in ten), no node is a +link, and the table is empty afterwards; (b) `reify` of a type with +10,000 variables, each shared twice: every variable is visited once +(steps linear); (c) 10,000 equations added and cut: linear, and every +row is its `previous`; (d) `mcomp` of two chains of 10,000 tuples that +share their elements pairwise: linear, by the table. + +**The parts.** + +- **R3h**: the fixed-row rule of `copy_refs` (decision 47's last + paragraph), then `Checker::{unify_pattern, cut_equations}` and the + pattern arms of `unify_expanded`, unify3 and `unify_pair` marked 1 + (unify.rs); `Checker::{reify, add_gadt_equation, duplicate, + non_recursive_abbrev, find_newtype_level}` (rows.rs); the equation + arm of `subst` (expand.rs). Tests: the hand-built tests of + tc/r3_tests.rs ignored for "R3h build", steps (a) to (c). +- **R3r**: `Checker::{mcomp, push_compatible, recover}`, the `Try` + arm of `run_frames`, the three places R3h + left `todo!("R3r")`. Tests: those ignored for "R3r build", step + (d), and `pattern_plan_agrees_on_generated_cases`. + +**Arms no hand-built test reaches**: the object-type arm of `reify` +(not built); `mcomp` of two fields (`Tfield _, Tfield _`, "actually +unused" in OCaml); a parameter past the 32nd of `injective_bits`; +`LocalDecls::compare` answering `Members` for two open types. + +**Open questions of the sixth turn.** + +1. *Ambiguity of GADT instances is not checked* by the Rust compiler + (decision 62: a program OCaml refuses is accepted). Recommended: a + slice of its own after R3r, designed with R3q in the next turn: the + instance levels of `Env` (`gadt_instance_level`, + `add_gadt_instance_chain`) as a side table of the kernel, and the + three places of unification that read them. +2. *The old checker links two structures in pattern mode* and no source + program shows it (decision 62). Recommended: keep `old_judges`; the + hand-built tests judge the success of a structure pair. +3. *`free_univar` visits a node again for each context* of a + polymorphic type WITH bound variables that shares it, as OCaml does + (decision 59). Recommended: accept. +4. *`Checker::lower_scoped` does not drop the name of a row whose path + escapes* (ctype.ml:618-623; decision 60). Recommended: a correction + with a hand-built test in the turn that designs R3q, which copies + named rows. +5. *`duplicate` is `generalize` and `copy` under a speculation* + (decision 67): it holds only if `generalize` writes nothing but + levels through the trail. The builder of R3h reports what it + writes; if it writes more, the copy gets a flag that copies every + node, in `Graph::copy_ref`. + +**Corrections owed after review** (the review of the sixth turn, pull +request 347). The prompts of R3h and R3r in 8.4 are not issued until a +design turn has made these, each with its test; nothing here is the +builder's to decide. + +1. *Decision 67, step 1, and the test + `a_recursive_equation_is_not_added` are not what OCaml does.* + `local_non_recursive_abbrev` is ctype.ml:1503-1536 (1478-1502 are + `occur_full`, `occur` and `occur_in`). It is called with `strict = + false`, and then: + - a node that is not a constructor is entered only `if strict` + (1525-1529): a tuple or an arrow at the top is not searched; + - a constructor of `source` answers false (1508); + - a constructor of a contractive type (`is_contractive`, + 1371-1375: a data type, or a predefined type without a manifest) + is not entered while not strict (1509); + - any other constructor is expanded once (`try_expand_head + try_expand_once`, 1515, not the `_opt` form) and the walk goes on + in the expansion; one that cannot be expanded has its arguments + walked, strictly for an argument whose parameter in the + declaration is not a variable (1516-1524). + Seen with both compilers on source programs (`Refl: eq<'a, 'a>` + matched at `eq` for a locally abstract `a`, the branch using a + value of type `a` at type `T`): the equations `a := (a, int)`, `a := + array` and `a := box` (a data type) are ADDED, both compilers + accept; `a := M.t` for an abstract `M.t` is not, both refuse. The + test must expect the tuple's equation added and have a case that is + refused. +2. *The occurrence test of a variable in pattern mode* (decisions 64 + and 69 say "as in expression mode"). In `Pattern` OCaml's `occur` + runs `occur_full` with `allow_recursive` (ctype.ml:1467-1490; + `occur_rec`, 1384-1408): a contractive constructor is not entered + (1390), a structure that is not a constructor is not entered + (1404-1408), objects and variants never are (1403). So a variable is + bound to a type that holds it below a tuple in pattern mode, where + expression mode clashes. The design says what the binding walk does + in pattern mode (the levels are still lowered everywhere), with a + hand-built test of the bind that succeeds and one of a clash through + a constructor that is not contractive, and what a binding in + expression mode pays for it. +3. *Decision 60 in pattern mode.* `unify_row` keeps row 1's name + arguments as children when `last1` is none, which is right because + unify3 has linked the left node in expression mode. Pattern mode + does not link it (decision 64): say which node is the home of row 1 + there and whether its name's arguments stop being children, with a + test. +4. `size_of` tests for `Equation` (20 bytes) and `Try` (40 bytes). +5. `mcomp`'s marker calls `leave_scopes`, which is private to unify.rs + and looks for its frame in `frames[base..]`, which `mcomp` has + popped by then: say how the scopes are cut there and make what it + needs visible. +6. `mcomp_at_ten_thousand` builds tuples of width 100, so its relation + holds with or without the table of pairs it is there for: width n, + or a second relation that fails without the table. +7. Step tests at 1,000 and 10,000 for `non_recursive_abbrev` and + `duplicate` (a destination of n nodes) and for `recover`. +8. Line ranges that are a few lines off, in doc comments, tests and + both prompts: the try for an argument that is not injective is + 2358-2375 (the cited 2359-2371 leaves out the two `reify` calls); + the variant arm is 2402-2411 (2411 is the `mcomp` call); the + package fallback is 2416-2424 (2422 reifies); two instantiable + types 2376-2387; the three one-sided arms 2388-2391, 2392-2395 and + 2396-2400. +9. The prompt of R3r gets the rule the prompt of R3h has for its + hand-built tests (a test that fails where the code does what the + ctype.ml lines say: stop and report, never edit the test); each + prompt names the functions that are the other slice's. Decision 71 + adds what it leaves out (the hash state made in `Checker::new`, the + growth of `Checker`, the two compares per pair of rows of decision + 60), and the cost paragraph of `unify_row` names those two compares. + +The five open questions above are answered as recommended in the same +review; the next design turn writes them into 8.6. + +Not designed in this turn: R3q (`instance_poly`, `instance_label`). +What is known for it: decision 47's fixed-row rule moves to R3h's first +commit, and question 4 above. + ## 9. R4 as designed Section 6.5 gave R4 as one slice of seven commits. It is replaced by this diff --git a/compiler-rust/src/tc/differential.rs b/compiler-rust/src/tc/differential.rs index 53f29a15a9..ec6d9b682b 100644 --- a/compiler-rust/src/tc/differential.rs +++ b/compiler-rust/src/tc/differential.rs @@ -184,6 +184,15 @@ pub(crate) const REFS: u16 = 1 << 7; /// two types whose binders pair by position and not by name print /// different names at one position and the same back references. pub(crate) const UNIVARS: u16 = 1 << 8; +/// Pattern mode (slice R3h; section 8.6, decision 61). The program +/// declares two abstract types `n0` and `n1`, which both sides make +/// locally abstract types of two newtype levels (the level the case +/// starts at, and one above it), an abstract type with a parameter +/// `k<'a>` (not injective) and a data type `d<'a>` (injective); its +/// values have types over them. The operations get [`Op::Gadt`]. Both +/// sides end their form with the equations in force, one line a type +/// that has one, sorted by name: `= name := body`. +pub(crate) const PATTERN: u16 = 1 << 9; /// A tier and what a case adds to it. 4 bytes, passed by value. #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -270,6 +279,14 @@ pub(crate) enum Op { /// 2424) and the old checker unifies its body (section 8.6, decision /// 49), and no generated case is to meet that. [`UNIVARS`]. Inside(u16, u8), + /// Unify two operands in pattern mode at the current level as the + /// newtype level (`unify_gadt`, compiler/ml/ctype.ml:2739-2750; + /// `Checker::unify_pattern`): bit 0 of the third field is + /// `generate_equations`, bit 1 `assume_injective` (`unify_gadt` + /// itself is 3; unify3 installs 1 and 0 for the arguments of a + /// constructor, ctype.ml:2352, 2368). The equations it adds stay in + /// force for the rest of the case. [`PATTERN`]. + Gadt(u16, u16, u8), } /// A deliberately wrong new side, for the tests that prove the harness sees @@ -305,6 +322,9 @@ pub(crate) enum Fault { /// polymorphic type in the other order: what pairing by name instead /// of by position looks like in the form. [`UNIVARS`]. SwappedBinders, + /// [`Op::Gadt`] unifies in expression mode: no equation, no + /// reification, a link where pattern mode assumes. [`PATTERN`]. + ExpressionMode, } /// What one side reports for one case. @@ -554,17 +574,57 @@ pub(crate) fn program_source(rng: &mut Rng, plan: impl Into) -> String { nullary.push("rv".to_string()); unary.push("pv".to_string()); } + if plan.has(PATTERN) { + source.push_str(PATTERN_TYPES); + for name in NEWTYPES { + nullary.push(name.to_string()); + } + unary.push("k".to_string()); + unary.push("d".to_string()); + } for index in 0..3 + rng.below(4) { let mut body = String::new(); type_text_with(rng, plan.with, 3, &["'a", "'b"], &nullary, &unary, &mut body); let _ = writeln!(source, "external v{index}: {body} = \"v{index}\""); } + if plan.has(PATTERN) { + source.push_str(PATTERN_VALUES); + if plan.has(VARIANTS) { + source.push_str(PATTERN_ROWS); + } + } source.push_str("let dup = x => (x, x)\n"); let last = nullary.last().unwrap(); let _ = writeln!(source, "let share = (x: {last}) => (x, x)"); source } +/// The predefined types a generated program names. +const PREDEFINED: [&str; 6] = ["int", "string", "unit", "float", "bool", "array"]; + +/// The two types a [`PATTERN`] case makes locally abstract. +const NEWTYPES: [&str; 2] = ["n0", "n1"]; + +/// The declarations of a [`PATTERN`] program: the two locally abstract +/// types, an abstract type with a parameter and a data type with one. +const PATTERN_TYPES: &str = "type n0\ntype n1\ntype k<'a>\ntype d<'a> = D('a)\n"; + +/// The values of a [`PATTERN`] program, after the generated ones: a +/// locally abstract type against a structure and against the other one, +/// constructors of one path with arguments (abstract: `k`; a data type: +/// `d`), and a function over both. +const PATTERN_VALUES: &str = "external q0: (n0, n1, int, (int, t0)) => unit = \"q0\"\n\ + external q1: (k, k, k, k<'a>) => unit = \"q1\"\n\ + external q2: (d, d, d, d<'a>) => unit = \"q2\"\n\ + external q3: (n0 => n1, int => t0, (n1, n0) => 'a) => unit = \"q3\"\n"; + +/// With [`VARIANTS`]: rows that clash and are compatible (`#A` over a +/// locally abstract type against `#A` over `int`, a tag the other closed +/// row lacks), and rows that clash and are not (a constant tag against +/// one with an argument). +const PATTERN_ROWS: &str = "external q4: ([#A(n0) | #B], [#A(int) | #B], [#A(int) | #C], [#A | #B]) => unit = \"q4\"\n\ + external q5: ([> #A(n0)], [< #A(int) | #B], [#C(n1)]) => unit = \"q5\"\n"; + /// O(the program): compile with the old checker. Panics with the source when /// the generator wrote a program the old checker rejects. pub(crate) fn compile(source: String) -> Program { @@ -646,6 +706,39 @@ pub(crate) fn operations(rng: &mut Rng, plan: impl Into, values: usize) -> } _ => Op::Inside(operand(rng), rng.below(3) as u8), } + } else if plan.has(PATTERN) && rng.below(3) == 0 { + // [`PATTERN`]: a pattern unification, twice in three as + // `unify_gadt` runs it, or a part of an operand to unify. + // Half of them are over an instance of one of the pattern + // values ([`PATTERN_VALUES`], before `dup` and `share`): + // one of its parameters against another, or against any + // operand; `follow` is last first. + match rng.below(6) { + 0..=2 => { + let count = if plan.has(VARIANTS) { 6 } else { 4 }; + let value = values - 2 - count + rng.below(count); + let instance = operands as u16; + let flags = if rng.below(4) == 0 { + rng.below(3) as u8 + } else { + 3 + }; + let against = if rng.below(3) == 0 { + operand(rng) + } else { + instance + 2 + }; + follow = vec![ + Op::Gadt(instance + 1, against, flags), + Op::Child(instance, rng.below(4) as u8), + Op::Child(instance, rng.below(4) as u8), + ]; + Op::Instance(value as u16) + } + 3 => Op::Gadt(operand(rng), operand(rng), 3), + 4 => Op::Gadt(operand(rng), operand(rng), rng.below(3) as u8), + _ => Op::Child(operand(rng), rng.below(4) as u8), + } } else if plan.has(BUILD) && rng.below(4) == 0 { match rng.below(4) { 0 => Op::Var, @@ -715,7 +808,7 @@ pub(crate) fn operations(rng: &mut Rng, plan: impl Into, values: usize) -> /// The old checker's types of a case: the context, the environment the /// unit's declarations are in, the schemes of its values and the level the /// case starts at. -struct Old { +pub(crate) struct Old { ctx: TypeContext, env: Env, values: Vec, @@ -745,6 +838,7 @@ fn old_context(program: &Program, plan: Plan) -> Old { let mut env: Env = initial_env(&ctx); let mut values = Vec::new(); let mut late = Vec::new(); + let mut newtypes = Vec::new(); for item in unit.ps_sig.iter() { match item { SignatureItem::SigValue(_, value) => values.push(value.val_type), @@ -752,6 +846,9 @@ fn old_context(program: &Program, plan: Plan) -> Old { if plan.has(SCOPED) && LATE.contains(&id.name()) { late.push((id.name().to_string(), declaration.clone())); } + if plan.has(PATTERN) && NEWTYPES.contains(&id.name()) { + newtypes.push((id.name().to_string(), declaration.clone())); + } env.add_type(*id, declaration.clone()); } _ => {} @@ -772,15 +869,32 @@ fn old_context(program: &Program, plan: Plan) -> Old { // 1`, `+ 2`, `+ 3`. A type of the unit is named through the persistent // unit and has binding time 0, so every node that names a late type is // renamed to the local identifier, in the schemes and in the manifests. + let mut renamed = Vec::new(); if plan.has(SCOPED) { assert_eq!(late.len(), LATE.len(), "{}", program.source); - let mut renamed = Vec::new(); for (offset, (name, declaration)) in late.iter().enumerate() { let id = ctx.create_ident(name.as_str()); assert_eq!(id.stamp(), base + 1 + offset as i32, "late stamp"); env.add_type(id, declaration.clone()); renamed.push((name.clone(), id)); } + } + // [`PATTERN`]: `n0` and `n1` are locally abstract types, as `(type + // a)` declares one (`type_newtype_level = Some (level, level)`, + // compiler/ml/typecore.ml, `Pexp_newtype`): local identifiers whose + // declarations have the newtype levels `base` and `base + 1`. + if plan.has(PATTERN) { + assert_eq!(newtypes.len(), NEWTYPES.len(), "{}", program.source); + for (offset, (name, declaration)) in newtypes.iter().enumerate() { + let id = ctx.create_ident(name.as_str()); + let level = base + offset as i32; + let mut declaration = (**declaration).clone(); + declaration.type_newtype_level = Some((level, level)); + env.add_type(id, declaration); + renamed.push((name.clone(), id)); + } + } + if !renamed.is_empty() { let mut seen = crate::fast_hash::AHashSet::default(); let mut work: Vec = values.clone(); for item in unit.ps_sig.iter() { @@ -908,9 +1022,57 @@ fn record_failure(outcome: &mut Outcome, index: usize, before: &str, after: (Str /// O(operations + printed nodes²): the old checker's side of a case. pub(crate) fn run_old(program: &Program, plan: impl Into, ops: &[Op]) -> Outcome { - let plan = plan.into(); + run_old_until(program, plan.into(), ops, &mut None) +} + +/// How many of `ops` the old checker is a judge of under [`PATTERN`] +/// (section 8.6, decision 62): the operations before the first pattern +/// unification that succeeds and leaves a structure of an operand a link. +/// In pattern mode OCaml records that the two structures are assumed +/// equal and links neither (`Pattern -> add_type_equality t1' t2'`, +/// compiler/ml/ctype.ml:2327-2331); the old checker links them, so its +/// graph after such an operation is not OCaml's. [`compare`] runs both +/// sides on this prefix. O(the case × operands). +pub(crate) fn old_judges(program: &Program, plan: Plan, ops: &[Op]) -> usize { + let mut cut = Some(ops.len()); + run_old_until(program, plan, ops, &mut cut); + cut.unwrap() +} + +/// O(nodes): the structures `ty` reaches: every node that is not a +/// variable, a universal variable or a constructor (those OCaml links in +/// pattern mode as well, ctype.ml:2247, 2250-2253, 2315-2324). +fn old_structures( + ctx: &TypeContext, + ty: TypeExprRef, + seen: &mut std::collections::HashSet, + out: &mut Vec, +) { + let ty = ctx.repr(ty); + if !seen.insert(ty) { + return; + } + if !matches!( + *ctx.get_desc(ty), + TypeDesc::Tvar { .. } | TypeDesc::Tunivar { .. } | TypeDesc::Tconstr { .. } + ) { + out.push(ty); + } + let mut children = Vec::new(); + crate::types::btype::iter_type_expr(ctx, ty, |child| children.push(child)); + for child in children { + old_structures(ctx, child, seen, out); + } +} + +/// [`run_old`]; with `cut` set, stops at the first operation the old +/// checker is not a judge of ([`old_judges`]) and writes its index. +fn run_old_until(program: &Program, plan: Plan, ops: &[Op], cut: &mut Option) -> Outcome { let old = old_context(program, plan); - let (ctx, env, base) = (&old.ctx, &old.env, old.base); + let (ctx, base) = (&old.ctx, old.base); + // The environment of the case: a pattern unification that succeeds + // leaves its equations and its reified types in it. + let mut current = old.env.clone(); let explain = plan.has(EXPLAIN); let mut outcome = Outcome { failed: None, @@ -921,10 +1083,14 @@ pub(crate) fn run_old(program: &Program, plan: impl Into, ops: &[Op]) -> O let mut operands: Vec = Vec::new(); for (index, op) in ops.iter().enumerate() { let before = match op { - Op::Unify(..) | Op::Lower(..) if explain => old_forms(&old, plan, &operands, &[]).0, + Op::Unify(..) | Op::Lower(..) | Op::Gadt(..) if explain => { + old_forms(&old, plan, &operands, &[]).0 + } _ => String::new(), }; let mut failure = None; + let env = ¤t; + let mut refined = None; match *op { Op::Instance(value) => operands.push(ctype::instance(ctx, old.values[value as usize])), Op::Copy(operand) => operands.push(ctype::instance(ctx, operands[operand as usize])), @@ -998,6 +1164,35 @@ pub(crate) fn run_old(program: &Program, plan: impl Into, ops: &[Op]) -> O } operands.push(ty); } + Op::Gadt(left, right, flags) => { + let mut structures = Vec::new(); + if cut.is_some() { + let mut seen = std::collections::HashSet::new(); + for &operand in &operands { + old_structures(ctx, operand, &mut seen, &mut structures); + } + } + let (result, after) = old_gadt( + ctx, + env, + flags, + operands[left as usize], + operands[right as usize], + ); + match result { + Ok(()) => { + if structures.iter().any(|&node| ctx.repr(node) != node) { + *cut = Some(index); + return outcome; + } + refined = Some(after); + } + Err(trace) => failure = Some(trace), + } + } + } + if let Some(env) = refined { + current = env; } if let Some(trace) = failure { if !explain { @@ -1008,9 +1203,66 @@ pub(crate) fn run_old(program: &Program, plan: impl Into, ops: &[Op]) -> O } } outcome.form.push_str(&old_forms(&old, plan, &operands, &[]).0); + if plan.has(PATTERN) { + outcome.form.push_str(&old_equations(&old, ¤t)); + } outcome } +/// One pattern unification on the old checker, as `unify_pattern_types_gadt` +/// runs one (types/typecore.rs): a state in pattern mode with the two +/// flags of `flags` ([`Op::Gadt`]) and the current level as the newtype +/// level. Returns the result and the environment the state ends with: +/// the equations and the reified types of a success. +pub(crate) fn old_gadt( + ctx: &TypeContext, + env: &Env, + flags: u8, + left: TypeExprRef, + right: TypeExprRef, +) -> (Result<(), Vec<(TypeExprRef, TypeExprRef)>>, Env) { + let mut state = ctype::UnifyState::new(); + state.set_env(env); + state.mode = crate::types::UnifyMode::Pattern; + state.generate_equations = flags & 1 != 0; + state.assume_injective = flags & 2 != 0; + state.newtype_level = Some(ctx.current_level()); + let result = match ctype::unify(ctx, &mut state, left, right) { + Ok(()) => Ok(()), + Err(ctype::TypeError::Unify(trace)) => Err(trace), + Err(_) => Err(Vec::new()), + }; + let after = state.env().cloned().unwrap_or_else(|| env.clone()); + (result, after) +} + +/// O(equations × printed nodes): the equations of `env`, one line a +/// type that has a local constraint, sorted by the type's name: `= name +/// := body`, the bodies numbered together (a body is generic: `[G]`). +pub(crate) fn old_equations(old: &Old, env: &Env) -> String { + let mut equations: Vec<(String, TypeExprRef)> = env + .iter_local_constraints() + .filter_map(|(path, declaration)| { + declaration + .type_manifest + .map(|body| (path.last().to_string(), body)) + }) + .collect(); + equations.sort_by(|left, right| left.0.cmp(&right.0)); + let mut out = String::new(); + let mut seen = OldSeen { + nodes: Vec::new(), + cells: None, + refs: None, + }; + for (name, body) in equations { + let _ = write!(out, "= {name} := "); + old_form(&old.ctx, old.base, true, body, &mut seen, &mut out); + out.push('\n'); + } + out +} + /// O(nodes reachable × nodes seen): print one old node; see the module text. fn old_form( ctx: &TypeContext, @@ -1211,6 +1463,48 @@ pub(crate) struct New { /// unopened node, the node and the field (bit 63 set): a reference /// nothing else holds. refs: Vec, + /// [`PATTERN`]: the names of the locally abstract types `reify` made, + /// by type index from the segment's last path on ([`FreshNames`]). + fresh: std::rc::Rc>>, +} + +/// The supplier of a [`PATTERN`] case (`rows::LocalDecls`): a fresh +/// abstract type gets the next type index after the segment's paths and +/// the name `get_new_abstract_name` gives it (compiler/ml/ctype.ml: +/// 908-922): `$'a`, then `$'a1`; `$0`, `$1` for a variable without a +/// name. The program has one data type, so two different data types are +/// never compared: `Clash`. +struct FreshNames { + first: u32, + strings: Vec, + names: std::rc::Rc>>, +} + +impl super::rows::LocalDecls for FreshNames { + fn fresh_abstract(&mut self, name: u32) -> u32 { + let prefix = match self.strings.get(name as usize) { + Some(name) if !name.is_empty() => format!("$'{name}"), + _ => "$".to_string(), + }; + let mut names = self.names.borrow_mut(); + let spelled = |index: usize| { + if index == 0 && !prefix.ends_with('$') { + prefix.clone() + } else { + format!("{prefix}{index}") + } + }; + let mut index = 0; + while names.contains(&spelled(index)) { + index += 1; + } + names.push(spelled(index)); + self.first + names.len() as u32 - 1 + } + + fn compare(&mut self, _: &mut Checker, _: u32, _: u32) -> super::rows::DeclMatch { + super::rows::DeclMatch::Clash + } } impl New { @@ -1272,6 +1566,7 @@ impl New { first: vec![None; roots.len()], cells: Vec::new(), refs: Vec::new(), + fresh: Default::default(), open, segment, checker, @@ -1283,9 +1578,107 @@ impl New { if plan.has(SCOPED) { new.scope_late_types(); } + if plan.has(PATTERN) { + new.declare_newtypes(); + } new } + /// O(paths): [`PATTERN`]: `n0` and `n1` are locally abstract types of + /// the newtype levels 1 and 2 (a graph starts at level 1), on every + /// `paths` row that names one, with the scope `get_level` gives a + /// newtype (its first level, compiler/ml/ctype.ml:562-569); and the + /// supplier of fresh abstract types is installed. + fn declare_newtypes(&mut self) { + use super::expand::{NEWTYPE, Newtype}; + for index in 0..self.segment.signature.paths.len() { + let name = match self.segment.signature.paths[index] { + PathRow::Ident(ident) => self.segment.signature.identifiers[ident as usize].name, + PathRow::Dot { name, .. } => name, + PathRow::Apply { .. } => continue, + }; + // The old side sees the unit from outside, as `Case`: its + // types are paths through a module there and only the + // predefined types are bare identifiers (`LOCAL`). + if PREDEFINED.contains(&self.string(name)) { + self.checker.types.set_local(index as u32); + } + let Some(offset) = NEWTYPES + .iter() + .position(|newtype| *newtype == self.string(name)) + else { + continue; + }; + let level = 1 + offset as u32; + let types = &mut self.checker.types; + types.newtypes.push(Newtype { + level, + expansion_level: level, + }); + let row = &mut types.rows[index]; + row.flags |= NEWTYPE; + row.newtype = types.newtypes.len() as u32 - 1; + if level > 1 { + if row.scope == 0 { + types.scoped += 1; + } + row.scope = level; + } + } + let pools = &self.segment.signature; + let strings = (0..pools.strings.len() as u32) + .map(|at| self.string(at).to_string()) + .collect(); + self.checker.locals = Some(Box::new(FreshNames { + first: pools.paths.len() as u32, + strings, + names: self.fresh.clone(), + })); + } + + /// O(equations × printed nodes): [`PATTERN`]: the equations in force + /// (`Checker::equations`), as [`old_equations`] prints them: one line + /// a type, sorted by name, the body an instance of the equation's + /// manifest made at the generic level, the bodies numbered together. + fn equations(&mut self) -> String { + let mut equations: Vec<(String, Ty)> = Vec::new(); + for at in 0..self.checker.equations.len() { + let index = self.checker.equations[at].type_index; + let info = self.checker.types.info(index); + let manifest = self.checker.types.manifests[info.manifest as usize]; + let base = self.checker.g.nodes.len() as u32; + let level = std::mem::replace(&mut self.checker.g.level, GENERIC); + self.checker.g.instance(manifest.template); + self.checker.g.level = level; + let body = if manifest.body.0 & RELATIVE != 0 { + Ty(base + (manifest.body.0 & !RELATIVE)) + } else { + manifest.body + }; + equations.push((self.type_name(index), body)); + } + equations.sort_by(|left, right| left.0.cmp(&right.0)); + let mut seen = vec![NONE; self.checker.g.nodes.len()]; + let mut count = 0; + let mut out = String::new(); + for (name, body) in equations { + let _ = write!(out, "= {name} := "); + self.node_form(body, &mut seen, &mut count, &mut out); + out.push('\n'); + } + out + } + + /// O(1): the name a type index prints with: the last component of its + /// path, or the name of a fresh abstract type. + fn type_name(&self, index: u32) -> String { + let paths = self.segment.signature.paths.len() as u32; + if index >= paths { + return self.fresh.borrow()[(index - paths) as usize].clone(); + } + self.path_name(index).to_string() + } + /// O(paths): give the late types their scopes: a graph starts at level /// 1, so `s0`, `s1`, `s2` are bound at levels 2, 3 and 4. Every `paths` /// row whose last name is a late type's is one (the unit's own @@ -1611,6 +2004,17 @@ impl New { } self.operands.push(ty); } + Op::Gadt(left, right, flags) => { + let (left, right) = (self.operands[left as usize], self.operands[right as usize]); + if fault == Fault::ExpressionMode { + return self.checker.unify(left, right).map_err(|_| ()); + } + let level = self.checker.g.level; + return self + .checker + .unify_pattern(level, flags & 1 != 0, flags & 2 != 0, left, right) + .map_err(|_| ()); + } } Ok(()) } @@ -1678,16 +2082,7 @@ impl New { } } CONSTR if node.head & super::segment::EXTENDED == 0 => { - let name = match self.segment.signature.paths[node.a as usize] { - PathRow::Ident(ident) => self.segment.signature.identifiers[ident as usize].name, - PathRow::Dot { name, .. } => name, - PathRow::Apply { .. } => NONE, - }; - if name == NONE { - out.push_str("?apply"); - } else { - out.push_str(self.string(name)); - } + out.push_str(&self.type_name(node.a)); if node.arity() != 0 { out.push('<'); for index in 0..node.arity() { @@ -1928,7 +2323,7 @@ pub(crate) fn run_new(program: &Program, plan: impl Into, ops: &[Op], faul program.source ); let before = match op { - Op::Unify(..) | Op::Lower(..) if explain => new.form(), + Op::Unify(..) | Op::Lower(..) | Op::Gadt(..) if explain => new.form(), _ => String::new(), }; if new.apply(op, fault).is_err() { @@ -1937,7 +2332,7 @@ pub(crate) fn run_new(program: &Program, plan: impl Into, ops: &[Op], faul } // A failed lowering has no trace on either side (`run_old`). let mut trace = match op { - Op::Unify(..) => new.checker.trace.clone(), + Op::Unify(..) | Op::Gadt(..) => new.checker.trace.clone(), _ => Vec::new(), }; match fault { @@ -1952,6 +2347,9 @@ pub(crate) fn run_new(program: &Program, plan: impl Into, ops: &[Op], faul } } outcome.form.push_str(&new.form()); + if plan.has(PATTERN) { + outcome.form.push_str(&new.equations()); + } outcome } @@ -1965,6 +2363,11 @@ pub(crate) fn compare( fault: Fault, ) -> Result { let plan = plan.into(); + // [`PATTERN`]: only the operations the old checker is a judge of. + let ops = match plan.has(PATTERN) { + true => &ops[..old_judges(program, plan, ops)], + false => ops, + }; let old = run_old(program, plan, ops); let new = run_new(program, plan, ops, fault); if old == new { @@ -3160,7 +3563,9 @@ mod tests { /// R3x. #[test] fn ignored_tests_name_the_slices_they_reach() { - let before: [(&str, &[&str]); 10] = [ + let before: [(&str, &[&str]); 12] = [ + ("R3q", &["R3m", "R3e", "R3f", "R3n", "R3g"]), + ("R3r", &["R3m", "R3e", "R3f", "R3n", "R3g", "R3h"]), ("R3p", &["R3m", "R3e"]), ("R3k", &[]), ("R3m", &[]), @@ -3258,9 +3663,12 @@ mod tests { let mut reaches: Vec<&str> = Vec::new(); let full = body.contains("Tier::Full"); let rows = body.contains("OBJECTS") || body.contains("VARIANTS"); - for (needle, reached) in - [("OBJECTS", "R3e"), ("VARIANTS", "R3f"), ("UNIVAR", "R3g")] - { + for (needle, reached) in [ + ("OBJECTS", "R3e"), + ("VARIANTS", "R3f"), + ("UNIVAR", "R3g"), + ("PATTERN", "R3h"), + ] { if full && body.contains(needle) { reaches.push(reached); } @@ -4012,6 +4420,670 @@ mod tests { } } + /// The corner of decision 53 on the old checker, built by hand + /// (section 8.6, decision 58; observed). Two `Reither` fields inside + /// two polymorphic types: `#A` with the types `[(u, x); x]` on the + /// left and `[(u', x'); x']` on the right. Each side would gain the + /// other's two types; the tuples hold a universal variable and are + /// unified instead, which makes `x` and `x'` one node; the lists the + /// fields gain were computed BEFORE that unification + /// (compiler/ml/ctype.ml:2685-2706): + /// + /// ```text + /// let tl2' = remq tl2 tl1 and tl1' = remq tl1 tl2 in + /// let tl1', tlu1 = split_univars tl1' and tl2', tlu2 = split_univars tl2' in + /// (match (tlu1, tlu2) with + /// | tu1 :: tlu1, _ :: _ -> + /// if not !(passive_variants ()) then List.iter (unify env tu1) (tlu1 @ tlu2) + /// ... + /// let f1' = Reither (c1 || c2, tl1', m1 || m2, e) + /// ``` + /// + /// so each field ends with THREE types, the last two one node (`@4 + /// @4`): the type it had, and the same node again as the type it + /// gained. The OCaml compiler was run on a source program that reaches + /// the corner (the pull request of the sixth skeleton turn has it): + /// its fields hold three types after the join too. + #[test] + fn old_checker_gains_a_type_the_holders_made_a_duplicate() { + use crate::types::type_expr::{RowDesc, RowField}; + let program = compile(UNIVAR_PROGRAM.to_string()); + let plan = Plan::new(Tier::Full, UNIVAR_PLAN); + let old = old_context(&program, plan); + let ctx = &old.ctx; + let side = || { + let u = ctx.new_univar(None); + let x = ctx.new_var(None); + let holder = ctx.new_tuple(vec![u, x]); + let field = RowField::Reither { + constant: false, + types: vec![holder, x], + matched: false, + link: ctx.alloc_row_field(None), + }; + let row = ctx.new_type(TypeDesc::Tvariant(ctx.new_row_desc(RowDesc { + row_fields: vec![("A".into(), field)].into(), + row_more: ctx.new_var(None), + row_closed: true, + row_fixed: false, + row_name: None, + }))); + (ctx.new_poly(row, vec![u]), row, x) + }; + let (left, row1, x1) = side(); + let (right, row2, x2) = side(); + let mut state = ctype::UnifyState::new(); + state.set_env(&old.env); + ctype::unify(ctx, &mut state, left, right).unwrap(); + let form = old_forms(&old, plan, &[left, right, row1, row2, x1, x2], &[]).0; + assert_eq!( + form, + "0 = &0[0]poly(&1[0][closed #A?^0(&2[0](&3[0]!, &4[0]') @4 @4 ) | &5[0]']; @3)\n\ + 1 = @0\n2 = @1\n3 = @1\n4 = @4\n5 = @4\n" + ); + } + + /// What the old checker's walks do with the arguments of a row name + /// that is no longer the name of the innermost row (section 8.6, + /// decision 60; observed). `[> #B]` is unified with `[> #A] as p`: + /// the named row gains `#B` through a new variant node its row + /// variable is linked to (`set_more`, compiler/ml/ctype.ml:2596-2616) + /// and the merged row has no name (ctype.ml:2574-2586), so `p` is + /// the name of a row whose `more` is a variant. `iter_row` visits a + /// name's arguments only for the row a chain ends in + /// (compiler/ml/btype.ml:229-241), so `y`, which is in no field, is + /// reached by no walk: a lowering of the row leaves `y` at its level + /// (`[1]`), a generalization leaves it there, `deep_occur` does not + /// find it, a copy of the generalized row makes no copy of it that + /// the row holds, and a variable bound to the row leaves it. + #[test] + fn old_checker_does_not_reach_the_arguments_of_a_stale_row_name() { + use crate::types::type_expr::{RowDesc, RowField}; + let program = compile(UNIVAR_PROGRAM.to_string()); + let plan = Plan::new(Tier::Full, UNIVAR_PLAN); + let old = old_context(&program, plan); + let ctx = &old.ctx; + let mut out = String::new(); + for walk in 0..5 { + ctx.begin_def(); + let y = ctx.new_var(None); + let path = crate::types::path::Path::pident(ctx.create_ident("p")); + let row = |tag: &str, name: Option<(crate::types::path::Path, Vec)>| { + ctx.new_type(TypeDesc::Tvariant(ctx.new_row_desc(RowDesc { + row_fields: vec![(tag.into(), RowField::Rpresent(None))].into(), + row_more: ctx.new_var(None), + row_closed: false, + row_fixed: false, + row_name: name, + }))) + }; + let named = row("A", Some((path, vec![y]))); + let other = row("B", None); + let mut state = ctype::UnifyState::new(); + state.set_env(&old.env); + ctype::unify(ctx, &mut state, other, named).unwrap(); + ctx.end_def(); + let mut operands = vec![named, y]; + match walk { + 0 => ctype::update_level(ctx, ctx.current_level(), named), + 1 => ctype::generalize(ctx, named), + 2 => { + let found = ctype::deep_occur(ctx, y, named); + out.push_str(&format!("deep_occur {found}\n")); + } + 3 => { + ctype::generalize(ctx, named); + ctype::generalize(ctx, y); + operands.push(ctype::instance(ctx, named)); + operands.push(ctype::instance(ctx, y)); + } + _ => { + let v = ctx.new_var(None); + ctype::unify(ctx, &mut state, v, named).unwrap(); + } + } + out.push_str(&old_forms(&old, plan, &operands, &[]).0); + } + assert_eq!( + out, + "0 = &0[0][#A #B | &1[0]']\n1 = &2[1]'\n\ + 0 = &0[G][#A #B | &1[G]']\n1 = &2[1]'\n\ + deep_occur false\n\ + 0 = &0[1][#A #B | &1[1]']\n1 = &2[1]'\n\ + 0 = &0[G][#A #B | &1[G]']\n1 = &2[G]'\n2 = &3[0][#A #B | &4[0]']\n3 = &5[0]'\n\ + 0 = &0[0][#A #B | &1[0]']\n1 = &2[1]'\n" + ); + } + + /// A program for the fixed cases of pattern mode ([`PATTERN`]): the + /// two locally abstract types `n0` (the older) and `n1`, an abstract + /// type with a parameter `k`, a data type `d`, and values whose + /// parameters are the types a case unifies: `p0` nullary types and a + /// tuple, `p1` and `p2` constructors of one path, `p3` variables, + /// `p4` closed rows, `p5` open rows and a row with an upper bound, + /// `p6` tuples and nested constructors. + const PATTERN_PROGRAM: &str = "type n0\ntype n1\ntype k<'a>\ntype d<'a> = D('a)\n\ + external p0: (n0, n1, int, (int, int)) => unit = \"p0\"\n\ + external p1: (k, k, k, k<'a>) => unit = \"p1\"\n\ + external p2: (d, d, d, d<'a>) => unit = \"p2\"\n\ + external p3: ('a, 'b, ('a, int)) => unit = \"p3\"\n\ + external p4: ([#A(n0) | #B], [#A(int) | #B], [#A(int) | #C], [#A | #B]) => unit = \"p4\"\n\ + external p5: ([> #A(n0)], [< #A(int) | #B], [#C(n1)], [> #A(int)], [> #A(int) | #B]) => unit = \"p5\"\n\ + external p6: ((n0, int), (int, (int, int)), k>, k>) => unit = \"p6\"\n"; + + /// The plan of the fixed cases of pattern mode. + const PATTERN_PLAN: u16 = OBJECTS | VARIANTS | REFS | BUILD | PATTERN; + + /// One definition, an instance of value `value` (operand 0), its + /// parameters `left` and `right` (operands 1 and 2), unified in + /// pattern mode with `flags` ([`Op::Gadt`]). + fn pattern(value: u16, left: u8, right: u8, flags: u8) -> Vec { + vec![ + Op::BeginDef, + Op::Instance(value), + Op::Child(0, left), + Op::Child(0, right), + Op::Gadt(1, 2, flags), + ] + } + + /// One definition, the parameter `structure` of an instance of value + /// `value` (operand 1) and `n0` (operand 3), unified in pattern mode + /// as `unify_gadt` does, `n0` on the left. + fn refine(value: u16, structure: u8) -> Vec { + vec![ + Op::BeginDef, + Op::Instance(value), + Op::Child(0, structure), + Op::Instance(0), + Op::Child(2, 0), + Op::Gadt(3, 1, 3), + ] + } + + /// O(text): the equations a form ends with. + fn equations(form: &str) -> Vec<&str> { + form.lines().filter(|line| line.starts_with("= ")).collect() + } + + /// The old checker on a fixed pattern case: the outcome, and that it + /// is a judge of every operation ([`old_judges`]). + fn old_pattern(ops: &[Op]) -> Outcome { + let program = compile(PATTERN_PROGRAM.to_string()); + let plan = Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN); + assert_eq!(old_judges(&program, plan, ops), ops.len(), "{ops:?}"); + run_old(&program, plan, ops) + } + + /// Which pairs get an equation, and in which direction (observed; + /// section 8.6, decision 62; the old checker is a judge of all of + /// it). A locally abstract type against a type that is not one gets + /// the equation whichever side it is on (`Tconstr (path, [], _), _ + /// when is_instantiable ...` and its mirror, compiler/ml/ctype.ml: + /// 2385-2392), and the two nodes stay two nodes. Two locally abstract + /// types: the one of the higher newtype level gets it (`if + /// find_newtype_level !env path > find_newtype_level !env path' then + /// (path, t2') else (path', t1')`, ctype.ml:2373-2384): `n1`, on + /// either side. Without `generate_equations` there is none and the + /// unification succeeds (`reify` and no `mcomp`, ctype.ml:2393-2397). + /// `int` against a tuple with equations fails: `mcomp`. A second + /// equation is for the expansion: `n1` against `n0` after `n0 := int` + /// gives `n1 := int` (`unify2` expands both, ctype.ml:2276-2280). + #[test] + fn old_checker_gives_the_equation_to_the_newer_locally_abstract_type() { + let left = old_pattern(&pattern(0, 0, 2, 3)); + assert_eq!(left.failed, None); + assert_eq!( + left.form, + "0 = &0[1]fn(&1[1]n0, &2[1]n1, &3[1]int, &4[1](&5[1]int, &6[1]int)) => &7[1]unit\n1 = @1\n2 = @3\n= n0 := &0[G]int\n" + ); + assert_eq!( + equations(&old_pattern(&pattern(0, 2, 0, 3)).form), + ["= n0 := &0[G]int"] + ); + assert_eq!( + equations(&old_pattern(&pattern(0, 0, 1, 3)).form), + ["= n1 := &0[G]n0"] + ); + assert_eq!( + equations(&old_pattern(&pattern(0, 1, 0, 3)).form), + ["= n1 := &0[G]n0"] + ); + assert_eq!( + equations(&old_pattern(&pattern(0, 0, 3, 3)).form), + ["= n0 := &0[G](&1[G]int, &2[G]int)"] + ); + let silent = old_pattern(&pattern(0, 0, 2, 2)); + assert_eq!((silent.failed, equations(&silent.form).len()), (None, 0)); + assert_eq!(old_pattern(&pattern(0, 2, 3, 3)).failed, Some(4)); + let mut chain = pattern(0, 0, 2, 3); + chain.extend([Op::Child(0, 1), Op::Gadt(3, 1, 3)]); + assert_eq!( + equations(&old_pattern(&chain).form), + ["= n0 := &0[G]int", "= n1 := &1[G]int"] + ); + } + + /// The arguments of two constructors of one path (observed; decision + /// 62; the old checker is a judge). As `unify_gadt` runs it + /// (`assume_injective`) the arguments are unified with equations + /// (`set_mode_pattern ~generate:true ~injective:false`, + /// compiler/ml/ctype.ml:2352-2354): `k` against `k` gives + /// `n0 := int`. Below that, `k`, abstract and not of the unit, is not + /// injective: its arguments are unified without equations and what + /// does not unify is reified (ctype.ml:2359-2371): no equation, a + /// success, the two `k` nodes two nodes, and a later unification of + /// the two in expression mode fails, so nothing of the pattern + /// unification is left to make them equal. `d`, a data type, is + /// injective (`expands_to_datatype`, ctype.ml:2355-2358): `n0 := int`. + /// Without `generate_equations` neither has one. A variable argument + /// is a link to the other argument in every mode. + #[test] + fn old_checker_refines_the_arguments_of_a_constructor_by_its_injectivity() { + assert_eq!( + equations(&old_pattern(&pattern(1, 0, 1, 3)).form), + ["= n0 := &0[G]int"] + ); + for flags in [0, 1] { + let abstract_type = old_pattern(&pattern(1, 0, 1, flags)); + assert_eq!( + (abstract_type.failed, equations(&abstract_type.form).len()), + (None, 0) + ); + assert!( + abstract_type.form.contains("\n1 = @1\n2 = @3\n"), + "{}", + abstract_type.form + ); + } + assert_eq!( + equations(&old_pattern(&pattern(2, 0, 1, 1)).form), + ["= n0 := &0[G]int"] + ); + assert_eq!(equations(&old_pattern(&pattern(2, 0, 1, 0)).form).len(), 0); + let nested = old_pattern(&pattern(6, 2, 3, 3)); + assert_eq!((nested.failed, equations(&nested.form).len()), (None, 0)); + let mut again = pattern(6, 2, 3, 3); + again.extend([Op::Child(1, 0), Op::Child(2, 0), Op::Unify(3, 4)]); + assert_eq!(old_pattern(&again).failed, Some(7)); + let mut direct = pattern(1, 0, 1, 1); + direct.push(Op::Unify(1, 2)); + assert_eq!(old_pattern(&direct).failed, Some(5)); + let variable = old_pattern(&pattern(1, 0, 3, 1)); + assert!(variable.form.contains("&7[1]k<@2>"), "{}", variable.form); + } + + /// What `reify` makes of a variable and of a row variable (observed; + /// decision 62; the old checker is a judge of both, and of the names + /// of variables without a name). The variable of `('a, int)` at the + /// newtype level becomes a new abstract type `$0` at the variable's + /// level, and the equation's body is a generic copy that names it + /// (compiler/ml/ctype.ml:1815-1840, 2085-2087). The row variable of + /// `[> #A(int)]` becomes `$0` as well, and the row a fixed row over + /// it (`{r with row_fields = []; row_fixed = true; row_more = t}`, + /// ctype.ml:1845-1851). A second one is `$1` (`get_new_abstract_name`, + /// ctype.ml:908-922). No operation gives the harness a variable with + /// a name: `$'a` is tested by hand in tc/r3_tests.rs. + #[test] + fn old_checker_reifies_a_variable_and_a_row_variable() { + let variable = old_pattern(&refine(3, 2)); + assert_eq!(variable.failed, None); + assert_eq!( + variable.form.lines().next(), + Some("0 = &0[1]fn(&1[1]$0, &2[1]', &3[1](@1, &4[1]int)) => &5[1]unit") + ); + assert_eq!( + equations(&variable.form), + ["= n0 := &0[G](&1[G]$0, &2[G]int)"] + ); + let row = old_pattern(&refine(5, 3)); + assert!( + row.form.contains("&10[1][fixed #A(&11[1]int) | &12[1]$0]"), + "{}", + row.form + ); + assert_eq!( + equations(&row.form), + ["= n0 := &0[G][fixed #A(&1[G]int) | &2[G]$0]"] + ); + let mut twice = refine(3, 2); + twice.extend([ + Op::Child(2, 1), + Op::Instance(3), + Op::Child(5, 2), + Op::Gadt(4, 6, 3), + ]); + assert_eq!( + equations(&old_pattern(&twice).form), + [ + "= n0 := &0[G](&1[G]$0, &2[G]int)", + "= n1 := &3[G](&4[G]$1, &5[G]int)" + ] + ); + } + + /// What a pattern unification that fails leaves (observed; decision + /// 62; the old checker is a judge of the graph, not of the + /// environment, which the harness drops with the state). `reify` + /// links before it refuses: a variable below the newtype level is a + /// link to its new abstract type when the unification fails with the + /// pair of that type and a new variable (`link_type ty t; if ty.level + /// < newtype_level then raise (Unify [(t, newvar2 ty.level)])`, + /// compiler/ml/ctype.ml:1837-1839), and a row that clashes is a fixed + /// row when `mcomp` refuses the pair after it (ctype.ml:2400-2408). + /// The handler of unify3 puts back only the description of the left + /// node (`t1'.desc <- d1`, ctype.ml:2437-2439). The caller that goes + /// on after a failure restores a snapshot, the levels and the + /// environment it kept (`set_state`, compiler/ml/typecore.ml: + /// 1299-1305); `unify_pat_types_gadt` itself reports the failure + /// (typecore.ml:488-495). A failure below a tuple whose first + /// elements gave an equation leaves the graph as it was. + #[test] + fn old_checker_keeps_what_reify_linked_when_a_pattern_unification_fails() { + let ops = [ + Op::Instance(3), + Op::Child(0, 2), + Op::BeginDef, + Op::Instance(0), + Op::Child(2, 0), + Op::Gadt(3, 1, 3), + ]; + let below = old_pattern(&ops); + assert_eq!(below.failed, Some(5)); + assert_eq!(trace_of(&below.form, 5), ["@7 ~ @3", "@1 ~ &14[0]'"]); + assert_eq!(below.form.lines().last(), Some("3 = @7"), "{}", below.form); + assert!( + below + .form + .contains("\n0 = &0[0]fn(&1[0]$0, &2[0]', &3[0](@1, &4[0]int)) => &5[0]unit\n"), + "{}", + below.form + ); + let row = old_pattern(&pattern(5, 0, 2, 3)); + assert_eq!(row.failed, Some(4)); + assert!( + row.form.contains("&1[1][fixed #A(&2[1]n0) | &3[1]$0]"), + "{}", + row.form + ); + let partial = old_pattern(&pattern(6, 0, 1, 3)); + assert_eq!(partial.failed, Some(4)); + assert_eq!(trace_of(&partial.form, 4), ["@1 ~ @4", "@3 ~ @6"]); + assert_eq!((partial.unchanged, partial.changed), (1, 0)); + } + + /// Rows in pattern mode (observed; decision 62; the old checker is a + /// judge of all three). Two closed rows of which one lacks a tag of + /// the other, and a constant tag against one with an argument, clash + /// and are not compatible: `mcomp` refuses them (`mcomp_row`, + /// compiler/ml/ctype.ml:1985-2009). A fixed row against an open row + /// with one more tag clashes (`unify_row` does not extend a fixed + /// row) and is compatible: the unification succeeds after the + /// backtrack, the two rows are two nodes, the open one is reified to + /// a fixed row over `$1` and nothing else of the attempt is left + /// (ctype.ml:2400-2408). + #[test] + fn old_checker_accepts_rows_that_clash_and_are_compatible() { + assert_eq!(old_pattern(&pattern(4, 0, 2, 3)).failed, Some(4)); + assert_eq!(old_pattern(&pattern(4, 0, 3, 3)).failed, Some(4)); + let mut ops = refine(5, 3); + ops.extend([Op::Child(0, 4), Op::Gadt(1, 4, 3)]); + let compatible = old_pattern(&ops); + assert_eq!(compatible.failed, None); + assert!( + compatible.form.contains( + "&10[1][fixed #A(&11[1]int) | &12[1]$0], &13[1][fixed #A(&14[1]int) #B | &15[1]$1]" + ), + "{}", + compatible.form + ); + assert!(compatible.form.contains("\n1 = @10\n") && compatible.form.contains("\n4 = @13\n")); + } + + /// Where the old checker is NOT a judge (observed; decision 62): two + /// structures that a pattern unification unifies are one node + /// afterwards. OCaml records the pair as assumed equal and links + /// neither (`Pattern -> add_type_equality t1' t2'`, + /// compiler/ml/ctype.ml:2327-2331): a type that is equal only under + /// the equations of one case must not become the other for good. Two + /// tuples of two instances are one node; of two closed rows that + /// differ in `n0` against `int` the left one prints `int` afterwards. + /// [`old_judges`] ends a case before such an operation. + #[test] + fn old_checker_links_the_structures_a_pattern_unification_unifies() { + let program = compile(PATTERN_PROGRAM.to_string()); + let plan = Plan::new(Tier::Full, PATTERN_PLAN); + let tuples = [ + Op::BeginDef, + Op::Instance(0), + Op::Child(0, 3), + Op::Instance(0), + Op::Child(2, 3), + Op::Gadt(1, 3, 3), + ]; + let linked = run_old(&program, plan, &tuples); + assert!(linked.form.contains("\n3 = @4\n"), "{}", linked.form); + assert_eq!(old_judges(&program, plan, &tuples), 5); + let rows = pattern(4, 0, 1, 3); + let linked = run_old(&program, plan, &rows); + assert!( + linked + .form + .starts_with("0 = &0[1]fn(&1[1][closed #A(&2[1]int) #B | &3[1]nil], @1,"), + "{}", + linked.form + ); + assert_eq!(old_judges(&program, plan, &rows), 4); + } + + /// The self-test of [`PATTERN`]: the two sides agree today on a + /// pattern program for everything that is not a pattern unification + /// (the locally abstract types, the abstract type with a parameter + /// and the data type convert, print and unify in expression mode as + /// any type), on the fixed program and on 300 generated cases with + /// the pattern unifications taken out; and the form of a pattern + /// plan ends with no equation there. + #[test] + fn the_two_sides_agree_on_pattern_programs_without_a_pattern_unification() { + let program = compile(PATTERN_PROGRAM.to_string()); + assert_eq!(program.values.len(), 7); + let plan = Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN); + let ops = [ + Op::BeginDef, + Op::Instance(0), + Op::Instance(1), + Op::Child(0, 0), + Op::Child(1, 3), + Op::Child(3, 0), + Op::Unify(2, 4), + Op::Instance(5), + Op::Child(0, 1), + Op::Unify(2, 6), + Op::EndDef, + Op::Generalize(1), + Op::Copy(1), + ]; + let outcome = compare(0, &program, plan, &ops, Fault::None) + .unwrap_or_else(|report| panic!("{report}")); + assert_eq!(outcome.failed, Some(9)); + assert!(outcome.form.contains("k<&"), "{}", outcome.form); + assert_eq!(equations(&outcome.form).len(), 0); + let plan = Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN); + let mut patterns = 0; + for seed in 1..=3u64 { + for sequence in 0..100u64 { + let case_seed = seed * 1_000_003 + sequence; + let (program, ops) = case(plan, case_seed); + patterns += ops.iter().filter(|op| matches!(op, Op::Gadt(..))).count(); + let ops: Vec = ops + .into_iter() + .filter(|op| !matches!(op, Op::Gadt(..))) + .collect(); + if let Err(report) = compare(case_seed, &program, plan, &ops, Fault::None) { + panic!("{report}"); + } + } + } + assert!(patterns >= 300, "{patterns}"); + } + + /// The old side of the pattern plan runs today and is not vacuous + /// (observed, 1,200 cases: 1136 judged to the end, 64 not; 950 + /// pattern unifications that succeed and 493 that fail; 130 cases end + /// with an equation, 3 with two, 183 with a reified type; the bounds + /// are a third of that): cases the old checker judges to the end and cases it + /// leaves at a pattern unification that linked two structures + /// ([`old_judges`]); among the judged operations, pattern + /// unifications that succeed and that fail; cases that end with an + /// equation, with two, and with a reified type. + #[test] + fn old_side_of_the_pattern_plan_is_not_vacuous() { + let plan = Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN); + let (mut whole, mut cut, mut ok, mut failed) = (0, 0, 0, 0); + let (mut equation, mut two, mut reified) = (0, 0, 0); + for seed in 1..=12u64 { + for sequence in 0..100u64 { + let (program, ops) = case(plan, seed * 1_000_003 + sequence); + let judged = old_judges(&program, plan, &ops); + if judged == ops.len() { + whole += 1; + } else { + cut += 1; + } + let ops = &ops[..judged]; + let outcome = run_old(&program, plan, ops); + for (index, op) in ops.iter().enumerate() { + if matches!(op, Op::Gadt(..)) { + match outcome.form.contains(&format!("! {index} trace\n")) { + true => failed += 1, + false => ok += 1, + } + } + } + let count = equations(&outcome.form).len(); + equation += usize::from(count >= 1); + two += usize::from(count >= 2); + reified += usize::from(outcome.form.contains('$')); + } + } + println!("pattern plan, old side: {whole} {cut} {ok} {failed} {equation} {two} {reified}"); + assert!( + whole >= 380 + && cut >= 20 + && ok >= 300 + && failed >= 160 + && equation >= 40 + && two >= 1 + && reified >= 60, + "{whole} {cut} {ok} {failed} {equation} {two} {reified}" + ); + } + + /// The deliberately wrong new side of [`PATTERN`] + /// ([`Fault::ExpressionMode`]: a pattern unification runs as an + /// expression unification, which the kernel has today) is reported by + /// the generated cases: by a case in which the old checker adds an + /// equation, and by one in which a pattern unification the old + /// checker accepts fails in expression mode (observed, 300 cases: 228 + /// with a pattern unification; reported with an equation on the old + /// side 21, with a failure only the wrong side has 58). + #[test] + fn generated_cases_find_a_pattern_unification_in_expression_mode() { + let plan = Plan::new(Tier::Full, PATTERN_PLAN); + let (mut cases, mut equation, mut refused) = (0, 0, 0); + for seed in 1..=3u64 { + for sequence in 0..100u64 { + let case_seed = seed * 1_000_003 + sequence; + let (program, ops) = case(plan, case_seed); + let ops = &ops[..old_judges(&program, plan, &ops)]; + if !ops.iter().any(|op| matches!(op, Op::Gadt(..))) { + continue; + } + cases += 1; + let old = run_old(&program, plan, ops); + if compare(case_seed, &program, plan, ops, Fault::ExpressionMode).is_err() { + let new = run_new(&program, plan, ops, Fault::ExpressionMode); + equation += usize::from(!equations(&old.form).is_empty()); + refused += usize::from(new.failed.is_some() && old.failed != new.failed); + } + } + } + println!("expression mode fault: {cases} {equation} {refused}"); + assert!( + cases >= 75 && equation >= 7 && refused >= 19, + "{cases} {equation} {refused}" + ); + } + + /// The acceptance test of pattern mode on generated cases (section + /// 8.9): 10,000 cases of the pattern plan, each run on the + /// operations the old checker is a judge of ([`old_judges`]), agree, + /// in the success mode and, 2,000 of them, in the explaining mode; + /// and the fixed cases of the observation tests above agree + /// (equations and their direction, the arguments of one path by the + /// flags, `reify`, what a failure leaves, rows that clash). No seed + /// is pinned: the design turn had no build to run them against, so a + /// seed that differs is read against the OCaml lines of 8.9 before + /// anything is pinned (the build prompt says how). + /// + /// Reaches: R3m, R3e, R3f, R3n, R3h, R3r. + #[test] + #[ignore = "R3r build"] + fn pattern_plan_agrees_on_generated_cases() { + let program = compile(PATTERN_PROGRAM.to_string()); + let plan = Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN); + let mut chain = pattern(0, 0, 2, 3); + chain.extend([Op::Child(0, 1), Op::Gadt(3, 1, 3)]); + let mut twice = refine(3, 2); + twice.extend([ + Op::Child(2, 1), + Op::Instance(3), + Op::Child(5, 2), + Op::Gadt(4, 6, 3), + ]); + let mut compatible = refine(5, 3); + compatible.extend([Op::Child(0, 4), Op::Gadt(1, 4, 3)]); + let mut again = pattern(6, 2, 3, 3); + again.extend([Op::Child(1, 0), Op::Child(2, 0), Op::Unify(3, 4)]); + let below = vec![ + Op::Instance(3), + Op::Child(0, 2), + Op::BeginDef, + Op::Instance(0), + Op::Child(2, 0), + Op::Gadt(3, 1, 3), + ]; + let mut cases: Vec> = vec![chain, twice, compatible, again, below, refine(5, 3)]; + for (value, left, right) in [ + (0, 0, 2), + (0, 2, 0), + (0, 0, 1), + (0, 1, 0), + (0, 0, 3), + (0, 2, 3), + (1, 0, 1), + (1, 0, 3), + (2, 0, 1), + (4, 0, 2), + (4, 0, 3), + (5, 0, 2), + (6, 0, 1), + (6, 2, 3), + ] { + for flags in 0..4 { + cases.push(pattern(value, left, right, flags)); + } + } + for (index, ops) in cases.iter().enumerate() { + compare(index as u64, &program, plan, ops, Fault::None) + .unwrap_or_else(|e| panic!("{e}")); + } + let coverage = run_cases_except(Plan::new(Tier::Full, PATTERN_PLAN), &[]); + assert_eq!(coverage.cases, 10_000); + assert!(coverage.failures * 10 > coverage.cases, "{coverage:?}"); + run_cases(Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN), 1, 20, 100); + } + /// 10,000 cases of everything on programs with polymorphic fields, /// universal variables, rows whose row variable is bound and /// hand-made polymorphic types, and 10,000 more with made nodes: the diff --git a/compiler-rust/src/tc/expand.rs b/compiler-rust/src/tc/expand.rs index 10e51d2a0b..16c5cc9789 100644 --- a/compiler-rust/src/tc/expand.rs +++ b/compiler-rust/src/tc/expand.rs @@ -51,6 +51,16 @@ pub const INHERITS: u8 = 32; /// with [`INHERITS`], in the same pass over the template /// ([`TypeTable::mark_manifests`]). (R3e) pub const OPEN_ROW: u8 = 64; +/// The path is a bare identifier for the unit being checked: one of its +/// own types or a predefined type (`in_current_module`, +/// compiler/ml/ctype.ml:1870-1872: `Pident`). With no [`NEWTYPE`] such a +/// type is `non_aliasable`: no equation refines it and it is compatible +/// with no other type ([`TypeTable::non_aliasable`]). Set by the caller +/// that owns the table, with [`TypeTable::set_local`] (R5a; the tests of +/// R3h): [`TypeTable::from_segment`] does not set it, since a segment +/// does not say whether it is the unit being checked or one it imports, +/// whose types are paths through a module. Read by R3h. +pub const LOCAL: u8 = 128; /// Head flag of a constructor node whose slot 0 holds a private expansion /// (`Mcons (Private, ..)`, compiler/ml/btype.ml:612). Bit 4 is /// `segment::EXTENDED`. @@ -114,6 +124,11 @@ pub struct TypeTable { /// Number of rows with a non-zero `scope`: when 0, lowering never tests a /// scope ([`Checker::lower`]). pub scoped: u32, + /// Per row of `rows`: bit `i` is set when parameter `i` is injective + /// (`Variance.(mem Inj)`). Empty, or shorter than `rows`, for types + /// without an injective parameter ([`TypeTable::injective`]). Filled + /// by the caller that owns the table (R5a). (R3h) + pub injective_bits: Vec, } impl TypeTable { @@ -303,6 +318,52 @@ impl TypeTable { } } + /// `Variance.(mem Inj)` of parameter `parameter` of the type + /// (compiler/ml/ctype.ml:2361, 2018): bit `parameter` of + /// `injective_bits[type_index]`; a type past the end of that table + /// and a parameter past the 32nd are not injective. O(1): one bounds + /// test and one load. (R3h; CHECKER_REWRITE 8.9, decision 66) + #[inline] + pub fn injective(&self, type_index: u32, parameter: u32) -> bool { + match self.injective_bits.get(type_index as usize) { + Some(&bits) => parameter < 32 && bits >> parameter & 1 != 0, + None => false, + } + } + + /// O(1): `type_index` is a bare identifier for the unit being checked + /// ([`LOCAL`]). One bounds test and one byte written. + pub fn set_local(&mut self, type_index: u32) { + self.rows[type_index as usize].flags |= LOCAL; + } + + /// `non_aliasable p decl` (compiler/ml/ctype.ml:1870-1872) for a + /// row: a bare identifier that is not a locally abstract type. O(1), + /// no load. + #[inline] + pub fn non_aliasable(row: TypeInfo) -> bool { + row.flags & (LOCAL | NEWTYPE) == LOCAL + } + + /// `is_instantiable env p` (compiler/ml/ctype.ml:1874-1881) for the + /// type of a constructor node WITHOUT arguments: the type has a row + /// (`Env.find_type` did not raise), is abstract (not [`DATATYPE`]), + /// has no manifest (none of [`ABBREV`], [`PRIVATE`], [`ALIAS`]: a + /// private abstract type always has one, so `type_private = Public` + /// follows) and is not [`TypeTable::non_aliasable`]. `type_arity = 0` + /// is the node's own arity. One bounds test and one row load. A type + /// an equation refined has [`ABBREV`] and is no longer instantiable, + /// as its local constraint has a manifest. (R3h) + #[inline] + pub fn instantiable(&self, type_index: u32) -> bool { + match self.rows.get(type_index as usize) { + Some(&row) => { + row.flags & (DATATYPE | ABBREV | PRIVATE | ALIAS) == 0 && !Self::non_aliasable(row) + } + None => false, + } + } + /// O(1): one bounds test and one 16-byte load. (R3b) #[inline] pub fn info(&self, type_index: u32) -> TypeInfo { diff --git a/compiler-rust/src/tc/r3_tests.rs b/compiler-rust/src/tc/r3_tests.rs index 149fbab12c..6b90efafa4 100644 --- a/compiler-rust/src/tc/r3_tests.rs +++ b/compiler-rust/src/tc/r3_tests.rs @@ -3352,6 +3352,480 @@ fn the_types_that_hold_a_universal_variable_are_unified_not_gained() { } } +/// A closed row with the one tag `#A`, `Reither (false, types, false, ref +/// None)` with any number of types (a conjunction), unopened, at level 2. +fn conjunctive_row(c: &mut Checker, types: &[Ty], more: Ty) -> Ty { + let name = name_index(c, "A"); + let start = c.extras.row_fields.len() as u32; + let kind = RowFieldKind::Either { + constant: false, + matched: false, + }; + let published = types.len() as u32 + 1; + c.extras.row_fields.push(RowFieldRow { + name, + types: Range { + start: 0, + len: types.len() as u32, + }, + link: published, + kind, + }); + c.extras.variants.push(VariantRow { + fields: Range { start, len: 1 }, + more: types.len() as u32, + name: NONE, + name_arguments: Range::default(), + closed: true, + fixed: false, + }); + let index = c.extras.variants.len() as u32 - 1; + let mut slots = types.to_vec(); + slots.push(more); + extended_at(c, 2, ExtraDescriptor::Variant(index), &slots) +} + +/// What two `Reither` fields gain is decided before the types that hold +/// a universal variable are unified (section 8.6, decisions 53 and 58; +/// compiler/ml/ctype.ml:2685-2706): +/// +/// ```text +/// let tl2' = remq tl2 tl1 and tl1' = remq tl1 tl2 in +/// let tl1', tlu1 = split_univars tl1' and tl2', tlu2 = split_univars tl2' in +/// (match (tlu1, tlu2) with +/// | [], [] -> () +/// | tu1 :: tlu1, _ :: _ -> +/// if not !(passive_variants ()) then List.iter (unify env tu1) (tlu1 @ tlu2) +/// | tu :: _, [] | [], tu :: _ -> occur_univar !env tu); +/// List.iter (update_level !env (repr more).level) (tl1' @ tl2'); +/// let e = ref None in +/// let f1' = Reither (c1 || c2, tl1', m1 || m2, e) +/// and f2' = Reither (c1 || c2, tl2', m1 || m2, e) in +/// ``` +/// +/// Witness (the old checker's side is `differential::tests:: +/// old_checker_gains_a_type_the_holders_made_a_duplicate`): `poly([< #A((u, +/// x)) & x]; u)` against `poly([< #A((u', x')) & x']; u')`. `tl1'` is +/// `[(u', x'); x']`: the tuple holds a universal variable, `x'` is kept. +/// The two tuples are unified, which makes `x'` a link to `x`... and the +/// field of row 1 still gains `x'`: it reads `[(u, x); x; x]`, three +/// types of which the last two are one node, and so does the field of +/// row 2. A join that compared the lists again after the unification +/// would gain nothing (two types): what `split_univars` did before the +/// sixth skeleton turn. The body is `Checker::join_either`'s second run +/// and the arm of `Checker::split_univars` with holders on both sides. +/// Reaches: R3f, R3g. +#[test] +fn a_field_gains_what_it_lacked_before_the_holders_were_unified() { + for mode in [Mode::Success, Mode::Explain] { + let mut c = checker(mode); + let side = |c: &mut Checker| { + let u = univar_at(c, 2); + let x = var(c); + let holder = tuple(c, &[u, x]); + let more = var(c); + let row = conjunctive_row(c, &[holder, x], more); + (poly_of(c, 2, row, &[u]), row, holder, x) + }; + let (left, _, holder1, x1) = side(&mut c); + let (right, row2, holder2, x2) = side(&mut c); + let rows = rows_now(&c); + c.unify(left, right).unwrap(); + assert_rows_kept(&c, &rows); + assert_eq!(c.g.repr(holder1), c.g.repr(holder2)); + assert_eq!(c.g.repr(x1), c.g.repr(x2)); + let (tags, _, _) = read_row(&mut c, row2); + assert_eq!(tags[0].1.types, 3, "{mode:?}"); + let mut types = Vec::new(); + let row2 = c.g.repr(row2); + let node = c.g.nodes[row2.0 as usize]; + let ExtraDescriptor::Variant(index) = c.extras.descriptors[node.a as usize] else { + panic!("a variant"); + }; + let field = c.extras.row_fields[c.extras.variants[index as usize].fields.start as usize]; + super::rows::field_types(&c.g, node.b, node.arity(), field, &mut types); + let types: Vec = types.into_iter().map(|ty| c.g.repr(ty)).collect(); + let x = c.g.repr(x1); + assert_eq!(types, [c.g.repr(holder1), x, x]); + assert!(c.scopes.is_empty() && c.frames.is_empty() && c.saved.is_empty()); + } +} + +/// An open row `[> #tag] as P` at `level`, unopened: slot 0 its +/// row variable (new, at `level`), slot 1 the argument of its name. +fn named_row(c: &mut Checker, level: u32, tag: &str, argument: Option) -> Ty { + let name = name_index(c, tag); + let start = c.extras.row_fields.len() as u32; + c.extras.row_fields.push(RowFieldRow { + name, + types: Range::default(), + link: NONE, + kind: RowFieldKind::Present, + }); + c.extras.variants.push(VariantRow { + fields: Range { start, len: 1 }, + more: 0, + name: if argument.is_some() { PA } else { NONE }, + name_arguments: Range { + start: 1, + len: u32::from(argument.is_some()), + }, + closed: false, + fixed: false, + }); + let index = c.extras.variants.len() as u32 - 1; + let mut slots = vec![var_at(c, level)]; + slots.extend(argument); + extended_at(c, level, ExtraDescriptor::Variant(index), &slots) +} + +/// The arguments of a row's name are children of the row only while the +/// row is the one its chain ends in (section 8.6, decision 60; +/// compiler/ml/btype.ml:229-241, `iter_row`): +/// +/// ```text +/// match (repr row.row_more).desc with +/// | Tvariant row -> iter_row f row +/// | Tvar _ | Tunivar _ | Tsubst _ | Tconstr _ | Tnil -> +/// Misc.may (fun (_, l) -> List.iter f l) row.row_name +/// ``` +/// +/// Witness, per walk: `[> #B]` unified with `[> #A] as P` at level 2, +/// `y` in no field. The named row is the right node, gains `#B` through +/// a variant node its row variable is linked to, and the merged row has +/// no name (ctype.ml:2574-2586, 2596-2616). Then, each on a graph of its +/// own: +/// - `Checker::lower` of the row to level 1 leaves `y` at 2 +/// (`update_level`, ctype.ml:584-630, through `iter_type_expr`); +/// - `Checker::generalize` after the definition leaves `y` at 2 +/// (`generalize`, ctype.ml:509-516); +/// - `Checker::deep_occur` of `y` in the row is false (ctype.ml:1735-1782); +/// - a variable of level 1 bound to the row (the binding walk, which +/// lowers below a row) leaves `y` at 2; +/// - with `y` a universal variable, `Checker::free_univar` of the row +/// finds none (`occur_univar`, ctype.ml:1563-1607); +/// - `Checker::univars_escape` of a binder of that universal variable in +/// a polymorphic type whose body is the row is false +/// (ctype.ml:1630-1652); +/// - a `Checker::copy` of the generalized row with `y` generic makes one +/// variable, the row variable's, and no copy of `y` (`row_repr` gives a +/// copy the innermost row's name, compiler/ml/btype.ml:126-135, +/// ctype.ml:800-868). +/// +/// The old checker's side, observed: `differential::tests:: +/// old_checker_does_not_reach_the_arguments_of_a_stale_row_name`. Before +/// the sixth skeleton turn every one of these walks reached `y` (each +/// pushes every slot of a node); the body is +/// `Checker::drop_name_children`, called by `Checker::unify_row`. While +/// the row IS the innermost, its name's arguments are children: the +/// last block. +/// Reaches: R3f, R3n, R3g. +#[test] +fn the_arguments_of_a_stale_row_name_are_not_children() { + // The named row after it gained `#B`, and the argument of its name. + let stale = |c: &mut Checker, y: Ty| { + let other = named_row(c, 2, "B", None); + let named = named_row(c, 2, "A", Some(y)); + let rows = rows_now(c); + c.unify(other, named).unwrap(); + assert_rows_kept(c, &rows); + assert_eq!(shape(c, named), "[A B | ']"); + named + }; + let level = |c: &mut Checker, ty: Ty| { + let ty = c.g.repr(ty); + c.g.nodes[ty.0 as usize].level + }; + for mode in [Mode::Success, Mode::Explain] { + let mut c = checker(mode); + let y = var(&mut c); + let named = stale(&mut c, y); + c.lower(named, 1).unwrap(); + assert_eq!((level(&mut c, named), level(&mut c, y)), (1, 2), "lower"); + + let mut c = checker(mode); + let y = var(&mut c); + let named = stale(&mut c, y); + c.g.end_def(); + c.generalize(named); + assert_eq!( + (level(&mut c, named), level(&mut c, y)), + (GENERIC, 2), + "generalize" + ); + + let mut c = checker(mode); + let y = var(&mut c); + let named = stale(&mut c, y); + assert!(!c.deep_occur(y, named), "deep_occur"); + + let mut c = checker(mode); + let y = var(&mut c); + let named = stale(&mut c, y); + let v = var_at(&mut c, 1); + c.unify(v, named).unwrap(); + assert_eq!((level(&mut c, named), level(&mut c, y)), (1, 2), "binding"); + + let mut c = checker(mode); + let u = univar_at(&mut c, 2); + let named = stale(&mut c, u); + assert_eq!(c.free_univar(named), Ty(NONE), "free_univar"); + let binder = poly_of(&mut c, 2, u, &[u]); + let w = univar_at(&mut c, 2); + let other = poly_of(&mut c, 2, named, &[w]); + let (binder, other) = (c.g.nodes[binder.0 as usize], c.g.nodes[other.0 as usize]); + assert!(!c.univars_escape(binder, other), "univars_escape"); + + let mut c = checker(mode); + let y = var_at(&mut c, GENERIC); + let named = stale(&mut c, y); + c.g.end_def(); + c.generalize(named); + let first = c.g.nodes.len(); + let copy = c.copy(named); + assert_eq!(shape(&mut c, copy), "[A B | ']"); + let variables = c.g.nodes[first..] + .iter() + .filter(|node| node.tag() == VAR) + .count(); + assert_eq!(variables, 1, "copy"); + } + // The innermost row's name: its argument is a child. + let mut c = checker(Mode::Success); + let y = var(&mut c); + let named = named_row(&mut c, 2, "A", Some(y)); + assert!(c.deep_occur(y, named)); + c.lower(named, 1).unwrap(); + assert_eq!(level(&mut c, y), 1); +} + +/// `free_univar` over a variant row (compiler/ml/btype.ml:229-241, +/// 263-265; ctype.ml:1563-1607): the fields' types, then the arguments +/// of the name, then the row variable: +/// +/// ```text +/// | Tvariant row -> +/// iter_row f row; +/// f (row_more row) +/// ``` +/// +/// Witnesses, each a row at level 2 with free universal variables in +/// three places, found in that order: +/// - `[> #A(u0)] as P` whose row variable is the universal variable +/// `u2`: `u0`; +/// - `[> #A(int)] as P` over `u2`: `u1`; +/// - `[> #A(int)]` over `u2`: `u2`; +/// - inside `poly(...; u2)`, the same row: none (the row variable is +/// bound); +/// - an OPENED row `[< #A(u0)]` (`Reither`, its type under the chain +/// node of its reference): `u0`; and after that reference is resolved +/// to absent, none: `iter_row` visits what `row_field_repr` gives. +/// +/// The body is the `Variant` arm of +/// `Checker::push_free_univar_children`. +/// Reaches: R3f, R3g. +#[test] +fn a_free_universal_variable_is_found_in_a_row_in_order() { + let mut c = checker(Mode::Success); + // `[> #A(argument)] as P` over `more`. + let row = |c: &mut Checker, argument: Ty, named: Option, more: Ty| { + let name = name_index(c, "A"); + let start = c.extras.row_fields.len() as u32; + c.extras.row_fields.push(RowFieldRow { + name, + types: Range { start: 0, len: 1 }, + link: NONE, + kind: RowFieldKind::Present, + }); + c.extras.variants.push(VariantRow { + fields: Range { start, len: 1 }, + more: 1, + name: if named.is_some() { PA } else { NONE }, + name_arguments: Range { + start: 2, + len: u32::from(named.is_some()), + }, + closed: false, + fixed: false, + }); + let index = c.extras.variants.len() as u32 - 1; + let mut slots = vec![argument, more]; + slots.extend(named); + extended_at(c, 2, ExtraDescriptor::Variant(index), &slots) + }; + let [u0, u1, u2] = [(); 3].map(|()| univar_at(&mut c, 2)); + let int = constr(&mut c, INT, &[]); + let first = row(&mut c, u0, Some(u1), u2); + assert_eq!(c.free_univar(first), u0); + let second = row(&mut c, int, Some(u1), u2); + assert_eq!(c.free_univar(second), u1); + let third = row(&mut c, int, None, u2); + assert_eq!(c.free_univar(third), u2); + let bound = poly_of(&mut c, 2, third, &[u2]); + assert_eq!(c.free_univar(bound), Ty(NONE)); + + let more = var(&mut c); + let either = variant_at( + &mut c, + 2, + &[("A", Tag::Either(false, Some(u0), false))], + more, + true, + NONE, + ); + c.open_row(either); + assert_eq!(c.free_univar(either), u0); + let (tags, _, _) = read_row(&mut c, either); + let holders = c.resolve(tags[0].1.end, super::rows::RES_ABSENT, Ty(NONE)); + c.settle(holders, super::rows::RES_ABSENT); + assert_eq!(c.free_univar(either), Ty(NONE)); +} + +/// The split of the types that hold a universal variable while the +/// variants are passive (compiler/ml/ctype.ml:2693-2703): +/// +/// ```text +/// | tu1 :: tlu1, _ :: _ -> +/// (* Attempt to merge all the types containing univars *) +/// if not !(passive_variants ()) then +/// List.iter (unify env tu1) (tlu1 @ tlu2) +/// | tu :: _, [] | [], tu :: _ -> occur_univar !env tu); +/// (* Is this handling of levels really principal? *) +/// List.iter (update_level !env (repr more).level) (tl1' @ tl2'); +/// ``` +/// +/// Witness: `poly([< #A(z & (u, x))]; u)` against `poly([< #A(z' & (u', +/// x'))]; u')`, every type at level 2 and the two row variables at level +/// 1, with `Extras::passive_variants` set (nothing in `tc` sets it: the +/// typing of a pattern for an exhaustiveness check does, `with_passive_ +/// variants`, ctype.ml:2189-2202, from typecore.ml:1931; section 8.9). +/// The two tuples hold a universal variable and are NOT unified and NOT +/// lowered: they stay two nodes at level 2. `z` and `z'` are gained and +/// lowered to the level of the merged row variable, 1. Each field then +/// has its two types and the one it gained. The holder comes after the +/// kept type in each list, so that the place between the two kept +/// ranges of the scratch holds a holder: the range `Checker::join_either` +/// lowered before the review of R3g (section 8.6, decision 54). The +/// bodies are the passive arm of `Checker::split_univars` and the two +/// lowering loops of `Checker::join_either`. +/// Reaches: R3f, R3g. +#[test] +fn passive_variants_keep_the_levels_of_the_types_that_hold_a_universal_variable() { + let mut c = checker(Mode::Success); + c.extras.passive_variants = true; + let side = |c: &mut Checker| { + let u = univar_at(c, 2); + let (x, z) = (var(c), var(c)); + let holder = tuple(c, &[u, x]); + let more = var_at(c, 1); + let row = conjunctive_row(c, &[z, holder], more); + (poly_of(c, 2, row, &[u]), row, holder, z) + }; + let (left, _, holder1, z1) = side(&mut c); + let (right, row2, holder2, z2) = side(&mut c); + c.unify(left, right).unwrap(); + let level = |c: &Checker, ty: Ty| c.g.nodes[ty.0 as usize].level; + assert_ne!(c.g.repr(holder1), c.g.repr(holder2)); + assert_eq!((level(&c, holder1), level(&c, holder2)), (2, 2)); + assert_eq!((level(&c, z1), level(&c, z2)), (1, 1)); + let (tags, _, more) = read_row(&mut c, row2); + assert_eq!(tags[0].1.types, 3); + assert_eq!(level(&c, more), 1); + assert!(c.scopes.is_empty() && c.frames.is_empty() && c.saved.is_empty()); +} + +/// The cost of the two walks for universal variables, as relations +/// between n = 1,000 and n = 10,000 (section 8.6, decisions 56 and 59). +/// - `Checker::free_univar` over a tuple of n variables inside one +/// binder: one visit a node, linear. +/// - `Checker::free_univar` over a chain of n polymorphic types WITHOUT +/// bound variables in which every type is held twice, once under such +/// a type and once outside it: `t(0)` is the bound variable, `t(i + 1)` +/// is `(poly(t(i);), t(i))`, the whole inside one binder. A +/// polymorphic type without bound variables adds nothing to `bound` +/// (`List.fold_right Type_set.add (List.map repr tyl) bound`, +/// compiler/ml/ctype.ml:1587-1589), so OCaml visits each node once +/// (ctype.ml:1569-1581: with the same `bound` a node is not visited +/// again). So does the walk: exactly `2n + 1` visits (n tuples, n +/// polymorphic types, the binder), ZERO extra. Before the sixth +/// skeleton turn such a type opened a context, a node first met inside +/// it was visited again after it was left, and this chain took `2^n` +/// visits: the test would not end. +/// - `Checker::univars_escape` of a known binder over a tuple of n +/// variables: the family pass and one visit a node, linear. +/// +/// Reaches: R3g. +#[test] +fn the_walks_for_universal_variables_at_ten_thousand() { + let wide = |n: u32| { + let mut c = checker(Mode::Success); + let u = univar_at(&mut c, 2); + let mut elements: Vec = (0..n).map(|_| var(&mut c)).collect(); + elements.push(u); + let body = tuple(&mut c, &elements); + let ty = poly_of(&mut c, 2, body, &[u]); + let mut found = u; + let count = steps::scope(|| found = c.free_univar(ty)); + assert_eq!(found, Ty(NONE)); + count + }; + let (small, large) = (wide(1_000), wide(10_000)); + assert!(large <= 11 * small, "wide: {small} {large}"); + + let shared = |n: u32| { + let mut c = checker(Mode::Success); + let u = univar_at(&mut c, 2); + let mut ty = u; + for _ in 0..n { + let inner = poly_of(&mut c, 2, ty, &[]); + ty = tuple(&mut c, &[inner, ty]); + } + let ty = poly_of(&mut c, 2, ty, &[u]); + let mut found = u; + let count = steps::scope(|| found = c.free_univar(ty)); + assert_eq!(found, Ty(NONE)); + count + }; + assert_eq!(shared(1_000), 2 * 1_000 + 1); + assert_eq!(shared(10_000), 2 * 10_000 + 1); + + let escape = |n: u32| { + let mut c = checker(Mode::Success); + // An open scope that pairs `u` with `u2`, so that a type that + // binds `u` again has a known binder. + let (u, u2) = (univar_at(&mut c, 2), univar_at(&mut c, 2)); + let (outer1, outer2) = (poly_of(&mut c, 2, u, &[u]), poly_of(&mut c, 2, u2, &[u2])); + let (n1, n2) = (c.g.nodes[outer1.0 as usize], c.g.nodes[outer2.0 as usize]); + let partners = c.g.push(partner_node(Ty(NONE))); + c.g.push(partner_node(Ty(NONE))); + c.scopes.push(PolyScope { + left: n1.b + 1, + right: n2.b + 1, + left_len: 1, + right_len: 1, + partners, + }); + let binder = poly_of(&mut c, 2, u, &[u]); + let w = univar_at(&mut c, 2); + let elements: Vec = (0..n).map(|_| var(&mut c)).collect(); + let body = tuple(&mut c, &elements); + let other = poly_of(&mut c, 2, body, &[w]); + let (binder, other) = (c.g.nodes[binder.0 as usize], c.g.nodes[other.0 as usize]); + let mut escapes = true; + let count = steps::scope(|| escapes = c.univars_escape(binder, other)); + assert!(!escapes); + // With `u2` in the body it escapes: `u2` is in the family of `u`. + let body = tuple(&mut c, &[u2]); + let other = poly_of(&mut c, 2, body, &[w]); + let other = c.g.nodes[other.0 as usize]; + assert!(c.univars_escape(binder, other)); + c.scopes.clear(); + count + }; + let (small, large) = (escape(1_000), escape(10_000)); + assert!(large <= 11 * small, "escape: {small} {large}"); +} + /// The cost of universal variables, as relations between n = 1,000 and /// n = 10,000. /// - n bound variables, the body the last of them: `unify_univar` scans @@ -3735,3 +4209,736 @@ fn packages_of_ten_thousand_names_are_linear() { let (small, large) = (run(1_000), run(10_000)); assert!(large <= 11 * small, "{small} {large}"); } + +// --------------------------------------------------------------------------- +// Pattern mode (R3h, R3r; CHECKER_REWRITE 8.9) +// --------------------------------------------------------------------------- + +use super::expand::{DATATYPE, NEWTYPE, Newtype}; +use super::rows::{DeclMatch, LocalDecls}; +use super::unify::UnifyMode; + +/// `(type n0)` at level 1 and `(type n1)` at level 2: locally abstract. +const N0: u32 = 14; +const N1: u32 = 15; +/// `type k<'a>` of another unit: abstract, not injective. +const K: u32 = 16; +/// `type d<'a> = D('a)` and `type e<'a> = E('a)` of another unit. +const D: u32 = 17; +const E: u32 = 18; +/// `type t2` of another unit: abstract, as `T`. +const T2: u32 = 19; +/// The first type `reify` makes. +const FRESH: u32 = 20; + +/// The supplier of the hand-built tests: counts the fresh types from +/// [`FRESH`], records the name index each was asked with, and answers +/// `compare` with `answer` after pushing `members`. +struct Fresh { + next: u32, + asked: std::rc::Rc>>, + answer: DeclMatch, + members: Vec<[Ty; 2]>, +} + +impl LocalDecls for Fresh { + fn fresh_abstract(&mut self, name: u32) -> u32 { + self.asked.borrow_mut().push(name); + self.next += 1; + self.next - 1 + } + + fn compare(&mut self, c: &mut Checker, _: u32, _: u32) -> DeclMatch { + for &[left, right] in &self.members { + c.push_compatible(left, right); + } + self.answer + } +} + +/// [`checker`] with the declarations of pattern mode: `int`, `string` +/// and `unit` are bare identifiers (`LOCAL`), `n0` and `n1` locally +/// abstract types of the newtype levels 1 and 2, `k`, `d`, `e` and `t2` +/// types of another unit, and a supplier whose record of asked names is +/// returned. +fn pattern_checker(mode: Mode) -> (Checker, std::rc::Rc>>) { + let mut c = checker(mode); + for ty in [INT, STRING, UNIT] { + c.types.set_local(ty); + } + let row = |arity: u16, flags: u8| TypeInfo { + manifest: NONE, + scope: 0, + newtype: NONE, + arity, + flags, + reserved: 0, + }; + for level in [1, 2] { + c.types.newtypes.push(Newtype { + level, + expansion_level: level, + }); + c.types.rows.push(TypeInfo { + scope: level, + newtype: c.types.newtypes.len() as u32 - 1, + ..row(0, NEWTYPE) + }); + c.types.scoped += 1; + } + c.types + .rows + .extend([row(1, 0), row(1, DATATYPE), row(1, DATATYPE), row(0, 0)]); + assert_eq!(c.types.rows.len() as u32, FRESH); + let asked = std::rc::Rc::new(std::cell::RefCell::new(Vec::new())); + c.locals = Some(Box::new(Fresh { + next: FRESH, + asked: asked.clone(), + answer: DeclMatch::Clash, + members: Vec::new(), + })); + (c, asked) +} + +/// A variable at `level` whose name is `text`. +fn named_var_at(c: &mut Checker, level: u32, text: &str) -> Ty { + let name = name_index(c, text); + let ty = var_at(c, level); + c.g.nodes[ty.0 as usize].a = name; + ty +} + +/// The checker is as a pattern unification must leave it, on either +/// result: expression mode, both flags off, no newtype level, no pair +/// assumed, no frame, no open attempt. +fn assert_pattern_left(c: &Checker) { + assert!(c.umode == UnifyMode::Expression); + assert!(!c.generate_equations && !c.assume_injective); + assert_eq!(c.newtype_level, 0); + assert!(c.pairs.keys.is_empty() && c.compatible.keys.is_empty()); + assert!(c.frames.is_empty() && c.tries.is_empty() && c.scopes.is_empty()); +} + +/// Witness: `let f = (type a, x: t) => switch x { | Int => ... }` +/// with `type rec t<_> = Int: t`: the result `t` of the +/// constructor against `t`, at the arguments. A locally abstract type +/// against a type that is not one gets the equation on either side +/// (compiler/ml/ctype.ml:2385-2392), the two nodes stay two nodes, and +/// the type is no longer `instantiable`. Under the equation an +/// expression unification of the two succeeds by expansion. After +/// `cut_equations(0)` the row is what it was and a new `n0` does not +/// unify with `int`. +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn an_equation_goes_to_the_locally_abstract_type_and_links_nothing() { + for swapped in [false, true] { + let (mut c, _) = pattern_checker(Mode::Success); + let before = c.types.rows[N0 as usize]; + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let pair = if swapped { [int, n0] } else { [n0, int] }; + assert!(c.unify_pattern(2, true, true, pair[0], pair[1]).is_ok()); + assert_pattern_left(&c); + assert_eq!(c.equations.len(), 1); + assert_eq!( + (c.equations[0].type_index, c.equations[0].previous), + (N0, before) + ); + let row = c.types.rows[N0 as usize]; + assert_eq!(row.flags & (ABBREV | NEWTYPE), ABBREV | NEWTYPE); + assert!(!c.types.instantiable(N0)); + assert_eq!((c.g.repr(n0), c.g.repr(int)), (n0, int)); + let (other, target) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + assert!(c.unify(other, target).is_ok()); + c.cut_equations(0); + assert!(c.equations.is_empty()); + assert_eq!(c.types.rows[N0 as usize], before); + let (other, target) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + assert!(c.unify(other, target).is_err()); + } +} + +/// Witness: `(type a, type b, x: eq) => switch x { | Refl => ... }` +/// with `Refl: eq<'c, 'c>`, `b` declared inside `a`. Of two locally +/// abstract types the one of the higher newtype level gets the equation, +/// on either side (compiler/ml/ctype.ml:2373-2384): `n1` (level 2), with +/// `n0` (level 1) its body. A locally abstract type against an abstract +/// type of another unit (`find_newtype_level` gives `(0, 0)` for it, +/// ctype.ml:2085-2091): the locally abstract one. Two abstract types of +/// another unit have one level: the RIGHT one gets it (`else (path', +/// t1')`). +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn of_two_instantiable_types_the_newer_gets_the_equation() { + for (left, right, source) in [ + (N0, N1, N1), + (N1, N0, N1), + (N0, T, N0), + (T, N0, N0), + (T, T2, T2), + (T2, T, T), + ] { + let (mut c, _) = pattern_checker(Mode::Success); + let (l, r) = (constr(&mut c, left, &[]), constr(&mut c, right, &[])); + assert!(c.unify_pattern(2, true, true, l, r).is_ok()); + assert_eq!(c.equations.len(), 1, "{left} {right}"); + assert_eq!(c.equations[0].type_index, source, "{left} {right}"); + let other = if source == left { right } else { left }; + let node = constr(&mut c, source, &[]); + let expansion = c.expand_head(node, ExpandMode::Unif).unwrap(); + assert_eq!(path(&mut c, expansion), other); + } +} + +/// Witness: `Pair: t<(int, string)>` against `t`. The body of an +/// equation is a generic copy (`duplicate_type destination`, +/// compiler/ml/ctype.ml:2095): every node of it is generic, it is not +/// the destination, and the destination has the levels it had. An +/// expansion of the type is a copy at the level of the node expanded. +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn the_body_of_an_equation_is_a_generic_copy() { + let (mut c, _) = pattern_checker(Mode::Success); + let (int, string) = (constr(&mut c, INT, &[]), constr(&mut c, STRING, &[])); + let destination = tuple(&mut c, &[int, string]); + let n0 = constr(&mut c, N0, &[]); + assert!(c.unify_pattern(2, true, true, n0, destination).is_ok()); + let manifest = c.types.manifests[c.types.rows[N0 as usize].manifest as usize]; + let body = manifest.body; + assert_ne!(body, destination); + for ty in [body, child(&mut c, body, 0), child(&mut c, body, 1)] { + assert_eq!(c.g.nodes[ty.0 as usize].level, GENERIC); + } + for ty in [destination, int, string] { + assert_eq!(c.g.nodes[ty.0 as usize].level, 2); + } + let expansion = c.expand_head(n0, ExpandMode::Unif).unwrap(); + assert!(expansion != body && expansion != destination); + assert_eq!(c.g.nodes[expansion.0 as usize].level, 2); + let second = child(&mut c, expansion, 1); + assert_eq!(path(&mut c, second), STRING); +} + +/// Witness: `Pack: t<('x, ('y, 'x))>` against `t` after the instance +/// of the constructor: its variables are at the newtype level. `reify` +/// makes one new abstract type for each variable, in the order +/// `iter_type_expr` visits them, each once (compiler/ml/ctype.ml: +/// 1828-1840): the supplier is asked with the name of `x`, then of `y`. +/// Each variable is a link to a nullary constructor of its type at the +/// variable's level; the type's row is a locally abstract type of the +/// newtype level (`new_declaration (Some (newtype_level, newtype_level)) +/// None`, ctype.ml:1816). +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn reify_makes_one_type_for_each_variable_in_visit_order() { + let (mut c, asked) = pattern_checker(Mode::Success); + let (x, y) = (named_var_at(&mut c, 2, "x"), named_var_at(&mut c, 3, "y")); + let inner = tuple(&mut c, &[y, x]); + let destination = tuple(&mut c, &[x, inner]); + let n0 = constr(&mut c, N0, &[]); + assert!(c.unify_pattern(2, true, true, n0, destination).is_ok()); + assert_eq!( + *asked.borrow(), + [name_index(&mut c, "x"), name_index(&mut c, "y")] + ); + for (variable, level, ty) in [(x, 2, FRESH), (y, 3, FRESH + 1)] { + let node = c.g.repr(variable); + assert_ne!(node, variable); + assert_eq!(path(&mut c, node), ty); + assert_eq!(c.g.nodes[node.0 as usize].level, level); + let row = c.types.rows[ty as usize]; + assert_eq!((row.flags, row.arity), (NEWTYPE, 0)); + let newtype = c.types.newtypes[row.newtype as usize]; + assert_eq!((newtype.level, newtype.expansion_level), (2, 2)); + } + assert_eq!(c.equations.len(), 1); +} + +/// Witness: a variable of the scrutinee's type that is older than the +/// pattern (`(x: t<'a>) => switch x { | Int => ... }` inside a function +/// whose `'a` is not generalized yet). `reify` links the variable to its +/// new type and THEN refuses it, because it is below the newtype level +/// (`link_type ty t; if ty.level < newtype_level then raise (Unify [(t, +/// newvar2 ty.level)])`, compiler/ml/ctype.ml:1837-1839): the +/// unification fails, the variable stays a link (observed on the old +/// checker: decision 62), no equation is added, and in the explaining +/// mode the last pair of the trace is the new type and a new variable at +/// the variable's level. +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn reify_links_a_variable_below_the_newtype_level_and_clashes() { + for mode in [Mode::Success, Mode::Explain] { + let (mut c, _) = pattern_checker(mode); + let old = var_at(&mut c, 1); + let int = constr(&mut c, INT, &[]); + let destination = tuple(&mut c, &[old, int]); + let n0 = constr(&mut c, N0, &[]); + assert!(c.unify_pattern(2, true, true, n0, destination).is_err()); + assert_pattern_left(&c); + assert!(c.equations.is_empty()); + let node = c.g.repr(old); + assert_ne!(node, old); + assert_eq!(path(&mut c, node), FRESH); + assert_eq!(c.g.nodes[node.0 as usize].level, 1); + if mode == Mode::Explain { + let [left, right] = *c.trace.last().unwrap(); + assert_eq!(left, node); + let variable = c.g.nodes[right.0 as usize]; + assert_eq!((variable.tag(), variable.level), (VAR, 1)); + } + } +} + +/// Witness: `Int: t` against `t>`'s argument and its +/// relatives: two constructors of one type in pattern mode +/// (compiler/ml/ctype.ml:2349-2358). As `unify_gadt` runs it +/// (`assume_injective`) the arguments are unified with equations: +/// `k` against `k` gives `n0 := int`. Without +/// `assume_injective` a data type (`d`) and a type of the unit (`LOCAL`, +/// here set on `box`) still do. Without `generate_equations` the +/// arguments are unified without equations: `n0` against `int` is then +/// reified and nothing more (ctype.ml:2393-2397): a success and no +/// equation. The two constructor nodes are never linked, and the flags +/// are what they were afterwards (`set_mode_pattern` puts them back, +/// ctype.ml:256-273). +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn the_arguments_of_one_type_are_unified_by_the_flags() { + for (ty, generate, injective, equation) in [ + (K, true, true, true), + (D, true, false, true), + (BOX, true, false, true), + (K, false, true, false), + (K, false, false, false), + (D, false, false, false), + ] { + let (mut c, _) = pattern_checker(Mode::Success); + c.types.set_local(BOX); + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let (left, right) = (constr(&mut c, ty, &[n0]), constr(&mut c, ty, &[int])); + assert!(c.unify_pattern(2, generate, injective, left, right).is_ok()); + assert_pattern_left(&c); + assert_eq!( + c.equations.len(), + usize::from(equation), + "{ty} {generate} {injective}" + ); + assert_eq!((c.g.repr(left), c.g.repr(right)), (left, right)); + assert_eq!((c.g.repr(n0), c.g.repr(int)), (n0, int)); + } +} + +/// Witness: `Pair: t<(int, int)>` against `t<(a, int)>`. Two structures +/// in pattern mode are assumed equal and NOT linked (`Pattern -> +/// add_type_equality t1' t2'`, compiler/ml/ctype.ml:2327-2331): the old +/// checker links them and is no judge here (decision 62). The two +/// tuples are two nodes afterwards, the equation `n0 := int` is added, +/// and the table of assumed pairs is empty again. A clash below +/// (`(n0, int)` against `(int, string)`) leaves both tuples as they +/// were, in both modes, and the equation its first element added stays +/// until the caller cuts it (`env` is not restored by a failure). +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn two_structures_are_assumed_equal_and_not_linked() { + for mode in [Mode::Success, Mode::Explain] { + for clash in [false, true] { + let (mut c, _) = pattern_checker(mode); + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let second = constr(&mut c, if clash { STRING } else { INT }, &[]); + let (left, right) = (tuple(&mut c, &[n0, int]), tuple(&mut c, &[int, second])); + let (l, r) = (c.g.nodes[left.0 as usize], c.g.nodes[right.0 as usize]); + let result = c.unify_pattern(2, true, true, left, right); + assert_eq!(result.is_err(), clash); + assert_pattern_left(&c); + assert_eq!( + (c.g.nodes[left.0 as usize], c.g.nodes[right.0 as usize]), + (l, r) + ); + assert_eq!(c.equations.len(), 1); + assert_eq!(c.equations[0].type_index, N0); + if clash && mode == Mode::Explain { + assert_eq!(c.trace.first(), Some(&[left, right])); + } + } + } +} + +/// Witness: `(type a, type b, x: eq, y: a): b => switch x { | Refl +/// => y }`: in the case, `a` against `b` in expression mode under `b := +/// a`. The arm of `unify` for two locally abstract types under local +/// constraints expands the NEWER one once and no more +/// (compiler/ml/ctype.ml:2258-2270): `n1` expands to a copy of `n0`, +/// which is linked to the other `n0` node by the arm for two nullary +/// constructors of one type, and that node is still `n0`, not its own +/// expansion `int`. +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn the_newer_of_two_locally_abstract_types_is_expanded_once() { + let (mut c, _) = pattern_checker(Mode::Success); + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + assert!(c.unify_pattern(2, true, true, n0, int).is_ok()); + let (n1, older) = (constr(&mut c, N1, &[]), constr(&mut c, N0, &[])); + assert!(c.unify_pattern(2, true, true, n1, older).is_ok()); + assert_eq!(c.equations.len(), 2); + assert_eq!(c.equations[1].type_index, N1); + let (left, right) = (constr(&mut c, N1, &[]), constr(&mut c, N0, &[])); + assert!(c.unify(left, right).is_ok()); + assert_eq!(path(&mut c, right), N0); + assert_eq!(c.g.repr(left), left); +} + +/// Witness: `Wrap: t<(a, int)>` against `t` for the same `a`: an +/// equation of a type with a type that names it. `add_gadt_equation` +/// adds nothing and does not clash (`if local_non_recursive_abbrev !env +/// source destination then ...`, compiler/ml/ctype.ml:2093-2094). +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn a_recursive_equation_is_not_added() { + // Review of pull request 347: this expectation is not what OCaml + // does (compiler/ml/ctype.ml:1503-1536, called with `strict = false`: + // a tuple at the top is not searched, so this equation IS added). + // A design turn corrects the test before the slice is built + // (CHECKER_REWRITE 8.9, "Corrections owed after review", item 1). + let (mut c, _) = pattern_checker(Mode::Success); + let (inner, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let destination = tuple(&mut c, &[inner, int]); + let n0 = constr(&mut c, N0, &[]); + assert!(c.unify_pattern(2, true, true, n0, destination).is_ok()); + assert!(c.equations.is_empty()); + assert_eq!(c.types.rows[N0 as usize].flags, NEWTYPE); +} + +/// Witness: a pattern whose type has 10,000 components. Relations +/// between 1,000 and 10,000 (CHECKER_REWRITE 8.9, "Step tests"): +/// - a chain of n nested tuples against a second chain of the same +/// shape in pattern mode: the steps are linear in the pairs, no node +/// of either chain is a link, the table of assumed pairs is empty; +/// - `reify` of a tuple of n variables, each named twice: every +/// variable is visited once, steps linear, n types made; +/// - n equations added and cut: steps linear, every row what it was. +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn pattern_mode_at_ten_thousand() { + let chain = |c: &mut Checker, n: u32| { + let mut ty = constr(c, INT, &[]); + for _ in 0..n { + let int = constr(c, INT, &[]); + ty = tuple(c, &[int, ty]); + } + ty + }; + let structures = |n: u32| { + let (mut c, _) = pattern_checker(Mode::Success); + let (left, right) = (chain(&mut c, n), chain(&mut c, n)); + let nodes = c.g.nodes.len(); + let mut result = Err(super::unify::Clash); + let count = steps::scope(|| result = c.unify_pattern(2, true, true, left, right)); + assert!(result.is_ok()); + assert_pattern_left(&c); + // The nullary `int` pairs are linked in every mode + // (ctype.ml:2248-2257); no tuple is. + for index in 0..nodes { + let node = c.g.nodes[index]; + assert!(node.tag() != LINK || path(&mut c, Ty(index as u32)) == INT); + } + count + }; + let (small, large) = (structures(1_000), structures(10_000)); + assert!(large <= 11 * small, "structures: {small} {large}"); + + let reified = |n: u32| { + let (mut c, asked) = pattern_checker(Mode::Success); + let variables: Vec = (0..n).map(|_| var(&mut c)).collect(); + let mut elements = variables.clone(); + elements.extend(&variables); + let ty = tuple(&mut c, &elements); + c.newtype_level = 2; + let mut result = Err(super::unify::Clash); + let count = steps::scope(|| result = c.reify(ty)); + assert!(result.is_ok()); + assert_eq!(asked.borrow().len(), n as usize); + count + }; + let (small, large) = (reified(1_000), reified(10_000)); + assert!(large <= 11 * small, "reify: {small} {large}"); + + let equations = |n: u32| { + let (mut c, _) = pattern_checker(Mode::Success); + let first = c.types.rows.len() as u32; + for _ in 0..n { + let row = c.types.rows[T as usize]; + c.types.rows.push(row); + } + let before = c.types.rows.clone(); + let count = steps::scope(|| { + for index in 0..n { + let (ty, int) = (constr(&mut c, first + index, &[]), constr(&mut c, INT, &[])); + assert!(c.unify_pattern(2, true, true, ty, int).is_ok()); + } + assert_eq!(c.equations.len(), n as usize); + c.cut_equations(0); + }); + assert_eq!(c.types.rows, before); + count + }; + let (small, large) = (equations(1_000), equations(10_000)); + assert!(large <= 11 * small, "equations: {small} {large}"); +} + +/// Witness: `Int: t>` against `t>` below a constructor +/// whose arguments were already entered (`generate_equations` on, +/// `assume_injective` off), `k` abstract and of another unit. Its +/// argument is not injective: it is unified without equations under a +/// snapshot, and when that fails the graph goes back and both arguments +/// are reified (compiler/ml/ctype.ml:2359-2371). +/// - `k` against `k`: no equation, a success. +/// - `k<'v>` against `k`: the attempt succeeds and is kept: `'v` +/// is a link to `int`. +/// - `k<('v, int)>` against `k`: the attempt fails; `'v` is not +/// bound by it and is reified: a link to the first fresh type. +/// - with the parameter marked injective (`injective_bits`), `k` +/// against `k` gives `n0 := int`. +/// +/// No attempt is left open and, in `Mode::Success`, no speculation. +/// +/// Reaches: R3h, R3r. +#[test] +#[ignore = "R3r build"] +fn an_argument_that_is_not_injective_is_tried_and_then_reified() { + for mode in [Mode::Success, Mode::Explain] { + let (mut c, _) = pattern_checker(mode); + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let (left, right) = (constr(&mut c, K, &[n0]), constr(&mut c, K, &[int])); + assert!(c.unify_pattern(2, true, false, left, right).is_ok()); + assert_pattern_left(&c); + assert!(c.equations.is_empty()); + assert_eq!(c.g.speculation_depth, 0); + + let (v, int) = (var(&mut c), constr(&mut c, INT, &[])); + let (left, right) = (constr(&mut c, K, &[v]), constr(&mut c, K, &[int])); + assert!(c.unify_pattern(2, true, false, left, right).is_ok()); + assert_eq!(c.g.repr(v), int); + + let (v, int, string) = ( + var(&mut c), + constr(&mut c, INT, &[]), + constr(&mut c, STRING, &[]), + ); + let pair = tuple(&mut c, &[v, int]); + let (left, right) = (constr(&mut c, K, &[pair]), constr(&mut c, K, &[string])); + assert!(c.unify_pattern(2, true, false, left, right).is_ok()); + assert_pattern_left(&c); + let node = c.g.repr(v); + assert_eq!(path(&mut c, node), FRESH); + assert_eq!(c.g.speculation_depth, 0); + + c.types.injective_bits = vec![0; K as usize + 1]; + c.types.injective_bits[K as usize] = 1; + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let (left, right) = (constr(&mut c, K, &[n0]), constr(&mut c, K, &[int])); + assert!(c.unify_pattern(2, true, false, left, right).is_ok()); + assert_eq!(c.equations.len(), 1); + } +} + +/// Witness: a constructor whose result is `t<[> #A]>` against a +/// scrutinee `t<[> #A | #B]>` whose row was fixed by an earlier case. +/// Two variants in pattern mode are unified under a snapshot; when that +/// fails the graph goes back, both are reified and, with equations, +/// compared with `mcomp` (compiler/ml/ctype.ml:2400-2408). +/// - a fixed row `[> #A]` (made by `reify`) against an open row with +/// one more tag: `unify_row` refuses to extend the fixed row, `mcomp` +/// accepts (no row is closed): a success, the two rows are two nodes, +/// the open one is a fixed row afterwards (observed on the old +/// checker: decision 62); +/// - two closed rows `[#A]` and `[#B]`: `mcomp` refuses (a present tag +/// the other closed row lacks, ctype.ml:1988-1995): a clash; +/// - the same without `generate_equations`: no `mcomp`, a success. +/// +/// Reaches: R3f, R3h, R3r. +#[test] +#[ignore = "R3r build"] +fn rows_that_clash_are_reified_and_compared() { + for mode in [Mode::Success, Mode::Explain] { + let (mut c, _) = pattern_checker(mode); + let (more1, more2) = (var(&mut c), var(&mut c)); + let fixed = variant_at(&mut c, 2, &[("A", Tag::Present(None))], more1, false, NONE); + c.newtype_level = 2; + assert!(c.reify(fixed).is_ok()); + c.newtype_level = 0; + assert!(shape(&mut c, fixed).contains("fixed")); + let open = variant_at( + &mut c, + 2, + &[("A", Tag::Present(None)), ("B", Tag::Present(None))], + more2, + false, + NONE, + ); + assert!(c.unify_pattern(2, true, true, fixed, open).is_ok()); + assert_pattern_left(&c); + assert_eq!((c.g.repr(fixed), c.g.repr(open)), (fixed, open)); + assert!(shape(&mut c, open).contains("fixed")); + assert_eq!(path(&mut c, more2), FRESH + 1); + + for (generate, clash) in [(true, true), (false, false)] { + let (nil1, nil2) = ( + extended_at(&mut c, 2, ExtraDescriptor::Nil, &[]), + extended_at(&mut c, 2, ExtraDescriptor::Nil, &[]), + ); + let left = variant_at(&mut c, 2, &[("A", Tag::Present(None))], nil1, true, NONE); + let right = variant_at(&mut c, 2, &[("B", Tag::Present(None))], nil2, true, NONE); + let result = c.unify_pattern(2, generate, true, left, right); + assert_eq!(result.is_err(), clash); + assert_pattern_left(&c); + assert_eq!((c.g.repr(left), c.g.repr(right)), (left, right)); + } + } +} + +/// Witness: `Int: t` against `t`, `t`, `t>`: +/// what `mcomp` answers (compiler/ml/ctype.ml:1905-2039), one line an +/// arm. It never links: every node is its own representative +/// afterwards, and its table is empty. +/// - a variable against anything: compatible; +/// - `int` against `string`, both bare identifiers that are not locally +/// abstract (`non_aliasable`): a clash; +/// - `M.t`, abstract and of another unit, against `int`: compatible +/// (`Type_abstract, Type_abstract`); +/// - `M.t` against a tuple: compatible; `int` against a tuple: a clash +/// (`non_aliasable p decl || is_datatype decl`); +/// - a tuple against an arrow: a clash; `(int, M.t)` against `(int, +/// myint)`: compatible, through the expansion of `myint`; +/// - `d` against `d`: compatible while the parameter is +/// not injective, a clash when it is; +/// - `d` against `e`, two data types: what the supplier +/// answers: `Clash`; `Members` with no pair: compatible; `Members` +/// after pushing the pair of `int` and `string`: a clash; with no +/// supplier: a clash (CHECKER_REWRITE 8.9, "No supplier"). +/// +/// Reaches: R3r. +#[test] +#[ignore = "R3r build"] +fn mcomp_answers_by_the_declarations() { + let (mut c, _) = pattern_checker(Mode::Success); + let int = |c: &mut Checker| constr(c, INT, &[]); + let (v, i, s, t) = ( + var(&mut c), + int(&mut c), + constr(&mut c, STRING, &[]), + constr(&mut c, T, &[]), + ); + let pair = { + let (a, b) = (int(&mut c), int(&mut c)); + tuple(&mut c, &[a, b]) + }; + let function = { + let (a, b) = (int(&mut c), int(&mut c)); + arrow(&mut c, &[a], b) + }; + let (with_t, with_myint) = { + let (a, b, m) = (int(&mut c), int(&mut c), constr(&mut c, MYINT, &[])); + let t = constr(&mut c, T, &[]); + (tuple(&mut c, &[a, t]), tuple(&mut c, &[b, m])) + }; + let (d_int, d_string) = { + let (a, b) = (int(&mut c), constr(&mut c, STRING, &[])); + (constr(&mut c, D, &[a]), constr(&mut c, D, &[b])) + }; + let nodes = c.g.nodes.len(); + for (left, right, compatible) in [ + (v, pair, true), + (i, s, false), + (t, i, true), + (t, pair, true), + (i, pair, false), + (pair, function, false), + (with_t, with_myint, true), + (d_int, d_string, true), + ] { + assert_eq!( + c.mcomp(left, right).is_ok(), + compatible, + "{left:?} {right:?}" + ); + assert!(c.compatible.keys.is_empty()); + } + for node in &c.g.nodes[..nodes] { + assert_ne!(node.tag(), LINK); + } + c.types.injective_bits = vec![0; D as usize + 1]; + c.types.injective_bits[D as usize] = 1; + assert!(c.mcomp(d_int, d_string).is_err()); + + let e_int = { + let a = int(&mut c); + constr(&mut c, E, &[a]) + }; + let supplier = |answer: DeclMatch, members: Vec<[Ty; 2]>| -> Option> { + Some(Box::new(Fresh { + next: FRESH, + asked: Default::default(), + answer, + members, + })) + }; + assert!(c.mcomp(d_int, e_int).is_err()); + c.locals = supplier(DeclMatch::Members, Vec::new()); + assert!(c.mcomp(d_int, e_int).is_ok()); + c.locals = supplier(DeclMatch::Members, vec![[i, s]]); + assert!(c.mcomp(d_int, e_int).is_err()); + c.locals = None; + assert!(c.mcomp(d_int, e_int).is_err()); + assert!(c.compatible.keys.is_empty()); +} + +/// Witness: two types of 10,000 components that `mcomp` compares. +/// Relation between 1,000 and 10,000: two tuples of n elements whose +/// elements are the SAME n-deep chain pairwise (every element of the +/// left is one node, every element of the right another): the table +/// makes the second and every later pair of the chains one lookup, so +/// the steps are linear in n and not n times the depth. +/// +/// Reaches: R3r. +#[test] +#[ignore = "R3r build"] +fn mcomp_at_ten_thousand() { + let run = |n: u32| { + let (mut c, _) = pattern_checker(Mode::Success); + let chain = |c: &mut Checker| { + let mut ty = constr(c, T, &[]); + for _ in 0..n { + let int = constr(c, INT, &[]); + ty = tuple(c, &[int, ty]); + } + ty + }; + let (a, b) = (chain(&mut c), chain(&mut c)); + let (left, right) = (tuple(&mut c, &vec![a; 100]), tuple(&mut c, &vec![b; 100])); + let mut result = Err(super::unify::Clash); + let count = steps::scope(|| result = c.mcomp(left, right)); + assert!(result.is_ok()); + count + }; + let (small, large) = (run(1_000), run(10_000)); + assert!(large <= 11 * small, "mcomp: {small} {large}"); +} diff --git a/compiler-rust/src/tc/rows.rs b/compiler-rust/src/tc/rows.rs index 86a0fa5628..5fd8292ef5 100644 --- a/compiler-rust/src/tc/rows.rs +++ b/compiler-rust/src/tc/rows.rs @@ -310,6 +310,60 @@ pub trait PackageDecls { fn compatible(&mut self, c: &mut Checker, left: Node, right: Node) -> bool; } +/// One equation of pattern mode in force (`Env.add_local_constraint`, +/// compiler/ml/ctype.ml:2093-2100): the type that got a manifest and the +/// `TypeInfo` row it had before, so that cutting the log back puts the +/// row back ([`Checker::cut_equations`]). 20 bytes; one per equation +/// added and not yet cut. (R3h; CHECKER_REWRITE 8.9) +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +#[repr(C)] +pub struct Equation { + pub type_index: u32, + pub previous: super::expand::TypeInfo, +} + +/// What two declarations answer when `mcomp` compares them +/// (`mcomp_type_decl`, compiler/ml/ctype.ml:2011-2039). +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum DeclMatch { + /// The kinds, the representations, the names, flags and number of the + /// members agree; the pairs of member types were pushed + /// ([`Checker::push_compatible`]). + Members, + /// They cannot be one type (`raise (Unify [])`). + Clash, +} + +/// What pattern mode asks of the declarations, which the kernel does not +/// have (CHECKER_REWRITE 8.9). R5a supplies it; the tests of R3h supply +/// one built by hand. Held in [`Checker::locals`]; without one, `reify` +/// of a type that has a variable stops the compiler (a caller that types +/// a pattern installs one first) and two different data types clash. +pub trait LocalDecls { + /// A new locally abstract type for `reify` (`create_fresh_constr`, + /// compiler/ml/ctype.ml:1815-1826): the supplier makes the identifier + /// (`get_new_abstract_name` of `"$'" ^ name`, or `"$"` when `name` is + /// `NONE`; `name` is the variable's name index) and whatever its own + /// tables hold for a declaration, and returns the type index. The + /// kernel writes the [`TypeTable`](super::expand::TypeTable) row + /// (abstract, [`NEWTYPE`](super::expand::NEWTYPE), both newtype levels + /// the current one). O(1) amortized. + fn fresh_abstract(&mut self, name: u32) -> u32; + /// `mcomp_type_decl` for two DIFFERENT types that are both data types + /// (a record, a variant or an open type) and not both names of the + /// unit being checked (ctype.ml:2026-2037): `Clash` unless both are + /// records of one representation, both variants, or both open; for + /// records and variants every pair of member types is pushed with + /// [`Checker::push_compatible`], in declaration order, the result + /// before the arguments, and a difference in names, mutability, + /// optionality, argument shape or number of members is a `Clash` + /// (`mcomp_variant_description`, `mcomp_record_description`, + /// ctype.ml:2047-2078). The member types are nodes of `c`'s graph: + /// the declarations' own types, not instances for the two argument + /// lists, as in OCaml. O(members). + fn compare(&mut self, c: &mut Checker, left: u32, right: u32) -> DeclMatch; +} + /// O(1): the `a` of a [`RES_EITHER`] node with `c` and `m`. #[inline] pub fn either_flags(constant: bool, matched: bool) -> u32 { @@ -1446,6 +1500,18 @@ impl Checker { return Err(clash); } }; + // A row that is not fixed has a variant node as its `more` now + // (`link_type rm ty`, ctype.ml:2616), so its name's arguments are + // children no longer (`iter_row`, btype.ml:229-241). + let homes = [ + (fixed1 || last1.0 == NONE, row1, last1), + (fixed2, row2, if last2.0 == NONE { t2x } else { last2 }), + ]; + for (kept, row, home) in homes { + if !kept && row.name != NONE && row.name_arguments.len != 0 { + self.drop_name_children(home, row); + } + } // 9. The static row below the common tags, which are popped in the // order of the pairs, then the pair a fixed row asked for. if closed { @@ -1466,6 +1532,62 @@ impl Checker { Ok(()) } + /// The arguments of a row's name stop being children of variant node + /// `home` (a representative, whose `VariantRow` is `row`) once the + /// row's `more` is a variant node: OCaml's `iter_row` visits them only + /// for the row a chain of rows ends in (compiler/ml/btype.ml:229-241): + /// + /// ```text + /// match (repr row.row_more).desc with + /// | Tvariant row -> iter_row f row + /// | Tvar _ | Tunivar _ | Tsubst _ | Tconstr _ | Tnil -> + /// Misc.may (fun (_, l) -> List.iter f l) row.row_name + /// ``` + /// + /// and every walk of a type goes through it (`iter_type_expr`, + /// btype.ml:263-265): `update_level`, `generalize`, `deep_occur`, + /// `occur_univar`, `univars_escape`; a copy takes the name of the + /// innermost row (`row_repr`, btype.ml:126-135). The kernel's walks + /// push every slot of a node, so the node is written + /// (`Graph::write`, logged in a speculation) with a new list in which + /// each slot of the name's arguments holds what the `more` slot + /// holds: a child the walk reaches anyway. The shared `VariantRow` is + /// not written, and nothing reads the name of a row that is not the + /// innermost (`row_repr`). In `Mode::Explain` a + /// [`FrameKind::Fields`] frame puts the node's descriptor back when + /// the unification fails, as it puts back the two rests, whose links + /// are what made the name stale (`rm1.desc <- md1`, ctype.ml: + /// 2637-2641). Called by [`Checker::unify_row`] for a row that has a + /// name with arguments and is not fixed: O(slots of the node), one + /// list of that length; a row without a name with arguments costs the + /// two compares of the caller. (CHECKER_REWRITE 8.6, decision 60) + #[cold] + #[inline(never)] + fn drop_name_children(&mut self, home: Ty, row: VariantRow) { + let node = self.g.nodes[home.0 as usize]; + if self.g.mode == Mode::Explain { + let extra = self.saved.len() as u32; + self.saved.push(node); + self.saved.push(node); + self.frames.push(Frame { + left: home, + right: home, + kind: FrameKind::Fields, + extra, + }); + } + let b = self.g.lists.len() as u32; + self.g + .lists + .extend_from_within(node.b as usize..(node.b + node.arity()) as usize); + let more = self.g.lists[(b + row.more) as usize]; + let arguments = row.name_arguments; + self.g.lists + [(b + arguments.start) as usize..(b + arguments.start + arguments.len) as usize] + .fill(more); + self.g.write(home, Node { b, ..node }); + } + /// O(chain): the field `extras.row_scratch[at]` is, read now /// ([`field_repr`]; its home is opened, so every `Reither` field of it /// has its reference slot). @@ -2080,13 +2202,35 @@ impl Checker { } } }; - missing(self, middle..end, base..middle); - let gained1 = end..self.extras.row_scratch.len(); - missing(self, base..middle, middle..end); - let gained2 = gained1.end..self.extras.row_scratch.len(); + // The second run, after the types that hold a universal variable + // were unified: `tl1'` and `tl2'` are the lists the first run kept + // (`let tl1', tlu1 = split_univars tl1'`, computed BEFORE `List.iter + // (unify env tu1) (tlu1 @ tlu2)`, ctype.ml:2685-2697), read back + // from `saved` after the record ([`Checker::split_univars`]), not + // the two lists compared again: a type those unifications made + // the same node as one the side already has is still gained + // (CHECKER_REWRITE 8.6, decision 58). + let (gained1, gained2) = if record.b & ROW_UNIVARS != 0 { + let counts = self.saved[frame.extra as usize + 1]; + let first = frame.extra as usize + 2; + for at in first..first + (counts.head + counts.level) as usize { + let ty = Ty(self.saved[at].head); + self.extras + .row_scratch + .push(FieldRef { name: 0, at: 0, ty }); + } + ( + end..end + counts.head as usize, + end + counts.head as usize..self.extras.row_scratch.len(), + ) + } else { + missing(self, middle..end, base..middle); + let gained1 = end..self.extras.row_scratch.len(); + missing(self, base..middle, middle..end); + (gained1.clone(), gained1.end..self.extras.row_scratch.len()) + }; // PR#6744 (ctype.ml:2685-2699), on one load when the graph has no - // universal variable; not on the second run, after the types that - // hold one were unified. + // universal variable; not on the second run. let (gained1, gained2) = if self.extras.univars != 0 && record.b & ROW_UNIVARS == 0 { match self.split_univars(frame, [gained1, gained2]) { Ok(Some([gained1, gained2])) => (gained1, gained2), @@ -2808,7 +2952,9 @@ impl Checker { /// root context is never visited again (ctype.ml:1569-1571); one /// visited in a context that has closed is visited again; /// - otherwise the node is marked with `current`, one step, and: - /// - a polymorphic type (ctype.ml:1587-1589): a new generation `g` + /// - a polymorphic type without bound variables: its body, in the + /// current context (`bound` is unchanged, ctype.ml:1587-1589); + /// - a polymorphic type with bound variables: a new generation `g` /// and `open_polys.push(true)`; each bound variable's /// representative whose mark is not live is marked `g` (one that /// an enclosing open type binds keeps that mark, so that leaving @@ -2915,6 +3061,15 @@ impl Checker { TUPLE if node.head & EXTENDED != 0 => match self.extras.descriptors[node.a as usize] { // `Tpoly (ty, tyl)`: the body with the bound variables // added to `bound` (compiler/ml/ctype.ml:1587-1589). + // `Tpoly (ty, [])` adds nothing to `bound`: no context, the + // body is a child like any other. A context for it would + // make every node first met below it be visited again + // after it is left, which doubles the walk at each such + // type that shares its body with the outside + // (CHECKER_REWRITE 8.6, decision 59). + ExtraDescriptor::Poly if arity == 1 => { + self.g.work.push(self.g.lists[node.b as usize]); + } ExtraDescriptor::Poly => { let generation = self.fresh_marks(); self.extras.open_polys.push(true); @@ -3002,9 +3157,12 @@ impl Checker { /// [`Checker::row_field_end`], which popped it) one is pushed; then /// the pairs of the first type of `tlu1` with each other type of /// `tlu1` and each type of `tlu2`, pushed so that they are popped - /// in that order. `Ok(None)`: the join is run again when they are - /// unified, and then every such type is on both sides and gained - /// by neither. + /// in that order; the types each range KEEPS are written to + /// `saved` after the record (an entry with the two counts, then one + /// entry a type). `Ok(None)`: the join is run again when the + /// holders are unified, and gains exactly those kept types, as + /// OCaml's `tl1'` and `tl2'` are computed before `unify env tu1` + /// (ctype.ml:2685-2706; CHECKER_REWRITE 8.6, decision 58). /// - One empty (`occur_univar !env tu`): [`Checker::occur_univar`] of /// the first type of the other, which fails and pushes its pair in /// `Mode::Explain`: `Err([NONE, NONE])`. @@ -3028,6 +3186,29 @@ impl Checker { // tlu2)`. (false, false) => { self.saved[frame.extra as usize].b |= ROW_UNIVARS; + // What the two fields gain is decided now, before the + // holders are unified (ctype.ml:2685-2706): the kept types + // go after the record, first an entry with the two + // counts (`head`, `level`), then one entry a type + // (`head`). Only this arm writes them: O(types kept), + // 16 bytes each, until the pair's `RowFieldEnd` frame is + // popped. + debug_assert_eq!(self.saved.len(), frame.extra as usize + 1); + self.saved.push(Node { + head: kept1.len() as u32, + level: kept2.len() as u32, + a: NONE, + b: 0, + }); + for at in kept1.clone().chain(kept2.clone()) { + let ty = self.extras.row_scratch[at].ty; + self.saved.push(Node { + head: ty.0, + level: 0, + a: NONE, + b: 0, + }); + } let end = Frame { kind: FrameKind::RowFieldEnd, ..frame }; if self.frames.last() != Some(&end) { self.frames.push(end); @@ -3229,27 +3410,72 @@ impl Checker { if compatible { Ok(()) } else { Err(Clash) } } - /// `reify` (compiler/ml/ctype.ml:1813-1870): replace the variables of a - /// type by fresh abstract types at the newtype level, in pattern mode. - /// O(nodes), marks. (R3h) - pub fn reify(&mut self, ty: Ty) { + /// `reify` (compiler/ml/ctype.ml:1813-1859): every variable `ty` + /// reaches becomes a new locally abstract type of the newtype level, + /// and the row variable of every variant that is neither static nor + /// fixed a fixed row without tags over one; the steps are in + /// CHECKER_REWRITE 8.9 ("`reify`"). Visits each node once (a new mark + /// generation, `g.work`), children in `iter_type_expr`'s order. A + /// variable below [`Checker::newtype_level`] is linked and then a + /// clash (ctype.ml:1837-1839). Needs [`Checker::locals`] when there + /// is a variable. O(nodes reached). (R3h) + pub fn reify(&mut self, ty: Ty) -> Result<(), Clash> { todo!("R3h") } - /// `add_gadt_equation` (compiler/ml/ctype.ml:2093-2100): give the - /// locally abstract type `source` the manifest `destination` when the - /// equation is well-founded, and clean the memos. O(nodes of - /// `destination`). (R3h) + /// `add_gadt_equation` (compiler/ml/ctype.ml:2093-2100): when + /// [`Checker::non_recursive_abbrev`], the locally abstract type + /// `source` gets [`Checker::duplicate`] of `destination` as its + /// manifest, the entry goes to [`Checker::equations`] and the memos + /// are cleaned; the five steps are in CHECKER_REWRITE 8.9, decision + /// 67. Otherwise nothing. O(nodes of `destination`) plus the + /// cleanup. (R3h) pub fn add_gadt_equation(&mut self, source: u32, destination: Ty) { todo!("R3h") } - /// `mcomp` (compiler/ml/ctype.ml:1905-2090): whether two types can have - /// a common instance; never links. O(pairs), pair set, explicit stack. - /// (R3h) - pub fn mcomp(&mut self, left: Ty, right: Ty) -> Result<(), Clash> { + /// `local_non_recursive_abbrev env p ty` (compiler/ml/ctype.ml: + /// 1478-1535): `ty` does not name the type `source`, through its + /// children or through the expansions of the constructors it names + /// ([`Checker::expand_once`] in `ExpandMode::Opt`). Marks and + /// `g.work`; O(nodes reached). (R3h) + pub fn non_recursive_abbrev(&mut self, source: u32, ty: Ty) -> bool { + todo!("R3h") + } + + /// `duplicate_type` for an equation: a copy of `ty` in which every + /// node is generic, sharing kept; `ty` is as it was afterwards. Made + /// of `g.speculate()`, `generalize(ty)` with `g.level` 0, + /// `copy(ty)` with `g.level` generic, and `g.restore`: the copy's + /// nodes are new and not on the trail (CHECKER_REWRITE 8.9, decision + /// 67). Three passes over `ty`. (R3h) + pub fn duplicate(&mut self, ty: Ty) -> Ty { + todo!("R3h") + } + + /// `find_newtype_level env path` (compiler/ml/ctype.ml:2085-2091): + /// the `Newtype` row's `(level, expansion_level)` of a type with + /// `NEWTYPE`, otherwise `(scope, scope)` (`Path.binding_time`). + /// Compared as pairs. A type without a row: `(0, 0)`. O(1). (R3h) + pub fn find_newtype_level(&self, type_index: u32) -> (u32, u32) { todo!("R3h") } + + /// `mcomp` (compiler/ml/ctype.ml:1905-2080): whether two types can + /// have a common instance; never links, never lowers. The arms are + /// in CHECKER_REWRITE 8.9, decision 70. Its table is + /// [`Checker::compatible`], emptied when it returns; its stack is + /// `g.work`. O(pairs). (R3r) + pub fn mcomp(&mut self, left: Ty, right: Ty) -> Result<(), Clash> { + todo!("R3r") + } + + /// One more pair for the `mcomp` in progress: what + /// [`LocalDecls::compare`] calls for each pair of member types. + /// O(1) amortized: two entries of `g.work`. (R3r) + pub fn push_compatible(&mut self, left: Ty, right: Ty) { + todo!("R3r") + } } #[cfg(test)] diff --git a/compiler-rust/src/tc/unify.rs b/compiler-rust/src/tc/unify.rs index 3991f42603..481ea24475 100644 --- a/compiler-rust/src/tc/unify.rs +++ b/compiler-rust/src/tc/unify.rs @@ -107,6 +107,39 @@ pub enum FrameKind { /// [`Checker::saved`]: the descriptor the left node had. Popped: /// [`Checker::package_end`]. (R3p) PackageEnd, + /// Pattern mode, below the pairs of the arguments of two + /// constructors of one type: `extra` holds the two flags as they + /// were (bit 0 `generate_equations`, bit 1 `assume_injective`). + /// Popped: the flags are put back (`set_mode_pattern`, + /// compiler/ml/ctype.ml:256-273). (R3h; CHECKER_REWRITE 8.9, + /// decision 64) + Flags, + /// Pattern mode, below what an arm tries: `left` and `right` are the + /// two types to reify when it fails, `extra` indexes + /// [`Checker::tries`]. Popped: the attempt succeeded + /// ([`Checker::recover`] says what a clash above it does). (R3r; + /// CHECKER_REWRITE 8.9, decision 68) + Try, +} + +/// One attempt of pattern mode that falls back on a clash (`let snap = +/// snapshot () in try ... with Unify _ -> backtrack snap; reify ...`, +/// compiler/ml/ctype.ml:2362-2371, 2400-2408). 40 bytes; one per open +/// attempt. (R3r; CHECKER_REWRITE 8.9, decision 68) +#[derive(Debug)] +pub struct Try { + /// What the graph is restored to. + pub speculation: super::graph::Speculation, + /// Index of the [`FrameKind::Try`] frame in `frames`. + pub frame: u32, + /// The lengths of `trace`, `saved` and `scopes` when it was opened. + pub trace: u32, + pub saved: u32, + pub scopes: u32, + /// Bit 0 `generate_equations`, bit 1 `assume_injective`, as they + /// were; bit 2: `mcomp` follows the reification when + /// `generate_equations` is on after the flags are put back. + pub flags: u8, } /// Bit 31 of a [`FrameKind::Linked`] frame's `extra`: `unify2` ran `unify3` @@ -172,6 +205,24 @@ pub struct Checker { /// What [`Checker::unify_package`] asks about module types; `None` /// until a caller that has declarations sets it (R5a). (R3p) pub packages: Option>, + /// `current_newtype_level` (compiler/ml/ctype.ml:1802-1808): the level + /// of the pattern being typed, or 0 outside + /// [`Checker::unify_pattern`] (`None`; no node is at level 0). (R3h) + pub newtype_level: u32, + /// The equations in force, oldest first (the local constraints of + /// `Env`, compiler/ml/ctype.ml:2093-2100). Empty for a unit that + /// never typed a pattern with an equation; `equations.len()` is what + /// a caller keeps to cut back to ([`Checker::cut_equations`]). (R3h) + pub equations: Vec, + /// What `reify` and `mcomp` ask about declarations; `None` until a + /// caller that has declarations sets it (R5a). (R3h) + pub locals: Option>, + /// The open attempts of pattern mode, innermost last; empty outside + /// a pattern unification. (R3r) + pub tries: Vec, + /// The pairs `mcomp` assumes compatible, ordered (left, right) in + /// one `u64` each; empty outside a call of it. (R3r) + pub compatible: PairSet, } impl Checker { @@ -196,6 +247,11 @@ impl Checker { pairs: PairSet::default(), scopes: Vec::new(), packages: None, + newtype_level: 0, + equations: Vec::new(), + locals: None, + tries: Vec::new(), + compatible: PairSet::default(), } } @@ -945,7 +1001,8 @@ impl Checker { return self.unify_fields(t1x, t2x); } if self.umode == UnifyMode::Pattern { - // `add_type_equality` in place of the link (ctype.ml:2329). + // `add_type_equality` in place of the link (ctype.ml:2329): + // CHECKER_REWRITE 8.9, decision 64. todo!("R3h") } // Two variant nodes get the slots of their references while both @@ -1224,12 +1281,69 @@ impl Checker { } /// `unify_gadt` (compiler/ml/ctype.ml:2739-2750): one unification in - /// pattern mode with equations, at `newtype_level`; clears the pair set - /// before and after. (R3h) + /// pattern mode with equations and injectivity assumed, at + /// `newtype_level`: [`Checker::unify_pattern`]. (R3h) pub fn unify_gadt(&mut self, newtype_level: u32, left: Ty, right: Ty) -> Result<(), Clash> { + self.unify_pattern(newtype_level, true, true, left, right) + } + + /// One top-level unification in pattern mode (`unify_gadt`, + /// compiler/ml/ctype.ml:2739-2750, with the two flags of + /// `set_mode_pattern`, ctype.ml:256-273, as arguments so that a test + /// can ask for each mode `unify3` installs, ctype.ml:2352, 2368): + /// 1. `scopes` must be empty (`current_univar_pairs () := []`); the + /// previous `umode`, `generate_equations`, `assume_injective` are + /// kept; `umode = Pattern`, the two flags as given, `newtype_level` + /// as given; + /// 2. [`Checker::unify`] of the pair; + /// 3. on either result: the three settings go back, `newtype_level` + /// is 0, and the table of assumed equalities is emptied + /// (`Type_pairs.clear (unify_eq_set ())`; `PairSet::clear`, + /// O(pairs assumed)). + /// + /// The equations it added stay in [`Checker::equations`]: the caller + /// cuts them back when the case they belong to ends or when it + /// restores a speculation ([`Checker::cut_equations`]). O(1) plus the + /// unification. (R3h; CHECKER_REWRITE 8.9) + pub fn unify_pattern( + &mut self, + newtype_level: u32, + generate: bool, + injective: bool, + left: Ty, + right: Ty, + ) -> Result<(), Clash> { + todo!("R3h") + } + + /// Cut the equations back to the first `len` (a length of + /// [`Checker::equations`] the caller kept), last first: the + /// `TypeTable` row of each popped entry is its `previous` again; + /// then [`Checker::cleanup_abbrev`] when one was popped. What a + /// caller does where OCaml drops the environment of a case + /// (`set_state`, compiler/ml/typecore.ml:1303-1305). O(entries + /// popped) plus the cleanup. (R3h; CHECKER_REWRITE 8.9, decision + /// 67) + pub fn cut_equations(&mut self, len: usize) { todo!("R3h") } + /// A clash reached `run_frames` whose frames start at `base`: `None` + /// when no [`Try`] record has its frame at or above `base` (the + /// clash is the caller's). Otherwise the last record is taken: + /// `frames` is cut to its frame, [`Checker::restore`] of its + /// speculation, `leave_scopes` to its scopes, `trace` and `saved` + /// cut to its lengths, the two flags put back; then `reify(left)`, + /// `reify(right)` and, with bit 2 of the record and + /// `generate_equations` on, `mcomp(left, right)`: `Some` of their + /// result. The equations and the fresh types made inside the attempt + /// stay (`env` is not part of OCaml's snapshot). O(frames above the + /// record + writes restored) plus the three callees. (R3r; + /// CHECKER_REWRITE 8.9, decision 68) + pub fn recover(&mut self, base: usize) -> Option> { + todo!("R3r") + } + /// The failure path of the explaining mode: the trace, outermost pair /// first (each enclosing `unify` adds its pair, ctype.ml:2272), and the /// descriptors of the linked structure nodes enclosing the failure put @@ -1450,6 +1564,13 @@ impl Checker { FrameKind::RowField => self.unify_row_field(frame)?, FrameKind::RowFieldEnd => self.row_field_end(frame)?, FrameKind::RowEnd => self.row_end(frame), + // The two flags go back (8.9, decision 64). + FrameKind::Flags => todo!("R3h"), + // The attempt succeeded: `g.keep` of the record's + // speculation, its flags put back, the record + // popped (8.9, decision 68). The clashes of this + // loop go through `Checker::recover` first. + FrameKind::Try => todo!("R3r"), FrameKind::PackageEnd => { // A clash here is the enclosing pair's (`Unify []`). if self.package_end(frame).is_err() {