From a24b06f92ac34cce897f837ad3d2c2ad56035978 Mon Sep 17 00:00:00 2001 From: "open-session-submodule[bot]" <341020755+open-session-submodule[bot]@users.noreply.github.com> Date: Tue, 17 Nov 2026 22:00:51 +0000 Subject: [PATCH 1/8] Checker rewrite R3: the record of the reviews of R3g and R3p (decisions 53 to 57) Section 8.6 gets what the reviews of the fifth skeleton turn and of the two slices built from it put on record: the join that split_univars puts off (53), the two gained ranges lowered each on its own (54), the descriptor package_end reads (55), the contexts of free_univar (56) and the bench record of R3g with the rule that no prompt prescribes a remedy for a figure that moves (57). Section 8.1: R3g and R3p merged, and the instance_poly slice as a row of its own (R3q). Co-authored-by: Jaap Frolich Session: https://os.submodule.nl/session/bks-9da31a3c-0d3b-7546-bc8e-f1fd3a9270ee Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com> --- .../checker-rewrite-r3-skeleton-6.internal.md | 1 + compiler-rust/docs/CHECKER_REWRITE.md | 65 +++++++++++++++++++ 2 files changed, 66 insertions(+) create mode 100644 changes/checker-rewrite-r3-skeleton-6.internal.md diff --git a/changes/checker-rewrite-r3-skeleton-6.internal.md b/changes/checker-rewrite-r3-skeleton-6.internal.md new file mode 100644 index 0000000000..6b5aca1bf8 --- /dev/null +++ b/changes/checker-rewrite-r3-skeleton-6.internal.md @@ -0,0 +1 @@ +- Checker rewrite R3, sixth design turn: the record of the reviews of universal variables and packages, three corrections of built code with their tests (what two variant fields gain beside types that hold a universal variable, the walk for free universal variables below a polymorphic type without bound variables, the arguments of a row name that is no longer the innermost row's), and the design of pattern mode, equations and `instance_poly`. diff --git a/compiler-rust/docs/CHECKER_REWRITE.md b/compiler-rust/docs/CHECKER_REWRITE.md index 85deda8e05..56cea6047e 100644 --- a/compiler-rust/docs/CHECKER_REWRITE.md +++ b/compiler-rust/docs/CHECKER_REWRITE.md @@ -2179,6 +2179,7 @@ it stops and reports a reproduction. | R3g | Universal variables and polymorphic types with variables: the scopes, `unify_univar`, `enter_poly`, the escape test, `occur_univar`, PR#6744 in `unify_row_field` (8.6, decisions 49 to 51; 8.8) | `rows.rs` | R3e, R3f, R3n | R3p | 600 | | R3p | Packages: the merge of two name lists, completed through `PackageDecls` (8.6, decision 52; 8.8) | `rows.rs` | R3e | R3g | 150 | | R3h | Pattern mode, equations, `reify`, `mcomp` | `rows.rs`, `unify.rs` | R3n, R3g | R3i, R3j | 600 | +| R3q | `instance_poly` and `instance_label`: the copy that goes through free universal variables without sharing (`copy_sep`), and the fixed-row rule of `copy_row` only it reaches (8.6, decision 47; 8.8, open question 1) | `rows.rs`, `expand.rs` | R3n, R3g | R3h, R3i, R3j | 300 | | R3i | `moregeneral`, `equal`, `matches` | `relate.rs` | R3n, R3g | R3h, R3j | 600 | | R3j | Subtyping without the declaration-directed coercions | `relate.rs` | R3n, R3g | R3h, R3i | 600 | | K2 | The hot paths of unification, no behaviour change (8.7) | `unify.rs` | R3c | every slice above | 150 | @@ -2253,6 +2254,14 @@ Packages are R3p and not part of R3g: no generated case can judge them arm that is already wired, and they ask the declarations three questions that R5a answers. R3h was not started in this turn. +State after the sixth skeleton turn (trunk `61f7f41292`). Merged: every +slice above R3h in the table, R3g and R3p included (built as designed; +what their reviews put on record is decisions 53 to 57 of 8.6), and K1 to +K4. `cargo test --lib tc::` on that trunk: 265 passed, 0 ignored. Without a design +before this turn: R3h, the `instance_poly` slice (R3q: a slice name has +three characters), R3i and R3j. What the sixth turn did is listed in 8.6 +("Decided in the sixth skeleton turn"). + The ignored tests are checked mechanically (`differential::tests::ignored_tests_name_the_slices_they_reach`): each names the slices it reaches, reaches its own, and reaches none that is @@ -4286,6 +4295,62 @@ unless it says "ignored". `Completion::Missing` are separate answers so that R3j can skip them without a second trait. +On record from the reviews of the fifth skeleton turn, of R3g and of R3p +(trunk `61f7f41292`). They bind the sixth skeleton turn and R5c. + +53. **`split_univars` puts the join off and runs it again.** OCaml + unifies the gained types that hold a universal variable and then + builds the two fields from the lists as they were before those + unifications (`List.iter (unify env tu1) (tlu1 @ tlu2)`, then `tl1'` + and `tl2'` as already computed, ctype.ml:2693-2706). The kernel + returns `Ok(None)`, lets the frame stack unify the holders, and runs + `join_either` from its start: the two type lists are read again and + compared again. The field holds the same SET of nodes either way. The + two differ in one corner: a gained type WITHOUT a universal variable + that those unifications make the same node as a type the gaining side + already has is not gained by the second run, and is kept, as a + duplicate entry of the list, by OCaml. Accepted as built. It is + reopened by a case in which a printed type or a later join shows the + duplicate; the sixth turn built the corner by hand on the old side + (decision 58). +54. **`join_either` lowers the two gained ranges each on its own.** With + `passive_variants` set and holders on both sides, `split_univars` + returns two ranges with a gap between them (what its compaction left + behind: holders, which OCaml does not lower, ctype.ml:2699-2703). The + caller lowered everything from the start of the first range to the end + of the second; corrected in the review of R3g. Nothing in `tc` sets + `passive_variants`, so no test reached the correction until the sixth + turn wrote one by setting the field (decision 59). +55. **`package_end` reads the right node's descriptor without following + a link.** For the right node to be linked away between the two frames + it would have to be nested in an argument of one of the two packages, + and every such shape ends in an occurrence clash before the + `PackageEnd` frame is popped. Whether pattern mode changes that is + answered by the design of R3h (decision 63: it does not, and why). +56. **A mark of `free_univar` is live while its context is open, and a + polymorphic type WITHOUT bound variables opens a context.** A node + first visited inside such a type is visited again after the type is + left, where OCaml's `visited` map, which compares the sets of bound + variables, does not visit it again (`bound' ⊆ bound` holds for two + equal sets, ctype.ml:1572-1581). The answer is the same; the visits + are more. The count is stated and asserted by the sixth turn's step + tests (decision 59). +57. **The bench record of R3g, for R5c.** The five unification operations + of `tc_kernel` at the builder's start and at the merged head: + `unify_vars` 118 and 122, `unify_ground1` 120 and 120, `unify_ground3` + 807 and 807, `unify_datatype2` 815 and 815, `unify_abbrev2` 2885.06 + and 2886.06. Tried and reverted, each measured: `#[inline(never)]` on + the five bodies (no figure moved) and `#[cold]` on the seven rare + functions (every operation dearer than the merged head). What moved + the figures between the two heads was a merge of trunk that changed + none of these bodies: code placement under one codegen unit. For + R3p, `#[cold] #[inline(never)]` on its two bodies put all five + figures back at their base, and is kept. Consequence, binding every + later prompt: no slice has a target for a `tc_kernel` figure + (decision 46), and **no prompt prescribes a remedy for a figure that + moves**; a build prompt asks for the five operations before and + after, as a report. + ### 8.7 The hot paths of unification (K2) From c8014373923fde6a170e478fc806b85fa0b13a76 Mon Sep 17 00:00:00 2001 From: "open-session-submodule[bot]" <341020755+open-session-submodule[bot]@users.noreply.github.com> Date: Tue, 17 Nov 2026 22:11:31 +0000 Subject: [PATCH 2/8] Checker rewrite R3: three corrections of built code, with the tests the reviews left owed (decisions 58 to 60) What two Reither fields gain is decided before the types that hold a universal variable are unified (compiler/ml/ctype.ml:2685-2706): the kept types are written after the pair's record and the second run of join_either reads them back. OCaml keeps a type those unifications made a duplicate; the kernel dropped it. free_univar opens no context for a polymorphic type without bound variables (ctype.ml:1587-1589). With one, a type shared between the inside and the outside of such a type was visited again at every level: 2^(n+1) - 1 visits on a chain of n, where OCaml and now the kernel make 2n + 1. The arguments of a row's name are children only of the innermost row (iter_row, compiler/ml/btype.ml:229-241): unify_row rewrites the home node of a named row that gained a variant tail. Tests, all running: the old checker's side of the first and the third, one hand-built case per walk, a row walked by free_univar, the split with passive variants, and the step relations of free_univar and univars_escape at 1,000 and 10,000. Co-authored-by: Jaap Frolich Session: https://os.submodule.nl/session/bks-9da31a3c-0d3b-7546-bc8e-f1fd3a9270ee Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com> --- compiler-rust/docs/CHECKER_REWRITE.md | 139 ++++++++ compiler-rust/src/tc/differential.rs | 135 ++++++++ compiler-rust/src/tc/r3_tests.rs | 442 ++++++++++++++++++++++++++ compiler-rust/src/tc/rows.rs | 128 +++++++- 4 files changed, 834 insertions(+), 10 deletions(-) diff --git a/compiler-rust/docs/CHECKER_REWRITE.md b/compiler-rust/docs/CHECKER_REWRITE.md index 56cea6047e..6fefdac626 100644 --- a/compiler-rust/docs/CHECKER_REWRITE.md +++ b/compiler-rust/docs/CHECKER_REWRITE.md @@ -4351,6 +4351,145 @@ On record from the reviews of the fifth skeleton turn, of R3g and of R3p moves**; a build prompt asks for the five operations before and after, as a report. +Decided in the sixth skeleton turn (trunk `61f7f41292`), each with what +was run. Decisions 58 to 60 are corrections and tests of BUILT code that +the reviews above left owed; every test named runs today. + +58. **`split_univars` keeps what OCaml keeps: decision 53 is reopened + and closed.** The corner was built by hand on the old checker + (`differential::tests::old_checker_gains_a_type_the_holders_made_a_ + duplicate`): `poly([< #A((u, x)) & x]; u)` against `poly([< #A((u', + x')) & x']; u')`. The two tuples hold a universal variable and are + unified, which makes `x'` and `x` one node; each field then holds + THREE types, the last two one node (`#A?^0((!, ') @4 @4)`). The OCaml + compiler was run on a source program that reaches the corner: + + ```rescript + let f = ( + o: {"m": 'u. [< #A(('u, 'a)) & ('a)] => unit}, + p: {"m": 'u. [< #A(('u, 'b)) & ('b)] => unit}, + ) => (if true { o } else { p }, o, p) + ``` + + With a counter added to `unify_row_field` (a local build, not + committed) the join runs once with two types a side, the split keeps + one and holds one a side, and after `set_row_field` both fields read + three types. So OCaml has the duplicate; the kernel, which compared + the two lists again after the holders were unified, had two types. + Can it show? A printed type lists every type of a `Reither` field + (`tree_of_row_field`, printtyp.ml:730-737), and a later join reads + the length of the list: the arm of PR#7496 needs `List.length tl1 = + List.length tl2` (ctype.ml:2652-2655) and `rigid_variants` reads + `List.length tl1 = 1` (ctype.ml:2667-2668), so a field of three + types and a field of two can take different arms against the same + third field. That is enough to correct it. (The diagnostic of the + program above, with `let g: int = f` appended, prints two types for + the tag in OCaml although the copies it is made from hold three; what + drops the third between the copy and the printed text was not + established, and is not needed for the decision.) + The correction, in `rows.rs`: the arm of `split_univars` with holders + on both sides writes the types each side KEEPS after the pair's + record in `saved` (one entry with the two counts, one entry a type; + 16 bytes each, only in this arm), and the second run of + `join_either` reads them back instead of comparing the two lists + again. Test: `r3_tests::a_field_gains_what_it_lacked_before_the_ + holders_were_unified` (both modes; three types, the last two one + node; it fails without the edit, observed: two types). +59. **`free_univar` opens no context for a polymorphic type without + bound variables; the step tests of the two walks; the passive + split.** + - *A fault of built code, found by the step test decision 56 asked + for.* Every object field is a `Tpoly (t, [])`. With a context per + such type, a node first met below one was visited again after it + was left, and the second visit met the types below it in closed + contexts again: on `t(i + 1) = (poly(t(i);), t(i))` the walk took + `2^(n + 1) - 1` visits (observed without the edit: 2,047, 4,095 + and 2,097,151 for n = 10, 11, 20). OCaml visits each node once: + such a type adds nothing to `bound` (ctype.ml:1587-1589), and a + node visited with a `bound` is not visited again with the same one + (ctype.ml:1569-1581). A type of that shape is what a value used + twice gives (`(x, {"m": x})`), and the walk runs at every binding + of a variable to a type with children once the unit has ANY + universal variable. Corrected: the `Poly` arm of + `push_free_univar_children` pushes the body of a type of arity 1 + in the current context. The extra visits are now ZERO, stated and + asserted: exactly `2n + 1` visits at n = 1,000 and n = 10,000 + (`r3_tests::the_walks_for_universal_variables_at_ten_thousand`, + which also has the linear relations of `free_univar` over a wide + type and of `univars_escape` with a known binder). + - *Left as it is, and why.* A polymorphic type WITH bound variables + still opens a context, and a node shared between the inside of two + sibling types with binders, or between the inside and the outside + of one, is visited once per context it is met in. OCaml visits + such a node again too, once per strictly smaller `bound` + (`Type_set.inter bound bound'`, ctype.ml:1574-1576: "that's way + too expensive", ctype.ml:1562). On a chain of n nested types WITH + binders that share their bodies with the outside the kernel's + visits double per type where OCaml's grow with the number of + distinct `bound` sets. No program has shown it: a polymorphic + method whose type shares a node with a type outside it needs a + value that is not generalized inside an annotated method type. + Open question 3 of this turn. + - *A row walked by `free_univar`*: `r3_tests::a_free_universal_ + variable_is_found_in_a_row_in_order` (fields, then the name's + arguments, then the row variable; an opened `Reither` field through + its chain; nothing below a reference resolved to absent). + - *The passive split*: `r3_tests::passive_variants_keep_the_levels_ + of_the_types_that_hold_a_universal_variable` sets + `Extras::passive_variants` by hand, with holders on both sides and + the holder AFTER the kept type in each list: the holders stay two + nodes at their level, the kept types are lowered to the merged row + variable's level. With the caller's loop as it was before decision + 54 the test fails (observed: a holder at level 1). +60. **The arguments of a row's name are children only of the innermost + row, in every walk.** OCaml's `iter_row` visits them only when + `row_more` is not itself a variant (btype.ml:229-241), every walk of + a type goes through `iter_type_expr` (btype.ml:263-265), and a copy + takes the name of the innermost row (`row_repr`, btype.ml:126-135). + The kernel's walks push every slot of a node. Before this turn: + + | Walk | Kernel | Visited them for a row whose `more` is a variant | OCaml | + |---|---|---|---| + | lowering | `Graph::lower`, `Checker::lower_scoped` | yes | no: `update_level`, ctype.ml:625-626 | + | generalization | `Graph::generalize`, `generalize_structure` | yes | no: ctype.ml:509-516, 524-540 | + | occurrence of a structure pair, of a binding | `Checker::reaches`, `occur_expanding`, the scan of `binding_scan` | not entered: a row stops the walk | not entered: `occur_rec`, ctype.ml:1403 | + | deep occurrence | `Checker::reaches` with `deep`, the routed `binding_scan` | yes | no: `deep_occur`, ctype.ml:1735-1782 | + | the lowering part of a binding | `binding_scan`, `LEVEL_ONLY` entries below a row | yes | no: `update_level` | + | free universal variable | `Checker::free_univar` | yes | no: `occur_univar`, ctype.ml:1601 | + | escape of a universal variable | `Checker::univars_escape` | yes | no: ctype.ml:1648 | + | closedness of an expansion | `Checker::closed_over` | yes | no: `closed_parameterized_type`, ctype.ml:441-451 | + | copy, instance | `Graph::copy`, `Graph::instance` | copied as slots of the copy | not copied: `copy_row` of `row_repr`, ctype.ml:800-868 | + + Observed on the old checker with `[> #B]` unified with `[> #A] as + p`, `y` in no field (`differential::tests::old_checker_does_not_ + reach_the_arguments_of_a_stale_row_name`): a lowering, a + generalization, `deep_occur`, a copy and a binding all leave `y` + alone. What can show it: the level of a type that is the argument of + a phantom parameter of a named open row, after the row gained a tag. + Corrected on the write side, so that no walk pays a descriptor load + per variant node: when `unify_row` has run both `set_more`, each row + that is not fixed has a variant node as its `more` (ctype.ml:2616), + and when such a row has a name with arguments its home node is + written with a new list in which the slots of those arguments hold + what the `more` slot holds (`Checker::drop_name_children`; in the + explaining mode a `Fields` frame puts the descriptor back on a + failure, as it puts back the rest whose link made the name stale). + Nothing reads the name of a row that is not the innermost. Cost: two + compares per unified pair of rows; for a named row with arguments + that gains a tail, one list of the node's slots. Test: + `r3_tests::the_arguments_of_a_stale_row_name_are_not_children`, one + block per walk of the table, in both modes (without the edit the + lowering already reaches `y`, observed). Two other places link a row + variable to a variant node, and each calls `drop_name_children` for + the row when it is built: `reify` (ctype.ml:1848-1852, R3h; 8.9) and + `rigidify_rec` (ctype.ml:3139-3147, R3i). + Found on the way and NOT corrected: `update_level` drops the name of + a row whose path would escape (`Some (p, _tl) when level < get_level + env p -> ... row_name = None`, ctype.ml:618-623), and + `Checker::lower_scoped` has no such arm. It needs a named row whose + type has a scope, which no generated program has. Open question 4 of + this turn. + ### 8.7 The hot paths of unification (K2) diff --git a/compiler-rust/src/tc/differential.rs b/compiler-rust/src/tc/differential.rs index 53f29a15a9..15a84fb611 100644 --- a/compiler-rust/src/tc/differential.rs +++ b/compiler-rust/src/tc/differential.rs @@ -4012,6 +4012,141 @@ mod tests { } } + /// The corner of decision 53 on the old checker, built by hand + /// (section 8.6, decision 58; observed). Two `Reither` fields inside + /// two polymorphic types: `#A` with the types `[(u, x); x]` on the + /// left and `[(u', x'); x']` on the right. Each side would gain the + /// other's two types; the tuples hold a universal variable and are + /// unified instead, which makes `x` and `x'` one node; the lists the + /// fields gain were computed BEFORE that unification + /// (compiler/ml/ctype.ml:2685-2706): + /// + /// ```text + /// let tl2' = remq tl2 tl1 and tl1' = remq tl1 tl2 in + /// let tl1', tlu1 = split_univars tl1' and tl2', tlu2 = split_univars tl2' in + /// (match (tlu1, tlu2) with + /// | tu1 :: tlu1, _ :: _ -> + /// if not !(passive_variants ()) then List.iter (unify env tu1) (tlu1 @ tlu2) + /// ... + /// let f1' = Reither (c1 || c2, tl1', m1 || m2, e) + /// ``` + /// + /// so each field ends with THREE types, the last two one node (`@4 + /// @4`): the type it had, and the same node again as the type it + /// gained. The OCaml compiler was run on a source program that reaches + /// the corner (the pull request of the sixth skeleton turn has it): + /// its fields hold three types after the join too. + #[test] + fn old_checker_gains_a_type_the_holders_made_a_duplicate() { + use crate::types::type_expr::{RowDesc, RowField}; + let program = compile(UNIVAR_PROGRAM.to_string()); + let plan = Plan::new(Tier::Full, UNIVAR_PLAN); + let old = old_context(&program, plan); + let ctx = &old.ctx; + let side = || { + let u = ctx.new_univar(None); + let x = ctx.new_var(None); + let holder = ctx.new_tuple(vec![u, x]); + let field = RowField::Reither { + constant: false, + types: vec![holder, x], + matched: false, + link: ctx.alloc_row_field(None), + }; + let row = ctx.new_type(TypeDesc::Tvariant(ctx.new_row_desc(RowDesc { + row_fields: vec![("A".into(), field)].into(), + row_more: ctx.new_var(None), + row_closed: true, + row_fixed: false, + row_name: None, + }))); + (ctx.new_poly(row, vec![u]), row, x) + }; + let (left, row1, x1) = side(); + let (right, row2, x2) = side(); + let mut state = ctype::UnifyState::new(); + state.set_env(&old.env); + ctype::unify(ctx, &mut state, left, right).unwrap(); + let form = old_forms(&old, plan, &[left, right, row1, row2, x1, x2], &[]).0; + assert_eq!( + form, + "0 = &0[0]poly(&1[0][closed #A?^0(&2[0](&3[0]!, &4[0]') @4 @4 ) | &5[0]']; @3)\n\ + 1 = @0\n2 = @1\n3 = @1\n4 = @4\n5 = @4\n" + ); + } + + /// What the old checker's walks do with the arguments of a row name + /// that is no longer the name of the innermost row (section 8.6, + /// decision 60; observed). `[> #B]` is unified with `[> #A] as p`: + /// the named row gains `#B` through a new variant node its row + /// variable is linked to (`set_more`, compiler/ml/ctype.ml:2596-2616) + /// and the merged row has no name (ctype.ml:2574-2586), so `p` is + /// the name of a row whose `more` is a variant. `iter_row` visits a + /// name's arguments only for the row a chain ends in + /// (compiler/ml/btype.ml:229-241), so `y`, which is in no field, is + /// reached by no walk: a lowering of the row leaves `y` at its level + /// (`[1]`), a generalization leaves it there, `deep_occur` does not + /// find it, a copy of the generalized row makes no copy of it that + /// the row holds, and a variable bound to the row leaves it. + #[test] + fn old_checker_does_not_reach_the_arguments_of_a_stale_row_name() { + use crate::types::type_expr::{RowDesc, RowField}; + let program = compile(UNIVAR_PROGRAM.to_string()); + let plan = Plan::new(Tier::Full, UNIVAR_PLAN); + let old = old_context(&program, plan); + let ctx = &old.ctx; + let mut out = String::new(); + for walk in 0..5 { + ctx.begin_def(); + let y = ctx.new_var(None); + let path = crate::types::path::Path::pident(ctx.create_ident("p")); + let row = |tag: &str, name: Option<(crate::types::path::Path, Vec)>| { + ctx.new_type(TypeDesc::Tvariant(ctx.new_row_desc(RowDesc { + row_fields: vec![(tag.into(), RowField::Rpresent(None))].into(), + row_more: ctx.new_var(None), + row_closed: false, + row_fixed: false, + row_name: name, + }))) + }; + let named = row("A", Some((path, vec![y]))); + let other = row("B", None); + let mut state = ctype::UnifyState::new(); + state.set_env(&old.env); + ctype::unify(ctx, &mut state, other, named).unwrap(); + ctx.end_def(); + let mut operands = vec![named, y]; + match walk { + 0 => ctype::update_level(ctx, ctx.current_level(), named), + 1 => ctype::generalize(ctx, named), + 2 => { + let found = ctype::deep_occur(ctx, y, named); + out.push_str(&format!("deep_occur {found}\n")); + } + 3 => { + ctype::generalize(ctx, named); + ctype::generalize(ctx, y); + operands.push(ctype::instance(ctx, named)); + operands.push(ctype::instance(ctx, y)); + } + _ => { + let v = ctx.new_var(None); + ctype::unify(ctx, &mut state, v, named).unwrap(); + } + } + out.push_str(&old_forms(&old, plan, &operands, &[]).0); + } + assert_eq!( + out, + "0 = &0[0][#A #B | &1[0]']\n1 = &2[1]'\n\ + 0 = &0[G][#A #B | &1[G]']\n1 = &2[1]'\n\ + deep_occur false\n\ + 0 = &0[1][#A #B | &1[1]']\n1 = &2[1]'\n\ + 0 = &0[G][#A #B | &1[G]']\n1 = &2[G]'\n2 = &3[0][#A #B | &4[0]']\n3 = &5[0]'\n\ + 0 = &0[0][#A #B | &1[0]']\n1 = &2[1]'\n" + ); + } + /// 10,000 cases of everything on programs with polymorphic fields, /// universal variables, rows whose row variable is bound and /// hand-made polymorphic types, and 10,000 more with made nodes: the diff --git a/compiler-rust/src/tc/r3_tests.rs b/compiler-rust/src/tc/r3_tests.rs index 149fbab12c..b21b4a5ae9 100644 --- a/compiler-rust/src/tc/r3_tests.rs +++ b/compiler-rust/src/tc/r3_tests.rs @@ -3352,6 +3352,448 @@ fn the_types_that_hold_a_universal_variable_are_unified_not_gained() { } } +/// A closed row with the one tag `#A`, `Reither (false, types, false, ref +/// None)` with any number of types (a conjunction), unopened, at level 2. +fn conjunctive_row(c: &mut Checker, types: &[Ty], more: Ty) -> Ty { + let name = name_index(c, "A"); + let start = c.extras.row_fields.len() as u32; + let kind = RowFieldKind::Either { constant: false, matched: false }; + let published = types.len() as u32 + 1; + c.extras.row_fields.push(RowFieldRow { + name, + types: Range { start: 0, len: types.len() as u32 }, + link: published, + kind, + }); + c.extras.variants.push(VariantRow { + fields: Range { start, len: 1 }, + more: types.len() as u32, + name: NONE, + name_arguments: Range::default(), + closed: true, + fixed: false, + }); + let index = c.extras.variants.len() as u32 - 1; + let mut slots = types.to_vec(); + slots.push(more); + extended_at(c, 2, ExtraDescriptor::Variant(index), &slots) +} + +/// What two `Reither` fields gain is decided before the types that hold +/// a universal variable are unified (section 8.6, decisions 53 and 58; +/// compiler/ml/ctype.ml:2685-2706): +/// +/// ```text +/// let tl2' = remq tl2 tl1 and tl1' = remq tl1 tl2 in +/// let tl1', tlu1 = split_univars tl1' and tl2', tlu2 = split_univars tl2' in +/// (match (tlu1, tlu2) with +/// | [], [] -> () +/// | tu1 :: tlu1, _ :: _ -> +/// if not !(passive_variants ()) then List.iter (unify env tu1) (tlu1 @ tlu2) +/// | tu :: _, [] | [], tu :: _ -> occur_univar !env tu); +/// List.iter (update_level !env (repr more).level) (tl1' @ tl2'); +/// let e = ref None in +/// let f1' = Reither (c1 || c2, tl1', m1 || m2, e) +/// and f2' = Reither (c1 || c2, tl2', m1 || m2, e) in +/// ``` +/// +/// Witness (the old checker's side is `differential::tests:: +/// old_checker_gains_a_type_the_holders_made_a_duplicate`): `poly([< #A((u, +/// x)) & x]; u)` against `poly([< #A((u', x')) & x']; u')`. `tl1'` is +/// `[(u', x'); x']`: the tuple holds a universal variable, `x'` is kept. +/// The two tuples are unified, which makes `x'` a link to `x`... and the +/// field of row 1 still gains `x'`: it reads `[(u, x); x; x]`, three +/// types of which the last two are one node, and so does the field of +/// row 2. A join that compared the lists again after the unification +/// would gain nothing (two types): what `split_univars` did before the +/// sixth skeleton turn. The body is `Checker::join_either`'s second run +/// and the arm of `Checker::split_univars` with holders on both sides. +/// Reaches: R3f, R3g. +#[test] +fn a_field_gains_what_it_lacked_before_the_holders_were_unified() { + for mode in [Mode::Success, Mode::Explain] { + let mut c = checker(mode); + let side = |c: &mut Checker| { + let u = univar_at(c, 2); + let x = var(c); + let holder = tuple(c, &[u, x]); + let more = var(c); + let row = conjunctive_row(c, &[holder, x], more); + (poly_of(c, 2, row, &[u]), row, holder, x) + }; + let (left, _, holder1, x1) = side(&mut c); + let (right, row2, holder2, x2) = side(&mut c); + let rows = rows_now(&c); + c.unify(left, right).unwrap(); + assert_rows_kept(&c, &rows); + assert_eq!(c.g.repr(holder1), c.g.repr(holder2)); + assert_eq!(c.g.repr(x1), c.g.repr(x2)); + let (tags, _, _) = read_row(&mut c, row2); + assert_eq!(tags[0].1.types, 3, "{mode:?}"); + let mut types = Vec::new(); + let row2 = c.g.repr(row2); + let node = c.g.nodes[row2.0 as usize]; + let ExtraDescriptor::Variant(index) = c.extras.descriptors[node.a as usize] else { + panic!("a variant"); + }; + let field = c.extras.row_fields[c.extras.variants[index as usize].fields.start as usize]; + super::rows::field_types(&c.g, node.b, node.arity(), field, &mut types); + let types: Vec = types.into_iter().map(|ty| c.g.repr(ty)).collect(); + let x = c.g.repr(x1); + assert_eq!(types, [c.g.repr(holder1), x, x]); + assert!(c.scopes.is_empty() && c.frames.is_empty() && c.saved.is_empty()); + } +} + +/// An open row `[> #tag] as P` at `level`, unopened: slot 0 its +/// row variable (new, at `level`), slot 1 the argument of its name. +fn named_row(c: &mut Checker, level: u32, tag: &str, argument: Option) -> Ty { + let name = name_index(c, tag); + let start = c.extras.row_fields.len() as u32; + c.extras.row_fields.push(RowFieldRow { + name, + types: Range::default(), + link: NONE, + kind: RowFieldKind::Present, + }); + c.extras.variants.push(VariantRow { + fields: Range { start, len: 1 }, + more: 0, + name: if argument.is_some() { PA } else { NONE }, + name_arguments: Range { start: 1, len: u32::from(argument.is_some()) }, + closed: false, + fixed: false, + }); + let index = c.extras.variants.len() as u32 - 1; + let mut slots = vec![var_at(c, level)]; + slots.extend(argument); + extended_at(c, level, ExtraDescriptor::Variant(index), &slots) +} + +/// The arguments of a row's name are children of the row only while the +/// row is the one its chain ends in (section 8.6, decision 60; +/// compiler/ml/btype.ml:229-241, `iter_row`): +/// +/// ```text +/// match (repr row.row_more).desc with +/// | Tvariant row -> iter_row f row +/// | Tvar _ | Tunivar _ | Tsubst _ | Tconstr _ | Tnil -> +/// Misc.may (fun (_, l) -> List.iter f l) row.row_name +/// ``` +/// +/// Witness, per walk: `[> #B]` unified with `[> #A] as P` at level 2, +/// `y` in no field. The named row is the right node, gains `#B` through +/// a variant node its row variable is linked to, and the merged row has +/// no name (ctype.ml:2574-2586, 2596-2616). Then, each on a graph of its +/// own: +/// - `Checker::lower` of the row to level 1 leaves `y` at 2 +/// (`update_level`, ctype.ml:584-630, through `iter_type_expr`); +/// - `Checker::generalize` after the definition leaves `y` at 2 +/// (`generalize`, ctype.ml:509-516); +/// - `Checker::deep_occur` of `y` in the row is false (ctype.ml:1735-1782); +/// - a variable of level 1 bound to the row (the binding walk, which +/// lowers below a row) leaves `y` at 2; +/// - with `y` a universal variable, `Checker::free_univar` of the row +/// finds none (`occur_univar`, ctype.ml:1563-1607); +/// - `Checker::univars_escape` of a binder of that universal variable in +/// a polymorphic type whose body is the row is false +/// (ctype.ml:1630-1652); +/// - a `Checker::copy` of the generalized row with `y` generic makes one +/// variable, the row variable's, and no copy of `y` (`row_repr` gives a +/// copy the innermost row's name, compiler/ml/btype.ml:126-135, +/// ctype.ml:800-868). +/// +/// The old checker's side, observed: `differential::tests:: +/// old_checker_does_not_reach_the_arguments_of_a_stale_row_name`. Before +/// the sixth skeleton turn every one of these walks reached `y` (each +/// pushes every slot of a node); the body is +/// `Checker::drop_name_children`, called by `Checker::unify_row`. While +/// the row IS the innermost, its name's arguments are children: the +/// last block. +/// Reaches: R3f, R3n, R3g. +#[test] +fn the_arguments_of_a_stale_row_name_are_not_children() { + // The named row after it gained `#B`, and the argument of its name. + let stale = |c: &mut Checker, y: Ty| { + let other = named_row(c, 2, "B", None); + let named = named_row(c, 2, "A", Some(y)); + let rows = rows_now(c); + c.unify(other, named).unwrap(); + assert_rows_kept(c, &rows); + assert_eq!(shape(c, named), "[A B | ']"); + named + }; + let level = |c: &mut Checker, ty: Ty| { + let ty = c.g.repr(ty); + c.g.nodes[ty.0 as usize].level + }; + for mode in [Mode::Success, Mode::Explain] { + let mut c = checker(mode); + let y = var(&mut c); + let named = stale(&mut c, y); + c.lower(named, 1).unwrap(); + assert_eq!((level(&mut c, named), level(&mut c, y)), (1, 2), "lower"); + + let mut c = checker(mode); + let y = var(&mut c); + let named = stale(&mut c, y); + c.g.end_def(); + c.generalize(named); + assert_eq!((level(&mut c, named), level(&mut c, y)), (GENERIC, 2), "generalize"); + + let mut c = checker(mode); + let y = var(&mut c); + let named = stale(&mut c, y); + assert!(!c.deep_occur(y, named), "deep_occur"); + + let mut c = checker(mode); + let y = var(&mut c); + let named = stale(&mut c, y); + let v = var_at(&mut c, 1); + c.unify(v, named).unwrap(); + assert_eq!((level(&mut c, named), level(&mut c, y)), (1, 2), "binding"); + + let mut c = checker(mode); + let u = univar_at(&mut c, 2); + let named = stale(&mut c, u); + assert_eq!(c.free_univar(named), Ty(NONE), "free_univar"); + let binder = poly_of(&mut c, 2, u, &[u]); + let w = univar_at(&mut c, 2); + let other = poly_of(&mut c, 2, named, &[w]); + let (binder, other) = (c.g.nodes[binder.0 as usize], c.g.nodes[other.0 as usize]); + assert!(!c.univars_escape(binder, other), "univars_escape"); + + let mut c = checker(mode); + let y = var_at(&mut c, GENERIC); + let named = stale(&mut c, y); + c.g.end_def(); + c.generalize(named); + let first = c.g.nodes.len(); + let copy = c.copy(named); + assert_eq!(shape(&mut c, copy), "[A B | ']"); + let variables = c.g.nodes[first..].iter().filter(|node| node.tag() == VAR).count(); + assert_eq!(variables, 1, "copy"); + } + // The innermost row's name: its argument is a child. + let mut c = checker(Mode::Success); + let y = var(&mut c); + let named = named_row(&mut c, 2, "A", Some(y)); + assert!(c.deep_occur(y, named)); + c.lower(named, 1).unwrap(); + assert_eq!(level(&mut c, y), 1); +} + +/// `free_univar` over a variant row (compiler/ml/btype.ml:229-241, +/// 263-265; ctype.ml:1563-1607): the fields' types, then the arguments +/// of the name, then the row variable: +/// +/// ```text +/// | Tvariant row -> +/// iter_row f row; +/// f (row_more row) +/// ``` +/// +/// Witnesses, each a row at level 2 with free universal variables in +/// three places, found in that order: +/// - `[> #A(u0)] as P` whose row variable is the universal variable +/// `u2`: `u0`; +/// - `[> #A(int)] as P` over `u2`: `u1`; +/// - `[> #A(int)]` over `u2`: `u2`; +/// - inside `poly(...; u2)`, the same row: none (the row variable is +/// bound); +/// - an OPENED row `[< #A(u0)]` (`Reither`, its type under the chain +/// node of its reference): `u0`; and after that reference is resolved +/// to absent, none: `iter_row` visits what `row_field_repr` gives. +/// +/// The body is the `Variant` arm of +/// `Checker::push_free_univar_children`. +/// Reaches: R3f, R3g. +#[test] +fn a_free_universal_variable_is_found_in_a_row_in_order() { + let mut c = checker(Mode::Success); + // `[> #A(argument)] as P` over `more`. + let row = |c: &mut Checker, argument: Ty, named: Option, more: Ty| { + let name = name_index(c, "A"); + let start = c.extras.row_fields.len() as u32; + c.extras.row_fields.push(RowFieldRow { + name, + types: Range { start: 0, len: 1 }, + link: NONE, + kind: RowFieldKind::Present, + }); + c.extras.variants.push(VariantRow { + fields: Range { start, len: 1 }, + more: 1, + name: if named.is_some() { PA } else { NONE }, + name_arguments: Range { start: 2, len: u32::from(named.is_some()) }, + closed: false, + fixed: false, + }); + let index = c.extras.variants.len() as u32 - 1; + let mut slots = vec![argument, more]; + slots.extend(named); + extended_at(c, 2, ExtraDescriptor::Variant(index), &slots) + }; + let [u0, u1, u2] = [(); 3].map(|()| univar_at(&mut c, 2)); + let int = constr(&mut c, INT, &[]); + let first = row(&mut c, u0, Some(u1), u2); + assert_eq!(c.free_univar(first), u0); + let second = row(&mut c, int, Some(u1), u2); + assert_eq!(c.free_univar(second), u1); + let third = row(&mut c, int, None, u2); + assert_eq!(c.free_univar(third), u2); + let bound = poly_of(&mut c, 2, third, &[u2]); + assert_eq!(c.free_univar(bound), Ty(NONE)); + + let more = var(&mut c); + let either = variant_at(&mut c, 2, &[("A", Tag::Either(false, Some(u0), false))], more, true, NONE); + c.open_row(either); + assert_eq!(c.free_univar(either), u0); + let (tags, _, _) = read_row(&mut c, either); + let holders = c.resolve(tags[0].1.end, super::rows::RES_ABSENT, Ty(NONE)); + c.settle(holders, super::rows::RES_ABSENT); + assert_eq!(c.free_univar(either), Ty(NONE)); +} + +/// The split of the types that hold a universal variable while the +/// variants are passive (compiler/ml/ctype.ml:2693-2703): +/// +/// ```text +/// | tu1 :: tlu1, _ :: _ -> +/// (* Attempt to merge all the types containing univars *) +/// if not !(passive_variants ()) then +/// List.iter (unify env tu1) (tlu1 @ tlu2) +/// | tu :: _, [] | [], tu :: _ -> occur_univar !env tu); +/// (* Is this handling of levels really principal? *) +/// List.iter (update_level !env (repr more).level) (tl1' @ tl2'); +/// ``` +/// +/// Witness: `poly([< #A(z & (u, x))]; u)` against `poly([< #A(z' & (u', +/// x'))]; u')`, every type at level 2 and the two row variables at level +/// 1, with `Extras::passive_variants` set (nothing in `tc` sets it: the +/// typing of a pattern for an exhaustiveness check does, `with_passive_ +/// variants`, ctype.ml:2189-2202, from typecore.ml:1931; section 8.9). +/// The two tuples hold a universal variable and are NOT unified and NOT +/// lowered: they stay two nodes at level 2. `z` and `z'` are gained and +/// lowered to the level of the merged row variable, 1. Each field then +/// has its two types and the one it gained. The holder comes after the +/// kept type in each list, so that the place between the two kept +/// ranges of the scratch holds a holder: the range `Checker::join_either` +/// lowered before the review of R3g (section 8.6, decision 54). The +/// bodies are the passive arm of `Checker::split_univars` and the two +/// lowering loops of `Checker::join_either`. +/// Reaches: R3f, R3g. +#[test] +fn passive_variants_keep_the_levels_of_the_types_that_hold_a_universal_variable() { + let mut c = checker(Mode::Success); + c.extras.passive_variants = true; + let side = |c: &mut Checker| { + let u = univar_at(c, 2); + let (x, z) = (var(c), var(c)); + let holder = tuple(c, &[u, x]); + let more = var_at(c, 1); + let row = conjunctive_row(c, &[z, holder], more); + (poly_of(c, 2, row, &[u]), row, holder, z) + }; + let (left, _, holder1, z1) = side(&mut c); + let (right, row2, holder2, z2) = side(&mut c); + c.unify(left, right).unwrap(); + let level = |c: &Checker, ty: Ty| c.g.nodes[ty.0 as usize].level; + assert_ne!(c.g.repr(holder1), c.g.repr(holder2)); + assert_eq!((level(&c, holder1), level(&c, holder2)), (2, 2)); + assert_eq!((level(&c, z1), level(&c, z2)), (1, 1)); + let (tags, _, more) = read_row(&mut c, row2); + assert_eq!(tags[0].1.types, 3); + assert_eq!(level(&c, more), 1); + assert!(c.scopes.is_empty() && c.frames.is_empty() && c.saved.is_empty()); +} + +/// The cost of the two walks for universal variables, as relations +/// between n = 1,000 and n = 10,000 (section 8.6, decisions 56 and 59). +/// - `Checker::free_univar` over a tuple of n variables inside one +/// binder: one visit a node, linear. +/// - `Checker::free_univar` over a chain of n polymorphic types WITHOUT +/// bound variables in which every type is held twice, once under such +/// a type and once outside it: `t(0)` is the bound variable, `t(i + 1)` +/// is `(poly(t(i);), t(i))`, the whole inside one binder. A +/// polymorphic type without bound variables adds nothing to `bound` +/// (`List.fold_right Type_set.add (List.map repr tyl) bound`, +/// compiler/ml/ctype.ml:1587-1589), so OCaml visits each node once +/// (ctype.ml:1569-1581: with the same `bound` a node is not visited +/// again). So does the walk: exactly `2n + 1` visits (n tuples, n +/// polymorphic types, the binder), ZERO extra. Before the sixth +/// skeleton turn such a type opened a context, a node first met inside +/// it was visited again after it was left, and this chain took `2^n` +/// visits: the test would not end. +/// - `Checker::univars_escape` of a known binder over a tuple of n +/// variables: the family pass and one visit a node, linear. +/// +/// Reaches: R3g. +#[test] +fn the_walks_for_universal_variables_at_ten_thousand() { + let wide = |n: u32| { + let mut c = checker(Mode::Success); + let u = univar_at(&mut c, 2); + let mut elements: Vec = (0..n).map(|_| var(&mut c)).collect(); + elements.push(u); + let body = tuple(&mut c, &elements); + let ty = poly_of(&mut c, 2, body, &[u]); + let mut found = u; + let count = steps::scope(|| found = c.free_univar(ty)); + assert_eq!(found, Ty(NONE)); + count + }; + let (small, large) = (wide(1_000), wide(10_000)); + assert!(large <= 11 * small, "wide: {small} {large}"); + + let shared = |n: u32| { + let mut c = checker(Mode::Success); + let u = univar_at(&mut c, 2); + let mut ty = u; + for _ in 0..n { + let inner = poly_of(&mut c, 2, ty, &[]); + ty = tuple(&mut c, &[inner, ty]); + } + let ty = poly_of(&mut c, 2, ty, &[u]); + let mut found = u; + let count = steps::scope(|| found = c.free_univar(ty)); + assert_eq!(found, Ty(NONE)); + count + }; + assert_eq!(shared(1_000), 2 * 1_000 + 1); + assert_eq!(shared(10_000), 2 * 10_000 + 1); + + let escape = |n: u32| { + let mut c = checker(Mode::Success); + // An open scope that pairs `u` with `u2`, so that a type that + // binds `u` again has a known binder. + let (u, u2) = (univar_at(&mut c, 2), univar_at(&mut c, 2)); + let (outer1, outer2) = (poly_of(&mut c, 2, u, &[u]), poly_of(&mut c, 2, u2, &[u2])); + let (n1, n2) = (c.g.nodes[outer1.0 as usize], c.g.nodes[outer2.0 as usize]); + let partners = c.g.push(partner_node(Ty(NONE))); + c.g.push(partner_node(Ty(NONE))); + c.scopes.push(PolyScope { left: n1.b + 1, right: n2.b + 1, left_len: 1, right_len: 1, partners }); + let binder = poly_of(&mut c, 2, u, &[u]); + let w = univar_at(&mut c, 2); + let elements: Vec = (0..n).map(|_| var(&mut c)).collect(); + let body = tuple(&mut c, &elements); + let other = poly_of(&mut c, 2, body, &[w]); + let (binder, other) = (c.g.nodes[binder.0 as usize], c.g.nodes[other.0 as usize]); + let mut escapes = true; + let count = steps::scope(|| escapes = c.univars_escape(binder, other)); + assert!(!escapes); + // With `u2` in the body it escapes: `u2` is in the family of `u`. + let body = tuple(&mut c, &[u2]); + let other = poly_of(&mut c, 2, body, &[w]); + let other = c.g.nodes[other.0 as usize]; + assert!(c.univars_escape(binder, other)); + c.scopes.clear(); + count + }; + let (small, large) = (escape(1_000), escape(10_000)); + assert!(large <= 11 * small, "escape: {small} {large}"); +} + /// The cost of universal variables, as relations between n = 1,000 and /// n = 10,000. /// - n bound variables, the body the last of them: `unify_univar` scans diff --git a/compiler-rust/src/tc/rows.rs b/compiler-rust/src/tc/rows.rs index 86a0fa5628..6f5c7b16e8 100644 --- a/compiler-rust/src/tc/rows.rs +++ b/compiler-rust/src/tc/rows.rs @@ -1446,6 +1446,18 @@ impl Checker { return Err(clash); } }; + // A row that is not fixed has a variant node as its `more` now + // (`link_type rm ty`, ctype.ml:2616), so its name's arguments are + // children no longer (`iter_row`, btype.ml:229-241). + let homes = [ + (fixed1 || last1.0 == NONE, row1, last1), + (fixed2, row2, if last2.0 == NONE { t2x } else { last2 }), + ]; + for (kept, row, home) in homes { + if !kept && row.name != NONE && row.name_arguments.len != 0 { + self.drop_name_children(home, row); + } + } // 9. The static row below the common tags, which are popped in the // order of the pairs, then the pair a fixed row asked for. if closed { @@ -1466,6 +1478,53 @@ impl Checker { Ok(()) } + /// The arguments of a row's name stop being children of variant node + /// `home` (a representative, whose `VariantRow` is `row`) once the + /// row's `more` is a variant node: OCaml's `iter_row` visits them only + /// for the row a chain of rows ends in (compiler/ml/btype.ml:229-241): + /// + /// ```text + /// match (repr row.row_more).desc with + /// | Tvariant row -> iter_row f row + /// | Tvar _ | Tunivar _ | Tsubst _ | Tconstr _ | Tnil -> + /// Misc.may (fun (_, l) -> List.iter f l) row.row_name + /// ``` + /// + /// and every walk of a type goes through it (`iter_type_expr`, + /// btype.ml:263-265): `update_level`, `generalize`, `deep_occur`, + /// `occur_univar`, `univars_escape`; a copy takes the name of the + /// innermost row (`row_repr`, btype.ml:126-135). The kernel's walks + /// push every slot of a node, so the node is written + /// (`Graph::write`, logged in a speculation) with a new list in which + /// each slot of the name's arguments holds what the `more` slot + /// holds: a child the walk reaches anyway. The shared `VariantRow` is + /// not written, and nothing reads the name of a row that is not the + /// innermost (`row_repr`). In `Mode::Explain` a + /// [`FrameKind::Fields`] frame puts the node's descriptor back when + /// the unification fails, as it puts back the two rests, whose links + /// are what made the name stale (`rm1.desc <- md1`, ctype.ml: + /// 2637-2641). Called by [`Checker::unify_row`] for a row that has a + /// name with arguments and is not fixed: O(slots of the node), one + /// list of that length; a row without a name with arguments costs the + /// two compares of the caller. (CHECKER_REWRITE 8.6, decision 60) + #[cold] + #[inline(never)] + fn drop_name_children(&mut self, home: Ty, row: VariantRow) { + let node = self.g.nodes[home.0 as usize]; + if self.g.mode == Mode::Explain { + let extra = self.saved.len() as u32; + self.saved.push(node); + self.saved.push(node); + self.frames.push(Frame { left: home, right: home, kind: FrameKind::Fields, extra }); + } + let b = self.g.lists.len() as u32; + self.g.lists.extend_from_within(node.b as usize..(node.b + node.arity()) as usize); + let more = self.g.lists[(b + row.more) as usize]; + let arguments = row.name_arguments; + self.g.lists[(b + arguments.start) as usize..(b + arguments.start + arguments.len) as usize].fill(more); + self.g.write(home, Node { b, ..node }); + } + /// O(chain): the field `extras.row_scratch[at]` is, read now /// ([`field_repr`]; its home is opened, so every `Reither` field of it /// has its reference slot). @@ -2080,13 +2139,30 @@ impl Checker { } } }; - missing(self, middle..end, base..middle); - let gained1 = end..self.extras.row_scratch.len(); - missing(self, base..middle, middle..end); - let gained2 = gained1.end..self.extras.row_scratch.len(); + // The second run, after the types that hold a universal variable + // were unified: `tl1'` and `tl2'` are the lists the first run kept + // (`let tl1', tlu1 = split_univars tl1'`, computed BEFORE `List.iter + // (unify env tu1) (tlu1 @ tlu2)`, ctype.ml:2685-2697), read back + // from `saved` after the record ([`Checker::split_univars`]), not + // the two lists compared again: a type those unifications made + // the same node as one the side already has is still gained + // (CHECKER_REWRITE 8.6, decision 58). + let (gained1, gained2) = if record.b & ROW_UNIVARS != 0 { + let counts = self.saved[frame.extra as usize + 1]; + let first = frame.extra as usize + 2; + for at in first..first + (counts.head + counts.level) as usize { + let ty = Ty(self.saved[at].head); + self.extras.row_scratch.push(FieldRef { name: 0, at: 0, ty }); + } + (end..end + counts.head as usize, end + counts.head as usize..self.extras.row_scratch.len()) + } else { + missing(self, middle..end, base..middle); + let gained1 = end..self.extras.row_scratch.len(); + missing(self, base..middle, middle..end); + (gained1.clone(), gained1.end..self.extras.row_scratch.len()) + }; // PR#6744 (ctype.ml:2685-2699), on one load when the graph has no - // universal variable; not on the second run, after the types that - // hold one were unified. + // universal variable; not on the second run. let (gained1, gained2) = if self.extras.univars != 0 && record.b & ROW_UNIVARS == 0 { match self.split_univars(frame, [gained1, gained2]) { Ok(Some([gained1, gained2])) => (gained1, gained2), @@ -2808,7 +2884,9 @@ impl Checker { /// root context is never visited again (ctype.ml:1569-1571); one /// visited in a context that has closed is visited again; /// - otherwise the node is marked with `current`, one step, and: - /// - a polymorphic type (ctype.ml:1587-1589): a new generation `g` + /// - a polymorphic type without bound variables: its body, in the + /// current context (`bound` is unchanged, ctype.ml:1587-1589); + /// - a polymorphic type with bound variables: a new generation `g` /// and `open_polys.push(true)`; each bound variable's /// representative whose mark is not live is marked `g` (one that /// an enclosing open type binds keeps that mark, so that leaving @@ -2915,6 +2993,15 @@ impl Checker { TUPLE if node.head & EXTENDED != 0 => match self.extras.descriptors[node.a as usize] { // `Tpoly (ty, tyl)`: the body with the bound variables // added to `bound` (compiler/ml/ctype.ml:1587-1589). + // `Tpoly (ty, [])` adds nothing to `bound`: no context, the + // body is a child like any other. A context for it would + // make every node first met below it be visited again + // after it is left, which doubles the walk at each such + // type that shares its body with the outside + // (CHECKER_REWRITE 8.6, decision 59). + ExtraDescriptor::Poly if arity == 1 => { + self.g.work.push(self.g.lists[node.b as usize]); + } ExtraDescriptor::Poly => { let generation = self.fresh_marks(); self.extras.open_polys.push(true); @@ -3002,9 +3089,12 @@ impl Checker { /// [`Checker::row_field_end`], which popped it) one is pushed; then /// the pairs of the first type of `tlu1` with each other type of /// `tlu1` and each type of `tlu2`, pushed so that they are popped - /// in that order. `Ok(None)`: the join is run again when they are - /// unified, and then every such type is on both sides and gained - /// by neither. + /// in that order; the types each range KEEPS are written to + /// `saved` after the record (an entry with the two counts, then one + /// entry a type). `Ok(None)`: the join is run again when the + /// holders are unified, and gains exactly those kept types, as + /// OCaml's `tl1'` and `tl2'` are computed before `unify env tu1` + /// (ctype.ml:2685-2706; CHECKER_REWRITE 8.6, decision 58). /// - One empty (`occur_univar !env tu`): [`Checker::occur_univar`] of /// the first type of the other, which fails and pushes its pair in /// `Mode::Explain`: `Err([NONE, NONE])`. @@ -3028,6 +3118,24 @@ impl Checker { // tlu2)`. (false, false) => { self.saved[frame.extra as usize].b |= ROW_UNIVARS; + // What the two fields gain is decided now, before the + // holders are unified (ctype.ml:2685-2706): the kept types + // go after the record, first an entry with the two + // counts (`head`, `level`), then one entry a type + // (`head`). Only this arm writes them: O(types kept), + // 16 bytes each, until the pair's `RowFieldEnd` frame is + // popped. + debug_assert_eq!(self.saved.len(), frame.extra as usize + 1); + self.saved.push(Node { + head: kept1.len() as u32, + level: kept2.len() as u32, + a: NONE, + b: 0, + }); + for at in kept1.clone().chain(kept2.clone()) { + let ty = self.extras.row_scratch[at].ty; + self.saved.push(Node { head: ty.0, level: 0, a: NONE, b: 0 }); + } let end = Frame { kind: FrameKind::RowFieldEnd, ..frame }; if self.frames.last() != Some(&end) { self.frames.push(end); From d4372fa0f8cafeb4f79404d581b423a6f1931b7d Mon Sep 17 00:00:00 2001 From: "open-session-submodule[bot]" <341020755+open-session-submodule[bot]@users.noreply.github.com> Date: Tue, 17 Nov 2026 22:33:00 +0000 Subject: [PATCH 3/8] Checker rewrite R3: the harness of pattern mode and what the old checker does in it (decisions 61 and 62) A plan bit PATTERN: programs with two locally abstract types of two newtype levels, an abstract type with a parameter and a data type; Op::Gadt for one pattern unification with the two flags of set_mode_pattern; both sides print the equations in force. The skeleton of R3h's entry: Checker::unify_pattern (unify_gadt calls it), the equations log, the supplier of declarations LocalDecls, the LOCAL flag with TypeTable::instantiable. Bodies are R3h's. Running tests record what the old checker does: which pairs get an equation and in which direction, what reify makes, what a failure leaves, rows that clash and are compatible. It links two structures a pattern unification unifies, where OCaml records an assumed equality: old_judges cuts a case before such an operation. Co-authored-by: Jaap Frolich Session: https://os.submodule.nl/session/bks-9da31a3c-0d3b-7546-bc8e-f1fd3a9270ee Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com> --- compiler-rust/docs/CHECKER_REWRITE.md | 138 +++++ compiler-rust/src/tc/differential.rs | 836 +++++++++++++++++++++++++- compiler-rust/src/tc/expand.rs | 43 ++ compiler-rust/src/tc/rows.rs | 54 ++ compiler-rust/src/tc/unify.rs | 48 +- 5 files changed, 1098 insertions(+), 21 deletions(-) diff --git a/compiler-rust/docs/CHECKER_REWRITE.md b/compiler-rust/docs/CHECKER_REWRITE.md index 6fefdac626..0f35f6c510 100644 --- a/compiler-rust/docs/CHECKER_REWRITE.md +++ b/compiler-rust/docs/CHECKER_REWRITE.md @@ -4490,6 +4490,144 @@ the reviews above left owed; every test named runs today. type has a scope, which no generated program has. Open question 4 of this turn. +61. **The harness of pattern mode: the plan bit `PATTERN`.** + `differential::PATTERN` (bit 9 of a plan) makes a case a case of + pattern mode. What the bit adds: + - *the program*: `type n0`, `type n1`, `type k<'a>` and + `type d<'a> = D('a)`, and after the generated values four values + whose parameters are the pairs R3h is about + (`differential::PATTERN_VALUES`): a locally abstract type against a + structure and against the other one; constructors of one path with + arguments, abstract (`k`) and a data type (`d`), over a locally + abstract type, over `int` and over a variable; arrows over them. + With `VARIANTS` two more (`PATTERN_ROWS`): rows that clash and are + not compatible (a tag the other closed row lacks, a constant tag + against one with an argument) and rows that unify under an + equation; a row that clashes and is compatible comes from `reify` + (a fixed row against an open row with one more tag). There is one + data type, so two different data types are never compared and the + supplier of declarations answers `Clash` (8.9); + - *both sides*: `n0` and `n1` are locally abstract types of two + newtype levels, the level the case starts at and one above it. The + old side binds them again as local identifiers whose declarations + have `type_newtype_level`; the new side sets `NEWTYPE` and a + `Newtype` row on their `TypeTable` rows. The old side sees the unit + from outside, as `Case`: its types are paths through a module and + only the predefined types are bare identifiers + (`in_current_module`). The new side says the same with the `LOCAL` + flag, set on the predefined types only; + - *the operation* `Op::Gadt(left, right, flags)`: one top-level + unification in pattern mode with the current level as the newtype + level; bit 0 of `flags` is `generate_equations`, bit 1 + `assume_injective`. `unify_gadt` itself is 3; 1, 0 and 2 are the + modes unify3 installs below it (ctype.ml:2352, 2368) and that no + top-level call of OCaml asks for: `Checker::unify_pattern` takes + the two flags so that a case reaches each. Half of the pattern + unifications of a generated case are over two parameters of a new + instance of a pattern value, the rest over any two operands; + - *the form*: both sides end it with the equations in force, one + line a type that has one, sorted by the type's name: `= name := + body`, the bodies numbered together. The old side prints the + manifests of the local constraints of the environment the last + successful pattern unification left; the new side instantiates the + manifest of every entry of `Checker::equations` at the generic + level. A type `reify` made prints by its name (`$0`, `$1`), which + the new side takes from the supplier of the harness + (`differential::FreshNames`, the rule of `get_new_abstract_name`, + ctype.ml:908-922). + - *the deliberately wrong side* `Fault::ExpressionMode`: a pattern + unification runs as `Checker::unify`. + + Running today: `the_two_sides_agree_on_pattern_programs_without_a_ + pattern_unification` (the self-test: the fixed program and 300 + generated cases with the pattern unifications taken out agree), + `old_side_of_the_pattern_plan_is_not_vacuous` (observed, 1,200 + cases: 950 pattern unifications that succeed and 493 that fail + among the judged operations; 130 cases end with an equation, 3 with + two, 183 with a reified type) and + `generated_cases_find_a_pattern_unification_in_expression_mode` (the + wrong side needs nothing of R3h: of 228 cases with a pattern + unification it is reported in 21 that have an equation on the old + side and in 58 where only it fails). + +62. **What the old checker does in pattern mode, and where it is a + judge.** Observed on `differential::tests::PATTERN_PROGRAM`; each + item names its test in tc/differential.rs. + - *Which pairs get an equation, and its direction* (a judge): + `old_checker_gives_the_equation_to_the_newer_locally_abstract_type`. + A locally abstract type against a type that is not one gets the + equation on either side (ctype.ml:2385-2392) and the two nodes + stay two nodes. Of two locally abstract types the one of the + higher newtype level gets it (ctype.ml:2373-2384). Without + `generate_equations`: none, and a success (`reify`, no `mcomp`). + `int` against a tuple with equations: a clash (`mcomp`). After + `n0 := int`, `n1` against `n0` gives `n1 := int`: the body is the + expanded head. + - *The arguments of one path* (a judge): + `old_checker_refines_the_arguments_of_a_constructor_by_its_ + injectivity`. With `assume_injective` the arguments are unified + with equations; below that an abstract type that is not of the + unit gets none and what does not unify is reified, and a data type + gets them. A later expression unification of two `k` nodes that + pattern mode assumed equal fails: nothing of the assumption is + left after the top-level call. + - *What `reify` makes* (a judge, also of the names of variables + without one): `old_checker_reifies_a_variable_and_a_row_variable`. + A variable at the newtype level is a link to a new abstract type + `$0` at the variable's level; a row variable is a link to a fixed + row without tags over `$0`; the second is `$1`. A variable with a + name (`$'a`) is not reachable from an operation: tested by hand. + - *What a failure leaves* (a judge of the graph): + `old_checker_keeps_what_reify_linked_when_a_pattern_unification_ + fails`. `reify` links before it refuses a variable below the + newtype level, and the row a clash reified stays fixed when + `mcomp` refuses the pair: unify3's handler puts back only the left + node's description (ctype.ml:2437-2439). A failure after an + equation was added leaves the graph as it was. The caller that + goes on restores a snapshot, the levels and the environment it + kept (`set_state`, compiler/ml/typecore.ml:1299-1305); the kernel's + caller restores a mark and cuts the equations + (`Checker::cut_equations`, 8.9). The harness drops the old side's + environment of a failed pattern unification, so it does not judge + the equations a failed one added. + - *Rows* (a judge): + `old_checker_accepts_rows_that_clash_and_are_compatible`. + - *Two structures unified in pattern mode are ONE node on the old + side* (NOT a judge): + `old_checker_links_the_structures_a_pattern_unification_unifies`. + OCaml records the pair as assumed equal and links neither + (`Pattern -> add_type_equality t1' t2'`, ctype.ml:2327-2331). The + old checker links them: two tuples of two instances are one node + afterwards, and of `[#A(n0) | #B]` against `[#A(int) | #B]` the + left row prints `int`: a type equal only under the equations of + one case has become the other for good. Three source programs + were tried (a constructor whose result names a tuple, the same + through a tuple pattern, the same through an abbreviation) and + both compilers accept all three: no source program is known that + shows it. `differential::old_judges` gives the number of + operations of a case before the first pattern unification that + succeeds and leaves a structure reachable from an operand a link; + `compare` runs both sides on that prefix under `PATTERN` (observed: + 64 of 1,200 cases are cut). So the generated cases judge failures + of structure pairs in pattern mode and never their success; the + hand-built tests of 8.9 do, from the OCaml lines. + - *Ambiguity is not checked on the old side* (NOT a judge, and not + in R3h): OCaml refuses + `type rec t<_> = Int: t` / + `let f = (type a, x: t, y: a) => switch x { | Int => if true { y } else { 0 } }` + with "This instance of int is ambiguous: it would escape the scope + of its equation" (`trace_gadt_instances`, `Env.gadt_instance_level`, + `add_gadt_instance_chain`, ctype.ml:176-201, 1220-1227, 2289-2298; + the report is typecore's). The Rust compiler accepts it (observed + with `compile_source_in_process`): the old checker has the tracing + flag and no instance levels. The kernel has neither after R3h; it + is a slice of its own after R3h (open question 1 of this turn). + - Not observed to differ, and not judged: the arm of `unify` for two + locally abstract types under local constraints, which expands the + newer one once (ctype.ml:2258-2270), is missing on the old side; + it expands both heads, and the equations that result were the + same on every case tried. + ### 8.7 The hot paths of unification (K2) diff --git a/compiler-rust/src/tc/differential.rs b/compiler-rust/src/tc/differential.rs index 15a84fb611..a006351e5a 100644 --- a/compiler-rust/src/tc/differential.rs +++ b/compiler-rust/src/tc/differential.rs @@ -184,6 +184,15 @@ pub(crate) const REFS: u16 = 1 << 7; /// two types whose binders pair by position and not by name print /// different names at one position and the same back references. pub(crate) const UNIVARS: u16 = 1 << 8; +/// Pattern mode (slice R3h; section 8.6, decision 61). The program +/// declares two abstract types `n0` and `n1`, which both sides make +/// locally abstract types of two newtype levels (the level the case +/// starts at, and one above it), an abstract type with a parameter +/// `k<'a>` (not injective) and a data type `d<'a>` (injective); its +/// values have types over them. The operations get [`Op::Gadt`]. Both +/// sides end their form with the equations in force, one line a type +/// that has one, sorted by name: `= name := body`. +pub(crate) const PATTERN: u16 = 1 << 9; /// A tier and what a case adds to it. 4 bytes, passed by value. #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -270,6 +279,14 @@ pub(crate) enum Op { /// 2424) and the old checker unifies its body (section 8.6, decision /// 49), and no generated case is to meet that. [`UNIVARS`]. Inside(u16, u8), + /// Unify two operands in pattern mode at the current level as the + /// newtype level (`unify_gadt`, compiler/ml/ctype.ml:2739-2750; + /// `Checker::unify_pattern`): bit 0 of the third field is + /// `generate_equations`, bit 1 `assume_injective` (`unify_gadt` + /// itself is 3; unify3 installs 1 and 0 for the arguments of a + /// constructor, ctype.ml:2352, 2368). The equations it adds stay in + /// force for the rest of the case. [`PATTERN`]. + Gadt(u16, u16, u8), } /// A deliberately wrong new side, for the tests that prove the harness sees @@ -305,6 +322,9 @@ pub(crate) enum Fault { /// polymorphic type in the other order: what pairing by name instead /// of by position looks like in the form. [`UNIVARS`]. SwappedBinders, + /// [`Op::Gadt`] unifies in expression mode: no equation, no + /// reification, a link where pattern mode assumes. [`PATTERN`]. + ExpressionMode, } /// What one side reports for one case. @@ -554,17 +574,57 @@ pub(crate) fn program_source(rng: &mut Rng, plan: impl Into) -> String { nullary.push("rv".to_string()); unary.push("pv".to_string()); } + if plan.has(PATTERN) { + source.push_str(PATTERN_TYPES); + for name in NEWTYPES { + nullary.push(name.to_string()); + } + unary.push("k".to_string()); + unary.push("d".to_string()); + } for index in 0..3 + rng.below(4) { let mut body = String::new(); type_text_with(rng, plan.with, 3, &["'a", "'b"], &nullary, &unary, &mut body); let _ = writeln!(source, "external v{index}: {body} = \"v{index}\""); } + if plan.has(PATTERN) { + source.push_str(PATTERN_VALUES); + if plan.has(VARIANTS) { + source.push_str(PATTERN_ROWS); + } + } source.push_str("let dup = x => (x, x)\n"); let last = nullary.last().unwrap(); let _ = writeln!(source, "let share = (x: {last}) => (x, x)"); source } +/// The predefined types a generated program names. +const PREDEFINED: [&str; 6] = ["int", "string", "unit", "float", "bool", "array"]; + +/// The two types a [`PATTERN`] case makes locally abstract. +const NEWTYPES: [&str; 2] = ["n0", "n1"]; + +/// The declarations of a [`PATTERN`] program: the two locally abstract +/// types, an abstract type with a parameter and a data type with one. +const PATTERN_TYPES: &str = "type n0\ntype n1\ntype k<'a>\ntype d<'a> = D('a)\n"; + +/// The values of a [`PATTERN`] program, after the generated ones: a +/// locally abstract type against a structure and against the other one, +/// constructors of one path with arguments (abstract: `k`; a data type: +/// `d`), and a function over both. +const PATTERN_VALUES: &str = "external q0: (n0, n1, int, (int, t0)) => unit = \"q0\"\n\ + external q1: (k, k, k, k<'a>) => unit = \"q1\"\n\ + external q2: (d, d, d, d<'a>) => unit = \"q2\"\n\ + external q3: (n0 => n1, int => t0, (n1, n0) => 'a) => unit = \"q3\"\n"; + +/// With [`VARIANTS`]: rows that clash and are compatible (`#A` over a +/// locally abstract type against `#A` over `int`, a tag the other closed +/// row lacks), and rows that clash and are not (a constant tag against +/// one with an argument). +const PATTERN_ROWS: &str = "external q4: ([#A(n0) | #B], [#A(int) | #B], [#A(int) | #C], [#A | #B]) => unit = \"q4\"\n\ + external q5: ([> #A(n0)], [< #A(int) | #B], [#C(n1)]) => unit = \"q5\"\n"; + /// O(the program): compile with the old checker. Panics with the source when /// the generator wrote a program the old checker rejects. pub(crate) fn compile(source: String) -> Program { @@ -646,6 +706,31 @@ pub(crate) fn operations(rng: &mut Rng, plan: impl Into, values: usize) -> } _ => Op::Inside(operand(rng), rng.below(3) as u8), } + } else if plan.has(PATTERN) && rng.below(3) == 0 { + // [`PATTERN`]: a pattern unification, twice in three as + // `unify_gadt` runs it, or a part of an operand to unify. + // Half of them are over an instance of one of the pattern + // values ([`PATTERN_VALUES`], before `dup` and `share`): + // one of its parameters against another, or against any + // operand; `follow` is last first. + match rng.below(6) { + 0..=2 => { + let count = if plan.has(VARIANTS) { 6 } else { 4 }; + let value = values - 2 - count + rng.below(count); + let instance = operands as u16; + let flags = if rng.below(4) == 0 { rng.below(3) as u8 } else { 3 }; + let against = if rng.below(3) == 0 { operand(rng) } else { instance + 2 }; + follow = vec![ + Op::Gadt(instance + 1, against, flags), + Op::Child(instance, rng.below(4) as u8), + Op::Child(instance, rng.below(4) as u8), + ]; + Op::Instance(value as u16) + } + 3 => Op::Gadt(operand(rng), operand(rng), 3), + 4 => Op::Gadt(operand(rng), operand(rng), rng.below(3) as u8), + _ => Op::Child(operand(rng), rng.below(4) as u8), + } } else if plan.has(BUILD) && rng.below(4) == 0 { match rng.below(4) { 0 => Op::Var, @@ -715,7 +800,7 @@ pub(crate) fn operations(rng: &mut Rng, plan: impl Into, values: usize) -> /// The old checker's types of a case: the context, the environment the /// unit's declarations are in, the schemes of its values and the level the /// case starts at. -struct Old { +pub(crate) struct Old { ctx: TypeContext, env: Env, values: Vec, @@ -745,6 +830,7 @@ fn old_context(program: &Program, plan: Plan) -> Old { let mut env: Env = initial_env(&ctx); let mut values = Vec::new(); let mut late = Vec::new(); + let mut newtypes = Vec::new(); for item in unit.ps_sig.iter() { match item { SignatureItem::SigValue(_, value) => values.push(value.val_type), @@ -752,6 +838,9 @@ fn old_context(program: &Program, plan: Plan) -> Old { if plan.has(SCOPED) && LATE.contains(&id.name()) { late.push((id.name().to_string(), declaration.clone())); } + if plan.has(PATTERN) && NEWTYPES.contains(&id.name()) { + newtypes.push((id.name().to_string(), declaration.clone())); + } env.add_type(*id, declaration.clone()); } _ => {} @@ -772,15 +861,32 @@ fn old_context(program: &Program, plan: Plan) -> Old { // 1`, `+ 2`, `+ 3`. A type of the unit is named through the persistent // unit and has binding time 0, so every node that names a late type is // renamed to the local identifier, in the schemes and in the manifests. + let mut renamed = Vec::new(); if plan.has(SCOPED) { assert_eq!(late.len(), LATE.len(), "{}", program.source); - let mut renamed = Vec::new(); for (offset, (name, declaration)) in late.iter().enumerate() { let id = ctx.create_ident(name.as_str()); assert_eq!(id.stamp(), base + 1 + offset as i32, "late stamp"); env.add_type(id, declaration.clone()); renamed.push((name.clone(), id)); } + } + // [`PATTERN`]: `n0` and `n1` are locally abstract types, as `(type + // a)` declares one (`type_newtype_level = Some (level, level)`, + // compiler/ml/typecore.ml, `Pexp_newtype`): local identifiers whose + // declarations have the newtype levels `base` and `base + 1`. + if plan.has(PATTERN) { + assert_eq!(newtypes.len(), NEWTYPES.len(), "{}", program.source); + for (offset, (name, declaration)) in newtypes.iter().enumerate() { + let id = ctx.create_ident(name.as_str()); + let level = base + offset as i32; + let mut declaration = (**declaration).clone(); + declaration.type_newtype_level = Some((level, level)); + env.add_type(id, declaration); + renamed.push((name.clone(), id)); + } + } + if !renamed.is_empty() { let mut seen = crate::fast_hash::AHashSet::default(); let mut work: Vec = values.clone(); for item in unit.ps_sig.iter() { @@ -908,9 +1014,57 @@ fn record_failure(outcome: &mut Outcome, index: usize, before: &str, after: (Str /// O(operations + printed nodes²): the old checker's side of a case. pub(crate) fn run_old(program: &Program, plan: impl Into, ops: &[Op]) -> Outcome { - let plan = plan.into(); + run_old_until(program, plan.into(), ops, &mut None) +} + +/// How many of `ops` the old checker is a judge of under [`PATTERN`] +/// (section 8.6, decision 62): the operations before the first pattern +/// unification that succeeds and leaves a structure of an operand a link. +/// In pattern mode OCaml records that the two structures are assumed +/// equal and links neither (`Pattern -> add_type_equality t1' t2'`, +/// compiler/ml/ctype.ml:2327-2331); the old checker links them, so its +/// graph after such an operation is not OCaml's. [`compare`] runs both +/// sides on this prefix. O(the case × operands). +pub(crate) fn old_judges(program: &Program, plan: Plan, ops: &[Op]) -> usize { + let mut cut = Some(ops.len()); + run_old_until(program, plan, ops, &mut cut); + cut.unwrap() +} + +/// O(nodes): the structures `ty` reaches: every node that is not a +/// variable, a universal variable or a constructor (those OCaml links in +/// pattern mode as well, ctype.ml:2247, 2250-2253, 2315-2324). +fn old_structures( + ctx: &TypeContext, + ty: TypeExprRef, + seen: &mut std::collections::HashSet, + out: &mut Vec, +) { + let ty = ctx.repr(ty); + if !seen.insert(ty) { + return; + } + if !matches!( + *ctx.get_desc(ty), + TypeDesc::Tvar { .. } | TypeDesc::Tunivar { .. } | TypeDesc::Tconstr { .. } + ) { + out.push(ty); + } + let mut children = Vec::new(); + crate::types::btype::iter_type_expr(ctx, ty, |child| children.push(child)); + for child in children { + old_structures(ctx, child, seen, out); + } +} + +/// [`run_old`]; with `cut` set, stops at the first operation the old +/// checker is not a judge of ([`old_judges`]) and writes its index. +fn run_old_until(program: &Program, plan: Plan, ops: &[Op], cut: &mut Option) -> Outcome { let old = old_context(program, plan); - let (ctx, env, base) = (&old.ctx, &old.env, old.base); + let (ctx, base) = (&old.ctx, old.base); + // The environment of the case: a pattern unification that succeeds + // leaves its equations and its reified types in it. + let mut current = old.env.clone(); let explain = plan.has(EXPLAIN); let mut outcome = Outcome { failed: None, @@ -921,10 +1075,14 @@ pub(crate) fn run_old(program: &Program, plan: impl Into, ops: &[Op]) -> O let mut operands: Vec = Vec::new(); for (index, op) in ops.iter().enumerate() { let before = match op { - Op::Unify(..) | Op::Lower(..) if explain => old_forms(&old, plan, &operands, &[]).0, + Op::Unify(..) | Op::Lower(..) | Op::Gadt(..) if explain => { + old_forms(&old, plan, &operands, &[]).0 + } _ => String::new(), }; let mut failure = None; + let env = ¤t; + let mut refined = None; match *op { Op::Instance(value) => operands.push(ctype::instance(ctx, old.values[value as usize])), Op::Copy(operand) => operands.push(ctype::instance(ctx, operands[operand as usize])), @@ -998,6 +1156,35 @@ pub(crate) fn run_old(program: &Program, plan: impl Into, ops: &[Op]) -> O } operands.push(ty); } + Op::Gadt(left, right, flags) => { + let mut structures = Vec::new(); + if cut.is_some() { + let mut seen = std::collections::HashSet::new(); + for &operand in &operands { + old_structures(ctx, operand, &mut seen, &mut structures); + } + } + let (result, after) = old_gadt( + ctx, + env, + flags, + operands[left as usize], + operands[right as usize], + ); + match result { + Ok(()) => { + if structures.iter().any(|&node| ctx.repr(node) != node) { + *cut = Some(index); + return outcome; + } + refined = Some(after); + } + Err(trace) => failure = Some(trace), + } + } + } + if let Some(env) = refined { + current = env; } if let Some(trace) = failure { if !explain { @@ -1008,9 +1195,64 @@ pub(crate) fn run_old(program: &Program, plan: impl Into, ops: &[Op]) -> O } } outcome.form.push_str(&old_forms(&old, plan, &operands, &[]).0); + if plan.has(PATTERN) { + outcome.form.push_str(&old_equations(&old, ¤t)); + } outcome } +/// One pattern unification on the old checker, as `unify_pattern_types_gadt` +/// runs one (types/typecore.rs): a state in pattern mode with the two +/// flags of `flags` ([`Op::Gadt`]) and the current level as the newtype +/// level. Returns the result and the environment the state ends with: +/// the equations and the reified types of a success. +pub(crate) fn old_gadt( + ctx: &TypeContext, + env: &Env, + flags: u8, + left: TypeExprRef, + right: TypeExprRef, +) -> (Result<(), Vec<(TypeExprRef, TypeExprRef)>>, Env) { + let mut state = ctype::UnifyState::new(); + state.set_env(env); + state.mode = crate::types::UnifyMode::Pattern; + state.generate_equations = flags & 1 != 0; + state.assume_injective = flags & 2 != 0; + state.newtype_level = Some(ctx.current_level()); + let result = match ctype::unify(ctx, &mut state, left, right) { + Ok(()) => Ok(()), + Err(ctype::TypeError::Unify(trace)) => Err(trace), + Err(_) => Err(Vec::new()), + }; + let after = state.env().cloned().unwrap_or_else(|| env.clone()); + (result, after) +} + +/// O(equations × printed nodes): the equations of `env`, one line a +/// type that has a local constraint, sorted by the type's name: `= name +/// := body`, the bodies numbered together (a body is generic: `[G]`). +pub(crate) fn old_equations(old: &Old, env: &Env) -> String { + let mut equations: Vec<(String, TypeExprRef)> = env + .iter_local_constraints() + .filter_map(|(path, declaration)| { + declaration.type_manifest.map(|body| (path.last().to_string(), body)) + }) + .collect(); + equations.sort_by(|left, right| left.0.cmp(&right.0)); + let mut out = String::new(); + let mut seen = OldSeen { + nodes: Vec::new(), + cells: None, + refs: None, + }; + for (name, body) in equations { + let _ = write!(out, "= {name} := "); + old_form(&old.ctx, old.base, true, body, &mut seen, &mut out); + out.push('\n'); + } + out +} + /// O(nodes reachable × nodes seen): print one old node; see the module text. fn old_form( ctx: &TypeContext, @@ -1211,6 +1453,48 @@ pub(crate) struct New { /// unopened node, the node and the field (bit 63 set): a reference /// nothing else holds. refs: Vec, + /// [`PATTERN`]: the names of the locally abstract types `reify` made, + /// by type index from the segment's last path on ([`FreshNames`]). + fresh: std::rc::Rc>>, +} + +/// The supplier of a [`PATTERN`] case (`rows::LocalDecls`): a fresh +/// abstract type gets the next type index after the segment's paths and +/// the name `get_new_abstract_name` gives it (compiler/ml/ctype.ml: +/// 908-922): `$'a`, then `$'a1`; `$0`, `$1` for a variable without a +/// name. The program has one data type, so two different data types are +/// never compared: `Clash`. +struct FreshNames { + first: u32, + strings: Vec, + names: std::rc::Rc>>, +} + +impl super::rows::LocalDecls for FreshNames { + fn fresh_abstract(&mut self, name: u32) -> u32 { + let prefix = match self.strings.get(name as usize) { + Some(name) if name != "" => format!("$'{name}"), + _ => "$".to_string(), + }; + let mut names = self.names.borrow_mut(); + let spelled = |index: usize| { + if index == 0 && !prefix.ends_with('$') { + prefix.clone() + } else { + format!("{prefix}{index}") + } + }; + let mut index = 0; + while names.contains(&spelled(index)) { + index += 1; + } + names.push(spelled(index)); + self.first + names.len() as u32 - 1 + } + + fn compare(&mut self, _: &mut Checker, _: u32, _: u32) -> super::rows::DeclMatch { + super::rows::DeclMatch::Clash + } } impl New { @@ -1272,6 +1556,7 @@ impl New { first: vec![None; roots.len()], cells: Vec::new(), refs: Vec::new(), + fresh: Default::default(), open, segment, checker, @@ -1283,9 +1568,100 @@ impl New { if plan.has(SCOPED) { new.scope_late_types(); } + if plan.has(PATTERN) { + new.declare_newtypes(); + } new } + /// O(paths): [`PATTERN`]: `n0` and `n1` are locally abstract types of + /// the newtype levels 1 and 2 (a graph starts at level 1), on every + /// `paths` row that names one, with the scope `get_level` gives a + /// newtype (its first level, compiler/ml/ctype.ml:562-569); and the + /// supplier of fresh abstract types is installed. + fn declare_newtypes(&mut self) { + use super::expand::{NEWTYPE, Newtype}; + for index in 0..self.segment.signature.paths.len() { + let name = match self.segment.signature.paths[index] { + PathRow::Ident(ident) => self.segment.signature.identifiers[ident as usize].name, + PathRow::Dot { name, .. } => name, + PathRow::Apply { .. } => continue, + }; + // The old side sees the unit from outside, as `Case`: its + // types are paths through a module there and only the + // predefined types are bare identifiers (`LOCAL`). + if PREDEFINED.contains(&self.string(name)) { + self.checker.types.set_local(index as u32); + } + let Some(offset) = NEWTYPES.iter().position(|newtype| *newtype == self.string(name)) + else { + continue; + }; + let level = 1 + offset as u32; + let types = &mut self.checker.types; + types.newtypes.push(Newtype { level, expansion_level: level }); + let row = &mut types.rows[index]; + row.flags |= NEWTYPE; + row.newtype = types.newtypes.len() as u32 - 1; + if level > 1 { + if row.scope == 0 { + types.scoped += 1; + } + row.scope = level; + } + } + let pools = &self.segment.signature; + let strings = (0..pools.strings.len() as u32).map(|at| self.string(at).to_string()).collect(); + self.checker.locals = Some(Box::new(FreshNames { + first: pools.paths.len() as u32, + strings, + names: self.fresh.clone(), + })); + } + + /// O(equations × printed nodes): [`PATTERN`]: the equations in force + /// (`Checker::equations`), as [`old_equations`] prints them: one line + /// a type, sorted by name, the body an instance of the equation's + /// manifest made at the generic level, the bodies numbered together. + fn equations(&mut self) -> String { + let mut equations: Vec<(String, Ty)> = Vec::new(); + for at in 0..self.checker.equations.len() { + let index = self.checker.equations[at].type_index; + let info = self.checker.types.info(index); + let manifest = self.checker.types.manifests[info.manifest as usize]; + let base = self.checker.g.nodes.len() as u32; + let level = std::mem::replace(&mut self.checker.g.level, GENERIC); + self.checker.g.instance(manifest.template); + self.checker.g.level = level; + let body = if manifest.body.0 & RELATIVE != 0 { + Ty(base + (manifest.body.0 & !RELATIVE)) + } else { + manifest.body + }; + equations.push((self.type_name(index), body)); + } + equations.sort_by(|left, right| left.0.cmp(&right.0)); + let mut seen = vec![NONE; self.checker.g.nodes.len()]; + let mut count = 0; + let mut out = String::new(); + for (name, body) in equations { + let _ = write!(out, "= {name} := "); + self.node_form(body, &mut seen, &mut count, &mut out); + out.push('\n'); + } + out + } + + /// O(1): the name a type index prints with: the last component of its + /// path, or the name of a fresh abstract type. + fn type_name(&self, index: u32) -> String { + let paths = self.segment.signature.paths.len() as u32; + if index >= paths { + return self.fresh.borrow()[(index - paths) as usize].clone(); + } + self.path_name(index).to_string() + } + /// O(paths): give the late types their scopes: a graph starts at level /// 1, so `s0`, `s1`, `s2` are bound at levels 2, 3 and 4. Every `paths` /// row whose last name is a late type's is one (the unit's own @@ -1611,6 +1987,17 @@ impl New { } self.operands.push(ty); } + Op::Gadt(left, right, flags) => { + let (left, right) = (self.operands[left as usize], self.operands[right as usize]); + if fault == Fault::ExpressionMode { + return self.checker.unify(left, right).map_err(|_| ()); + } + let level = self.checker.g.level; + return self + .checker + .unify_pattern(level, flags & 1 != 0, flags & 2 != 0, left, right) + .map_err(|_| ()); + } } Ok(()) } @@ -1678,16 +2065,7 @@ impl New { } } CONSTR if node.head & super::segment::EXTENDED == 0 => { - let name = match self.segment.signature.paths[node.a as usize] { - PathRow::Ident(ident) => self.segment.signature.identifiers[ident as usize].name, - PathRow::Dot { name, .. } => name, - PathRow::Apply { .. } => NONE, - }; - if name == NONE { - out.push_str("?apply"); - } else { - out.push_str(self.string(name)); - } + out.push_str(&self.type_name(node.a)); if node.arity() != 0 { out.push('<'); for index in 0..node.arity() { @@ -1928,7 +2306,7 @@ pub(crate) fn run_new(program: &Program, plan: impl Into, ops: &[Op], faul program.source ); let before = match op { - Op::Unify(..) | Op::Lower(..) if explain => new.form(), + Op::Unify(..) | Op::Lower(..) | Op::Gadt(..) if explain => new.form(), _ => String::new(), }; if new.apply(op, fault).is_err() { @@ -1937,7 +2315,7 @@ pub(crate) fn run_new(program: &Program, plan: impl Into, ops: &[Op], faul } // A failed lowering has no trace on either side (`run_old`). let mut trace = match op { - Op::Unify(..) => new.checker.trace.clone(), + Op::Unify(..) | Op::Gadt(..) => new.checker.trace.clone(), _ => Vec::new(), }; match fault { @@ -1952,6 +2330,9 @@ pub(crate) fn run_new(program: &Program, plan: impl Into, ops: &[Op], faul } } outcome.form.push_str(&new.form()); + if plan.has(PATTERN) { + outcome.form.push_str(&new.equations()); + } outcome } @@ -1965,6 +2346,11 @@ pub(crate) fn compare( fault: Fault, ) -> Result { let plan = plan.into(); + // [`PATTERN`]: only the operations the old checker is a judge of. + let ops = match plan.has(PATTERN) { + true => &ops[..old_judges(program, plan, ops)], + false => ops, + }; let old = run_old(program, plan, ops); let new = run_new(program, plan, ops, fault); if old == new { @@ -3160,7 +3546,8 @@ mod tests { /// R3x. #[test] fn ignored_tests_name_the_slices_they_reach() { - let before: [(&str, &[&str]); 10] = [ + let before: [(&str, &[&str]); 11] = [ + ("R3q", &["R3m", "R3e", "R3f", "R3n", "R3g"]), ("R3p", &["R3m", "R3e"]), ("R3k", &[]), ("R3m", &[]), @@ -3259,7 +3646,7 @@ mod tests { let full = body.contains("Tier::Full"); let rows = body.contains("OBJECTS") || body.contains("VARIANTS"); for (needle, reached) in - [("OBJECTS", "R3e"), ("VARIANTS", "R3f"), ("UNIVAR", "R3g")] + [("OBJECTS", "R3e"), ("VARIANTS", "R3f"), ("UNIVAR", "R3g"), ("PATTERN", "R3h")] { if full && body.contains(needle) { reaches.push(reached); @@ -4147,6 +4534,417 @@ mod tests { ); } + /// A program for the fixed cases of pattern mode ([`PATTERN`]): the + /// two locally abstract types `n0` (the older) and `n1`, an abstract + /// type with a parameter `k`, a data type `d`, and values whose + /// parameters are the types a case unifies: `p0` nullary types and a + /// tuple, `p1` and `p2` constructors of one path, `p3` variables, + /// `p4` closed rows, `p5` open rows and a row with an upper bound, + /// `p6` tuples and nested constructors. + const PATTERN_PROGRAM: &str = "type n0\ntype n1\ntype k<'a>\ntype d<'a> = D('a)\n\ + external p0: (n0, n1, int, (int, int)) => unit = \"p0\"\n\ + external p1: (k, k, k, k<'a>) => unit = \"p1\"\n\ + external p2: (d, d, d, d<'a>) => unit = \"p2\"\n\ + external p3: ('a, 'b, ('a, int)) => unit = \"p3\"\n\ + external p4: ([#A(n0) | #B], [#A(int) | #B], [#A(int) | #C], [#A | #B]) => unit = \"p4\"\n\ + external p5: ([> #A(n0)], [< #A(int) | #B], [#C(n1)], [> #A(int)], [> #A(int) | #B]) => unit = \"p5\"\n\ + external p6: ((n0, int), (int, (int, int)), k>, k>) => unit = \"p6\"\n"; + + /// The plan of the fixed cases of pattern mode. + const PATTERN_PLAN: u16 = OBJECTS | VARIANTS | REFS | BUILD | PATTERN; + + /// One definition, an instance of value `value` (operand 0), its + /// parameters `left` and `right` (operands 1 and 2), unified in + /// pattern mode with `flags` ([`Op::Gadt`]). + fn pattern(value: u16, left: u8, right: u8, flags: u8) -> Vec { + vec![ + Op::BeginDef, + Op::Instance(value), + Op::Child(0, left), + Op::Child(0, right), + Op::Gadt(1, 2, flags), + ] + } + + /// One definition, the parameter `structure` of an instance of value + /// `value` (operand 1) and `n0` (operand 3), unified in pattern mode + /// as `unify_gadt` does, `n0` on the left. + fn refine(value: u16, structure: u8) -> Vec { + vec![ + Op::BeginDef, + Op::Instance(value), + Op::Child(0, structure), + Op::Instance(0), + Op::Child(2, 0), + Op::Gadt(3, 1, 3), + ] + } + + /// O(text): the equations a form ends with. + fn equations(form: &str) -> Vec<&str> { + form.lines().filter(|line| line.starts_with("= ")).collect() + } + + /// The old checker on a fixed pattern case: the outcome, and that it + /// is a judge of every operation ([`old_judges`]). + fn old_pattern(ops: &[Op]) -> Outcome { + let program = compile(PATTERN_PROGRAM.to_string()); + let plan = Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN); + assert_eq!(old_judges(&program, plan, ops), ops.len(), "{ops:?}"); + run_old(&program, plan, ops) + } + + /// Which pairs get an equation, and in which direction (observed; + /// section 8.6, decision 62; the old checker is a judge of all of + /// it). A locally abstract type against a type that is not one gets + /// the equation whichever side it is on (`Tconstr (path, [], _), _ + /// when is_instantiable ...` and its mirror, compiler/ml/ctype.ml: + /// 2385-2392), and the two nodes stay two nodes. Two locally abstract + /// types: the one of the higher newtype level gets it (`if + /// find_newtype_level !env path > find_newtype_level !env path' then + /// (path, t2') else (path', t1')`, ctype.ml:2373-2384): `n1`, on + /// either side. Without `generate_equations` there is none and the + /// unification succeeds (`reify` and no `mcomp`, ctype.ml:2393-2397). + /// `int` against a tuple with equations fails: `mcomp`. A second + /// equation is for the expansion: `n1` against `n0` after `n0 := int` + /// gives `n1 := int` (`unify2` expands both, ctype.ml:2276-2280). + #[test] + fn old_checker_gives_the_equation_to_the_newer_locally_abstract_type() { + let left = old_pattern(&pattern(0, 0, 2, 3)); + assert_eq!(left.failed, None); + assert_eq!( + left.form, + "0 = &0[1]fn(&1[1]n0, &2[1]n1, &3[1]int, &4[1](&5[1]int, &6[1]int)) => &7[1]unit\n1 = @1\n2 = @3\n= n0 := &0[G]int\n" + ); + assert_eq!(equations(&old_pattern(&pattern(0, 2, 0, 3)).form), ["= n0 := &0[G]int"]); + assert_eq!(equations(&old_pattern(&pattern(0, 0, 1, 3)).form), ["= n1 := &0[G]n0"]); + assert_eq!(equations(&old_pattern(&pattern(0, 1, 0, 3)).form), ["= n1 := &0[G]n0"]); + assert_eq!( + equations(&old_pattern(&pattern(0, 0, 3, 3)).form), + ["= n0 := &0[G](&1[G]int, &2[G]int)"] + ); + let silent = old_pattern(&pattern(0, 0, 2, 2)); + assert_eq!((silent.failed, equations(&silent.form).len()), (None, 0)); + assert_eq!(old_pattern(&pattern(0, 2, 3, 3)).failed, Some(4)); + let mut chain = pattern(0, 0, 2, 3); + chain.extend([Op::Child(0, 1), Op::Gadt(3, 1, 3)]); + assert_eq!( + equations(&old_pattern(&chain).form), + ["= n0 := &0[G]int", "= n1 := &1[G]int"] + ); + } + + /// The arguments of two constructors of one path (observed; decision + /// 62; the old checker is a judge). As `unify_gadt` runs it + /// (`assume_injective`) the arguments are unified with equations + /// (`set_mode_pattern ~generate:true ~injective:false`, + /// compiler/ml/ctype.ml:2352-2354): `k` against `k` gives + /// `n0 := int`. Below that, `k`, abstract and not of the unit, is not + /// injective: its arguments are unified without equations and what + /// does not unify is reified (ctype.ml:2359-2371): no equation, a + /// success, the two `k` nodes two nodes, and a later unification of + /// the two in expression mode fails, so nothing of the pattern + /// unification is left to make them equal. `d`, a data type, is + /// injective (`expands_to_datatype`, ctype.ml:2355-2358): `n0 := int`. + /// Without `generate_equations` neither has one. A variable argument + /// is a link to the other argument in every mode. + #[test] + fn old_checker_refines_the_arguments_of_a_constructor_by_its_injectivity() { + assert_eq!(equations(&old_pattern(&pattern(1, 0, 1, 3)).form), ["= n0 := &0[G]int"]); + for flags in [0, 1] { + let abstract_type = old_pattern(&pattern(1, 0, 1, flags)); + assert_eq!((abstract_type.failed, equations(&abstract_type.form).len()), (None, 0)); + assert!(abstract_type.form.contains("\n1 = @1\n2 = @3\n"), "{}", abstract_type.form); + } + assert_eq!(equations(&old_pattern(&pattern(2, 0, 1, 1)).form), ["= n0 := &0[G]int"]); + assert_eq!(equations(&old_pattern(&pattern(2, 0, 1, 0)).form).len(), 0); + let nested = old_pattern(&pattern(6, 2, 3, 3)); + assert_eq!((nested.failed, equations(&nested.form).len()), (None, 0)); + let mut again = pattern(6, 2, 3, 3); + again.extend([Op::Child(1, 0), Op::Child(2, 0), Op::Unify(3, 4)]); + assert_eq!(old_pattern(&again).failed, Some(7)); + let mut direct = pattern(1, 0, 1, 1); + direct.push(Op::Unify(1, 2)); + assert_eq!(old_pattern(&direct).failed, Some(5)); + let variable = old_pattern(&pattern(1, 0, 3, 1)); + assert!(variable.form.contains("&7[1]k<@2>"), "{}", variable.form); + } + + /// What `reify` makes of a variable and of a row variable (observed; + /// decision 62; the old checker is a judge of both, and of the names + /// of variables without a name). The variable of `('a, int)` at the + /// newtype level becomes a new abstract type `$0` at the variable's + /// level, and the equation's body is a generic copy that names it + /// (compiler/ml/ctype.ml:1815-1840, 2085-2087). The row variable of + /// `[> #A(int)]` becomes `$0` as well, and the row a fixed row over + /// it (`{r with row_fields = []; row_fixed = true; row_more = t}`, + /// ctype.ml:1845-1851). A second one is `$1` (`get_new_abstract_name`, + /// ctype.ml:908-922). No operation gives the harness a variable with + /// a name: `$'a` is tested by hand in tc/r3_tests.rs. + #[test] + fn old_checker_reifies_a_variable_and_a_row_variable() { + let variable = old_pattern(&refine(3, 2)); + assert_eq!(variable.failed, None); + assert_eq!( + variable.form.lines().next(), + Some("0 = &0[1]fn(&1[1]$0, &2[1]', &3[1](@1, &4[1]int)) => &5[1]unit") + ); + assert_eq!(equations(&variable.form), ["= n0 := &0[G](&1[G]$0, &2[G]int)"]); + let row = old_pattern(&refine(5, 3)); + assert!(row.form.contains("&10[1][fixed #A(&11[1]int) | &12[1]$0]"), "{}", row.form); + assert_eq!(equations(&row.form), ["= n0 := &0[G][fixed #A(&1[G]int) | &2[G]$0]"]); + let mut twice = refine(3, 2); + twice.extend([Op::Child(2, 1), Op::Instance(3), Op::Child(5, 2), Op::Gadt(4, 6, 3)]); + assert_eq!( + equations(&old_pattern(&twice).form), + ["= n0 := &0[G](&1[G]$0, &2[G]int)", "= n1 := &3[G](&4[G]$1, &5[G]int)"] + ); + } + + /// What a pattern unification that fails leaves (observed; decision + /// 62; the old checker is a judge of the graph, not of the + /// environment, which the harness drops with the state). `reify` + /// links before it refuses: a variable below the newtype level is a + /// link to its new abstract type when the unification fails with the + /// pair of that type and a new variable (`link_type ty t; if ty.level + /// < newtype_level then raise (Unify [(t, newvar2 ty.level)])`, + /// compiler/ml/ctype.ml:1837-1839), and a row that clashes is a fixed + /// row when `mcomp` refuses the pair after it (ctype.ml:2400-2408). + /// The handler of unify3 puts back only the description of the left + /// node (`t1'.desc <- d1`, ctype.ml:2437-2439). The caller that goes + /// on after a failure restores a snapshot, the levels and the + /// environment it kept (`set_state`, compiler/ml/typecore.ml: + /// 1299-1305); `unify_pat_types_gadt` itself reports the failure + /// (typecore.ml:488-495). A failure below a tuple whose first + /// elements gave an equation leaves the graph as it was. + #[test] + fn old_checker_keeps_what_reify_linked_when_a_pattern_unification_fails() { + let ops = [ + Op::Instance(3), + Op::Child(0, 2), + Op::BeginDef, + Op::Instance(0), + Op::Child(2, 0), + Op::Gadt(3, 1, 3), + ]; + let below = old_pattern(&ops); + assert_eq!(below.failed, Some(5)); + assert_eq!(trace_of(&below.form, 5), ["@7 ~ @3", "@1 ~ &14[0]'"]); + assert_eq!( + below.form.lines().last(), + Some("3 = @7"), + "{}", + below.form + ); + assert!( + below.form.contains("\n0 = &0[0]fn(&1[0]$0, &2[0]', &3[0](@1, &4[0]int)) => &5[0]unit\n"), + "{}", + below.form + ); + let row = old_pattern(&pattern(5, 0, 2, 3)); + assert_eq!(row.failed, Some(4)); + assert!(row.form.contains("&1[1][fixed #A(&2[1]n0) | &3[1]$0]"), "{}", row.form); + let partial = old_pattern(&pattern(6, 0, 1, 3)); + assert_eq!(partial.failed, Some(4)); + assert_eq!(trace_of(&partial.form, 4), ["@1 ~ @4", "@3 ~ @6"]); + assert_eq!((partial.unchanged, partial.changed), (1, 0)); + } + + /// Rows in pattern mode (observed; decision 62; the old checker is a + /// judge of all three). Two closed rows of which one lacks a tag of + /// the other, and a constant tag against one with an argument, clash + /// and are not compatible: `mcomp` refuses them (`mcomp_row`, + /// compiler/ml/ctype.ml:1985-2009). A fixed row against an open row + /// with one more tag clashes (`unify_row` does not extend a fixed + /// row) and is compatible: the unification succeeds after the + /// backtrack, the two rows are two nodes, the open one is reified to + /// a fixed row over `$1` and nothing else of the attempt is left + /// (ctype.ml:2400-2408). + #[test] + fn old_checker_accepts_rows_that_clash_and_are_compatible() { + assert_eq!(old_pattern(&pattern(4, 0, 2, 3)).failed, Some(4)); + assert_eq!(old_pattern(&pattern(4, 0, 3, 3)).failed, Some(4)); + let mut ops = refine(5, 3); + ops.extend([Op::Child(0, 4), Op::Gadt(1, 4, 3)]); + let compatible = old_pattern(&ops); + assert_eq!(compatible.failed, None); + assert!( + compatible.form.contains( + "&10[1][fixed #A(&11[1]int) | &12[1]$0], &13[1][fixed #A(&14[1]int) #B | &15[1]$1]" + ), + "{}", + compatible.form + ); + assert!(compatible.form.contains("\n1 = @10\n") && compatible.form.contains("\n4 = @13\n")); + } + + /// Where the old checker is NOT a judge (observed; decision 62): two + /// structures that a pattern unification unifies are one node + /// afterwards. OCaml records the pair as assumed equal and links + /// neither (`Pattern -> add_type_equality t1' t2'`, + /// compiler/ml/ctype.ml:2327-2331): a type that is equal only under + /// the equations of one case must not become the other for good. Two + /// tuples of two instances are one node; of two closed rows that + /// differ in `n0` against `int` the left one prints `int` afterwards. + /// [`old_judges`] ends a case before such an operation. + #[test] + fn old_checker_links_the_structures_a_pattern_unification_unifies() { + let program = compile(PATTERN_PROGRAM.to_string()); + let plan = Plan::new(Tier::Full, PATTERN_PLAN); + let tuples = [ + Op::BeginDef, + Op::Instance(0), + Op::Child(0, 3), + Op::Instance(0), + Op::Child(2, 3), + Op::Gadt(1, 3, 3), + ]; + let linked = run_old(&program, plan, &tuples); + assert!(linked.form.contains("\n3 = @4\n"), "{}", linked.form); + assert_eq!(old_judges(&program, plan, &tuples), 5); + let rows = pattern(4, 0, 1, 3); + let linked = run_old(&program, plan, &rows); + assert!( + linked.form.starts_with("0 = &0[1]fn(&1[1][closed #A(&2[1]int) #B | &3[1]nil], @1,"), + "{}", + linked.form + ); + assert_eq!(old_judges(&program, plan, &rows), 4); + } + + /// The self-test of [`PATTERN`]: the two sides agree today on a + /// pattern program for everything that is not a pattern unification + /// (the locally abstract types, the abstract type with a parameter + /// and the data type convert, print and unify in expression mode as + /// any type), on the fixed program and on 300 generated cases with + /// the pattern unifications taken out; and the form of a pattern + /// plan ends with no equation there. + #[test] + fn the_two_sides_agree_on_pattern_programs_without_a_pattern_unification() { + let program = compile(PATTERN_PROGRAM.to_string()); + assert_eq!(program.values.len(), 7); + let plan = Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN); + let ops = [ + Op::BeginDef, + Op::Instance(0), + Op::Instance(1), + Op::Child(0, 0), + Op::Child(1, 3), + Op::Child(3, 0), + Op::Unify(2, 4), + Op::Instance(5), + Op::Child(0, 1), + Op::Unify(2, 6), + Op::EndDef, + Op::Generalize(1), + Op::Copy(1), + ]; + let outcome = compare(0, &program, plan, &ops, Fault::None).unwrap_or_else(|report| panic!("{report}")); + assert_eq!(outcome.failed, Some(9)); + assert!(outcome.form.contains("k<&"), "{}", outcome.form); + assert_eq!(equations(&outcome.form).len(), 0); + let plan = Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN); + let mut patterns = 0; + for seed in 1..=3u64 { + for sequence in 0..100u64 { + let case_seed = seed * 1_000_003 + sequence; + let (program, ops) = case(plan, case_seed); + patterns += ops.iter().filter(|op| matches!(op, Op::Gadt(..))).count(); + let ops: Vec = + ops.into_iter().filter(|op| !matches!(op, Op::Gadt(..))).collect(); + if let Err(report) = compare(case_seed, &program, plan, &ops, Fault::None) { + panic!("{report}"); + } + } + } + assert!(patterns >= 300, "{patterns}"); + } + + /// The old side of the pattern plan runs today and is not vacuous + /// (observed, 1,200 cases: 1136 judged to the end, 64 not; 950 + /// pattern unifications that succeed and 493 that fail; 130 cases end + /// with an equation, 3 with two, 183 with a reified type; the bounds + /// are a third of that): cases the old checker judges to the end and cases it + /// leaves at a pattern unification that linked two structures + /// ([`old_judges`]); among the judged operations, pattern + /// unifications that succeed and that fail; cases that end with an + /// equation, with two, and with a reified type. + #[test] + fn old_side_of_the_pattern_plan_is_not_vacuous() { + let plan = Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN); + let (mut whole, mut cut, mut ok, mut failed) = (0, 0, 0, 0); + let (mut equation, mut two, mut reified) = (0, 0, 0); + for seed in 1..=12u64 { + for sequence in 0..100u64 { + let (program, ops) = case(plan, seed * 1_000_003 + sequence); + let judged = old_judges(&program, plan, &ops); + if judged == ops.len() { + whole += 1; + } else { + cut += 1; + } + let ops = &ops[..judged]; + let outcome = run_old(&program, plan, ops); + for (index, op) in ops.iter().enumerate() { + if matches!(op, Op::Gadt(..)) { + match outcome.form.contains(&format!("! {index} trace\n")) { + true => failed += 1, + false => ok += 1, + } + } + } + let count = equations(&outcome.form).len(); + equation += usize::from(count >= 1); + two += usize::from(count >= 2); + reified += usize::from(outcome.form.contains('$')); + } + } + println!("pattern plan, old side: {whole} {cut} {ok} {failed} {equation} {two} {reified}"); + assert!( + whole >= 380 + && cut >= 20 + && ok >= 300 + && failed >= 160 + && equation >= 40 + && two >= 1 + && reified >= 60, + "{whole} {cut} {ok} {failed} {equation} {two} {reified}" + ); + } + + /// The deliberately wrong new side of [`PATTERN`] + /// ([`Fault::ExpressionMode`]: a pattern unification runs as an + /// expression unification, which the kernel has today) is reported by + /// the generated cases: by a case in which the old checker adds an + /// equation, and by one in which a pattern unification the old + /// checker accepts fails in expression mode (observed, 300 cases: 228 + /// with a pattern unification; reported with an equation on the old + /// side 21, with a failure only the wrong side has 58). + #[test] + fn generated_cases_find_a_pattern_unification_in_expression_mode() { + let plan = Plan::new(Tier::Full, PATTERN_PLAN); + let (mut cases, mut equation, mut refused) = (0, 0, 0); + for seed in 1..=3u64 { + for sequence in 0..100u64 { + let case_seed = seed * 1_000_003 + sequence; + let (program, ops) = case(plan, case_seed); + let ops = &ops[..old_judges(&program, plan, &ops)]; + if !ops.iter().any(|op| matches!(op, Op::Gadt(..))) { + continue; + } + cases += 1; + let old = run_old(&program, plan, ops); + if compare(case_seed, &program, plan, ops, Fault::ExpressionMode).is_err() { + let new = run_new(&program, plan, ops, Fault::ExpressionMode); + equation += usize::from(equations(&old.form).len() > 0); + refused += usize::from(new.failed.is_some() && old.failed != new.failed); + } + } + } + println!("expression mode fault: {cases} {equation} {refused}"); + assert!(cases >= 75 && equation >= 7 && refused >= 19, "{cases} {equation} {refused}"); + } + /// 10,000 cases of everything on programs with polymorphic fields, /// universal variables, rows whose row variable is bound and /// hand-made polymorphic types, and 10,000 more with made nodes: the diff --git a/compiler-rust/src/tc/expand.rs b/compiler-rust/src/tc/expand.rs index 10e51d2a0b..32243fbadb 100644 --- a/compiler-rust/src/tc/expand.rs +++ b/compiler-rust/src/tc/expand.rs @@ -51,6 +51,16 @@ pub const INHERITS: u8 = 32; /// with [`INHERITS`], in the same pass over the template /// ([`TypeTable::mark_manifests`]). (R3e) pub const OPEN_ROW: u8 = 64; +/// The path is a bare identifier for the unit being checked: one of its +/// own types or a predefined type (`in_current_module`, +/// compiler/ml/ctype.ml:1870-1872: `Pident`). With no [`NEWTYPE`] such a +/// type is `non_aliasable`: no equation refines it and it is compatible +/// with no other type ([`TypeTable::non_aliasable`]). Set by the caller +/// that owns the table, with [`TypeTable::set_local`] (R5a; the tests of +/// R3h): [`TypeTable::from_segment`] does not set it, since a segment +/// does not say whether it is the unit being checked or one it imports, +/// whose types are paths through a module. Read by R3h. +pub const LOCAL: u8 = 128; /// Head flag of a constructor node whose slot 0 holds a private expansion /// (`Mcons (Private, ..)`, compiler/ml/btype.ml:612). Bit 4 is /// `segment::EXTENDED`. @@ -303,6 +313,39 @@ impl TypeTable { } } + /// O(1): `type_index` is a bare identifier for the unit being checked + /// ([`LOCAL`]). One bounds test and one byte written. + pub fn set_local(&mut self, type_index: u32) { + self.rows[type_index as usize].flags |= LOCAL; + } + + /// `non_aliasable p decl` (compiler/ml/ctype.ml:1870-1872) for a + /// row: a bare identifier that is not a locally abstract type. O(1), + /// no load. + #[inline] + pub fn non_aliasable(row: TypeInfo) -> bool { + row.flags & (LOCAL | NEWTYPE) == LOCAL + } + + /// `is_instantiable env p` (compiler/ml/ctype.ml:1874-1881) for the + /// type of a constructor node WITHOUT arguments: the type has a row + /// (`Env.find_type` did not raise), is abstract (not [`DATATYPE`]), + /// has no manifest (none of [`ABBREV`], [`PRIVATE`], [`ALIAS`]: a + /// private abstract type always has one, so `type_private = Public` + /// follows) and is not [`TypeTable::non_aliasable`]. `type_arity = 0` + /// is the node's own arity. One bounds test and one row load. A type + /// an equation refined has [`ABBREV`] and is no longer instantiable, + /// as its local constraint has a manifest. (R3h) + #[inline] + pub fn instantiable(&self, type_index: u32) -> bool { + match self.rows.get(type_index as usize) { + Some(&row) => { + row.flags & (DATATYPE | ABBREV | PRIVATE | ALIAS) == 0 && !Self::non_aliasable(row) + } + None => false, + } + } + /// O(1): one bounds test and one 16-byte load. (R3b) #[inline] pub fn info(&self, type_index: u32) -> TypeInfo { diff --git a/compiler-rust/src/tc/rows.rs b/compiler-rust/src/tc/rows.rs index 6f5c7b16e8..abb4dc311f 100644 --- a/compiler-rust/src/tc/rows.rs +++ b/compiler-rust/src/tc/rows.rs @@ -310,6 +310,60 @@ pub trait PackageDecls { fn compatible(&mut self, c: &mut Checker, left: Node, right: Node) -> bool; } +/// One equation of pattern mode in force (`Env.add_local_constraint`, +/// compiler/ml/ctype.ml:2093-2100): the type that got a manifest and the +/// [`TypeInfo`] row it had before, so that cutting the log back puts the +/// row back ([`Checker::cut_equations`]). 20 bytes; one per equation +/// added and not yet cut. (R3h; CHECKER_REWRITE 8.9) +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +#[repr(C)] +pub struct Equation { + pub type_index: u32, + pub previous: super::expand::TypeInfo, +} + +/// What two declarations answer when `mcomp` compares them +/// (`mcomp_type_decl`, compiler/ml/ctype.ml:2011-2039). +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum DeclMatch { + /// The kinds, the representations, the names, flags and number of the + /// members agree; the pairs of member types were pushed + /// ([`Checker::push_compatible`]). + Members, + /// They cannot be one type (`raise (Unify [])`). + Clash, +} + +/// What pattern mode asks of the declarations, which the kernel does not +/// have (CHECKER_REWRITE 8.9). R5a supplies it; the tests of R3h supply +/// one built by hand. Held in [`Checker::locals`]; without one, `reify` +/// of a type that has a variable stops the compiler (a caller that types +/// a pattern installs one first) and two different data types clash. +pub trait LocalDecls { + /// A new locally abstract type for `reify` (`create_fresh_constr`, + /// compiler/ml/ctype.ml:1815-1826): the supplier makes the identifier + /// (`get_new_abstract_name` of `"$'" ^ name`, or `"$"` when `name` is + /// `NONE`; `name` is the variable's name index) and whatever its own + /// tables hold for a declaration, and returns the type index. The + /// kernel writes the [`TypeTable`](super::expand::TypeTable) row + /// (abstract, [`NEWTYPE`](super::expand::NEWTYPE), both newtype levels + /// the current one). O(1) amortized. + fn fresh_abstract(&mut self, name: u32) -> u32; + /// `mcomp_type_decl` for two DIFFERENT types that are both data types + /// (a record, a variant or an open type) and not both names of the + /// unit being checked (ctype.ml:2026-2037): `Clash` unless both are + /// records of one representation, both variants, or both open; for + /// records and variants every pair of member types is pushed with + /// [`Checker::push_compatible`], in declaration order, the result + /// before the arguments, and a difference in names, mutability, + /// optionality, argument shape or number of members is a `Clash` + /// (`mcomp_variant_description`, `mcomp_record_description`, + /// ctype.ml:2047-2078). The member types are nodes of `c`'s graph: + /// the declarations' own types, not instances for the two argument + /// lists, as in OCaml. O(members). + fn compare(&mut self, c: &mut Checker, left: u32, right: u32) -> DeclMatch; +} + /// O(1): the `a` of a [`RES_EITHER`] node with `c` and `m`. #[inline] pub fn either_flags(constant: bool, matched: bool) -> u32 { diff --git a/compiler-rust/src/tc/unify.rs b/compiler-rust/src/tc/unify.rs index 3991f42603..59aa39d112 100644 --- a/compiler-rust/src/tc/unify.rs +++ b/compiler-rust/src/tc/unify.rs @@ -172,6 +172,18 @@ pub struct Checker { /// What [`Checker::unify_package`] asks about module types; `None` /// until a caller that has declarations sets it (R5a). (R3p) pub packages: Option>, + /// `current_newtype_level` (compiler/ml/ctype.ml:1802-1808): the level + /// of the pattern being typed, or 0 outside + /// [`Checker::unify_pattern`] (`None`; no node is at level 0). (R3h) + pub newtype_level: u32, + /// The equations in force, oldest first (the local constraints of + /// `Env`, compiler/ml/ctype.ml:2093-2100). Empty for a unit that + /// never typed a pattern with an equation; `equations.len()` is what + /// a caller keeps to cut back to ([`Checker::cut_equations`]). (R3h) + pub equations: Vec, + /// What `reify` and `mcomp` ask about declarations; `None` until a + /// caller that has declarations sets it (R5a). (R3h) + pub locals: Option>, } impl Checker { @@ -196,6 +208,9 @@ impl Checker { pairs: PairSet::default(), scopes: Vec::new(), packages: None, + newtype_level: 0, + equations: Vec::new(), + locals: None, } } @@ -1224,9 +1239,38 @@ impl Checker { } /// `unify_gadt` (compiler/ml/ctype.ml:2739-2750): one unification in - /// pattern mode with equations, at `newtype_level`; clears the pair set - /// before and after. (R3h) + /// pattern mode with equations and injectivity assumed, at + /// `newtype_level`: [`Checker::unify_pattern`]. (R3h) pub fn unify_gadt(&mut self, newtype_level: u32, left: Ty, right: Ty) -> Result<(), Clash> { + self.unify_pattern(newtype_level, true, true, left, right) + } + + /// One top-level unification in pattern mode (`unify_gadt`, + /// compiler/ml/ctype.ml:2739-2750, with the two flags of + /// `set_mode_pattern`, ctype.ml:256-273, as arguments so that a test + /// can ask for each mode `unify3` installs, ctype.ml:2352, 2368): + /// 1. `scopes` must be empty (`current_univar_pairs () := []`); the + /// previous `umode`, `generate_equations`, `assume_injective` are + /// kept; `umode = Pattern`, the two flags as given, `newtype_level` + /// as given; + /// 2. [`Checker::unify`] of the pair; + /// 3. on either result: the three settings go back, `newtype_level` + /// is 0, and the table of assumed equalities is emptied + /// (`Type_pairs.clear (unify_eq_set ())`; `PairSet::clear`, + /// O(pairs assumed)). + /// + /// The equations it added stay in [`Checker::equations`]: the caller + /// cuts them back when the case they belong to ends or when it + /// restores a speculation ([`Checker::cut_equations`]). O(1) plus the + /// unification. (R3h; CHECKER_REWRITE 8.9) + pub fn unify_pattern( + &mut self, + newtype_level: u32, + generate: bool, + injective: bool, + left: Ty, + right: Ty, + ) -> Result<(), Clash> { todo!("R3h") } From 2b54f965460e3de61ef7e7e0f897f2a3da082349 Mon Sep 17 00:00:00 2001 From: "open-session-submodule[bot]" <341020755+open-session-submodule[bot]@users.noreply.github.com> Date: Tue, 17 Nov 2026 22:50:15 +0000 Subject: [PATCH 4/8] Checker rewrite R3: the design of pattern mode in two slices, with their acceptance tests and build prompts (section 8.9, decisions 63 to 71) Section 8.9: the state of pattern mode and what a unit that never types a pattern pays; what replaces the link in each arm of unify3; the arm of unify for two locally abstract types; an equation as a manifest whose body is a generic copy, its log and its cut; reify; the two arms that try and fall back as frames, not recursion; mcomp with a supplier that compares declarations, and a clash without one. Two slices: R3h (state, equations, reify, the arms without a backtrack) and R3r (mcomp and the two arms that try). Skeleton: FrameKind::{Flags, Try}, Try, Checker::{cut_equations, recover, tries, compatible, duplicate, non_recursive_abbrev, find_newtype_level, push_compatible}, TypeTable::{injective, injective_bits}; reify can fail. Bodies are todo. Fourteen ignored tests: eleven hand-built, two step tests at 1,000 and 10,000, and the generated cases of the pattern plan. No prototype was made: they compile and the check of the ignored tests passes. Co-authored-by: Jaap Frolich Session: https://os.submodule.nl/session/bks-9da31a3c-0d3b-7546-bc8e-f1fd3a9270ee Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com> --- compiler-rust/docs/CHECKER_REWRITE.md | 474 +++++++++++++++- compiler-rust/src/tc/differential.rs | 213 +++++-- compiler-rust/src/tc/expand.rs | 18 + compiler-rust/src/tc/r3_tests.rs | 776 +++++++++++++++++++++++++- compiler-rust/src/tc/rows.rs | 102 +++- compiler-rust/src/tc/unify.rs | 79 ++- 6 files changed, 1595 insertions(+), 67 deletions(-) diff --git a/compiler-rust/docs/CHECKER_REWRITE.md b/compiler-rust/docs/CHECKER_REWRITE.md index 0f35f6c510..3712a3d98d 100644 --- a/compiler-rust/docs/CHECKER_REWRITE.md +++ b/compiler-rust/docs/CHECKER_REWRITE.md @@ -2178,7 +2178,8 @@ it stops and reports a reproduction. | R3n | The references of a copied variant row: shared when the row variable is, new otherwise (8.6, decision 34) | `rows.rs`, `expand.rs` | R3e, R3f | R3g | 150 | | R3g | Universal variables and polymorphic types with variables: the scopes, `unify_univar`, `enter_poly`, the escape test, `occur_univar`, PR#6744 in `unify_row_field` (8.6, decisions 49 to 51; 8.8) | `rows.rs` | R3e, R3f, R3n | R3p | 600 | | R3p | Packages: the merge of two name lists, completed through `PackageDecls` (8.6, decision 52; 8.8) | `rows.rs` | R3e | R3g | 150 | -| R3h | Pattern mode, equations, `reify`, `mcomp` | `rows.rs`, `unify.rs` | R3n, R3g | R3i, R3j | 600 | +| R3h | Pattern mode: state, equations, `reify`, the arms without a backtrack (8.9) | `rows.rs`, `unify.rs`, `expand.rs` | R3n, R3g | R3i, R3j, R3q | 450 | +| R3r | Pattern mode: `mcomp`, the two arms that try and fall back (8.9) | `rows.rs`, `unify.rs` | R3h | R3i, R3j, R3q | 450 | | R3q | `instance_poly` and `instance_label`: the copy that goes through free universal variables without sharing (`copy_sep`), and the fixed-row rule of `copy_row` only it reaches (8.6, decision 47; 8.8, open question 1) | `rows.rs`, `expand.rs` | R3n, R3g | R3h, R3i, R3j | 300 | | R3i | `moregeneral`, `equal`, `matches` | `relate.rs` | R3n, R3g | R3h, R3j | 600 | | R3j | Subtyping without the declaration-directed coercions | `relate.rs` | R3n, R3g | R3h, R3i | 600 | @@ -2262,6 +2263,15 @@ before this turn: R3h, the `instance_poly` slice (R3q: a slice name has three characters), R3i and R3j. What the sixth turn did is listed in 8.6 ("Decided in the sixth skeleton turn"). +| Slice | Decisions | Tests | Run in this turn | +|---|---|---|---| +| R3h | 61 to 67, 69, 71 | nine ignored in `r3_tests.rs`: eight hand-built and the step test `pattern_mode_at_ten_thousand` | none: no prototype was made; they compile against the skeleton and the check of the ignored tests passes. The harness and the old checker's side (decisions 61, 62) are nine tests that run today | +| R3r | 63, 68, 70 | five ignored: three hand-built and `mcomp_at_ten_thousand` in `r3_tests.rs`, the fixed cases and 12,000 generated cases in `differential.rs` (`pattern_plan_agrees_on_generated_cases`) | none, as above | + +After the sixth turn: designed and not built, R3h and R3r (8.9); without +a design, R3q, R3i, R3j and the ambiguity levels of GADT instances +(decision 62). + The ignored tests are checked mechanically (`differential::tests::ignored_tests_name_the_slices_they_reach`): each names the slices it reaches, reaches its own, and reaches none that is @@ -2565,6 +2575,9 @@ End state: no `todo!("R3p")`; the three ignored tests pass. One commit. Failure: a frame or a saved descriptor left after a clash in the merge; a supplier not put back; a `Vec` of pairs. +**R3h, R3r.** Functions, end state, commits and gates: section 8.9 +("The parts") and the two prompts of 8.4. + **R3h to R3j.** Functions: every signature of `rows.rs` and `relate.rs` carrying the slice's name. End state for each: no `todo!` with its name; its generator extension and old-side vacuity test (first commit); the @@ -3013,9 +3026,128 @@ for two equal paths; a change in unify.rs or outside rows.rs and the three ignore lines. At most 150 lines. ``` +```text +R3h. Read section 8.9 whole and section 8.6, decisions 47, 60, 61 and +62; compiler/ml/ctype.ml 253-273, 1478-1535, 1802-1881, 2085-2104, +2229-2440 and 2739-2750; in src/tc/unify.rs the doc comments of +PairSet, FrameKind::Flags, Checker::{unify_pattern, cut_equations} and +the bodies of unify_pair, unify_expanded, unify3 and run_frames; in +src/tc/rows.rs the doc comments of Equation, LocalDecls and +Checker::{reify, add_gadt_equation, non_recursive_abbrev, duplicate, +find_newtype_level, drop_name_children, copy_refs}; in src/tc/expand.rs +TypeTable::{instantiable, non_aliasable, injective}, LOCAL, and the body +of Checker::subst. R3g, R3n and R3p are merged. + +Commits, in this order, each pushed with + cargo test --manifest-path compiler-rust/Cargo.toml --lib tc:: +seen to pass: +1. The fixed-row rule of Checker::copy_refs (decision 47, last + paragraph), with the hand-built row of that decision as a running + test in src/tc/r3_tests.rs. +2. Checker::{find_newtype_level, non_recursive_abbrev, duplicate, + add_gadt_equation} in rows.rs, Checker::cut_equations in unify.rs, + and the equation arm at the top of Checker::subst in expand.rs + (decision 67). +3. Checker::reify in rows.rs (section 8.9, "reify"). +4. Checker::unify_pattern; `pattern` loaded once in unify_expanded and + passed to unify3, with the lookup of the table before unify3; the + arms of decision 64 marked 1, each `todo!("R3h")` of unify.rs + replaced; the Flags arm of run_frames; the arm of decision 65 in + unify_pair. The three places marked 2 in the table of decision 64 + become `todo!("R3r")`. Then the `#[ignore = "R3h build"]` lines of + src/tc/r3_tests.rs removed and nothing else in those tests changed. +End your turn after the fourth commit is pushed and no test is ignored +for "R3h build". + +Steps: one `#[cfg(test)] super::steps::tick()` per node `reify`, +`non_recursive_abbrev` and `duplicate` visit, and per entry +`cut_equations` pops; unify_pair's own tick counts the pairs. + +Relations to hold: every tc:: test that passed before passes +unchanged; pattern_mode_at_ten_thousand as written; after +unify_pattern, on either result, what +r3_tests::assert_pattern_left asserts. + +Report, before and after, the instructions of the tc_kernel bench +operations unify_vars, unify_ground1, unify_ground3, unify_datatype2 +and unify_abbrev2 (compiler-rust/docs/TC_KERNEL_R3.md, "Machine and +method"). No target is set (decision 46). Report also what +Checker::generalize does to a memo or to anything that is not a node's +level when `duplicate` calls it: if it writes something +`Graph::restore` does not put back, stop and report it. + +If a test of yours fails where your code does what the ctype.ml lines +say, stop and report the test with the lines: do not change the test. + +Shortcuts that count as failure: linking two structures in pattern +mode; an occurrence test for a structure pair in pattern mode; reading +`pairs` or `umode` in unify_pair or anywhere in expression mode beyond +the one load of unify_expanded; a recursive walk (use g.work and +Checker::marks with a new generation); a hash set or a `Vec` allocated +per call; cutting equations in Graph::restore or on a failed +unification; a body of an equation that shares a node with its +destination; writing a TypeTable row of an equation anywhere but +add_gadt_equation and cut_equations; a change in graph.rs, level.rs or +template.rs, or under src/types. At most 450 lines. +``` + +```text +R3r. R3h is merged. Read section 8.9, decisions 63, 64 (the rows +marked 2), 68 and 70 and "No supplier"; compiler/ml/ctype.ml 1886-2080, +2359-2371, 2400-2408 and 2416-2421; in src/tc/unify.rs the doc comments +of Try, FrameKind::Try and Checker::recover, and the body of +run_frames; in src/tc/rows.rs the doc comments of LocalDecls, +DeclMatch, Checker::{mcomp, push_compatible, enter_poly, unify_univar, +unify_row, unify_fields, flatten_row, merge_rows, unify_package} and +the body of Checker::leave_scopes in unify.rs. + +Commits, in this order, each pushed with the tc:: tests seen to pass: +1. Checker::{mcomp, push_compatible} in rows.rs (decision 70), and the + call of mcomp in the constructor arm of unify3 that R3h left + `todo!("R3r")`. +2. Checker::recover and the Try arm of run_frames in unify.rs: every + place where run_frames returns a clash first asks `recover(base)` + and goes on with the loop when it answers `Some(Ok(()))`; a + `Some(Err(_))` is a clash again and asks again. Then the argument + arm and the variant arm of unify3 (decision 64, the rows marked 2) + and the package arm of decision 63. +3. The `#[ignore = "R3r build"]` lines of src/tc/r3_tests.rs and + src/tc/differential.rs removed, nothing else in those tests changed. +End your turn after the third commit is pushed and no test is ignored +for "R3r build". + +Steps: one tick per pair `mcomp` takes off its stack. + +Relations to hold: every tc:: test that passed before passes +unchanged; mcomp_at_ten_thousand as written; `Checker::compatible` and +`Checker::tries` are empty when unify_pattern and mcomp return; +`g.speculation_depth` is what it was. + +If pattern_plan_agrees_on_generated_cases reports a seed, read the +ctype.ml lines of the arm first. If your code does what they say, stop +and report the seed with the lines: do not copy the old checker, and do +not pin the seed yourself. The old checker is not a judge of a pattern +unification that succeeds on two structures (decision 62), and +`differential::old_judges` already ends a case before one; a seed that +differs there means old_judges missed a link: report it. + +Report the same five bench operations as R3h, before and after. + +Shortcuts that count as failure: a nested call of Checker::unify for +an attempt (the attempt is frames above a Try frame); cutting +`equations` in recover; a second table of pairs for unification (the +table of mcomp is `compatible`, the table of unification `pairs`); +comparing declarations in the kernel (LocalDecls::compare is the only +reader of a declaration); answering compatible for two data types +without a supplier; calling leave_scopes anywhere but in recover and +mcomp's marker; a change in graph.rs, level.rs, template.rs or +expand.rs, or under src/types. At most 450 lines. +``` + The prompt of K2 is in section 8.7. The last prompt below is superseded by decision 5 of 8.6: no builder starts a slice from it. R3e, R3f, R3n, R3g -and R3p have their own prompts above it; R3h to R3j have none yet. +and R3p, R3h and R3r have their own prompts above it; R3q, R3i and R3j +have none yet. ```text R3e..R3j. First commit, before any body: extend @@ -4628,6 +4760,14 @@ the reviews above left owed; every test named runs today. it expands both heads, and the equations that result were the same on every case tried. +63 to 71. **The design of R3h** is section 8.9, where each is written + with its reasons: 63 `package_end` in pattern mode; 64 what replaces + the link, per arm of unify3; 65 the arm of `unify` for two locally + abstract types; 66 the declaration facts; 67 an equation, its + storage and its cut; 68 the two arms that try, as frames; 69 `occur` + in pattern mode; 70 `mcomp` and the answer without a supplier; 71 + what a unit that never types a pattern pays. + ### 8.7 The hot paths of unification (K2) @@ -4937,6 +5077,336 @@ it follows and names its witness. binder nested in a binder is a method inside a polymorphic field, and the depth is the nesting of the source. +### 8.9 Pattern mode, equations, `reify` and `mcomp` (R3h) + +What 8.6 decided for this slice: the harness and where the old checker +is a judge (decisions 61 and 62). This section is the design; decisions +63 to 71 of 8.6 are its numbered decisions, written here so that they +are read with their reasons. The work is cut in two slices, **R3h** +(state, equations, `reify`, the arms that need no backtrack) and +**R3r** (`mcomp`, the two arms that try and fall back; a slice name +has one letter, which the check of the ignored tests reads), built in +that order, each under 450 lines. + +**State.** Five things, all on `Checker`. + +| state | where | written by | read by | +|---|---|---|---| +| `umode` | `Checker::umode` | `unify_pattern` (entry, exit) | `unify_expanded`, once a call | +| `generate_equations`, `assume_injective` | two bytes of `Checker` | `unify_pattern`; the `Flags` frame and the `Try` record put back what they found | unify3's pattern arms | +| newtype level | `Checker::newtype_level`, 0 outside | `unify_pattern` | `reify`, `add_gadt_equation` | +| assumed equalities | `Checker::pairs` (`PairSet`) | unify3 in pattern mode | `unify_expanded` in pattern mode | +| equations | `Checker::equations`, and the `TypeTable` row of each | `add_gadt_equation`, `cut_equations` | the row: `expand_once`; the length: `unify_pair`'s arm for two newtypes | + +*The table of assumed equalities* (`unify_eq_set`, +compiler/ml/ctype.ml:2102-2104). Key: the two node indices of the pair, +the smaller first, in one `u64` (`order_type_pair`, ctype.ml:2104; the +kernel's index is OCaml's `id` for this purpose: a total order on +nodes). It is a hash set: pairs of nodes have no dense index. Size: 8 +bytes a pair plus the set's control bytes; empty and unallocated for a +unit that never types a pattern. Cleared when `unify_pattern` returns, +on either result (ctype.ml:2746, 2749): `if !keys.is_empty() { +keys.clear() }`, which is one byte a bucket of the capacity the largest +pattern unification of the unit left. A pattern unification of 10,000 +structure pairs costs 10,000 inserts and at most 20,000 lookups; the +next one, of one pair, costs one clear of that capacity (16 KiB of +control bytes written), and that is the only cost that outlives the +call. Never read or written in expression mode. + +**63. `package_end` reads the right node as it does today.** In pattern +mode no structure node is linked, so the right node of a `PackageEnd` +frame is the node the frame was pushed with, and decision 55's argument +is not needed. In expression mode nothing changed. The one difference +of pattern mode for packages is the arm of `Not_found` +(ctype.ml:2417-2421): where `unify_package` answers a clash because a +completion was not `Completion::Type`, pattern mode reifies the types of +both lists, left list first, and succeeds (R3r). + +**64. What replaces the link, per arm of unify3.** `unify_expanded` +loads `umode` once and passes `pattern: bool` to unify3 (the load unify3 +makes today moves; the count of loads of a unification in expression +mode does not change). In pattern mode `unify_expanded` answers `Ok` +before unify3 when the pair of the two expanded heads is in the table +(`unify_eq`, ctype.ml:2233, 2286); the two tests `unify_eq t1 t1'` and +`unify_eq t2 t2'` that choose the swapped call (ctype.ml:2299) stay +tests of identity: a pair in the table is two heads that were distinct +when unify3 took them, and a node and its own expanded head are never +that (an expansion is made from the node, after it stopped being a +head). The first three arms of unify3 (a variable on either side, two +universal variables, two fields) are as in expression mode, the +occurrence test of a variable included (`occur !env t1' t2`, +ctype.ml:2316-2323). Then, with `pattern`: + +1. the pair `(t1', t2')` goes into the table (`add_type_equality`, + ctype.ml:2329). No occurrence test (decision 69), no link, no + `Linked` frame, nothing in `saved`: a failure has nothing of this + node to put back (`t1'.desc <- d1` writes what is there); +2. `create_recursion` is computed and its `Recursion` frame pushed as + today (ctype.ml:2308, 2428-2436: the repair is outside the match on + `umode`); +3. the arm, by the two heads, first that applies: + +| heads | `generate` | what runs | part | +|---|---|---|---| +| arrow, arrow of one length | any | labels match when equal or when neither is optional (ctype.ml:2337-2343); the pairs of the parameters, then of the results | 1 | +| tuple, tuple | any | the pairs of the elements | 1 | +| constructor, constructor, one type | off | the pairs of the arguments | 1 | +| the same | on, `assume_injective` on | a `Flags` frame, the flags become (on, off), the pairs of the arguments above it (ctype.ml:2352-2354) | 1 | +| the same | on, injective off, the type's row has `LOCAL` or `DATATYPE` | the pairs of the arguments (ctype.ml:2355-2358; decision 66) | 1 | +| the same | on, otherwise | per argument, last first: when `TypeTable::injective` the pair; otherwise a `Try` frame (flags (off, off), fallback: reify both) and the pair above it (ctype.ml:2359-2371) | 2 | +| nullary constructors of two types, both `instantiable` | on | `add_gadt_equation`: the source is the left type when its newtype level is greater (`find_newtype_level`, compared as pairs), otherwise the right one; the destination is the other side's head (ctype.ml:2373-2384) | 1 | +| nullary `instantiable` constructor, anything | on | `reify` of the right head, then `add_gadt_equation(left type, right head)` (ctype.ml:2385-2388) | 1 | +| anything, nullary `instantiable` constructor | on | `reify` of the left head, then `add_gadt_equation(right type, left head)` (ctype.ml:2389-2392) | 1 | +| a constructor on either side | any | `reify` of the left head, of the right head; with `generate` on, `mcomp` of the two (ctype.ml:2393-2397) | 1; `mcomp` 2 | +| object, object | any | `unify_fields` | 1 | +| variant, variant | any | `open_row` of both, a `Try` frame (flags unchanged, fallback: reify both, and `mcomp` with `generate` on), `unify_row` above it (ctype.ml:2400-2408; decision 68) | 2 | +| `nil`, `nil` | any | nothing | 1 | +| polymorphic types | any | as in expression mode | 1 | +| packages | any | `unify_package`; decision 63 | 1; the fallback 2 | +| anything else | any | a clash | 1 | + +A `Flags` frame (`FrameKind::Flags`) holds in `extra` the two flags as +they were (bit 0 `generate_equations`, bit 1 `assume_injective`) and +puts them back when it is popped; a clash leaves them to the `Try` +record or to `unify_pattern`, which put back their own. + +The rows are in OCaml's order; "left" and "right" are unify3's own +`t1'` and `t2'`, after the swap of unify2, so an equation's direction +does not depend on which side was expanded. Part 1 leaves +`todo!("R3r")` in the three places marked 2 and in `mcomp`. + +**65. The arm of `unify` for two locally abstract types** +(ctype.ml:2258-2270) is in `unify_pair`, after the arm for two nullary +constructors of one type, in every mode: when both heads are nullary +constructors (in registers), `equations` is not empty (one load, made +only for such a pair) and both rows are locally abstract types +(`flags & (NEWTYPE | DATATYPE | PRIVATE) == NEWTYPE`: `is_newtype`, +ctype.ml:1862-1868): `expand_once(·, ExpandMode::Unif)` of the right +node when the left type's newtype level is lower, otherwise of the left +node; when it answers `NONE` (`Cannot_expand`) the pair goes to +`unify_expanded` as it is; otherwise the pair of the unexpanded side and +the expansion is pushed with `push_pair` and the arm returns `Ok`: the +pushed pair is the nested `unify` of OCaml, with its own entry in the +trace. Cost for a unit without equations: one load for a pair of two +nullary constructors of different types, nothing for any other pair. + +**66. Declaration facts.** Held today and read by this slice: +`NEWTYPE` and the `Newtype` row (`type_newtype_level`), `DATATYPE` +(`is_datatype`: a record, a variant or an open type), `ABBREV`, +`PRIVATE`, `ALIAS` (a manifest), `scope` (`Path.binding_time` for a +type that is not a newtype: `get_level`, ctype.ml:562-569, and +`find_newtype_level`, ctype.ml:2085-2091, read the same thing). Added: + +- `LOCAL` (bit 7 of `TypeInfo::flags`): `in_current_module`. Set by + the caller that owns the table (`TypeTable::set_local`; R5a), not by + `from_segment`: a segment does not say whether it is the unit being + checked. `TypeTable::non_aliasable` and `TypeTable::instantiable` are + built and read it. +- `TypeTable::injective_bits: Vec`, indexed by type index, bit + `i` the parameter `i` (`Variance.(mem Inj)` of `type_variance`). + Empty, or shorter than the index, reads as not injective, which is + what an abstract type without an annotation is (observed: `k`, + decision 62). Read only by the last constructor arm, for a type that + is neither `LOCAL` nor `DATATYPE`. A parameter past the 32nd reads as + not injective. R5a fills it; the tests of R3h write it by hand. +- `expands_to_datatype` of `t1'`, `t1` and `t2` (ctype.ml:2356) is the + `DATATYPE` flag of the row of the two heads' type: the heads are of + one type, and `t1` and `t2` expand to them, so the three chains end + at that row; a type on the way that is a data type with a manifest + (`type t = M.t = A | B`) names the same constructors as its head. + +`find_newtype_level(type_index)`: the `Newtype` row's `(level, +expansion_level)` with `NEWTYPE`, otherwise `(scope, scope)`. + +**67. An equation.** `add_gadt_equation(source, destination)` +(ctype.ml:2093-2100): + +1. `non_recursive_abbrev(source, destination)` + (`local_non_recursive_abbrev`, ctype.ml:1478-1535: the type does not + name `source` through its children or through the expansions of the + constructors it names). False: nothing happens, no clash. A walk + with marks and `g.work`, expanding with `expand_once(·, + ExpandMode::Opt)` (`try_expand_once_opt`), O(nodes reached); +2. `duplicate(destination)`: the generic copy (`duplicate_type`). Made + from parts that exist: `g.speculate()`; with `g.level` 0, + `generalize(destination)`, which makes every node of it generic; + with `g.level` generic, `copy(destination)`; `g.level` put back; + `g.restore(token)`. The copy's nodes are new and are not on the + trail, so they stay; the destination has its levels again. Three + passes over the destination; +3. a `Manifest` row `{ template: an empty template, params: 0, body: + the copy }`. A body that is a node of the graph and not a + `RELATIVE` reference marks an equation: `subst` gets one arm at its + top for it, which copies the body at the node's level + (`Checker::copy` with `g.level` the node's), memorizes the copy and + returns it (one compare of a loaded field for every other + expansion); +4. `equations.push(Equation { type_index: source, previous: the row + })`, then the row gets `ABBREV` and the manifest. Its `NEWTYPE` and + `Newtype` row stay (`new_declaration (Some source_lev) (Some + destination)`), so the type is still a locally abstract type for + decision 65 and no longer `instantiable`; +5. `cleanup_abbrev()`. + +`cut_equations(len)` pops the entries above `len`, last first, writing +each `previous` row back, and calls `cleanup_abbrev()` when it popped +one. It is what a caller does where OCaml drops the environment of a +case (`set_state`, typecore.ml:1303-1305, and the end of a case). The +`Manifest` rows and the copies are not reclaimed: 20 bytes and the +body's nodes for every equation a unit ever adds. Nothing of an +equation is on the graph's trail: `Graph::restore` does not cut +equations, and a caller that restores a mark also cuts. Decision 68 +says why the kernel's own backtrack must not. + +**`reify(ty) -> Result<(), Clash>`** (ctype.ml:1813-1859). The +signature changes: it can fail. A walk with a new mark generation and +`g.work`, children pushed so that they pop in `iter_type_expr`'s order +(the fresh names are numbered in visit order). Per representative not +yet marked: + +- a variable: `fresh = locals.fresh_abstract(name)`; the `TypeTable` + row of `fresh` is pushed (`rows.len()` must equal `fresh`: the + supplier counts with the table) as abstract with `NEWTYPE` and a + `Newtype` row of `(newtype_level, newtype_level)`, scope + `newtype_level`; a constructor node of it at the variable's level; + `g.link(variable, node)`; then, when the variable's level was below + `newtype_level`, a clash whose trace pair in `Mode::Explain` is the + new node and a new variable at the variable's level + (ctype.ml:1837-1839). The link stays (decision 62); +- a variant: unless the row is static, for a fixed row the walk goes + on into `more`; otherwise `more` is a variable (anything else is an + invariant failure) and becomes, through `g.link`, a variant node at + its level over a new `VariantRow` without tags, `fixed`, not closed, + whose `more` is the fresh constructor made as above; then + `drop_name_children` for the row (decision 60) and the same level + test. Then the row's children in `iter_row`'s order; +- a constructor whose path is an object type (`is_object_type`): no + kernel type has such a path; not built; +- anything else: its children. + +No supplier (`locals` is `None`) and a variable to reify: a panic that +names `Checker::locals`; a caller that types a pattern installs one. + +**68. The two arms that try** (the argument of a constructor that is +not injective, ctype.ml:2362-2371; two variants, ctype.ml:2400-2408) +are frames, not recursion. A `Try` record in `Checker::tries` (40 +bytes; empty outside pattern mode) holds the index of its frame, the +`Speculation`, the lengths of `trace`, `saved` and `scopes`, the two +flags as they were, and whether `mcomp` follows the reification; its +frame `FrameKind::Try` holds the two types and the record's index. What +is tried is pushed above the frame. + +- The frame is popped (everything above it succeeded): `g.keep`, the + flags put back, the record popped. +- `run_frames` gets a clash from any callee: `recover(base)`. No + record whose frame is at or above `base`: the clash is the caller's, + as today. Otherwise the last record is taken: `frames` is cut to its + frame, `Checker::restore` of its speculation, `leave_scopes` to its + scopes, `trace` and `saved` cut to its lengths, the flags put back; + then `reify(left)`, `reify(right)` and, when the record says so and + `generate_equations` is now on, `mcomp(left, right)`. A clash of + these three is a new clash, handled the same way; otherwise the loop + goes on. + +In both modes of the graph: `Mode::Success` logs to the trail while a +speculation is open and stops when the last one is kept; `Mode::Explain` +has its own speculation below. The descriptors a `Linked` or `Fields` +frame above the `Try` frame would put back are put back by the restore. +`leave_scopes` is the one place that cuts scopes (decision 51): the +record keeps the length it found and `recover` calls it. +What the backtrack does NOT undo, as in OCaml, where `env` is a +reference that `backtrack` does not know: an equation added inside the +attempt of two variants stays in `equations`, and so does a type +`reify` made. Only the graph goes back. + +**69. `occur` in pattern mode.** Only the two variable arms of unify3 +test occurrence (ctype.ml:2316-2323), and they are shared. A structure +pair is not tested (`occur` is in the `Expression` branch, +ctype.ml:2327-2329): nothing is linked, so no cycle is made. +`add_gadt_equation` has its own test (step 1 of decision 67), and a +type that fails it gets no equation without a clash. + +**70. `mcomp(left, right)`** (ctype.ml:1905-1984). Never links, never +lowers; it expands (`expand_head(·, ExpandMode::Opt)`), which +memorizes. Its own table `Checker::compatible` (a second `PairSet`; +ORDERED pairs, `Type_pairs.find type_pairs (t1', t2')`), emptied when +it returns; its stack is `g.work`, two entries a pair. Per pair, after +`repr`: one node, a variable on either side, or two nullary +constructors of one type: compatible. Otherwise both heads expanded; +one node or a pair in the table: compatible; else the pair is added +and, by the heads: + +- arrows of one length whose labels match as in decision 64: the + parameters and the results; tuples of one length: the elements; +- two constructors (`mcomp_type_decl`, ctype.ml:2011-2039): a type + without a row: compatible. One type: the pairs of the arguments + whose parameter is `TypeTable::injective`. Two types, both + `non_aliasable`: a clash. Otherwise by the two rows: both + `DATATYPE`: the pairs of the arguments (a clash when the lists + differ in length), then `LocalDecls::compare`, which pushes the + pairs of the member types with `push_compatible` or answers + `DeclMatch::Clash`; neither `DATATYPE`: compatible; one `DATATYPE` + and the other not `non_aliasable`: compatible; else a clash; +- a constructor against anything else: a clash when its row is + `non_aliasable` or `DATATYPE`, else compatible; +- packages, `nil`: compatible. Objects: `mcomp_fields` + (ctype.ml:1969-1981) over `flatten_fields` and the merge + `unify_fields` uses. Variants: `mcomp_row` (ctype.ml:1985-2009) over + `flatten_row` and `merge_rows`, the table of ctype.ml:1996-2008; +- polymorphic types without variables: the bodies. With variables: + `enter_poly(left, right)`, which pushes a `LeavePoly` frame and the + pair of the bodies on `frames`; `mcomp` pops both and pushes on its + own stack a marker with the frame's scope length and the pair, and a + popped marker calls `leave_scopes`. Two universal variables: + `unify_univar`; +- anything else: a clash. A clash has no trace pair (`Unify []`). + +**No supplier.** `Checker::locals` is `None` and two different data +types meet: a clash, as decision 52 answers for packages: the kernel +has no declaration to compare and must not accept a pattern on a +guess. R5a installs a supplier before it types a pattern, so the +answer is reached only by a test. + +**71. What a unit that never types a pattern pays.** Reads added to +expression mode: none in `unify_expanded` and unify3 (the load of +`umode` moves from one to the other); in `unify_pair` one load of +`equations.len()` for a pair of two nullary constructors of different +types; in `subst` one compare of a field it has loaded; in +`run_frames` nothing on the success path (`recover` is called from the +failure path, where it loads `tries.len()`). Memory: three empty `Vec`s +and two empty sets on `Checker`. The benchmark operations of +TC_KERNEL_R3.md are reported before and after by each part; no target +is set (decision 46). + +**Step tests** (relations, 1,000 against 10,000; tc/r3_tests.rs): +`pattern_mode_at_ten_thousand`. (a) a chain of 10,000 nested tuples +against its copy in pattern mode: the steps are linear in the pairs +(ten times the steps of 1,000, within one part in ten), no node is a +link, and the table is empty afterwards; (b) `reify` of a type with +10,000 variables, each shared twice: every variable is visited once +(steps linear); (c) 10,000 equations added and cut: linear, and every +row is its `previous`; (d) `mcomp` of two chains of 10,000 tuples that +share their elements pairwise: linear, by the table. + +**The parts.** + +- **R3h**: the fixed-row rule of `copy_refs` (decision 47's last + paragraph), then `Checker::{unify_pattern, cut_equations}` and the + pattern arms of `unify_expanded`, unify3 and `unify_pair` marked 1 + (unify.rs); `Checker::{reify, add_gadt_equation, duplicate, + non_recursive_abbrev, find_newtype_level}` (rows.rs); the equation + arm of `subst` (expand.rs). Tests: the hand-built tests of + tc/r3_tests.rs ignored for "R3h build", steps (a) to (c). +- **R3r**: `Checker::{mcomp, push_compatible, recover}`, the `Try` + arm of `run_frames`, the three places R3h + left `todo!("R3r")`. Tests: those ignored for "R3r build", step + (d), and `pattern_plan_agrees_on_generated_cases`. + +**Arms no hand-built test reaches**: the object-type arm of `reify` +(not built); `mcomp` of two fields (`Tfield _, Tfield _`, "actually +unused" in OCaml); a parameter past the 32nd of `injective_bits`; +`LocalDecls::compare` answering `Members` for two open types. + ## 9. R4 as designed Section 6.5 gave R4 as one slice of seven commits. It is replaced by this diff --git a/compiler-rust/src/tc/differential.rs b/compiler-rust/src/tc/differential.rs index a006351e5a..ec6d9b682b 100644 --- a/compiler-rust/src/tc/differential.rs +++ b/compiler-rust/src/tc/differential.rs @@ -718,8 +718,16 @@ pub(crate) fn operations(rng: &mut Rng, plan: impl Into, values: usize) -> let count = if plan.has(VARIANTS) { 6 } else { 4 }; let value = values - 2 - count + rng.below(count); let instance = operands as u16; - let flags = if rng.below(4) == 0 { rng.below(3) as u8 } else { 3 }; - let against = if rng.below(3) == 0 { operand(rng) } else { instance + 2 }; + let flags = if rng.below(4) == 0 { + rng.below(3) as u8 + } else { + 3 + }; + let against = if rng.below(3) == 0 { + operand(rng) + } else { + instance + 2 + }; follow = vec![ Op::Gadt(instance + 1, against, flags), Op::Child(instance, rng.below(4) as u8), @@ -1235,7 +1243,9 @@ pub(crate) fn old_equations(old: &Old, env: &Env) -> String { let mut equations: Vec<(String, TypeExprRef)> = env .iter_local_constraints() .filter_map(|(path, declaration)| { - declaration.type_manifest.map(|body| (path.last().to_string(), body)) + declaration + .type_manifest + .map(|body| (path.last().to_string(), body)) }) .collect(); equations.sort_by(|left, right| left.0.cmp(&right.0)); @@ -1473,7 +1483,7 @@ struct FreshNames { impl super::rows::LocalDecls for FreshNames { fn fresh_abstract(&mut self, name: u32) -> u32 { let prefix = match self.strings.get(name as usize) { - Some(name) if name != "" => format!("$'{name}"), + Some(name) if !name.is_empty() => format!("$'{name}"), _ => "$".to_string(), }; let mut names = self.names.borrow_mut(); @@ -1593,13 +1603,18 @@ impl New { if PREDEFINED.contains(&self.string(name)) { self.checker.types.set_local(index as u32); } - let Some(offset) = NEWTYPES.iter().position(|newtype| *newtype == self.string(name)) + let Some(offset) = NEWTYPES + .iter() + .position(|newtype| *newtype == self.string(name)) else { continue; }; let level = 1 + offset as u32; let types = &mut self.checker.types; - types.newtypes.push(Newtype { level, expansion_level: level }); + types.newtypes.push(Newtype { + level, + expansion_level: level, + }); let row = &mut types.rows[index]; row.flags |= NEWTYPE; row.newtype = types.newtypes.len() as u32 - 1; @@ -1611,7 +1626,9 @@ impl New { } } let pools = &self.segment.signature; - let strings = (0..pools.strings.len() as u32).map(|at| self.string(at).to_string()).collect(); + let strings = (0..pools.strings.len() as u32) + .map(|at| self.string(at).to_string()) + .collect(); self.checker.locals = Some(Box::new(FreshNames { first: pools.paths.len() as u32, strings, @@ -3546,8 +3563,9 @@ mod tests { /// R3x. #[test] fn ignored_tests_name_the_slices_they_reach() { - let before: [(&str, &[&str]); 11] = [ + let before: [(&str, &[&str]); 12] = [ ("R3q", &["R3m", "R3e", "R3f", "R3n", "R3g"]), + ("R3r", &["R3m", "R3e", "R3f", "R3n", "R3g", "R3h"]), ("R3p", &["R3m", "R3e"]), ("R3k", &[]), ("R3m", &[]), @@ -3645,9 +3663,12 @@ mod tests { let mut reaches: Vec<&str> = Vec::new(); let full = body.contains("Tier::Full"); let rows = body.contains("OBJECTS") || body.contains("VARIANTS"); - for (needle, reached) in - [("OBJECTS", "R3e"), ("VARIANTS", "R3f"), ("UNIVAR", "R3g"), ("PATTERN", "R3h")] - { + for (needle, reached) in [ + ("OBJECTS", "R3e"), + ("VARIANTS", "R3f"), + ("UNIVAR", "R3g"), + ("PATTERN", "R3h"), + ] { if full && body.contains(needle) { reaches.push(reached); } @@ -4616,9 +4637,18 @@ mod tests { left.form, "0 = &0[1]fn(&1[1]n0, &2[1]n1, &3[1]int, &4[1](&5[1]int, &6[1]int)) => &7[1]unit\n1 = @1\n2 = @3\n= n0 := &0[G]int\n" ); - assert_eq!(equations(&old_pattern(&pattern(0, 2, 0, 3)).form), ["= n0 := &0[G]int"]); - assert_eq!(equations(&old_pattern(&pattern(0, 0, 1, 3)).form), ["= n1 := &0[G]n0"]); - assert_eq!(equations(&old_pattern(&pattern(0, 1, 0, 3)).form), ["= n1 := &0[G]n0"]); + assert_eq!( + equations(&old_pattern(&pattern(0, 2, 0, 3)).form), + ["= n0 := &0[G]int"] + ); + assert_eq!( + equations(&old_pattern(&pattern(0, 0, 1, 3)).form), + ["= n1 := &0[G]n0"] + ); + assert_eq!( + equations(&old_pattern(&pattern(0, 1, 0, 3)).form), + ["= n1 := &0[G]n0"] + ); assert_eq!( equations(&old_pattern(&pattern(0, 0, 3, 3)).form), ["= n0 := &0[G](&1[G]int, &2[G]int)"] @@ -4650,13 +4680,26 @@ mod tests { /// is a link to the other argument in every mode. #[test] fn old_checker_refines_the_arguments_of_a_constructor_by_its_injectivity() { - assert_eq!(equations(&old_pattern(&pattern(1, 0, 1, 3)).form), ["= n0 := &0[G]int"]); + assert_eq!( + equations(&old_pattern(&pattern(1, 0, 1, 3)).form), + ["= n0 := &0[G]int"] + ); for flags in [0, 1] { let abstract_type = old_pattern(&pattern(1, 0, 1, flags)); - assert_eq!((abstract_type.failed, equations(&abstract_type.form).len()), (None, 0)); - assert!(abstract_type.form.contains("\n1 = @1\n2 = @3\n"), "{}", abstract_type.form); + assert_eq!( + (abstract_type.failed, equations(&abstract_type.form).len()), + (None, 0) + ); + assert!( + abstract_type.form.contains("\n1 = @1\n2 = @3\n"), + "{}", + abstract_type.form + ); } - assert_eq!(equations(&old_pattern(&pattern(2, 0, 1, 1)).form), ["= n0 := &0[G]int"]); + assert_eq!( + equations(&old_pattern(&pattern(2, 0, 1, 1)).form), + ["= n0 := &0[G]int"] + ); assert_eq!(equations(&old_pattern(&pattern(2, 0, 1, 0)).form).len(), 0); let nested = old_pattern(&pattern(6, 2, 3, 3)); assert_eq!((nested.failed, equations(&nested.form).len()), (None, 0)); @@ -4689,15 +4732,33 @@ mod tests { variable.form.lines().next(), Some("0 = &0[1]fn(&1[1]$0, &2[1]', &3[1](@1, &4[1]int)) => &5[1]unit") ); - assert_eq!(equations(&variable.form), ["= n0 := &0[G](&1[G]$0, &2[G]int)"]); + assert_eq!( + equations(&variable.form), + ["= n0 := &0[G](&1[G]$0, &2[G]int)"] + ); let row = old_pattern(&refine(5, 3)); - assert!(row.form.contains("&10[1][fixed #A(&11[1]int) | &12[1]$0]"), "{}", row.form); - assert_eq!(equations(&row.form), ["= n0 := &0[G][fixed #A(&1[G]int) | &2[G]$0]"]); + assert!( + row.form.contains("&10[1][fixed #A(&11[1]int) | &12[1]$0]"), + "{}", + row.form + ); + assert_eq!( + equations(&row.form), + ["= n0 := &0[G][fixed #A(&1[G]int) | &2[G]$0]"] + ); let mut twice = refine(3, 2); - twice.extend([Op::Child(2, 1), Op::Instance(3), Op::Child(5, 2), Op::Gadt(4, 6, 3)]); + twice.extend([ + Op::Child(2, 1), + Op::Instance(3), + Op::Child(5, 2), + Op::Gadt(4, 6, 3), + ]); assert_eq!( equations(&old_pattern(&twice).form), - ["= n0 := &0[G](&1[G]$0, &2[G]int)", "= n1 := &3[G](&4[G]$1, &5[G]int)"] + [ + "= n0 := &0[G](&1[G]$0, &2[G]int)", + "= n1 := &3[G](&4[G]$1, &5[G]int)" + ] ); } @@ -4730,20 +4791,21 @@ mod tests { let below = old_pattern(&ops); assert_eq!(below.failed, Some(5)); assert_eq!(trace_of(&below.form, 5), ["@7 ~ @3", "@1 ~ &14[0]'"]); - assert_eq!( - below.form.lines().last(), - Some("3 = @7"), - "{}", - below.form - ); + assert_eq!(below.form.lines().last(), Some("3 = @7"), "{}", below.form); assert!( - below.form.contains("\n0 = &0[0]fn(&1[0]$0, &2[0]', &3[0](@1, &4[0]int)) => &5[0]unit\n"), + below + .form + .contains("\n0 = &0[0]fn(&1[0]$0, &2[0]', &3[0](@1, &4[0]int)) => &5[0]unit\n"), "{}", below.form ); let row = old_pattern(&pattern(5, 0, 2, 3)); assert_eq!(row.failed, Some(4)); - assert!(row.form.contains("&1[1][fixed #A(&2[1]n0) | &3[1]$0]"), "{}", row.form); + assert!( + row.form.contains("&1[1][fixed #A(&2[1]n0) | &3[1]$0]"), + "{}", + row.form + ); let partial = old_pattern(&pattern(6, 0, 1, 3)); assert_eq!(partial.failed, Some(4)); assert_eq!(trace_of(&partial.form, 4), ["@1 ~ @4", "@3 ~ @6"]); @@ -4805,7 +4867,9 @@ mod tests { let rows = pattern(4, 0, 1, 3); let linked = run_old(&program, plan, &rows); assert!( - linked.form.starts_with("0 = &0[1]fn(&1[1][closed #A(&2[1]int) #B | &3[1]nil], @1,"), + linked + .form + .starts_with("0 = &0[1]fn(&1[1][closed #A(&2[1]int) #B | &3[1]nil], @1,"), "{}", linked.form ); @@ -4839,7 +4903,8 @@ mod tests { Op::Generalize(1), Op::Copy(1), ]; - let outcome = compare(0, &program, plan, &ops, Fault::None).unwrap_or_else(|report| panic!("{report}")); + let outcome = compare(0, &program, plan, &ops, Fault::None) + .unwrap_or_else(|report| panic!("{report}")); assert_eq!(outcome.failed, Some(9)); assert!(outcome.form.contains("k<&"), "{}", outcome.form); assert_eq!(equations(&outcome.form).len(), 0); @@ -4850,8 +4915,10 @@ mod tests { let case_seed = seed * 1_000_003 + sequence; let (program, ops) = case(plan, case_seed); patterns += ops.iter().filter(|op| matches!(op, Op::Gadt(..))).count(); - let ops: Vec = - ops.into_iter().filter(|op| !matches!(op, Op::Gadt(..))).collect(); + let ops: Vec = ops + .into_iter() + .filter(|op| !matches!(op, Op::Gadt(..))) + .collect(); if let Err(report) = compare(case_seed, &program, plan, &ops, Fault::None) { panic!("{report}"); } @@ -4936,13 +5003,85 @@ mod tests { let old = run_old(&program, plan, ops); if compare(case_seed, &program, plan, ops, Fault::ExpressionMode).is_err() { let new = run_new(&program, plan, ops, Fault::ExpressionMode); - equation += usize::from(equations(&old.form).len() > 0); + equation += usize::from(!equations(&old.form).is_empty()); refused += usize::from(new.failed.is_some() && old.failed != new.failed); } } } println!("expression mode fault: {cases} {equation} {refused}"); - assert!(cases >= 75 && equation >= 7 && refused >= 19, "{cases} {equation} {refused}"); + assert!( + cases >= 75 && equation >= 7 && refused >= 19, + "{cases} {equation} {refused}" + ); + } + + /// The acceptance test of pattern mode on generated cases (section + /// 8.9): 10,000 cases of the pattern plan, each run on the + /// operations the old checker is a judge of ([`old_judges`]), agree, + /// in the success mode and, 2,000 of them, in the explaining mode; + /// and the fixed cases of the observation tests above agree + /// (equations and their direction, the arguments of one path by the + /// flags, `reify`, what a failure leaves, rows that clash). No seed + /// is pinned: the design turn had no build to run them against, so a + /// seed that differs is read against the OCaml lines of 8.9 before + /// anything is pinned (the build prompt says how). + /// + /// Reaches: R3m, R3e, R3f, R3n, R3h, R3r. + #[test] + #[ignore = "R3r build"] + fn pattern_plan_agrees_on_generated_cases() { + let program = compile(PATTERN_PROGRAM.to_string()); + let plan = Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN); + let mut chain = pattern(0, 0, 2, 3); + chain.extend([Op::Child(0, 1), Op::Gadt(3, 1, 3)]); + let mut twice = refine(3, 2); + twice.extend([ + Op::Child(2, 1), + Op::Instance(3), + Op::Child(5, 2), + Op::Gadt(4, 6, 3), + ]); + let mut compatible = refine(5, 3); + compatible.extend([Op::Child(0, 4), Op::Gadt(1, 4, 3)]); + let mut again = pattern(6, 2, 3, 3); + again.extend([Op::Child(1, 0), Op::Child(2, 0), Op::Unify(3, 4)]); + let below = vec![ + Op::Instance(3), + Op::Child(0, 2), + Op::BeginDef, + Op::Instance(0), + Op::Child(2, 0), + Op::Gadt(3, 1, 3), + ]; + let mut cases: Vec> = vec![chain, twice, compatible, again, below, refine(5, 3)]; + for (value, left, right) in [ + (0, 0, 2), + (0, 2, 0), + (0, 0, 1), + (0, 1, 0), + (0, 0, 3), + (0, 2, 3), + (1, 0, 1), + (1, 0, 3), + (2, 0, 1), + (4, 0, 2), + (4, 0, 3), + (5, 0, 2), + (6, 0, 1), + (6, 2, 3), + ] { + for flags in 0..4 { + cases.push(pattern(value, left, right, flags)); + } + } + for (index, ops) in cases.iter().enumerate() { + compare(index as u64, &program, plan, ops, Fault::None) + .unwrap_or_else(|e| panic!("{e}")); + } + let coverage = run_cases_except(Plan::new(Tier::Full, PATTERN_PLAN), &[]); + assert_eq!(coverage.cases, 10_000); + assert!(coverage.failures * 10 > coverage.cases, "{coverage:?}"); + run_cases(Plan::new(Tier::Full, PATTERN_PLAN | EXPLAIN), 1, 20, 100); } /// 10,000 cases of everything on programs with polymorphic fields, diff --git a/compiler-rust/src/tc/expand.rs b/compiler-rust/src/tc/expand.rs index 32243fbadb..16c5cc9789 100644 --- a/compiler-rust/src/tc/expand.rs +++ b/compiler-rust/src/tc/expand.rs @@ -124,6 +124,11 @@ pub struct TypeTable { /// Number of rows with a non-zero `scope`: when 0, lowering never tests a /// scope ([`Checker::lower`]). pub scoped: u32, + /// Per row of `rows`: bit `i` is set when parameter `i` is injective + /// (`Variance.(mem Inj)`). Empty, or shorter than `rows`, for types + /// without an injective parameter ([`TypeTable::injective`]). Filled + /// by the caller that owns the table (R5a). (R3h) + pub injective_bits: Vec, } impl TypeTable { @@ -313,6 +318,19 @@ impl TypeTable { } } + /// `Variance.(mem Inj)` of parameter `parameter` of the type + /// (compiler/ml/ctype.ml:2361, 2018): bit `parameter` of + /// `injective_bits[type_index]`; a type past the end of that table + /// and a parameter past the 32nd are not injective. O(1): one bounds + /// test and one load. (R3h; CHECKER_REWRITE 8.9, decision 66) + #[inline] + pub fn injective(&self, type_index: u32, parameter: u32) -> bool { + match self.injective_bits.get(type_index as usize) { + Some(&bits) => parameter < 32 && bits >> parameter & 1 != 0, + None => false, + } + } + /// O(1): `type_index` is a bare identifier for the unit being checked /// ([`LOCAL`]). One bounds test and one byte written. pub fn set_local(&mut self, type_index: u32) { diff --git a/compiler-rust/src/tc/r3_tests.rs b/compiler-rust/src/tc/r3_tests.rs index b21b4a5ae9..5f9e0c1052 100644 --- a/compiler-rust/src/tc/r3_tests.rs +++ b/compiler-rust/src/tc/r3_tests.rs @@ -3357,11 +3357,17 @@ fn the_types_that_hold_a_universal_variable_are_unified_not_gained() { fn conjunctive_row(c: &mut Checker, types: &[Ty], more: Ty) -> Ty { let name = name_index(c, "A"); let start = c.extras.row_fields.len() as u32; - let kind = RowFieldKind::Either { constant: false, matched: false }; + let kind = RowFieldKind::Either { + constant: false, + matched: false, + }; let published = types.len() as u32 + 1; c.extras.row_fields.push(RowFieldRow { name, - types: Range { start: 0, len: types.len() as u32 }, + types: Range { + start: 0, + len: types.len() as u32, + }, link: published, kind, }); @@ -3460,7 +3466,10 @@ fn named_row(c: &mut Checker, level: u32, tag: &str, argument: Option) -> Ty fields: Range { start, len: 1 }, more: 0, name: if argument.is_some() { PA } else { NONE }, - name_arguments: Range { start: 1, len: u32::from(argument.is_some()) }, + name_arguments: Range { + start: 1, + len: u32::from(argument.is_some()), + }, closed: false, fixed: false, }); @@ -3539,7 +3548,11 @@ fn the_arguments_of_a_stale_row_name_are_not_children() { let named = stale(&mut c, y); c.g.end_def(); c.generalize(named); - assert_eq!((level(&mut c, named), level(&mut c, y)), (GENERIC, 2), "generalize"); + assert_eq!( + (level(&mut c, named), level(&mut c, y)), + (GENERIC, 2), + "generalize" + ); let mut c = checker(mode); let y = var(&mut c); @@ -3571,7 +3584,10 @@ fn the_arguments_of_a_stale_row_name_are_not_children() { let first = c.g.nodes.len(); let copy = c.copy(named); assert_eq!(shape(&mut c, copy), "[A B | ']"); - let variables = c.g.nodes[first..].iter().filter(|node| node.tag() == VAR).count(); + let variables = c.g.nodes[first..] + .iter() + .filter(|node| node.tag() == VAR) + .count(); assert_eq!(variables, 1, "copy"); } // The innermost row's name: its argument is a child. @@ -3625,7 +3641,10 @@ fn a_free_universal_variable_is_found_in_a_row_in_order() { fields: Range { start, len: 1 }, more: 1, name: if named.is_some() { PA } else { NONE }, - name_arguments: Range { start: 2, len: u32::from(named.is_some()) }, + name_arguments: Range { + start: 2, + len: u32::from(named.is_some()), + }, closed: false, fixed: false, }); @@ -3646,7 +3665,14 @@ fn a_free_universal_variable_is_found_in_a_row_in_order() { assert_eq!(c.free_univar(bound), Ty(NONE)); let more = var(&mut c); - let either = variant_at(&mut c, 2, &[("A", Tag::Either(false, Some(u0), false))], more, true, NONE); + let either = variant_at( + &mut c, + 2, + &[("A", Tag::Either(false, Some(u0), false))], + more, + true, + NONE, + ); c.open_row(either); assert_eq!(c.free_univar(either), u0); let (tags, _, _) = read_row(&mut c, either); @@ -3772,7 +3798,13 @@ fn the_walks_for_universal_variables_at_ten_thousand() { let (n1, n2) = (c.g.nodes[outer1.0 as usize], c.g.nodes[outer2.0 as usize]); let partners = c.g.push(partner_node(Ty(NONE))); c.g.push(partner_node(Ty(NONE))); - c.scopes.push(PolyScope { left: n1.b + 1, right: n2.b + 1, left_len: 1, right_len: 1, partners }); + c.scopes.push(PolyScope { + left: n1.b + 1, + right: n2.b + 1, + left_len: 1, + right_len: 1, + partners, + }); let binder = poly_of(&mut c, 2, u, &[u]); let w = univar_at(&mut c, 2); let elements: Vec = (0..n).map(|_| var(&mut c)).collect(); @@ -4177,3 +4209,731 @@ fn packages_of_ten_thousand_names_are_linear() { let (small, large) = (run(1_000), run(10_000)); assert!(large <= 11 * small, "{small} {large}"); } + +// --------------------------------------------------------------------------- +// Pattern mode (R3h, R3r; CHECKER_REWRITE 8.9) +// --------------------------------------------------------------------------- + +use super::expand::{DATATYPE, NEWTYPE, Newtype}; +use super::rows::{DeclMatch, LocalDecls}; +use super::unify::UnifyMode; + +/// `(type n0)` at level 1 and `(type n1)` at level 2: locally abstract. +const N0: u32 = 14; +const N1: u32 = 15; +/// `type k<'a>` of another unit: abstract, not injective. +const K: u32 = 16; +/// `type d<'a> = D('a)` and `type e<'a> = E('a)` of another unit. +const D: u32 = 17; +const E: u32 = 18; +/// `type t2` of another unit: abstract, as `T`. +const T2: u32 = 19; +/// The first type `reify` makes. +const FRESH: u32 = 20; + +/// The supplier of the hand-built tests: counts the fresh types from +/// [`FRESH`], records the name index each was asked with, and answers +/// `compare` with `answer` after pushing `members`. +struct Fresh { + next: u32, + asked: std::rc::Rc>>, + answer: DeclMatch, + members: Vec<[Ty; 2]>, +} + +impl LocalDecls for Fresh { + fn fresh_abstract(&mut self, name: u32) -> u32 { + self.asked.borrow_mut().push(name); + self.next += 1; + self.next - 1 + } + + fn compare(&mut self, c: &mut Checker, _: u32, _: u32) -> DeclMatch { + for &[left, right] in &self.members { + c.push_compatible(left, right); + } + self.answer + } +} + +/// [`checker`] with the declarations of pattern mode: `int`, `string` +/// and `unit` are bare identifiers (`LOCAL`), `n0` and `n1` locally +/// abstract types of the newtype levels 1 and 2, `k`, `d`, `e` and `t2` +/// types of another unit, and a supplier whose record of asked names is +/// returned. +fn pattern_checker(mode: Mode) -> (Checker, std::rc::Rc>>) { + let mut c = checker(mode); + for ty in [INT, STRING, UNIT] { + c.types.set_local(ty); + } + let row = |arity: u16, flags: u8| TypeInfo { + manifest: NONE, + scope: 0, + newtype: NONE, + arity, + flags, + reserved: 0, + }; + for level in [1, 2] { + c.types.newtypes.push(Newtype { + level, + expansion_level: level, + }); + c.types.rows.push(TypeInfo { + scope: level, + newtype: c.types.newtypes.len() as u32 - 1, + ..row(0, NEWTYPE) + }); + c.types.scoped += 1; + } + c.types + .rows + .extend([row(1, 0), row(1, DATATYPE), row(1, DATATYPE), row(0, 0)]); + assert_eq!(c.types.rows.len() as u32, FRESH); + let asked = std::rc::Rc::new(std::cell::RefCell::new(Vec::new())); + c.locals = Some(Box::new(Fresh { + next: FRESH, + asked: asked.clone(), + answer: DeclMatch::Clash, + members: Vec::new(), + })); + (c, asked) +} + +/// A variable at `level` whose name is `text`. +fn named_var_at(c: &mut Checker, level: u32, text: &str) -> Ty { + let name = name_index(c, text); + let ty = var_at(c, level); + c.g.nodes[ty.0 as usize].a = name; + ty +} + +/// The checker is as a pattern unification must leave it, on either +/// result: expression mode, both flags off, no newtype level, no pair +/// assumed, no frame, no open attempt. +fn assert_pattern_left(c: &Checker) { + assert!(c.umode == UnifyMode::Expression); + assert!(!c.generate_equations && !c.assume_injective); + assert_eq!(c.newtype_level, 0); + assert!(c.pairs.keys.is_empty() && c.compatible.keys.is_empty()); + assert!(c.frames.is_empty() && c.tries.is_empty() && c.scopes.is_empty()); +} + +/// Witness: `let f = (type a, x: t) => switch x { | Int => ... }` +/// with `type rec t<_> = Int: t`: the result `t` of the +/// constructor against `t`, at the arguments. A locally abstract type +/// against a type that is not one gets the equation on either side +/// (compiler/ml/ctype.ml:2385-2392), the two nodes stay two nodes, and +/// the type is no longer `instantiable`. Under the equation an +/// expression unification of the two succeeds by expansion. After +/// `cut_equations(0)` the row is what it was and a new `n0` does not +/// unify with `int`. +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn an_equation_goes_to_the_locally_abstract_type_and_links_nothing() { + for swapped in [false, true] { + let (mut c, _) = pattern_checker(Mode::Success); + let before = c.types.rows[N0 as usize]; + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let pair = if swapped { [int, n0] } else { [n0, int] }; + assert!(c.unify_pattern(2, true, true, pair[0], pair[1]).is_ok()); + assert_pattern_left(&c); + assert_eq!(c.equations.len(), 1); + assert_eq!( + (c.equations[0].type_index, c.equations[0].previous), + (N0, before) + ); + let row = c.types.rows[N0 as usize]; + assert_eq!(row.flags & (ABBREV | NEWTYPE), ABBREV | NEWTYPE); + assert!(!c.types.instantiable(N0)); + assert_eq!((c.g.repr(n0), c.g.repr(int)), (n0, int)); + let (other, target) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + assert!(c.unify(other, target).is_ok()); + c.cut_equations(0); + assert!(c.equations.is_empty()); + assert_eq!(c.types.rows[N0 as usize], before); + let (other, target) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + assert!(c.unify(other, target).is_err()); + } +} + +/// Witness: `(type a, type b, x: eq) => switch x { | Refl => ... }` +/// with `Refl: eq<'c, 'c>`, `b` declared inside `a`. Of two locally +/// abstract types the one of the higher newtype level gets the equation, +/// on either side (compiler/ml/ctype.ml:2373-2384): `n1` (level 2), with +/// `n0` (level 1) its body. A locally abstract type against an abstract +/// type of another unit (`find_newtype_level` gives `(0, 0)` for it, +/// ctype.ml:2085-2091): the locally abstract one. Two abstract types of +/// another unit have one level: the RIGHT one gets it (`else (path', +/// t1')`). +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn of_two_instantiable_types_the_newer_gets_the_equation() { + for (left, right, source) in [ + (N0, N1, N1), + (N1, N0, N1), + (N0, T, N0), + (T, N0, N0), + (T, T2, T2), + (T2, T, T), + ] { + let (mut c, _) = pattern_checker(Mode::Success); + let (l, r) = (constr(&mut c, left, &[]), constr(&mut c, right, &[])); + assert!(c.unify_pattern(2, true, true, l, r).is_ok()); + assert_eq!(c.equations.len(), 1, "{left} {right}"); + assert_eq!(c.equations[0].type_index, source, "{left} {right}"); + let other = if source == left { right } else { left }; + let node = constr(&mut c, source, &[]); + let expansion = c.expand_head(node, ExpandMode::Unif).unwrap(); + assert_eq!(path(&mut c, expansion), other); + } +} + +/// Witness: `Pair: t<(int, string)>` against `t`. The body of an +/// equation is a generic copy (`duplicate_type destination`, +/// compiler/ml/ctype.ml:2095): every node of it is generic, it is not +/// the destination, and the destination has the levels it had. An +/// expansion of the type is a copy at the level of the node expanded. +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn the_body_of_an_equation_is_a_generic_copy() { + let (mut c, _) = pattern_checker(Mode::Success); + let (int, string) = (constr(&mut c, INT, &[]), constr(&mut c, STRING, &[])); + let destination = tuple(&mut c, &[int, string]); + let n0 = constr(&mut c, N0, &[]); + assert!(c.unify_pattern(2, true, true, n0, destination).is_ok()); + let manifest = c.types.manifests[c.types.rows[N0 as usize].manifest as usize]; + let body = manifest.body; + assert_ne!(body, destination); + for ty in [body, child(&mut c, body, 0), child(&mut c, body, 1)] { + assert_eq!(c.g.nodes[ty.0 as usize].level, GENERIC); + } + for ty in [destination, int, string] { + assert_eq!(c.g.nodes[ty.0 as usize].level, 2); + } + let expansion = c.expand_head(n0, ExpandMode::Unif).unwrap(); + assert!(expansion != body && expansion != destination); + assert_eq!(c.g.nodes[expansion.0 as usize].level, 2); + let second = child(&mut c, expansion, 1); + assert_eq!(path(&mut c, second), STRING); +} + +/// Witness: `Pack: t<('x, ('y, 'x))>` against `t` after the instance +/// of the constructor: its variables are at the newtype level. `reify` +/// makes one new abstract type for each variable, in the order +/// `iter_type_expr` visits them, each once (compiler/ml/ctype.ml: +/// 1828-1840): the supplier is asked with the name of `x`, then of `y`. +/// Each variable is a link to a nullary constructor of its type at the +/// variable's level; the type's row is a locally abstract type of the +/// newtype level (`new_declaration (Some (newtype_level, newtype_level)) +/// None`, ctype.ml:1816). +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn reify_makes_one_type_for_each_variable_in_visit_order() { + let (mut c, asked) = pattern_checker(Mode::Success); + let (x, y) = (named_var_at(&mut c, 2, "x"), named_var_at(&mut c, 3, "y")); + let inner = tuple(&mut c, &[y, x]); + let destination = tuple(&mut c, &[x, inner]); + let n0 = constr(&mut c, N0, &[]); + assert!(c.unify_pattern(2, true, true, n0, destination).is_ok()); + assert_eq!( + *asked.borrow(), + [name_index(&mut c, "x"), name_index(&mut c, "y")] + ); + for (variable, level, ty) in [(x, 2, FRESH), (y, 3, FRESH + 1)] { + let node = c.g.repr(variable); + assert_ne!(node, variable); + assert_eq!(path(&mut c, node), ty); + assert_eq!(c.g.nodes[node.0 as usize].level, level); + let row = c.types.rows[ty as usize]; + assert_eq!((row.flags, row.arity), (NEWTYPE, 0)); + let newtype = c.types.newtypes[row.newtype as usize]; + assert_eq!((newtype.level, newtype.expansion_level), (2, 2)); + } + assert_eq!(c.equations.len(), 1); +} + +/// Witness: a variable of the scrutinee's type that is older than the +/// pattern (`(x: t<'a>) => switch x { | Int => ... }` inside a function +/// whose `'a` is not generalized yet). `reify` links the variable to its +/// new type and THEN refuses it, because it is below the newtype level +/// (`link_type ty t; if ty.level < newtype_level then raise (Unify [(t, +/// newvar2 ty.level)])`, compiler/ml/ctype.ml:1837-1839): the +/// unification fails, the variable stays a link (observed on the old +/// checker: decision 62), no equation is added, and in the explaining +/// mode the last pair of the trace is the new type and a new variable at +/// the variable's level. +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn reify_links_a_variable_below_the_newtype_level_and_clashes() { + for mode in [Mode::Success, Mode::Explain] { + let (mut c, _) = pattern_checker(mode); + let old = var_at(&mut c, 1); + let int = constr(&mut c, INT, &[]); + let destination = tuple(&mut c, &[old, int]); + let n0 = constr(&mut c, N0, &[]); + assert!(c.unify_pattern(2, true, true, n0, destination).is_err()); + assert_pattern_left(&c); + assert!(c.equations.is_empty()); + let node = c.g.repr(old); + assert_ne!(node, old); + assert_eq!(path(&mut c, node), FRESH); + assert_eq!(c.g.nodes[node.0 as usize].level, 1); + if mode == Mode::Explain { + let [left, right] = *c.trace.last().unwrap(); + assert_eq!(left, node); + let variable = c.g.nodes[right.0 as usize]; + assert_eq!((variable.tag(), variable.level), (VAR, 1)); + } + } +} + +/// Witness: `Int: t` against `t>`'s argument and its +/// relatives: two constructors of one type in pattern mode +/// (compiler/ml/ctype.ml:2349-2358). As `unify_gadt` runs it +/// (`assume_injective`) the arguments are unified with equations: +/// `k` against `k` gives `n0 := int`. Without +/// `assume_injective` a data type (`d`) and a type of the unit (`LOCAL`, +/// here set on `box`) still do. Without `generate_equations` the +/// arguments are unified without equations: `n0` against `int` is then +/// reified and nothing more (ctype.ml:2393-2397): a success and no +/// equation. The two constructor nodes are never linked, and the flags +/// are what they were afterwards (`set_mode_pattern` puts them back, +/// ctype.ml:256-273). +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn the_arguments_of_one_type_are_unified_by_the_flags() { + for (ty, generate, injective, equation) in [ + (K, true, true, true), + (D, true, false, true), + (BOX, true, false, true), + (K, false, true, false), + (K, false, false, false), + (D, false, false, false), + ] { + let (mut c, _) = pattern_checker(Mode::Success); + c.types.set_local(BOX); + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let (left, right) = (constr(&mut c, ty, &[n0]), constr(&mut c, ty, &[int])); + assert!(c.unify_pattern(2, generate, injective, left, right).is_ok()); + assert_pattern_left(&c); + assert_eq!( + c.equations.len(), + usize::from(equation), + "{ty} {generate} {injective}" + ); + assert_eq!((c.g.repr(left), c.g.repr(right)), (left, right)); + assert_eq!((c.g.repr(n0), c.g.repr(int)), (n0, int)); + } +} + +/// Witness: `Pair: t<(int, int)>` against `t<(a, int)>`. Two structures +/// in pattern mode are assumed equal and NOT linked (`Pattern -> +/// add_type_equality t1' t2'`, compiler/ml/ctype.ml:2327-2331): the old +/// checker links them and is no judge here (decision 62). The two +/// tuples are two nodes afterwards, the equation `n0 := int` is added, +/// and the table of assumed pairs is empty again. A clash below +/// (`(n0, int)` against `(int, string)`) leaves both tuples as they +/// were, in both modes, and the equation its first element added stays +/// until the caller cuts it (`env` is not restored by a failure). +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn two_structures_are_assumed_equal_and_not_linked() { + for mode in [Mode::Success, Mode::Explain] { + for clash in [false, true] { + let (mut c, _) = pattern_checker(mode); + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let second = constr(&mut c, if clash { STRING } else { INT }, &[]); + let (left, right) = (tuple(&mut c, &[n0, int]), tuple(&mut c, &[int, second])); + let (l, r) = (c.g.nodes[left.0 as usize], c.g.nodes[right.0 as usize]); + let result = c.unify_pattern(2, true, true, left, right); + assert_eq!(result.is_err(), clash); + assert_pattern_left(&c); + assert_eq!( + (c.g.nodes[left.0 as usize], c.g.nodes[right.0 as usize]), + (l, r) + ); + assert_eq!(c.equations.len(), 1); + assert_eq!(c.equations[0].type_index, N0); + if clash && mode == Mode::Explain { + assert_eq!(c.trace.first(), Some(&[left, right])); + } + } + } +} + +/// Witness: `(type a, type b, x: eq, y: a): b => switch x { | Refl +/// => y }`: in the case, `a` against `b` in expression mode under `b := +/// a`. The arm of `unify` for two locally abstract types under local +/// constraints expands the NEWER one once and no more +/// (compiler/ml/ctype.ml:2258-2270): `n1` expands to a copy of `n0`, +/// which is linked to the other `n0` node by the arm for two nullary +/// constructors of one type, and that node is still `n0`, not its own +/// expansion `int`. +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn the_newer_of_two_locally_abstract_types_is_expanded_once() { + let (mut c, _) = pattern_checker(Mode::Success); + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + assert!(c.unify_pattern(2, true, true, n0, int).is_ok()); + let (n1, older) = (constr(&mut c, N1, &[]), constr(&mut c, N0, &[])); + assert!(c.unify_pattern(2, true, true, n1, older).is_ok()); + assert_eq!(c.equations.len(), 2); + assert_eq!(c.equations[1].type_index, N1); + let (left, right) = (constr(&mut c, N1, &[]), constr(&mut c, N0, &[])); + assert!(c.unify(left, right).is_ok()); + assert_eq!(path(&mut c, right), N0); + assert_eq!(c.g.repr(left), left); +} + +/// Witness: `Wrap: t<(a, int)>` against `t` for the same `a`: an +/// equation of a type with a type that names it. `add_gadt_equation` +/// adds nothing and does not clash (`if local_non_recursive_abbrev !env +/// source destination then ...`, compiler/ml/ctype.ml:2093-2094). +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn a_recursive_equation_is_not_added() { + let (mut c, _) = pattern_checker(Mode::Success); + let (inner, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let destination = tuple(&mut c, &[inner, int]); + let n0 = constr(&mut c, N0, &[]); + assert!(c.unify_pattern(2, true, true, n0, destination).is_ok()); + assert!(c.equations.is_empty()); + assert_eq!(c.types.rows[N0 as usize].flags, NEWTYPE); +} + +/// Witness: a pattern whose type has 10,000 components. Relations +/// between 1,000 and 10,000 (CHECKER_REWRITE 8.9, "Step tests"): +/// - a chain of n nested tuples against a second chain of the same +/// shape in pattern mode: the steps are linear in the pairs, no node +/// of either chain is a link, the table of assumed pairs is empty; +/// - `reify` of a tuple of n variables, each named twice: every +/// variable is visited once, steps linear, n types made; +/// - n equations added and cut: steps linear, every row what it was. +/// +/// Reaches: R3h. +#[test] +#[ignore = "R3h build"] +fn pattern_mode_at_ten_thousand() { + let chain = |c: &mut Checker, n: u32| { + let mut ty = constr(c, INT, &[]); + for _ in 0..n { + let int = constr(c, INT, &[]); + ty = tuple(c, &[int, ty]); + } + ty + }; + let structures = |n: u32| { + let (mut c, _) = pattern_checker(Mode::Success); + let (left, right) = (chain(&mut c, n), chain(&mut c, n)); + let nodes = c.g.nodes.len(); + let mut result = Err(super::unify::Clash); + let count = steps::scope(|| result = c.unify_pattern(2, true, true, left, right)); + assert!(result.is_ok()); + assert_pattern_left(&c); + // The nullary `int` pairs are linked in every mode + // (ctype.ml:2248-2257); no tuple is. + for index in 0..nodes { + let node = c.g.nodes[index]; + assert!(node.tag() != LINK || path(&mut c, Ty(index as u32)) == INT); + } + count + }; + let (small, large) = (structures(1_000), structures(10_000)); + assert!(large <= 11 * small, "structures: {small} {large}"); + + let reified = |n: u32| { + let (mut c, asked) = pattern_checker(Mode::Success); + let variables: Vec = (0..n).map(|_| var(&mut c)).collect(); + let mut elements = variables.clone(); + elements.extend(&variables); + let ty = tuple(&mut c, &elements); + c.newtype_level = 2; + let mut result = Err(super::unify::Clash); + let count = steps::scope(|| result = c.reify(ty)); + assert!(result.is_ok()); + assert_eq!(asked.borrow().len(), n as usize); + count + }; + let (small, large) = (reified(1_000), reified(10_000)); + assert!(large <= 11 * small, "reify: {small} {large}"); + + let equations = |n: u32| { + let (mut c, _) = pattern_checker(Mode::Success); + let first = c.types.rows.len() as u32; + for _ in 0..n { + let row = c.types.rows[T as usize]; + c.types.rows.push(row); + } + let before = c.types.rows.clone(); + let count = steps::scope(|| { + for index in 0..n { + let (ty, int) = (constr(&mut c, first + index, &[]), constr(&mut c, INT, &[])); + assert!(c.unify_pattern(2, true, true, ty, int).is_ok()); + } + assert_eq!(c.equations.len(), n as usize); + c.cut_equations(0); + }); + assert_eq!(c.types.rows, before); + count + }; + let (small, large) = (equations(1_000), equations(10_000)); + assert!(large <= 11 * small, "equations: {small} {large}"); +} + +/// Witness: `Int: t>` against `t>` below a constructor +/// whose arguments were already entered (`generate_equations` on, +/// `assume_injective` off), `k` abstract and of another unit. Its +/// argument is not injective: it is unified without equations under a +/// snapshot, and when that fails the graph goes back and both arguments +/// are reified (compiler/ml/ctype.ml:2359-2371). +/// - `k` against `k`: no equation, a success. +/// - `k<'v>` against `k`: the attempt succeeds and is kept: `'v` +/// is a link to `int`. +/// - `k<('v, int)>` against `k`: the attempt fails; `'v` is not +/// bound by it and is reified: a link to the first fresh type. +/// - with the parameter marked injective (`injective_bits`), `k` +/// against `k` gives `n0 := int`. +/// +/// No attempt is left open and, in `Mode::Success`, no speculation. +/// +/// Reaches: R3h, R3r. +#[test] +#[ignore = "R3r build"] +fn an_argument_that_is_not_injective_is_tried_and_then_reified() { + for mode in [Mode::Success, Mode::Explain] { + let (mut c, _) = pattern_checker(mode); + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let (left, right) = (constr(&mut c, K, &[n0]), constr(&mut c, K, &[int])); + assert!(c.unify_pattern(2, true, false, left, right).is_ok()); + assert_pattern_left(&c); + assert!(c.equations.is_empty()); + assert_eq!(c.g.speculation_depth, 0); + + let (v, int) = (var(&mut c), constr(&mut c, INT, &[])); + let (left, right) = (constr(&mut c, K, &[v]), constr(&mut c, K, &[int])); + assert!(c.unify_pattern(2, true, false, left, right).is_ok()); + assert_eq!(c.g.repr(v), int); + + let (v, int, string) = ( + var(&mut c), + constr(&mut c, INT, &[]), + constr(&mut c, STRING, &[]), + ); + let pair = tuple(&mut c, &[v, int]); + let (left, right) = (constr(&mut c, K, &[pair]), constr(&mut c, K, &[string])); + assert!(c.unify_pattern(2, true, false, left, right).is_ok()); + assert_pattern_left(&c); + let node = c.g.repr(v); + assert_eq!(path(&mut c, node), FRESH); + assert_eq!(c.g.speculation_depth, 0); + + c.types.injective_bits = vec![0; K as usize + 1]; + c.types.injective_bits[K as usize] = 1; + let (n0, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); + let (left, right) = (constr(&mut c, K, &[n0]), constr(&mut c, K, &[int])); + assert!(c.unify_pattern(2, true, false, left, right).is_ok()); + assert_eq!(c.equations.len(), 1); + } +} + +/// Witness: a constructor whose result is `t<[> #A]>` against a +/// scrutinee `t<[> #A | #B]>` whose row was fixed by an earlier case. +/// Two variants in pattern mode are unified under a snapshot; when that +/// fails the graph goes back, both are reified and, with equations, +/// compared with `mcomp` (compiler/ml/ctype.ml:2400-2408). +/// - a fixed row `[> #A]` (made by `reify`) against an open row with +/// one more tag: `unify_row` refuses to extend the fixed row, `mcomp` +/// accepts (no row is closed): a success, the two rows are two nodes, +/// the open one is a fixed row afterwards (observed on the old +/// checker: decision 62); +/// - two closed rows `[#A]` and `[#B]`: `mcomp` refuses (a present tag +/// the other closed row lacks, ctype.ml:1988-1995): a clash; +/// - the same without `generate_equations`: no `mcomp`, a success. +/// +/// Reaches: R3f, R3h, R3r. +#[test] +#[ignore = "R3r build"] +fn rows_that_clash_are_reified_and_compared() { + for mode in [Mode::Success, Mode::Explain] { + let (mut c, _) = pattern_checker(mode); + let (more1, more2) = (var(&mut c), var(&mut c)); + let fixed = variant_at(&mut c, 2, &[("A", Tag::Present(None))], more1, false, NONE); + c.newtype_level = 2; + assert!(c.reify(fixed).is_ok()); + c.newtype_level = 0; + assert!(shape(&mut c, fixed).contains("fixed")); + let open = variant_at( + &mut c, + 2, + &[("A", Tag::Present(None)), ("B", Tag::Present(None))], + more2, + false, + NONE, + ); + assert!(c.unify_pattern(2, true, true, fixed, open).is_ok()); + assert_pattern_left(&c); + assert_eq!((c.g.repr(fixed), c.g.repr(open)), (fixed, open)); + assert!(shape(&mut c, open).contains("fixed")); + assert_eq!(path(&mut c, more2), FRESH + 1); + + for (generate, clash) in [(true, true), (false, false)] { + let (nil1, nil2) = ( + extended_at(&mut c, 2, ExtraDescriptor::Nil, &[]), + extended_at(&mut c, 2, ExtraDescriptor::Nil, &[]), + ); + let left = variant_at(&mut c, 2, &[("A", Tag::Present(None))], nil1, true, NONE); + let right = variant_at(&mut c, 2, &[("B", Tag::Present(None))], nil2, true, NONE); + let result = c.unify_pattern(2, generate, true, left, right); + assert_eq!(result.is_err(), clash); + assert_pattern_left(&c); + assert_eq!((c.g.repr(left), c.g.repr(right)), (left, right)); + } + } +} + +/// Witness: `Int: t` against `t`, `t`, `t>`: +/// what `mcomp` answers (compiler/ml/ctype.ml:1905-2039), one line an +/// arm. It never links: every node is its own representative +/// afterwards, and its table is empty. +/// - a variable against anything: compatible; +/// - `int` against `string`, both bare identifiers that are not locally +/// abstract (`non_aliasable`): a clash; +/// - `M.t`, abstract and of another unit, against `int`: compatible +/// (`Type_abstract, Type_abstract`); +/// - `M.t` against a tuple: compatible; `int` against a tuple: a clash +/// (`non_aliasable p decl || is_datatype decl`); +/// - a tuple against an arrow: a clash; `(int, M.t)` against `(int, +/// myint)`: compatible, through the expansion of `myint`; +/// - `d` against `d`: compatible while the parameter is +/// not injective, a clash when it is; +/// - `d` against `e`, two data types: what the supplier +/// answers: `Clash`; `Members` with no pair: compatible; `Members` +/// after pushing the pair of `int` and `string`: a clash; with no +/// supplier: a clash (CHECKER_REWRITE 8.9, "No supplier"). +/// +/// Reaches: R3r. +#[test] +#[ignore = "R3r build"] +fn mcomp_answers_by_the_declarations() { + let (mut c, _) = pattern_checker(Mode::Success); + let int = |c: &mut Checker| constr(c, INT, &[]); + let (v, i, s, t) = ( + var(&mut c), + int(&mut c), + constr(&mut c, STRING, &[]), + constr(&mut c, T, &[]), + ); + let pair = { + let (a, b) = (int(&mut c), int(&mut c)); + tuple(&mut c, &[a, b]) + }; + let function = { + let (a, b) = (int(&mut c), int(&mut c)); + arrow(&mut c, &[a], b) + }; + let (with_t, with_myint) = { + let (a, b, m) = (int(&mut c), int(&mut c), constr(&mut c, MYINT, &[])); + let t = constr(&mut c, T, &[]); + (tuple(&mut c, &[a, t]), tuple(&mut c, &[b, m])) + }; + let (d_int, d_string) = { + let (a, b) = (int(&mut c), constr(&mut c, STRING, &[])); + (constr(&mut c, D, &[a]), constr(&mut c, D, &[b])) + }; + let nodes = c.g.nodes.len(); + for (left, right, compatible) in [ + (v, pair, true), + (i, s, false), + (t, i, true), + (t, pair, true), + (i, pair, false), + (pair, function, false), + (with_t, with_myint, true), + (d_int, d_string, true), + ] { + assert_eq!( + c.mcomp(left, right).is_ok(), + compatible, + "{left:?} {right:?}" + ); + assert!(c.compatible.keys.is_empty()); + } + for node in &c.g.nodes[..nodes] { + assert_ne!(node.tag(), LINK); + } + c.types.injective_bits = vec![0; D as usize + 1]; + c.types.injective_bits[D as usize] = 1; + assert!(c.mcomp(d_int, d_string).is_err()); + + let e_int = { + let a = int(&mut c); + constr(&mut c, E, &[a]) + }; + let supplier = |answer: DeclMatch, members: Vec<[Ty; 2]>| -> Option> { + Some(Box::new(Fresh { + next: FRESH, + asked: Default::default(), + answer, + members, + })) + }; + assert!(c.mcomp(d_int, e_int).is_err()); + c.locals = supplier(DeclMatch::Members, Vec::new()); + assert!(c.mcomp(d_int, e_int).is_ok()); + c.locals = supplier(DeclMatch::Members, vec![[i, s]]); + assert!(c.mcomp(d_int, e_int).is_err()); + c.locals = None; + assert!(c.mcomp(d_int, e_int).is_err()); + assert!(c.compatible.keys.is_empty()); +} + +/// Witness: two types of 10,000 components that `mcomp` compares. +/// Relation between 1,000 and 10,000: two tuples of n elements whose +/// elements are the SAME n-deep chain pairwise (every element of the +/// left is one node, every element of the right another): the table +/// makes the second and every later pair of the chains one lookup, so +/// the steps are linear in n and not n times the depth. +/// +/// Reaches: R3r. +#[test] +#[ignore = "R3r build"] +fn mcomp_at_ten_thousand() { + let run = |n: u32| { + let (mut c, _) = pattern_checker(Mode::Success); + let chain = |c: &mut Checker| { + let mut ty = constr(c, T, &[]); + for _ in 0..n { + let int = constr(c, INT, &[]); + ty = tuple(c, &[int, ty]); + } + ty + }; + let (a, b) = (chain(&mut c), chain(&mut c)); + let (left, right) = (tuple(&mut c, &vec![a; 100]), tuple(&mut c, &vec![b; 100])); + let mut result = Err(super::unify::Clash); + let count = steps::scope(|| result = c.mcomp(left, right)); + assert!(result.is_ok()); + count + }; + let (small, large) = (run(1_000), run(10_000)); + assert!(large <= 11 * small, "mcomp: {small} {large}"); +} diff --git a/compiler-rust/src/tc/rows.rs b/compiler-rust/src/tc/rows.rs index abb4dc311f..5fd8292ef5 100644 --- a/compiler-rust/src/tc/rows.rs +++ b/compiler-rust/src/tc/rows.rs @@ -312,7 +312,7 @@ pub trait PackageDecls { /// One equation of pattern mode in force (`Env.add_local_constraint`, /// compiler/ml/ctype.ml:2093-2100): the type that got a manifest and the -/// [`TypeInfo`] row it had before, so that cutting the log back puts the +/// `TypeInfo` row it had before, so that cutting the log back puts the /// row back ([`Checker::cut_equations`]). 20 bytes; one per equation /// added and not yet cut. (R3h; CHECKER_REWRITE 8.9) #[derive(Clone, Copy, Debug, PartialEq, Eq)] @@ -1569,13 +1569,22 @@ impl Checker { let extra = self.saved.len() as u32; self.saved.push(node); self.saved.push(node); - self.frames.push(Frame { left: home, right: home, kind: FrameKind::Fields, extra }); + self.frames.push(Frame { + left: home, + right: home, + kind: FrameKind::Fields, + extra, + }); } let b = self.g.lists.len() as u32; - self.g.lists.extend_from_within(node.b as usize..(node.b + node.arity()) as usize); + self.g + .lists + .extend_from_within(node.b as usize..(node.b + node.arity()) as usize); let more = self.g.lists[(b + row.more) as usize]; let arguments = row.name_arguments; - self.g.lists[(b + arguments.start) as usize..(b + arguments.start + arguments.len) as usize].fill(more); + self.g.lists + [(b + arguments.start) as usize..(b + arguments.start + arguments.len) as usize] + .fill(more); self.g.write(home, Node { b, ..node }); } @@ -2206,9 +2215,14 @@ impl Checker { let first = frame.extra as usize + 2; for at in first..first + (counts.head + counts.level) as usize { let ty = Ty(self.saved[at].head); - self.extras.row_scratch.push(FieldRef { name: 0, at: 0, ty }); + self.extras + .row_scratch + .push(FieldRef { name: 0, at: 0, ty }); } - (end..end + counts.head as usize, end + counts.head as usize..self.extras.row_scratch.len()) + ( + end..end + counts.head as usize, + end + counts.head as usize..self.extras.row_scratch.len(), + ) } else { missing(self, middle..end, base..middle); let gained1 = end..self.extras.row_scratch.len(); @@ -3188,7 +3202,12 @@ impl Checker { }); for at in kept1.clone().chain(kept2.clone()) { let ty = self.extras.row_scratch[at].ty; - self.saved.push(Node { head: ty.0, level: 0, a: NONE, b: 0 }); + self.saved.push(Node { + head: ty.0, + level: 0, + a: NONE, + b: 0, + }); } let end = Frame { kind: FrameKind::RowFieldEnd, ..frame }; if self.frames.last() != Some(&end) { @@ -3391,27 +3410,72 @@ impl Checker { if compatible { Ok(()) } else { Err(Clash) } } - /// `reify` (compiler/ml/ctype.ml:1813-1870): replace the variables of a - /// type by fresh abstract types at the newtype level, in pattern mode. - /// O(nodes), marks. (R3h) - pub fn reify(&mut self, ty: Ty) { + /// `reify` (compiler/ml/ctype.ml:1813-1859): every variable `ty` + /// reaches becomes a new locally abstract type of the newtype level, + /// and the row variable of every variant that is neither static nor + /// fixed a fixed row without tags over one; the steps are in + /// CHECKER_REWRITE 8.9 ("`reify`"). Visits each node once (a new mark + /// generation, `g.work`), children in `iter_type_expr`'s order. A + /// variable below [`Checker::newtype_level`] is linked and then a + /// clash (ctype.ml:1837-1839). Needs [`Checker::locals`] when there + /// is a variable. O(nodes reached). (R3h) + pub fn reify(&mut self, ty: Ty) -> Result<(), Clash> { todo!("R3h") } - /// `add_gadt_equation` (compiler/ml/ctype.ml:2093-2100): give the - /// locally abstract type `source` the manifest `destination` when the - /// equation is well-founded, and clean the memos. O(nodes of - /// `destination`). (R3h) + /// `add_gadt_equation` (compiler/ml/ctype.ml:2093-2100): when + /// [`Checker::non_recursive_abbrev`], the locally abstract type + /// `source` gets [`Checker::duplicate`] of `destination` as its + /// manifest, the entry goes to [`Checker::equations`] and the memos + /// are cleaned; the five steps are in CHECKER_REWRITE 8.9, decision + /// 67. Otherwise nothing. O(nodes of `destination`) plus the + /// cleanup. (R3h) pub fn add_gadt_equation(&mut self, source: u32, destination: Ty) { todo!("R3h") } - /// `mcomp` (compiler/ml/ctype.ml:1905-2090): whether two types can have - /// a common instance; never links. O(pairs), pair set, explicit stack. - /// (R3h) - pub fn mcomp(&mut self, left: Ty, right: Ty) -> Result<(), Clash> { + /// `local_non_recursive_abbrev env p ty` (compiler/ml/ctype.ml: + /// 1478-1535): `ty` does not name the type `source`, through its + /// children or through the expansions of the constructors it names + /// ([`Checker::expand_once`] in `ExpandMode::Opt`). Marks and + /// `g.work`; O(nodes reached). (R3h) + pub fn non_recursive_abbrev(&mut self, source: u32, ty: Ty) -> bool { + todo!("R3h") + } + + /// `duplicate_type` for an equation: a copy of `ty` in which every + /// node is generic, sharing kept; `ty` is as it was afterwards. Made + /// of `g.speculate()`, `generalize(ty)` with `g.level` 0, + /// `copy(ty)` with `g.level` generic, and `g.restore`: the copy's + /// nodes are new and not on the trail (CHECKER_REWRITE 8.9, decision + /// 67). Three passes over `ty`. (R3h) + pub fn duplicate(&mut self, ty: Ty) -> Ty { + todo!("R3h") + } + + /// `find_newtype_level env path` (compiler/ml/ctype.ml:2085-2091): + /// the `Newtype` row's `(level, expansion_level)` of a type with + /// `NEWTYPE`, otherwise `(scope, scope)` (`Path.binding_time`). + /// Compared as pairs. A type without a row: `(0, 0)`. O(1). (R3h) + pub fn find_newtype_level(&self, type_index: u32) -> (u32, u32) { todo!("R3h") } + + /// `mcomp` (compiler/ml/ctype.ml:1905-2080): whether two types can + /// have a common instance; never links, never lowers. The arms are + /// in CHECKER_REWRITE 8.9, decision 70. Its table is + /// [`Checker::compatible`], emptied when it returns; its stack is + /// `g.work`. O(pairs). (R3r) + pub fn mcomp(&mut self, left: Ty, right: Ty) -> Result<(), Clash> { + todo!("R3r") + } + + /// One more pair for the `mcomp` in progress: what + /// [`LocalDecls::compare`] calls for each pair of member types. + /// O(1) amortized: two entries of `g.work`. (R3r) + pub fn push_compatible(&mut self, left: Ty, right: Ty) { + todo!("R3r") + } } #[cfg(test)] diff --git a/compiler-rust/src/tc/unify.rs b/compiler-rust/src/tc/unify.rs index 59aa39d112..481ea24475 100644 --- a/compiler-rust/src/tc/unify.rs +++ b/compiler-rust/src/tc/unify.rs @@ -107,6 +107,39 @@ pub enum FrameKind { /// [`Checker::saved`]: the descriptor the left node had. Popped: /// [`Checker::package_end`]. (R3p) PackageEnd, + /// Pattern mode, below the pairs of the arguments of two + /// constructors of one type: `extra` holds the two flags as they + /// were (bit 0 `generate_equations`, bit 1 `assume_injective`). + /// Popped: the flags are put back (`set_mode_pattern`, + /// compiler/ml/ctype.ml:256-273). (R3h; CHECKER_REWRITE 8.9, + /// decision 64) + Flags, + /// Pattern mode, below what an arm tries: `left` and `right` are the + /// two types to reify when it fails, `extra` indexes + /// [`Checker::tries`]. Popped: the attempt succeeded + /// ([`Checker::recover`] says what a clash above it does). (R3r; + /// CHECKER_REWRITE 8.9, decision 68) + Try, +} + +/// One attempt of pattern mode that falls back on a clash (`let snap = +/// snapshot () in try ... with Unify _ -> backtrack snap; reify ...`, +/// compiler/ml/ctype.ml:2362-2371, 2400-2408). 40 bytes; one per open +/// attempt. (R3r; CHECKER_REWRITE 8.9, decision 68) +#[derive(Debug)] +pub struct Try { + /// What the graph is restored to. + pub speculation: super::graph::Speculation, + /// Index of the [`FrameKind::Try`] frame in `frames`. + pub frame: u32, + /// The lengths of `trace`, `saved` and `scopes` when it was opened. + pub trace: u32, + pub saved: u32, + pub scopes: u32, + /// Bit 0 `generate_equations`, bit 1 `assume_injective`, as they + /// were; bit 2: `mcomp` follows the reification when + /// `generate_equations` is on after the flags are put back. + pub flags: u8, } /// Bit 31 of a [`FrameKind::Linked`] frame's `extra`: `unify2` ran `unify3` @@ -184,6 +217,12 @@ pub struct Checker { /// What `reify` and `mcomp` ask about declarations; `None` until a /// caller that has declarations sets it (R5a). (R3h) pub locals: Option>, + /// The open attempts of pattern mode, innermost last; empty outside + /// a pattern unification. (R3r) + pub tries: Vec, + /// The pairs `mcomp` assumes compatible, ordered (left, right) in + /// one `u64` each; empty outside a call of it. (R3r) + pub compatible: PairSet, } impl Checker { @@ -211,6 +250,8 @@ impl Checker { newtype_level: 0, equations: Vec::new(), locals: None, + tries: Vec::new(), + compatible: PairSet::default(), } } @@ -960,7 +1001,8 @@ impl Checker { return self.unify_fields(t1x, t2x); } if self.umode == UnifyMode::Pattern { - // `add_type_equality` in place of the link (ctype.ml:2329). + // `add_type_equality` in place of the link (ctype.ml:2329): + // CHECKER_REWRITE 8.9, decision 64. todo!("R3h") } // Two variant nodes get the slots of their references while both @@ -1274,6 +1316,34 @@ impl Checker { todo!("R3h") } + /// Cut the equations back to the first `len` (a length of + /// [`Checker::equations`] the caller kept), last first: the + /// `TypeTable` row of each popped entry is its `previous` again; + /// then [`Checker::cleanup_abbrev`] when one was popped. What a + /// caller does where OCaml drops the environment of a case + /// (`set_state`, compiler/ml/typecore.ml:1303-1305). O(entries + /// popped) plus the cleanup. (R3h; CHECKER_REWRITE 8.9, decision + /// 67) + pub fn cut_equations(&mut self, len: usize) { + todo!("R3h") + } + + /// A clash reached `run_frames` whose frames start at `base`: `None` + /// when no [`Try`] record has its frame at or above `base` (the + /// clash is the caller's). Otherwise the last record is taken: + /// `frames` is cut to its frame, [`Checker::restore`] of its + /// speculation, `leave_scopes` to its scopes, `trace` and `saved` + /// cut to its lengths, the two flags put back; then `reify(left)`, + /// `reify(right)` and, with bit 2 of the record and + /// `generate_equations` on, `mcomp(left, right)`: `Some` of their + /// result. The equations and the fresh types made inside the attempt + /// stay (`env` is not part of OCaml's snapshot). O(frames above the + /// record + writes restored) plus the three callees. (R3r; + /// CHECKER_REWRITE 8.9, decision 68) + pub fn recover(&mut self, base: usize) -> Option> { + todo!("R3r") + } + /// The failure path of the explaining mode: the trace, outermost pair /// first (each enclosing `unify` adds its pair, ctype.ml:2272), and the /// descriptors of the linked structure nodes enclosing the failure put @@ -1494,6 +1564,13 @@ impl Checker { FrameKind::RowField => self.unify_row_field(frame)?, FrameKind::RowFieldEnd => self.row_field_end(frame)?, FrameKind::RowEnd => self.row_end(frame), + // The two flags go back (8.9, decision 64). + FrameKind::Flags => todo!("R3h"), + // The attempt succeeded: `g.keep` of the record's + // speculation, its flags put back, the record + // popped (8.9, decision 68). The clashes of this + // loop go through `Checker::recover` first. + FrameKind::Try => todo!("R3r"), FrameKind::PackageEnd => { // A clash here is the enclosing pair's (`Unify []`). if self.package_end(frame).is_err() { From 20b7290baa3248c2d1039d96bb49c738404d8a47 Mon Sep 17 00:00:00 2001 From: "open-session-submodule[bot]" <341020755+open-session-submodule[bot]@users.noreply.github.com> Date: Tue, 17 Nov 2026 22:51:34 +0000 Subject: [PATCH 5/8] Checker rewrite R3: the open questions of the sixth skeleton turn Co-authored-by: Jaap Frolich Session: https://os.submodule.nl/session/bks-9da31a3c-0d3b-7546-bc8e-f1fd3a9270ee Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com> --- compiler-rust/docs/CHECKER_REWRITE.md | 28 +++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/compiler-rust/docs/CHECKER_REWRITE.md b/compiler-rust/docs/CHECKER_REWRITE.md index 3712a3d98d..ac0fb71490 100644 --- a/compiler-rust/docs/CHECKER_REWRITE.md +++ b/compiler-rust/docs/CHECKER_REWRITE.md @@ -5407,6 +5407,34 @@ share their elements pairwise: linear, by the table. unused" in OCaml); a parameter past the 32nd of `injective_bits`; `LocalDecls::compare` answering `Members` for two open types. +**Open questions of the sixth turn.** + +1. *Ambiguity of GADT instances is not checked* by the Rust compiler + (decision 62: a program OCaml refuses is accepted). Recommended: a + slice of its own after R3r, designed with R3q in the next turn: the + instance levels of `Env` (`gadt_instance_level`, + `add_gadt_instance_chain`) as a side table of the kernel, and the + three places of unification that read them. +2. *The old checker links two structures in pattern mode* and no source + program shows it (decision 62). Recommended: keep `old_judges`; the + hand-built tests judge the success of a structure pair. +3. *`free_univar` visits a node again for each context* of a + polymorphic type WITH bound variables that shares it, as OCaml does + (decision 59). Recommended: accept. +4. *`Checker::lower_scoped` does not drop the name of a row whose path + escapes* (ctype.ml:618-623; decision 60). Recommended: a correction + with a hand-built test in the turn that designs R3q, which copies + named rows. +5. *`duplicate` is `generalize` and `copy` under a speculation* + (decision 67): it holds only if `generalize` writes nothing but + levels through the trail. The builder of R3h reports what it + writes; if it writes more, the copy gets a flag that copies every + node, in `Graph::copy_ref`. + +Not designed in this turn: R3q (`instance_poly`, `instance_label`). +What is known for it: decision 47's fixed-row rule moves to R3h's first +commit, and question 4 above. + ## 9. R4 as designed Section 6.5 gave R4 as one slice of seven commits. It is replaced by this From bd8d5333d75ba116e820ea974c785cc2fa95d089 Mon Sep 17 00:00:00 2001 From: "open-session-submodule[bot]" <341020755+open-session-submodule[bot]@users.noreply.github.com> Date: Tue, 17 Nov 2026 22:51:53 +0000 Subject: [PATCH 6/8] Checker rewrite R3: the changelog fragment says what the sixth turn designed Co-authored-by: Jaap Frolich Session: https://os.submodule.nl/session/bks-9da31a3c-0d3b-7546-bc8e-f1fd3a9270ee Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com> --- changes/checker-rewrite-r3-skeleton-6.internal.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/changes/checker-rewrite-r3-skeleton-6.internal.md b/changes/checker-rewrite-r3-skeleton-6.internal.md index 6b5aca1bf8..eeb9733a39 100644 --- a/changes/checker-rewrite-r3-skeleton-6.internal.md +++ b/changes/checker-rewrite-r3-skeleton-6.internal.md @@ -1 +1 @@ -- Checker rewrite R3, sixth design turn: the record of the reviews of universal variables and packages, three corrections of built code with their tests (what two variant fields gain beside types that hold a universal variable, the walk for free universal variables below a polymorphic type without bound variables, the arguments of a row name that is no longer the innermost row's), and the design of pattern mode, equations and `instance_poly`. +- Checker rewrite R3, sixth design turn: the record of the reviews of universal variables and packages, three corrections of built code with their tests (what two variant fields gain beside types that hold a universal variable, the walk for free universal variables below a polymorphic type without bound variables, the arguments of a row name that is no longer the innermost row's), and the design of pattern mode and its equations in two slices, with a harness, acceptance tests and build prompts. From df56d4ad29b00f72b443f0b2ac48fb411fffb77a Mon Sep 17 00:00:00 2001 From: "open-session-submodule[bot]" <341020755+open-session-submodule[bot]@users.noreply.github.com> Date: Tue, 17 Nov 2026 23:03:36 +0000 Subject: [PATCH 7/8] Checker rewrite R3: the counts of the sixth turn's tests as run Co-authored-by: Jaap Frolich Session: https://os.submodule.nl/session/bks-9da31a3c-0d3b-7546-bc8e-f1fd3a9270ee Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com> --- compiler-rust/docs/CHECKER_REWRITE.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/compiler-rust/docs/CHECKER_REWRITE.md b/compiler-rust/docs/CHECKER_REWRITE.md index ac0fb71490..dfb8945063 100644 --- a/compiler-rust/docs/CHECKER_REWRITE.md +++ b/compiler-rust/docs/CHECKER_REWRITE.md @@ -2258,14 +2258,15 @@ that R5a answers. R3h was not started in this turn. State after the sixth skeleton turn (trunk `61f7f41292`). Merged: every slice above R3h in the table, R3g and R3p included (built as designed; what their reviews put on record is decisions 53 to 57 of 8.6), and K1 to -K4. `cargo test --lib tc::` on that trunk: 265 passed, 0 ignored. Without a design +K4. `cargo test --lib tc::` on that trunk: 265 passed, 0 ignored. At the end +of the sixth turn: 281 passed, 15 ignored (ten for R3h, five for R3r). Without a design before this turn: R3h, the `instance_poly` slice (R3q: a slice name has three characters), R3i and R3j. What the sixth turn did is listed in 8.6 ("Decided in the sixth skeleton turn"). | Slice | Decisions | Tests | Run in this turn | |---|---|---|---| -| R3h | 61 to 67, 69, 71 | nine ignored in `r3_tests.rs`: eight hand-built and the step test `pattern_mode_at_ten_thousand` | none: no prototype was made; they compile against the skeleton and the check of the ignored tests passes. The harness and the old checker's side (decisions 61, 62) are nine tests that run today | +| R3h | 61 to 67, 69, 71 | ten ignored in `r3_tests.rs`: nine hand-built and the step test `pattern_mode_at_ten_thousand` | none: no prototype was made; they compile against the skeleton and the check of the ignored tests passes. The harness and the old checker's side (decisions 61, 62) are nine tests that run today | | R3r | 63, 68, 70 | five ignored: three hand-built and `mcomp_at_ten_thousand` in `r3_tests.rs`, the fixed cases and 12,000 generated cases in `differential.rs` (`pattern_plan_agrees_on_generated_cases`) | none, as above | After the sixth turn: designed and not built, R3h and R3r (8.9); without From 2e50e185b3368b41c4bee61420333f68657b9032 Mon Sep 17 00:00:00 2001 From: "open-session-submodule[bot]" <341020755+open-session-submodule[bot]@users.noreply.github.com> Date: Wed, 18 Nov 2026 00:31:24 +0100 Subject: [PATCH 8/8] Checker rewrite R3: the corrections the review of the sixth turn found owed, and the two prompts held until they are made The review read the design of pattern mode against compiler/ml/ctype.ml and ran four source programs through both compilers. Section 8.9 gets the list of corrections a design turn makes before either slice is built: the rule of local_non_recursive_abbrev (the test a_recursive_equation_is_not_added expects the opposite of what OCaml does), the occurrence test of a variable in pattern mode, the stale row name in pattern mode, two size tests, the visibility mcomp's marker needs, a step test that does not test its table, three step tests, line ranges, and the prompts' rules. The prompts of both slices say they are not issued as they stand. No code changes: one comment in the test. Co-authored-by: Jaap Frolich Session: https://os.submodule.nl/session/os-01a1c9fa-0c5d-7912-a93b-3e7f38e53547 Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com> --- compiler-rust/docs/CHECKER_REWRITE.md | 82 +++++++++++++++++++++++++++ compiler-rust/src/tc/r3_tests.rs | 5 ++ 2 files changed, 87 insertions(+) diff --git a/compiler-rust/docs/CHECKER_REWRITE.md b/compiler-rust/docs/CHECKER_REWRITE.md index dfb8945063..77aeb7a262 100644 --- a/compiler-rust/docs/CHECKER_REWRITE.md +++ b/compiler-rust/docs/CHECKER_REWRITE.md @@ -3027,6 +3027,10 @@ for two equal paths; a change in unify.rs or outside rows.rs and the three ignore lines. At most 150 lines. ``` +Not issued as it stands: the review of this turn found corrections the +design owes first (8.9, "Corrections owed after review"). The prompt is +issued when they are in the document and the tests. + ```text R3h. Read section 8.9 whole and section 8.6, decisions 47, 60, 61 and 62; compiler/ml/ctype.ml 253-273, 1478-1535, 1802-1881, 2085-2104, @@ -3092,6 +3096,10 @@ add_gadt_equation and cut_equations; a change in graph.rs, level.rs or template.rs, or under src/types. At most 450 lines. ``` +Not issued as it stands: the review of this turn found corrections the +design owes first (8.9, "Corrections owed after review"). The prompt is +issued when they are in the document and the tests. + ```text R3r. R3h is merged. Read section 8.9, decisions 63, 64 (the rows marked 2), 68 and 70 and "No supplier"; compiler/ml/ctype.ml 1886-2080, @@ -5432,6 +5440,80 @@ unused" in OCaml); a parameter past the 32nd of `injective_bits`; writes; if it writes more, the copy gets a flag that copies every node, in `Graph::copy_ref`. +**Corrections owed after review** (the review of the sixth turn, pull +request 347). The prompts of R3h and R3r in 8.4 are not issued until a +design turn has made these, each with its test; nothing here is the +builder's to decide. + +1. *Decision 67, step 1, and the test + `a_recursive_equation_is_not_added` are not what OCaml does.* + `local_non_recursive_abbrev` is ctype.ml:1503-1536 (1478-1502 are + `occur_full`, `occur` and `occur_in`). It is called with `strict = + false`, and then: + - a node that is not a constructor is entered only `if strict` + (1525-1529): a tuple or an arrow at the top is not searched; + - a constructor of `source` answers false (1508); + - a constructor of a contractive type (`is_contractive`, + 1371-1375: a data type, or a predefined type without a manifest) + is not entered while not strict (1509); + - any other constructor is expanded once (`try_expand_head + try_expand_once`, 1515, not the `_opt` form) and the walk goes on + in the expansion; one that cannot be expanded has its arguments + walked, strictly for an argument whose parameter in the + declaration is not a variable (1516-1524). + Seen with both compilers on source programs (`Refl: eq<'a, 'a>` + matched at `eq` for a locally abstract `a`, the branch using a + value of type `a` at type `T`): the equations `a := (a, int)`, `a := + array` and `a := box` (a data type) are ADDED, both compilers + accept; `a := M.t` for an abstract `M.t` is not, both refuse. The + test must expect the tuple's equation added and have a case that is + refused. +2. *The occurrence test of a variable in pattern mode* (decisions 64 + and 69 say "as in expression mode"). In `Pattern` OCaml's `occur` + runs `occur_full` with `allow_recursive` (ctype.ml:1467-1490; + `occur_rec`, 1384-1408): a contractive constructor is not entered + (1390), a structure that is not a constructor is not entered + (1404-1408), objects and variants never are (1403). So a variable is + bound to a type that holds it below a tuple in pattern mode, where + expression mode clashes. The design says what the binding walk does + in pattern mode (the levels are still lowered everywhere), with a + hand-built test of the bind that succeeds and one of a clash through + a constructor that is not contractive, and what a binding in + expression mode pays for it. +3. *Decision 60 in pattern mode.* `unify_row` keeps row 1's name + arguments as children when `last1` is none, which is right because + unify3 has linked the left node in expression mode. Pattern mode + does not link it (decision 64): say which node is the home of row 1 + there and whether its name's arguments stop being children, with a + test. +4. `size_of` tests for `Equation` (20 bytes) and `Try` (40 bytes). +5. `mcomp`'s marker calls `leave_scopes`, which is private to unify.rs + and looks for its frame in `frames[base..]`, which `mcomp` has + popped by then: say how the scopes are cut there and make what it + needs visible. +6. `mcomp_at_ten_thousand` builds tuples of width 100, so its relation + holds with or without the table of pairs it is there for: width n, + or a second relation that fails without the table. +7. Step tests at 1,000 and 10,000 for `non_recursive_abbrev` and + `duplicate` (a destination of n nodes) and for `recover`. +8. Line ranges that are a few lines off, in doc comments, tests and + both prompts: the try for an argument that is not injective is + 2358-2375 (the cited 2359-2371 leaves out the two `reify` calls); + the variant arm is 2402-2411 (2411 is the `mcomp` call); the + package fallback is 2416-2424 (2422 reifies); two instantiable + types 2376-2387; the three one-sided arms 2388-2391, 2392-2395 and + 2396-2400. +9. The prompt of R3r gets the rule the prompt of R3h has for its + hand-built tests (a test that fails where the code does what the + ctype.ml lines say: stop and report, never edit the test); each + prompt names the functions that are the other slice's. Decision 71 + adds what it leaves out (the hash state made in `Checker::new`, the + growth of `Checker`, the two compares per pair of rows of decision + 60), and the cost paragraph of `unify_row` names those two compares. + +The five open questions above are answered as recommended in the same +review; the next design turn writes them into 8.6. + Not designed in this turn: R3q (`instance_poly`, `instance_label`). What is known for it: decision 47's fixed-row rule moves to R3h's first commit, and question 4 above. diff --git a/compiler-rust/src/tc/r3_tests.rs b/compiler-rust/src/tc/r3_tests.rs index 5f9e0c1052..6b90efafa4 100644 --- a/compiler-rust/src/tc/r3_tests.rs +++ b/compiler-rust/src/tc/r3_tests.rs @@ -4611,6 +4611,11 @@ fn the_newer_of_two_locally_abstract_types_is_expanded_once() { #[test] #[ignore = "R3h build"] fn a_recursive_equation_is_not_added() { + // Review of pull request 347: this expectation is not what OCaml + // does (compiler/ml/ctype.ml:1503-1536, called with `strict = false`: + // a tuple at the top is not searched, so this equation IS added). + // A design turn corrects the test before the slice is built + // (CHECKER_REWRITE 8.9, "Corrections owed after review", item 1). let (mut c, _) = pattern_checker(Mode::Success); let (inner, int) = (constr(&mut c, N0, &[]), constr(&mut c, INT, &[])); let destination = tuple(&mut c, &[inner, int]);