Repository navigation
An instance of a type that is ambiguous outside its equation is refused, as in OCaml - #350
Conversation
Why:
OCaml records, per `switch` whose patterns may add equations, the
types known only through an equation (`Env.gadt_instances`,
compiler/ml/env.ml:1305-1345), and refuses to expand or lower one
below that level (ctype.ml:587-589, 1215-1227). Rust did not track
them (`Env::add_gadt_instance_level` took no level), so it accepted
and compiled programs OCaml refuses.
Typing modified (expanding an equation's type below its switch):
Example:
type rec t<_> = Int: t<int>
let f = (type a, x: t<a>, y: a) =>
switch x {
| Int => if true { y } else { 0 }
}
Before (diagnostic): accepted; `function f(x, y) { return y; }`
After (diagnostic):
We've found a bug for you!
Main.res:4:33
This has type: int
But this if statement is expected to return: a (defined as int)
This instance of int is ambiguous:
it would escape the scope of its equation
if expressions must return the same type in all branches (if, else if, else).
The instance sets live in the type context, one per scope, each holding
its enclosing scope's handle; the environment holds the innermost
handle (`EnvData::gadt_instances`), so cloning an environment copies a
`u32` and no set. Additions are journaled and undone on backtrack, as
OCaml trails `set_typeset`. Ported: the expansion level of an equation
(env.ml:1805-1810), `cleanup_abbrev` when tracing starts (ctype.ml:182),
the test and the additions of `expand_abbrev_gen` (ctype.ml:1215-1227),
the chain of `try_expand_head` (ctype.ml:1270-1276), `unify2`'s
expansions failing with the pair (ctype.ml:2289-2292), the test of
`update_level` (ctype.ml:587-589), and the arm of `unify` for two
locally abstract types (ctype.ml:2255-2268), without which an accepted
`eq<a, b>` program became refused.
Review: What does a program without GADTs pay? The test in the level
update is one load of `has_gadt_instance_scopes` and a branch; every
other read and write runs only while tracing is on (under local
constraints) or in an environment with an instance scope.
Co-authored-by: Jaap Frolich <jfrolich@gmail.com>
Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b
Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Why:
OCaml's `copy ?env` (compiler/ml/ctype.ml:770-774), reached through
`instance env` and `instance_list env` (ctype.ml:879-894), adds the
copy of a node in an instance set to that set when the environment
has local constraints. Rust's `instance` takes no environment, so a
function generalized inside a case gave copies that were no
instances, and its result left the case.
Typing modified (instantiating a generalized instance below its switch):
Example:
type rec t<_> = Int: t<int>
let f = (type a, x: t<a>, y: a) =>
switch x {
| Int => {
let k = () => if true { y } else { 0 }
k()
}
}
Before (diagnostic): accepted; `function f(x, y) { return y; }`
After (diagnostic):
Main.res:6:7-9
This has type: a (defined as int)
But it's expected to have type: 'a
This instance of int is ambiguous:
it would escape the scope of its equation
`instance_in` and `instance_list_in` take the environment; the type
checker's `instance env` sites use them (constants and predefined types
keep `instance`, OCaml's `instance_def`). The copy is made as before;
then, only in an environment with local constraints and an instance
scope, the copy table is walked once and the copy of each instance is
added. Without a scope in the unit an `instance` call pays one load and
one branch.
Co-authored-by: Jaap Frolich <jfrolich@gmail.com>
Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b
Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
The third round of CHECKER_DEPARTURES.md gets the place A1: the 64 programs run through both compilers before and after the two commits of the port, the causes on both sides, and the OCaml sites left open because no program shows them. The twelve probes that differed on trunk and had no fixture become super_errors fixtures. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Why:
The instruction count of a clean tests/tests build rose by 0.313%
with the port: the expansion of a head returned a `Result` through
two more call layers, which changed what was inlined
(`try_expand_head_with_mode` -29.4M, `try_expand_head_traced`
+23.6M, `expand_head_traced` +15.1M, `find_abbrev_expansion` +8.2M
instructions), and every `instance_in` call tested for an instance
scope before copying (+2.3M).
The expansion of one head is again trunk's function returning an
`Option`; an ambiguous instance is written to an out-slot that only the
unifier reads, and the traced end of `expand_abbrev_gen` is a cold
function reached only while tracing. The head-expansion loop is one
function, generic over whether it propagates the refusal, so the safe
instantiation is trunk's loop. The test of `try_expand_head`'s chain is
made only while tracing: one load and one branch outside it. `instance`
and `instance_in` share one body; the instance-scope test follows a copy
of a generic root only. `unify` reuses its local-constraints test for the
newtype arm. All 64 programs of A1 agree as before.
Co-authored-by: Jaap Frolich <jfrolich@gmail.com>
Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b
Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Why:
The checker tests on trunk drive the old checker's pattern
unification on an environment that has an equation (expansion
level 999) but no instance scope, which the type checker never
builds: both callers of `Env::add_gadt_instance_level` add the scope
before the patterns that add equations. `add_gadt_instances`
panicked there, where OCaml's `List.assoc` asserts
(compiler/ml/env.ml:1326-1328). It now adds nothing;
`tc::differential::tests::old_checker_gives_the_equation_to_the_newer_locally_abstract_type`,
`generated_cases_find_a_pattern_unification_in_expression_mode` and
`old_side_of_the_pattern_plan_is_not_vacuous` pass unchanged.
Co-authored-by: Jaap Frolich <jfrolich@gmail.com>
Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b
Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
tests/tests gained gadt_instance_ambiguous_accepted, one more unit the new checker declines (NotBuilt 832 -> 833); recorded with `scripts/tc_coverage.py --rescript rewatch/target/release/rescript --write`. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
The instruction counts the CI check measured, by function, and the line numbers after the expansion was moved off the default path. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
`instance` and `instance_list` are trunk's again. The `instance env` sites test the unit's flag and then the environment at hand; only when it has an instance scope do they call the copy that walks the finished copy table (`instance_with_gadt_instances`, not inlined). Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
…as an instance scope Trunk's head expansion runs unchanged outside an instance scope. In an environment with one, a traced loop calls the traced instantiation of the expansion, which makes the test of expand_abbrev_gen, and adds try_expand_head's chain whenever the result is an instance, as OCaml does whatever the tracing flag. The unifier's expansions take the same choice. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
The chain of try_expand_head (compiler/ml/env.ml:1342) has its own scan of the memorized expansions, so find_abbrev_expansion keeps trunk's one caller, the head expansion, and is inlined there as on trunk. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
rustfmt's layout for the lines the three attempts changed; no code change. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
383798e to
8d0abc3
Compare
Why:
An instance scope exists inside almost every `switch` of more than
one case: OCaml's `type_cases` adds the level whenever the
exhaustiveness check will run (compiler/ml/typecore.ml:4814-4818,
4913-4914), so 434 of the 788 units of tests/tests add one. Every
read of a set is a no-op while every set is empty, and a set gets
its first member only from `expand_abbrev_gen` (ctype.ml:1217-1227)
under the tracing flag.
The type context's flag is now "a type has been added to an instance
set in this unit", set where a member is pushed, not where a scope is
created, and kept in one byte with the tracing flag. Copies and level
updates test it; the expansion sites test the byte (tracing or a
member), so the test that adds the first member still runs. Without
either, every site runs trunk's code after one load and one branch.
Co-authored-by: Jaap Frolich <jfrolich@gmail.com>
Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b
Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
… copy The traced expansion was a second instantiation of `try_expand_head_with_mode`, so `find_abbrev_expansion` and `find_pdot_expansion` had two callers and were no longer inlined into trunk's. `try_expand_head_traced` now wraps trunk's function: while tracing and with no memorized expansion, it looks up the equation and its expansion level, refuses an ambiguous instance before expanding (nothing to undo), and otherwise records the type and its expansion in their instance set after it. A type that is an instance through another route is tested after the expansion, whose memo is then forgotten as OCaml's `try_expand_safe` backtracks it. The memo test is a scan of its own (`has_memorized_expansion`). Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
… can be expanded `expand_head_for_unify` returned a `Result` from both arms and tested the guard once per side. The pair sites now test it once (`expand_heads_unif`) and, while it is false, make trunk's two expansions directly; the traced pair, which refuses an ambiguous instance, is a separate function. The fully expanded comparison's site (`expand_head_in_unify`) does the same for one side. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Corrects why 434 units add an instance scope: type_cases adds one for every switch whose exhaustiveness is checked. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Inlined, it grew unify's frame from 0x1e8 to 0x238 bytes, so every call spilled more registers. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
…+1,914,955) This reverts commit 89b7ae1. Measured locally against trunk 459b87c (fat LTO, one codegen unit, five runs a side), the total fell from +5,213,073 to +4,521,654, but only through functions the commit does not touch (`IdTbl::find_by_ident` -802,026, the path iterator), which move by as much between runs. `unify` itself rose from +1,130,695 to +1,914,955: with `#[inline(never)]` on `unify_newtypes` its frame grows from 0x238 to 0x258 bytes (trunk: 0x1e8), so the attribute adds to what a unit without an instance pays. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
… link to a private item The documentation build refused the link from a public function to a private one. The sentence about the guard now says what the guard is: a type has joined an instance set in this unit. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/os-01a1c9fa-0c5d-7912-a93b-3e7f38e53547 Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
No change to the tree. The port measures about +0.12 to +0.14% on the tests/tests build: one load and one branch at each copy, expansion and level-update site while no type has joined an instance set, a larger frame in unify, and the instance lookups OCaml also makes in the units that add an instance. The split by function is in the pull request and under A1 of CHECKER_DEPARTURES.md. Instruction-count-increase: GADT instance tracking as OCaml has it; guard tests at the ported sites and the instance work in units that add an instance, split by function in the pull request Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/os-01a1c9fa-0c5d-7912-a93b-3e7f38e53547 Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
|
First-pass checklist review at ed5a1a8 (a checklist for the final reviewer, not an approval). First-pass review (checklist, not an approval): pull request 350, 1. ScopeThe task is to port OCaml's GADT instance sets (env.ml 1305-1345) and the ctype.ml sites that read or add to them, under rule (i)–(iii).
Not done, as the body states under "Open": unify2's chain (ctype.ml 2289-2298), Done but not asked for:
Line-number note: the body cites "unify2's expansions ( PASS (every asked item is done or listed as open). 2. Default pathThese changed functions run in an ordinary build. "In table" means the body's cost split has a row for it.
The three "no" rows are in the guard or under local constraints, but no figure is reported for them. PASS (each is listed; three without a measurement). 3. Engineering standardThe greps run over the 802 added lines of compiler-rust/src ( (a) Cost in the doc comment. Present for
(b) Step tests at 10,000. Two exist:
No step test for:
The copy test has one member. With k members, a copy of n nodes costs n·k. (c) No HashMap, HashSet or BTreeMap added (grep: none). (d) Allocations:
None inside a per-node walk on the default path. (e) One (f) A second pass: (g) No (h) No
UNCLEAR: the cost comments listed in (a) are missing, and the chain has no step test. 4. Tests and oracles
This is a count of declined units that may not rise ( Three PASS (one re-recorded ratchet, justified by a new unit). 5. New scripts and fixturesNo new script. The 30 PASS 6. Privacy
PASS 7. Changelog
PASS 8. Numbers
The document disagrees with the body by a few thousand on some rows, because it takes the fe245f2 run:
UNCLEAR: the local and count figures have no output. 9. Focused testsNot run (host review; cargo is not run here). UNCLEAR 10. Where to look hardest
11. Files
No file under compiler-rust/src/tc ( PASS 12. The sets against env.ml 1305-1345OCaml (env.ml): let add_gadt_instance_level lv env =
{env with gadt_instances = (lv, ref Type_set.empty) :: env.gadt_instances}
let gadt_instance_level env t =
let rec find_instance = function
| [] -> None
| (lv, r) :: rem ->
if Type_set.exists is_Tlink !r then
r := Type_set.fold (fun ty -> Type_set.add (repr ty)) !r Type_set.empty;
if Type_set.mem t !r then Some lv else find_instance rem
in find_instance env.gadt_instances
let add_gadt_instances env lv tl =
let r = try List.assoc lv env.gadt_instances with Not_found -> assert false in
set_typeset r (List.fold_right Type_set.add tl !r)
let add_gadt_instance_chain env lv t = … let rec add_instance t = let t = repr t in
if not (Type_set.mem t !r) then (set_typeset r (Type_set.add t !r);
match t.desc with Tconstr (p, _, memo) -> may add_instance (find_expans Private p !memo) | _ -> ())Rust (context.rs): pub(crate) fn gadt_instance_level(&self, scope: GadtScopeRef, ty: TypeExprRef) -> Option<i32> {
… while !current.is_none() { let entry = &scopes[current.index()];
for &member in &entry.members { … if self.repr(member) == ty { return Some(entry.level); } }
current = entry.parent; } None }
fn add_gadt_instances(&self, scope, level, types) {
let Some(scope) = self.gadt_instance_scope_at(scope, level) else { return; };
for &ty in types { self.push_gadt_instance(scope, ty); } }
pub(crate) fn add_gadt_instance_chain(&self, scope, level, ty) { … loop: let ty = self.repr(ty);
if members.contains(&ty) { return; } self.push_gadt_instance(scope, ty);
current = Tconstr path's memo scan `path.same(cached_path).then_some(expansion)` }
Membership after links:
I found no case where the answers differ. Neither side follows Cost:
PASS (same answers; duplicate growth noted in section 10). 13. Scope creation and what an environment holdsOld (env.rs): /// Rust does not track GADT instances, so only the fresh scope identity …
pub fn add_gadt_instance_level(&mut self) { self.bump_scope(); }New: pub fn add_gadt_instance_level(&mut self, ctx: &TypeContext, level: i32) {
self.bump_scope();
let parent = self.data.gadt_instances;
self.data_mut().gadt_instances = ctx.new_gadt_instance_scope(level, parent);
}The handle is Callers:
In Rust's UNCLEAR: the Rust behaviour is likely equivalent; the stated reason for the count is inaccurate. 14. Ported reads and writes in ctype.rs(a) if ty.level > level then (
(match Env.gadt_instance_level env ty with
| Some lv -> if level < lv then raise (Unify [(ty, newvar2 level)]) | None -> ());
match ty.desc with …Rust if level < current {
if let Some(instance_level) = gadt_instance_level(ctx, env, ty) && level < instance_level { return Err(ty); }
let desc = ctx.borrow_desc(ty); match &*desc { …The caller maps (b) The copy, ctype.ml:770-774: (match env with Some env when Env.has_local_constraints env -> (
match Env.gadt_instance_level env ty with Some lv -> Env.add_gadt_instances env lv [t] | None -> ()) | _ -> ())Rust (c) let ty' = subst env level kind abbrev (Some ty) params args body in
(if !(trace_gadt_instances ()) then
match Ext_pervasives.max_int_option lv (Env.gadt_instance_level env ty) with
| None -> () | Some lv -> if level < lv then raise (Unify [(ty, newvar2 level)]);
Env.add_gadt_instances env lv [ty; ty']);Rust
Before-versus-after is observable only if Other details:
SAME (ordering differs, not observable as far as I can tell). (d) let ty' = try_expand_head try_once env ty in
(match Env.gadt_instance_level env ty' with None -> () | Some lv -> Env.add_gadt_instance_chain env lv ty); ty'Rust Rust skips the chain when there was no expansion ( (e)
(f) ignore (expand_head_unif !env t1); ignore (expand_head_unif !env t2);
let t1' = expand_head_unif !env t1 in let t2' = expand_head_unif !env t2 inRust (if !(trace_gadt_instances ()) then
let ilevel t = match Env.gadt_instance_level !env t with None -> 0 | Some lv -> lv in
let lv1 = ilevel t1 and lv2 = ilevel t2 in
if lv1 > lv2 then Env.add_gadt_instance_chain !env lv1 t2
else if lv2 > lv1 then Env.add_gadt_instance_chain !env lv2 t1);That part makes the less ambiguous side's expansion chain join the more ambiguous side's set before (g) PASS (one open site, documented). 15. The guardFlags (context.rs new 572-574):
Calls and inlined tests:
With the flag false, body by body against the base (an awk extraction of each
UNCLEAR (only 16. BacktrackingRecorded (context.rs): TypeMutation::GadtInstance(scope) => { self.gadt_instance_scopes.borrow_mut()[scope.index()].members.pop(); … }The journal replays in reverse, so the pops are last-in, first-out per scope. Scope creation is not journaled (a scope outlives a restore, harmless), and the member flag stays set (rule iii). The task says OCaml's sets are not undone by
So OCaml undoes every addition on a backtrack ( Remaining risk: a UNCLEAR (matches OCaml's trail; Rust-only restore sites not enumerated). 17. Copy sites of typecore.rs
Converted where OCaml passes no environment:
Left on
Correctly left: the constant and predef The extra conversions copy generic label and declaration nodes, which are unlikely to be set members. The missed FAIL (per the task's rule: 6 converted where OCaml passes none, 2 to 3 sites left where it passes 18. Fixtures
Five shapes quoted (expected files, ANSI stripped):
Each has the two lines followed by OCaml's context text.
Fixtures per row of section 14, from shape and the document's mapping (not verified by a run):
The two "guard" programs of the document (refused at 9:33, and the three-function one) are not committed: no expected file has 9:33. UNCLEAR: row (a) has no identified fixture, and the guard programs are not committed. 19. CHECKER_DEPARTURES.md, A1Section "A1 - an instance of a type that is ambiguous outside its equation is refused" is present. Its contents:
The open list, quoted:
Every required item is there. Two caveats:
PASS 20. Counts as failureOver the added lines of compiler-rust/src:
Functions without a cost in their doc comment are listed in 3(a). FAIL (two 21.
|
| # | Row | Body |
|---|---|---|
| 1 | −12,250,409 expand_head_with_mode |
(a) |
| 2 | 12,100,981 expand_head_plain |
(a) |
| 3 | 1,130,683 unify |
(a) |
| 4 | 706,988 IdTbl::find_by_ident |
rest |
| 5 | 679,023 gadt_instance_level |
(b) |
| 6 | 642,364 repr_chain |
(b) |
| 7 | 497,792 expand_heads_unif |
site |
| 8 | 398,277 type_application_parts |
site |
| 9 | 242,590 try_expand_head_traced |
(b) |
| 10 | 190,067 Env::data_mut |
(a) |
| 11 | −166,879 unify_types |
(a) |
| 12 | −147,336 mi_theap_malloc_zero_aligned_at_generic |
rest (allocator) |
| 13 | 140,409 update_level_checked_above |
site |
| 14 | 130,678 type_identifier |
site |
| 15 | 129,160 moregen |
site |
| 16 | −115,008 mi_…_overalloc |
rest |
| 17 | 113,214 mi_free |
rest |
| 18 | 108,040 expand_head_traced::<true> |
(b) |
| 19 | 97,242 restore_type_state |
(a) |
| 20 | 96,633 type_function_nary |
site |
| 21 | 89,597 Env::add_gadt_instance_level |
(a) |
| 22 | 89,441 update_level_pass |
(b) |
| 23 | −72,110 _mi_theap_malloc_zero |
rest |
| 24 | 65,613 instance_in |
site |
| 25 | 64,273 expand_head_traced::<false> |
(b) |
| 26 | 56,221 try_expand_head |
site |
| 27 | −54,768 lower_plain_levels::lower |
(a) |
| 28 | 51,707 mi_page_free_list_extend |
rest |
| 29 | 51,117 _mi_page_malloc_zero |
rest |
| 30 | −46,387 mi_bin |
rest |
Every base function in the first 30 is placed. Below row 30, three base functions are not named in the body's split:
<bitmaps::bitmap::Iter…>+45,852 (the "path iterator" in prose only);Parmatch::head_path+44,076;typecore::type_construct+42,656 (aninstance_insite, not listed among the site rows).
The body's gates, as stated, on 8cd9bde:
- "All 71 programs agree with OCaml";
test_error_output_parity.sh --release: "total: 2267 pass: 2267 fail: 0";test_js_output_parity.sh --release: "Total: 783 / 783 tests passed (100.0% parity)";- clippy
--all-targets -- -D warnings: "exit 0, no warnings"; tc::differential: "65 passed; 0 failed; 1 ignored".
Not rerun here.
The trailer: git show -s --format=%B ed5a1a84ac | git interpret-trailers --parse | sed -n 's/^Instruction-count-increase: *//Ip' prints "GADT instance tracking as OCaml has it; guard tests at the ported sites and the instance work in units that add an instance, split by function in the pull request". This is the parse the job's "Find an accepted increase" step makes (instruction-count.yml 180-196).
The job found the trailer. Its compare-job log for ed5a1a8 (run 44230574575) contains Accepted increase: GADT instance tracking as OCaml has it; …, and its compare step printed:
run 1: base 4,136,102,697 head 4,141,002,098
run 2: base 4,135,442,114 head 4,140,844,178
run 3: base 4,136,596,882 head 4,142,785,288
run 4: base 4,135,865,588 head 4,140,937,567
run 5: base 4,136,512,391 head 4,142,699,271
More instructions than the threshold of +0.1%, accepted by the pull request.
| change | +5,402,064 (+0.131%) | | |
This run measures +5,402,064 (+0.131%), against the body's +5,752,626 (+0.139%) from 8cd9bde. The tree is the same, so the gap is run-to-run noise.
PASS (the trailer is accepted and every completed check passes; compiler-tests is still pending)
|
Coordinator review at ed5a1a8, first round (the author did not write this review). One correction is asked before the merge; the port is otherwise accepted. What I checked myself
Asked before the merge (sent to the builder)
Accepted as it stands
On record, open without a program: the sites listed under A1 (unify2's chain after expansion, Not rerun by me: the builder's 71 programs, the parity suites (CI ran them: 22 of 23 checks passed when I read them, |
OCaml's copy adds to an instance set only when given the environment (`instance env`, `instance_list env`; ctype.ml:877-894); `instance_def` and typetexp's `instance_list` (`Ctype.instance_list Env.empty`, typetexp.ml:61) add nothing. Back to `instance` / `instance_list`, where OCaml passes no environment: the declaration parameters of a type constructor (typetexp.ml:373), the constructor result under local constraints (typecore.ml:4739 `instance_def ty_res`), and the label result and argument of a record field and of a field assignment (typecore.ml:4315, 4319). To `instance_in`, where OCaml passes it: `rue` (typecore.ml:2607 `instance env ty_expected`; `rue_unify`, `finish_expected` and `finish_expected_at` take the environment), `object_field_use_type` (typecore.ml:2553 `instance env ty`), and the expected type of a record (typecore.ml:3044, 3148) and of a coercion's `rue`, through `instance_if_generic_in`. The tuple's copy has no OCaml counterpart (typecore.ml:2940 unifies `ty_expected` itself) and keeps trunk's. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
…them Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
No behaviour change. Cost notes for `expand_head_plain`, `expand_heads_traced`, `ambiguous_instance`, `instance_list_in` and `instance_list_with_gadt_instances`; a step test of `add_gadt_instance_chain` at 1,000 and 10,000 members (its membership scan now ticks the test-only step counter). Under A1: where Rust creates instance scopes, which corrects the reason given for the 434 units; the scope OCaml adds for a `switch` without a generalized constructor and Rust does not, as a difference without a program; duplicates in the member sets; the newtype arm; the journal; and the figures of the two remaining `#[inline(never)]`. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
OCaml has no copy there: typecore.ml:3165 stores the copy of 3153 (`ty_exp = instance env exp.exp_type`) as the base's type, so this Rust-only copy adds to no instance set. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
typecore.ml:3780 gives `assert(false)` the type `instance env ty_expected`; Rust used the expected type uncopied, which adds nothing to an instance set. A type that is not generic is returned unchanged by the copy, as before. Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
Co-authored-by: Jaap Frolich <jfrolich@gmail.com> Session: https://os.submodule.nl/session/bks-bd5a5126-dea8-7249-82a4-e8fae2447c0b Signed-off-by: open-session-submodule[bot] <341020755+open-session-submodule[bot]@users.noreply.github.com>
|
Coordinator review, final, at 4150bc8 (the author did not write this review). Merging: the port agrees with OCaml on every program that differed and that it was aimed at, changes no output, and what it leaves open is listed, with one new witness below that goes to a follow-up. The correction of the first round
What I ran on this head
A witness the port does not cover yet (trunk and this head both accept; OCaml refuses), found with programs beyond the 34: type rec t<_> = Int: t<int>
let f = (type a, x: t<a>, y: a) => switch x { | Int => ((z: a) => z)(y) }OCaml: "This has type: a (defined as int) / But this switch is expected to return: 'a / This instance of int is ambiguous: it would escape the scope of its equation". The same with On record, open without a program: the sites listed under A1 (unify2's chain after expansion, Not rerun by me: the builder's programs and the parity suites (CI ran the suites on this head). |
Note
Cost accepted. CI
Instruction counton5cfe8a23f(fifth turn): base 4,136,194,490, head 4,140,293,342, +4,098,852 (+0.099%), within the job's +0.1% threshold. The split below is from8cd9bdec6. Since then, the copies that now take the environment add guard sites:type_identifier+357,632 (was +130,678, asrue's copy tests the flag),type_application_parts+431,847 (was +398,277) andtype_expect_with_recarg_unscoped+67,765. Part (a) and part (b) are otherwise unchanged.Cost
Guard. Every read of an instance set answers nothing while all sets are empty. A set gets its first member only from
expand_abbrev_gen's test (ctype.ml:1217-1227), which runs while tracing. So the type context's flag now means "a type has been added to an instance set in this unit" (has_gadt_instances). It is set inpush_gadt_instance, not where a scope is created, and shares one byte with the tracing flag.instance_in,instance_list_in) and level updates test the flag first and then the environment's scope handle. While the flag is false they run trunk'sinstanceand the plain level update.try_expand_head,expand_head_with_mode, the unifier'sexpand_heads_unifandexpand_head_in_unify) test the byte (expands_gadt_instances: tracing or a member), so the test that adds the first member still runs. Otherwise they run trunk's expansion and loop.unifyruns under local constraints, which also turn tracing on.Counts.
type_function_nary'sinit_env, for a function whose parameters contain a generalized constructor, and for one with a parameter that is not a plain variable (such as()), whose patterns get an exhaustiveness check (typecore.ml:4196-4197). Most of the 434 are this second kind.type_match, for aswitchwith a generalized constructor only (typecore.ml:4819-4828).type_casesadds one for aswitchwithout a generalized constructor that gets an exhaustiveness check (typecore.ml:4913-4914). This is recorded under A1 as a difference without a program. It can't be seen: a member is only ever added at the expansion level of an equation or at the level of a set that already holds the type, so OCaml's inner set stays empty.Split of the final CI table (
8cd9bdec6). "site" is the one load and one branch per site; "a" is the rest of what a unit that adds no instance pays; "b" is work while tracing or after a member.expand_heads_unif)instance_in/instance_list_inat typecore's sites:type_application_parts+398,277,type_identifier+130,678,type_function_nary+96,633,instance_in+65,613,instance_list_in+44,386update_level_checked_aboveexpand_head_with_mode's guard, inlined intomoregenat its twoexpand_headcalls (placement)try_expand_headunify: the newtype arm is inlined into it, frame 0x1e8 → 0x238 bytesexpand_head_plain+12,100,981 againstexpand_head_with_mode−12,250,409Env::data_mut+190,067,Env::add_gadt_instance_level+89,597,make_mut+122,560 (from thefe245f217run; this run's list is cut off before it)restore_type_state(the journal's new entry)unify_types−166,879,lower_plain_levels::lower−54,768gadt_instance_level+679,023,repr_chain+642,364try_expand_head_traced+242,590,expand_head_traced::<true>+108,040,::<false>+64,273,Env::find_local_constraint+43,608update_level_pass+89,441,instance_with_gadt_instances+45,419IdTbl::find_by_ident+706,988 and the allocator; the run onfe245f217, with the same checker code, hadfind_by_identat −501,294 and +5,138,728 in all)Cost of a lookup.
gadt_instance_levelcosts one step per scope visited, plus onereprper member of those scopes, counted as added: a type added twice is two members.reprcompresses the links it follows (repr_chain).Attempts (third and fourth turns). Measured locally with fat LTO, one codegen unit and five runs a side. Base is trunk 459b87c; the previous head
bcc2a4ef4measured +4,401,645.82e309f01cbaf8d70dtry_expand_head_with_mode, testing the ambiguity before the callfe245f21789b7ae1e8#[inline(never)]onunify_newtypes8cd9bdec6bcc2a4ef4is within the noise of the earlier measurement (CI hadbcc2a4ef4's code at +7,466,613), but it also caused theunifyinlining.find_abbrev_expansion+8.18M andfind_pdot_expansion+1.41M from the table.find_by_ident−802,026, the path iterator).unifyitself rose from +1,130,695 to +1,914,955 because its frame grew to 0x258 bytes, so I reverted it.#[inline(always)]or#[cold]remains. Two#[inline(never)]do, oninstance_with_gadt_instancesandinstance_list_with_gadt_instances. They come from the copy-path attemptae16a975dof the second turn, which measured +9,118,887 against +11,678,261 before it.What and why
OCaml keeps, per
switch(or function) whose patterns may add GADT equations, the set of types known only through an equation (Env.gadt_instances,compiler/ml/env.ml:1305-1345). It refuses to expand such a type, or lower its level, below that set's level: "This instance of int is ambiguous: it would escape the scope of its equation". The Rust checker tracked none of this, so it accepted and compiled programs OCaml refuses:Both compilers now refuse it at 4:33 with the same text. The witness
[y, 0](refused at 4:16) and 28 more programs that differed agree as well. Programs that must stay accepted still are: a result annotation,(y: int),y + 1,h(y), an existential that doesn't escape,eq<a, b>, and others.Commits
92862d207stores the instance sets and makes the expansion and level tests:u32handle, and the environment's shared bindings record holds the innermost handle. Both callers ofadd_gadt_instance_levelbuild the case environment as a clone of the outer one, so this form copies au32per clone and never a set.set_typeset.env.ml:1805-1810);cleanup_abbrevwhen tracing starts (ctype.ml:182);expand_abbrev_gen(ctype.ml:1215-1227);try_expand_head(ctype.ml:1270-1276);unify2's expansions (ctype.ml:2289-2292);update_leveltest (ctype.ml:587-589);unify's arm for two locally abstract types (ctype.ml:2255-2268). Without it the acceptedeq<a, b>program became refused, and a trace showed why: Rust lowered the copy ofbto the outer level before expanding it.22a53a612makes a copy of an instance join its set (copy ?env,ctype.ml:770-774):instance_in/instance_list_intake the environment, and the typecore sites of OCaml'sinstance envuse them. Without this, a function generalized inside the case (let k = () => ...; k()) let the instance out.7d08707fdadds the document entry A1 incompiler-rust/docs/CHECKER_DEPARTURES.md(all 64 programs, before and after) and 12 more fixtures.752db1e00keeps the tests off the default path: the expansion is again trunk's function returningOption; a refused node goes to an out-slot only the unifier reads; the traced end ofexpand_abbrev_genis a cold function; the head-expansion loop is generic over propagating the refusal; the chain test is made only while tracing;instanceandinstance_inshare one body.c61856deamerges trunk.b46a3af77makes a missing instance scope at an equation's level add nothing. Trunk's new checker tests build that state, which the type checker never does and where OCaml asserts; threetc::differentialtests now pass unchanged.c0daf094fre-recordscompiler-rust/tc_coverage.txt(tests/tests NotBuilt 832 -> 833: the new accepted-programs unit), withscripts/tc_coverage.py --write.c8346e661records the cost in the document.ae16a975d,c3cd63b5d,bcc2a4ef4(copy path, head loop, chain memo),4679c1d94(format only),8d0abc3bb(document).82e309f01,cbaf8d70d,fe245f217(above),d1faed4e5(document: the guard, the two programs, the split).89b7ae1e8and its revert8cd9bdec6,43e204bef(the lookup's cost in its doc comment),f3af82e02(document).de5bef189: the copy sites of typecore.rs are OCaml's in both directions.66dfd8330: a record base's second copy gets trunk's call back.5f8d821f9:assert(false)copies its expected type.0684ebd56,848fc888a,5cfe8a23f: document.702f4bd66: cost notes, the chain's step test, the scope difference.Decisions this PR makes
Envclone then copies nothing more.Vecper scope; a lookup compares each member's representative, as OCaml normalizes the set byreprbefore testing. No hash map: membership must follow links made after insertion, which a table keyed by node can't see. An addition is O(1); the chain keeps OCaml's membership test, which ends it.TypeExprRefMap::for_eachintype_expr.rs, a file outside the listed functions.try_expand_headchain gate. Sincec3cd63b5dthe chain runs whenever the environment at hand has an instance scope, as OCaml runs it whatever the tracing flag (for example when the result of aswitchis lowered in the outer environment). Five more programs aimed at that moment agree; all five are accepted by OCaml.Cost
instance_incall (has_gadt_instances). Expansions pay the same, testingexpands_gadt_instances.gadt_instance_level_is_one_step_per_member_at_ten_thousand: a lookup is one step per member visited.instance_in_tests_each_copied_node_once_at_ten_thousand: one lookup per copied node; 0 steps outside local constraints.gadt_instance_chain_is_linear_in_the_members_of_its_set: a chain of one type against 1,000 and 10,000 members; ten times the members is ten times the steps.TypeExprRefMap::for_eachhas no step counter intype_expr.rs; its cost is in its doc comment.Instruction count: see the warning above.
Copy sites of typecore.rs
OCaml's copy adds to an instance set only when it is given the environment:
instance envandinstance_list env(ctype.ml:877-894).instance_defand typetexp'sinstance_list(Ctype.instance_list Env.empty,typetexp.ml:61) add nothing. Every copy of typecore.rs, matched against its OCaml line:instance_in,instance_list_in,instance_if_generic_in), as OCaml'sinstance env/instance_list env/ typetexp'sinstance envinstance,instance_list), as OCaml'sinstance_def/ typetexp'sinstance_listrueis one site (2607,rue_unify), shared by its 21 callers.instance_if_generic_incopies only a generic type. OCaml's copy returns any other type as is and adds nothing for it (ctype.ml:750).instance envsites have no Rust copy:is_ignoreand theignorecase oftype_application(typecore.ml:2498,4496): Rust tests the primitive by name and copies nothing.letwith locally abstract types (typecore.ml:5092): Rust types the rewritten expression, whoseruemakes that copy.All 84 sites
instanceinstance_definstanceinstance_definstanceinstance_definstanceinstance_definstanceinstance_definstance_ininstance envinstance_ininstance envinstance_listinstance_listinstance_ininstance envinstance_ininstance envinstance_ininstance envinstance_ininstance envinstance_if_genericPexp_tuplety_expecteditselfinstance_inassert(false)instance envinstance_inPexp_arrayinstance envinstance_inPexp_dictinstance envinstance_inPexp_spreadinstance envinstancePexp_templateinstance_definstance_inPexp_tagged_templateinstance envinstance_inPexp_identinstance envinstance_inrueinstance envinstance_ininstance envinstance_ininstance envinstance_ininstance envinstance_ininstance envinstance_inswitchresult, unused checkinstance envinstance_ininstance envinstance_ininstance envinstance_ininstance envinstance_inunify_patsinstance envinstance_inunify_patsinstance envinstance_ininstance envinstance_inswitchwith GADTs: shared resultinstance envinstance_inswitchresultinstance envinstance_inswitchresultinstance envinstanceswitchexception casesPredef.type_exnis passed as isinstance_inPexp_matchresultinstance envinstance_ininstance envinstance_ininstance envinstancetryhandlerPredef.type_exnis passed as isinstance_ininstance envinstance_ininstance envinstance_ininstance envinstance_ininstance envinstance_ininstance envinstance_ininstance envinstance_ininstance envinstanceinstance_definstance_ininstance envinstance_list_ininstance_list envinstance_ininstance envinstance_ininstance envinstance_if_generic_ininstance envinstance_list_ininstance_list envinstanceinstance_definstance_ininstance envinstanceinstance_definstance_ininstance envinstance_ininstance envinstance_ininstance envinstance_ininstance envinstance_ininstance envinstanceinstance_ininstance envinstanceinstance_definstance_ininstance envinstanceinstance_definstance_ininstance envinstance_ininstance envinstance_inPexp_constraintinstance envinstance_inPexp_constraintinstance envinstance_if_generic_inPexp_coerce(rue)instance envinstance_ininstance envinstance_ininstance envinstance_inPexp_packinstance envinstance_inwrap_unpacksidentifierinstance envinstance_inletresultinstance envinstanceinstance_definstanceinstance_definstanceinstance_definstanceinstance_definstanceinstance_definstanceinstance_definstanceinstance_defPrograms for the corrected sites. I tried 21 programs, each in a case with
Int: t<int>, through both compilers on the head beforede5bef189and on the final head:rueand the otherinstance envsites, 15 programs: a function generalized in the case, passed through a constructor, a record field, a coercion, an object field, a tuple,Some(g), a polymorphic field and a destructuringlet;Box(y, 0); nested constructors; labels{p: y, q: 0}; andassert(false)in a branch and in a generalized record field.v: a, built with0ory, assignedm.v = 0, read after the case and behind anif, beside a second fieldw: int, and beside a type parameter.None is a witness, so none became a fixture. The sites are corrected because the copy must do what OCaml's does.
Open
No program of the 64 shows a difference at these OCaml sites, which are not ported (the port stopped there, at the task's size limit):
unify2after expansion (ctype.ml:2293-2298);unify_fields(ctype.ml:2490) and onunify_row's row variable (ctype.ml:2609-2611);unify_var(2759-2766),expand_head_trace(2782-2784), the application'slower_args(typecore.ml:2828) andlocal_non_recursive_abbrev(1533-1534);instance ~partial env.type_casesadds attypecore.ml:4913-4914(see Counts), a difference without a program.instance envsites with no Rust copy (see the copy-site table).The probes written for each of them agree; they are listed in the document. The printer arm was not touched: its condition is OCaml's.
Verification
All 64 programs ran through both compilers, Rust before (trunk
109b2bd4c) and after. Summary (the full table is in the document):Fixtures: 30 super_errors
gadt_instance_ambiguous_*(snapshots produced by the OCaml bsc) andtests/tests/src/gadt_instance_ambiguous_accepted.res(31 accepted programs).Gates (on
5f8d821f9, the last code commit; the commits after it change only the document)Programs: 26/26 agree with OCaml. They are the 21 above, three exhaustiveness checks of a
switchwith a generalized type under_, and t01/t02 aimed at the guard.Note
The sandbox was reset before this turn, which deleted the scratch copies of the earlier 64 + 5 programs. Of the 64:
gadt_instance_ambiguous_accepted.res../scripts/test_error_output_parity.sh --release:./scripts/test_js_output_parity.sh --release:cargo clippy --locked --manifest-path compiler-rust/Cargo.toml --all-targets -- -D warnings: exit 0.RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --manifest-path compiler-rust/Cargo.toml: exit 0.cargo test --manifest-path compiler-rust/Cargo.toml --lib tc::differential:cargo test --lib gadt_instance: 2 passed, including the new chain step test.cargo test --lib types::has 367 passed and 2-3 failures intypes::module_view("shared type index refers to an unpublished page"). They pass alone, and trunk109b2bd4cfails the same way in the same run (366 passed; 2 failed, same tests, same message), so this is a pre-existing race between parallel tests.On a 3,735-module application,
scripts/compare_checker_builds.pybetween trunk and this branch:tests/ERROR_VARIANTS.mdis unchanged: no error variant was added or removed.Started by Jaap Frolich in this Assistant session