Repository navigation
GEAK fallback can promote candidates rejected by native accuracy validation #1433
Description
Hyperloom's GEAK fallback can promote a throughput result after native validation rejects accuracy. This reproduces on the former PR #1428 head ecf07f0b7bd8ffb2ae65bcafea1927e0275d1d8f using the actual Coordinator, native sweep and promotion methods with CPU fixture subprocesses.
With baseline accuracy 0.94 and current-best throughput 110, missing evaluation or fresh accuracy 0.2 can still lead to throughput 150 being promoted and a 50% validated gain recorded. Native quality controls reject both. A healthy-quality control and a failed-process control establish the intended positive and negative paths.
The route has two defects:
- Explore can emit
per_variant_outcomeswithREVERT / accuracy_droporREVERT / accuracy_unavailable, no winner, and no output throughput while the task itself succeeds because it produced measurements. GEAK writeback treats missing throughput as inconclusive and invokes another harness. - The fallback sweep drops fresh evaluation artifacts, and
_validate_geak_via_geak_harnesschecks throughput and GEAK's claimed speedup without applying the native accuracy requirement.
The correction is published in existing PR #1428 at f89eedaeff3a4499d7f068394c05aff633bba841 and awaits review/merge. It:
- Preserves the matching current-round native accuracy REVERT, closes the candidate with the existing no-promote tombstone, and withdraws provisional journey KEEPs.
- Reads evaluation only from the fresh replay workspace through the existing native
parse_eval_resultsfunction. - Requires valid fresh accuracy and the unchanged native drop tolerance when a positive baseline accuracy is recorded. Old result or throughput-summary scalar accuracy cannot substitute for evaluation artifacts.
- Carries the accepted fresh accuracy into the optimization stack, which CLOSE already reads.
The same sweep has an independent return-configuration defect: it ignores authoritative accepted_config.env_map. The correction reuses the existing shared normalizer, with the actual shell transport correction published in AMD-AGI/GEAK#452 at 812819d82f6baf6ff9fc694eeaff33aba06a1ca5. Twelve real Hyperloom → copied GEAK adapters → Python-S decoder → child-environment controls pass. Parser-only round trips are insufficient.
Final immutable candidate validation: 20,719 full-suite tests passed, 43 skipped, eight expected failures and 15 passing subtests, plus whole-repository Ruff, formatting, REUSE and whitespace checks. Independent source review and 128 focused tests pass. The actual-source reproduction retains 13 corrected cases, fresh accuracy-retention controls, empty-map no-material closure and artifact-only fallback controls. Original vulnerable-source controls and all outputs are retained. No accuracy or performance threshold changed. Final receipt SHA256 688bc268081e0acd1c22725ef5d11fd99e8b9208e3e0f96d0940062226df1bec.
This establishes a current control-flow defect. It does not establish that this exact route caused the historical MiniMax KEEP with degraded accuracy, whose exact executable/state remain incomplete. It is also not a GPU performance or dashboard-recovery claim.
Evidence under /home/sapmajum/work/geak_ci_delivery_20260907/composed_validation/hyperloom_fallback_accuracy_audit:
- Original replay receipt SHA256
beed72623418c17dd16a4e5bf1f94ab26f744348d73fdede0d5083b962b1ca93. - Corrected-source 13-case receipt SHA256
50412445696274f15429782d6c38f0d7c8a305ccb189036ba46eab1577d53173. - Independent review SHA256
8d6579c855d937a436e99b1c58ff631d75d21d2449249a7ff6029ded0c74f778. - Fresh-accuracy stack-retention receipt SHA256
1c10c10972f2e948c5dee9c737c902688222989de810fb6b68526f2508659aad.
Activity
chaojhou commentedon Oct 20, 2026
Checked against current
main: both defects are fixed by #1428 (bacd8e7) and still hold after the later writeback refactors.WritebackCollaborator.promote_to_shared_state(loop/writeback.py) readsper_variant_outcomes. A matching current-roundREVERTwithaccuracy_drop,accuracy_unavailableorintvty_regressioncloses the candidate as no-promote and never reaches the GEAK-harness fallback.validate_geak_via_geak_harnessrequires a valid fresh replay accuracy whenever a positive baseline accuracy is recorded and applies the nativeaccuracy_passedtolerance. Missing or failing accuracy returnsvalidated: False, status: "no_promote".Pinned by
test_geak_gain_alignment.py::test_2b_native_revert_is_conclusive,::test_geak_harness_rejects_missing_or_failed_fresh_accuracyand::test_geak_harness_accepts_fresh_accuracy_within_native_tolerance, and bytest_geak_sweep_units.py::test_sweep_replays_normalized_return_environment_and_fresh_accuracy.Nothing is left to do here, so this issue will close on its own. If a run on current
mainshows the problem again, please open a new issue with the session and head.github-actions commentedon Oct 22, 2026
This issue has been quiet for about 2 days since our last reply.
It will be closed automatically in 1 day if there is no further activity.
If you are still working on this, please leave a comment with any update — that will reset the timer.