Skip to content

fix(selfhost): warn at boot when the public-score-terms allowlist is unset - #9437

Merged
JSONbored merged 1 commit into
mainfrom
fix/ai-review-summary-fallback-detection
Jul 27, 2026
Merged

fix(selfhost): warn at boot when the public-score-terms allowlist is unset#9437
JSONbored merged 1 commit into
mainfrom
fix/ai-review-summary-fallback-detection

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Closes part of #9433.

The #public-score-terms-scoping fix is config-dependent: the code path shipped and is correctly wired at ai-review.ts (allowBareScoreTerm: isPublicScoreTermSafeForRepo(env, input.repoFullName)), but it stays inert until an operator sets LOOPOVER_PUBLIC_SCORE_TERMS_ALLOWED_REPOS.

Unset is the shipped default, and it was the state found in production. In that state the allowlist fails closed, so BARE_SCORE_TERM_PATTERN = /\bscore\w*\b/i is enforced for every repo. sanitizePublicComment throws on a match and toPublicSafe catches it and returns null — all-or-nothing — so any AI review narrative containing score / scores / scoring / scored had its entire assessment discarded in favour of the generic "did not include a separate narrative summary" placeholder. Nits survived, because those are filtered individually.

That is exactly the behaviour composeAdvisoryNotes' own doc comment already documents:

"Metagraphed#8038-class bug: without this, ANY review of code with a legitimately-public score-named field ... had its entire narrative assessment silently discarded in favor of the generic 'did not include a separate narrative summary' fallback -- observed live, recurring."

So the fix shipped, was considered done, and then sat inert with no signal anywhere.

Why a warning rather than a test

No unit test can catch this class. The existing tests verify the mechanism works when enabled; they cannot assert that a production env var was actually set. The failure mode is a correct code path plus an unset variable, which is invisible to CI by construction. Shouting once at boot is the only place this is observable.

Design

Mirrors the existing shouldWarnRagEmbedUnavailable precedent exactly: a pure predicate in queue-intelligence.ts (next to the exemption it guards, so the two can never disagree about what "configured" means), and a single structured console.error at the boot site in server.ts.

Warn-only, deliberately not a hard boot failure. An empty allowlist is the correct configuration for a deployment whose repos genuinely carry private trust/reward data — the operator just needs to make that an informed choice rather than an unnoticed default.

Test plan

  • New tests pin the inert-fix states (unset / empty / whitespace / separators-only) and the configured states
  • A test pins that the warning and isPublicScoreTermSafeForRepo agree on what counts as configured — if they drift, the shout either fires for a configured deployment (noise) or stays silent for an unconfigured one (the failure it exists to catch)
  • Covers the "*" case: it parses as a non-empty entry so the warning is silent, but it grants no repo the exemption (no wildcard escape hatch)
  • npm run typecheck clean
  • Full local suite: 1242 files / 23,358 tests, 0 failures

Note on scope

An earlier draft of this branch also changed hasPublicReviewAssessment to reject the persisted no-narrative placeholder. That was reverted: it made the orchestration hold PRs for manual review and discard the model's nits, which two existing tests correctly pin as unwanted. See the correction on #9432.

…unset

The #public-score-terms-scoping fix is config-dependent: the code path
shipped and is correctly wired, but stays inert until an operator sets
LOOPOVER_PUBLIC_SCORE_TERMS_ALLOWED_REPOS. Unset (the shipped default,
and the state found in production) the bare-`score` public-comment check
is enforced for every repo, so any AI review narrative using the ordinary
word "score"/"scoring" has its WHOLE assessment discarded in favour of the
generic no-narrative placeholder -- exactly the metagraphed#8038 behaviour
the exemption was written to fix, with no signal anywhere.

No unit test can catch this class: tests verify the mechanism works WHEN
enabled, never that a production env var was actually set. Shout once at
boot instead, mirroring shouldWarnRagEmbedUnavailable (pure predicate here,
the console.error in server.ts). Warn-only, not a hard boot failure: an
empty allowlist is the correct configuration for a deployment whose repos
genuinely carry private trust/reward data -- the operator just needs to
make that an informed choice rather than an unnoticed default.
@loopover-orb

loopover-orb Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Warning

⏸️ LoopOver review result - manual review recommended

Review updated: 2026-07-27 19:07:51 UTC

3 files · 1 AI reviewer · 2 blockers · CI green · clean

⏸️ Suggested Action - Manual Review

  • AI review did not produce public notes: The configured AI reviewer returned no usable public assessment for this PR head.

Review summary
AI review is unavailable for this PR head. LoopOver is holding this PR for manual review until the configured AI provider returns a usable public review summary.

Nits — 1 non-blocking
  • AI review did not produce public notes — Fix the configured AI provider, then re-run LoopOver review before relying on the result.

Concerns raised — review before merging

  • No linked issue detected: No closing reference or linked issue number was found in the PR metadata/body. — If this PR is intended to solve an issue, link it explicitly in the PR body.
  • Maintainer requires a linked issue: This repo's maintainer focus manifest requires every PR to reference a tracked issue. — Link the relevant issue (for example Closes #123) before opening the PR.
📋 Copy for AI agents — paste into your coding agent
Fix the following blocker(s) from this PR review:

1. No linked issue detected: No closing reference or linked issue number was found in the PR metadata/body. — If this PR is intended to solve an issue, link it explicitly in the PR body.

2. Maintainer requires a linked issue: This repo's maintainer focus manifest requires every PR to reference a tracked issue. — Link the relevant issue (for example `Closes #123`) before opening the PR.

Decision drivers

  • ❌ Code review — 2 blockers (1 reviewer)
  • ❌ Gate result — Blocking (Repo-configured hard blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ⚠️ Missing No linked issue or no-issue rationale found.
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (no linked issue context).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 13 registered-repo PR(s), 13 merged, 313 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 13 PR(s), 313 issue(s).
Improvement ✅ Minor risk: clean · value: minor
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: not available
  • Official Gittensor activity: 13 PR(s), 313 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Then work through the remaining 2 steps in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

Decision record
  • action: hold · clause: missing_linked_issue
  • config: e22471d3bce2d32df7457fbd1d07df659608224a9829cad347d1cf3290e81f05 · pack: oss-anti-slop · ci: passed
  • record: a69e35dc0b1f730a03c57063b98cd2ba58419ea800609c8521d6ecc0a7724a1b (schema v5, head 8439b72)

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 27, 2026
@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.63%. Comparing base (17045e8) to head (8439b72).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9437      +/-   ##
==========================================
- Coverage   89.53%   88.63%   -0.91%     
==========================================
  Files         843      843              
  Lines      109959   109960       +1     
  Branches    26184    26184              
==========================================
- Hits        98457    97467     -990     
- Misses      10239    11521    +1282     
+ Partials     1263      972     -291     
Flag Coverage Δ
backend 93.61% <100.00%> (-1.65%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/queue-intelligence.ts 100.00% <100.00%> (ø)

... and 3 files with indirect coverage changes

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 27, 2026
@JSONbored
JSONbored merged commit a4894da into main Jul 27, 2026
7 checks passed
@JSONbored
JSONbored deleted the fix/ai-review-summary-fallback-detection branch July 27, 2026 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant