diff --git a/CHANGELOG.md b/CHANGELOG.md index 9648896..db99673 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,23 @@ All notable changes to `learn`. Versions follow semantic versioning; each minor release was built behind the `feat/learning-loop` branch and reviewed before merge. +## Unreleased + +- `tutor/reverify.mjs`: tutor-receipt re-verification with typed failure codes. `reverifyReceipt` + recomputes a receipt's own evidence instead of trusting its stored booleans (`verified` / + `ledgerVerified` are author-controlled and deliberately ignored): the hash chain over the + witnessed practice entries must recompute (a break is typed `CHAIN_BROKEN` with the offending + entry's seq and hash; a hash-consistent truncation is caught by attempt accounting) and the + stored mastery verdict must re-derive from the recorded attempts under the recorded policy (a + divergence is typed `VERDICT_MISMATCH` with both projections). A chainless receipt re-verifies + as `UNVERIFIED`, never as verified. A clean re-check carries a witnessed summary digest. +- CLI: `learn tutor reverify [--file ]` exits 0 only when every checked receipt + re-verifies as VERIFIED; any typed failure or UNVERIFIED receipt exits 1. +- MCP: `learn_tutor_reverify` (advisory, read-only). +- `doctor` gains `tutor.reverify_rejects_known_bad`: the re-verifier must pass a clean receipt and + reject each known-bad fixture (tampered chain entry, hand-edited verdict, chainless receipt). + A verifier that cannot fail on a known-bad input is not a verifier. + ## 1.5.0 The learning loop reaches its first complete shape: every planned teach-you capability is shipped diff --git a/README.md b/README.md index 508c09d..4728d08 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ ## Try it ```bash -node --test # 167 tests +node --test # 206 tests node src/cli.mjs status node src/cli.mjs doctor ``` @@ -68,6 +68,12 @@ capability matches where you are stuck: - **`learn tutor study`** (`study.mjs`): the orchestrator. Composes due + misconceptions + interleaved order + prerequisite readiness + the mastery-gate into one plan, and a witnessed, hash-chained `study-receipt` you can keep. +- **Receipt re-verification** (`reverify.mjs`): `learn tutor reverify` recomputes an emitted + receipt's own evidence instead of trusting its stored booleans. The hash chain over the + witnessed practice entries must recompute (a break is typed `CHAIN_BROKEN` with the offending + entry) and the stored mastery verdict must re-derive from the recorded attempts under the + recorded policy (a divergence is typed `VERDICT_MISMATCH`). A chainless receipt is `UNVERIFIED`, + never verified; a clean re-check exits 0 with a witnessed summary digest. Underneath the learning loop, the original credential-logistics engine still runs workflows (`learn run` / `learn resume`), halts at every graded `assess` step for you to complete yourself, @@ -89,13 +95,14 @@ and emits a provenance receipt separating automated logistics from your own grad - **Release:** `1.5.0`; command `learn`; Node >= 20; zero external dependencies (ES modules, `node:test`). - **Operator surface:** `learn status`, `learn doctor`, `learn run/resume/verify/receipt`, - `learn assist`, `learn visualize`, and `learn tutor `. The zero-dep MCP server (`src/mcp.mjs`) exposes the advisory/read tools: `learn_doctor`, `learn_status`, `learn_verify`, `learn_receipt`, `learn_dry_run`, `learn_tutor_plan`, `learn_tutor_record`, `learn_tutor_mastery`, `learn_tutor_due`, `learn_tutor_studyplan`, `learn_tutor_misconceptions`, - and `learn_visualize_dry_run`. Actuation (real runs) stays operator-driven on the CLI. -- **Test floor:** 167 tests across the runtime, adapters, receipt, tutor/learning-loop, and telos + `learn_tutor_reverify`, and `learn_visualize_dry_run`. Actuation (real runs) stays + operator-driven on the CLI. +- **Test floor:** 206 tests across the runtime, adapters, receipt, tutor/learning-loop, and telos interop, including a falsifiable test per integrity invariant. - **Housekeeping:** see [CHANGELOG.md](CHANGELOG.md) for version history. @@ -133,7 +140,8 @@ a render, a visualization, or a pending prediction as if it were a graded answer `schedule.mjs` (spaced repetition), `misconception.mjs` (aggregation), `retrieval.mjs` (cloze generation + interleaving), `explain.mjs` (self-explanation grading), `predict.mjs` (predict-then-observe), `map.mjs` (concept map + prerequisite gating), `study.mjs` (the - orchestrator + witnessed study-receipt), `tutorstore.mjs` (session persistence). + orchestrator + witnessed study-receipt), `reverify.mjs` (receipt re-verification: recomputed + chain + re-derived verdict, typed failures), `tutorstore.mjs` (session persistence). - `interop/telos.mjs`: the visualization bridge. `concept -> math_physics scene-spec -> witnessed AID render`. Delegates rendering to the telos engine over `LEARN_TELOS_CMD` (fail-closed); learn never imports telos internals or picks the renderer profile. @@ -157,6 +165,10 @@ a render, a visualization, or a pending prediction as if it were a graded answer study) generates practice, structures study, or checks the operator's own work, and the `mastery()` verdict is a function of the operator's own scored practice attempts only, never of a render, a visualization, or an unscored pending prediction. +8. A tutor receipt re-verifies from its own recorded evidence. `learn tutor reverify` recomputes + the hash chain and re-derives the mastery verdict; failures are typed (`CHAIN_BROKEN`, + `VERDICT_MISMATCH`), author-controlled booleans never gate, and a chainless receipt is + `UNVERIFIED`, never verified. ## Interop diff --git a/USAGE.md b/USAGE.md index dcc2b57..b2b1464 100644 --- a/USAGE.md +++ b/USAGE.md @@ -35,12 +35,20 @@ node src/cli.mjs tutor misconceptions mysession node src/cli.mjs tutor mastery mysession node src/cli.mjs tutor study mysession --now 2026-06-30T00:00:00Z node src/cli.mjs tutor study-receipt mysession --now 2026-06-30T00:00:00Z +node src/cli.mjs tutor receipt mysession +node src/cli.mjs tutor reverify mysession ``` `learn tutor study` is the one command to run first on an existing session: it composes what is due, what you keep getting wrong, a mixed practice order, and the mastery-gate verdict, all from your own recorded attempts. +`learn tutor reverify` recomputes an emitted receipt's own evidence (hash chain + verdict +re-derivation) instead of trusting its stored booleans. It exits 0 with a witnessed summary only +when every checked receipt re-verifies clean; a tampered chain is typed `CHAIN_BROKEN`, an edited +verdict is typed `VERDICT_MISMATCH`, and a chainless receipt is `UNVERIFIED`, never verified +(all exit 1). Use `--file ` to check a single receipt file directly. + ## Basic usage: the credential/coursework engine ```bash @@ -61,7 +69,7 @@ node src/mcp.mjs Exposes the advisory/read tools (`learn_doctor`, `learn_status`, `learn_verify`, `learn_receipt`, `learn_dry_run`, `learn_tutor_plan`, `learn_tutor_record`, `learn_tutor_mastery`, -`learn_tutor_due`, `learn_tutor_studyplan`, `learn_tutor_misconceptions`, +`learn_tutor_due`, `learn_tutor_studyplan`, `learn_tutor_misconceptions`, `learn_tutor_reverify`, `learn_visualize_dry_run`) over stdio JSON-RPC. Actuation (real workflow runs) stays operator-driven on the CLI; the MCP surface never performs a real course action or answers a graded step. diff --git a/docs/ENTERPRISE-READINESS.md b/docs/ENTERPRISE-READINESS.md index 48b84d5..d5e89ee 100644 --- a/docs/ENTERPRISE-READINESS.md +++ b/docs/ENTERPRISE-READINESS.md @@ -29,7 +29,8 @@ what exact material it relied on, what it changed, what verified, and what remai - `node src/cli.mjs verify ID` and `node src/cli.mjs receipt ID` for ledger replay and receipt emission (JSON, Markdown, HTML). - `node src/cli.mjs tutor plan|record|due|misconceptions|retrieval|explain|predict|score|path| - study|study-receipt|mastery` for the learning loop. + study|study-receipt|mastery` for the learning loop, and `tutor reverify` to recompute an emitted + receipt's evidence (typed `CHAIN_BROKEN` / `VERDICT_MISMATCH` failures; exit 0 only when clean). - `node src/mcp.mjs` for a stdio MCP host exposing the advisory/read surface only. ## Context Envelope Contribution diff --git a/docs/HOW-IT-WORKS.md b/docs/HOW-IT-WORKS.md index f7eb521..18bd3fe 100644 --- a/docs/HOW-IT-WORKS.md +++ b/docs/HOW-IT-WORKS.md @@ -103,6 +103,20 @@ record. The receipt is a quiet floor: it proves the study happened as recorded. grade or unlock anything beyond what `studyPlan` already computed, and it never contains an answer to a certified assessment, only the operator's own recorded practice. +### 9. Receipt re-verification + +```bash +node src/cli.mjs tutor receipt mysession +node src/cli.mjs tutor reverify mysession +``` + +`tutor/reverify.mjs` recomputes an emitted receipt's own evidence instead of trusting its stored +booleans. The hash chain over the witnessed practice entries must recompute (a break is typed +`CHAIN_BROKEN` with the offending entry's seq and hash) and the stored mastery verdict must +re-derive from the recorded attempts under the recorded policy (a divergence is typed +`VERDICT_MISMATCH`). A receipt without chain evidence is `UNVERIFIED`, never verified. A clean +re-check exits 0 with a witnessed summary digest; any failure exits 1. + --- ## Fail-closed, the same way the rest of Project Telos is diff --git a/src/cli.mjs b/src/cli.mjs index b8c56a5..6f425f6 100644 --- a/src/cli.mjs +++ b/src/cli.mjs @@ -151,6 +151,12 @@ export async function main(argv, { dir = process.cwd() } = {}) { ` order: ${plan.order.join(", ")}\n` + ` readiness: ${plan.readiness.map((r) => `${r.objective}:${r.unlocked ? "unlocked" : "locked"}`).join(", ")}` }; } + if (sub === "reverify") { + const { reverifyFiles, formatReverify } = await import("./tutor/reverify.mjs"); + const file = arg(argv, "--file"); + const r = reverifyFiles(dir, id, { file }); + return { code: r.ok ? 0 : 1, out: formatReverify(r, file || id) }; + } if (sub === "study-receipt") { const s = loadSession(dir, id); if (!s) return { code: 1, out: `no tutor session: ${id}` }; const { studyReceipt } = await import("./tutor/study.mjs"); @@ -159,7 +165,7 @@ export async function main(argv, { dir = process.cwd() } = {}) { writeFileSync(join(dir, "tutor", id + ".study-receipt.json"), JSON.stringify(r, null, 2)); return { code: 0, out: `tutor study-receipt ${id}: verified ${r.verified}, mastery ${r.mastery.ready ? "READY" : "not yet"} -> tutor/${id}.study-receipt.json` }; } - return { code: 1, out: "usage: learn tutor ..." }; + return { code: 1, out: "usage: learn tutor ..." }; } if (cmd === "assist") { const { assistArtifacts } = await import("./assist/assist.mjs"); diff --git a/src/doctor.mjs b/src/doctor.mjs index 2f3ac76..fe84b60 100644 --- a/src/doctor.mjs +++ b/src/doctor.mjs @@ -11,6 +11,7 @@ import { telosRender, toAidLedgerEntry } from "./interop/telos.mjs"; import { buildReceipt } from "./receipt/receipt.mjs"; import { newSession, recordAttempt, mastery, recordVisualization } from "./tutor/tutor.mjs"; import { recordPrediction } from "./tutor/predict.mjs"; +import { reverifySelfCheck } from "./tutor/reverify.mjs"; export async function doctor() { const checks = []; @@ -69,6 +70,11 @@ export async function doctor() { masteryAfterPrediction.perObjective[0].attempts === masteryBefore.perObjective[0].attempts + 1 && masteryAfterPrediction.perObjective[0].correct === masteryBefore.perObjective[0].correct); + // 8. the tutor-receipt re-verifier can FAIL: it must pass a clean receipt and reject each + // known-bad fixture (tampered chain entry, hand-edited verdict, chainless receipt). A verifier + // that cannot fail on a known-bad input is not a verifier. + add("tutor.reverify_rejects_known_bad", reverifySelfCheck()); + const ok = checks.every((c) => c.status === "MATCH"); return { tool: "learn", version, status: ok ? "MATCH" : "DEGRADED", checks }; } diff --git a/src/mcp.mjs b/src/mcp.mjs index 11aeb2f..337d567 100644 --- a/src/mcp.mjs +++ b/src/mcp.mjs @@ -14,6 +14,7 @@ import { saveSession, loadSession } from "./tutor/tutorstore.mjs"; import { due } from "./tutor/schedule.mjs"; import { misconceptions } from "./tutor/misconception.mjs"; import { studyPlan } from "./tutor/study.mjs"; +import { reverifyFiles } from "./tutor/reverify.mjs"; import "./adapters/fake.mjs"; import "./adapters/generic.mjs"; import "./adapters/lms.mjs"; @@ -31,6 +32,7 @@ export const TOOLS = [ { name: "learn_tutor_due", description: "Advisory, read-only: list objectives due for spaced-repetition review in a saved tutor session, most-overdue first.", inputSchema: { type: "object", properties: { sessionId: { type: "string" }, now: { type: ["string", "number"] }, asOf: { type: ["string", "number"] } }, required: ["sessionId", "now"] } }, { name: "learn_tutor_studyplan", description: "Advisory, read-only: return the composed study plan for a saved tutor session (due list, ranked misconceptions, interleaved order, prerequisite readiness, mastery-gate verdict).", inputSchema: { type: "object", properties: { sessionId: { type: "string" }, now: { type: ["string", "number"] }, seed: { type: ["string", "number"] } }, required: ["sessionId", "now"] } }, { name: "learn_tutor_misconceptions", description: "Advisory, read-only: return the ranked misconception aggregation (wrong attempts + the operator's own feedback) for a saved tutor session.", inputSchema: { type: "object", properties: { sessionId: { type: "string" } }, required: ["sessionId"] } }, + { name: "learn_tutor_reverify", description: "Advisory, read-only: re-verify a session's emitted tutor receipts from their own recorded evidence (recomputed hash chain + re-derived mastery verdict). Failures are typed CHAIN_BROKEN / VERDICT_MISMATCH; a chainless receipt is UNVERIFIED, never verified.", inputSchema: { type: "object", properties: { sessionId: { type: "string" }, file: { type: "string" } }, required: ["sessionId"] } }, ]; export async function dispatch(name, args = {}, { dir = process.cwd() } = {}) { @@ -76,6 +78,9 @@ export async function dispatch(name, args = {}, { dir = process.cwd() } = {}) { const s = loadSession(dir, args.sessionId); if (!s) throw new Error("no tutor session: " + args.sessionId); return { sessionId: args.sessionId, misconceptions: misconceptions(s) }; } + case "learn_tutor_reverify": { + return { sessionId: args.sessionId, ...reverifyFiles(dir, args.sessionId, { file: args.file }) }; + } default: throw new Error(`unknown tool: ${name}`); } } diff --git a/src/status.mjs b/src/status.mjs index f99437a..23fad65 100644 --- a/src/status.mjs +++ b/src/status.mjs @@ -26,6 +26,7 @@ export function status() { predict: "predict-then-observe: records the operator's OWN prediction as a pending attempt, scored only after they compare it to a rendered aid observation", map: "normalizes objectives (string or {id,text,requires}), computes a topological learningPath, and gates readiness on prerequisite mastery", study: "orchestrator composing due + misconceptions + interleaved order + readiness + mastery into one studyPlan, and a witnessed hash-chained studyReceipt", + reverify: "re-verifies an emitted tutor receipt from its own recorded evidence: the hash chain is recomputed and the mastery verdict re-derived under the recorded policy; failures are typed CHAIN_BROKEN / VERDICT_MISMATCH, and a chainless receipt is UNVERIFIED, never verified", boundary: "every learning-loop capability generates practice, structures study, or checks the operator's OWN work — never produces, hints, or auto-fills an answer to a certified/graded assessment", }, integrityInvariants: [ @@ -35,6 +36,7 @@ export function status() { "the receipt separates automated logistics from human assessment", "credentials, payment, CAPTCHA, and account creation halt for the operator", "aid visualizations are learning aids only — they are witnessed but never satisfy an assess step or enter the graded receipt channels", + "tutor receipts re-verify from their own recorded evidence, never from author-controlled booleans (CHAIN_BROKEN / VERDICT_MISMATCH; chainless receipts are never verified)", ], boundary: "The engine never produces graded work. It is a learning aid, not a bypass.", }; diff --git a/src/tutor/reverify.mjs b/src/tutor/reverify.mjs new file mode 100644 index 0000000..44085fb --- /dev/null +++ b/src/tutor/reverify.mjs @@ -0,0 +1,173 @@ +// Tutor-receipt re-verification. Recomputes a receipt's own evidence instead of trusting its +// self-reported booleans, with two typed failure codes: +// +// CHAIN_BROKEN -- the hash chain over the receipt's witnessed practice entries does not +// recompute (tampered, reordered, or unaccounted-for entry); reported with +// the offending entry's seq and stored hash. +// VERDICT_MISMATCH -- the stored mastery verdict does not re-derive from the receipt's own +// recorded practice evidence under the receipt's own recorded policy +// (threshold / minAttempts). +// +// INTEGRITY: the verdict here keys off recomputed hashes and a re-derived mastery result ONLY. +// The receipt's own `verified` / `ledgerVerified` fields are author-controlled and deliberately +// ignored. A receipt without chain evidence (no `entries`) re-verifies as UNVERIFIED, never as +// verified. The policy inputs (threshold, minAttempts) are part of the receipt's claim surface: +// re-verification proves the verdict follows from the recorded evidence under the recorded +// policy, not that the policy itself was a good one. +import { readFileSync, existsSync } from "node:fs"; +import { join } from "node:path"; +import { Ledger } from "../accountability/ledger.mjs"; +import { observe } from "../accountability/witness.mjs"; +import { newSession, recordAttempt, mastery, masteryReceipt } from "./tutor.mjs"; + +export const CHAIN_BROKEN = "CHAIN_BROKEN"; +export const VERDICT_MISMATCH = "VERDICT_MISMATCH"; + +const GENESIS_HEAD = "0".repeat(64); + +// The comparable claim a mastery verdict makes, as a canonical string (stable key order). +function masteryProjection(m) { + return JSON.stringify({ + ready: m.ready === true, + weakest: [...(m.weakest || [])], + perObjective: (m.perObjective || []).map((p) => ({ + objective: p.objective, attempts: p.attempts, correct: p.correct, accuracy: p.accuracy, ready: p.ready === true, + })), + }); +} + +// Structural gaps make a receipt UNVERIFIED (cannot be checked), never verified. +function structuralReasons(receipt) { + const reasons = []; + if (!receipt || typeof receipt !== "object" || Array.isArray(receipt)) return ["not a receipt object"]; + if (!Array.isArray(receipt.entries)) reasons.push("no hash-chained entries recorded (chainless receipt); chainless receipts are never verified"); + const m = receipt.mastery; + if (!m || typeof m !== "object" || typeof m.ready !== "boolean") reasons.push("no stored mastery verdict to re-derive"); + else if (!Number.isFinite(m.threshold) || !Number.isFinite(m.minAttempts)) reasons.push("stored mastery verdict lacks its recorded policy (threshold / minAttempts)"); + return reasons; +} + +// (a) Chain integrity: seq ordering, recomputed hash linkage, and attempt accounting. +function checkChain(receipt, failures) { + const rows = receipt.entries; + for (let i = 0; i < rows.length; i++) { + if (!rows[i] || rows[i].seq !== i) { + failures.push({ code: CHAIN_BROKEN, seq: i, hash: (rows[i] && rows[i].hash) || null, detail: `entry at index ${i} carries seq ${rows[i] && rows[i].seq} (expected ${i})` }); + return; + } + } + const v = Ledger.fromEntries(rows).verify(); + if (!v.ok) { + failures.push({ code: CHAIN_BROKEN, seq: v.brokenAt, hash: rows[v.brokenAt].hash, detail: `hash chain does not recompute at entry seq ${v.brokenAt}` }); + return; + } + if (typeof receipt.totalAttempts === "number" && receipt.totalAttempts !== rows.length) { + failures.push({ code: CHAIN_BROKEN, seq: rows.length, hash: null, detail: `chain carries ${rows.length} entr(ies) but the receipt claims totalAttempts ${receipt.totalAttempts} (truncated or padded)` }); + } +} + +// (b) Verdict re-derivation: recompute mastery() from the chained practice entries under the +// receipt's recorded policy and compare it to the stored verdict. +function checkVerdict(receipt, failures) { + const stored = receipt.mastery; + const attempts = receipt.entries + .map((r) => r && r.entry) + .filter((e) => e && e.kind === "practice") + .map((e) => ({ objective: e.objective, correct: e.correct })); + const rederived = mastery( + { topic: receipt.topic, objectives: receipt.objectives || [], attempts }, + { threshold: stored.threshold, minAttempts: stored.minAttempts }, + ); + if (masteryProjection(rederived) !== masteryProjection(stored)) { + failures.push({ + code: VERDICT_MISMATCH, + detail: `stored mastery verdict (ready=${stored.ready}) does not re-derive from the receipt's own ${attempts.length} recorded practice entr(ies) (re-derived ready=${rederived.ready})`, + stored: JSON.parse(masteryProjection(stored)), + rederived: JSON.parse(masteryProjection(rederived)), + }); + } + return rederived; +} + +// reverifyReceipt(receipt) -> {verdict, failures, reasons, summary, witness} +// verdict: "VERIFIED" (evidence recomputes clean) | "FAILED" (typed failures) | "UNVERIFIED" +// (structurally uncheckable, e.g. chainless). The witness is a content-addressed digest of the +// re-check summary, so the outcome itself is carried as evidence, not as prose. +export function reverifyReceipt(receipt) { + const reasons = structuralReasons(receipt); + if (reasons.length) return { verdict: "UNVERIFIED", failures: [], reasons, summary: null, witness: null }; + + const failures = []; + checkChain(receipt, failures); + const rederived = checkVerdict(receipt, failures); + const headHash = receipt.entries.length ? receipt.entries[receipt.entries.length - 1].hash : GENESIS_HEAD; + const verdict = failures.length ? "FAILED" : "VERIFIED"; + const summary = { + entries: receipt.entries.length, + headHash, + storedReady: receipt.mastery.ready, + rederivedReady: rederived.ready, + failures: failures.map((f) => f.code), + }; + const witness = observe({ organ: "learn.tutor.reverify", subject: "head:" + headHash, summary: "reverify " + verdict, payload: JSON.stringify(summary), data: summary }); + return { verdict, failures, reasons: [], summary, witness }; +} + +// reverifyFiles(dir, id, {file}) -> {ok, results[, error]}. Without --file it re-verifies every +// receipt the CLI emits for a session (tutor/.mastery.json, tutor/.study-receipt.json). +// `ok` is true only when every checked receipt re-verifies as VERIFIED; UNVERIFIED is fail-closed. +export function reverifyFiles(dir, id, { file = null } = {}) { + const candidates = file + ? [file] + : [join(dir, "tutor", id + ".mastery.json"), join(dir, "tutor", id + ".study-receipt.json")].filter((p) => existsSync(p)); + if (!candidates.length) { + return { ok: false, results: [], error: `no tutor receipt found for ${id} (looked for tutor/${id}.mastery.json and tutor/${id}.study-receipt.json; run \`learn tutor receipt\` or \`learn tutor study-receipt\` first)` }; + } + const results = candidates.map((p) => { + let receipt; + try { receipt = JSON.parse(readFileSync(p, "utf8")); } + catch (e) { return { file: p, verdict: "UNVERIFIED", failures: [], reasons: ["unreadable or malformed receipt file: " + ((e && e.message) || e)], summary: null, witness: null }; } + return { file: p, ...reverifyReceipt(receipt) }; + }); + return { ok: results.every((r) => r.verdict === "VERIFIED"), results }; +} + +// CLI presentation for a reverifyFiles result. +export function formatReverify(r, label) { + if (r.error) return `tutor reverify ${label}: ${r.error}`; + const lines = [`tutor reverify ${label}: ${r.ok ? "VERIFIED" : "NOT VERIFIED"} (${r.results.length} receipt(s))`]; + for (const res of r.results) { + lines.push(` [${res.verdict}] ${res.file}`); + for (const f of res.failures) { + lines.push(` ${f.code}${f.seq !== undefined ? ` @entry seq=${f.seq}${f.hash ? " hash=" + f.hash : ""}` : ""}: ${f.detail}`); + } + for (const why of res.reasons) lines.push(` ${why}`); + if (res.verdict === "VERIFIED") { + lines.push(` witnessed: ${res.witness.digest} (${res.summary.entries} entr(ies), head ${res.summary.headHash.slice(0, 12)}, mastery ${res.summary.rederivedReady ? "READY" : "not yet"} re-derived)`); + } + } + return lines.join("\n"); +} + +// Falsifiable self-check for doctor: the re-verifier must PASS a clean receipt and FAIL each +// known-bad input. A verifier that cannot fail on a known-bad input is not a verifier. +export function reverifySelfCheck() { + const s = newSession({ topic: "reverify-probe", objectives: ["x"] }); + for (const q of ["q1", "q2", "q3"]) recordAttempt(s, { objective: "x", prompt: q, answer: "a", correct: true }); + const clean = JSON.parse(JSON.stringify(masteryReceipt(s))); + if (reverifyReceipt(clean).verdict !== "VERIFIED") return false; + + const tampered = JSON.parse(JSON.stringify(clean)); + tampered.entries[1].entry.correct = false; + const t = reverifyReceipt(tampered); + if (!(t.verdict === "FAILED" && t.failures.some((f) => f.code === CHAIN_BROKEN && f.seq === 1))) return false; + + const edited = JSON.parse(JSON.stringify(clean)); + edited.mastery.ready = false; + const e = reverifyReceipt(edited); + if (!(e.verdict === "FAILED" && e.failures.some((f) => f.code === VERDICT_MISMATCH))) return false; + + const chainless = JSON.parse(JSON.stringify(clean)); + delete chainless.entries; + return reverifyReceipt(chainless).verdict === "UNVERIFIED"; +} diff --git a/tests/tutor-reverify.test.mjs b/tests/tutor-reverify.test.mjs new file mode 100644 index 0000000..a00fc59 --- /dev/null +++ b/tests/tutor-reverify.test.mjs @@ -0,0 +1,192 @@ +// Tutor-receipt re-verification: the verifier must PASS a clean receipt and FAIL each known-bad +// input. A verifier that cannot fail on a known-bad input is not a verifier, so every typed +// failure code here has a negative fixture the check rejects. +import { test } from "node:test"; +import assert from "node:assert/strict"; +import { mkdtempSync, writeFileSync, readFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { newSession, recordAttempt, masteryReceipt } from "../src/tutor/tutor.mjs"; +import { studyReceipt } from "../src/tutor/study.mjs"; +import { reverifyReceipt, reverifyFiles, reverifySelfCheck, CHAIN_BROKEN, VERDICT_MISMATCH } from "../src/tutor/reverify.mjs"; +import { main } from "../src/cli.mjs"; +import { dispatch } from "../src/mcp.mjs"; + +const NOW = "2026-06-30T00:00:00.000Z"; + +function masteredSession() { + const s = newSession({ topic: "t", objectives: ["x"] }); + for (const q of ["q1", "q2", "q3"]) recordAttempt(s, { objective: "x", prompt: q, answer: "a", correct: true }); + return s; +} + +// JSON round-trip mirrors what a consumer reads back from disk. +const roundTrip = (r) => JSON.parse(JSON.stringify(r)); + +test("reverifyReceipt: a clean mastery receipt re-verifies as VERIFIED with a witnessed summary", () => { + const r = reverifyReceipt(roundTrip(masteryReceipt(masteredSession()))); + assert.equal(r.verdict, "VERIFIED"); + assert.deepEqual(r.failures, []); + assert.match(r.witness.digest, /^sha256:[0-9a-f]{64}$/); + assert.equal(r.summary.entries, 3); + assert.equal(r.summary.rederivedReady, true); +}); + +test("reverifyReceipt: a clean study receipt re-verifies as VERIFIED", () => { + const r = reverifyReceipt(roundTrip(studyReceipt(masteredSession(), { now: NOW }))); + assert.equal(r.verdict, "VERIFIED"); + assert.deepEqual(r.failures, []); +}); + +test("reverifyReceipt: a tampered MIDDLE entry yields CHAIN_BROKEN with the offending entry id", () => { + const receipt = roundTrip(masteryReceipt(masteredSession())); + receipt.entries[1].entry.correct = false; // tamper the middle receipt entry + const r = reverifyReceipt(receipt); + assert.equal(r.verdict, "FAILED"); + const broken = r.failures.find((f) => f.code === CHAIN_BROKEN); + assert.ok(broken, "expected a CHAIN_BROKEN failure"); + assert.equal(broken.seq, 1); + assert.equal(broken.hash, receipt.entries[1].hash); +}); + +test("reverifyReceipt: author-controlled booleans never gate; a tampered receipt claiming verified still FAILS", () => { + const receipt = roundTrip(masteryReceipt(masteredSession())); + receipt.entries[1].entry.correct = false; + receipt.ledgerVerified = true; // the author says it is fine; the evidence says otherwise + receipt.verified = true; + const r = reverifyReceipt(receipt); + assert.equal(r.verdict, "FAILED"); + assert.ok(r.failures.some((f) => f.code === CHAIN_BROKEN)); +}); + +test("reverifyReceipt: a hand-edited overall verdict yields VERDICT_MISMATCH", () => { + const receipt = roundTrip(masteryReceipt(masteredSession())); + receipt.mastery.ready = false; // chain intact, verdict edited + const r = reverifyReceipt(receipt); + assert.equal(r.verdict, "FAILED"); + const mm = r.failures.find((f) => f.code === VERDICT_MISMATCH); + assert.ok(mm, "expected a VERDICT_MISMATCH failure"); + assert.equal(mm.stored.ready, false); + assert.equal(mm.rederived.ready, true); +}); + +test("reverifyReceipt: a hand-edited per-objective stat yields VERDICT_MISMATCH", () => { + const receipt = roundTrip(masteryReceipt(masteredSession())); + receipt.mastery.perObjective[0].accuracy = 0.5; + const r = reverifyReceipt(receipt); + assert.equal(r.verdict, "FAILED"); + assert.ok(r.failures.some((f) => f.code === VERDICT_MISMATCH)); +}); + +test("reverifyReceipt: a truncated chain (dropped tail entry) yields CHAIN_BROKEN via attempt accounting", () => { + const receipt = roundTrip(masteryReceipt(masteredSession())); + receipt.entries.pop(); // hash-consistent truncation; totalAttempts still claims 3 + const r = reverifyReceipt(receipt); + assert.equal(r.verdict, "FAILED"); + assert.ok(r.failures.some((f) => f.code === CHAIN_BROKEN)); +}); + +test("reverifyReceipt: a chainless receipt is UNVERIFIED, never verified", () => { + const receipt = roundTrip(masteryReceipt(masteredSession())); + delete receipt.entries; + const r = reverifyReceipt(receipt); + assert.equal(r.verdict, "UNVERIFIED"); + assert.ok(r.reasons.length > 0); +}); + +test("reverifyReceipt: a receipt with no stored verdict is UNVERIFIED", () => { + const receipt = roundTrip(masteryReceipt(masteredSession())); + delete receipt.mastery; + const r = reverifyReceipt(receipt); + assert.equal(r.verdict, "UNVERIFIED"); +}); + +test("reverifySelfCheck: the doctor probe passes (clean passes, every known-bad input fails)", () => { + assert.equal(reverifySelfCheck(), true); +}); + +// ---- CLI ---- + +async function seedReceipts(dir, id) { + await main(["tutor", "plan", id, "--topic", "t", "--objectives", "x"], { dir }); + for (const q of ["q1", "q2", "q3"]) { + await main(["tutor", "record", id, "--objective", "x", "--prompt", q, "--answer", "a", "--correct", "true"], { dir }); + } + await main(["tutor", "receipt", id], { dir }); + await main(["tutor", "study-receipt", id, "--now", NOW], { dir }); +} + +test("learn tutor reverify : clean receipts exit 0 with a witnessed summary", async () => { + const dir = mkdtempSync(join(tmpdir(), "learn-")); + await seedReceipts(dir, "rv1"); + const r = await main(["tutor", "reverify", "rv1"], { dir }); + assert.equal(r.code, 0); + assert.match(r.out, /VERIFIED/); + assert.match(r.out, /witnessed: sha256:[0-9a-f]{64}/); +}); + +test("learn tutor reverify : a tampered middle entry on disk exits 1 with CHAIN_BROKEN", async () => { + const dir = mkdtempSync(join(tmpdir(), "learn-")); + await seedReceipts(dir, "rv2"); + const p = join(dir, "tutor", "rv2.mastery.json"); + const receipt = JSON.parse(readFileSync(p, "utf8")); + receipt.entries[1].entry.correct = false; + writeFileSync(p, JSON.stringify(receipt, null, 2)); + const r = await main(["tutor", "reverify", "rv2"], { dir }); + assert.equal(r.code, 1); + assert.match(r.out, /CHAIN_BROKEN/); + assert.match(r.out, /seq=1/); +}); + +test("learn tutor reverify : a hand-edited verdict on disk exits 1 with VERDICT_MISMATCH", async () => { + const dir = mkdtempSync(join(tmpdir(), "learn-")); + await seedReceipts(dir, "rv3"); + const p = join(dir, "tutor", "rv3.study-receipt.json"); + const receipt = JSON.parse(readFileSync(p, "utf8")); + receipt.mastery.ready = false; + writeFileSync(p, JSON.stringify(receipt, null, 2)); + const r = await main(["tutor", "reverify", "rv3"], { dir }); + assert.equal(r.code, 1); + assert.match(r.out, /VERDICT_MISMATCH/); +}); + +test("learn tutor reverify --file: a chainless receipt file exits 1 as UNVERIFIED", async () => { + const dir = mkdtempSync(join(tmpdir(), "learn-")); + const p = join(dir, "chainless.json"); + writeFileSync(p, JSON.stringify({ topic: "t", objectives: ["x"], mastery: { ready: true, threshold: 0.8, minAttempts: 3, perObjective: [], weakest: [] } })); + const r = await main(["tutor", "reverify", "--file", p], { dir }); + assert.equal(r.code, 1); + assert.match(r.out, /UNVERIFIED/); +}); + +test("learn tutor reverify : no receipt on disk exits 1 with a pointer to emit one first", async () => { + const dir = mkdtempSync(join(tmpdir(), "learn-")); + const r = await main(["tutor", "reverify", "nosuch"], { dir }); + assert.equal(r.code, 1); + assert.match(r.out, /no tutor receipt/i); +}); + +test("usage string mentions reverify", async () => { + const dir = mkdtempSync(join(tmpdir(), "learn-")); + const r = await main(["tutor"], { dir }); + assert.equal(r.code, 1); + assert.match(r.out, /reverify/); +}); + +// ---- MCP ---- + +test("mcp learn_tutor_reverify: clean receipts return ok true; a tampered file returns ok false with CHAIN_BROKEN", async () => { + const dir = mkdtempSync(join(tmpdir(), "learn-")); + await seedReceipts(dir, "rv4"); + const clean = await dispatch("learn_tutor_reverify", { sessionId: "rv4" }, { dir }); + assert.equal(clean.ok, true); + assert.ok(clean.results.every((x) => x.verdict === "VERIFIED")); + + const p = join(dir, "tutor", "rv4.mastery.json"); + const receipt = JSON.parse(readFileSync(p, "utf8")); + receipt.entries[1].entry.correct = false; + writeFileSync(p, JSON.stringify(receipt, null, 2)); + const bad = await dispatch("learn_tutor_reverify", { sessionId: "rv4" }, { dir }); + assert.equal(bad.ok, false); + assert.ok(bad.results.some((x) => x.failures.some((f) => f.code === "CHAIN_BROKEN"))); +});