Skip to content

feat!: remove the knowledge store - #60

Draft
cryptotavares wants to merge 2 commits into
cryptotavares/mmcli-p1-03-no-prior-knowledgefrom
cryptotavares/mmcli-p1-04-remove-knowledge-store
Draft

cryptotavares wants to merge 2 commits into
cryptotavares/mmcli-p1-03-no-prior-knowledgefrom
cryptotavares/mmcli-p1-04-remove-knowledge-store

Conversation

@cryptotavares

Copy link
Copy Markdown
Collaborator

Description

Every tool call wrote a step record to test-artifacts/llm-knowledge/<session>/steps/ (recordToolStep in create-server.ts). The four knowledge_* tools read those records back. Nothing uses this at run time. The store was ~2250 lines of source, 4 tools, 2 launch flags and an always-on disk write per call. This PR removes it, which is decision D6 of the mm CLI v2 plan.

  • Delete src/knowledge-store/ (store, tokenization, redaction), src/tools/knowledge.ts, src/tools/types/knowledge.ts and src/tools/types/step-record.ts.
  • Remove the knowledge_last/search/summarize/sessions tools, schemas and mm knowledge-* commands, and ErrorCodes.MM_KNOWLEDGE_ERROR.
  • Remove launch goal/flowTags (--goal, --flow-tags), and knowledgeStore from ToolContext/ServerConfig.
  • Remove step recording from the server, and the helpers that only fed it.
  • Kept and moved:
    • StepRecordObservation and createDefaultObservation → src/tools/types/observation.ts
    • SessionMetadata → src/tools/types/session.ts (without goal/flowTags)
  • New writeSessionMetadata(artifactsDir, metadata) writes <artifactsDir>/<sessionId>/session.json. It rejects unsafe ids, returns the absolute path and ends the file with a newline.
  • Observation collection is unchanged. A later PR in the stack limits it to mutating tools.

Source: +107 / −3173 lines. Tests: 1636 → 1397, because the deleted modules' tests are gone. Coverage goes up.

Part of the mm CLI v2 effort: ticket P1-04 (Phase 1).
Stacked on #59 (review that first).

Breaking changes

  • Removed public exports:
    • Store: KnowledgeStore, KnowledgeStoreConfig, createKnowledgeStore, setKnowledgeStore, getKnowledgeStore, hasKnowledgeStore, knowledgeStore
    • Tokenization: tokenize, tokenizeIdentifier, expandWithSynonyms
    • Tools and schemas: the knowledge*Tool functions, knowledge*InputSchema, knowledgeScopeSchema, knowledgeFiltersSchema, Knowledge*Z
    • Constants and types: FLOW_TAGS, FlowTag, STEP_LABELS, StepLabel, the StepRecord* types except StepRecordObservation, the PriorKnowledge*, Knowledge*, RecipeStep and SessionSummary types
    • Other: ErrorCodes.MM_KNOWLEDGE_ERROR, extractTargetFromInput, extractScreenshotInfo, extractToolOutcome
  • ToolName no longer includes knowledge_*.
  • ServerConfig.knowledgeStore and ToolContext.knowledgeStore are removed.
  • Launch input no longer accepts goal/flowTags, and SessionMetadata no longer has them. mm launch --goal now gives the unknown-flag warning.
  • mm knowledge-* commands are removed (unknown command, exit 1).
  • Path change: session metadata moves from test-artifacts/llm-knowledge/<id>/session.json to <artifactsDir>/<id>/session.json (test-artifacts/<id>/session.json with the consumers' artifactsDir: 'test-artifacts'). Consumers call writeSessionMetadata() themselves.

Consumer impact

Consumer Change needed Done on branch
Extension test/e2e/playwright/llm-workflow/ Drop KnowledgeStore/setKnowledgeStore from daemon.ts; metamask-provider.ts calls writeSessionMetadata(artifactsDir, …); no goal/flowTags; tests; README, AGENTS.md cryptotavares/mm-cli-v2-extension 45c821d168a
Mobile tests/llm-workflow/ Same (write stays non-blocking) cryptotavares/mm-cli-v2-mobile 4e2661f209f

Proof

Tested with a yalc build of this branch (0.12.0+f2caa574) in the Extension (e2e test build 13.56.0, headless) and Mobile (dev client 8.19.0, iOS sim + Android emulator), with the consumer commits above.

Check Before (#59) After
Files written under test-artifacts/llm-knowledge/ during a session one step file per tool call none (ext 116 → 116 files, mobile 260 → 260)
session.json per launch llm-knowledge/<id>/session.json test-artifacts/<id>/session.json (ext ×2, iOS, Android)
mm knowledge-last step list Error: unknown command 'knowledge-last', exit 1
mm launch --goal x accepted Warning: unknown launch flag '--goal', launch ok
Smoke (ext / iOS / Android) pass pass / pass / pass (Android list-testids 30 s timeout = known B11)
rg -i knowledge src many only the tests asserting the commands are gone
Coverage (stmts / branch / funcs / lines) 96.38 / 91.7 / 94.63 / 96.62 96.92 / 93.38 / 94.84 / 97.1
Transcripts
$ yarn mm knowledge-last
Error: unknown command 'knowledge-last'
exit=1

$ yarn mm launch --context e2e --state default --goal x
Warning: unknown launch flag '--goal'
... (launch ok, exit 0)

$ find test-artifacts -newer <marker> -type f      # Extension
test-artifacts/mm-mvl3vmad-zd909k/session.json
test-artifacts/mm-mvl3whgy-3sm2ye/session.json
test-artifacts/screenshots/p1-04-ext-home-1792768182733.png

$ cat test-artifacts/mm-mvl3vmad-zd909k/session.json
{ "schemaVersion": 1, "sessionId": "mm-mvl3vmad-zd909k", "createdAt": "...", "tags": [], "launch": { "stateMode": "default", ... } }
Unit tests
core:       Test Files 46 passed (46) · Tests 1397 passed (1397) · Type Errors no errors
            All files | 96.92 | 93.38 | 94.84 | 97.1
Extension:  yarn jest test/e2e/playwright/llm-workflow → 12 suites, 144 tests passed
Mobile:     yarn test:unit:llm-workflow → 18 suites, 231 tests passed

Every tool call wrote a step record to test-artifacts/llm-knowledge/ and the
knowledge_* tools read them back. Nothing uses this at run time, so it goes:

- delete src/knowledge-store (store, tokenization, redaction),
  src/tools/knowledge.ts and the knowledge/step-record types
- remove the knowledge_last/search/summarize/sessions tools, schemas and
  `mm knowledge-*` commands, and ErrorCodes.MM_KNOWLEDGE_ERROR
- remove launch `goal`/`flowTags` (`--goal`, `--flow-tags`)
- remove `knowledgeStore` from ToolContext and ServerConfig, and step
  recording from the server

Kept and moved:
- StepRecordObservation and createDefaultObservation -> tools/types/observation.ts
- SessionMetadata -> tools/types/session.ts (without goal/flowTags)
- new writeSessionMetadata(artifactsDir, metadata) writes
  <artifactsDir>/<sessionId>/session.json

Observation collection is unchanged.

BREAKING CHANGE: Removed public exports: KnowledgeStore, KnowledgeStoreConfig,
createKnowledgeStore, setKnowledgeStore, getKnowledgeStore, hasKnowledgeStore,
knowledgeStore, tokenize, tokenizeIdentifier, expandWithSynonyms, the
knowledge*Tool functions, knowledge*InputSchema and Knowledge*Z, FLOW_TAGS,
STEP_LABELS, StepRecord* types (except StepRecordObservation), PriorKnowledge*
and Knowledge* types, ErrorCodes.MM_KNOWLEDGE_ERROR, extractTargetFromInput,
extractScreenshotInfo, extractToolOutcome. ServerConfig and ToolContext no
longer have knowledgeStore. Launch input no longer accepts goal/flowTags and
SessionMetadata no longer has them. `mm knowledge-*` commands are gone.
Session metadata moves from test-artifacts/llm-knowledge/<id>/session.json to
<artifactsDir>/<id>/session.json, written by writeSessionMetadata().

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant