Skip to content

feat(marginnote4): Phase 1 — connected KB type, read-only tools, and sync bridge - #836

Open
evan188199-tech wants to merge 3 commits into
HKUDS:devfrom
evan188199-tech:codex/feat-marginnote4-protocol
Open

feat(marginnote4): Phase 1 — connected KB type, read-only tools, and sync bridge#836
evan188199-tech wants to merge 3 commits into
HKUDS:devfrom
evan188199-tech:codex/feat-marginnote4-protocol

Conversation

@evan188199-tech

Copy link
Copy Markdown
Contributor

Summary

Adds MarginNote 4 as a connected knowledge base type that syncs study data from MN4 via the official Add-on API into a dedicated SQLite store, then exposes it to the chat loop through seven read-only tools.

This follows the same architectural pattern as the Obsidian integration (#756): a KnowledgeCapability that owns the turn exclusively when an MN4 library is selected, backed by a connected KB type that skips indexing.

What's included (Phase 1)

New package deeptutor/capabilities/marginnote4/:

  • models.pyMarginNoteObject, SyncBatch, PairedDevice, LearningEvent
  • store.py — SQLite CRUD + search + device pairing with SHA-256 hashed tokens
  • binding.py — per-turn resolution of the selected MN4 library
  • tools.py — 7 read-only tools: search, read, list, documents, links, tags, cards
  • capability.pyMarginNoteCapability (KnowledgeCapability, exclusive turn ownership)
  • prompts/{en,zh}/system.md — bilingual system prompts

HTTP bridge deeptutor/api/routers/marginnote4.py:

Endpoint Auth Purpose
POST /pair Session Register a device, get one-time token
POST /sync Device token Incremental upsert + tombstones
POST /heartbeat Device token Liveness + object count
GET /devices Session List paired devices
DELETE /devices/{id} Session Revoke a device
GET /status Session Health check + stats

Integration points:

  • kb_types.py: MARGINNOTE4_KB_TYPE added to CONNECTED_KB_TYPES
  • manager.py: db_path surfaced in get_metadata allowlist
  • registry.py: MarginNoteCapability registered in LOOP_CAPABILITIES
  • builtin/__init__.py: MARGINNOTE_TOOL_TYPES in BUILTIN_TOOL_TYPES
  • api/main.py: router mounted at /api/v1/marginnote4

Data sovereignty

  • MarginNote 4 owns: original text, excerpts, mindmap, native cards, FSRS scheduling
  • DeepTutor owns: RAG, reasoning, mastery tracking, long-term memory
  • DeepTutor never reads MN4's private database or modifies its FSRS state

Test coverage

40 new tests across:

  • Store: pairing, token verification, revocation, ingest (store/update/delete), search, list, documents, tags, links, cursor
  • Tools: all 7 tools with seeded data, graceful failure on missing store
  • Capability: binding resolution, caching, exclusivity, system blocks (en/zh), owned KBs
  • KB manager: orphan prune survival, metadata surfacing, reconcile safety

All 73 tests pass (40 new + 33 existing capability/knowledge), 0 regressions. Ruff clean.

Roadmap

  • Phase 2: Diff-confirmed write-back to MN4 (propose → confirm → apply → verify → rollback)
  • Phase 3: Card learning event sync, mastery bridge, cross-device setup docs

Refs: #756 (Obsidian pattern), #180 (multimodal learning)

@evan188199-tech
evan188199-tech marked this pull request as ready for review August 14, 2026 23:03
XZH and others added 3 commits August 14, 2026 16:43
…ridge

Add MarginNote 4 as a connected knowledge base type that syncs study data
(notes, excerpts, cards, mindmap nodes) from MN4 via the official Add-on API
into a dedicated SQLite store, then exposes it to the chat loop through seven
read-only tools.

New package: deeptutor/capabilities/marginnote4/
- models.py: MarginNoteObject, SyncBatch, PairedDevice, LearningEvent
- store.py: SQLite CRUD + search + device pairing with hashed tokens
- binding.py: per-turn resolution of the selected MN4 library
- tools.py: search, read, list, documents, links, tags, cards
- capability.py: KnowledgeCapability (exclusive turn ownership)
- prompts/{en,zh}/system.md

HTTP bridge: deeptutor/api/routers/marginnote4.py
- POST /pair (session auth) -- register a device, get one-time token
- POST /sync (device-token auth) -- incremental upsert + tombstones
- POST /heartbeat (device-token auth) -- liveness + object count
- GET /devices, DELETE /devices/{id}, GET /status (session auth)

Integration points:
- kb_types.py: MARGINNOTE4_KB_TYPE added to CONNECTED_KB_TYPES
- manager.py: db_path surfaced in get_metadata allowlist
- registry.py: MarginNoteCapability registered in LOOP_CAPABILITIES
- builtin/__init__.py: MARGINNOTE_TOOL_TYPES in BUILTIN_TOOL_TYPES
- api/main.py: router mounted at /api/v1/marginnote4

Phase 1 scope: device pairing, read-only sync, tool navigation.
Write-back (propose/apply/verify), learning events, and mastery bridge
are planned for Phase 2-3.

Refs: HKUDS#756 (Obsidian external knowledge source pattern), HKUDS#180
Testing and experience improvements:

- Search now includes document_title in scope: searching for a textbook
  name returns results instead of zero hits
- Store instances cached by db_path so schema init runs once, not per
  tool call within a turn
- object_type parameters carry enum constraints so the LLM knows valid
  types without guessing
- System prompts (en/zh) rewritten with domain context: explains the
  five object types, link traversal strategy, and citation guidance
- KB registration: register_marginnote4_kb manager method +
  /connect-marginnote4 API endpoint, mirroring the Obsidian connect flow
- Fix indentation inconsistencies in manager.py and main.py
- New tests: document_title search scope, KB registration (create,
  default path, duplicate rejection)

65 tests pass, 0 regressions, ruff clean.
@evan188199-tech
evan188199-tech force-pushed the codex/feat-marginnote4-protocol branch from 6be7544 to d23bbf2 Compare August 15, 2026 00:00
@evan188199-tech

Copy link
Copy Markdown
Contributor Author

Rebase and validation update

  • Replayed both Phase 1 commits onto the latest dev (23ad80d1).
  • Added a formatter-only commit for the new MarginNote4 files.
  • Re-ran validation with Python 3.11.15:
    • pytest -q tests/capabilities/marginnote4 tests/knowledge/test_marginnote4_kb.py: 44 passed
    • ruff check on changed integration files: PASS
    • ruff format --check on changed integration files: PASS
    • git diff --check: PASS

Phase 2 write-back and Phase 3 mastery bridging remain intentionally out of scope.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant