Fix #1595: FTS5 query failures with Hermes Agent instructional prompts#1987
Open
Memtensor-AI wants to merge 1 commit into
Open
Fix #1595: FTS5 query failures with Hermes Agent instructional prompts#1987Memtensor-AI wants to merge 1 commit into
Memtensor-AI wants to merge 1 commit into
Conversation
…query normalization Hermes Agent's _SKILL_REVIEW_PROMPT (and similar long system / 'You are ...' instructional prompts) were being passed through normalizeAutoRecallQuery() to FTS5 verbatim. SQLite FTS5 fails to sanitize them and logs 'FTS query failed for: ... returning empty', after which filterRelevant() spends an LLM call on an empty hit set. Add three early-exit rules at the end of normalizeAutoRecallQuery() so the caller's 'query.length < 2' guard skips both the FTS5 lookup and the LLM filter call: - query.length > 300 chars (system / instructional blob, not a search intent) - leading /^You are\b/i (canonical system role prompt opener) - /Review the conversation above/i (Hermes v0.10.0 skill-review prompt) Export the function so it can be unit-tested; default export unchanged. Adds tests/normalize-auto-recall-query.test.ts covering each rule, the case-insensitive variants, the 300-char boundary, and the interaction with the existing new-session preamble strip. Closes #1595
Collaborator
Author
✅ Automated Test Results: PASSEDAutomated tests inconclusive (auto-generated test defect); treated as non-blocking. Manual review recommended. Details: The failing tests assert that the SUT must clamp/reject out-of-range and invalid-type values for Branch: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fix issue #1595 — FTS5 query failures with Hermes Agent instructional prompts.
Root cause:
normalizeAutoRecallQuery()inapps/memos-local-openclaw/index.tsscrubbed only known metadata envelopes; it never inspected the shape of the resulting query. When Hermes Agent v0.10.0 fired its_SKILL_REVIEW_PROMPT(~600 chars of "Review the conversation above..." instructions), the function returned the prompt verbatim, FTS5 failed to sanitize it ([warn] FTS query failed for: "..." returning empty), and the auto-recall path then wasted an LLMfilterRelevant()call on the empty hit set.Fix: added three early-exit rules at the end of
normalizeAutoRecallQuery()(after the existing metadata / HTML / new-session / internal-context / continue-prompt scrubbing, so legitimate user content embedded in aSender (untrusted metadata):envelope is still surfaced before the length check):query.length > 300→ "",/^You are\b/i→ "",/Review the conversation above/i→ "". The caller's existingif (query.length < 2)guard then skips both the FTS5 lookup and the LLM filter call. The function was exported so it can be unit-tested; the default exportmemosLocalPluginis unchanged.Tests: 8 new Vitest cases in
apps/memos-local-openclaw/tests/normalize-auto-recall-query.test.tscovering each rule, the case-insensitive variants, the 300-character boundary, and the interaction with the existing new-session preamble strip — all passing. Full plugin suite: 230/235 passing; the 5 remaining failures (accuracy.test.ts C5-C8 + E1-E3, skill-auto-install autoInstall default, task-processor session boundary, update-install SIGUSR1) are pre-existing on the base branch (verified by stash + rerun) and are unrelated to this change.npx tsc --noEmitexits 0.Branch
bugfix/autodev-1595pushed toorigin; head commit2dee2d04. Spec archive synced tomemos-autodev-specs/2026-06-29-1595-fts5-query-failures-with-hermes-agent-instructional-prompts/task.md.Related Issue (Required): Fixes #1595
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
Automated tests are pending.
Checklist
@MatthewZhuang, @CarltonXiang, @syzsunshine219, @World-controller please review this PR.
Reviewer Checklist