Skip to content

Cortex 2.0 for Harper 5#24

Open
daniel-harperdb wants to merge 17 commits intomainfrom
harper5-upgrade
Open

Cortex 2.0 for Harper 5#24
daniel-harperdb wants to merge 17 commits intomainfrom
harper5-upgrade

Conversation

@daniel-harperdb
Copy link
Copy Markdown
Collaborator

@daniel-harperdb daniel-harperdb commented Apr 20, 2026

Cortex 2.0 for Harper 5

Cortex 2.0 ships Harper 5.0 compatibility as a major release. It adopts
Harper 5's Resource patterns (static-method endpoints, table @export
directives), aligns the error surface with RFC 9457, handles Harper 5's
Promise-based request and CRUD semantics, and brings the embeddings
pipeline into compatibility with Harper 5's module loader.

Breaking changes

Change Impact
Requires Harper 5.0+ Harper 4 users stay on Cortex 1.x
harperdbharper dependency Update package.json
All Resource endpoints are now static Any direct instantiation needs updating
Error shape { error } → RFC 9457 { type, title, status, detail } Update client error handling

See CHANGELOG.md for the full migration guide.

Deployment requirements

Harper server harper-config.yaml must set:

applications:
  moduleLoader: node
  dependencyLoader: native

These are required for @huggingface/transformers named exports and
sharp native binding loading under Harper 5's VM sandbox.

Testing

Unit tests: 332 passing, 9 skipped. All test files updated to the
static call pattern and the RFC 9457 assertion shape.

Live endpoint validation: All 9 Cortex endpoints validated against
a clean Harper 5.0.1 install — 18/18 happy-path and error-path requests
PASS. Full PUT/GET/DELETE cycle on the Memory table confirmed.
MemorySearch end-to-end confirmed with the embeddings pipeline
(sharp + @huggingface/transformers).

Harper 5 returns Promises from singular CRUD operations (Memory.get,
Memory.put, Memory.delete, Memory.patch, Memory.invalidate, and the
equivalents on other tables). Harper 4 returned values synchronously for
primary-key lookups, so a number of callsites in the codebase dropped the
`await`. Under Harper 5 those become unhandled promises that leak into
downstream logic — the consumer sees `Promise {}` where it expected a
record object.

Two callsites fixed:

- `packages/cortex/resources/memory.js`: `MemoryTable.get` extends the
  auto-generated Memory table to strip embeddings from GET responses. The
  override called `super.get(target)` without awaiting, so under Harper 5
  the `typeof record === 'object'` branch matched the Promise, and the
  function returned an object literal containing Promise-valued fields.

- `packages/cortex-mcp-server/harper/resources/mcp-endpoint.ts`: the
  `memoryRecall` function called `Memory.get(params.id)` without awaiting
  and then checked `if (!record)`. Under Harper 5 the Promise is always
  truthy, so "not found" never triggered and downstream property access
  on the Promise returned `undefined` for every field.

All other singular CRUD callsites in `packages/cortex/` and
`packages/cortex-mcp-server/harper/` were already awaited correctly. For-await
iterators over `Memory.search(...)` are a separate code path and are not
affected.

Tests: full workspace suite green (23 files, 331 passed, 9 skipped).
Harper 5.0 ships under the new 'harper' package name (formerly 'harperdb').
This commit updates all imports, scripts, and dependency declarations
in the Cortex package and the harper/ subfolder of cortex-mcp-server.

- Rename 'harperdb' -> 'harper' in all ESM/TS imports (4 sites)
- Update vitest mocks in 15 test files to target 'harper'
- Bump @harperfast/cortex to 2.0.0 (Harper 5 compat is breaking)
- Update devDep harperdb ^4.7.24 -> harper ^5.0.0 in cortex
- Update peerDep harperdb * -> harper * in cortex-mcp-server/harper
- Update 'harperdb run/dev/deploy' scripts -> 'harper run/dev/deploy'
- Update 'npm install -g harperdb' -> 'harper' in CONTRIBUTING.md and README.md
- Refresh package-lock.json for new dep tree

External URLs (harperdb.io/docs, discord.gg/harperdb) and references to
@harperdb/mcp-server remain unchanged — those are separate packages.

Tests: 331 passed, 9 skipped (pre-existing openclaw-memory integration
skips). No regressions.
Under Harper 5, batch upserts need explicit per-record transaction
boundaries to ensure one failing record does not poison the entire
batch or leave partial writes hanging in an implicit transaction.

- Wrap each tableRef.put(record) call in transaction(() => ...)
- Import transaction from the default harper export (not re-exported
  as a named export in harper@5.0.x)
- Update vitest mocks across 15 test files to stub default.transaction
  as a pass-through invoker
- Add 'wraps each record put in its own transaction' case to
  batch-upsert.test.js verifying one-bad-row isolation and per-record
  transaction boundary count

Tests: 332 passed, 9 skipped.
Harper 5's component VM loader provides harper exports as named exports
only — no default export is synthesised. The previous workaround of
importing harper default and destructuring transaction failed at runtime
because harper resolved to undefined inside the VM.

Switch to named import: import { Resource, tables, transaction } from 'harper'.
Updated all vitest mocks to expose transaction as a named export alongside
the existing default stub.

Discovered during live V2 validation against Harper 5.0.1.
Harper 5 requires explicit @export directive on table types for REST
endpoint exposure. In v4, jsResource config.yaml was sufficient; v5
requires schema-level declaration per the new database/schema docs.
MemorySearch, MemoryCount, MemoryStore, and VectorSearch now expose
static async post(target, data) per Harper 5's jsResource loader
requirements. Instance methods on these classes were silently returning
404 under v5. MemoryTable (table subclass) keeps its instance get()
override unchanged -- table subclasses are not loaded by jsResource.

Update 7 test files to call ClassName.post(null, data) directly.
BatchUpsert.post(target, data) is now static. Per-record transaction
wrapping from commit 3b34ee0 is preserved unchanged.

Update batch-upsert.test.js to use static call pattern.
SynapseSearch, SynapseIngest, and SynapseEmit now expose
static async post(_req, data) per Harper 5's jsResource loader
requirements. Private helpers _parseContent and _emitForTarget
are also made static; internal this.* calls updated to class-name
references. SynapseEntry (table subclass) keeps its instance
get() override unchanged.

First parameter renamed _req (from target) across all Resource
static methods to avoid collision with the 'target' field in
SynapseEmit's request payload.

Update synapse-search, synapse-ingest, synapse-emit, and
score-normalization test files to use static call pattern.
… returns

SlackWebhook.post is now static async post(_req, data). The private
_processMessage helper is also made static; the setTimeout callback
updated from this._processMessage to SlackWebhook._processMessage.

All HTTP-visible returns now use getResponse(status, body, headers)
from harper, making SlackWebhook a proper v5-native HTTP endpoint.
The url_verification challenge return stays as a plain object since
Slack requires that specific shape verbatim.

Update webhook.test.js: static call pattern + getResponse mock that
spreads body into the result so existing message/status assertions
continue to work unchanged.
All Cortex-level validation errors now return RFC 9457 problem details
instead of { error: 'message string' }. Shape: { type, title, status,
detail }. A cortexError(slug, title, status, detail?) helper in
shared.js ensures consistent construction across all endpoints.

18 error return sites updated across memory.js (10) and synapse.js (8).
Type URIs use https://github.com/HarperFast/cortex/errors/<slug>.

Update 7 test files: result.type replaces result.error for presence
checks; result.detail replaces result.error for message-content checks.
Documents the full 2.0 scope: Harper 5 dependency rename, static-method
migration, RFC 9457 error shape, SlackWebhook getResponse adoption, and
the five fixes from Dev 1/2. Includes a four-step migration guide from
1.x.
…ming body

Harper 5's REST layer passes the request body as a streaming deserializer
Promise (getDeserializer with streaming=true returns streamToBuffer(...).then(deserialize)).
This Promise is passed directly to resource.post() without being awaited in
the transactional wrapper, so static post handlers receive an unresolved
Promise as their data argument.

Add 'data = await data;' at the top of all nine static post() methods across
memory.js, synapse.js, and slack-webhook.js. Awaiting a plain object is a
no-op, so unit tests calling post(null, {...}) directly are unaffected.
…urns

Harper 5.0.1 getResponse() takes no args — returns context storage response
object, not the status/body/headers passed in. All 5 call sites returned
{"headers":{}} silently. Replace with plain { status, message } objects.
Remove getResponse from harper import and vitest mock.
…Harper 5 VM compat

Harper 5 runs app code in a Node VM sandbox by default. Native addon
packages (sharp, onnxruntime-node) fail to load their binary bindings in
the VM context. dependencyLoader: native bypasses the VM loader for all
npm dependencies, letting Node load them natively.

allowInstallScripts: true ensures sharp postinstall build step runs when
Harper deploys (without --ignore-scripts).
… pipeline import

Harper module loader wraps dynamic imports such that named exports are
unavailable — everything lands on .default. Use nullish coalescing to
handle both environments: hft.pipeline (plain Node ESM) ?? hft.default?.pipeline
(Harper module loader).
Reorganize the 2.0.0 section top-down around what users need to know
(what's new, what breaks, how to migrate, what to deploy). Fold the
"Dev 4 & Dev 6 live validation" subsection into the main Fixed list
since those entries are fixes, not implementation history. Correct
the SlackWebhook entry to describe the final behavior (plain
{ status, message } returns) rather than the pre-fix getResponse()
shape. Remove the Node-PATH paragraph under deployment requirements;
npm's engines enforcement handles that case without Cortex-specific
guidance.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant