MemoryCD (ICLR 2026 workshop): cross-domain personalization eval for AutoMem relation linking #30

MemoryCD: cross-domain lifelong personalization

• Paper: arXiv:2603.25973, Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalization (Lifelong Agent workshop, ICLR 2026)
• Code: https://github.com/AgentMemoryWorld/MemoryCD
• Data: https://huggingface.co/datasets/WZDavid/MemoryCD

What it tests

The benchmark is built from real Amazon review histories across 12 domains, not LLM-simulated transcripts. It has four tasks: rating prediction, review summarization, review generation and item ranking. Each runs in a single-domain setting and a cross-domain setting, where memory comes from other domains and the test is on a target domain. The paper's baselines (Mem0, A-Mem, MemoryBank, ReadAgent, long-context, BM25 RAG) fall short across 14 LLMs, and cross-domain is the hardest setting.

Why AutoMem

Our other adapters (BEAM, Memora/FAMA, Mem2ActBench, AMemGym) are dialogue-centric with one domain of discourse, so none of them exercise cross-domain relation linking. MemoryCD's cross-domain setting needs a memory formed in one domain (a Books review) to inform a prediction in another (Electronics) for the same user. That is what RELATES_TO/PART_OF linking plus expand_relations is for, and what lexical and recency baselines handle badly.

Status

• An AutoMemMethod adapter for the harness's BaseMethod.select_memory() exists. It stores interaction and per-domain profile nodes, links them with PART_OF and cross-domain RELATES_TO, and recalls with expand_relations=true scoped to a run/user tag. A draft PR is linked below.
• A 4-user smoke (Books target, Electronics source, rating prediction, K=4, claude-sonnet-4-6) showed no measurable cross-domain gain yet:

Four users is far too small to conclude anything.

Next

• Run a statistically meaningful subset (≥100 users per setting) on rating prediction, then the other three tasks.
• Compare plain recall with expand_relations to isolate the effect of relation linking.
• Decide go/no-go on a full cross-domain run.

Moved here from a private tracker, where it was filed in the wrong repo.