feat(agent): keep early chat context with a rolling summary of older messages

[viktoravelino]

The Agent resends the newest n_messages stored messages (100 by default) on every model call and silently drops anything older. So you either pay for 100 raw messages per call, or lower N and the Agent forgets what was said at the start of the session (the project, the customer, a decision).

This is a POC for #15799. It adds an opt-in Summarize Older Messages switch to the Agent, off by default:

• The last N messages are still sent as they are.
• Messages older than that are folded into a short per-session summary, one model call per 20 messages.
• The summary is sent first in the history as a user-role message.

The system prompt is unchanged, and the summary is never stored as a chat message.

How it works

Storage. A new session_memory_state table (EXPAND migration) holds one row per flow, message owner, session and context. Each row has the summary, a cursor at the last folded message, and a version used for optimistic concurrency.

Each turn, the Agent loads the newest N messages as today, then fetches the unfolded messages between the cursor and that window:

• Fewer than 20: they're sent as-is, so the model never misses messages between the summary and the window.
• 20 or more: they're folded into the summary with the Agent's own model. This call doesn't stream and has no run callbacks, so summary tokens never appear in the chat.
• Folding happens at most once per turn, even when the Agent retries.

Fallbacks. If the summary call fails, the turn runs with the previous summary and folding pauses for 10 minutes for that session. A long backlog folds at most 3 batches per turn; unfolded messages are sent as-is only when fewer than 20 remain, so the prompt never gets an out-of-order block. If another run saves first, the winner's summary is used. Standalone lfx (no database) and runs whose messages aren't stored fall back to today's last-N.

Cleanup. State rows are deleted with their session (both session-delete endpoints), with their flow (cascade_delete_flow), and by the end-user erase step.

Toggle off is unchanged behaviour.

Results

I tested against a real model (gpt-4o-mini): one session, a first message stating "project PRJ-123 on the EU cluster", then 60 filler turns including a ~5,000-character pasted log, then "Which cluster and project were we on?":

Setting	Input tokens	Answer
Toggle off, N=10	1,037	made-up project and cluster
Toggle off, N=100 (today's default)	6,129	"I'm unable to access specific details…"
Toggle on, N=10	1,722	"You are working on project PRJ-123 on the EU cluster."

• Tokens: 72% fewer input tokens than the default, and the right answer. Even N=100 loses the first message once a session passes 100 rows.
• Latency: a fold adds about 2–3 s to one turn in ten at N=10.
• Summary quality: the stored summary kept the project, the cluster, the short-answers preference and the out-of-memory diagnosis, and reduced the pasted log to one line.

How to test

Run the backend with components loaded from source, so the new input shows up, and with INFO logs, so you can see the folds:

LFX_DEV=1 LANGFLOW_LOG_LEVEL=info make backend   # terminal 1
make frontend                                     # terminal 2

1. Recall after the window

1. Create a flow: Chat Input → Agent → Chat Output, with an OpenAI model such as gpt-4o-mini.
2. In the Agent's advanced settings, set Number of Chat History Messages to 2 and leave Summarize Older Messages off.
3. In the Playground, send "My customer is Acme, region APAC. Keep answers short.", then 11 or more short messages ("ok 1", "ok 2", …), then "Which customer and region?"
   Expected: the Agent doesn't know.
4. Turn Summarize Older Messages on. In the same session, ask again. More than 20 messages are already older than the window, so this turn folds them.
   Expected:
   • the answer names Acme and APAC;
   • the backend log shows session_summary_fold latency_ms=… folded=20;
   • the turn that folds is about 2–3 s slower;
   • no summary text appears in the chat.

Doing the toggle-off check first keeps the right answer out of the recent window.

2. Inspect the stored summary. Query the session_memory_state table in your Langflow database, for example select session_id, version, folded_count, summary from session_memory_state;.
Expected: a "Key facts" plus "Conversation so far" summary of at most 300 words. version counts the folds.

3. Cleanup. Delete that session from the Playground's session list.
Expected: its session_memory_state row is gone.

4. Edge cases

• Toggle on with Number of Chat History Messages = 0: memory is off, and no summary is read or written.
• Toggle off: flows behave exactly as before.

Known gaps (POC)

• Content from edited or deleted messages stays in the summary until the session is deleted.
• Renaming a session orphans its summary.
• Only verified on OpenAI. Some providers may reject two user messages in a row (the summary, then a window that starts with a user message).
• Fold calls aren't counted in the Agent's token usage.
• Starter projects and the component index aren't updated. New Agents show the switch only with LFX_DEV, or after make build_component_index.
• The fold logs are INFO, so they're hidden at the default log level.

Refs #15799

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

✅ Migration Validation Passed

All migrations follow the Expand-Contract pattern correctly.