Examples/voice agent live retrieval UI - #409
Conversation
Enhances the customer-service voice-agent example with: - A Next.js web UI that shows live Moss retrieval side by side with the conversation, lighting up per turn with per-query latency. - Region metadata filtering: a US/EU picker in the UI publishes the region on a data channel; the agent scopes queries with a metadata filter (region in [selected, all]) so US and EU callers get the right policy (e.g. 30-day vs 14-day returns) from one index. - seed_index.py and data/faqs.json (20 FAQs tagged by category/region). - A demo script. The agent stays customer-facing and never mentions indexes or regions.
- README: list Cartesia as TTS (OpenAI=LLM, Deepgram=STT), add CARTESIA_API_KEY, fix DEMO_SCRIPT link to DEMO_SCRIPT_METADATA.md. - next.config: derive dir from import.meta.url (Node 18 compatible). - agent.py: validate region from data channel against an allow-list; fix stale 'upload.py' hint to 'seed_index.py'. - seed_index.py: fail fast if Moss credentials are missing. - RetrievalPanel: add type=button + aria-pressed on the region toggle.
… validates env + uses crypto.randomUUID (no hard-coded cred fallback)
- Require the index to load locally (fatal on failure): region metadata filtering is ignored on cloud fallback, so a silent warning would break the demo's point. - Skip empty/interim transcriptions before querying Moss. - Validate Moss creds in entrypoint; publish_data uses payload= keyword. - Add roomName to the token response type.
…p transcript history, Node 18.18+, note unauth token route is local-dev only
Address FAQ legal accuracy, region race conditions, accessibility, token hardening, and make npm run lint usable with ESLint config. Co-authored-by: Cursor <cursoragent@cursor.com>
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 22 files
Tip: instead of fixing issues one by one fix them all with cubic
Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.
Re-trigger cubic
Harden the local token route, align docs with uv run / Cloud setup, fix FAQ contradictions and UI region races, and migrate lint to ESLint flat config. Co-authored-by: Cursor <cursoragent@cursor.com>
|
@cubic.dev |
@samanyugoyal2010 I have started the AI code review. It will take a few minutes to complete. |
|
@claude how does this PR look to you |
There was a problem hiding this comment.
All reported issues were addressed across 22 files
Tip: instead of fixing issues one by one fix them all with cubic
Tip: cubic can generate docs of your entire codebase and keep them up to date. Try it here.
Re-trigger cubic
…ad limits Bind the Next server to loopback, stop trusting Host for auth, serialize region publishes, re-query on mid-flight region changes, truncate retrieval packets, restore the base voice pipeline, and fix remaining FAQ/CSS compatibility issues. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
All reported issues were addressed across 9 files (changes from recent commits).
Tip: instead of fixing issues one by one fix them all with cubic
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Require verified loopback binding for null peer IPs, take XFF from the trusted rightmost hop, make retrieval region mandatory, commit successful publishes before UI suppression, and revalidate region after every await. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
All reported issues were addressed across 8 files (changes from recent commits).
Tip: instead of fixing issues one by one fix them all with cubic
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
…ader, region deferral Use cross-env for portable scripts, allow loopback-bound production starts, make TRUST_PROXY_HEADER an exclusive strategy, and defer region changes until the current reply finishes so spoken policy matches retrieval. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
All reported issues were addressed across 5 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
…code Extract a pure tokenGuard module with focused node:test coverage, simplify turn retrieval to a single query/publish under deferral, and remove unused session_holder. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
All reported issues were addressed across 6 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
…st abuse Validate 127/8 octets, require TRUSTED_PROXIES before honoring forwarded IPs, restore TRUST_PROXY_HEADER misconfig warnings, and allowlist Host on the loopback listen path to block DNS rebinding. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
All reported issues were addressed across 4 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
Move empty TRUSTED_PROXIES and invalid TRUST_PROXY_HEADER warnings out of per-request peerIp so token traffic cannot flood the logs. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
All reported issues were addressed across 1 file (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Fix all with cubic | Re-trigger cubic
Codex reviewThe PR adds a useful voice demo surface, but the agent currently wires metadata filtering through the wrong Python SDK API. There are also correctness gaps in the documented proxy mode and the region-scoped sample data. |
There was a problem hiding this comment.
Pull request overview
This PR enhances the experimental moss-live-labs/examples/voice-agent demo by adding a Next.js web UI that displays per-turn Moss retrieval results in real time, and by introducing region-scoped metadata filtering (US/EU) so the same index can serve different policy answers depending on the caller’s region.
Changes:
- Added a new Next.js (
web/) UI showing call status, transcript, and a “Moss · knowledge base” retrieval panel fed via LiveKit data channels. - Implemented region selection (US/EU) in the UI and enforced region-scoped retrieval in
agent.pyvia Moss metadata filters (region ∈ {selected, all}). - Added seed script + sample FAQ dataset tagged by
region/category, plus updated docs/demo script for the new workflow.
Reviewed changes
Copilot reviewed 22 out of 24 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| moss-live-labs/examples/voice-agent/web/tsconfig.json | TypeScript config for the new Next.js UI app. |
| moss-live-labs/examples/voice-agent/web/public/moss-wordmark.svg | Wordmark asset used in the UI header branding. |
| moss-live-labs/examples/voice-agent/web/package.json | Web app dependencies/scripts (Next.js + LiveKit components + tests). |
| moss-live-labs/examples/voice-agent/web/package-lock.json | Lockfile for deterministic web dependency installs. |
| moss-live-labs/examples/voice-agent/web/next.config.mjs | Next.js config (monorepo tracing root pin). |
| moss-live-labs/examples/voice-agent/web/lib/types.ts | Shared payload/types for retrieval panel rendering. |
| moss-live-labs/examples/voice-agent/web/lib/tokenGuard.ts | Local-dev token endpoint guard (loopback/trusted proxy logic). |
| moss-live-labs/examples/voice-agent/web/lib/tokenGuard.test.ts | Unit tests for the token-guard logic. |
| moss-live-labs/examples/voice-agent/web/eslint.config.mjs | ESLint flat config for the new Next.js app. |
| moss-live-labs/examples/voice-agent/web/components/Transcript.tsx | Live transcript rendering (STT/TTS segments) with accessibility roles. |
| moss-live-labs/examples/voice-agent/web/components/RetrievalPanel.tsx | Retrieval UI + region picker; consumes moss.retrieval and publishes moss.region. |
| moss-live-labs/examples/voice-agent/web/components/AgentSide.tsx | Agent visualization + control bar + transcript composition. |
| moss-live-labs/examples/voice-agent/web/app/page.tsx | Main UI flow: connect, render LiveKit room, and layout the two panels. |
| moss-live-labs/examples/voice-agent/web/app/layout.tsx | App layout + metadata + font/style setup. |
| moss-live-labs/examples/voice-agent/web/app/globals.css | Styling for the UI, including LiveKit component overrides. |
| moss-live-labs/examples/voice-agent/web/app/api/token/route.ts | Token-minting endpoint for browser participants (room-per-visitor). |
| moss-live-labs/examples/voice-agent/web/.gitignore | Web app ignore rules (e.g., .next). |
| moss-live-labs/examples/voice-agent/web/.env.local.example | Web UI env example (LiveKit dev defaults + token-guard options). |
| moss-live-labs/examples/voice-agent/seed_index.py | Seeds a demo index from data/faqs.json with region/category metadata. |
| moss-live-labs/examples/voice-agent/README.md | Updated setup/run instructions and explanation of retrieval panel + region behavior. |
| moss-live-labs/examples/voice-agent/DEMO_SCRIPT_METADATA.md | Demo script focused on region-scoped retrieval behavior. |
| moss-live-labs/examples/voice-agent/data/faqs.json | Sample FAQ dataset with region-specific policy entries. |
| moss-live-labs/examples/voice-agent/agent.py | Adds per-turn retrieval publishing + region filter control via LiveKit data channel. |
| moss-live-labs/examples/voice-agent/.env.example | Adds MOSS_REGION default and clarifies provider key usage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "baseUrl": ".", | ||
| "paths": { "@/*": ["./*"] } | ||
| }, | ||
| "include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"], |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| results = await self.moss.query( | ||
| INDEX_NAME, | ||
| user_query, | ||
| QueryOptions(top_k=5, alpha=0.8, filter=region_filter), |
There was a problem hiding this comment.
BLOCKING QueryOptions(top_k=5, alpha=0.8, filter=region_filter)
The Python Moss SDK documents metadata filters as a filter= keyword on query(...), after QueryOptions, so this construction will fail before retrieval or leave the query unfiltered depending on the installed version. Move the filter to the query call:
results = await self.moss.query(
INDEX_NAME,
user_query,
QueryOptions(top_k=5, alpha=0.8),
filter=region_filter,
)(pypi.org)
|
|
||
| /** Socket / platform peer when available (NextRequest.ip). Never trust Host for this. */ | ||
| export function immediatePeerFromRequest(request: Request): string | null { | ||
| const ip = (request as Request & { ip?: string | null }).ip; |
There was a problem hiding this comment.
CONSIDER const ip = (request as Request & { ip?: string | null }).ip;
This app is pinned to Next 15, where NextRequest.ip was removed, so reverse-proxy mode never gets an immediate peer IP from the route request. With TRUST_PROXY=1, peerIp() returns null and /api/token 403s even for the documented trusted-proxy setup. Either remove this mode from the example or source the peer IP from a real deployment-specific/custom-server boundary and pass it into assertLocalDevOnly.
| "id": "returns-policy", | ||
| "category": "returns", | ||
| "region": "all", | ||
| "text": "For Northwind's standard (voluntary) returns program, eligible items can be returned for a refund when they are unworn, unwashed, and in their original packaging with tags attached — subject to your region's returns policy. In the EU, the separate 14-day statutory right of withdrawal is not lost merely because packaging or tags are missing; see the EU returns policy for statutory exceptions. Start a standard return from the Orders page in your account and print the prepaid label." |
There was a problem hiding this comment.
CONSIDER "region": "all", ... "In the EU, the separate 14-day statutory right of withdrawal..."
Because the agent queries region in [selectedRegion, "all"], this EU-only policy is included in US retrieval context and can leak the wrong answer for US callers. Keep all documents strictly region-neutral and move the EU statutory exception into an "EU" document, or split this FAQ into separate US/EU variants.
Summary
Enhances the existing
moss-live-labs/examples/voice-agentwith a live retrieval web UI and region metadata filtering, including review fixes from the prior iteration.Closes #348
Live retrieval web UI (
web/)A Next.js app that renders the call next to a Moss · Knowledge Base panel, showing the documents retrieved for each turn with scores and per-query latency. Data is published on a
moss.retrievaldata channel.Region metadata filtering
A US / EU picker in the UI publishes the region on a
moss.regiondata channel. The agent scopes each query with a metadata filter so US and EU callers get the right policy from the same index, while region-agnostic FAQs (region: all) always match.Type of Change
Try it
Docs: https://docs.moss.dev/docs/integrate/metadata-filtering