Skip to content

feat(live-voice): honest human→agent GPT-Live bridge (v0.0.14) - #32

Open
robotlearning123 wants to merge 48 commits into
mainfrom
release/v0.0.14-live-voice
Open

feat(live-voice): honest human→agent GPT-Live bridge (v0.0.14)#32
robotlearning123 wants to merge 48 commits into
mainfrom
release/v0.0.14-live-voice

Conversation

@robotlearning123

@robotlearning123 robotlearning123 commented Jul 11, 2026

Copy link
Copy Markdown
Owner

v0.0.14 — GPT-Live → coding-agent bridge (human → agent, observe-only)

A human talks to ChatGPT voice in a real signed-in browser; the observed human transcript routes to a coding agent; the reply reaches the human out-of-band (text overlay). GPT-Live silently drops client-injected speech, so there is no agent→Live "speak" path — this ships as an honest observe-only bridge, not a controllable TTS.

Reliable path

Real signed-in Chrome + sidecar/extension (TAP) + sidecar/agent-gateway.mjs (agent adapter + control plane). browser/sidecar.mjs (puppeteer + fake WAV mic) is a test harness — Live does not transcribe synthetic audio. Cloudflare Turnstile bypass is out of scope. Spec: docs/superpowers/plans/2026-07-11-gpt-live-bridge-layer-spec.md.

Honesty remediation (this update)

Cross-model review (cx GPT-5.6 + Opus) flagged the earlier Mode-B design as BLOCK. Fixed:

  • Removed the agent→Live write channel (voice_live_send_text, POST /send_text, speak-queue) — it reported false delivered:true for a server-dropped op.
  • Unified every path through the real chat_message_delta parser + isActionable; the reliable extension path now flows through the shared bridge and the control plane, so voice_live_* observe the real path.
  • /end deadlock; group-killing agent timeout (src/agent-runner.mjs); gateway loopback + no-CORS + body cap + optional token; recursive secret redaction (py+js); version 0.0.14 + tool count 30 / 12 modules aligned across all surfaces; source-vs-wheel packaging disclosure.

MCP surface: voice_live_status / voice_live_get_transcript / voice_live_end / voice_live_export_help (observe + lifecycle only).

Gates (real output)

  • sidecar npm test: 50 passed / 0 failed / 3 skipped
  • pytest: 366 passed / 15 skipped · ruff clean
  • Verify receipt: artifacts/verify/gpt-live-v0.0.14-remediation-2026-07-11.md

⚠️ Merge hold

Base is main, so this diff also includes the 0.0.13 voice-catalog lane. Per the release ordering (0.0.12 → 0.0.13 → 0.0.14), hold merge pending 0.0.12/0.0.13 landing and owner authorization. Do not merge without a "Go".

🤖 Generated with Claude Code

https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG

Summary by CodeRabbit

  • New Features

    • Added a read-only voice catalog tool, increasing available MCP tools to 30.
    • Added an experimental, observe-only GPT-Live bridge for human voice transcripts and agent replies displayed as text.
    • Added local controls for bridge status, transcripts, help, and session termination.
    • Added automated voice testing support using transcript, speech-to-text, and text-to-speech test doubles.
  • Documentation

    • Updated product documentation, skill references, roadmap, FAQs, and release notes for voice capabilities and limitations.
  • Bug Fixes

    • Improved session ending, timeout handling, reliability, and sensitive-data redaction.
  • Chores

    • Updated the release version to 0.0.14 and removed an outdated QA report.

robotlearning123 and others added 30 commits July 10, 2026 16:32
Reject a voices list above _MAX_VOICES (128) as contract drift instead of
normalizing an unbounded private response, and add tests at and above the bound.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
Document list_voices as the 26th tool across README, docs, skill, and reference;
state that catalog discovery is supported while GPT-Live realtime audio,
transport, synthesis, and transcript guarantees are not. Add docs/roadmap.md
(version lanes, GPT-Live boundary, language policy, release gates) and an install
regression asserting the bundled skill allowlists all 26 tools.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
Bump pyproject, package, plugin, and server metadata to 0.0.13 and add the dated
CHANGELOG section for the read-only Voice catalog. Release is held (no tag or
publish) until 0.0.12 lands on its own lane.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
…dings

Frame the GPT-Live bridge (Mode A vs Mode B), enumerate the four evidence gaps,
and log the read-only web-app inspection: the realtime/WebRTC engine is absent
from all eager chunks and lives in a lazy voice chunk, so closing the gaps
requires an owner-gated live session capture.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
Owner-approved live session capture: recorded the entry flow, the voice_mode
catalog variants (advanced/wingman), voice-selection PATCH, and conversation
init. The realtime WebRTC handshake could not be captured — the headless browser
has no mic (getUserMedia NotFoundError x3) and a synthetic silent stream did not
make the app advance to RTCPeerConnection/SDP/datachannel. Documents the
fake-media-device path to capture it next.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
Forward an optional voice_mode query param (standard/advanced/live/wingman and
future modes) to /backend-api/settings/voices; charset-validate it and reject
malformed values before any request. Cover mode passthrough, default omission,
and rejection offline, and parametrize the opt-in live contract test over the
real modes. list_voices(voice_mode="live") targets the GPT-Live catalog.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
…h use

Record that the bridge is a TypeScript WebRTC sidecar plus an MCP control plane
(not one tool); the load-bearing reliability parts (real/virtual audio device,
session bootstrap, sentinel, TURN fallback, reconnect/heartbeat); and the two
tool/search models (native Mode A, likely disabled; brain-swap Mode B, reliable)
— both gated on the same datachannel capability capture.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
… core

GPT-Live is WebRTC audio and cannot be an MCP tool, so the bridge is a Node/TS
sidecar (owns WebRTC + mic + datachannel) plus an MCP control plane. Adds the
verifiable core — reconnect backoff, half-open liveness detection, and a
datachannel event router with Mode B glue (input transcript -> agent -> speak) —
all unit-tested (16/16). The consumer handshake (bootstrap + SDP routes) is an
isolated adapter stub that throws NotYetCapturedError, plus a paste-in capture
harness to fill it. No end-to-end bridge is claimed yet.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
Captured the real consumer handshake from ChatGPT's public web bundle (no
session/mic/credentials): endpoint ${origin}/realtime/{vp,vps,wm}?dcid=0, a
single-shot SDP exchange (POST offer as application/sdp + Bearer, response body
is the answer SDP), and a negotiated datachannel id:0 with a Realtime-style
event model. Replaces the throwing adapter stub with a real, unit-tested
implementation (voicePath/realtimeUrl/exchangeSdp), wires session.mjs to it, and
records provenance in the handshake evidence doc. Suite now 21/21. Remaining is a
single live confirmation POST (token source, ICE, exact event enum).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
…t token

Two authenticated POSTs to /realtime/vp?dcid=0 with the account bearer
(~/.codex/auth.json, curl_cffi Chrome impersonation) — no browser/mic/sentinel:
a datachannel-only offer returned 400 invalid_offer (auth passed, route correct),
and an Opus-audio offer returned HTTP 201 + a full SDP answer with server ICE
candidates. Confirms token source = account bearer (sidecar needs no browser) and
ICE arrives in the answer. Updates adapter/README/evidence accordingly; remaining
work is the WebRTC media stack (werift) + datachannel event enum.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
…er/mic)

connect_live.mjs (werift) + sdp_exchange.py established a real session to live
GPT-Live using only the account bearer: ICE completed, DTLS connected, negotiated
datachannel opened, server accepted the session (state_update idle->listening).
Discovered the consumer datachannel protocol: a {type:data_message, data:<json>}
envelope wrapping inner events (state_update with previous_state/new_state) — not
raw Realtime API events. Session drops when no audio is sent (recvonly); remaining
media step (Opus send via ffmpeg+werift) tracked in task #3. Adds werift dep to
the isolated sidecar subproject only; node_modules gitignored.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
…Live)

connect_live_audio.mjs streams a TTS utterance as Opus RTP into the live session
via werift rtpSource. Connection + datachannel + idle->listening confirmed with
PT (96/96) and SSRC aligned, but GPT-Live does not yet decode the sent audio (no
transcription event; session drops ~1s). Egress path (werift SRTP send of the
injected RTP) is the open item, tracked in task #3. Documents exact status.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
connect_live_audio.mjs: rtpSource returns [track,port,dispose] not {track};
ffmpeg -ssrc overflows on werift's >2^31 SSRC (dropped it); addTransceiver needs
the track as first arg. After fixes ffmpeg delivers 328 RTP packets and the track
is wired to the sender. Session still closes ~1s after 'listening' (same as
no-audio) -> not a VAD timeout but a missing outbound datachannel init message,
which is obfuscated in the bundle. Records exact status; task #3 = reverse the
outbound envelope. Full spoken round-trip not yet demonstrated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
…tive result)

Session still closes ~1s after 'listening' with a data_message-wrapped
session.update, identical to no-audio/unwrapped runs. Confirms the blocker is not
guessable from Node; the real authenticated client's outbound datachannel traffic
must be observed via CDP. Documents the negative result in the experiment.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
…wall

Voice client sends via publishData -> {type:data_message, data:<string>} (envelope
confirmed both ways). Connection phases and quality channel identified, but the
specific init/keepalive inner message that holds the session past 'listening' is
in a 4.5MB minified voice-command layer static grep can't trace, and blind Node
guessing is ruled out. Autonomous reverse-engineering exhausted; next step is
runtime observation of the authenticated client's datachannel (no mic/audio).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
…ift media

Owner-approved CDP observation of the real authenticated client (forced-silent
mic, spoofed permission/enumerateDevices; WebRTC is main-thread) captured the
complete datachannel protocol: only two outbound types in the data_message
envelope — track_state (mic live, on open) and client_metrics (~5-7/sec
keepalive). Applied both in connect_live_audio.mjs. Session STILL app-closes ~1s
after 'listening' while the real browser client stayed open 20s+/152 msgs with
the same protocol — so the blocker is werift<->OpenAI media/SRTP interop, not any
ChatGPT-specific unknown. All account-side unknowns (auth/routes/SDP/protocol)
resolved. Viable path: browser-based sidecar (headless Chrome + fake-audio file)
reusing the app's working media stack. Full spoken round-trip not yet demonstrated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
SILENCE=1 (continuous silence, what the browser client survives 20s+ on) still
app-closes ~1s after 'listening'. Confirms werift SRTP audio egress never reaches
the OpenAI server -> browser-based sidecar is the path. Adds SILENCE flag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
…dia internals

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
Adds browser/sidecar.mjs (puppeteer-core): drives a ChatGPT-logged-in Chrome with
a WAV fed as the mic (--use-file-for-fake-audio-capture, no real mic), injects the
datachannel hook via evaluateOnNewDocument (before app scripts — fixes the
hook-too-late problem), opens voice, and prints the transcription + response
event stream. Reuses the app's working media stack, sidestepping the werift SRTP
egress issue. onAgentTurn() is the Mode B hook for routing the transcript to your
agent. Ready to run on the Mac; voicing the agent reply back needs the one
speak-injection command still to capture.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
Headless-Chrome demo (browser/demo-headless.mjs: browser WebRTC + curl_cffi SDP
POST) closes ~1s after 'listening' IDENTICALLY to werift — so it was never
werift media egress. The real voice handshake (bundle 4813494d) posts
FormData(sdp + session JSON) with a Sentinel ProofToken + auth/routing headers,
NOT raw application/sdp + Bearer. A bare-token POST returns a lenient 201 but an
ephemeral session the server drops. Tested FORMDATA=1 (FormData + guessed
session, no ProofToken): 201 but connecting->failed. Remaining work = the
complete authenticated handshake (correct session object + realtime Sentinel
proof), which the logged-in client supplies natively. Adds puppeteer-core dep +
demo; retracts the werift-media conclusion.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
sandia777 and others added 10 commits July 11, 2026 11:59
Hardcoded Linux dev paths (/home/robot venv, /usr/bin/google-chrome) blocked
running the live experiments on the Mac, which is the machine with a real mic
for voice testing. Route the Python SDP helper via SDP_PY and Chrome via
CHROME_BIN, defaulting to the macOS Chrome.app path.

Confirms the corrected diagnosis on macOS: headless real-Chrome demo also
closes ~1s after 'listening' on a bare-bearer application/sdp POST — not a
werift-specific media issue.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fu4gZgWmCxAJ5JFozWSQb6
…lare Turnstile

sdp_exchange_full.py reconstructs ML.startTransceiverSession: FormData(sdp+session)
+ Bearer + OAI-Device-Id + matching UA + Sentinel Chat-Requirements + Proof(POW)
tokens (via gpt2agent's BackendClient/SentinelGate). POW solves; Turnstile does
NOT solve headlessly (gpt2agent's own solver fails here). Proof-only -> HTTP 201
but the session is invalid and the browser peer goes connecting->failed (ICE torn
down). So the autonomous no-login path is blocked by Cloudflare Turnstile, an
anti-bot challenge designed to require a real interactive browser — which the
logged-in Mac browser (browser/sidecar.mjs) solves natively. Not a code gap; the
intended security boundary. Definitive corrected conclusion.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
Reconstruct the voice handshake with gpt2agent's own machinery: BackendClient's
authenticated curl_cffi session (oai-* headers) + SentinelGate proof tokens,
posting FormData(sdp+session) to /realtime. demo-headless gains SDP_HELPER env
to swap the SDP helper.

Empirical finding (macOS, real Chrome media): bare application/sdp+Bearer
CONNECTS and reaches 'listening' (then closes ~1s later); FormData+guessed
session goes connecting->FAILED (unconnectable answer). So the guessed session
object is wrong, and bare-sdp is the better-connecting path — the ~1s close is
the open question, not werift media.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fu4gZgWmCxAJ5JFozWSQb6
… release/v0.0.14-live-voice

# Conflicts:
#	sidecar/browser/demo-headless.mjs
#	sidecar/experiments/sdp_exchange_full.py
…s fine)

macOS headless real-Chrome probe with getStats(): outbound audio packetsSent
climbs 35->287 while the server issues a 'User-Initiated Abort' SCTP close ~1s
after 'listening'. Confirms the ~1s death is a server-side session-validation
rejection (Turnstile boundary), not media/werift — a real browser sends audio
fine and the token-only session still dies. Records the logged-in-browser Mode B
path for the agent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fu4gZgWmCxAJ5JFozWSQb6
The logged-in-browser Mode B path uses a local Chrome profile (.chrome-gptlive)
that holds session cookies — never commit it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fu4gZgWmCxAJ5JFozWSQb6
Real logged-in Chrome (claude-in-chrome, ChatGPT Pro) opened advanced voice via
composer-speech-button and held a full session (produced a conversation) — the
human path works natively. Records that the fully-headless agent path is gated by
Cloudflare Turnstile by design and should not be bypassed; the supportable agent
shape is a real human-authenticated browser, not a token-only headless bridge.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fu4gZgWmCxAJ5JFozWSQb6
Add ModeBExport plane, localhost control HTTP, speak-wire contract, browser
sidecar agent hook, and MCP voice_live_* control tools. Audio stays in the
headed browser; Turnstile bypass remains out of scope.
POST /send_text no longer drainSpeakQueue() after a successful delivery,
which wiped earlier undelivered wires. Build+record first; enqueue only
when sendSpeak does not deliver.
@coderabbitai

coderabbitai Bot commented Jul 11, 2026

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Changes

The release updates the project to version 0.0.14, adds a read-only list_voices MCP tool, introduces an observe-only GPT-Live human-to-agent bridge, adds sidecar runtime and browser tooling, and documents the supported boundaries and testing strategy.

Voice catalog and MCP surface

Layer / File(s) Summary
Voice catalog and registration
gpt2agent/tools/voice.py, gpt2agent/tools/__init__.py, gpt2agent/tools/voice_live.py
Adds normalized, redacted voice catalog access and four observe-only GPT-Live control tools.
Skill and product metadata
README.md, gpt2agent/skills/*, server.json, pyproject.toml, gpt2agent/__init__.py
Aligns tool counts, descriptions, package versions, and documented voice capabilities.
Contract and integration tests
tests/test_tools.py, tests/test_backend_tools.py, tests/test_voice_live.py, tests/test_install.py
Validates voice normalization, registration, annotations, live-contract gating, redaction, control behavior, and installed skill contents.

GPT-Live bridge and sidecar

Layer / File(s) Summary
Transcript and bridge runtime
sidecar/src/transcript.mjs, sidecar/src/events.mjs, sidecar/src/export.mjs
Parses consumer chat_message_delta events, filters actionable inbound utterances, buffers text-only transcripts, and records agent replies without a Live speak channel.
Control and session orchestration
sidecar/src/control.mjs, sidecar/src/session.mjs, sidecar/src/adapter.mjs, sidecar/src/reconnect.mjs, sidecar/src/liveness.mjs
Adds localhost control routes, SDP exchange helpers, session states, reconnect backoff, and datachannel liveness tracking.
Gateway and browser integration
sidecar/agent-gateway.mjs, sidecar/browser/*, sidecar/extension/*
Routes validated utterances to an agent command, applies body and token controls, renders replies as overlays, and provides browser-side transcript forwarding.
Protocol experiments and captures
sidecar/experiments/*, sidecar/capture/*
Adds WebRTC, SDP, audio, bundle, handshake, injection, and live-monitoring harnesses for observed protocol behavior.
Runtime validation
sidecar/test/*, sidecar/package.json
Tests control routes, event envelopes, transcript assembly, session safety, agent timeouts, reconnects, liveness, and opt-in voice loops.

Documentation and release evidence

Layer / File(s) Summary
GPT-Live specifications and findings
docs/superpowers/plans/*, docs/superpowers/specs/*, docs/superpowers/reviews/*
Documents the consumer protocol, handshake evidence, supported boundaries, architecture decisions, and deferred behavior.
Release and contributor documentation
CHANGELOG.md, CLAUDE.md, CONTRIBUTING.md, docs/README.md, docs/roadmap.md, docs/faq.md, docs/how-it-works.md
Updates release notes, tool inventories, roadmap navigation, FAQ boundaries, and request-path descriptions.
Verification receipt
artifacts/verify/gpt-live-v0.0.14-remediation-2026-07-11.md
Records remediation scope, validation gates, review findings, and the remaining real-browser validation boundary.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Chrome
  participant BrowserHook
  participant ModeBExport
  participant AgentGateway
  participant CodingAgent
  Chrome->>BrowserHook: Emit inbound chat_message_delta events
  BrowserHook->>ModeBExport: Forward consumer datachannel payload
  ModeBExport->>ModeBExport: Assemble and filter human utterance
  ModeBExport->>AgentGateway: Submit actionable text
  AgentGateway->>CodingAgent: Run configured agent command
  CodingAgent-->>AgentGateway: Return text reply
  AgentGateway-->>ModeBExport: Return agent reply
  ModeBExport-->>BrowserHook: Expose text-only reply
  BrowserHook-->>Chrome: Render reply overlay
Loading
🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 32.14% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The PR description is detailed, but it misses the template's Type section and checklist items. Add the required Type section and checklist checkboxes, and align the wording under What & why to the repository template.
✅ Passed checks (3 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is concise and accurately summarizes the main v0.0.14 GPT-Live bridge change.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch release/v0.0.14-live-voice

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1d0f9f5738

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sidecar/src/control.mjs Outdated
}
if (req.method === "POST" && path === "/end") {
exportPlane.close();
if (typeof onEnd === "function") await onEnd();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Send the /end response before running shutdown

When voice_live_end hits the real browser sidecar, onEnd is the callback in browser/sidecar.mjs that calls await shutdown(), and shutdown() awaits control.server.close(). Because this line awaits onEnd before the current /end response is written, server.close() can wait on the still-open request, causing the MCP call to time out and the sidecar to remain stuck; send the JSON response first or schedule shutdown after res.end.

Useful? React with 👍 / 👎.

Comment thread sidecar/src/session.mjs Outdated
offerSdp: offer.sdp,
});
await pc.setRemoteDescription({ type: "answer", sdp: answerSdp });
this._dc.send(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Wait for the data channel before sending session updates

For a real RTCDataChannel created in connect(), the channel is still connecting immediately after setRemoteDescription; it only becomes usable after its open event. Sending the session update here can throw InvalidStateError in browsers/werift before the session ever reaches LIVE, so this should be deferred until the channel is open or guarded by an awaitable open step.

Useful? React with 👍 / 👎.

try {
dc.addEventListener("message", (m) => {
try {
const t = JSON.parse(m.data).type;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Unwrap data_message before recording event types

When ChatGPT Live emits the documented {type:"data_message", data:"<inner json>"} envelope, this records only data_message in eventTypes and never captures the inner transcript/response event names the harness is supposed to discover. That makes the capture evidence misleading for the exact consumer wire format used elsewhere in this change; parse outer.data before deduping, as the sidecar hook does.

Useful? React with 👍 / 👎.

sandia777 and others added 8 commits July 11, 2026 17:39
Pure-logic TranscriptAssembler that reconstructs human utterances from the chat_message_delta JSON-patch stream (direction:"in"), the real consumer protocol (not the OpenAI Realtime API names). +8 unit tests; no voice/human/LLM.

Co-Authored-By: Claude <noreply@anthropic.com>
Add CONSUMER_EVENTS (real wire vocabulary) + re-export TranscriptAssembler. Deprecate the Realtime-API event names and the buildSpeakWire/response.create speak-injection — verified 2026-07-11 the server silently drops response.create/conversation.item.create/session.update. Behavior retained for source/test compat. Narrow npm test to test/*.test.mjs so experiments/ isn't picked up.

Co-Authored-By: Claude <noreply@anthropic.com>
Consumer GPT-Live rejects synthetic audio (only transcribes real-mic), so its STT can't be driven human-free. RealtimeVoiceProvider + realtime-provider use the GA Realtime API (gpt-realtime-2.1-mini), which accepts synthetic TTS audio → transcript, enabling human-free voice STT tests. VoiceProvider abstracts prod (ConsumerGptLive, real mic) vs test (Realtime, synthetic). +2 STT tests + opt-in full-loop test.

Co-Authored-By: Claude <noreply@anthropic.com>
The working voice→agent bridge (experiments/voice-to-agent.mjs: CDP-tap real consumer GPT-Live → transcript → coding agent), the minimum-footprint Chrome extension (extension/), the local agent gateway, the layered voice test-case suite, and the reverse-engineering experiment scripts that produced the protocol findings. Gitignore generated audio.

Co-Authored-By: Claude <noreply@anthropic.com>
End-to-end pipeline + the authoritative protocol reference (handshake, WebRTC, the real datachannel event vocabulary, turn lifecycle, capabilities, Turnstile boundary, 1:1 reproduction notes) — live capture + the 4.5MB shipped voice-client bundle, all cited. Supersedes the earlier investigation doc's unverified gaps.

Co-Authored-By: Claude <noreply@anthropic.com>
Summarize the GPT-Live voice→agent lane: reverse-engineering, bridge, VoiceProvider, human-free testing stack, events.mjs correction, and the evidence-backed findings (synthetic-audio rejection, no speak-injection, Turnstile boundary, voice=backend conversation).

Co-Authored-By: Claude <noreply@anthropic.com>
A generated pre-release QA report from v0.0.2 (2026-05-27) committed to the
repo root; its "25 tools" count is long stale and it is regenerable, not a
current advertising surface.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
Re-scope to human→agent only. The consumer datachannel silently drops
client-injected speech, so there is no agent→Live "speak" path; the reply
reaches the human out-of-band (text overlay).

- Cut the write channel: remove voice_live_send_text, POST /send_text, the
  export.mjs speak-queue/buildSpeakWire, and session.mjs.speak (they reported
  false delivery for a server-dropped op).
- Real parser on every path: ModeBExport.ingest/handleUtterance use
  TranscriptAssembler + isActionable; agent-gateway.mjs runs the reliable
  extension path through the shared bridge and serves the control plane so the
  voice_live_* MCP tools observe it.
- Fixes: /end deadlock; group-killing agent timeout (src/agent-runner.mjs);
  gateway loopback + no-CORS + body cap + optional token; recursive secret
  redaction (py+js); version 0.0.14 + tool count 30 / 12 modules across all
  surfaces; source-vs-wheel packaging disclosure.
- Reliable path = signed-in Chrome + sidecar/extension + agent-gateway.mjs;
  browser/sidecar.mjs is a test harness. Spec + verify receipt included.

Gates: sidecar npm test 50/0/3, pytest 366 passed/15 skipped, ruff clean.
Cross-model reviewed (cx GPT-5.6 + Opus).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAbFRELj9M3MYQt7SkUaLG
@robotlearning123 robotlearning123 changed the title feat(live-voice): Mode B GPT-Live export plane + MCP control tools feat(live-voice): honest human→agent GPT-Live bridge (v0.0.14) Jul 13, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 15

🧹 Nitpick comments (4)
sidecar/extension/hook.js (1)

57-57: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add optional debug logging to the empty catch block.

Silently swallowing all parse errors makes it impossible to diagnose why utterances may be silently dropped. Consider a debug flag or console.debug to surface unexpected message formats without breaking resilience.

♻️ Optional: add debug logging
-        } catch {}
+        } catch (e) {
+          if (window.__gptliveDebug) console.debug("[gptlive-hook] parse error:", e?.message || e);
+        }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/extension/hook.js` at line 57, Update the empty catch block in the
utterance parsing flow to optionally emit debug information about the caught
parse error, using the existing debug configuration if available or a
non-intrusive console.debug fallback. Preserve the current resilient behavior by
continuing to swallow the error and avoid interrupting hook processing.
gpt2agent/tools/voice_live.py (1)

172-177: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Neither new tool module's register() matches the mandated conv=None signature. Path instructions for gpt2agent/tools/*.py state new tools must implement register(mcp, client, conv=None); both new files here omit conv, though this mirrors several pre-existing sibling files (e.g. account.py, instructions.py) that also skip it when unused.

  • gpt2agent/tools/voice_live.py#L172-L177: add conv=None to register(mcp, client: Any = None) -> None.
  • gpt2agent/tools/voice.py#L88-L88: add conv=None to register(mcp, client: BackendClient) -> None.

As per path instructions: "New tools must be implemented in gpt2agent/tools/<name>.py with a register(mcp, client, conv=None) function."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@gpt2agent/tools/voice_live.py` around lines 172 - 177, Update register in
gpt2agent/tools/voice_live.py (lines 172-177) to accept the optional conv=None
parameter while preserving its existing client handling. Also update register in
gpt2agent/tools/voice.py (line 88) to include conv=None, ensuring both new tool
modules match the mandated register(mcp, client, conv=None) signature.

Source: Path instructions

sidecar/browser/demo-headless.mjs (1)

21-21: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Same hardcoded personal machine path repeated as the default Python interpreter in three experiment scripts.

All three default to /home/robot/workspace/47-chatgpt2agent/gpt2agent/.venv/bin/python when SDP_PY/PY isn't set, so anyone else following the header run instructions gets a silent ENOENT instead of the venv the repo actually ships.

  • sidecar/browser/demo-headless.mjs#L21: resolve SDP_PY against the repo-relative venv (e.g. fileURLToPath(new URL("../../.venv/bin/python", import.meta.url))) or throw a clear error naming the required env var when unset, instead of falling back to the hardcoded absolute path.
  • sidecar/experiments/connect_live.mjs#L15: apply the same repo-relative fallback / explicit-error fix.
  • sidecar/experiments/connect_live_audio.mjs#L15: apply the same repo-relative fallback / explicit-error fix.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/browser/demo-headless.mjs` at line 21, Replace the hardcoded
personal-machine Python fallback in the interpreter configuration of
sidecar/browser/demo-headless.mjs:21, sidecar/experiments/connect_live.mjs:15,
and sidecar/experiments/connect_live_audio.mjs:15 with the repository-relative
.venv/bin/python resolution based on each module’s location, or fail clearly
when the required SDP_PY/PY environment variable is unset; preserve the existing
environment-variable override behavior in all three scripts.
sidecar/experiments/voice-agent-inject.mjs (1)

38-88: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Avoid eval() for the page hook; pass args directly to evaluateOnNewDocument.

eval(code) at Line 87 is flagged by static analysis (noGlobalEval) and is also fragile: if the target page enforces a CSP without unsafe-eval, the injected hook will throw and silently fail to install. voice-to-agent.mjs in this same PR shows the cleaner pattern — pass a real function plus its argument to evaluateOnNewDocument instead of stringifying and eval-ing.

♻️ Proposed refactor
-const PAGE_HOOK = `
-(wavB64) => {
+function pageHook(wavB64) {
   window.__pcs = []; window.__src = null; window.__replaced = false; window.__wavB64 = wavB64;
   ...
-}`;
+}
 
 const wavB64 = readFileSync(WAV).toString("base64");
 const browser = await puppeteer.connect({ browserURL: CDP, defaultViewport: null });
 const page = await browser.newPage();
-await page.evaluateOnNewDocument((code) => { eval(code); }, `(${PAGE_HOOK})(${JSON.stringify(wavB64)})`);
+await page.evaluateOnNewDocument(pageHook, wavB64);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/experiments/voice-agent-inject.mjs` around lines 38 - 88, Replace the
eval-based page-hook installation in evaluateOnNewDocument with a directly
passed function and argument, following the pattern used by voice-to-agent.mjs.
Preserve PAGE_HOOK’s execution and wavB64 initialization while removing
stringification and global eval so the hook remains CSP-compatible.

Source: Linters/SAST tools

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@gpt2agent/tools/voice_live.py`:
- Around line 199-225: Wrap each blocking _request call in voice_live_status,
voice_live_get_transcript, and voice_live_end with asyncio.to_thread, awaiting
the result before passing it to _strip_secrets. Ensure the module imports
asyncio and preserves the existing HTTP methods, paths, and clear query
behavior.

In `@sidecar/agent-gateway.mjs`:
- Around line 40-49: Update the request-body accumulation in the req data
handler to store raw chunks as byte buffers and track total byte length against
MAX_BODY. Decode the concatenated buffer once after all chunks are received,
preserving the existing oversized-body rejection behavior and avoiding
incremental UTF-8 decoding.

In `@sidecar/experiments/fetch-bundle-and-bootstrap.mjs`:
- Around line 26-33: Update the datachannel logging around __onBoot and its
BOOT_LOG persistence to avoid writing unredacted chat content: sanitize inbound
and outbound payloads by retaining only type and structural metadata, or ensure
any full-payload log is created with restrictive 0o600 permissions. Preserve the
existing bootstrap diagnostics while preventing sensitive transcript fields from
being stored at the predictable temporary path.

In `@sidecar/experiments/realtime-spike.mjs`:
- Around line 14-16: Replace the global WebSocket construction with a WebSocket
client/API that explicitly supports custom handshake headers, preserving the
Realtime URL and MODEL while ensuring the Authorization bearer token from KEY is
sent during connection setup. Do not rely on changing the Node engine version or
passing headers through the built-in WebSocket options.

In `@sidecar/experiments/voice-agent-inject.mjs`:
- Around line 73-81: Update the injection branch in the interval callback so
window.__replaced is set synchronously before invoking makeTrack(), preventing
later poll ticks from starting duplicate asynchronous injections. Keep the
existing makeTrack(), replaceTrack(), success reporting, and error reporting
behavior unchanged.

In `@sidecar/extension/background.js`:
- Around line 15-23: Apply explicit deadlines to both network boundaries: in
sidecar/extension/background.js lines 15-23, add an abortable timeout to the
gateway fetch and return a clear timeout response; in sidecar/src/adapter.mjs
lines 50-58, accept a timeout or abort signal for the SDP exchange, abort it
when the deadline expires, and add coverage for the timeout path. Use the
existing request-handling symbols at each site and preserve normal success and
error responses.

In `@sidecar/extension/relay.js`:
- Around line 6-17: Strengthen the message handler in the inbound relay by
requiring an extension-only capability check before calling
chrome.runtime.sendMessage. Add a per-session secret or equivalent unforgeable
validation shared with the legitimate extension hook, validate it alongside
__gptlive_utterance in the window message listener, and ignore messages that
fail it while preserving the existing forwarding and reply behavior for valid
messages.

In `@sidecar/src/adapter.mjs`:
- Around line 65-74: In sidecar/src/adapter.mjs lines 65-74, set LIVE_CONFIRMED
to false until media establishment and a complete round trip are verified, while
preserving the existing transport evidence and NEEDS_LIVE_CONFIRMATION markers.
In sidecar/experiments/test-speak-inject.mjs lines 178-183, update the
validation to require correlated output-transcript and received-audio evidence
rather than accepting a marker found anywhere.

In `@sidecar/src/control.mjs`:
- Around line 73-111: Strengthen the control-plane request handling in the
http.createServer callback by validating the request Origin/Host before serving
/status or /transcript and before allowing POST /end. Reject missing or
non-local origins/hosts with an appropriate 4xx response, while preserving
access for trusted loopback callers and leaving /health and /help behavior
unchanged as intended.

In `@sidecar/src/export.mjs`:
- Around line 41-75: Update the isBlocked logic in redactForAgent to also reject
lowercased keys ending with "secret" and "cookie", alongside the existing
underscored checks. Preserve the current blocklist and all other suffix matching
behavior.

In `@sidecar/src/session.mjs`:
- Around line 127-135: Update close() so a failure from this._dc.close() cannot
prevent this._pc.close() from being attempted; isolate the two close operations
with separate error-handling scopes while preserving the existing
exportPlane.close() and State.CLOSED finalization.
- Around line 79-107: Store the newly created peer connection in this._pc
immediately after createPeer succeeds, before any asynchronous handshake steps
in connect(). Ensure partial failures remain reachable by close(), while
preserving the existing successful-handshake state transition and reconnect
reset behavior.
- Around line 44-64: Update the session loop around the liveness monitor to
periodically call `isDead()`. When it reports expiration, close the current
session and trigger the existing reconnect flow, reusing the configured
`LivenessMonitor`, `_pc`, and `ReconnectPolicy` behavior without changing normal
activity handling.

In `@sidecar/src/voice-provider.mjs`:
- Around line 60-69: Update start() to clear any existing interval in this._poll
before assigning the new setInterval handle, ensuring repeated starts do not
leave the previous polling timer running.

In `@sidecar/test/voice-test-cases.md`:
- Around line 24-25: Insert a blank line between each Layer A, B, C, D, and E
heading and its following Markdown table in voice-test-cases.md, preserving the
existing headings and table contents.

---

Nitpick comments:
In `@gpt2agent/tools/voice_live.py`:
- Around line 172-177: Update register in gpt2agent/tools/voice_live.py (lines
172-177) to accept the optional conv=None parameter while preserving its
existing client handling. Also update register in gpt2agent/tools/voice.py (line
88) to include conv=None, ensuring both new tool modules match the mandated
register(mcp, client, conv=None) signature.

In `@sidecar/browser/demo-headless.mjs`:
- Line 21: Replace the hardcoded personal-machine Python fallback in the
interpreter configuration of sidecar/browser/demo-headless.mjs:21,
sidecar/experiments/connect_live.mjs:15, and
sidecar/experiments/connect_live_audio.mjs:15 with the repository-relative
.venv/bin/python resolution based on each module’s location, or fail clearly
when the required SDP_PY/PY environment variable is unset; preserve the existing
environment-variable override behavior in all three scripts.

In `@sidecar/experiments/voice-agent-inject.mjs`:
- Around line 38-88: Replace the eval-based page-hook installation in
evaluateOnNewDocument with a directly passed function and argument, following
the pattern used by voice-to-agent.mjs. Preserve PAGE_HOOK’s execution and
wavB64 initialization while removing stringification and global eval so the hook
remains CSP-compatible.

In `@sidecar/extension/hook.js`:
- Line 57: Update the empty catch block in the utterance parsing flow to
optionally emit debug information about the caught parse error, using the
existing debug configuration if available or a non-intrusive console.debug
fallback. Preserve the current resilient behavior by continuing to swallow the
error and avoid interrupting hook processing.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8d0d73ae-696f-4cfd-a1a8-5b8c1994f3c1

📥 Commits

Reviewing files that changed from the base of the PR and between 04e3d93 and 68dab90.

⛔ Files ignored due to path filters (1)
  • sidecar/package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (86)
  • .claude-plugin/marketplace.json
  • .claude-plugin/plugin.json
  • CHANGELOG.md
  • CLAUDE.md
  • CONTRIBUTING.md
  • QA_REPORT.html
  • README.md
  • artifacts/verify/gpt-live-v0.0.14-remediation-2026-07-11.md
  • docs/README.md
  • docs/faq.md
  • docs/how-it-works.md
  • docs/roadmap.md
  • docs/superpowers/plans/2026-07-10-v0.0.13-voice-release.md
  • docs/superpowers/plans/2026-07-11-gpt-live-bridge-layer-spec.md
  • docs/superpowers/plans/2026-07-11-gpt-live-full-pipeline.md
  • docs/superpowers/plans/2026-07-11-gpt-live-handshake-evidence.md
  • docs/superpowers/plans/2026-07-11-gpt-live-protocol-spec.md
  • docs/superpowers/plans/2026-07-11-v0.0.14-live-voice-investigation.md
  • docs/superpowers/reviews/2026-07-10-account-native-feature-coverage-cross-model-review.md
  • docs/superpowers/specs/2026-07-10-account-native-feature-coverage-design.md
  • gpt2agent/__init__.py
  • gpt2agent/install.py
  • gpt2agent/skills/gpt2agent/SKILL.md
  • gpt2agent/skills/gpt2agent/tools-reference.md
  • gpt2agent/tools/__init__.py
  • gpt2agent/tools/voice.py
  • gpt2agent/tools/voice_live.py
  • pyproject.toml
  • server.json
  • sidecar/.gitignore
  • sidecar/README.md
  • sidecar/agent-gateway.mjs
  • sidecar/browser/demo-headless.mjs
  • sidecar/browser/sidecar.mjs
  • sidecar/capture/gpt-live-capture.js
  • sidecar/experiments/chunk-graph-fetch.mjs
  • sidecar/experiments/connect_live.mjs
  • sidecar/experiments/connect_live_audio.mjs
  • sidecar/experiments/coop-fetch.mjs
  • sidecar/experiments/diag-voice-entry.mjs
  • sidecar/experiments/fakemic-diag.mjs
  • sidecar/experiments/fetch-bundle-and-bootstrap.mjs
  • sidecar/experiments/live-monitor.mjs
  • sidecar/experiments/realtime-spike.mjs
  • sidecar/experiments/sdp_exchange.py
  • sidecar/experiments/sdp_exchange_full.py
  • sidecar/experiments/test-inject-candidates.mjs
  • sidecar/experiments/test-speak-inject-cdp.mjs
  • sidecar/experiments/test-speak-inject.mjs
  • sidecar/experiments/text-steer-watcher.mjs
  • sidecar/experiments/voice-agent-inject.mjs
  • sidecar/experiments/voice-to-agent.mjs
  • sidecar/extension/background.js
  • sidecar/extension/hook.js
  • sidecar/extension/manifest.json
  • sidecar/extension/relay.js
  • sidecar/package.json
  • sidecar/src/adapter.mjs
  • sidecar/src/agent-runner.mjs
  • sidecar/src/control.mjs
  • sidecar/src/events.mjs
  • sidecar/src/export.mjs
  • sidecar/src/liveness.mjs
  • sidecar/src/realtime-provider.mjs
  • sidecar/src/reconnect.mjs
  • sidecar/src/session.mjs
  • sidecar/src/transcript.mjs
  • sidecar/src/voice-provider.mjs
  • sidecar/test/adapter.test.mjs
  • sidecar/test/agent-runner.test.mjs
  • sidecar/test/control.test.mjs
  • sidecar/test/events.test.mjs
  • sidecar/test/export.test.mjs
  • sidecar/test/liveness.test.mjs
  • sidecar/test/realtime-stt.test.mjs
  • sidecar/test/reconnect.test.mjs
  • sidecar/test/session.test.mjs
  • sidecar/test/transcript.test.mjs
  • sidecar/test/voice-loop.test.mjs
  • sidecar/test/voice-test-cases.md
  • tests/test_audit_2026_07_09_tools.py
  • tests/test_backend_tools.py
  • tests/test_install.py
  • tests/test_none_guards.py
  • tests/test_tools.py
  • tests/test_voice_live.py
💤 Files with no reviewable changes (1)
  • QA_REPORT.html

Comment on lines +199 to +225
@mcp.tool(
name="voice_live_status",
annotations=annotations_ro,
)
async def voice_live_status() -> dict[str, Any]:
"""Status of the local GPT-Live bridge control plane (no audio/secrets)."""
result = _request("GET", "/status")
return _strip_secrets(result)

@mcp.tool(
name="voice_live_get_transcript",
annotations=annotations_ro,
)
async def voice_live_get_transcript(clear: bool = False) -> dict[str, Any]:
"""Drain the observed human/agent transcript text from the bridge (human → agent)."""
path = "/transcript?clear=1" if clear else "/transcript"
result = _request("GET", path)
return _strip_secrets(result)

@mcp.tool(
name="voice_live_end",
annotations=annotations_write,
)
async def voice_live_end() -> dict[str, Any]:
"""End the GPT-Live bridge session via the local control plane."""
result = _request("POST", "/end")
return _strip_secrets(result)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== voice_live.py outline ==\n'
ast-grep outline gpt2agent/tools/voice_live.py --view expanded || true

printf '\n== relevant line ranges ==\n'
nl -ba gpt2agent/tools/voice_live.py | sed -n '1,280p'

Repository: robotlearning123/gpt2agent

Length of output: 997


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== imports and helper bodies ==\n'
sed -n '1,260p' gpt2agent/tools/voice_live.py | cat -n

printf '\n== search for asyncio usage in file ==\n'
rg -n "asyncio|to_thread|_request\\(" gpt2agent/tools/voice_live.py

Repository: robotlearning123/gpt2agent

Length of output: 10113


Wrap the blocking control-plane calls in asyncio.to_thread().

voice_live_status, voice_live_get_transcript, and voice_live_end are async def but call _request(), which uses blocking urllib.request.urlopen with a 5s timeout. That will stall the MCP event loop during each request; run these calls in a thread instead.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@gpt2agent/tools/voice_live.py` around lines 199 - 225, Wrap each blocking
_request call in voice_live_status, voice_live_get_transcript, and
voice_live_end with asyncio.to_thread, awaiting the result before passing it to
_strip_secrets. Ensure the module imports asyncio and preserves the existing
HTTP methods, paths, and clear query behavior.

Source: Path instructions

Comment thread sidecar/agent-gateway.mjs
Comment on lines +40 to +49
let body = "";
let over = false;
req.on("data", (chunk) => {
if (over) return;
body += chunk;
if (body.length > MAX_BODY) {
over = true;
req.destroy();
reject(new Error("body too large"));
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

Does concatenating Node.js Buffer chunks into a string with += corrupt multi-byte UTF-8 characters split across chunks?

💡 Result:

Yes, concatenating Node.js Buffer chunks into a string using the += operator (or by calling.toString on individual chunks) will corrupt multi-byte UTF-8 characters if they are split across chunk boundaries [1][2]. When a multi-byte character (such as many CJK characters or emojis) is split between two chunks, the bytes at the end of the first chunk and the beginning of the next chunk will be incomplete on their own [3][2]. If you convert each chunk to a string independently, the decoder will be unable to recognize the multi-byte sequence and will typically replace the incomplete bytes with the Unicode replacement character (U+FFFD,), resulting in permanent data corruption [3][4][2]. To avoid this corruption, you should ensure that multi-byte character sequences are decoded only when they are complete. You can achieve this using one of the following methods: 1. Collect all chunks into an array, concatenate them using Buffer.concat, and then convert the resulting single Buffer to a string [3][1][2]: const bufs = []; for await (const chunk of readableStream) { bufs.push(Buffer.isBuffer(chunk)? chunk: Buffer.from(chunk)); } const result = Buffer.concat(bufs).toString('utf8'); 2. Use the node:string_decoder module, which is specifically designed to handle this scenario. The StringDecoder class maintains an internal buffer to hold incomplete multi-byte sequences until the next chunk arrives [5][6]: const { StringDecoder } = require('node:string_decoder'); const decoder = new StringDecoder('utf8'); let result = ''; for await (const chunk of readableStream) { result += decoder.write(chunk); } result += decoder.end; 3. Use readable.setEncoding('utf8') on a stream, which internally manages a StringDecoder to ensure that chunks are correctly reassembled into valid strings before they are emitted [7][8]. Note that this relies on the stream implementation correctly utilizing the internal encoding logic [3][2].

Citations:


Accumulate request chunks as bytes before decoding.
body += chunk decodes each Buffer independently, so multi-byte UTF-8 characters split across chunks can be corrupted. body.length also tracks string length, not raw bytes, so the size limit can drift from the actual request size.

Switch to byte counting and decode once at the end
 function readBodyLimited(req) {
   return new Promise((resolve, reject) => {
-    let body = "";
+    const chunks = [];
+    let size = 0;
     let over = false;
     req.on("data", (chunk) => {
       if (over) return;
-      body += chunk;
-      if (body.length > MAX_BODY) {
+      size += chunk.length;
+      if (size > MAX_BODY) {
         over = true;
         req.destroy();
         reject(new Error("body too large"));
+        return;
       }
+      chunks.push(chunk);
     });
     req.on("end", () => {
-      if (!over) resolve(body);
+      if (!over) resolve(Buffer.concat(chunks).toString("utf8"));
     });
     req.on("error", reject);
   });
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
let body = "";
let over = false;
req.on("data", (chunk) => {
if (over) return;
body += chunk;
if (body.length > MAX_BODY) {
over = true;
req.destroy();
reject(new Error("body too large"));
}
const chunks = [];
let size = 0;
let over = false;
req.on("data", (chunk) => {
if (over) return;
size += chunk.length;
if (size > MAX_BODY) {
over = true;
req.destroy();
reject(new Error("body too large"));
return;
}
chunks.push(chunk);
});
req.on("end", () => {
if (!over) resolve(Buffer.concat(chunks).toString("utf8"));
});
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/agent-gateway.mjs` around lines 40 - 49, Update the request-body
accumulation in the req data handler to store raw chunks as byte buffers and
track total byte length against MAX_BODY. Decode the concatenated buffer once
after all chunks are received, preserving the existing oversized-body rejection
behavior and avoiding incremental UTF-8 decoding.

Comment on lines +26 to +33
dc.addEventListener("open", () => { window.__dcOpen = true; try { window.__onBoot({ dir: "sys", t: "dc_open" }); } catch {} });
dc.addEventListener("message", (ev) => {
try {
let o = JSON.parse(String(ev.data));
let inner = o && o.type === "data_message" && typeof o.data === "string" ? JSON.parse(o.data) : o;
window.__onBoot({ dir: "in", t: inner && inner.type, raw: JSON.stringify(inner).slice(0, 4000) });
} catch {}
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Captured datachannel payloads may include real chat content, written unredacted to a predictable /tmp path.

window.__onBoot persists up to 4000 chars of inbound (and 2000 of outbound) raw datachannel JSON to BOOT_LOG (/tmp/gptlive-bootstrap.jsonl), which per this file's own protocol notes can include actual chat_message_delta conversation content, not just protocol metadata. On a shared machine this is a predictable, unprotected path for potentially sensitive transcript data to land.

Consider redacting/truncating message text fields before logging (keep only type/structural info), or write with restrictive permissions ({ mode: 0o600 }) if full payloads are needed for debugging.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/experiments/fetch-bundle-and-bootstrap.mjs` around lines 26 - 33,
Update the datachannel logging around __onBoot and its BOOT_LOG persistence to
avoid writing unredacted chat content: sanitize inbound and outbound payloads by
retaining only type and structural metadata, or ensure any full-payload log is
created with restrictive 0o600 permissions. Preserve the existing bootstrap
diagnostics while preventing sensitive transcript fields from being stored at
the predictable temporary path.

Comment on lines +14 to +16
const ws = new WebSocket(`wss://api.openai.com/v1/realtime?model=${MODEL}`, {
headers: { Authorization: `Bearer ${KEY}` },
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Check for engines field in sidecar/package.json
rg -A2 '"engines"' sidecar/package.json

Repository: robotlearning123/gpt2agent

Length of output: 203


🏁 Script executed:

#!/bin/bash
sed -n '1,80p' sidecar/package.json
printf '\n---\n'
sed -n '1,80p' sidecar/experiments/realtime-spike.mjs

Repository: robotlearning123/gpt2agent

Length of output: 3201


🌐 Web query:

Node.js global WebSocket headers option undici stable version support custom headers constructor

💡 Result:

As of July 2026, the global WebSocket client built into Node.js (which uses the Undici library) does not natively support passing custom headers via the standard WebSocket constructor [1][2]. While the standalone Undici package historically added support for a WebSocketInit options object that included a headers field [3][4][5], this extension is generally not available in the global, stable WebSocket implementation provided by Node.js [2][6]. Node.js aligns its global WebSocket API closely with the browser-based standard, which intentionally excludes options for custom handshake headers [4]. Key points regarding the current status: - Stable Implementation: The built-in, global WebSocket client in Node.js reached stable status in v22.4.0 [7]. It is designed to be compliant with standard browser-based WebSocket APIs [4]. - Custom Headers Limitation: Because the global WebSocket implementation follows standard web specs, there is no standardized way to provide custom headers (such as Authorization) during the handshake process [4][2][6]. - Alternatives: If your application requires custom headers for the handshake (e.g., for authentication or custom routing), you must continue to use the established third-party 'ws' library, which explicitly supports these features [6]. In summary, despite some custom extensions existing within the underlying Undici library code, they are not exposed in the stable, global Node.js WebSocket API [2][6].

Citations:


Global WebSocket won’t send these headers

new WebSocket(url, { headers: ... }) uses an option the built-in Node WebSocket doesn’t expose, so the Authorization header won’t reach the handshake. Use a client/API that supports handshake headers instead; bumping the Node engine floor won’t fix this.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/experiments/realtime-spike.mjs` around lines 14 - 16, Replace the
global WebSocket construction with a WebSocket client/API that explicitly
supports custom handshake headers, preserving the Realtime URL and MODEL while
ensuring the Authorization bearer token from KEY is sent during connection
setup. Do not rely on changing the Node engine version or passing headers
through the built-in WebSocket options.

Comment on lines +73 to +81
setInterval(()=>{ if(window.__replaced) return;
try { window.__on({ sys: "poll", pcs: window.__pcs.length, trs: window.__pcs.reduce((s,pc)=>{try{return s+pc.getTransceivers().length}catch{return s}},0) }); } catch {}
for(const pc of window.__pcs){ try{
for(const tr of pc.getTransceivers()){
const isAudio = (tr.receiver&&tr.receiver.track&&tr.receiver.track.kind==="audio") || (tr.sender&&tr.sender.track&&tr.sender.track.kind==="audio");
if(isAudio && tr.sender){ makeTrack().then(t=>tr.sender.replaceTrack(t).then(()=>{window.__replaced=true; window.__on({sys:"inject", same: tr.sender.track===t, kind: tr.sender.track&&tr.sender.track.kind, senderTrackId: tr.sender.track&&tr.sender.track.id, injectedId: t.id});})).catch(e=>window.__on({sys:"inject_err",e:String(e&&e.message||e)})); return; }
}
}catch{} } }
, 400);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Set window.__replaced synchronously before the async makeTrack() call to avoid a duplicate-injection race.

The guard is only flipped to true inside the resolved promise chain, so a slow AudioContext/decodeAudioData can let the next 400ms poll tick re-enter the same branch and start a second concurrent makeTrack()/replaceTrack().

🔒️ Proposed fix
-        if(isAudio && tr.sender){ makeTrack().then(t=>tr.sender.replaceTrack(t).then(()=>{window.__replaced=true; window.__on(...)})).catch(...); return; }
+        if(isAudio && tr.sender){ window.__replaced = true; makeTrack().then(t=>tr.sender.replaceTrack(t).then(()=>{ window.__on(...) })).catch(e=>{ window.__replaced=false; window.__on({sys:"inject_err",e:String(e&&e.message||e)}); }); return; }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
setInterval(()=>{ if(window.__replaced) return;
try { window.__on({ sys: "poll", pcs: window.__pcs.length, trs: window.__pcs.reduce((s,pc)=>{try{return s+pc.getTransceivers().length}catch{return s}},0) }); } catch {}
for(const pc of window.__pcs){ try{
for(const tr of pc.getTransceivers()){
const isAudio = (tr.receiver&&tr.receiver.track&&tr.receiver.track.kind==="audio") || (tr.sender&&tr.sender.track&&tr.sender.track.kind==="audio");
if(isAudio && tr.sender){ makeTrack().then(t=>tr.sender.replaceTrack(t).then(()=>{window.__replaced=true; window.__on({sys:"inject", same: tr.sender.track===t, kind: tr.sender.track&&tr.sender.track.kind, senderTrackId: tr.sender.track&&tr.sender.track.id, injectedId: t.id});})).catch(e=>window.__on({sys:"inject_err",e:String(e&&e.message||e)})); return; }
}
}catch{} } }
, 400);
setInterval(()=>{ if(window.__replaced) return;
try { window.__on({ sys: "poll", pcs: window.__pcs.length, trs: window.__pcs.reduce((s,pc)=>{try{return s+pc.getTransceivers().length}catch{return s}},0) }); } catch {}
for(const pc of window.__pcs){ try{
for(const tr of pc.getTransceivers()){
const isAudio = (tr.receiver&&tr.receiver.track&&tr.receiver.track.kind==="audio") || (tr.sender&&tr.sender.track&&tr.sender.track.kind==="audio");
if(isAudio && tr.sender){ window.__replaced = true; makeTrack().then(t=>tr.sender.replaceTrack(t).then(()=>{window.__on({sys:"inject", same: tr.sender.track===t, kind: tr.sender.track&&tr.sender.track.kind, senderTrackId: tr.sender.track&&tr.sender.track.id, injectedId: t.id});})).catch(e=>{ window.__replaced=false; window.__on({sys:"inject_err",e:String(e&&e.message||e)}); }); return; }
}
}catch{} } }
, 400);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/experiments/voice-agent-inject.mjs` around lines 73 - 81, Update the
injection branch in the interval callback so window.__replaced is set
synchronously before invoking makeTrack(), preventing later poll ticks from
starting duplicate asynchronous injections. Keep the existing makeTrack(),
replaceTrack(), success reporting, and error reporting behavior unchanged.

Comment thread sidecar/src/session.mjs
Comment on lines +44 to +64
constructor(opts) {
this.opts = opts;
this.state = State.IDLE;
this.exportPlane =
opts.exportPlane ??
new ModeBExport({
onAgentTurn: async (text) => {
const r = await opts.onUserSaid?.(text);
return typeof r === "string" ? r : null;
},
});
this.liveness = new LivenessMonitor({ timeoutMs: opts.livenessTimeoutMs ?? 15_000 });
this.reconnect = new ReconnectPolicy(opts.reconnect ?? { maxAttempts: 8 });
this._pc = null;
this._dc = null;

// Keep liveness fresh on every human utterance the bridge layer sees.
this.exportPlane.onHumanUtterance(() => {
this.liveness.seen(Date.now());
});
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
rg -n "isDead" -g '*.mjs' sidecar

Repository: robotlearning123/gpt2agent

Length of output: 729


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '\n== session.mjs outline ==\n'
ast-grep outline sidecar/src/session.mjs --view expanded || true

printf '\n== liveness usage ==\n'
rg -n "liveness|isDead|planReconnect|ReconnectPolicy|close\(" sidecar/src sidecar -g '*.mjs'

printf '\n== session.mjs relevant slice ==\n'
sed -n '1,220p' sidecar/src/session.mjs

printf '\n== liveness.mjs relevant slice ==\n'
sed -n '1,220p' sidecar/src/liveness.mjs

Repository: robotlearning123/gpt2agent

Length of output: 12785


Liveness monitor needs a dead check
seen() is updated on human utterances and datachannel messages, but isDead() is never polled here. Half-open sessions can stay alive indefinitely; add a periodic check in this session loop that closes and reconnects when the timeout expires.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/src/session.mjs` around lines 44 - 64, Update the session loop around
the liveness monitor to periodically call `isDead()`. When it reports
expiration, close the current session and trigger the existing reconnect flow,
reusing the configured `LivenessMonitor`, `_pc`, and `ReconnectPolicy` behavior
without changing normal activity handling.

Comment thread sidecar/src/session.mjs
Comment on lines +79 to +107
async connect() {
this._setState(this.state === State.IDLE ? State.CONNECTING : State.RECONNECTING);
const pc = this.opts.createPeer(this.opts.iceServers ?? []);
const track = await this.opts.getMicTrack();
pc.addTrack(track);
this._dc = pc.createDataChannel("", { negotiated: true, id: adapter.DATACHANNEL_ID });
this._dc.addEventListener("message", (m) => {
this.liveness.seen(Date.now());
// Fire-and-forget async agent turn. The reply is buffered as text for
// out-of-band egress; it is NOT sent back to Live (injection is dropped).
void this.exportPlane.ingest(m.data);
});
const offer = await pc.createOffer();
await pc.setLocalDescription(offer);
const url = adapter.realtimeUrl({
mode: this.opts.voiceMode,
sessionType: this.opts.sessionType,
});
const { answerSdp } = await adapter.exchangeSdp({
url,
token: this.opts.auth.token,
offerSdp: offer.sdp,
});
await pc.setRemoteDescription({ type: "answer", sdp: answerSdp });
// No client-side session.update: the consumer channel silently drops it.
this._pc = pc;
this.reconnect.reset();
this._setState(State.LIVE);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

connect() leaks the peer connection on partial failure.

this._pc is only assigned after the full handshake succeeds (line 104). If any step between mic acquisition and setRemoteDescription throws (mic permission error, network failure, auth failure), the locally-created pc is never stored and becomes unreachable — close() can never reach it, leaking the RTCPeerConnection/ICE agent/track.

🔧 Proposed fix
   async connect() {
     this._setState(this.state === State.IDLE ? State.CONNECTING : State.RECONNECTING);
     const pc = this.opts.createPeer(this.opts.iceServers ?? []);
-    const track = await this.opts.getMicTrack();
-    pc.addTrack(track);
-    this._dc = pc.createDataChannel("", { negotiated: true, id: adapter.DATACHANNEL_ID });
-    this._dc.addEventListener("message", (m) => {
-      this.liveness.seen(Date.now());
-      void this.exportPlane.ingest(m.data);
-    });
-    const offer = await pc.createOffer();
-    await pc.setLocalDescription(offer);
-    const url = adapter.realtimeUrl({
-      mode: this.opts.voiceMode,
-      sessionType: this.opts.sessionType,
-    });
-    const { answerSdp } = await adapter.exchangeSdp({
-      url,
-      token: this.opts.auth.token,
-      offerSdp: offer.sdp,
-    });
-    await pc.setRemoteDescription({ type: "answer", sdp: answerSdp });
-    // No client-side session.update: the consumer channel silently drops it.
-    this._pc = pc;
-    this.reconnect.reset();
-    this._setState(State.LIVE);
+    try {
+      const track = await this.opts.getMicTrack();
+      pc.addTrack(track);
+      this._dc = pc.createDataChannel("", { negotiated: true, id: adapter.DATACHANNEL_ID });
+      this._dc.addEventListener("message", (m) => {
+        this.liveness.seen(Date.now());
+        void this.exportPlane.ingest(m.data);
+      });
+      const offer = await pc.createOffer();
+      await pc.setLocalDescription(offer);
+      const url = adapter.realtimeUrl({
+        mode: this.opts.voiceMode,
+        sessionType: this.opts.sessionType,
+      });
+      const { answerSdp } = await adapter.exchangeSdp({
+        url,
+        token: this.opts.auth.token,
+        offerSdp: offer.sdp,
+      });
+      await pc.setRemoteDescription({ type: "answer", sdp: answerSdp });
+      this._pc = pc;
+      this.reconnect.reset();
+      this._setState(State.LIVE);
+    } catch (err) {
+      pc.close();
+      this._dc = null;
+      throw err;
+    }
   }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
async connect() {
this._setState(this.state === State.IDLE ? State.CONNECTING : State.RECONNECTING);
const pc = this.opts.createPeer(this.opts.iceServers ?? []);
const track = await this.opts.getMicTrack();
pc.addTrack(track);
this._dc = pc.createDataChannel("", { negotiated: true, id: adapter.DATACHANNEL_ID });
this._dc.addEventListener("message", (m) => {
this.liveness.seen(Date.now());
// Fire-and-forget async agent turn. The reply is buffered as text for
// out-of-band egress; it is NOT sent back to Live (injection is dropped).
void this.exportPlane.ingest(m.data);
});
const offer = await pc.createOffer();
await pc.setLocalDescription(offer);
const url = adapter.realtimeUrl({
mode: this.opts.voiceMode,
sessionType: this.opts.sessionType,
});
const { answerSdp } = await adapter.exchangeSdp({
url,
token: this.opts.auth.token,
offerSdp: offer.sdp,
});
await pc.setRemoteDescription({ type: "answer", sdp: answerSdp });
// No client-side session.update: the consumer channel silently drops it.
this._pc = pc;
this.reconnect.reset();
this._setState(State.LIVE);
}
async connect() {
this._setState(this.state === State.IDLE ? State.CONNECTING : State.RECONNECTING);
const pc = this.opts.createPeer(this.opts.iceServers ?? []);
try {
const track = await this.opts.getMicTrack();
pc.addTrack(track);
this._dc = pc.createDataChannel("", { negotiated: true, id: adapter.DATACHANNEL_ID });
this._dc.addEventListener("message", (m) => {
this.liveness.seen(Date.now());
void this.exportPlane.ingest(m.data);
});
const offer = await pc.createOffer();
await pc.setLocalDescription(offer);
const url = adapter.realtimeUrl({
mode: this.opts.voiceMode,
sessionType: this.opts.sessionType,
});
const { answerSdp } = await adapter.exchangeSdp({
url,
token: this.opts.auth.token,
offerSdp: offer.sdp,
});
await pc.setRemoteDescription({ type: "answer", sdp: answerSdp });
this._pc = pc;
this.reconnect.reset();
this._setState(State.LIVE);
} catch (err) {
pc.close();
this._dc = null;
throw err;
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/src/session.mjs` around lines 79 - 107, Store the newly created peer
connection in this._pc immediately after createPeer succeeds, before any
asynchronous handshake steps in connect(). Ensure partial failures remain
reachable by close(), while preserving the existing successful-handshake state
transition and reconnect reset behavior.

Comment thread sidecar/src/session.mjs
Comment on lines +127 to +135
close() {
try {
this._dc?.close();
this._pc?.close();
} finally {
this.exportPlane.close();
this._setState(State.CLOSED);
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

close(): a throwing dc.close() skips pc.close().

Since both calls share one try, an exception from this._dc.close() jumps straight to finally, leaving this._pc never closed.

🔧 Proposed fix
   close() {
-    try {
-      this._dc?.close();
-      this._pc?.close();
-    } finally {
-      this.exportPlane.close();
-      this._setState(State.CLOSED);
-    }
+    try { this._dc?.close(); } catch { /* ignore */ }
+    try { this._pc?.close(); } catch { /* ignore */ }
+    this.exportPlane.close();
+    this._setState(State.CLOSED);
   }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
close() {
try {
this._dc?.close();
this._pc?.close();
} finally {
this.exportPlane.close();
this._setState(State.CLOSED);
}
}
close() {
try { this._dc?.close(); } catch { /* ignore */ }
try { this._pc?.close(); } catch { /* ignore */ }
this.exportPlane.close();
this._setState(State.CLOSED);
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/src/session.mjs` around lines 127 - 135, Update close() so a failure
from this._dc.close() cannot prevent this._pc.close() from being attempted;
isolate the two close operations with separate error-handling scopes while
preserving the existing exportPlane.close() and State.CLOSED finalization.

Comment on lines +60 to +69
async start() {
const { page } = this.opts;
// Drain completed human utterances the hook parked on the page.
this._poll = setInterval(async () => {
try {
const us = await page.evaluate(() => { const a = window.__utterances || []; window.__utterances = []; return a; });
for (const u of us || []) for (const cb of this._cbs) cb(u);
} catch {}
}, 400);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Guard against leaking the previous poll interval on repeated start() calls.

If start() runs again before a matching stop() (e.g. a reconnect path), the prior setInterval handle in this._poll is overwritten and never cleared, leaking a polling timer that keeps calling page.evaluate indefinitely.

🔧 Proposed fix
   async start() {
     const { page } = this.opts;
+    if (this._poll) clearInterval(this._poll);
     // Drain completed human utterances the hook parked on the page.
     this._poll = setInterval(async () => {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
async start() {
const { page } = this.opts;
// Drain completed human utterances the hook parked on the page.
this._poll = setInterval(async () => {
try {
const us = await page.evaluate(() => { const a = window.__utterances || []; window.__utterances = []; return a; });
for (const u of us || []) for (const cb of this._cbs) cb(u);
} catch {}
}, 400);
}
async start() {
const { page } = this.opts;
if (this._poll) clearInterval(this._poll);
// Drain completed human utterances the hook parked on the page.
this._poll = setInterval(async () => {
try {
const us = await page.evaluate(() => { const a = window.__utterances || []; window.__utterances = []; return a; });
for (const u of us || []) for (const cb of this._cbs) cb(u);
} catch {}
}, 400);
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/src/voice-provider.mjs` around lines 60 - 69, Update start() to clear
any existing interval in this._poll before assigning the new setInterval handle,
ensuring repeated starts do not leave the previous polling timer running.

Comment on lines +24 to +25
## Layer A — STT (does GPT-Live transcribe the utterance?) [T2/T3]
| id | utterance | expected transcript (≈) |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Add blank lines before tables (MD058).

Static analysis flags 5 tables (Layer A/B/C/D/E) that immediately follow their heading with no blank line, which can break table rendering in stricter markdown parsers.

📝 Proposed fix (repeat for each flagged heading)
 ## Layer A — STT (does GPT-Live transcribe the utterance?) [T2/T3]
+
 | id | utterance | expected transcript (≈) |

Also applies to: 35-36, 44-45, 55-56, 62-63

🧰 Tools
🪛 markdownlint-cli2 (0.23.0)

[warning] 25-25: Tables should be surrounded by blank lines

(MD058, blanks-around-tables)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@sidecar/test/voice-test-cases.md` around lines 24 - 25, Insert a blank line
between each Layer A, B, C, D, and E heading and its following Markdown table in
voice-test-cases.md, preserving the existing headings and table contents.

Source: Linters/SAST tools

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants