Skip to content

fix(voice): align session.say transcripts with TTS - #2225

Open
swayamg20 wants to merge 2 commits into
livekit:mainfrom
swayamg20:fix/2067-align-say-transcripts
Open

fix(voice): align session.say transcripts with TTS#2225
swayamg20 wants to merge 2 commits into
livekit:mainfrom
swayamg20:fix/2067-align-say-transcripts

Conversation

@swayamg20

Copy link
Copy Markdown

Summary

  • use TTS-provided timed transcripts for session.say() when aligned transcripts are enabled and supported
  • preserve raw scripted text when alignment is disabled, unsupported, bypassed by caller-provided audio, or audio output is disabled
  • add regression coverage for the aligned path and each direct fallback path

Fixes #2067

Testing

  • pnpm test agents/src/voice/agent_activity_say_aligned_transcript.test.ts agents/src/voice/agent_activity_close_commit.test.ts agents/src/voice/agent_activity_interrupted_commit.test.ts agents/src/voice/generation_tts_timeout.test.ts --silent (20 passed)
  • pnpm test agents/src --exclude agents/src/voice/remote_session.test.ts --silent (1,561 passed, 5 skipped; the excluded file has an unrelated current-main event assertion mismatch)
  • pnpm build:agents
  • pnpm format:check
  • targeted ESLint for the changed voice files
  • pnpm exec changeset status --since=origin/main

@swayamg20
swayamg20 requested a review from a team as a code owner August 5, 2026 11:56
@changeset-bot

changeset-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 5e0fee0

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 39 packages
Name Type
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-anthropic Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-azure Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-did Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-fishaudio Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-hume Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-krisp Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-perplexity Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-protoface Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-soniox Patch
@livekit/agents-plugin-tavus Patch
@livekit/agents-plugins-test Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

devin-ai-integration[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

session.say() transcription is never TTS-aligned — scripted turns drift then flush in one chunk at playback end

1 participant