Skip to content
BranchesTags

Repository files navigation

ADK Sonar

ADK Sonar

A live voice agent for work in motion.

An ADK reference implementation demonstrating how to build a low-latency Gemini Live voice orchestrator that stays conversationally responsive while dispatching long-running background coding agents in isolated cloud sandboxes, managing daily productivity across Google Workspace, GitHub, Spotify, Maps, Search, and Slack, and rendering live visual surfaces to a mobile-first control plane.

Note

This is an educational recipe, not a packaged library. It demonstrates how to combine bidirectional audio streaming (run_live), non-blocking background tool scheduling (WHEN_IDLE), interchangeable coding agent harnesses (horizon, antigravity, claude) inside a shared Vertex AI Agent Engine Sandbox, and scoped A2UI v0.9 visual cards. You are not meant to pip install it — you are meant to read how the pieces fit together and lift the patterns you need into your own agent.

  • Study the components → AGENTS.md maps every interface to the file and function that implements it
  • Run it yourself → Quickstart gets the full local voice + sandbox stack running in ~5 minutes
  • Build your own → Use a coding agent to adapt these patterns to your own domain
  • Deploy → Deploy ships the FastAPI WebSocket backend + React PWA to Cloud Run

🎬 Demo Video Coming Soon — Live voice orchestration across Google Workspace, GitHub, Spotify, and multi-agent coding sandboxes from mobile.

Features

Live Voice & Non-Blocking Orchestration

  • Sub-second bidirectional voice (run_live) — Continuous PCM 16kHz/24kHz audio streaming and barge-in over WebSockets (app/main.py), powered by gemini-live-2.5-flash-native-audio on Vertex AI (us-central1).
  • Non-blocking background tool dispatch (WHEN_IDLE) — Every tool (app/tools/async_wrapper.py) immediately yields {status: "RUNNING"} to the Gemini Live stream so the voice persona (Charon) acknowledges the request in natural conversation without stalling audio frames, then narrates completion once the background coroutine finishes.
  • Voice-first conciseness guardrails — System instruction (app/agent.py) enforces 1–2 spoken sentences per turn while offloading code diffs, PR lists, and schedules to the visual stage.

Shared Cloud Sandbox & Interchangeable Coding Harnesses

  • Persistent Vertex AI Agent Engine Sandbox — All background coding tasks run inside a live remote Linux container (app/workers/sandbox.py) provisioned on Vertex AI Agent Engine (us-central1) with automated local fallback and one-command re-provisioning (make sandbox).
  • Git worktree isolation per task — Each coding task executes in its own isolated git worktree (workspaces/<repo>/.worktrees/<task_id>) on branch task/<task_id>, preventing concurrent agents from stepping on each other's working trees.
  • Two-phase Plan → Execute with human gate — In plan mode, the coding harness analyzes the repo and writes PLAN.md without mutating code, rendering a live Plan Approval Gate in the UI (Approve & Execute / Request Changes / Switch Harness).
  • Mid-task harness switching — Seamlessly hand off an active task (POST /api/v1/tasks/{id}/switch-harness) between ADK Long-Horizon (horizon), Antigravity CLI (antigravity), and Claude Code (claude). Because PLAN.md, PROGRESS.md, and git commits live inside the shared sandbox worktree, the incoming harness picks up right where the previous one left off.

Scoped A2UI v0.9 Stage & Mobile Control Plane

  • Single-surface visual stage — Instead of scrolling chat transcripts, the UI (web/src/components/a2ui/A2UISurfaceDeck.tsx) renders a focused A2UI v0.9 surface card (TaskStatusCard, PlanReviewCard, GitHubPRCard, WorkspaceDigestCard, CalendarAgendaCard, SpotifyPlayerCard, PlaceCard) synchronized with live voice turns.
  • Interactive bottom sheets — Dedicated slide-up drawers for Tasks (live terminal diffs, PLAN.md inspection, harness switching), Workspaces (clone repos, inspect branches, auto-discover personal GitHub forks), Connected Apps (1-click OAuth & CLI auth detection), and Transcript & Text Input.

Zero-Friction OAuth & Live Integrations

  • Unified OAuth 2.0 + Refresh Token lifecycle (app/auth.py) — Built-in PKCE + refresh token auto-rotation for Spotify and Google Workspace (Calendar, Gmail, Drive), 1-click gcloud / gh CLI detection, dynamic personal fork discovery (GET /user → <owner>/<repo>), and automatic Slack workspace discovery (auth.test).

The stack

Built on Google ADK, Gemini Live API, and Vertex AI Agent Engine Sandboxes:

Capability What provides it
Bidirectional voice & barge-in ADK Runner.run_live() + LiveRequestQueue (gemini-live-2.5-flash-native-audio)
Non-blocking tool execution Custom @non_blocking_tool decorator (app/tools/async_wrapper.py) emitting WHEN_IDLE results
Remote code execution Vertex AI Agent Engine sandboxEnvironments (app/workers/sandbox.py)
Multi-harness worker execution CodingHarness protocol (app/workers/harnesses.py) with HorizonHarness, AntigravityHarness, and ClaudeCodeHarness
Generative UI cards Custom A2UI v0.9 event emitter (app/callbacks/a2ui_emitter.py) + React surface deck
Connected app authentication Declarative config/integrations.yaml + OAuth 2.0 / ADC manager (app/auth.py)

Everything else is custom glue (~2,800 lines across app/ and web/), built on six core interfaces:

  1. Non-blocking Live Voice tool wrapper — app/tools/async_wrapper.py (non_blocking_tool)
  2. Pluggable Multi-Harness Sandbox protocol — app/workers/harnesses.py (CodingHarness) + app/workers/sandbox.py (SandboxProvisioner)
  3. Two-phase Plan → Execute & worktree handoff — app/workers/task_worker.py (TaskWorker.dispatch_task, switch_harness)
  4. Scoped A2UI v0.9 surface deck emitter — app/callbacks/a2ui_emitter.py (emit_surface_for_tool)
  5. Declarative OAuth 2.0 + ADC Workspace bridge — app/auth.py (IntegrationAuthManager)
  6. Human + AI-Agent dual-mode onboarding — scripts/provision_sandbox.py (--non-interactive)

See AGENTS.md for the complete architecture map, start-here file table, and troubleshooting reference.

Quickstart

Prerequisites

  • Python 3.11+ with uv installed
  • Node.js 20+ and npm
  • Google Cloud SDK (gcloud) authenticated against a GCP project with Vertex AI API enabled (gcloud auth login)
  • (Optional) GitHub CLI (gh) logged in (gh auth login) to automatically clone your personal forks into the sandbox

1. Clone & run interactive onboarding (5 minutes)

The onboarding wizard (make onboard) configures your GCP project, provisions your live Vertex AI Agent Engine Sandbox, clones your personal GitHub forks into /workspaces, and walks through connecting Google Workspace, Google Search, Google Maps, GitHub, Spotify, and Slack (or press Enter to skip any integration and toggle it later from the mobile UI):

git clone https://github.com/allen-stephen/adk-sonar.git
cd adk-sonar
make onboard

2. Verify live connections

Run the health check suite at any time to verify your Vertex AI credentials, remote sandbox reachability, and connected apps:

make check

3. Start the voice orchestrator

make dev

Tap the center Mic button to start a live voice session, or open Connected Apps (plug icon in the top header) to toggle integrations with 1-click OAuth.

Essential Environment Variables

Variable Description Default / Example
GOOGLE_CLOUD_PROJECT Active Google Cloud project ID hosting Vertex AI e.g. agents-cli-test-dev-qi5zi1
GOOGLE_CLOUD_LOCATION Region for Gemini Live native audio and Vertex Sandbox us-central1
SANDBOX_GCP_PROJECT GCP project ID hosting the Vertex Sandbox container Inherits GOOGLE_CLOUD_PROJECT
LIVE_VOICE_NAME Default Gemini Live persona voice (Aoede, Puck, Charon, Kore) Aoede
GITHUB_PERSONAL_ACCESS_TOKEN GitHub PAT with repo, read:org, and workflow scopes. Auto-populated from gh auth token by make onboard. ghp_...
SPOTIFY_CLIENT_ID Spotify Developer Dashboard App Client ID. Create an app at developer.spotify.com/dashboard. —
SPOTIFY_CLIENT_SECRET Spotify Developer Dashboard App Client Secret. —
SLACK_BOT_TOKEN Slack Bot User OAuth Token. Create a Slack app and install it at api.slack.com/apps. xoxb-...

Build your own with a coding agent

This repo includes an AGENTS.md designed to be read by both humans and coding agents (Claude Code, Gemini CLI, Antigravity, Cursor). Point your coding agent at this repo and ask it to lift the patterns you need:

git clone https://github.com/allen-stephen/adk-sonar.git
cd adk-sonar
# Launch your coding agent (claude, gemini, etc.)

Example prompts:

  • "Read AGENTS.md. I want to build a voice-controlled DevOps incident responder that uses the @non_blocking_tool wrapper and A2UI surface cards for PagerDuty and Cloud Logging."
  • "Read AGENTS.md and show me how TaskWorker.switch_harness preserves PLAN.md and git worktree state when switching between horizon and claude."
  • "Run uv run python scripts/provision_sandbox.py --non-interactive --check-only to inspect my current environment, then add a new Linear integration to config/integrations.yaml and app/tools/integration_tools.py."

Learn & adapt

If you want to understand... Read this file
How the Gemini Live voice persona and 13 tools are wired app/agent.py (root_agent, SYSTEM_INSTRUCTION)
How tools return {status: "RUNNING"} immediately without stalling voice audio app/tools/async_wrapper.py (non_blocking_tool)
How background coding tasks run inside Vertex AI Agent Engine Sandboxes app/workers/sandbox.py (SandboxProvisioner)
How horizon, antigravity, and claude share worktrees and PLAN.md app/workers/harnesses.py & app/workers/task_worker.py
How tool results map to scoped A2UI v0.9 visual cards app/callbacks/a2ui_emitter.py & web/src/components/a2ui/A2UISurfaceDeck.tsx
How OAuth 2.0 PKCE, refresh tokens, and gcloud ADC scopes work app/auth.py (IntegrationAuthManager)

For the full file map, maintenance rules, and troubleshooting guide (X-Goog-User-Project headers, Spotify 127.0.0.1 loopback, Sandbox 502 eviction recovery), read AGENTS.md.

A2UI Visual Surface Catalog

Surface Card Component File Triggered By Visual Payload
Task Status Card TaskStatusCard.tsx Task dispatch & execution Real-time terminal output, elapsed time, and active harness
Plan Review Gate PlanReviewCard.tsx mode="plan" completion Markdown plan diff, clarify questions, approve / steer buttons
GitHub PR Card GitHubPRCard.tsx github_operations tool PR status badges, diff lines, author, and branch labels
Spotify Player Card SpotifyPlayerCard.tsx spotify_playback tool Album art, playback state, active device, and track details
Calendar Agenda Card CalendarAgendaCard.tsx calendar_events tool Upcoming event titles, start/end times, attendee list, and video call join links
Workspace Digest Card WorkspaceDigestCard.tsx workspace_digest tool Unread message count, priority email summaries, and recent Drive file activity
Google Place Card PlaceCard.tsx maps_search tool Place name, address, star rating, opening hours, and Google Maps link

Deploy

Deploy the full-stack application (compiled React PWA + FastAPI WebSocket server) to Google Cloud Run and sync your local .env secrets in two commands:

# 1. Build and deploy container to Cloud Run (us-central1)
make deploy

# 2. Push your local .env integration tokens & APP_URL to the live Cloud Run service
make sync-secrets

Tip

After deploying to Cloud Run, add https://<your-cloud-run-url>/api/v1/auth/<integration_id>/callback to your Spotify Developer Dashboard and Google Cloud Console OAuth 2.0 Redirect URIs so 1-click OAuth works on your mobile device anywhere.

Disclaimer

This repository is for demonstrative and educational purposes only. It is not an officially supported Google product.

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

2 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages