feat(ask): render tool questions as a native form - #854
Open
snowboardit wants to merge 7 commits into
Open
snowboardit wants to merge 7 commits into
snowboardit wants to merge 7 commits into
Conversation
Pi has no wire protocol for "ask the user a question". Extensions build a terminal component and hand it to ctx.ui.custom(), which leaves the host with rendered text and no idea what was asked. Add a registry that maps one known tool call onto a question description, checks a submitted answer against that question, and produces the value the extension expects back. The first entry covers the ask_user tool from pi-ask-user. Option parsing accepts the key aliases models fall back to when a proxy mangles the schema. Nothing uses this yet.
Correlate a custom UI request with the tool call that runs when the request arrives. The extension API supplies no tool identity with ctx.ui.custom(), and an asking tool blocks while its question is open, so the running tool calls are both the only correlation available and a sufficient one. A recognized question rides along on the extension_ui_request event, and a new extension_ui_ask_response command answers it. The submission is checked against the question before the extension's promise resolves; one that does not fit leaves the question open. Unrecognized custom UIs are untouched and keep streaming terminal lines.
A terminal panel cannot be used with a pointer and is close to unusable on a phone, so a recognized question now gets a real form: single select that submits on tap, multi-select with a count, a freeform row, an optional comment, and Skip. Multi-select keeps a typed answer beside the checked options, because a custom answer should not cost the user their checks. Keyboard support sits on top of real buttons, not in place of them. The header chevron minimizes the card instead of dismissing it, so the conversation stays readable while the user decides. Only Skip cancels. Unrecognized custom UIs keep the terminal panel, and a recognized question can still be answered there through the terminal-view button.
A question the user answered is a decision, not tool output. Render a finished ask_user call as a decision card built from the tool result details: the question, the options offered, which ones were chosen, any comment, and whether the question was dismissed. A question still waiting on an answer renders the same card from the tool arguments.
A turn's tool calls are folded into the process-details group, which collapses once the turn has a final answer. That hid every decision card behind a disclosure, so an answered question looked lost. Pull question tool calls out of that group and render them in the turn.
A blocked custom UI notified with generic text. A recognized question carries its own words, so show them and let the user decide from the notification whether to come back now.
Write down why Pi Web holds knowledge of one extension's shapes, how a custom UI is correlated with a tool call, and what keeps that coupling contained.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Pi has no wire protocol for asking the user a question. Extensions that ask one, such as the
ask_usertool frompi-ask-user, build a terminal component and pass it toctx.ui.custom().Pi Web runs that component headless, renders it to text lines, and streams the lines to the browser with raw keystrokes forwarded back. That bridge is correct for arbitrary extension UIs, but a text terminal cannot be used with a pointer, and it is close to unusable on a phone. In practice the user reads a fake terminal in an "Extension panel" and then types the answer into the composer by hand.
Answering a question is a core part of an agent session, so it gets a real interface.
Approach
A small adapter layer in
lib/structured-ask.tsmaps one known question-asking tool onto a structured description (question, context, options, multi-select, freeform, comment), and maps a submitted answer back onto the value that the extension'sctx.ui.custom()promise must resolve with.The server correlates a custom UI request with the tool call that is running when the request arrives. The extension API supplies no tool identity with
ctx.ui.custom(), and a question-asking tool blocks while its question is open, so the running tool calls are both the only correlation available and a sufficient one.When an adapter matches, the
extension_ui_requestevent carries anaskfield and the browser renders a native form. The browser answers with anextension_ui_ask_responsecommand. The server checks the submission against the question it answers, then resolves the extension's promise. A submission that does not fit leaves the question open. No keystrokes are synthesized anywhere.UI
Answered questions render as decision cards in the transcript, rebuilt from the tool result details. They are also kept out of the collapsed process-details group, which previously hid every answered question behind a disclosure.
Example (single selection)
Asked question
Answered question
Fallbacks
Known limits
docs/adr/0004-structured-ask-adapter.mdrecords the tradeoff and the alternative of upstreaming a descriptor protocol to the extension.ask_user. If two asks were ever in flight in one session, the newest match wins.Testing
tsc --noEmit,eslint, andnpm teston its own.Docs
docs/adr/0004-structured-ask-adapter.mdplus a file-map entry and a design note inAGENTS.md.