Skip to content

feat(agent-server): resolve server default tools for profile launches#4030

Open
simonrosenberg wants to merge 17 commits into
mainfrom
openhands/launch-time-agent-overrides
Open

feat(agent-server): resolve server default tools for profile launches#4030
simonrosenberg wants to merge 17 commits into
mainfrom
openhands/launch-time-agent-overrides

Conversation

@simonrosenberg

@simonrosenberg simonrosenberg commented Jul 8, 2026

Copy link
Copy Markdown
Member

HUMAN:

Allow Canvas to launch the real default or named Agent Profile without losing deployment context or Canvas-owned tools.


AGENT:

Why

On main, Canvas bypasses the displayed default Agent Profile and reconstructs an inline agent because profile launch cannot inherit deployment-provided tools or append deployment prompt context. This PR gives the agent-server one default-tool resolution path while letting a preloaded tool declare its own default status beside its SDK registration.

Summary

  • Add agent_launch_additions.system_message_suffix_append for deployment context applied after inline-agent or Agent Profile resolution.
  • Extend register_tool(...) with default=True and expose the usable registered defaults through the SDK registry.
  • Resolve inherited OpenHands profile tools in the agent-server from the canonical SDK set, profile-dependent delegation, usable browser support, and usable registered defaults.
  • Expose that same resolved list as /server_info.default_tools so clients can build compatible inline agents without duplicating server policy.
  • Preserve the profile contract: tools: null inherits server defaults, while any list is exact, including tools: [] for no tools. ACP toolsets remain subprocess-owned.
  • Migrate only a v1, untouched revision-0 OpenHands profile named default from the legacy tools: [] seed to tools: null; new, unversioned, and v2 tools: [] remain explicitly bare.
  • Persist the materialized conversation agent while excluding launch-only context, so resolved tools and the appended suffix survive restore exactly once.

The agent-server does not contain Canvas-specific tool names or deployment-specific tool configuration. A deployment preloads its tool modules; each module owns its registration name and whether it participates in inherited defaults.

REST API contract changes

Compared with base OpenAPI 56ac31719f91 for public /api/** paths.

--- base public OpenAPI
+++ head public OpenAPI
@@ -755,0 +756,2 @@
+schema AgentLaunchAdditions property system_message_suffix_append optional schema=anyOf=[type="string" maxLength=32768,type="null"]
+schema AgentLaunchAdditions type="object" additionalProperties=false
@@ -2128,0 +2131 @@
+schema StartConversationRequest property agent_launch_additions optional schema=anyOf=[AgentLaunchAdditions,type="null"]

Lifecycle

  1. The deployment preloads optional tool modules at server startup.
  2. A tool that should join inherited defaults registers with default=True.
  3. The agent-server composes one resolved default list for profile inheritance and /server_info.
  4. An OpenHands profile inherits that list only when tools is null; explicit lists are unchanged.
  5. Deployment prompt context is appended after agent resolution, and the materialized agent is persisted with the conversation.

The suffix and resolved tools are conversation snapshots. Moving a conversation to a different runtime later still requires an explicit rebinding mechanism.

How to Test

Focused SDK and agent-server tests: 129 passed. Ruff formatting/lint, pycodestyle, Pyright, import-boundary, and tool-registration checks pass. A cross-repository smoke test preloads Canvas's canvas_ui_tool and verifies that it becomes a usable registered default and appears in the agent-server's resolved default list.

Type

  • Bug fix
  • Feature
  • Breaking change
  • Docs / chore

Downstream

OpenHands/agent-canvas#1650 preloads its existing server-side canvas_ui_tool, whose registration declares it as a default, and reads /server_info.default_tools for inline launches.

Fixes #4029.

This PR was updated by an AI agent on behalf of the user.


Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.13-nodejs22-slim Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:c284488-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-c284488-python \
  ghcr.io/openhands/agent-server:c284488-python

All tags pushed for this build

ghcr.io/openhands/agent-server:c284488-golang-amd64
ghcr.io/openhands/agent-server:c284488b694806c50d71373f93928a85c354275a-golang-amd64
ghcr.io/openhands/agent-server:openhands-launch-time-agent-overrides-golang-amd64
ghcr.io/openhands/agent-server:c284488-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:c284488-golang-arm64
ghcr.io/openhands/agent-server:c284488b694806c50d71373f93928a85c354275a-golang-arm64
ghcr.io/openhands/agent-server:openhands-launch-time-agent-overrides-golang-arm64
ghcr.io/openhands/agent-server:c284488-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:c284488-java-amd64
ghcr.io/openhands/agent-server:c284488b694806c50d71373f93928a85c354275a-java-amd64
ghcr.io/openhands/agent-server:openhands-launch-time-agent-overrides-java-amd64
ghcr.io/openhands/agent-server:c284488-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:c284488-java-arm64
ghcr.io/openhands/agent-server:c284488b694806c50d71373f93928a85c354275a-java-arm64
ghcr.io/openhands/agent-server:openhands-launch-time-agent-overrides-java-arm64
ghcr.io/openhands/agent-server:c284488-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:c284488-python-amd64
ghcr.io/openhands/agent-server:c284488b694806c50d71373f93928a85c354275a-python-amd64
ghcr.io/openhands/agent-server:openhands-launch-time-agent-overrides-python-amd64
ghcr.io/openhands/agent-server:c284488-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-slim-amd64
ghcr.io/openhands/agent-server:c284488-python-arm64
ghcr.io/openhands/agent-server:c284488b694806c50d71373f93928a85c354275a-python-arm64
ghcr.io/openhands/agent-server:openhands-launch-time-agent-overrides-python-arm64
ghcr.io/openhands/agent-server:c284488-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-slim-arm64
ghcr.io/openhands/agent-server:c284488-golang
ghcr.io/openhands/agent-server:c284488b694806c50d71373f93928a85c354275a-golang
ghcr.io/openhands/agent-server:openhands-launch-time-agent-overrides-golang
ghcr.io/openhands/agent-server:c284488-golang_tag_1.21-bookworm
ghcr.io/openhands/agent-server:c284488-java
ghcr.io/openhands/agent-server:c284488b694806c50d71373f93928a85c354275a-java
ghcr.io/openhands/agent-server:openhands-launch-time-agent-overrides-java
ghcr.io/openhands/agent-server:c284488-eclipse-temurin_tag_17-jdk
ghcr.io/openhands/agent-server:c284488-python
ghcr.io/openhands/agent-server:c284488b694806c50d71373f93928a85c354275a-python
ghcr.io/openhands/agent-server:openhands-launch-time-agent-overrides-python
ghcr.io/openhands/agent-server:c284488-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-slim

About Multi-Architecture Support

  • Each variant tag (e.g., c284488-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., c284488-python-amd64) are also available if needed

…path

Introduce AgentLaunchOverrides with a system_message_suffix_append field on
StartConversationRequest. The server applies it to the resolved agent after
agent/profile resolution, reusing the same seam that appends worktree guidance.
This lets clients carry deployment-specific, client-computed enrichments (e.g.
a <RUNTIME_SERVICES> block) onto the agent_profile_id launch path, which
otherwise discards the client agent.

Overrides are strictly additive (append-only, cannot swap the LLM or replace
config) and apply uniformly to both launch paths. They are folded into the
resolved agent at creation and excluded from persistence to avoid double
application on resume.

Fixes #4029

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Python API breakage checks — ✅ PASSED

Result:PASSED

Action log

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

REST API breakage checks (OpenAPI) — ✅ PASSED

Result:PASSED

Action log

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Coverage

Coverage Report •
FileStmtsMissCoverMissing
openhands-agent-server/openhands/agent_server
   conversation_service.py68210585%158–159, 168, 194–195, 199–200, 205, 290–291, 294–295, 307–308, 322, 468–469, 502, 505, 512–518, 545, 551, 647, 653, 658, 664, 672–673, 682–685, 694, 706, 714, 766–767, 823, 864–868, 870–871, 874–875, 885, 897–902, 1000, 1007–1011, 1014–1015, 1019–1023, 1026–1027, 1031–1035, 1038–1039, 1068, 1073, 1077, 1117, 1124–1126, 1145, 1166, 1176, 1180, 1182–1183, 1188–1189, 1195–1196, 1206, 1255, 1270, 1308, 1622, 1625
   server_details_router.py58297%27–28
openhands-sdk/openhands/sdk/conversation
   request.py911089%75, 292, 298, 304, 306–307, 310–311, 322, 331
openhands-sdk/openhands/sdk/profiles
   agent_profile.py101694%297, 300, 361, 367, 372, 400
openhands-sdk/openhands/sdk/tool
   registry.py105793%40, 60–61, 72, 85, 121, 163
TOTAL36335731980% 

@simonrosenberg simonrosenberg marked this pull request as ready for review July 8, 2026 16:57
@simonrosenberg simonrosenberg self-assigned this Jul 8, 2026

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ QA Report: PASS

Profile-launched conversations now carry client-supplied launch-time suffix additions through the real agent-server API, and the folded override does not persist as a separate request field.

Does this PR achieve its stated goal?

Yes. The PR set out to let clients send deployment-specific prompt enrichments on the agent_profile_id launch path; I verified this by creating real LLM and agent profiles through the API, then creating a conversation with agent_profile_id plus agent_launch_overrides. On origin/main, the same request created the conversation but the resolved agent suffix remained only PROFILE_BASELINE; on the PR branch, the resolved suffix became `PROFILE_BASELINE

<RUNTIME_SERVICES>..., confirming the enrichment survives profile resolution. I also restarted the API app and reloaded the conversation; the runtime block appeared exactly once and agent_launch_overrides` was absent from the response, matching the no-double-application goal.

Phase Result
Environment Setup make build completed successfully with uv sync --dev and editable packages installed.
CI Status ⏳ 23 successful, 6 pending, 1 skipped, 0 failing at review time.
Functional Verification ✅ Real API requests exercised profile creation, profile-backed conversation launch, and reload behavior.
Functional Verification

Test 1: Profile launch path carries runtime suffix after the PR

Step 1 — Reproduce / establish baseline without the fix:
Checked out origin/main and ran a FastAPI API scenario that:

  1. POST /api/profiles/qa-llm
  2. POST /api/agent-profiles/qa-agent with system_message_suffix: PROFILE_BASELINE
  3. POST /api/conversations with agent_profile_id and agent_launch_overrides.system_message_suffix_append
  4. GET /api/conversations/{id} and inspected the resolved agent context

Command:

git checkout --detach origin/main
QA_HOME=$(mktemp -d) QA_WS=$(mktemp -d) HOME="$QA_HOME" QA_LABEL=BASE   OPENHANDS_SUPPRESS_BANNER=1 uv run python /tmp/qa_agent_launch_overrides.py

Observed output excerpt:

POST /api/profiles/qa-llm 201
POST /api/agent-profiles/qa-agent 201
BASE_HEAD 98f56248
POST /api/conversations 201
launched_agent_profile {'agent_profile_id': '7f6066f8-90bd-49b3-8fe6-707f87ebced6', 'revision': 0}
agent_launch_overrides_in_response <missing>
suffix_exact PROFILE_BASELINE
suffix_contains_runtime False

This confirms the old behavior: a client could include the launch-time field in the JSON payload, but the profile-resolved agent did not receive the runtime-services suffix.

Step 2 — Apply the PR's changes:
Checked out openhands/launch-time-agent-overrides at 29a30cd2.

Step 3 — Re-run with the fix in place:
Command:

git checkout openhands/launch-time-agent-overrides
QA_HOME=$(mktemp -d) QA_WS=$(mktemp -d) HOME="$QA_HOME" QA_LABEL=PR   OPENHANDS_SUPPRESS_BANNER=1 uv run python /tmp/qa_agent_launch_overrides.py

Observed output excerpt:

POST /api/profiles/qa-llm 201
POST /api/agent-profiles/qa-agent 201
PR_HEAD 29a30cd2
POST /api/conversations 201
launched_agent_profile {'agent_profile_id': 'd607827e-14bd-44f1-ad0c-e3028c3b95c8', 'revision': 0}
agent_launch_overrides_in_response <missing>
suffix_exact PROFILE_BASELINE

<RUNTIME_SERVICES>backend=/api/automation</RUNTIME_SERVICES>
suffix_contains_runtime True

This confirms the PR behavior: the same profile-backed launch request now folds the client-owned runtime-services block into the resolved agent's agent_context.system_message_suffix.

Test 2: Folded override survives reload without double application

Step 1 — Establish the relevant prior state:
The PR description says overrides should be excluded from persistence after being folded into the resolved agent, avoiding double application on resume.

Step 2 — Apply the PR's changes:
Used the PR branch (29a30cd2) and created a conversation without an initial message so no LLM execution was needed.

Step 3 — Run create + reload verification:
Command:

QA_HOME=$(mktemp -d) QA_WS=$(mktemp -d) HOME="$QA_HOME"   OPENHANDS_SUPPRESS_BANNER=1 uv run python - <<'PY'
# Creates API profiles, POSTs /api/conversations with agent_profile_id and
# agent_launch_overrides, then creates a new TestClient app and GETs the same
# conversation after startup reload.
PY

Observed output excerpt:

create_status 201
created_execution_status idle
created_override_field <missing>
created_runtime_count 1
created_suffix PROFILE_BASELINE

<RUNTIME_SERVICES>backend=/api/automation</RUNTIME_SERVICES>
reload_status_present True
reloaded_execution_status idle
reloaded_override_field <missing>
reloaded_runtime_count 1
reloaded_suffix PROFILE_BASELINE

<RUNTIME_SERVICES>backend=/api/automation</RUNTIME_SERVICES>

This shows the override is persisted only as part of the resolved agent suffix, not as a separate request field, and reloading the conversation does not append it a second time.

Issues Found

None.

This QA review was created by an AI agent (OpenHands) on behalf of the user.

Final verdict: PASS

@enyst enyst left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure this design is the best? Maybe we could consider first if we can detach all/most of the runtime config from the profile or even from the agent context?

@simonrosenberg

Copy link
Copy Markdown
Member Author

@enyst — good call, I went and traced this end-to-end rather than just wiring the field, and I think you're right: runtime/deployment topology doesn't belong on agent_context. This PR reuses the worktree-guidance seam, which is itself the exact config/runtime conflation you're pointing at — _append_worktree_guidance mutates agent_context.system_message_suffix (a persisted, profile-round-tripped config field) with per-launch data and then freezes the mutated agent into meta.json. This PR just extends that to client-supplied text.

What I found on the way:

The two "obvious" clean fixes don't apply here:

  • Server computes it (the way it computes worktree paths/branch): can't. <RUNTIME_SERVICES> is canvas-layer topology — the automation backend at /api/automation, the X-Session-API-Key / OPENHANDS_AUTOMATION_API_KEY convention, ingress/frontend ports. The agent-server only knows itself; canvas is the authoritative source, so the channel has to accept client-supplied data.
  • Deliver it as an environment event/observation: there's no environment-event channel in the SDK today — the only agent-injected-at-start event is SystemPromptEvent. That'd be net-new infra and cuts against the current grain.

The channel the framework actually wants isn't agent_context — it's the secret_registry pattern. ConversationState.secret_registry is a conversation-scoped object (not agent config, not persisted into the agent record) that Agent.get_dynamic_context(state) merges into the DYNAMIC (uncached) system-prompt tier and renders via a dedicated <CUSTOM_SECRETS> PromptSection. <RUNTIME_SERVICES> is the same shape.

Proposed redesign (replaces the agent_launch_overrides approach in this PR):

  • Add a conversation-scoped runtime_context to StartConversationRequestConversationStatenot on agent_context, not on the profile, not frozen into meta.json.
  • New DYNAMIC-tier RuntimeServicesSection renders it as <RUNTIME_SERVICES>…, mirroring CustomSecretsSection.
  • Because it never touches the agent, it works uniformly on both the agent_profile_id and agent_settings paths — so canvas can delete the agent-canvas#1571 default-profile → agent_settings special-case, and named/custom profiles get it too.
  • Stays a system-role block (DYNAMIC renders as a second system content block), so the mock-llm-automation.spec.ts assertion still holds.
  • Bonus: worktree guidance could later migrate off the suffix-mutation onto the same mechanism.

(The canvas_ui tool half is orthogonal and already tracked by #3978 — this is only the prompt-text half.)

Tradeoff to flag: this is a real SDK-core change (agent / prompt-registry / request / state) + a canvas change, versus the minimal PR here that's already green. I think it's worth it — it's the correct layering and it lets canvas drop the special-case. Before I rebuild: does conversation-scoped runtime_context → dynamic <RUNTIME_SERVICES> section match what you had in mind, or were you picturing pushing it even further off the prompt (e.g. a tool-provided resource the agent queries on demand)?

@enyst

enyst commented Jul 9, 2026

Copy link
Copy Markdown
Member

@OpenHands understand this PR’s goal and the discussion here in top comments.

(clone yourself also smolpaws/smolpaws and look at the skills .agents/skills/ teach-me and show-me)

I want to understand the alternative designs if we move runtime services out of agent context; and their implications. Where else could/should it be, how will it work then for prompt construction (also including at conversation restore time, not only new conversation)

Make an explainer for this or these alternative designs, on enyst.github.io, using those skills as you see fit. Goal is think with me.

Do not modify this PR. Push to main on enyst io the explainer.

@simonrosenberg

Copy link
Copy Markdown
Member Author

@enyst I traced this through current SDK + Canvas main, including create, persistence, cold restore, prompt rendering, plugins/hooks, and both tool-registration paths. My recommendation is: do not merge the generic system_message_suffix_append API as-is. Detach deployment/runtime facts from both AgentProfile and AgentContext, but keep durable agent instructions in AgentContext.

The important lifecycle split is:

  • Profile / durable agent config: agent kind, LLM/profile refs, base server tools, MCP refs, disabled skills, condenser/verification, personality/custom instructions. A profile's existing system_message_suffix is valid for durable instructions.
  • Conversation runtime binding: workspace/worktree and the deployment service topology reachable by this conversation. <RUNTIME_SERVICES> belongs here.
  • Client UI capability: canvas_ui. This exists only while a compatible Canvas client is attached.
  • Server capability: registered executable tools, browser usability, MCP providers, module/plugin implementations.

Why #4030 is the wrong boundary even though it works: the override field is excluded, but its arbitrary text is folded into agent.agent_context.system_message_suffix; that mutated agent is then persisted in meta.json and base_state.json, and the rendered SystemPromptEvent is persisted too. On cold restore the server reconstructs the stored agent, then Agent.init_state() sees the existing SystemPromptEvent and deliberately does not render again. So this is not really an ephemeral launch override: it permanently converts deployment data into agent config, with no typed ownership or refresh semantics. The existing worktree-guidance mutation has the same layering debt; it is precedent for behavior, not for the boundary.

I suggest replacing this PR's model with a narrow typed ConversationRuntimeContext / RuntimeServices input and state field:

  1. Carry structured facts (service kind, URL from the agent's network namespace, API prefix/docs URL, auth env-var name), not pre-rendered prompt text or a generic AgentContext overlay.
  2. Populate it from a server/deployment provider where possible. Pure agent-server discovery is insufficient—the server cannot infer Canvas ingress/automation topology—but Canvas's launcher already owns that topology and can configure the server. If a request-supplied form is retained for separately deployed clients, make that an explicit authenticated/trusted-deployment policy; do not create a raw system-prompt injection bypass.
  3. Add a dedicated dynamic-tier prompt section that renders <RUNTIME_SERVICES> from the typed conversation context. This works identically after either agent_profile_id or agent_settings resolution and keeps profiles portable.
  4. Persist the resolved context as a conversation snapshot, separate from the agent/profile. On restore, the persisted SystemPromptEvent remains the exact-once prompt snapshot. If deployments need rebinding to changed URLs, make that an explicit runtime-context update/versioned event (or a live provider/tool); silently re-reading a profile or appending again would make history nondeterministic.

For canvas_ui, use the existing client_tools channel rather than adding extra_tools or keeping tool_module_qualnames. Its current Python executor is already a no-op acknowledgment while Canvas handles the ActionEvent, which is exactly ClientTool's contract. client_tools is applied after profile resolution, persisted in conversation metadata, re-registered on cold restore, and preserved on forks. It also has defined collision behavior: duplicate request names fail, collisions with server tools fail, and same-name/different-schema reuse fails. By contrast, tool_module_qualnames merely imports an implementation; it does not attach the tool to a profile-resolved agent, and global register_tool currently warns then last-write-wins on collisions. Canvas should migrate the JSON schema/description to client_tools and remove the Python module registration.

Alternatives are weaker:

  • A generic append-only or additive launch-augmentation model is cheap but preserves the ownership/trust/persistence bug.
  • Pure server discovery cannot see Canvas-owned services; server-side provisioning can.
  • A runtime-services tool would stay fresh and reduce permanent prompt text, but the model must know to call it and ACP agents do not receive SDK tools. It is a reasonable later redesign, not a parity-safe replacement today.
  • Plugins/hooks have the wrong lifecycle and semantics: plugins merge skills/MCP/hooks at first use; SessionStart output is not prompt context, and UserPromptSubmit context is user-message augmentation.
  • Removing <RUNTIME_SERVICES> in favor of stable env vars or an MCP/tool resource may ultimately be cleaner, but the automation skills currently depend on deterministic prompt discovery; remove it only with an end-to-end replacement.

This boundary gives profile portability and local/cloud parity, prevents deployment data from entering stored profiles, keeps resume deterministic, preserves API compatibility through optional additive fields, and uses the stricter existing collision rules for UI tools. Cost is higher than this green PR (SDK request/state/prompt section + Canvas migration), but it fixes the model rather than adding another generic escape hatch.

@simonrosenberg simonrosenberg changed the title Allow client-supplied launch-time agent overrides on the agent_profile_id path Model runtime services as conversation context outside Agent Profiles Jul 10, 2026
@simonrosenberg

simonrosenberg commented Jul 10, 2026

Copy link
Copy Markdown
Member Author

@enyst I rebuilt the PR around the detached boundary we discussed.

The generic AgentLaunchOverrides / raw suffix append is gone. ConversationRuntimeContext is now typed conversation state, persisted as a deterministic snapshot and rendered by a dedicated dynamic <RUNTIME_SERVICES> prompt section. It never becomes an AgentProfile or AgentContext field; for ACP, the conversation context is combined by the shared prompt builder only after AgentContext has produced its own inputs. Profile resolution therefore remains authoritative and portable.

The request accepts constrained service facts rather than prompt prose (machine identifier, URL from the agent namespace, API/docs locations, optional auth header/env-var metadata), with validation against extra fields, duplicate/prompt-shaped names, multiline URLs, and invalid auth identifiers. Restore retains the stored context and existing SystemPromptEvent, so it does not re-render or pick up silently changed deployment data. Local, remote, profile, legacy-agent, ACP, and fork paths share the same lifecycle.

I left canvas_ui out of this new model: it belongs on existing client_tools, not in runtime topology or tool_module_qualnames. Canvas can migrate that independently while removing the default-profile agent_settings special-case.

Local verification: 268 selected profile/service/prompt/ACP/remote/OpenAPI tests pass; pre-commit formatting, lint, typing, import rules, and tool-registration checks pass. CI is now running on e77ec5ff4.

simonrosenberg and others added 2 commits July 10, 2026 11:00
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
@simonrosenberg simonrosenberg changed the title Model runtime services as conversation context outside Agent Profiles Add typed runtime services to Agent Profile launches Jul 10, 2026
@simonrosenberg

Copy link
Copy Markdown
Member Author

@enyst I simplified the implementation after weighing the migration cost more heavily.

The final diff is now two production files plus one focused test file, not a new conversation-runtime subsystem. runtime_services is a narrow typed launch input; after profile/direct-agent resolution the server renders it through the existing suffix seam into the resolved conversation agent. The stored Agent Profile remains untouched, while the existing resolved-agent persistence gives deterministic cold restore and exact-once prompt behavior for both OpenHands and ACP without new state/prompt/factory plumbing.

This keeps the important boundary—deployment data cannot be saved into a profile and the client cannot submit arbitrary system-prompt prose—while accepting the pragmatic compromise that the rendered launch snapshot lives in the conversation's resolved AgentContext.

Canvas's canvas_ui capability remains orthogonal: the follow-up uses existing client_tools for OpenHands profile launches. ACP subprocesses own their tools and reject SDK custom tools, so they keep their existing tool behavior.

Local verification: 160 profile/service/OpenAPI tests pass; pre-commit and Python API compatibility checks pass. Head: a267c1da9.

@simonrosenberg

Copy link
Copy Markdown
Member Author

Canvas follow-up: OpenHands/agent-canvas#1650. It removes the local default-profile workaround, sends typed runtime services on profile launches, and uses client_tools for OpenHands canvas_ui. It is draft and explicitly depends on this SDK change.

@simonrosenberg simonrosenberg changed the title Add typed runtime services to Agent Profile launches Add trusted launch augmentation to Agent Profile launches Jul 10, 2026
@simonrosenberg simonrosenberg changed the title Add trusted launch augmentation to Agent Profile launches feat(conversation): add deployment launch additions Jul 14, 2026
@simonrosenberg simonrosenberg added review-this This label triggers a PR review by OpenHands qa-this labels Jul 14, 2026

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ QA Report: PASS WITH ISSUES

Launch-time suffix/tool additions work for inline and Agent Profile launches, survive cold restore once, and legacy revision-0 default profiles regain standard tools; one invalid-input path currently returns HTTP 500.

Does this PR achieve its stated goal?

Yes, with one API-quality issue. I verified the stated Canvas path by running a real local agent-server, creating conversations over HTTP with agent_launch_additions, restarting the server, and fetching the restored conversations: the materialized agent kept the appended suffix/tools and agent_launch_additions was not persisted/reapplied. I also verified profile launch through /api/agent-profiles after seeding a real LLM profile, and the legacy default/revision-0/tools: [] SDK profile now resolves to default tools while edited/custom bare profiles remain bare.

Phase Result
Environment Setup uv sync --dev completed; local PR and baseline main agent-server instances started successfully.
CI Status 🟡 Most checks were successful; 2 Build & Push jobs (golang-amd64, java-amd64) were still in progress when checked. No tests were run manually.
Functional Verification ✅ Core launch additions, profile launch, legacy default-tool upgrade, and cold restore behavior verified via real HTTP/API and SDK calls.
Functional Verification

Test 1: Inline launch additions are ignored before the PR and applied after the PR

Step 1 — Reproduce / establish baseline without the fix:
Started origin/main on 127.0.0.1:8124 and posted the same real /api/conversations payload containing an inline agent plus agent_launch_additions.

Baseline summary:

{
  "tools": [],
  "suffix": "PROFILE_BASELINE",
  "launch_additions": null
}

This confirms the old server accepted the request but did not add the deployment-owned suffix or tool.

Step 2 — Apply the PR's changes:
Started the PR branch server on 127.0.0.1:8123.

Step 3 — Re-run with the fix in place:
Posted the same payload to the PR server and fetched the created conversation.

PR summary:

{
  "tools": ["terminal"],
  "suffix": "PROFILE_BASELINE

<RUNTIME_SERVICES>Automation=http://localhost:18001</RUNTIME_SERVICES>",
  "launch_additions": null
}

This shows the launch-time tool and runtime-services suffix were folded into the materialized conversation agent, while launch-only input was not persisted.

Test 2: Cold restore preserves the materialized agent without duplicating launch additions

Step 1 — Baseline old behavior:
On main, there was no applied launch addition to restore from Test 1.

Step 2 — Apply the PR's changes:
Restarted the PR server and fetched the same conversation ID.

Step 3 — Re-run with the fix in place:
Restored conversation summary:

{
  "tools": ["terminal"],
  "suffix": "PROFILE_BASELINE

<RUNTIME_SERVICES>Automation=http://localhost:18001</RUNTIME_SERVICES>",
  "runtime_services_count": 1,
  "launch_additions": null
}

This confirms the materialized agent survives cold restore exactly once; the suffix was not appended a second time.

Test 3: Agent Profile launch keeps deployment context and standard tools

Step 1 — Reproduce / establish baseline:
Initial profile launch failed until I seeded an LLM profile, which matches real profile dependency behavior:

{"detail":"LLM profile 'default' not found"}
HTTP 404

Step 2 — Apply usable profile setup on the PR server:
Saved a dummy default LLM profile via POST /api/profiles/default, then launched using the active default Agent Profile plus agent_launch_additions.

Step 3 — Re-run with the fix in place:
Profile launch summary:

{
  "tools": ["terminal", "file_editor", "task_tracker", "browser_tool_set"],
  "suffix": "<RUNTIME_SERVICES>ProfileLaunch=http://localhost:18002</RUNTIME_SERVICES>",
  "launched_agent_profile": {
    "agent_profile_id": "99e00f7b-3e30-4fd6-bd8e-019fb134a837",
    "revision": 0
  },
  "launch_additions": null
}

After another server restart, the restored profile conversation still had the same four tools, one runtime-services block, and launch_additions: null. This verifies the stated Canvas/default-profile path: profile resolution happens, launch additions are applied, and the stored profile is not modified.

Test 4: Legacy seeded default profile tools are upgraded, but edited/custom bare profiles stay bare

Step 1 — Reproduce / establish baseline without the fix:
Ran the SDK profile model on origin/main:

main SDK:
default rev=0 tools=[]
default rev=1 tools=[]
bare rev=0 tools=[]

This confirms the legacy seeded default profile remained bare before the PR.

Step 2 — Apply the PR's changes:
Ran the same SDK construction against the PR branch.

Step 3 — Re-run with the fix in place:

pr SDK:
default rev=0 tools=None
default rev=1 tools=[]
bare rev=0 tools=[]

This confirms only untouched revision-0 default profiles are upgraded to inherit standard tools; edited/default and custom bare profiles remain bare.

Test 5: Conflicting launch tool parameters are rejected

Posted an inline agent with terminal params {mode: "profile"} and launch additions with terminal params {mode: "canvas"}.

Observed response:

{"detail":"Internal Server Error","exception":"Launch tool 'terminal' conflicts with the resolved agent","error_id":"d65625f4e7784b47afb7458a59d08c0b"}
HTTP 500

This verifies conflict detection exists, but the API reports invalid user input as a server error; see issue below.

Issues Found

  • 🟠 Issue: Conflicting launch tool parameters are rejected with HTTP 500 instead of a client-facing 4xx validation error.

This review was created by an AI agent (OpenHands) on behalf of the user.

Final verdict: PASS WITH ISSUES

Comment thread openhands-agent-server/openhands/agent_server/conversation_service.py Outdated
@simonrosenberg simonrosenberg added review-this This label triggers a PR review by OpenHands qa-this and removed review-this This label triggers a PR review by OpenHands qa-this labels Jul 14, 2026

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ QA Report: PASS

The PR achieves its goal: real POST /api/conversations calls now launch a named/default Agent Profile with deployment suffix/tool additions while preserving the stored profile and restoring legacy default tools.

Does this PR achieve its stated goal?

Yes. I verified the before/after behavior through the agent-server HTTP API: on the baseline branch, the same launch-additions payload was ignored and produced a conversation with suffix: null and tools: []; on the PR branch, the resolved default profile launched with standard tools plus the added tool and the runtime suffix. I also verified the additions are not persisted as launch input, survive server restart exactly once as materialized agent state, duplicate identical tools are no-ops, and conflicting tool params return HTTP 409.

Phase Result
Environment Setup ✅ Baseline and PR-head agent-server instances ran with isolated HOME/workspace/profile stores.
CI Status 🟡 Observed 35 successful, 14 skipped, and 1 in-progress check (qa-changes); no local test suite was run.
Functional Verification ✅ Real API conversation creation, profile launch, persistence/restore, module-qualified tool addition, and conflict paths worked.
Functional Verification

Test 1: Baseline ignores launch additions and exposes the original failure mode

Step 1 — Reproduce / establish baseline without the fix:

Started the baseline server from origin/main / 56ac3171 on 127.0.0.1:8765, seeded a legacy revision-0 default profile with raw tools: [], then created a conversation with agent_profile_id plus launch additions:

curl -sS -X POST http://127.0.0.1:8765/api/conversations   -H 'Content-Type: application/json'   --data-binary @/tmp/ohqa-pr4030-base-run/start_payload.json   | jq '{id, status:.execution_status, profile:.launched_agent_profile, suffix:.agent.agent_context.system_message_suffix, tools:[.agent.tools[]?.name], detail}'

Observed:

{
  "id": "6f762dd3-bbdb-4126-ac24-cc3ad49feab5",
  "status": "idle",
  "profile": {
    "agent_profile_id": "d70e8f49-b8a7-4b00-912a-48e67f679917",
    "revision": 0
  },
  "suffix": null,
  "tools": [],
  "detail": null
}

This confirms the pre-PR profile launch path loses the deployment-owned suffix/tool additions and that the legacy bare default profile launches without the standard toolset.

Step 2 — Apply the PR's changes:

Started the PR branch openhands/launch-time-agent-overrides at 1b5bab83 on 127.0.0.1:8766, using the same isolated setup and raw legacy default profile shape.

Step 3 — Re-run with the fix in place:

curl -sS -X POST http://127.0.0.1:8766/api/conversations   -H 'Content-Type: application/json'   --data-binary @/tmp/ohqa-pr4030-head-run/start_payload.json   | jq '{id, status:.execution_status, profile:.launched_agent_profile, suffix:.agent.agent_context.system_message_suffix, tools:[.agent.tools[]?.name], detail}'

Observed:

{
  "id": "ba74c601-8d60-42b8-a2ad-bdaa44fec203",
  "status": "idle",
  "profile": {
    "agent_profile_id": "ead48b90-75a1-4ac9-8aaf-aad107c3bbf5",
    "revision": 0
  },
  "suffix": "<RUNTIME_SERVICES>
* QA baseline marker
</RUNTIME_SERVICES>",
  "tools": [
    "terminal",
    "file_editor",
    "task_tracker",
    "browser_tool_set",
    "qa_launch_tool"
  ],
  "detail": null
}

This shows the PR delivers the intended additive launch path and restores standard tools for an untouched legacy default profile.

Test 2: Stored profile is not modified by launch additions

After the PR-head conversation launch, queried the profile API:

curl -sS http://127.0.0.1:8766/api/agent-profiles   | jq '{profiles:[.profiles[] | {name,revision,tools,system_message_suffix}]}'

Observed:

{
  "profiles": [
    {
      "name": "default",
      "revision": 0,
      "tools": null,
      "system_message_suffix": null
    }
  ]
}

The materialized conversation changed, but the durable profile did not gain the launch suffix or appended tool.

Test 3: Cold restore keeps materialized additions exactly once

Restarted the PR-head server and fetched the previously created conversation:

curl -sS http://127.0.0.1:8766/api/conversations/ba74c601-8d60-42b8-a2ad-bdaa44fec203   | jq '{suffix:.agent.agent_context.system_message_suffix, runtime_services_count:([.agent.agent_context.system_message_suffix | scan("<RUNTIME_SERVICES>")] | length), launch_additions:.agent_launch_additions, tools:[.agent.tools[]?.name]}'

Observed:

{
  "suffix": "<RUNTIME_SERVICES>
* QA baseline marker
</RUNTIME_SERVICES>",
  "runtime_services_count": 1,
  "launch_additions": null,
  "tools": [
    "terminal",
    "file_editor",
    "task_tracker",
    "browser_tool_set",
    "qa_launch_tool"
  ]
}

This confirms the launch input is excluded from persistence while the materialized agent survives restore without duplicating the suffix.

Test 4: Conflict and no-op duplicate tool handling

Created a PR-head conversation adding terminal with different params:

curl -sS -o /tmp/ohqa-pr4030-head-run/conflict_response.json -w '%{http_code}
'   -X POST http://127.0.0.1:8766/api/conversations   -H 'Content-Type: application/json'   --data-binary @/tmp/ohqa-pr4030-head-run/conflict_payload.json
jq . /tmp/ohqa-pr4030-head-run/conflict_response.json

Observed:

409
{
  "detail": "Launch tool 'terminal' conflicts with the resolved agent"
}

Then added identical terminal params:

curl -sS -X POST http://127.0.0.1:8766/api/conversations   -H 'Content-Type: application/json'   --data-binary @/tmp/ohqa-pr4030-head-run/noop_payload.json   | jq '{id, status:.execution_status, terminal_count:([.agent.tools[]?.name | select(.=="terminal")] | length), tools:[.agent.tools[]?.name], detail}'

Observed:

{
  "id": "35f9ca6e-bede-4a0b-b6fc-23c9d0f9f0d8",
  "status": "idle",
  "terminal_count": 1,
  "tools": [
    "terminal",
    "file_editor",
    "task_tracker",
    "browser_tool_set"
  ],
  "detail": null
}

This matches the PR's claimed conflict/no-op semantics.

Test 5: Added tools can still use tool_module_qualnames

Created a PR-head conversation adding planning_file_editor with a module-qualified import:

curl -sS -X POST http://127.0.0.1:8766/api/conversations   -H 'Content-Type: application/json'   --data-binary @/tmp/ohqa-pr4030-head-run/module_payload.json   | jq '{id,status:.execution_status, tools:[.agent.tools[]?.name], detail}'

Observed:

{
  "id": "652685af-22d9-4c08-a483-7bcefd9b9cda",
  "status": "idle",
  "tools": [
    "terminal",
    "file_editor",
    "task_tracker",
    "browser_tool_set",
    "planning_file_editor"
  ],
  "detail": null
}

This verifies a real launch-added tool can be materialized while using the existing module-import path.

Issues Found

None.

This QA review was created by an AI agent (OpenHands) on behalf of the user.

Co-authored-by: openhands <openhands@all-hands.dev>
@simonrosenberg simonrosenberg changed the title feat(conversation): add deployment launch additions feat(agent-server): resolve runtime defaults for profile launches Jul 14, 2026
@simonrosenberg

Copy link
Copy Markdown
Member Author

Architecture update in 30a50d812: removed launch-time tool appending. Tool implementations now declare whether they are runtime defaults; the agent-server registers runtime modules before resolving a profile and adds usable runtime defaults only for OpenHands profiles with tools: null. Explicit [] and explicit tool lists remain authoritative. agent_launch_additions now carries deployment prompt context only.

@simonrosenberg simonrosenberg added review-this This label triggers a PR review by OpenHands qa-this and removed review-this This label triggers a PR review by OpenHands qa-this labels Jul 14, 2026

@all-hands-bot all-hands-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ QA Report: PASS

Profile launches now preserve runtime deployment context and runtime-default tools when exercised through the real agent-server API.

Does this PR achieve its stated goal?

Yes. The PR set out to let Canvas/profile launches use the real default or named Agent Profile while keeping runtime-owned defaults and launch-only deployment context. I verified that the same profile-launch request on main lost the suffix and did not include a dynamically registered canvas_ui runtime default, while the PR branch materialized both canvas_ui and the trimmed runtime suffix, persisted them exactly once, and kept explicit bare profiles bare.

Phase Result
Environment Setup uv sync --dev completed; local agent-server started on 127.0.0.1:8765
CI Status ⏳ 29 checks successful, 2 in progress (qa-changes, Build & Push (java-arm64)), 12 skipped; no local test suite run
Functional Verification ✅ Real HTTP profile-launch requests and SDK scripts verified the changed behavior
Functional Verification

Test 1: Profile launch with Canvas-like runtime tool and deployment suffix

Step 1 — Reproduce / establish baseline without the fix:
Checked out main at 56ac3171, started the real server with --extra-python-path /tmp/qa-runtime-tools, saved a minimal default LLM profile through the API, then posted this Canvas-like request to POST /api/conversations:

{
  "agent_profile_id": "<active default profile id>",
  "workspace": {"kind": "LocalWorkspace", "working_dir": "/tmp/qa-main-workspace-custom"},
  "autotitle": false,
  "tool_module_qualnames": {"canvas_ui": "canvas_ui_tool"},
  "agent_launch_additions": {
    "system_message_suffix_append": "<RUNTIME_SERVICES>custom module</RUNTIME_SERVICES>"
  }
}

Observed on main:

HTTP=201
id= ccecfdbd-ed22-4fec-97c0-5fdc9b3c917d
status= idle
tools= ['terminal', 'file_editor', 'task_tracker', 'browser_tool_set']
suffix= None
detail= None

This confirms the old profile-launch path created a conversation but lost the launch-time suffix and did not include the dynamically supplied canvas_ui runtime-default tool.

Step 2 — Apply the PR's changes:
Checked out openhands/launch-time-agent-overrides at 30a50d81 and restarted the same local agent-server with the same custom tool module path.

Step 3 — Re-run with the fix in place:
Posted the equivalent request to POST /api/conversations on the PR branch and then fetched the conversation back with GET /api/conversations/{id}:

HTTP=201
id= 077b002f-ecfc-4f9a-a570-e228ae993779
status= idle
tools= ['terminal', 'file_editor', 'task_tracker', 'browser_tool_set', 'canvas_ui']
suffix_count= 1
suffix= <RUNTIME_SERVICES>
* QA runtime: http://localhost:18001
</RUNTIME_SERVICES>
launched= {'agent_profile_id': '45eca3b0-0ed8-44e5-9388-07ea1f052393', 'revision': 0}
has_agent_launch_additions= False
detail= None

Fetch after creation showed the materialized state persisted as expected:

HTTP=200
suffix_count= 1
tools= ['terminal', 'file_editor', 'task_tracker', 'browser_tool_set', 'canvas_ui']
has_agent_launch_additions= False

This shows the PR imports the runtime tool module before profile resolution, adds the runtime default to a default-tool profile, applies the deployment suffix after resolution, and stores the materialized agent without retaining the launch-only request field.

Test 2: SDK-visible launch additions, runtime defaults, and legacy default profile behavior

Step 1 — Baseline on main:
Ran a small SDK script that constructed a legacy default profile, attempted to import AgentLaunchAdditions, and attempted to query runtime-default tools after importing the custom canvas_ui_tool module:

legacy_default_tools= []
AgentLaunchAdditions= ImportError cannot import name 'AgentLaunchAdditions' from 'openhands.sdk.conversation.request'
runtime_defaults= ImportError cannot import name 'list_usable_runtime_default_tools' from 'openhands.sdk.tool.registry'

This establishes that the base branch had no launch-additions API, no runtime-default registry API, and left untouched revision-0 default profiles with tools=[].

Step 2 — Re-run on the PR branch:
Ran the same script on 30a50d81:

legacy_default_tools= None
AgentLaunchAdditions= {'system_message_suffix_append': 'ctx'}
runtime_defaults= ['canvas_ui']

This confirms the PR exposes the new launch-additions model, lets runtime tool implementations opt into the runtime-default set, and upgrades the legacy untouched default profile shape to default-tool semantics.

Test 3: Explicit bare profile remains authoritative

Created a named OpenHands profile through POST /api/agent-profiles/qa-bare with tools: [], then launched it with the same tool_module_qualnames that registered canvas_ui as a runtime default:

SAVE_HTTP=201
{"name": "qa-bare", "message": "Agent profile 'qa-bare' saved"}
CREATE_HTTP=201
id= 91b94724-e08d-40e3-b559-6d6ac81c3a9a
detail= None
tools= []
launched= {'agent_profile_id': 'dc416d62-1779-4ecd-a613-d06e06bf720d', 'revision': 0}

This verifies the PR preserves explicit tools: [] as a bare profile rather than injecting runtime defaults into every OpenHands profile.

Issues Found

None.

This review was posted by an AI agent (OpenHands) on behalf of the user.

Co-authored-by: openhands <openhands@all-hands.dev>
@simonrosenberg simonrosenberg changed the title feat(agent-server): resolve runtime defaults for profile launches feat(agent-server): resolve server default tools for profile launches Jul 14, 2026
@simonrosenberg

Copy link
Copy Markdown
Member Author

Architecture simplification in f698a22ab: the SDK runtime_default marker and registry bookkeeping are gone. The agent-server now owns default selection through extra_default_tools, resolves tools: null before agent construction, and publishes the same resolved deployment list through /server_info.default_tools. Explicit empty and non-empty profile lists remain authoritative, and request-level module imports cannot alter startup defaults. Canvas consumes this contract in OpenHands/agent-canvas#1650.

simonrosenberg and others added 2 commits July 14, 2026 17:52
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
@simonrosenberg

Copy link
Copy Markdown
Member Author

Migration follow-up in f0555d6ee and 1f6307f20: the risk is real because an older SDK represented global default tools as [], and the profile seed copied that value. Profile schema v2 now migrates only a v1, untouched revision-0 OpenHands profile named default from [] to null. New/unversioned/v2 [], revised v1 profiles, and custom v1 profiles remain explicitly bare. A cross-repository launch smoke test with the Canvas module produced legacy/default → terminal,file_editor,task_tracker,canvas_ui, explicit [] → no tools, and [terminal] → only terminal.

Co-authored-by: openhands <openhands@all-hands.dev>
@simonrosenberg

Copy link
Copy Markdown
Member Author

Superseding the earlier extra_default_tools proposal: default membership now lives beside tool registration via register_tool(..., default=True). The agent-server composes canonical defaults, delegation/browser policy, and usable registered defaults; it has no Canvas-specific name or deployment tool list. Canvas only preloads its module. The cross-repo smoke test resolves the standard tools plus canvas_ui through this registration path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

qa-this review-this This label triggers a PR review by OpenHands

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow client-supplied launch-time agent overrides on the agent_profile_id path (carry canvas-only enrichments like <RUNTIME_SERVICES>)

4 participants