fix(local): stop BashCommand when an automation run is cancelled #514

Why

On the OSS Agent Canvas VM (Automation 1.13.3), I cancelled a running local automation through POST /api/automation/v1/runs/{id}/cancel. The run changed to CANCELLED, but its python3 worker.py child kept running and making GitHub requests for several minutes. I had to stop the child process separately. This makes cancellation unreliable as a resource-safety control.

The observed run was e7b70fc0-d599-4101-bd7b-f67e5058998a at 2026-09-23 04:24 UTC. The callback/status path worked; process termination did not. Related but separate from #283, which prevents overlapping cron dispatches.

Desired behavior

Cancelling a local automation run terminates its active Agent Server BashCommand and descendants, then releases the run's execution resources. A terminal CANCELLED run must not keep executing its entrypoint.

Actual behavior

cancel_run (openhands/automation/router.py) only performs process cleanup when run.sandbox_id is set. Local runs have no sandbox — LocalAgentServerBackend.get_execution_context returns sandbox_id=None — so nothing stops the entrypoint dispatched through POST /api/bash/start_bash_command. The run row flips to CANCELLED while its BashCommand process group keeps running until the command's own timeout.

Reproduce: dispatch a local run whose entrypoint is long-running (for example python3 worker.py sleeping well past the run timeout), then curl -X POST "$AUTOMATION_URL/api/automation/v1/runs/$RUN_ID/cancel". The run returns CANCELLED but pgrep -f 'python3 worker.py' still shows the process. The regression tests for this fix live in tests/test_cancel_run.py and run with uv run pytest tests/test_cancel_run.py.

Design decision

Use a targeted per-command stop operation on the Agent Server. The Agent Server (pinned openhands-sdk==1.49.4, and current main) currently exposes no way to stop a running BashCommand: bash_router offers only start, execute, event search/get, and DELETE /api/bash/bash_events (which deletes history, not the process), and the process group is signalled only when the Bash service closes or the command hits its own timeout.

• Automation cancels the run's stored entrypoint bash_command_id through that API.
• Stopping that command terminates its process group/descendants.
• It must not stop unrelated BashCommands or the agent's TerminalTool activity on the shared local Agent Server.
• Successful stop, already-finished, and not-found are all treated as idempotent cancellation.
• The SDK endpoint is tracked in [Feature]: Agent Server operation to stop a single running BashCommand software-agent-sdk#5257; Automation depends on it.

Rejected alternatives: a host-level process signal issued by Automation, and deleting the shared conversation runtime. Cloud cancellation works only because deleting the sandbox kills the container — local mode cannot reuse that, and that mode-specific difference is documented below.

Acceptance Criteria

• Cancelling a RUNNING local automation stops the run's entrypoint BashCommand (its recorded bash_command_id) through the Agent Server's per-command stop API ([Feature]: Agent Server operation to stop a single running BashCommand software-agent-sdk#5257), terminating the command's process group and descendants without waiting for the command's normal timeout.
• Cancellation targets only that command id: other concurrently running BashCommands and the agent's TerminalTool activity on the shared local Agent Server are left running.
• Cancellation is idempotent — a successful stop, an already-finished command, and an unknown/not-found command id all leave the run CANCELLED and do not surface an error to the caller.
• A cancelled run stays CANCELLED: a late completion callback cannot revert it to COMPLETED or FAILED (the existing optimistic-locking guarantee is preserved).
• The run's execution resources are released on cancel: the stopped command is observable as finished (its exit status is visible to the verifier, so a later watchdog scan does not report it as still running), and the shared Agent Server and its conversation runtime are left intact.
• Cloud mode keeps its existing behavior (sandbox deletion kills the container); the mode-specific difference between cloud and local cancellation is documented in code or docs.
• No host-level signal workaround is added, and the shared conversation runtime is not deleted.
• Focused tests exercise a long-running entrypoint, cancel it, and verify the process exits and resources are released — including isolation from a concurrent command and idempotency for a finished or unknown command id.

Non-goals

• Releasing the run's conversation subject on cancel (tracked separately in Release conversation subjects when runs are cancelled #472).
• Changing cloud-mode cancellation or sandbox lifecycle.
• Making the watchdog's timeout path cancel commands.

Dependency

• [Feature]: Agent Server operation to stop a single running BashCommand software-agent-sdk#5257 adds the Agent Server/SDK per-command stop operation this issue calls. Automation's implementation depends on that endpoint shipping in a pinned SDK release.

OpenHands AI triage

The following comments and acceptance criteria were added by the OpenHands AI agent.

Triage

Reconciled against the maintainer decision on this issue: the termination mechanism is a targeted per-command stop operation on the Agent Server/SDK (option (a)), called by cancel_run with the run's stored entrypoint bash_command_id, and the command scope is that single entrypoint command — not the agent's concurrent TerminalTool activity on the shared local Agent Server. The Agent Server currently has no such operation (bash_router offers start/execute/event search-get plus DELETE /api/bash/bash_events, which deletes history rather than the process, and BashEventService only signals a command's process group on service close or on that command's own timeout), so the endpoint is tracked in OpenHands/software-agent-sdk#5257 and Automation depends on it.

The bug label is added to match this repository's convention for fix issues (see #500, #498, #472, #470) and so the type-specific readiness criteria can be evaluated against this issue.

Desired Behavior

Cancelling a RUNNING local automation run stops its entrypoint BashCommand through the Agent Server's per-command stop API and leaves the run CANCELLED, without waiting for the command's normal timeout. The stop is scoped to the run's recorded bash_command_id so unrelated BashCommands and the agent's TerminalTool activity on the shared local Agent Server keep running. Cloud mode continues to rely on sandbox deletion, and the mode-specific difference is documented rather than worked around.

Acceptance Criteria

• Cancelling a RUNNING local automation stops the run's entrypoint BashCommand (its recorded bash_command_id) through the Agent Server's per-command stop API ([Feature]: Agent Server operation to stop a single running BashCommand software-agent-sdk#5257), terminating the command's process group and descendants without waiting for the command's normal timeout.
• Cancellation targets only that command id: other concurrently running BashCommands and the agent's TerminalTool activity on the shared local Agent Server are left running.
• Cancellation is idempotent — a successful stop, an already-finished command, and an unknown/not-found command id all leave the run CANCELLED and do not surface an error to the caller.
• A cancelled run stays CANCELLED: a late completion callback cannot revert it to COMPLETED or FAILED (the existing optimistic-locking guarantee in complete_run is preserved).
• The run's execution resources are released on cancel: the stopped command is observable as finished (its exit status is visible to verify_run_on_agent_server, so a later watchdog scan does not report it as still running), and the shared Agent Server and its conversation runtime are left intact.
• Cloud mode keeps its existing behavior (sandbox deletion kills the container); the mode-specific difference between cloud and local cancellation is documented in code or docs.
• No host-level signal workaround is added, and the shared conversation runtime is not deleted.
• Focused tests exercise a long-running entrypoint, cancel it, and verify the process exits and resources are released — including isolation from a concurrent command and idempotency for a finished or unknown command id.