feat: fail-closed capability negotiation for gateway↔supervisor signals under version skew #2949

References: originates from PR #2852 (--no-login-shell, which Closes #2668) and review finding GATOR-7d79a247-01 on that PR.

Related issues: #1731 (push gateway-owned desired state — deliberately excludes capability negotiation for the config-push pipeline; see Relationship below), #2589 (in-place supervisor upgrade via re-exec — complementary root-cause fix; see Alternatives).

Suggested labels: area:gateway, area:supervisor, topic:security (this gates a security-relevant opt-out; not a vulnerability disclosure — no live exploit — so it is filed as a feature per SECURITY.md).

User Story

As an operator running long-lived sandboxes who upgrades the gateway in place, I want the gateway to reject a requested sandbox behavior when the sandbox's supervisor doesn't support it, so that a security- or correctness-relevant option never silently degrades to the old behavior without any signal.

Problem Statement

The supervisor is baked into each sandbox at creation and never updates for that sandbox's lifetime, while the gateway is upgraded independently. When a newer gateway sends a behavior signal to an older supervisor that predates it, the supervisor accepts the request but ignores the signal — there is no mechanism for the gateway to learn which behaviors the live supervisor actually understands. Today these signals are carried as SSH environment requests, which an older supervisor acknowledges with success even when it does not act on them.

Impact / Why This Matters

Without this, a gateway-issued option can silently no-op against an already-running sandbox, and the caller receives a normal success as if it had taken effect.

• Concrete case: --no-login-shell (PR feat(sandbox): add --no-login-shell to skip shell startup files on exec #2852, addressing sandbox exec always runs commands through a login shell, so sandbox-user startup files run before the requested command #2668). Against a pre-change supervisor, the command still runs through a login shell (bash -lc), the user's profile files are sourced, and the caller sees a clean success. For a security opt-out on a trusted exec boundary, silent failure is worse than an explicit error.
• This is not exec-only. There are already two such signals riding SSH env requests — no_login_shell (exec) and main_read_only (main attachment) — so the skew spans more than one pipeline (exec, interactive shell/PTY, direct-tcpip tunnels, subsystems, main attach all share the same gateway↔supervisor SSH connection).
• The skew can occur within a single normal deploy (a sandbox created just before a gateway upgrade), not only across releases, so a pre-1.0 "we break things" stance does not remove the exposure.

Current workaround: none. The gateway cannot tell an honored signal from a silently dropped one.

Proposed Design

From the operator's and caller's perspective:

• When a caller requests a behavior the live supervisor supports, it works as today.
• When a caller requests a behavior the live supervisor does not support, the operation fails fast with a clear, actionable error (e.g. gRPC FailedPrecondition: "supervisor predates this capability; recreate the sandbox") instead of silently running the old behavior.
• The capability check is negotiated once per SSH session and applies to every pipeline on that connection (exec, interactive, tunnels, subsystems, main attach), not re-solved per feature.
• Adding a new skew-sensitive behavior in the future should not require re-inventing detection — it should reuse the same negotiation and the same fail-closed path.

Internal wire format is left to implementers; directions are in Alternatives.

Relationship to #1731 (why this is not the rejected negotiation)

#1731 ("Push gateway-owned desired state to supervisors") resolved to not add capability negotiation, a protocol-version field, or a rollout mode, using a required bootstrap exchange plus bounded timeout as its compatibility gate. That decision is scoped to the config/policy/provider/inference push pipeline over the reverse ConnectSupervisor session, where the gateway controls both ends of a single required exchange and can fail the whole init if the bootstrap does not complete.

This issue is a different pipeline: per-request behavior signals carried as SSH env requests (exec, interactive/PTY, tunnels, subsystems, main attach). Two properties make #1731's gate insufficient here:

• These signals are optional and per-operation, not a single required init exchange. An older supervisor accepts the SSH session, the env request, and the channel — and replies channel_success unconditionally (crates/openshell-supervisor-process/src/ssh.rs:726) — so there is no bootstrap-style all-or-nothing gate to hang the result on.
• The failure is silent per-command, not at init: the session is healthy, only the individual behavior no-ops.

So the fail-closed check must be consulted at each affected call site, keyed on something learned once per session. This does not reintroduce config-push negotiation; it reuses #1731's spirit (learn compatibility once, fail clearly) on the SSH transport. If maintainers prefer to fold this into #1731's bootstrap result (advertise SSH-behavior capabilities in SessionAccepted rather than a new SupervisorHello), that satisfies this issue's acceptance criteria — see Alternative 4.

Acceptance Criteria

• A newer gateway paired with an older supervisor rejects an unsupported behavior with a clear FailedPrecondition-style error rather than silently falling back.
• The reproduction for no_login_shell fails closed: with a profile marker seeded, requesting a non-login exec against a pre-change supervisor returns an unsupported-feature error and does not source the profile.
• Capability support is determined once per session and consulted by all affected pipelines (exec unary + interactive at minimum).
• When the supervisor supports the capability, behavior is unchanged (no regression, no added per-command round-trip).
• Adding a future skew-sensitive signal reuses the negotiation without a new detection mechanism.

Alternatives Considered

1. SupervisorHello capability set (preferred direction). Supervisor advertises a set of supported capability strings once per session; gateway caches it on the live session and gates each behavior against it. Additive (new behavior = new string), composes across pipelines, no version-ordering tables. Cost: a new negotiation message/contract to define and maintain, and one exchange at session open.
2. SSH identification banner version gate. Supervisor stamps its OpenShell version into the SSH identification string (SSH-2.0-<software>), already exchanged at connection start; gateway parses the remote id and gates on a version threshold. No new message, no proto field, no extra round-trip. Narrower: carries a version, not a capability list, so each future behavior needs its own "added in version ≥ X" mapping on the gateway side. (Verified feasible in russh 0.62: supervisor can set server::Config.server_id; client exposes Session::remote_sshid() — though it lives on the client Session, not the Handle, so the gateway must capture it in a Handler callback and thread it out to the exec path.)
3. Do nothing (rely on pre-1.0 breakage tolerance). Rejected: the exposure occurs in normal in-place upgrades, and the failure mode is a silent no-op on a security-relevant option.
4. Fold capabilities into feat(config): introduce push config delivery over the supervisor session #1731's SessionAccepted bootstrap. Rather than a new SupervisorHello, the supervisor advertises its SSH-behavior capability set as one field of the existing bootstrap result feat(config): introduce push config delivery over the supervisor session #1731 already requires. The gateway caches it on the live session and gates behaviors identically. Reuses an exchange that must exist anyway (no new message, no extra round-trip) and stays consistent with feat(config): introduce push config delivery over the supervisor session #1731's "learn compatibility once" gate. Cost: couples this fail-closed path to feat(config): introduce push config delivery over the supervisor session #1731 landing first, and mixes SSH-transport concerns into a config-push message. Viable if feat(config): introduce push config delivery over the supervisor session #1731 ships before or with this work.
5. In-place supervisor upgrade via re-exec (feat: support in-place supervisor upgrade via re-exec #2589). If the supervisor can replace its own binary in place, a live sandbox's supervisor gains new capabilities without recreation and the skew disappears at the source. Complementary, not competing: re-exec requires release-matched artifacts and does not help a sandbox whose supervisor has not yet been re-exec'd, so a fail-closed negotiation is still needed for the window before/without upgrade. Out of scope here; tracked in feat: support in-place supervisor upgrade via re-exec #2589.

Direction (1)/(4) is preferred because the skew already spans multiple pipelines and is expected to grow; a session-scoped capability set turns "re-solve skew per feature" into "add one string," at zero steady-state cost. Choice between a standalone SupervisorHello (1) and folding into #1731's bootstrap (4) depends on sequencing of #1731.

Agent Investigation

• Gateway emits the signal at two exec sites: crates/openshell-server/src/grpc/sandbox.rs:2170 (interactive) and :2359 (unary) — both via set_env(false, NO_LOGIN_SHELL_ENV.0, ...) (want_reply=false). These are the sites gator flagged.
• Supervisor consumes it in crates/openshell-supervisor-process/src/ssh.rs: env_request (fn at :701) — no_login_shell variable check at :715/set at :718, main_read_only at :720; the flag maps to the shell argument in login_shell_flag at :1058 (-c vs -lc), covered by test login_shell_flag_controls_profile_sourcing at :1778.
• env_request unconditionally replies channel_success at :726 for any variable, so an older supervisor acks an unknown signal with success — there is no reply-based signal to detect a dropped capability (this rules out flipping want_reply on the env request).
• Two signals already ride SSH env requests today: OPENSHELL_NO_LOGIN_SHELL (exec) and OPENSHELL_MAIN_READ_ONLY (main attachment), confirming the pattern spans pipelines.
• russh 0.62 supports both halves of the banner approach: server::Config.server_id (set) and Session::remote_sshid() (read, client side), with the plumbing caveat noted above.
• Duplicate scan (issues, all states): no existing issue proposes capability negotiation. Closest are feat(config): introduce push config delivery over the supervisor session #1731 (config-push, explicitly excludes negotiation — see Relationship) and feat: support in-place supervisor upgrade via re-exec #2589 (re-exec upgrade, complementary — see Alternatives). Parent bug sandbox exec always runs commands through a login shell, so sandbox-user startup files run before the requested command #2668 is state:accepted; origin PR feat(sandbox): add --no-login-shell to skip shell startup files on exec #2852 is still open.