Skip to content

feat(sandbox): validate effective policy before workload activation #3145

Closed
#3259
Closed
feat(sandbox): validate effective policy before workload activation#3145
#3259
Assignees
johntmyers
Labels
state:acceptedA maintainer decided OpenShell should pursue this issue

Description

@johntmyers
johntmyers
opened on Sep 2, 2026

User Story

As an OpenShell operator, I want a sandbox's complete effective policy validated before its workload starts, so that an incompatible image policy, user policy, or provider configuration cannot produce a running sandbox whose traffic is only rejected later at connection time.

Problem Statement

A sandbox image may contain a policy that is not available when the gateway initially validates sandbox creation and attached providers. The supervisor later publishes that image policy to the gateway. That delivery path cannot reject the policy safely today because doing so can crash-loop the sandbox, so it may install an effective policy that would have failed the normal credential and policy gates.

This creates a split lifecycle: the sandbox can become usable before OpenShell has validated the exact combination of image policy, user-authored policy, provider layers, and gateway-derived credential provenance that the runtime will enforce. PR #3129 demonstrates one consequence: a legacy image policy contains L4-only GitHub endpoints, an attached credentialed GitHub provider causes those endpoints to become credentialed, and the network supervisor must deny the connection at runtime.

OpenShell is moving toward an unopinionated model in which the project provides only a minimal default image and users, vendors, and organizations own compatible image, policy, and provider bundles. Removing project-authored default policies reduces accidental conflicts, but it does not establish the required safety invariant for third-party bundles.

Impact / Why This Matters

Operators can currently receive a sandbox that appears to have started successfully but cannot perform expected work because its effective policy is invalid. The failure occurs late, potentially as an HTTP proxy error from an application, rather than as an actionable sandbox configuration error.

The current workaround is to diagnose the runtime denial and replace or edit the policy after startup. That is insufficient because the workload may already be running, incremental edits may be blocked by whole-policy validation, and behavior differs depending on which policy delivery path admitted the configuration.

A runtime connection-time backstop remains necessary for defense in depth, but it should not substitute for validating configuration before workload activation.

Proposed Design

Before a sandbox enters its usable state or launches its workload, OpenShell evaluates the exact effective configuration that will be activated. The candidate includes any policy supplied by the image, user-authored policy or overrides, attached provider policy layers, credential bindings, and gateway-derived credential provenance.

If the effective policy is valid, OpenShell activates that policy and allows the workload to start. The active runtime policy must correspond to the validated generation.

If validation fails, OpenShell does not start the workload or enable its egress. The sandbox exposes a stable configuration-error state and an actionable, bounded diagnostic identifying the invalid policy surface without including credentials or policy payloads. Management operations remain available so the operator can replace the policy, detach or change providers, or otherwise submit a fully valid configuration. Once the complete effective configuration validates, the sandbox can proceed to workload activation.

Images without a policy remain supported. Absence of an image policy must not imply project-authored network access or provider behavior; any effective permissions come from configuration selected by the operator.

The same externally observable invariant applies on initial creation, restart, and any transition that changes the effective policy or provider set: OpenShell never exposes a workload running against an effective configuration that has not passed the applicable validation gates.

Acceptance Criteria

  • When an image contains a policy, OpenShell composes it with the user policy, attached provider layers, credential bindings, and derived credential provenance before the workload starts.
  • A sandbox does not enter its usable/running state or enable workload egress until the exact effective policy generation has passed validation.
  • An invalid effective policy leaves the workload stopped and places the sandbox in a stable, externally visible configuration-error state rather than a crash loop.
  • The validation failure reports an actionable, bounded diagnostic that identifies the relevant rule or endpoint and reason without exposing credential values, policy payloads, or request content.
  • Operators can repair a rejected configuration through management operations, including submitting an atomic replacement policy or changing the provider set, without first activating the invalid workload configuration.
  • After a repaired effective configuration validates, OpenShell can activate it and start the workload normally.
  • Images with no embedded policy are supported and do not receive implicit network or provider permissions from the default image.
  • Initial creation, restart, and effective-policy/provider changes preserve the same validation-before-activation invariant across supported supervisor topologies.
  • Runtime enforcement retains its connection-time fail-closed checks as defense in depth.
  • Tests cover an image policy that is valid alone but becomes invalid after composition with a credentialed provider, a valid image/provider bundle, an image without a policy, configuration repair, restart, and policy-generation consistency at activation.
  • Architecture and user documentation describe the activation gate, failure state, repair workflow, and no-policy behavior.

Alternatives Considered

Rely on connection-time denial. The network supervisor already fails closed when it encounters unsafe traffic, but this produces late application failures and allows a workload to run in a configuration OpenShell already knows is invalid.

Grandfather violations inherited from the current policy. PR #3129 proposes differential edit-time validation so unrelated changes can proceed while inherited violations remain runtime-denied. This can improve compatibility with existing sandboxes, but it creates separate notions of policy admission and runtime usability. It should not replace validation before workload activation.

Validate image policies in isolation. An image policy can be valid by itself and become invalid only after provider composition and credential-provenance stamping. Validation must cover the exact effective candidate.

Ship project-maintained matching image policies and provider profiles. This couples OpenShell to particular workload images and becomes stale as images, tools, and provider behavior evolve. Users, vendors, and organizations should be able to own these bundles while OpenShell enforces the composition contract.

Agent Investigation

This concern is related to but distinct from #3092, which requires validation and normalization at policy-loading and supervisor-activation boundaries. This issue covers lifecycle ordering and the externally observable requirement that the workload itself must not activate until the fully composed image, user, and provider policy has been accepted.

Related context: #2998 and #3129.

Activity

  1. johntmyers
    self-assigned this
    on Sep 9, 2026
  2. johntmyers
    added
    state:acceptedA maintainer decided OpenShell should pursue this issue
    on Sep 9, 2026

  3. johntmyers commented on Sep 9, 2026

    @johntmyers
    johntmyers
    on Sep 9, 2026
    CollaboratorAuthor

    🏗️ build-plan

    Implementation Plan

    Issue type: feat
    Complexity: High
    Confidence: Medium — startup admission is clear; configuration consistency and independently restarting sidecars require coordinated protocol changes.

    Summary

    Introduce explicit configuration admission before workload activation. The supervisor must obtain and install a validated effective policy with matching provider state before launching a workload; rejected startup configurations remain available for gateway-managed repair without a crash loop. Policy loading and fail-closed networking already exist on main, but neither currently gates workload launch.

    Scope

    • proto/openshell.proto and crates/openshell-core/src/grpc_client.rs: configuration admission and exact-generation status/report contracts.
    • crates/openshell-server/src/grpc/{policy,sandbox,provider}.rs and compute/mod.rs: complete effective-policy validation, repairable desired configurations, generation-aware status, and readiness protection.
    • crates/openshell-sandbox/src/{lib,sidecar_control}.rs: startup reconciliation, accepted launch state, staged policy/provider updates, and coherent sidecar bootstrap/update handling.
    • Process launch and client status handling where needed to enforce and expose the new contract.
    • Existing architecture/user documentation, related skills, and focused gateway, supervisor, sidecar, and e2e tests.

    Implementation Steps

    1. Distinguish desired configuration from an accepted runtime generation. Identify a generation using the effective policy identity, configuration revision, and provider-environment revision. Keep rejected desired state inspectable and repairable. Use bounded diagnostics identifying the failing configuration surface without payloads or credentials.
    2. Apply the complete admission gates after provider composition and provenance derivation, including image discovery. A rejected image candidate must not cause supervisor exit or become a launchable restrictive fallback. Preserve restrictive no-image-policy behavior and existing explicit policy precedence.
    3. Add a pre-launch reconciliation loop. Wait for a valid candidate, matching provider snapshot, and successful runtime construction before capturing process identity, filesystem restrictions, and child environment. On rejection, keep management access available through the gateway and await repair; do not start the main process or workload services.
    4. Bind activation reports to the accepted generation and supervisor instance. Prevent stale reports and compute readiness observations from overriding a blocked startup. Restart must revalidate before launching.
    5. Prepare policy and provider updates together before installing them. Rejected desired updates must never mix new credentials with the previous policy. Preserve an already accepted generation under the existing last-valid/fail-closed posture; a rejected new candidate does not retroactively make the old accepted generation invalid.
    6. Carry the same contract across sidecar bootstrap, update, and reconnect. Obtain image policy from the workload image, and prevent independently restarting supervisors or interleaved messages from activating stale or mixed state.
    7. Verify atomic policy replacement and provider changes while blocked, automatic first launch after repair, and actionable status/watch output. Update existing documentation and related skills.

    Test Plan

    • Gateway unit/integration: valid-alone image policy rejected after credentialed-provider composition; valid composition; retained rejected candidate; policy replacement/provider repair; bounded diagnostics; stale report rejection; readiness cannot hide blocked startup.
    • Supervisor unit/integration: rejected candidate never launches; repair launches once; policy/provider revisions must match; rejected updates cannot install partial state; restart and no-policy behavior.
    • Sidecar tests: rejected/pending bootstrap, repaired activation, reconnect, independent restart, and interleaved configuration messages.
    • E2E: custom-image entrypoint writes a marker; verify absence while blocked, then repair and verify launch. Cover valid/no-policy bundles, provider repair, restart, update consistency, and runtime connection-time denial. Exercise combined Docker and Kubernetes sidecar paths.
    • Run repository-required mise run test, mise run pre-commit, relevant sandbox e2e lanes, and mise run ci before opening a PR. Report environmental limitations rather than claiming unexecuted checks pass.

    Risks & Open Questions

    • A startup-only wait would leave update consistency and sidecar acceptance criteria incomplete.
    • Preserve current explicit user/global policy precedence; do not silently union image permissions into an explicit replacement policy.
    • Distinguish a blocked first launch from completed/stopped workloads so repair cannot accidentally restart a completed main process.
    • Keep sidecar authentication and credential isolation intact. Resolve identity/filesystem restrictions from the repaired accepted configuration; do not add privileges or rely on cross-label /proc access in SELinux tests.
    • Wire compatibility and generation checks must fail closed when an activation contract cannot be established.

    Documentation Impact

    Update architecture/sandbox.md, architecture/security-policy.md, sandbox policy/lifecycle documentation, and relevant provider guidance. Review related skills with sync-agent-infra. No new gateway TOML option is planned; update docs/reference/gateway-config.mdx if validation-failure semantics change. Update deployment/debugging guidance if sidecar wiring changes.

    Revision 1 — initial plan, investigated against main 3ea0ce896ca31a62f377e61ffe0ebb2843c84786

  4. johntmyers commented on Sep 10, 2026

    @johntmyers
    johntmyers
    on Sep 10, 2026
    CollaboratorAuthor

    🏗️ build-from-issue-agent

    Implementation Complete

    PR: #3259

    Added generation-specific configuration admission before workload launch, repairable configuration errors, coherent policy/provider updates, sidecar discovery/bootstrap, and CLI/Go status support. Architecture, user documentation, and related skills describe the activation gate and repair workflow.

    Validation passed: full mise run ci, mise run pre-commit, Docker smoke plus the new activation/repair/restart E2E test, and Kubernetes sidecar smoke. The PR includes a detailed E2E attestation and scope/compatibility notes. The server unit suite reported 1,449 passed and 7 ignored; the sandbox library reported 125 passed.

    The issue will auto-close when the PR is merged.

  5. johntmyers
    added 12 commits that reference this issue on Sep 14, 2026

    feat(sandbox): validate configuration before workload activation

    65dad0f

    feat(sandbox): validate configuration before workload activation

    0b11017

    feat(sandbox): validate configuration before workload activation

    84e3d3b

    feat(sandbox): validate configuration before workload activation

    21cdda1

    feat(sandbox): validate configuration before workload activation

    7f88389

    feat(sandbox): validate configuration before workload activation

    8ee7e23

    feat(sandbox): validate configuration before workload activation

    3ac61d1

    feat(sandbox): validate configuration before workload activation

    454d28e

    feat(sandbox): validate configuration before workload activation

    b41d1fd

    feat(sandbox): validate configuration before workload activation

    ac3c2d9

    feat(sandbox): validate configuration before workload activation

    e79edbf

    feat(sandbox): validate configuration before workload activation

    db18050
  6. johntmyers
    closed this as completedin #3259on Sep 18, 2026
  7. johntmyers
    added a commit that references this issue on Sep 18, 2026

    feat(sandbox): validate configuration before workload activation (NVI…

    1d010f4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

state:acceptedA maintainer decided OpenShell should pursue this issue

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions