Skip to content

Commit 65dad0f

Browse files
committed
feat(sandbox): validate configuration before workload activation
Closes #3145 Signed-off-by: John Myers <9696606+johntmyers@users.noreply.github.com>
1 parent cc780d4 commit 65dad0f

39 files changed

Lines changed: 4266 additions & 1365 deletions

File tree

‎architecture/sandbox.md‎

Lines changed: 43 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -473,6 +473,44 @@ the structured 403 and authors the narrowest rule. Mechanistically mapping L7
473473
would either over-broaden rules or require path-templating logic that rots
474474
quickly.
475475

476+
## Configuration Admission
477+
478+
Gateway-managed supervisors reconcile configuration before launching the main
479+
process or exposing workload services. Admission covers the effective policy,
480+
provider layers, credential bindings, and gateway-derived provenance. Explicit
481+
user and global policy precedence is unchanged; an image without a policy uses
482+
the restrictive baseline. An invalid image policy does not become a launchable
483+
default.
484+
485+
The gateway tracks configuration admission independently of compute health.
486+
A blocked startup remains `Provisioning` with a `ConfigurationInvalid` readiness
487+
condition, even when the container backend reports readiness. Gateway management
488+
operations remain available. Replacing the policy or repairing providers allows
489+
the same supervisor to reconcile and launch; it does not recreate the sandbox.
490+
Static policy fields can be replaced before the first accepted activation.
491+
Admission validates policy composition; image and host setup failures, such as
492+
an unresolved OCI user or unavailable isolation facilities, retain their existing
493+
startup error behavior.
494+
495+
Acceptance identifies the effective policy hash/version, configuration revision,
496+
provider-environment revision, and reporting supervisor instance. Startup captures
497+
the matching provider environment and constructs the runtime before reporting
498+
acceptance. Live reconciliation begins only after the main process has spawned,
499+
so it cannot replace the configuration captured for that launch. Restart resets
500+
admission and requires a fresh accepted configuration.
501+
502+
Policy and provider refreshes are prepared before publication. Publication
503+
invalidates prior policy guards before exposing new provider material and swaps
504+
the policy under the same publication locks. Rejected candidates cannot install
505+
their credentials alongside the previous policy. Existing runtime fail-closed
506+
checks remain necessary for in-flight traffic and invalid live updates.
507+
508+
In sidecar topology, the authenticated process supervisor supplies discovery
509+
from the workload image over the existing control socket. The network supervisor
510+
withholds bootstrap until admission succeeds, then sends the accepted policy and
511+
child environment together. Subsequent configuration messages carry both parts
512+
and an ordered generation; older messages cannot restore stale child credentials.
513+
476514
## Policy Revision Acknowledgement
477515

478516
When the supervisor loads a sandbox-scoped policy from the gateway, it retains
@@ -508,11 +546,11 @@ outages cannot block policy polling, enforcement, settings, or provider
508546
refreshes and cannot permanently lose the initial acknowledgement.
509547

510548
Only sandbox-scoped revisions (`PolicySource::Sandbox`, version greater than
511-
zero) are acknowledged. Global policies and local-file development policies do
512-
not use the sandbox revision API and produce no acknowledgement. When explicit
513-
local Rego and data files are configured, the supervisor continues polling the
514-
gateway for settings and provider refreshes but never replaces the local OPA
515-
engine with a gateway policy revision.
549+
zero) use the policy revision acknowledgement API. Global policies use the
550+
configuration admission contract without a sandbox policy revision acknowledgement.
551+
Local Rego/data overrides remain available for standalone development; combining
552+
them with a gateway-managed sandbox is rejected because the gateway cannot admit
553+
the runtime policy it would enforce.
516554

517555
## Failure Behavior
518556

‎architecture/security-policy.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,14 @@ flag defaults to `false` and is security-flagged in policy approval flows.
142142
Incremental merges only ever add the flag to a matching endpoint; clearing it
143143
requires removing the endpoint or replacing the policy.
144144

145+
Image discovery may persist a desired policy for repair, but does not authorize
146+
workload activation. The gateway applies the credential gate after full provider
147+
composition and provenance derivation. A rejected effective configuration keeps
148+
startup blocked with a bounded diagnostic; the supervisor waits for management
149+
repair instead of launching with connection-time denials or a fallback policy.
150+
Accepted runtime state includes the matching provider-environment revision, so
151+
policy and credential updates cannot activate independently.
152+
145153
The network supervisor independently enforces the same boundary. Credentialed
146154
WebSocket upgrades use the parsed relay, binary frames fail closed, and text
147155
placeholders require rewrite. REST bodies continue streaming when body rewrite is disabled. The relay holds

‎crates/openshell-cli/src/run.rs‎

Lines changed: 118 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1510,11 +1510,23 @@ pub async fn sandbox_get(
15101510
sandbox.object_id().to_string()
15111511
};
15121512

1513-
let config = client
1513+
let config_result = client
15141514
.get_sandbox_config(GetSandboxConfigRequest { sandbox_id })
1515-
.await
1516-
.into_diagnostic()?
1517-
.into_inner();
1515+
.await;
1516+
let config = match config_result {
1517+
Ok(response) => response.into_inner(),
1518+
Err(_) if !policy_only && configuration_failure_message(&sandbox).is_some() => {
1519+
// An invalid desired policy must not hide the status needed to
1520+
// repair it. Keep payload-only reads strict.
1521+
GetSandboxConfigResponse {
1522+
configuration_error: configuration_failure_message(&sandbox)
1523+
.unwrap_or_default()
1524+
.to_string(),
1525+
..Default::default()
1526+
}
1527+
}
1528+
Err(error) => return Err(error).into_diagnostic(),
1529+
};
15181530

15191531
if policy_only {
15201532
let Some(ref policy) = config.policy else {
@@ -1548,6 +1560,22 @@ pub async fn sandbox_get(
15481560
println!(" {} {}", "Id:".dimmed(), id);
15491561
println!(" {} {}", "Name:".dimmed(), name);
15501562
println!(" {} {}", "Phase:".dimmed(), phase_name(sandbox.phase()));
1563+
if let Some(status) = sandbox.status.as_ref() {
1564+
for condition in &status.conditions {
1565+
if matches!(
1566+
condition.r#type.as_str(),
1567+
"ConfigurationReady" | "DesiredConfigurationReady"
1568+
) && condition.status.eq_ignore_ascii_case("false")
1569+
{
1570+
println!(
1571+
" {} {}: {}",
1572+
"Configuration:".dimmed(),
1573+
condition.reason,
1574+
condition.message
1575+
);
1576+
}
1577+
}
1578+
}
15511579
if let Some(exit_code) = sandbox.status.as_ref().and_then(|status| status.exit_code) {
15521580
println!(" {} {}", "Exit Code:".dimmed(), exit_code);
15531581
}
@@ -2375,7 +2403,19 @@ pub async fn sandbox_list(
23752403
Ok(())
23762404
}
23772405

2406+
fn configuration_failure_message(sandbox: &Sandbox) -> Option<&str> {
2407+
sandbox
2408+
.status
2409+
.as_ref()?
2410+
.configuration_admission
2411+
.as_ref()
2412+
.map(|admission| admission.error.as_str())
2413+
.filter(|message| !message.is_empty())
2414+
}
2415+
23782416
fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
2417+
use openshell_core::proto::ConfigurationAdmissionState;
2418+
23792419
let meta = sandbox.metadata.as_ref();
23802420
let labels = meta.map_or_else(|| serde_json::json!({}), |m| serde_json::json!(m.labels));
23812421
let annotations = meta.map_or_else(
@@ -2392,6 +2432,39 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
23922432
"resource_version": provenance.resource_version,
23932433
})
23942434
});
2435+
let conditions: Vec<_> = sandbox
2436+
.status
2437+
.as_ref()
2438+
.into_iter()
2439+
.flat_map(|status| &status.conditions)
2440+
.map(|condition| {
2441+
serde_json::json!({
2442+
"type": condition.r#type,
2443+
"status": condition.status,
2444+
"reason": condition.reason,
2445+
"message": condition.message,
2446+
})
2447+
})
2448+
.collect();
2449+
let admission = sandbox
2450+
.status
2451+
.as_ref()
2452+
.and_then(|status| status.configuration_admission.as_ref())
2453+
.map(|admission| {
2454+
serde_json::json!({
2455+
"state": match ConfigurationAdmissionState::try_from(admission.state) {
2456+
Ok(ConfigurationAdmissionState::Pending) => "pending",
2457+
Ok(ConfigurationAdmissionState::Accepted) => "accepted",
2458+
Ok(ConfigurationAdmissionState::Rejected) => "rejected",
2459+
_ => "unknown",
2460+
},
2461+
"error": admission.error,
2462+
"policy_version": admission.policy_version,
2463+
"policy_hash": admission.policy_hash,
2464+
"config_revision": admission.config_revision,
2465+
"provider_env_revision": admission.provider_env_revision,
2466+
})
2467+
});
23952468
serde_json::json!({
23962469
"id": sandbox.object_id(),
23972470
"name": sandbox.object_name(),
@@ -2403,6 +2476,8 @@ fn sandbox_to_json(sandbox: &Sandbox) -> serde_json::Value {
24032476
"phase": phase_name(sandbox.phase()),
24042477
"current_policy_version": sandbox.current_policy_version(),
24052478
"exit_code": sandbox.status.as_ref().and_then(|status| status.exit_code),
2479+
"conditions": conditions,
2480+
"configuration_admission": admission,
24062481
"created_from_workload_template": created_from_workload_template,
24072482
})
24082483
}
@@ -7081,6 +7156,45 @@ mod tests {
70817156
assert_eq!(json["resources"]["gpu"], 2);
70827157
}
70837158

7159+
#[test]
7160+
fn sandbox_json_exposes_repair_diagnostic_and_accepted_generation() {
7161+
use openshell_core::proto::{ConfigurationAdmissionState, SandboxConfigurationAdmission};
7162+
7163+
let mut sandbox = Sandbox::default();
7164+
sandbox.set_phase(SandboxPhase::Provisioning as i32);
7165+
let status = sandbox.status.as_mut().unwrap();
7166+
status.configuration_admission = Some(SandboxConfigurationAdmission {
7167+
state: ConfigurationAdmissionState::Rejected as i32,
7168+
error: "rule image_api requires L7 inspection".to_string(),
7169+
policy_hash: "candidate-hash".to_string(),
7170+
..Default::default()
7171+
});
7172+
status.conditions.push(SandboxCondition {
7173+
r#type: "ConfigurationReady".to_string(),
7174+
status: "False".to_string(),
7175+
reason: "ConfigurationInvalid".to_string(),
7176+
message: "rule image_api requires L7 inspection".to_string(),
7177+
..Default::default()
7178+
});
7179+
let json = super::sandbox_to_json(&sandbox);
7180+
assert_eq!(json["configuration_admission"]["state"], "rejected");
7181+
assert_eq!(json["conditions"][0]["reason"], "ConfigurationInvalid");
7182+
assert_eq!(
7183+
super::configuration_failure_message(&sandbox),
7184+
Some("rule image_api requires L7 inspection")
7185+
);
7186+
sandbox
7187+
.status
7188+
.as_mut()
7189+
.unwrap()
7190+
.configuration_admission
7191+
.as_mut()
7192+
.unwrap()
7193+
.error
7194+
.clear();
7195+
assert_eq!(super::configuration_failure_message(&sandbox), None);
7196+
}
7197+
70847198
#[test]
70857199
fn sandbox_detail_to_json_includes_policy_fields() {
70867200
let mut sandbox = Sandbox {

‎crates/openshell-cli/tests/ensure_providers_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -551,6 +551,13 @@ impl OpenShell for TestOpenShell {
551551
Err(Status::unimplemented("not implemented in test"))
552552
}
553553

554+
async fn report_sandbox_configuration(
555+
&self,
556+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
557+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
558+
Err(Status::unimplemented("not implemented in test"))
559+
}
560+
554561
async fn report_policy_status(
555562
&self,
556563
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/mtls_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -408,6 +408,13 @@ impl OpenShell for TestOpenShell {
408408
Err(Status::unimplemented("not implemented in test"))
409409
}
410410

411+
async fn report_sandbox_configuration(
412+
&self,
413+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
414+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
415+
Err(Status::unimplemented("not implemented in test"))
416+
}
417+
411418
async fn report_policy_status(
412419
&self,
413420
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/provider_commands_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -981,6 +981,13 @@ impl OpenShell for TestOpenShell {
981981
Err(Status::unimplemented("not implemented in test"))
982982
}
983983

984+
async fn report_sandbox_configuration(
985+
&self,
986+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
987+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
988+
Err(Status::unimplemented("not implemented in test"))
989+
}
990+
984991
async fn report_policy_status(
985992
&self,
986993
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -826,6 +826,13 @@ impl OpenShell for TestOpenShell {
826826
Err(Status::unimplemented("not implemented in test"))
827827
}
828828

829+
async fn report_sandbox_configuration(
830+
&self,
831+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
832+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
833+
Err(Status::unimplemented("not implemented in test"))
834+
}
835+
829836
async fn report_policy_status(
830837
&self,
831838
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-cli/tests/sandbox_name_fallback_integration.rs‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -496,6 +496,13 @@ impl OpenShell for TestOpenShell {
496496
Err(Status::unimplemented("not implemented in test"))
497497
}
498498

499+
async fn report_sandbox_configuration(
500+
&self,
501+
_request: tonic::Request<openshell_core::proto::ReportSandboxConfigurationRequest>,
502+
) -> Result<Response<openshell_core::proto::ReportSandboxConfigurationResponse>, Status> {
503+
Err(Status::unimplemented("not implemented in test"))
504+
}
505+
499506
async fn report_policy_status(
500507
&self,
501508
_request: tonic::Request<openshell_core::proto::ReportPolicyStatusRequest>,

‎crates/openshell-core/src/grpc_client.rs‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -822,6 +822,40 @@ pub async fn sync_policy_and_fetch_snapshot(
822822
fetch_settings_snapshot_with_client(&mut client, sandbox_id).await
823823
}
824824

825+
/// Report an exact runtime configuration generation. Pending registration uses
826+
/// the snapshot's instance fence; retain that snapshot across registration retries.
827+
pub async fn report_sandbox_configuration(
828+
endpoint: &str,
829+
sandbox_id: &str,
830+
instance_id: &str,
831+
snapshot: Option<&SettingsPollResult>,
832+
state: crate::proto::ConfigurationAdmissionState,
833+
error: &str,
834+
) -> Result<()> {
835+
let mut client = connect(endpoint).await?;
836+
client
837+
.report_sandbox_configuration(crate::proto::ReportSandboxConfigurationRequest {
838+
sandbox_id: sandbox_id.to_string(),
839+
expected_instance_id: snapshot.map_or_else(String::new, |snapshot| {
840+
snapshot.configuration_instance_id.clone()
841+
}),
842+
admission: Some(crate::proto::SandboxConfigurationAdmission {
843+
instance_id: instance_id.to_string(),
844+
state: state.into(),
845+
policy_version: snapshot.map_or(0, |snapshot| snapshot.version),
846+
policy_hash: snapshot
847+
.map_or_else(String::new, |snapshot| snapshot.policy_hash.clone()),
848+
config_revision: snapshot.map_or(0, |snapshot| snapshot.config_revision),
849+
provider_env_revision: snapshot
850+
.map_or(0, |snapshot| snapshot.provider_env_revision),
851+
error: error.to_string(),
852+
}),
853+
})
854+
.await
855+
.into_diagnostic()?;
856+
Ok(())
857+
}
858+
825859
/// Fetch provider environment variables for a sandbox from `OpenShell` server via gRPC.
826860
///
827861
/// Returns a map of environment variable names to values derived from provider
@@ -920,6 +954,9 @@ pub struct CachedOpenShellClient {
920954
/// Settings poll result returned by [`CachedOpenShellClient::poll_settings`].
921955
#[derive(Clone, Debug)]
922956
pub struct SettingsPollResult {
957+
pub configuration_instance_id: String,
958+
pub configuration_admitted: bool,
959+
pub configuration_error: String,
923960
pub policy: Option<ProtoSandboxPolicy>,
924961
pub version: u32,
925962
pub policy_hash: String,
@@ -941,6 +978,9 @@ pub struct SettingsPollResult {
941978

942979
fn settings_poll_result(inner: crate::proto::GetSandboxConfigResponse) -> SettingsPollResult {
943980
SettingsPollResult {
981+
configuration_instance_id: inner.configuration_instance_id,
982+
configuration_admitted: inner.configuration_admitted,
983+
configuration_error: inner.configuration_error,
944984
policy: inner.policy,
945985
version: inner.version,
946986
policy_hash: inner.policy_hash,

0 commit comments

Comments
 (0)