Skip to content

Commit 07a1460

Browse files
committed
feat(driver-oci): add OCI compute driver on the rootfs provider boundary
Adds openshell-driver-oci, an OCI/runc-based compute driver that builds sandboxes from Linux namespaces and cgroups v2. The driver is a sandbox provisioner: it generates the OCI runtime spec, assembles the bundle, manages per-sandbox network namespaces + veth + nftables, and drives the configured low-level OCI runtime (runc/crun) directly through its create/start/state/kill/delete CLI contract. containerd never creates a Container or Task for these sandboxes. All image handling goes through openshell-rootfs's ContainerdRootfsProvider from the previous commit: this crate has no containerd-client dependency and consumes only provider-neutral PreparedRootfs mounts, so the daemon-backed provider can later be swapped for a daemonless one without touching the driver. Also wires ComputeDriverKind::Oci through openshell-core and every openshell-server compute-driver selection path. Hardening from review: sandbox names are validated against path traversal, the runtime's --root is scoped under the driver's own state directory instead of the shared global default, the sandbox endpoint no longer defaults to an unreachable loopback address, per-sandbox subnet allocation detects and avoids collisions instead of a stateless hash, cleanup after a failed create no longer leaves a mounted rootfs and bundle directory behind, and sandbox token files are written with owner-only permissions. Verified end to end against a real containerd 2.x + runc/crun install (see tests/containerd_integration.rs, #[ignore]d in CI since CI has no containerd). Known gaps (rootless mode, image-based supervisor injection, full CDI GPU support, polling-based watch) are documented in the driver README and docs/reference/sandbox-compute-drivers.mdx. Related: #2255 Signed-off-by: Eric Curtin <eric.curtin@docker.com>
1 parent 8f3ab94 commit 07a1460

25 files changed

Lines changed: 3824 additions & 18 deletions

File tree

‎Cargo.lock‎

Lines changed: 47 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎architecture/build.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,8 @@ collect telemetry: `openshell-server` (gateway), `openshell-sandbox`
2929
(supervisor), and `openshell-driver-vm`. Every crate depends on
3030
`openshell-core` with `default-features = false`, so the binary crate's feature
3131
is the single switch that enables `openshell-core/telemetry` for its build
32-
graph. In-process drivers (`docker`, `kubernetes`, `podman`) inherit the
33-
gateway's setting through feature unification and carry no passthrough.
32+
graph. In-process drivers (`docker`, `kubernetes`, `podman`, `oci`) inherit
33+
the gateway's setting through feature unification and carry no passthrough.
3434

3535
Building a binary with `--no-default-features` compiles out telemetry entirely:
3636
no endpoint, no telemetry HTTP client, and no emission code. With telemetry

‎architecture/compute-runtimes.md‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -73,9 +73,9 @@ provisioner on top of it.
7373

7474
`ContainerdRootfsProvider` is the first implementation, backed by a
7575
system-provided containerd used only for image pull/unpack and snapshot
76-
management. The VM driver's image acquisition and ext4 materialization
77-
pipeline is the same boundary shape; converging it onto the shared types is
78-
tracked follow-up work.
76+
management. The OCI driver consumes it; the VM driver's image acquisition
77+
and ext4 materialization pipeline is the same boundary shape, and converging
78+
it onto the shared types is tracked follow-up work.
7979

8080
## Runtime Summary
8181

@@ -85,12 +85,14 @@ tracked follow-up work.
8585
| Podman | Rootless or single-machine deployments. | Container plus nested sandbox namespace. | Uses the Podman REST API, OCI image volumes, and CDI GPU devices when available. |
8686
| Kubernetes | Cluster deployment through Helm. | Pod plus nested sandbox namespace. | Uses Kubernetes API objects, service accounts, secrets, PVC-backed workspace storage, and GPU resources. |
8787
| VM | Experimental microVM isolation. | Per-sandbox libkrun VM. | Managed endpoint-backed driver. The gateway spawns `openshell-driver-vm`, waits for its Unix socket, and then consumes it through the same remote `compute_driver.proto` path used by unmanaged endpoint drivers. The VM driver boots a cached bootstrap `rootfs.ext4`, prepares requested OCI images inside a bootstrap VM with `umoci`, attaches the prepared image disk read-only, and gives each sandbox a writable `overlay.ext4` for merged-root changes and runtime material. The driver persists each accepted launch request beside the overlay and restarts those VMs on driver startup without recreating the overlay. |
88-
| Extension | Out-of-tree drivers operated alongside the gateway. | Whatever boundary the driver implements. | Selected by a non-reserved custom `compute_drivers = ["<name>"]` entry with `[openshell.drivers.<name>].socket_path`, or at launch time by pairing `--drivers <name>` with `--compute-driver-socket=<path>`. Reserved built-in names such as `vm`, `docker`, `podman`, and `kubernetes` cannot be used as unmanaged socket endpoints. The gateway connects to a UDS the operator already provisioned, runs `GetCapabilities`, logs the advertised `driver_name`, and dispatches all sandbox lifecycle calls through `compute_driver.proto`. The driver process and socket lifecycle are operator-owned; the gateway does not spawn, supervise, or remove unmanaged extension drivers. The trust boundary is the socket's filesystem permissions: the operator must ensure only the gateway uid can read/write it. |
88+
| OCI | Early/opt-in kernel-primitive isolation against an existing system `containerd`. | Container built from Linux namespaces + cgroups v2. | In-process driver, like Docker/Podman/Kubernetes. The driver itself — not containerd — spawns the configured `runtime_binary` (`runc` by default, `crun`, or another OCI-runtime-spec-compatible binary) directly through its `create`/`start`/`state`/`delete` CLI contract; containerd is used only for image pull/unpack and snapshot management — consumed through the `openshell-rootfs` provider boundary (see Rootfs Provisioning Boundary above), which protects each sandbox's snapshot from containerd's GC via a lease since no containerd `Container`/`Task` is ever created — and never bundled. Sandboxes are tracked by scanning the driver's own state directory rather than querying containerd or the runtime for a global list. Per-sandbox network namespace + veth pair + nftables ruleset (shared `openshell-nft-ruleset` crate, also used by the VM driver). GPU support is kernel-device-node passthrough only. Rootless mode is implemented but not yet functional (defaults off) — see `crates/openshell-driver-oci/README.md`. |
89+
| Extension | Out-of-tree drivers operated alongside the gateway. | Whatever boundary the driver implements. | Selected by a non-reserved custom `compute_drivers = ["<name>"]` entry with `[openshell.drivers.<name>].socket_path`, or at launch time by pairing `--drivers <name>` with `--compute-driver-socket=<path>`. Reserved built-in names such as `vm`, `docker`, `podman`, `kubernetes`, and `oci` cannot be used as unmanaged socket endpoints. The gateway connects to a UDS the operator already provisioned, runs `GetCapabilities`, logs the advertised `driver_name`, and dispatches all sandbox lifecycle calls through `compute_driver.proto`. The driver process and socket lifecycle are operator-owned; the gateway does not spawn, supervise, or remove unmanaged extension drivers. The trust boundary is the socket's filesystem permissions: the operator must ensure only the gateway uid can read/write it. |
8990

9091
Per-sandbox CPU and memory values currently enter the driver layer through
9192
template resource limits. Docker and Podman apply them as runtime limits.
9293
Kubernetes mirrors each limit into the matching request. VM accepts the fields
93-
but currently ignores them.
94+
but currently ignores them. The OCI driver applies them as cgroup v2 CPU quota/period
95+
and memory limits in the generated OCI spec.
9496

9597
Docker and Podman also accept per-sandbox driver-config mounts for existing
9698
runtime-managed named volumes and tmpfs mounts. Podman additionally accepts
@@ -123,6 +125,7 @@ Runtime-specific implementation notes belong in the driver crate README:
123125
- `crates/openshell-driver-podman/README.md`
124126
- `crates/openshell-driver-kubernetes/README.md`
125127
- `crates/openshell-driver-vm/README.md`
128+
- `crates/openshell-driver-oci/README.md`
126129

127130
The combined VM topology runs `openshell-sandbox` as guest PID 1. libkrun
128131
executes the driver-owned guest bootstrap as PID 1, and the bootstrap preserves
@@ -138,6 +141,7 @@ The supervisor must be available inside each sandbox workload:
138141
| Podman | Read-only OCI image volume containing the supervisor binary. |
139142
| Kubernetes | Supervisor image side-loaded into the sandbox pod by image volume or init container. |
140143
| VM | Embedded in the guest rootfs bundle. |
144+
| OCI | Bind-mounted from a host path (`supervisor_binary_path`), not sourced from an image like the other in-tree drivers — see the driver README's Known Gaps. |
141145
| Extension | Defined by the out-of-tree driver. |
142146

143147
Driver-controlled environment variables must override sandbox image or template

‎crates/openshell-core/src/config.rs‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@ pub enum ComputeDriverKind {
8787
Vm,
8888
Docker,
8989
Podman,
90+
Oci,
9091
}
9192

9293
impl ComputeDriverKind {
@@ -97,6 +98,7 @@ impl ComputeDriverKind {
9798
Self::Vm => "vm",
9899
Self::Docker => "docker",
99100
Self::Podman => "podman",
101+
Self::Oci => "oci",
100102
}
101103
}
102104
}
@@ -137,8 +139,9 @@ impl FromStr for ComputeDriverKind {
137139
"vm" => Ok(Self::Vm),
138140
"docker" => Ok(Self::Docker),
139141
"podman" => Ok(Self::Podman),
142+
"oci" => Ok(Self::Oci),
140143
other => Err(format!(
141-
"unsupported compute driver '{other}'. expected one of: kubernetes, vm, docker, podman"
144+
"unsupported compute driver '{other}'. expected one of: kubernetes, vm, docker, podman, oci"
142145
)),
143146
}
144147
}
@@ -1012,6 +1015,10 @@ mod tests {
10121015
"docker".parse::<ComputeDriverKind>().unwrap(),
10131016
ComputeDriverKind::Docker
10141017
);
1018+
assert_eq!(
1019+
"oci".parse::<ComputeDriverKind>().unwrap(),
1020+
ComputeDriverKind::Oci
1021+
);
10151022
}
10161023

10171024
#[test]

‎crates/openshell-core/src/telemetry.rs‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,7 @@ pub enum TelemetryComputeDriver {
161161
Kubernetes,
162162
Podman,
163163
Vm,
164+
Oci,
164165
Unknown,
165166
}
166167

@@ -172,6 +173,7 @@ impl TelemetryComputeDriver {
172173
Self::Kubernetes => "kubernetes",
173174
Self::Podman => "podman",
174175
Self::Vm => "vm",
176+
Self::Oci => "oci",
175177
Self::Unknown => "unknown",
176178
}
177179
}
@@ -183,6 +185,7 @@ impl TelemetryComputeDriver {
183185
"k8s" | "kubernetes" => Self::Kubernetes,
184186
"podman" => Self::Podman,
185187
"vm" => Self::Vm,
188+
"oci" => Self::Oci,
186189
_ => Self::Unknown,
187190
}
188191
}
@@ -194,6 +197,7 @@ impl TelemetryComputeDriver {
194197
Some(crate::ComputeDriverKind::Kubernetes) => Self::Kubernetes,
195198
Some(crate::ComputeDriverKind::Podman) => Self::Podman,
196199
Some(crate::ComputeDriverKind::Vm) => Self::Vm,
200+
Some(crate::ComputeDriverKind::Oci) => Self::Oci,
197201
None => Self::Unknown,
198202
}
199203
}
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
2+
# SPDX-License-Identifier: Apache-2.0
3+
4+
[package]
5+
name = "openshell-driver-oci"
6+
description = "OCI/runc-based compute driver for OpenShell, backed by a system-provided containerd"
7+
version.workspace = true
8+
edition.workspace = true
9+
rust-version.workspace = true
10+
license.workspace = true
11+
repository.workspace = true
12+
13+
[lib]
14+
name = "openshell_driver_oci"
15+
path = "src/lib.rs"
16+
17+
[[bin]]
18+
name = "openshell-driver-oci"
19+
path = "src/main.rs"
20+
21+
[dependencies]
22+
openshell-core = { path = "../openshell-core", default-features = false }
23+
openshell-nft-ruleset = { path = "../openshell-nft-ruleset" }
24+
openshell-policy = { path = "../openshell-policy" }
25+
# Rootfs provider boundary. The containerd-backed provider lives there:
26+
# this driver never talks to containerd directly (no containerd-client
27+
# dependency here) and consumes only provider-neutral PreparedRootfs
28+
# mounts. The driver itself, not containerd, spawns the configured
29+
# low-level OCI runtime (runc/crun/etc.) directly (see src/runtime.rs).
30+
openshell-rootfs = { path = "../openshell-rootfs" }
31+
32+
# OCI runtime-spec types (namespaces, cgroups v2 resources, seccomp).
33+
oci-spec = { version = "0.10", default-features = false, features = [
34+
"runtime",
35+
] }
36+
37+
tokio = { workspace = true }
38+
tokio-stream = { workspace = true }
39+
tonic = { workspace = true, features = ["transport"] }
40+
prost = { workspace = true }
41+
prost-types = { workspace = true }
42+
futures = { workspace = true }
43+
clap = { workspace = true }
44+
tracing = { workspace = true }
45+
tracing-subscriber = { workspace = true }
46+
miette = { workspace = true }
47+
serde = { workspace = true }
48+
serde_json = { workspace = true }
49+
nix = { workspace = true }
50+
rand = { workspace = true }
51+
thiserror = { workspace = true }
52+
sha2 = "0.10"
53+
54+
[dev-dependencies]
55+
tempfile = "3"
56+
57+
[lints]
58+
workspace = true

0 commit comments

Comments
 (0)