Skip to content

Commit e73162e

Browse files
committed
fix(docker): document CDI context mount
Signed-off-by: Evan Lezar <elezar@nvidia.com>
1 parent 93c9ace commit e73162e

4 files changed

Lines changed: 21 additions & 16 deletions

File tree

‎architecture/compute-runtimes.md‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -172,10 +172,11 @@ For all in-tree drivers, this is equivalent to selecting a single GPU.
172172
For Docker GPU sandboxes, the driver treats CDI specs as runtime metadata for
173173
both outer injection and inner sandbox policy. It selects opaque CDI device IDs,
174174
passes them to Docker, mounts daemon-reported CDI spec directories into
175-
supervisor-only paths, and uploads a versioned CDI context before starting the
176-
container. The supervisor resolves that context inside the sandbox and derives
177-
Landlock paths and supplemental groups from CDI `containerEdits`. Host-side CDI
178-
spec paths are diagnostic only and are never treated as sandbox policy paths.
175+
supervisor-only paths, and bind-mounts a gateway-owned versioned CDI context
176+
read-only before creating the container. The supervisor resolves that context
177+
inside the sandbox and derives Landlock paths and supplemental groups from CDI
178+
`containerEdits`. Host-side CDI spec paths are diagnostic only and are never
179+
treated as sandbox policy paths.
179180
Kubernetes must not infer CDI device IDs from the `nvidia.com/gpu` resource
180181
request; it needs a node-local selected-device handoff before using the same
181182
supervisor resolver.

‎crates/openshell-driver-docker/Cargo.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,10 +24,10 @@ serde_json = { workspace = true }
2424
prost-types = { workspace = true }
2525
bollard = { version = "0.20" }
2626
url = { workspace = true }
27-
tar = "0.4"
2827

2928
[dev-dependencies]
3029
prost-types = { workspace = true }
30+
tar = "0.4"
3131
temp-env = "0.3"
3232
tempfile = "3"
3333

‎crates/openshell-driver-docker/README.md‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -86,7 +86,7 @@ contract:
8686
| `restart_policy = unless-stopped` | Keeps managed sandboxes resumable across daemon or gateway restarts. |
8787
| `PidsLimit` | Enforces the sandbox PID budget at the Docker cgroup layer. Set `[openshell.drivers.docker].sandbox_pids_limit = 0` to inherit the Docker/runtime default. |
8888
| CDI GPU request | Uses opaque `driver_config.cdi_devices` values when set; otherwise selects the requested count of NVIDIA CDI GPUs in round-robin order when daemon CDI support is detected. Docker daemon `/info` can permit `nvidia.com/gpu=all` as a WSL2 all-only compatibility fallback, where it counts as one selectable device. Exact CDI device lists must not contain duplicates and must match the effective GPU count. |
89-
| CDI context upload | For GPU/CDI sandboxes only, mounts daemon-reported CDI spec directories read-only under `/run/openshell/supervisor/cdi-specs/<n>` and uploads `/run/openshell/supervisor/cdi-context.json` after container create and before start. |
89+
| CDI context mount | For GPU/CDI sandboxes only, creates a gateway-owned context file and bind-mounts it read-only at `/run/openshell/supervisor/cdi-context.json`; daemon-reported CDI spec directories are mounted read-only under `/run/openshell/supervisor/cdi-specs/<n>`. |
9090

9191
The agent child process does not retain these supervisor privileges.
9292

@@ -97,11 +97,13 @@ CDI device IDs from `driver_config.cdi_devices` or the daemon's discovered CDI
9797
inventory, then passes the same IDs to Docker with a CDI `DeviceRequest`.
9898

9999
When a GPU/CDI request is present, the driver also mounts the Docker
100-
daemon-reported `Info.CDISpecDirs` into supervisor-only paths and uploads a
101-
small versioned CDI context through Docker's container archive API. The context
102-
uses container-side spec paths for resolution and keeps host-side spec sources
103-
diagnostic-only. If the upload fails, the driver removes the created container
104-
and sandbox token file before reporting the failure.
100+
daemon-reported `Info.CDISpecDirs` into supervisor-only paths. Before container
101+
creation, it writes a small versioned CDI context in gateway-owned state and
102+
bind-mounts it read-only into the supervisor. The context uses container-side
103+
spec paths for resolution and keeps host-side spec sources diagnostic-only. If
104+
context or token creation fails, the driver removes any created state files; if
105+
container creation or start fails, it also removes the container and state
106+
files before reporting the failure.
105107

106108
The sandbox supervisor resolves the selected IDs from those mounted specs
107109
before it launches agent processes. CDI device nodes become read-write

‎docs/reference/sandbox-compute-drivers.mdx‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -148,11 +148,13 @@ For GPU-backed Docker sandboxes, configure Docker CDI before starting the gatewa
148148
For Docker GPU/CDI sandboxes, OpenShell uses Docker's selected CDI device IDs
149149
and daemon-reported CDI spec directories to build a supervisor-only CDI
150150
context. The driver mounts the spec directories read-only into the sandbox
151-
container, uploads `cdi-context.json` before starting the container, and removes
152-
the created container if that upload fails. The supervisor resolves the context
153-
inside the sandbox and derives the inner filesystem and supplemental group
154-
requirements from CDI specs. Non-GPU Docker sandboxes do not receive the CDI
155-
context, spec mounts, or CDI-derived policy changes.
151+
container. Before creation, it writes a gateway-owned `cdi-context.json` and
152+
bind-mounts it read-only into the supervisor. If context or token creation
153+
fails, the driver removes the created state files; if container creation or
154+
start fails, it also removes the container and state files. The supervisor
155+
resolves the context inside the sandbox and derives the inner filesystem and
156+
supplemental group requirements from CDI specs. Non-GPU Docker sandboxes do not
157+
receive the CDI context, spec mounts, or CDI-derived policy changes.
156158

157159
### Docker Driver Config Mounts
158160

0 commit comments

Comments
 (0)