SR: Phase 4: Full restore with incremental container adoption

[Cameron E Baird (Camelron)]

Consume live and completed container accounting during the snapshot phase.
This tells us which containers were naturally completed
(init or ephemeral containers) and which are currently running
(workload, pause).

For live containers only, the restore path adopts the guest processes,
storing mappings for the new containerd-provided ctr ids to the
snapshotted instance's guest ids held by the agent. We then
refresh the mounts and resume the cgroup, which was paused during
snapshot.

For incoming CreateContainers for completed containers,
handle this synthetically. Respond with normal TaskStart and
then TaskExit events.

[Copilot]

Pull request overview

Adds full snapshot restoration with live-container adoption, synthetic completion handling, identity mapping, and recursive snapshot support.

Changes:

• Records live/completed container identities and persists restore state.
• Adopts restored guest processes and refreshes node-local mounts.
• Adds restored rootfs/device lifecycle and agent mount-preparation APIs.

Reviewed changes

Copilot reviewed 31 out of 32 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/runtime-rs/crates/runtimes/virt_container/src/sandbox.rs	Builds and restores expanded snapshot manifests.
src/runtime-rs/crates/runtimes/virt_container/src/sandbox_persist.rs	Persists restore state.
src/runtime-rs/crates/runtimes/virt_container/src/restore.rs	Implements adoption, identity hashing, and completion tracking.
src/runtime-rs/crates/runtimes/virt_container/src/container_manager/process.rs	Separates host/guest process identities.
src/runtime-rs/crates/runtimes/virt_container/src/container_manager/manager.rs	Handles adopted and synthetic containers.
src/runtime-rs/crates/runtimes/virt_container/src/container_manager/container.rs	Routes operations through guest IDs.
src/runtime-rs/crates/runtimes/virt_container/src/container_manager/container_inner.rs	Applies guest IDs to agent RPCs.
src/runtime-rs/crates/runtimes/virt_container/Cargo.toml	Adds SHA-256 support.
src/runtime-rs/crates/runtimes/src/manager.rs	Persists lifecycle transitions and emits events.
src/runtime-rs/crates/runtimes/common/src/types/mod.rs	Defines snapshot inventory types.
src/runtime-rs/crates/runtimes/common/src/sandbox.rs	Adds runtime-state persistence API.
src/runtime-rs/crates/runtimes/common/src/container_manager.rs	Adds synthetic completion API.
src/runtime-rs/crates/resource/src/volume/share_fs_volume.rs	Refreshes copied guest mounts.
src/runtime-rs/crates/resource/src/volume/mod.rs	Exposes snapshot mount metadata.
src/runtime-rs/crates/resource/src/rootfs/snapshot.rs	Rewrites recursive snapshot disk configuration.
src/runtime-rs/crates/resource/src/rootfs/restored_rootfs.rs	Implements restored rootfs lifecycle.
src/runtime-rs/crates/resource/src/rootfs/mod.rs	Registers and manages restored rootfs objects.
src/runtime-rs/crates/resource/src/manager.rs	Exposes restore resource operations.
src/runtime-rs/crates/resource/src/manager_inner.rs	Performs guest mount refreshes.
src/runtime-rs/crates/hypervisor/src/lib.rs	Adds restored-device removal API.
src/runtime-rs/crates/hypervisor/src/device/device_manager.rs	Delegates restored-device removal.
src/runtime-rs/crates/hypervisor/src/ch/mod.rs	Implements removal for Cloud Hypervisor.
src/runtime-rs/crates/hypervisor/src/ch/inner_hypervisor.rs	Documents paused restore/network behavior.
src/runtime-rs/crates/hypervisor/src/ch/inner_device.rs	Removes restored CLH devices by ID.
src/runtime-rs/crates/agent/src/types.rs	Defines guest-mount preparation request.
src/runtime-rs/crates/agent/src/lib.rs	Extends the agent interface.
src/runtime-rs/crates/agent/src/kata/trans.rs	Converts the new request type.
src/runtime-rs/crates/agent/src/kata/agent.rs	Registers the agent RPC.
src/libs/protocols/protos/agent.proto	Defines PrepareGuestMount.
src/agent/src/rpc.rs	Implements guest mount clearing.
src/agent/src/netlink.rs	Clarifies restored interface replacement.
Cargo.lock	Locks the SHA-256 dependency.

Suppressed comments (2)

src/runtime-rs/crates/runtimes/src/manager.rs:723

• delete_process has already removed and cleaned the task before this save runs. If persistence fails, Delete returns an error and suppresses TaskDelete, but a retry sees ContainerNotFound, leaving containerd unable to complete deletion. The durable state update and externally visible delete result need failure semantics that do not report a reversible failure after the irreversible mutation.

                sandbox
                    .persist_runtime_state()
                    .await
                    .context("persist process deletion")?;

src/runtime-rs/crates/runtimes/src/manager.rs:830

• This save can fail after the process was resumed (or synthetically completed) and after TaskStart was published. Returning a Start error then contradicts the already-observed event and may cause containerd to retry a running task. Persist before publishing success where possible, or treat a post-commit save failure as a fatal recovery condition rather than a failed Start RPC.

                    sandbox
                        .persist_runtime_state()
                        .await
                        .context("persist container start")?;

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Sumedh Alok Sharma (Sumynwa)]

nit: Use known global to construct this path.