diff --git a/CLAUDE.md b/CLAUDE.md index 26b75d1..d6b63e1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1089,15 +1089,15 @@ occurrence wins (`crashkernel=`) takes the user's value. `root=`, ltvm's values, and rc.local configures the guest from `fc_*`. Fixed at create time like the disks; `cluster create` applies it to every node. -**Nested KVM on Apple Silicon:** a Linux host on an Apple CPU (`CPU -implementer : 0x61`) is a VM under Apple Virtualization -- UTM, M3 or -later, nested virtualization on -- and `qemu_machine_for_arch` gives its -KVM guests `its=off` (`vm_state._nested_kvm_on_apple`). With the ITS, -virtio-pci MSIs lose wakeups there and the guest idles in `cpu_do_idle` -forever a few seconds after init; wired INTx works. It changes nothing -on any other host, macOS's HVF included. KVM is 3-40x faster than TCG -there; the benchmarks, against TCG and against ltvm on the Mac itself, -are in [docs/NESTED_VIRTUALIZATION.md](docs/NESTED_VIRTUALIZATION.md). +**Nested KVM on Apple Silicon:** on macOS, `qemu_run` checks the host's +EL2 capability and the selected QEMU binary at boot. Native Arm HVF +guests get EL2 on macOS 15+ with compatible hardware and QEMU 11.1+. +`create --disable-nesting` saves `DISABLE_NESTING=1` in the VM's +`.info` file. Inside an Apple-hosted Linux VM, `qemu_machine_for_arch` +gives KVM guests `its=off` (`vm_state._nested_kvm_on_apple`). This +uses wired INTx to avoid a virtio interrupt hang observed under +UTM's Apple Virtualization backend. See the setup and benchmarks in +[docs/NESTED_VIRTUALIZATION.md](docs/NESTED_VIRTUALIZATION.md). **Naming:** `-`, where `` is the first two consonants of the host's short name (`bl` on buildhost, `ts` on testnode) and N is unused diff --git a/README.md b/README.md index 281d0d6..4b1a271 100644 --- a/README.md +++ b/README.md @@ -71,10 +71,11 @@ a podman-managed Linux VM + QEMU microvms. A few things differ from Linux: ends), and `export LTVM_PODMAN_AUTOSTOP=0` keeps it up for good. A machine that lists as running but does not answer -- a failed start leaves it so -- is restarted by the next build. -- **To confine an agent to a VM**, run ltvm inside an aarch64 Linux VM - on the Mac instead: UTM on the Apple Virtualization backend with - nested virtualization on, which needs an M3 or later and macOS 15+. - ltvm then gets KVM there; see +- **Native Arm guests can run KVM by default** on a Mac with hardware + EL2 support, macOS 15+, and QEMU 11.1+ (included in `./ltvm install`). + Pass `--disable-nesting` at create time to opt out. To confine an agent + to a Linux VM, run ltvm inside that guest or inside a UTM Linux guest + with nested virtualization enabled on the Apple Virtualization backend. See [docs/NESTED_VIRTUALIZATION.md](docs/NESTED_VIRTUALIZATION.md#apple-silicon----aarch64). ```bash diff --git a/docs/NESTED_VIRTUALIZATION.md b/docs/NESTED_VIRTUALIZATION.md index 055a033..70b4cde 100644 --- a/docs/NESTED_VIRTUALIZATION.md +++ b/docs/NESTED_VIRTUALIZATION.md @@ -153,9 +153,36 @@ gcloud compute instances update \ ### Apple Silicon -- aarch64 -ltvm runs on macOS itself, with HVF. To run it inside a Linux VM on -the Mac instead -- so an agent driving ltvm is confined to that VM -- -the Linux VM needs nested virtualization, and that needs all of: +#### Expose KVM in a VM created by ltvm + +ltvm exposes Arm EL2 to native aarch64 guests by default on compatible +Apple Silicon (M3 or later) with macOS 15+ and QEMU 11.1+. A Linux guest +kernel with KVM enabled then provides `/dev/kvm`. On a capable Mac, +`ltvm doctor` flags an older QEMU. `ltvm doctor --fix` or +`ltvm install --qemu` installs the signed Apple Silicon QEMU bundle +from [ltvm's GitHub releases](https://github.com/lustre-tools/lustre-test-vms/releases). + +```bash +ltvm create nested-host --target rocky10 --arch aarch64 \ + --mem 6144 --vcpus 4 --root-size 16G --mdt-disks 0 --ost-disks 0 +ltvm vm exec nested-host -- ls -l /dev/kvm +ltvm vm exec nested-host -- dmesg | grep -E 'started at EL2|kvm \[' +``` + +ltvm checks support at each cold boot, including for existing VMs. +On an older host or QEMU, the VM boots with ordinary HVF. To opt out, +pass `--disable-nesting` to `ltvm create` or `ltvm cluster create`. +ltvm saves the opt-out for each VM. QEMU suppresses Arm SME when it +exposes EL2 through HVF, so opt out for workloads that need SME. + +The example allocates space for inner VMs and their images. +For an inner QEMU, use +`-accel kvm -cpu host -machine virt,gic-version=max,its=off`. + +#### Run ltvm inside a Linux VM on the Mac + +Use the Linux guest above, or use UTM to create the outer VM. For the +UTM path, use: - an **M3 or later** chip (M1 and M2 have no nested virtualization); - **macOS 15** or later; @@ -176,13 +203,13 @@ sudo usermod -aG kvm $USER # if you are not in the kvm it: `accel=kvm` and `-cpu host`. `ps -o args -C qemu-system-aarch64` shows what a running VM got. -**ltvm turns off the GIC ITS here.** On an Apple CPU (`CPU +**ltvm turns off the GIC ITS for the inner VM.** On an Apple CPU (`CPU implementer : 0x61` in `/proc/cpuinfo`), ltvm launches KVM guests with -`virt,...,its=off`. With the ITS, virtio-pci MSIs lose wakeups under -this nesting: the guest boots past init, then sits in `cpu_do_idle` -for good while its vCPU thread shows 100% CPU. Without the ITS, -virtio falls back to wired INTx interrupts, which work. Trapping WFI -in the L1 KVM does not help, so it is the MSI delivery itself. +`virt,...,its=off`. This avoids a hang observed under UTM's Apple +Virtualization backend. With the ITS, virtio-pci MSIs lose wakeups and +the guest sits in `cpu_do_idle` while its vCPU thread shows 100% CPU. +Without the ITS, virtio falls back to wired INTx interrupts. +Trapping WFI in the L1 KVM does not help. `LTVM_FORCE_TCG=1` still forces emulation, if you ever need it. Parallels Desktop and VMware Fusion have not been tested with ltvm. diff --git a/ltvm b/ltvm index 6c2127c..2a060c4 100755 --- a/ltvm +++ b/ltvm @@ -1595,7 +1595,16 @@ def build_parser() -> argparse.ArgumentParser: target_parser.set_defaults(func=_target_no_action) # -- VM lifecycle: create -- - _vm_create_args = argparse.ArgumentParser(add_help=False) + _nesting_args = argparse.ArgumentParser(add_help=False) + _nesting_args.add_argument( + "--disable-nesting", + action="store_true", + help="Keep Arm EL2 unavailable to guests on macOS HVF. " + "Fixed at create time.", + ) + _vm_create_args = argparse.ArgumentParser( + add_help=False, parents=[_nesting_args] + ) _vm_create_args.add_argument( "--vcpus", type=int, default=2, help="vCPU count (default: 2)" ) @@ -2568,7 +2577,7 @@ def build_parser() -> argparse.ArgumentParser: ccp = cluster_sub.add_parser( "create", - parents=[common], + parents=[common, _nesting_args], help="Create a cluster from node specs (needs root)", ) ccp.add_argument("name", help="Cluster name (e.g. co2)") diff --git a/ltvm_pkg/cli/cluster.py b/ltvm_pkg/cli/cluster.py index 6dfef1b..f5991e4 100644 --- a/ltvm_pkg/cli/cluster.py +++ b/ltvm_pkg/cli/cluster.py @@ -220,6 +220,7 @@ def cmd_cluster_create(args: argparse.Namespace) -> int: eth0_speed=args.eth0_speed, nic=list(args.nic or []), kernel_args=args.kernel_args, + disable_nesting=getattr(args, "disable_nesting", False), numa_nodes=args.numa_nodes, numa_dist=args.numa_dist, numa_dist_file=args.numa_dist_file, diff --git a/ltvm_pkg/qemu_run.py b/ltvm_pkg/qemu_run.py index bb1bd53..48b4852 100644 --- a/ltvm_pkg/qemu_run.py +++ b/ltvm_pkg/qemu_run.py @@ -7,6 +7,7 @@ import functools import math import os +import platform import pwd import re import secrets @@ -51,6 +52,40 @@ def die(msg: str, code: int = EXIT_ERROR) -> NoReturn: sys.exit(code) +def hvf_el2_supported() -> bool: + """Return whether this Apple Silicon host can expose Arm EL2.""" + if not is_macos() or platform.machine() != "arm64": + return False + + import ctypes + + try: + hvf = ctypes.CDLL( + "/System/Library/Frameworks/Hypervisor.framework/Hypervisor" + ) + query = hvf.hv_vm_config_get_el2_supported + except (OSError, AttributeError): + return False + query.argtypes = [ctypes.POINTER(ctypes.c_bool)] + query.restype = ctypes.c_int + supported = ctypes.c_bool() + return query(ctypes.byref(supported)) == 0 and supported.value + + +def qemu_supports_hvf_vgic(binary: str) -> bool: + """QEMU 11.1 introduced both the HVF platform GIC and EL2 support.""" + try: + result = run([binary, "--version"], check=True, timeout=5) + except (OSError, subprocess.SubprocessError): + return False + version = re.search(r"\bversion (\d+)\.(\d+)\.(\d+)\b", result.stdout) + return version is not None and tuple(map(int, version.groups())) >= ( + 11, + 1, + 0, + ) + + def _read_meminfo_mb(key: str) -> int: """Return /proc/meminfo's value in MiB, or 0 if unreadable. @@ -1294,11 +1329,23 @@ def _start_qemu(vm: VMInfo) -> None: f"{str(_nic)!r} on VM {vm.name!r}" ) - import platform as _platform - arch = vm.arch qemu_bin = qemu_binary_for_arch(arch) machine = qemu_machine_for_arch(arch) + accel_args = [] + if arch == "aarch64" and "accel=hvf" in machine: + macos_major = int(platform.mac_ver()[0].partition(".")[0] or 0) + if macos_major < 15: + if qemu_supports_hvf_vgic(qemu_bin): + # Newer QEMU defaults to a GIC that needs macOS 15. + machine = machine.replace(",accel=hvf", "") + accel_args = ["-accel", "hvf,kernel-irqchip=off"] + elif ( + not vm.disable_nesting + and hvf_el2_supported() + and qemu_supports_hvf_vgic(qemu_bin) + ): + machine += ",virtualization=on" # q35 (x86) and virt (aarch64) both have a PCI bus, so virtio # devices attach as virtio-*-pci. Previously x86 used microvm and @@ -1317,7 +1364,7 @@ def _start_qemu(vm: VMInfo) -> None: # those, so /sbin/init aborts with "Fatal glibc error: CPU does not # support x86-64-v2" and the kernel panics. Nehalem (Intel 2008) is # the baseline CPU model that satisfies v2 in full. - host_arch = _platform.machine() + host_arch = platform.machine() import os as _os _force_tcg = _os.environ.get("LTVM_FORCE_TCG") == "1" @@ -1345,6 +1392,7 @@ def _start_qemu(vm: VMInfo) -> None: qemu_bin, "-name", vm.name, + *accel_args, "-machine", machine, "-cpu", diff --git a/ltvm_pkg/vm_cluster.py b/ltvm_pkg/vm_cluster.py index 4080dd3..e7b8497 100644 --- a/ltvm_pkg/vm_cluster.py +++ b/ltvm_pkg/vm_cluster.py @@ -606,6 +606,7 @@ def _create_one_node( disk_cache: str = "", disk_prealloc: str = "", eth0_speed: int | None = None, + disable_nesting: bool = False, ) -> tuple[str, int, str]: """Create a single cluster VM via ltvm subprocess. @@ -679,6 +680,8 @@ def _create_one_node( if kernel_args: # One argv word, so a value starting with '-' is not an option. cmd += [f"--kernel-args={kernel_args}"] + if disable_nesting: + cmd.append("--disable-nesting") if numa_nodes: cmd += ["--numa-nodes", str(numa_nodes)] if numa_dist: @@ -880,6 +883,7 @@ def _print_cluster_plan( disk_dir: str = "", disk_cache: str = "", disk_prealloc: str = "", + disable_nesting: bool = False, ) -> None: """Report what `cluster create` would do, for --dry-run. @@ -930,6 +934,8 @@ def _print_cluster_plan( print(f" nics: eth0 (mgmt) + {', '.join(nics)}") if kernel_args: print(f" cmdline: + {kernel_args}") + if disable_nesting: + print(" nesting: disabled for every node (macOS HVF)") print(f" owner: {owner_id}") print("Nothing was written. Re-run without --dry-run to create it.") @@ -1140,6 +1146,7 @@ def cmd_cluster_create(args: argparse.Namespace) -> None: die(network_issue) nics = [str(n) for n in nic_specs] kernel_args = _validate_kernel_args(getattr(args, "kernel_args", None)) + disable_nesting = getattr(args, "disable_nesting", False) wait_seconds: int = getattr(args, "wait", 0) numa_nodes: int = getattr(args, "numa_nodes", 0) or 0 problem = nic_numa_problem(nic_specs, numa_nodes) @@ -1202,6 +1209,7 @@ def cmd_cluster_create(args: argparse.Namespace) -> None: nics=nics, kernel_args=kernel_args, owner_id=owner_id, + disable_nesting=disable_nesting, **disk_opts, ) return @@ -1256,6 +1264,7 @@ def create_wave(wave: list[ClusterNode]) -> list[str]: variant="base" if node_targets[node.name] else variant, share_disks=node.name in shared, eth0_speed=getattr(args, "eth0_speed", None), + disable_nesting=disable_nesting, **disk_opts, ): node for node in wave diff --git a/ltvm_pkg/vm_commands.py b/ltvm_pkg/vm_commands.py index bcb43fa..8dd8462 100644 --- a/ltvm_pkg/vm_commands.py +++ b/ltvm_pkg/vm_commands.py @@ -33,10 +33,12 @@ from .qemu_run import ( die, host_node_problem, + hvf_el2_supported, is_running, kill_qemu, launch_qemu, numa_problem, + qemu_supports_hvf_vgic, run, ) from .vm_net import ( @@ -85,6 +87,7 @@ drop_orphan_clusters, lustre_libdir, nic_ip6, + qemu_binary_for_arch, resolve_os_artifacts, ) @@ -708,6 +711,7 @@ def _validate_kernel_args(raw: str | None) -> str: "disk_size": "--disk-size", "root_size": "--root-size", "kernel_args": "--kernel-args", + "disable_nesting": "--disable-nesting", "numa_nodes": "--numa-nodes", "numa_dist": "--numa-dist", "disk_dir": "--disk-dir", @@ -950,6 +954,7 @@ def _allocate_and_persist_vm( os_id=os_id, kver=kver, arch=os_arts.arch, + disable_nesting=getattr(args, "disable_nesting", False), # Track the invoking human. SUDO_USER preserves the legacy # ``sudo ltvm create`` path; the unified privilege model runs # the command as the user and elevates only individual writes. @@ -1057,6 +1062,7 @@ def _print_create_plan( "target": os_target, "variant": variant, "arch": getattr(args, "arch", None) or os_arch_of(image), + "disable_nesting": getattr(args, "disable_nesting", False), "kernel": kernel, "kernel_version": kver, "image": image, @@ -1096,6 +1102,8 @@ def _print_create_plan( print(f" {kernel}") print(f" image: {image}") print(f" cpu/mem: {args.vcpus} vcpus, {args.mem} MB") + if plan["disable_nesting"]: + print(" nesting: disabled (macOS HVF)") if getattr(args, "numa_nodes", 0): print(f" numa: {args.numa_nodes} nodes") if getattr(args, "numa_dist", ""): @@ -4005,6 +4013,33 @@ def _unreadable_vm_taps(name: str) -> set[str]: return taps_for_vm(name, count) +def _check_macos_nesting_qemu(fix: bool) -> tuple[int, int]: + if not hvf_el2_supported(): + return 0, 0 + binary = qemu_binary_for_arch("aarch64") + if qemu_supports_hvf_vgic(binary): + return 0, 0 + + print( + f"nested virtualization needs QEMU 11.1 or newer at {binary}; " + "run `ltvm install --qemu` to install ltvm's macOS bundle from GitHub" + ) + if not fix: + return 1, 0 + + from .host_setup import install_qemu_macos_vhost + + try: + install_qemu_macos_vhost() + if not qemu_supports_hvf_vgic(qemu_binary_for_arch("aarch64")): + raise RuntimeError("selected QEMU is still missing or too old") + except (RuntimeError, OSError, subprocess.CalledProcessError) as exc: + print(f" FAILED to update macOS QEMU: {exc}") + return 1, 1 + print(" fixed: installed QEMU with nesting support") + return 1, 0 + + def cmd_doctor(args: argparse.Namespace) -> int: issues = 0 # Counts repairs that were attempted and did not work, so --fix @@ -4293,6 +4328,9 @@ def cmd_doctor(args: argparse.Namespace) -> int: if is_macos(): print("note: `ltvm target export` needs a Linux host; not checked") + found, failed = _check_macos_nesting_qemu(args.fix) + issues += found + fix_failures += failed else: from ltvm_pkg import image_export diff --git a/ltvm_pkg/vm_state.py b/ltvm_pkg/vm_state.py index 91ea091..e9182d3 100644 --- a/ltvm_pkg/vm_state.py +++ b/ltvm_pkg/vm_state.py @@ -136,9 +136,8 @@ def qemu_machine_for_arch(arch: str = "x86_64") -> str: def _nested_kvm_on_apple(cpuinfo: Path = Path("/proc/cpuinfo")) -> bool: """True when this Linux host runs on an Apple CPU (implementer 0x61). - On Apple Silicon /dev/kvm only exists inside a VM with nested - virtualization on -- UTM's Apple Virtualization backend on M3 or - later, macOS 15 or later -- so the KVM guests ltvm starts here are + On Apple Silicon /dev/kvm exists inside a VM with nested + virtualization enabled by the macOS hypervisor, so KVM guests here are nested ones. There, a guest whose virtio-pci MSIs go through the GICv3 ITS boots past init and then stops taking interrupts: the vCPU idles in WFI with nothing ever waking it (verified on an M3 @@ -619,6 +618,7 @@ class VMInfo: image_id: str = "" os_id: str = "" # OS identifier (e.g. rocky9, ubuntu24) arch: str = "x86_64" # CPU architecture (x86_64, aarch64) + disable_nesting: bool = False # Keep EL2 off under macOS HVF. creator: str = ( "" # username that created the VM (SUDO_USER, or "" for legacy) ) @@ -803,6 +803,7 @@ def save(self) -> None: f"IMAGE_ID={self.image_id}\n" f"OS_ID={self.os_id}\n" f"ARCH={self.arch}\n" + f"DISABLE_NESTING={int(self.disable_nesting)}\n" f"CREATOR={self.creator}\n" f"OWNER_ID={self.owner_id or ''}\n" f"VARIANT={self.variant}\n" @@ -1008,6 +1009,7 @@ def _int(key: str, default: int) -> int: image_id=vals.get("IMAGE_ID", ""), os_id=vals.get("OS_ID", ""), arch=vals.get("ARCH", "x86_64"), + disable_nesting=vals.get("DISABLE_NESTING") == "1", creator=vals.get("CREATOR", ""), owner_id=vals.get("OWNER_ID") or None, variant=vals.get("VARIANT", "base"), diff --git a/skills/ltvm/SKILL.md b/skills/ltvm/SKILL.md index f2797ad..a4fabe6 100644 --- a/skills/ltvm/SKILL.md +++ b/skills/ltvm/SKILL.md @@ -27,6 +27,15 @@ On macOS it is `./ltvm install` with no `sudo`: it refuses to run as root there and prompts for sudo itself where it must (README.md, "Running on macOS"). +Native aarch64 VMs on macOS expose Arm EL2 by default on capable +Apple Silicon (M3 or later), macOS 15+, and QEMU 11.1+. A Linux guest +with KVM enabled gets `/dev/kvm` and can run its own accelerated VMs. +Older setups boot with ordinary HVF. Pass `--disable-nesting` to +`ltvm create` or `ltvm cluster create` to save a per-VM opt-out. +On a capable Mac, `ltvm doctor` flags an older QEMU. Run +`ltvm doctor --fix` to install the current bundle from GitHub. +Use `ltvm vm exec -- ls -l /dev/kvm` to check the guest. + `install` also sets up tab completion for the human's shell (bash, zsh, fish), which needs a new shell before it works -- worth saying when you walk someone through setup, since nothing else announces it. `ltvm diff --git a/tests/test_cluster_cli.py b/tests/test_cluster_cli.py index d7d025d..0f259dd 100644 --- a/tests/test_cluster_cli.py +++ b/tests/test_cluster_cli.py @@ -428,6 +428,12 @@ def test_kernel_args_parsed(self) -> None: self._captured_ns().kernel_args == "slub_debug=FZPU page_owner=on" ) + def test_disable_nesting_parsed(self) -> None: + cmd_cluster( + _ns("create", "co1", "--disable-nesting", "mgs+mds:co1-mds:1") + ) + assert self._captured_ns().disable_nesting + def test_unknown_flag_errors(self) -> None: err = _expect_usage_error( "create", "co1", "mgs+mds:co1-mds:1", "--frobnicate" @@ -2315,6 +2321,15 @@ def test_role_match_invokes_execvp(self, tmp_sockets: Path) -> None: class TestClusterNodeDiskArgs: """`cluster create` must state disk counts explicitly.""" + def test_disable_nesting_reaches_each_node(self) -> None: + node = vm_cluster.parse_node_spec("oss:co9-oss:3") + with patch.object(vm_cluster, "_run_prefixed") as run: + run.return_value = MagicMock(returncode=0, stdout="", stderr="") + vm_cluster._create_one_node( + node, vcpus=2, mem=None, disable_nesting=True + ) + assert "--disable-nesting" in run.call_args.args[0] + def test_kernel_and_variant_reach_each_node(self) -> None: node = vm_cluster.parse_node_spec("oss:co9-oss:3") with patch.object(vm_cluster, "_run_prefixed") as run: diff --git a/tests/test_qemu_run.py b/tests/test_qemu_run.py index 093267f..8fb29a0 100644 --- a/tests/test_qemu_run.py +++ b/tests/test_qemu_run.py @@ -988,6 +988,51 @@ def _launch_macos(self, vm: VMInfo, h: _LaunchHarness) -> None: ): qemu_run.launch_qemu(vm) + @pytest.mark.parametrize( + ("macos", "el2", "qemu", "disabled", "want_el2", "want_userspace_gic"), + [ + ("26.6", True, True, False, True, False), + ("26.6", True, True, True, False, False), + ("26.6", True, False, False, False, False), + ("15.7", False, True, False, False, False), + ("14.8", False, True, False, False, True), + ("14.8", False, False, False, False, False), + ], + ) + def test_arm_hvf_nesting( + self, + tmp_vmdir: Path, + macos: str, + el2: bool, + qemu: bool, + disabled: bool, + want_el2: bool, + want_userspace_gic: bool, + ) -> None: + vm = _make_vm(tmp_vmdir, arch="aarch64") + vm.disable_nesting = disabled + h = _LaunchHarness() + with ( + patch("platform.system", return_value="Darwin"), + patch("platform.machine", return_value="arm64"), + patch("platform.mac_ver", return_value=(macos, (), "arm64")), + patch("ltvm_pkg.qemu_run.hvf_el2_supported", return_value=el2), + patch( + "ltvm_pkg.qemu_run.qemu_supports_hvf_vgic", return_value=qemu + ), + ): + self._launch_macos(vm, h) + args = h.qemu_args or [] + machine = args[args.index("-machine") + 1] + assert ("virtualization=on" in machine) is want_el2 + if want_userspace_gic: + assert args[args.index("-accel") + 1] == "hvf,kernel-irqchip=off" + assert machine == "virt,gic-version=max" + else: + assert "-accel" not in args + assert "accel=hvf" in machine + assert args[args.index("-cpu") + 1] == "host" + def test_mgmt_nic_uses_stream_unix(self, tmp_vmdir: Path) -> None: """-netdev stream,addr.type=unix,addr.path=,server=off.""" vm = _make_vm(tmp_vmdir) diff --git a/tests/test_vm_create_phases.py b/tests/test_vm_create_phases.py index 5061349..20130b8 100644 --- a/tests/test_vm_create_phases.py +++ b/tests/test_vm_create_phases.py @@ -234,6 +234,13 @@ def test_kernel_args_persisted_on_vm(self, tmp_vmdir: Path) -> None: ) assert VMInfo.load("co1-kargs").kernel_args == "slub_debug=FZPU" + def test_disable_nesting_persists(self, tmp_vmdir: Path) -> None: + with _create_env(tmp_vmdir): + vm_commands.cmd_create( + _create_args(name="co1-noel2", disable_nesting=True) + ) + assert VMInfo.load("co1-noel2").disable_nesting + def test_refused_kernel_args_write_nothing(self, tmp_vmdir: Path) -> None: with _create_env(tmp_vmdir) as env, pytest.raises(SystemExit): vm_commands.cmd_create( diff --git a/tests/test_vm_doctor.py b/tests/test_vm_doctor.py index 918e23d..07e1644 100644 --- a/tests/test_vm_doctor.py +++ b/tests/test_vm_doctor.py @@ -81,6 +81,7 @@ def doctor_env(tmp_vmdir: Path, tmp_path: Path) -> Iterator[dict]: "ltvm_pkg.vm_commands._check_completion", return_value=([], [], 0), ), + patch("ltvm_pkg.vm_commands.hvf_el2_supported", return_value=False), patch("ltvm_pkg.vm_commands.is_running", return_value=False), patch("shutil.disk_usage", return_value=plenty), ): @@ -98,6 +99,43 @@ def _make_args(fix: bool = False) -> argparse.Namespace: return argparse.Namespace(fix=fix, json=False) +@pytest.mark.parametrize( + ("fix", "supported_after_fix"), + [ + (False, False), + (True, True), + (True, False), + ], +) +def test_doctor_checks_and_repairs_selected_qemu_for_nesting( + doctor_env: dict, + capsys: pytest.CaptureFixture[str], + fix: bool, + supported_after_fix: bool, +) -> None: + binary = "/custom/qemu/bin/qemu-system-aarch64" + with ( + patch("ltvm_pkg.vm_commands.is_macos", return_value=True), + patch("ltvm_pkg.vm_commands.hvf_el2_supported", return_value=True), + patch("ltvm_pkg.vm_commands.qemu_binary_for_arch", return_value=binary), + patch( + "ltvm_pkg.vm_commands.qemu_supports_hvf_vgic", + side_effect=[False, supported_after_fix], + ) as check, + patch("ltvm_pkg.host_setup.install_qemu_macos_vhost") as install, + ): + rc = vm_commands.cmd_doctor(_make_args(fix=fix)) + output = capsys.readouterr().out + assert ( + f"nested virtualization needs QEMU 11.1 or newer at {binary}" in output + ) + assert install.call_count == int(fix) + assert check.call_count == 1 + int(fix) + assert all(call.args == (binary,) for call in check.call_args_list) + assert ("fixed: installed QEMU" in output) is supported_after_fix + assert (rc == 0) is supported_after_fix + + # ── clean ─────────────────────────────────────────────────