-
Notifications
You must be signed in to change notification settings - Fork 0
EN_Virtualization
A summary of virtualization concepts for DevOps engineers. Click each link for full details.
- Virtualization: Technology that creates virtual versions of physical devices or resources in software
- Type 1 Hypervisor: Runs directly on bare-metal hardware (VMware ESXi, Microsoft Hyper-V)
- Type 2 Hypervisor: Runs on top of a host OS — used for dev/test (VirtualBox, VMware Workstation)
- Full Virtualization: Complete hardware emulation — guest OS requires no modification
- Para-Virtualization: Guest OS is aware of the hypervisor and communicates directly — better I/O performance
- Container Virtualization: Shares host OS kernel, isolates only the app and its dependencies — lighter than VMs (Docker, Kubernetes)
- RAID: Combines multiple physical disks into one logical unit — improves redundancy and performance
- LVM (Logical Volume Manager): Abstraction layer over physical storage — flexible resizing without physical constraints
- Docker: Container platform leveraging Linux cgroups, namespaces, and OverlayFS
- Dockerfile: Text file defining image build instructions (FROM, RUN, COPY, CMD, EXPOSE, ENV)
-
Docker Compose: Tool to define and run multi-container apps using
docker-compose.yml - Docker Hub: Public registry for Docker images
- cgroups: Limits CPU, memory, and I/O resources per process group
- Namespaces: Isolates PID, network, mount, and other resources per container
-
Docker-in-Docker (DinD): Running a separate Docker daemon (dockerd) inside a CI runner container to build images — the
docker:dindsidecar must run privileged so dockerd can mount cgroups -
Kaniko: Daemonless tool that builds images by executing a Dockerfile in userspace without a Docker daemon — runs non-privileged, making it CI/Kubernetes-friendly, with layer cache support (
--cache) -
BuildKit: Docker's next-generation build engine (parallel builds, cache mounts, multi-arch) — drivers:
docker(built-in),docker-container(on a local container / dind),kubernetes,remote -
buildx: The Docker CLI frontend that drives BuildKit (
docker buildx) — builds amd64+arm64 multi-arch manifest lists viabinfmt/QEMU
Covers the concept of abstracting physical resources, comparison with RAID and LVM, full vs para-virtualization, and the differences between host, container, and hypervisor virtualization with their use cases.
→ Details
Covers how Docker works (cgroups, namespaces, OverlayFS), Dockerfile syntax and the image build process, and how to define multi-container services with Docker Compose.
→ Details
Core problem: The CI runner is itself a container, but running docker build inside it needs a Docker daemon — three approaches solve this.
-
DinD: Run a
docker:dindsidecar in privileged mode. Because a real dockerd runs, you accept the privileged cost in exchange for full Docker parity — which is why it stays popular despite the security trade-off. Advantages:-
Full Docker compatibility: every Docker feature and command works as-is — buildx multi-arch,
docker compose, advanced BuildKit features, special builds. None of Kaniko's Dockerfile-instruction / feature gaps. -
Build-and-run in the same job: the image you just built can be launched right away with
docker runin the same job for integration / smoke tests (a daemonless builder cannot run images). -
Familiar workflow: the same
docker buildcommands and Dockerfile you use locally run unchanged in CI — low learning curve and local↔CI behavior parity. - Isolated daemon per job: each CI job gets its own dockerd sidecar, so the build environment is isolated and there is no daemon-state / cache contamination from other jobs.
- Daemon layer cache: dockerd's layer cache can be reused to speed up repeated builds (when a volume / cache volume is mounted).
Cons: privileged means host-kernel access (larger security blast radius), nested overhead, and fiddly daemon-cache management.
-
Full Docker compatibility: every Docker feature and command works as-is — buildx multi-arch,
-
Kaniko: Daemonless and non-privileged. Pros: security (no privileged needed), Kubernetes-friendly, layer cache. Cons: limited multi-arch and advanced BuildKit features, plus compatibility gaps on some Dockerfile instructions.
-
BuildKit + buildx: The next-generation builder. The
docker-containerdriver runs on a local container (e.g. dind); theremote/kubernetesdrivers connect to a separate BuildKit pod. Strong at multi-arch (binfmt), parallel builds, and cache mounts.
| Builder | Docker daemon | Privileged | Multi-arch | Cache | CI / K8s fit |
|---|---|---|---|---|---|
| DinD | Required (dockerd) | Required | Yes (via buildx) | Daemon cache | Works, but privileged |
| Kaniko | Not needed | Not needed | Limited | Layer cache (--cache) |
Strong |
| BuildKit (buildx) | Depends on driver | Only with dind driver | Yes (binfmt) | Cache mount | Strong (remote/k8s driver) |
Practical pattern: It is common to split by purpose — build application images with Kaniko (non-privileged), and build amd64+arm64 multi-arch shared images with dind + buildx (docker-container driver).