How KVM Works: VM Exits, EPT, and Why a Guest Is Scheduled Like a Process

A process that starts a virtual machine does not execute guest instructions the way it executes its own. When the guest is running, the CPU is in a special non-root mode. The host kernel only regains control when the hardware decides an exit is required. That boundary is what makes KVM possible without rewriting every guest instruction.

This article explains that contract: how the Linux kernel becomes a hypervisor, what a VM exit is, how memory translation works with EPT, and why the guest is still scheduled like any other process on the host.

Related reading: How Linux Namespaces and cgroups Work, How Virtual Memory Works, and How Linux CFS Works.

The contract

KVM turns the Linux kernel into a type-2 hypervisor. User space (almost always QEMU) creates a VM, allocates guest memory, and asks the kernel to run a virtual CPU. The kernel uses hardware virtualization extensions (Intel VT-x or AMD-V) so that guest code runs directly on the physical CPU until something forces an exit back to the host.

The guest sees its own kernel, its own page tables, and its own devices. From the host scheduler's point of view, the vCPU is just a thread that spends most of its time inside a long-running ioctl.

The QEMU / KVM split

QEMU runs in user space. It owns the device models and the memory backing for the guest. KVM is a kernel module. It owns the virtual CPU and the memory-translation machinery that the hardware needs.

The interface is a file descriptor. Opening /dev/kvm and issuing ioctls creates a VM, creates vCPUs, registers memory regions, and finally runs the vCPU. Most of the time KVM_RUN does not return. The thread is blocked in the kernel while the guest executes. It returns only when a VM exit occurs that KVM cannot handle entirely inside the kernel.

Hardware modes and the VMCS

Intel VT-x adds VMX root mode and VMX non-root mode. The host kernel runs in root mode. The guest runs in non-root mode. The Virtual Machine Control Structure (VMCS) holds the guest and host state that must be saved and restored, plus the controls that decide which events cause an exit.

Entering the guest is a VMLAUNCH or VMRESUME. Leaving it is a VM exit. On exit the hardware loads the host state from the VMCS and jumps to the host's exit handler.

What causes a VM exit

Ordinary computation and memory accesses that hit in both the guest page tables and the EPT stay inside non-root mode. Exits happen for events the hardware has been configured to trap: I/O port instructions for emulated devices, certain MSR accesses, CPUID, EPT violations, external interrupts the host wants to handle, halt, and hypercalls.

Each exit has a cost. The CPU must save and restore architectural state. Hardware features such as APICv, posted interrupts, and EPT exist to keep the exit rate down.

Memory virtualization: shadow paging versus EPT

A guest has its own page tables that translate guest virtual addresses to guest physical addresses. The host must still translate those to host physical addresses.

Shadow paging is the software approach: KVM builds page tables that map guest virtual addresses directly to host physical addresses and intercepts guest page-table writes. It works on older hardware but is expensive.

Extended Page Tables (EPT) is the hardware approach. The CPU walks two tables. The guest's own page tables do guest-virtual to guest-physical. The EPT (controlled by the host) does guest-physical to host-physical. Guest page-table updates no longer force exits. An access that misses in the EPT causes an EPT violation; KVM installs the mapping and re-enters. On modern processors this is the normal path.

The guest still believes it owns physical memory. It sees guest physical addresses. The real frames live wherever the host's allocator and the EPT say they live.

The guest is still a process

Each vCPU is a thread inside the QEMU process. When that thread is inside KVM_RUN the scheduler sees a runnable task executing in kernel mode. When an exit returns to user space, the thread becomes an ordinary user-mode task again and can be preempted or migrated like any other process.

Host NUMA placement, cgroup limits, and the scheduler all apply to the QEMU threads. This is why a container and a VM are not the same thing. A container shares the host kernel and the host page tables. A KVM guest has its own kernel, its own page tables, and a hardware-enforced boundary that only the hypervisor can cross. The cost of that boundary is the VM exit.

Misconceptions

KVM is not a full hypervisor by itself. It is the CPU and memory virtualization engine. Device emulation lives in QEMU.

Not every guest instruction traps. Ordinary arithmetic and most memory accesses do not.

The guest does not see real physical addresses. It sees guest physical addresses.

VM exits are not the same as system calls. A system call is a deliberate request inside one operating system. A VM exit is a hardware-forced transition from non-root to root mode.

Takeaways

KVM makes the Linux kernel a hypervisor by using hardware virtualization extensions. Guest code runs directly until a configured event forces a VM exit. Memory translation is two-dimensional: the guest's page tables plus the host's EPT (or a software shadow). From the host's perspective each vCPU is still a schedulable thread; most of its CPU time is spent inside a long kernel ioctl that the hardware itself is executing.

The exit is the price of isolation. Hardware features exist to make that price as small as possible for the common case.