Virtualization Tip 2: Relation between physical & virtual Memory

Hello guys, I hope you started 2020 as the best year of your life. In the next part of virtualization tips about the resource management via the ESXi, I will deep dive into the Memory Management.
 We have two concepts of memory: physical memory & virtual memory that Both of them are block-based that are called page. In the case of physical memory is full (over-commit) the other data of virtual pages are stored on disk (called memory swapping). There is two types of memory virtualization: Software-based & Hardware-assisted.

 But before everything, let's look at two general concepts:

1. Regardless of virtual machine's required memory, the ESXi host toke a piece of physical memory for running its own daemons and agents, when the host is normally up and is ready to work. (and also a dynamic overhead memory for its code and data) Remaining total memory is available for assignment to all of the virtual machines via sharing, reserving or other memory allocation techniques.

2. There is an abstraction layer for the memory allocation between the ESXi host and its associated VMs. Hypervisor do it without the knowledge or any required management activities of the guest OS of VM. To improve memory utilization, the ESXi host transfers memory from idle VMs to VMs that need more memory.

For better understanding of memory allocation in a virtualized environment, I should explain about three main definitions:

1. Machine Page Number(MPN): Physical memory resources that is available inside the ESXi. (host physical memory)
2. Physical Page Number(PPN): Assigned memory from available physical resources of the ESXi host to the virtual machines.(VM physical memory)
3. Virtual Page Number(VPN): Guest OS virtual memory.

  Hypervisor (VMM) is the responsible for translating / mapping PPN (physical) to MPN (machine) to provide memory for each VM. The VMM maintains the combined virtual-to-machine page mappings in the shadow page tables, so the PPN to MPN mappings maintained by the VMM. But for both of memory virtualization types the Guest OS of each VM wil map VPN to PPN. Then the  tables are kept up to date with the VPN to PPN mapping via the guest OS. (No need to more computation for maintaining the coherency of the shadow page tables) Also there is a good point in this method of memory allocation: The server can remap a physical page by changing its PPN2MPN mapping, in a manner that is completely transparent to the VM.

 The TLB (called address translation cache) stores the recent translations of VPN2PPN. The hardware TLB permits ordinary memory references to execute without additional overhead, so it will directly cache the VPN2MPN read from the shadow page table. In the software-based method, there is a combination of VPN2MPN as a software technique that saves them in the shadow page tables managed by the hypervisor. But in other method, as VMware said hardware resources will be used to generate the combined mappings with the guest's page tables and the nested page tables maintained by the hypervisor.

 For the calculation of required memory in our datacenter, some important points about the memory allocation & assignment are exist that we need to be aware. Let's review them:

1. An ESXi host will provide an extra layer of address translation on the underlay physical memory to grant memory access for each VM, because the Guest OS needs a zero-based physical address space at the memont of power-on operation, So basically the host virtualized its physical memory via this method. VMware also carefully defined that "The virtual memory of a VM is really a software abstraction layer to provide the illusion of hardware memory for that machine."

2. Consider a VM with configured 4GB RAM. Regardless of share, reservation and limit considerations, the guest OS behaves like it has absolutely a dedicated 4GB memory. But based on many factors include of memory over-head/over-commit and the level of memory pressure on the ESXi host, it might not really use exact the configured size of vSphere Documentarymemory. 

3. The software MMU has a higher overhead memory requirement than hardware MMU. Sometimes software memory virtualization may have some performance benefit over hardware-assisted approach if the workload induces a huge amount of TLB misses. But the performance of hardware MMU has improved since it was first introduced with extensive caching implemented in the hardware layer.

4. In the case of memory reservation you must remember if the VM becomes idle after awhile, the host does not reclaim its memory and VM can retain it. So the VMkernel can do the memory allocation for the other VMs (that are out of reserved memory capacity via the resource pools) from remained portion of its physical memory resources.

5. Transparent Page Sharing (TPS) is a wonderful technique for consuming less memory via the removing of memory page's redundant copies. So in compare with the traditional implementation of physical system, a workload running on a VM often consumes less memory. Just know it as the security approach, inter-VM TPS is disabled by default and page sharing is being restricted to intra-VM memory sharing. (Page sharing will occure only inside of VMs, not across them)

 Obviously before this, we could say that overall overhead of memory virtualization highly depends on the VMs & hosts workload. But you should understand each of the above definitions and tips before the design and calculation of required memory resources of the virtualization infrastructure. If you want to read more details about the memory virtualization and ESXi management mechanisms, please look at the vSphere Documentary about the Memory Virtualization