Memory resource provisioning is one of the biggest challenges for IT administrators and virtualization designers. Although the provision of CPU is the major factor in the virtual infrastructure, most of the time there is lesser attention to CPU, because many applications that are deployed in virtualization need more memory, not the processor. And also today CPU technologies are very powerful, but applications like SQL and Oracle for some of their processes need more memory. Lack of enough memory resources may restrict the development of many virtual infra, so what should we do in this situation, before providing more physical resources for our hypervisors?
There are many technologies to handle the problem of memory over-commitment such as Swapping and Ballooning and VMware ESXi use some of them to confronting these issues. in this post I will review act and importance both mentioned mechanism:
1. Virtual Memory Ballooning is just a memory reclaiming technique that lets the VMkernel retrieve idle memory pages. When the ESXi host has less than 6% free memory (actually >=6%), Ballooning will come into the ring to handle out of memory problem! If a VM has many idle pages, its host borrows them to use as the temporary overhead for the VMs with more memory demand because probably they have some memory-intensive processes.
When a virtual machine wants to release some of the old used page files dedicated from host physical memory, does not remove it exactly, just change the address space pointer of the allocated memory list to the free memory list. So VM with Balloon Driver (vmmemctl.sys) can decide which pages (idle pages) can be reclaimed back to the host (up to 65% of VM guest memory) and which ones are needed for itself yet (already used pages) without involving host in this decision procedure. Now time for inflating step to happen.
If you disable memory ballooning driver inside the guest OS, VM will not be aware of host memory state and amount of available or unused physical memory and hypervisor cannot understand how much memory can take it back for other VMs memory requests.
2. Host Swapping is another mechanism used in low memory status, but unlike the Ballooning, it does not relate to VM guest OS. Host Swapping (or vRAM) has occurred If the host has less than 2% free memory and needs to provide more memory resources for memory-intensive VMs, so hypervisors should have enough swap space. although swapping is a reliable way against memory over-commitment, but it may cause total performance degrading because the efficiency of generated swap files is highly depended on IOPS rate of datastore specified for swapping, So if we don't provision suitable Disks (Like SSD) for swap files, it can lead to low performance.
There is one crucial point: If Memory Compression could happen, there is no need for hypervisor swap procedure execution. (I will discus more about it later on another post)
Remember, ESXi Host's memory will stay in share state until total required memory for them (memory requests) is greater than available physical memory (over-commitment), So if we want to prevent this situation before ballooning activation, we need to do these jobs:
1. Estimate carefully about overall required memory and also provide at least 25% more memory in the design phase.
2. Use VMware DRS in cluster-level resources to distribute VMs between hosts and balance resource usage, then no need to worried about low-memory. It's recommended to provision equal memory for all of ESXi hosts.
3. Use Resource Pools, especially in cluster-level and reserved estimated memory for mission-critical application and high resource-demand VMs and ensure they access to enough resources. Also, reserved memory is protected against memory reclaiming, even contention happened.
4. Do Not disable Balloon Driver, because it's the best intelligent memory reclamation technique. but if you want to do that, you can do it via the vSphere Client setting (Remove
Thanks to frank Danneman for this post in the VMware blog about memory reclaiming, I will talk more about VMware memory management technologies in the next section. For more information, you can refer to below links:
There is one crucial point: If Memory Compression could happen, there is no need for hypervisor swap procedure execution. (I will discus more about it later on another post)
Remember, ESXi Host's memory will stay in share state until total required memory for them (memory requests) is greater than available physical memory (over-commitment), So if we want to prevent this situation before ballooning activation, we need to do these jobs:
1. Estimate carefully about overall required memory and also provide at least 25% more memory in the design phase.
2. Use VMware DRS in cluster-level resources to distribute VMs between hosts and balance resource usage, then no need to worried about low-memory. It's recommended to provision equal memory for all of ESXi hosts.
3. Use Resource Pools, especially in cluster-level and reserved estimated memory for mission-critical application and high resource-demand VMs and ensure they access to enough resources. Also, reserved memory is protected against memory reclaiming, even contention happened.
4. Do Not disable Balloon Driver, because it's the best intelligent memory reclamation technique. but if you want to do that, you can do it via the vSphere Client setting (Remove
sched.mem.maxmemctl entry from .vmx file) or Guest OS Windows Registry (Set value of HKLM\SYSTEM\CurrentControlSet\services\VMMEMCTL from 2 to 4)Thanks to frank Danneman for this post in the VMware blog about memory reclaiming, I will talk more about VMware memory management technologies in the next section. For more information, you can refer to below links:
