Problem solve Get help with specific problems with your technologies, process and projects.

How many virtual machines on a Hyper-V server?

Virtualization has changed how admins think about data center capacity – but there are still limits, and consequences for hitting them.

Virtual administrators are regularly faced with the question of, how many? For the sake of solid forecasting in...

your data center as well as your budget, someone always wants to know how many virtual machines can be consolidated to a physical machine. The answer is always the same: it depends.

It was certainly simpler to come up with the number in the days of physical boxes. When faced with an application, you would propose a configuration per application that usually included more power than necessary to account for growth. Now that space is shared with other virtual machines, the appropriate number depends on the types of machines, their workloads, and their requirements for RAM, storage, and CPUs. The overhead of managing complex, multi-threaded workloads can also affect the final number. With virtualization, it’s all about efficient use of resources.

When considering virtual machines, it’s still a good idea to start with your processor needs. Although many of even the most basic machines sold today come with multiple processor cores, this doesn’t mean your server needs them. You can assign up to four virtual processors to a virtual machine depending on the operating system, but the effort to sync multiple physical processor cores to service a multiple-CPU virtual machine configuration does contribute to overhead. The overhead increases as integration decreases. Cross-processor communication is much less of an issue with Windows Server 2008 R2 than with a Windows 2000 Server virtual machine. You can tailor the number of virtual processors to give the virtual machine only the resources it needs.  

There is a practical limit to how many virtual processors can run on a single server. Microsoft Hyper-V Server R2 will support up to 512 virtual CPUs per physical host. That’s a big number, but that doesn’t mean you can hit this number without consequences. Different software vendors will give you different stories as to their virtual CPU requirements, but in the end you should be thinking about how many virtual CPUs will run on each logical CPU as a ratio. Unlike some virtualization platforms, this does not refer to cores, but cores and threads. A shining example of this is hyper-threading (HT) technology present in most Intel server chips. Hyper-Threading is the reason Windows reports eight CPUs when there are physically only four CPU cores. In this situation, you can count the ability to execute two threads in a single clock cycle as Hyper-Threading allows you to do. In this case, you’ll usually double up against the actual physical cores present in the server.

So how can you estimate your CPU needs? Common recommendations are to allocate a single logical processor to eight virtual processors for servers, and 12 virtual processors to each logical processor for virtualized desktops like Windows 7 virtual machines. In a Hyper-V host running two six-core HT processors and server-class virtual machines each running a single virtual processor, you would likely limit your number of virtual machines to 192.

The performance of current generation server processors has jumped significantly for virtualization. Holding up to six cores per socket and other optimizations like increased second- and third-level cache, along with faster buses, today’s servers can handle more than ever. At a recent conference, Microsoft published that they hosted a lab that contained 225 virtual machines per server blade, which had 128 GB of RAM and two Intel Xeon 5600 processors with six cores per socket in each host. That density didn’t even touch the capacity of the CPU and memory for their workload, with CPU utilization generally running at 15%.

This is fine when talking basic servers, but what if the server is running a CPU intensive load? You can obviously assign additional CPU cores to a virtual machine. A SQL Server, for example, would not only need processors, but plenty of RAM. In this case, you may run into the memory allocation limit of 64 GB to a single virtual machine. Even though it’s popular to push virtual machines onto slim, two-socket hardware, Hyper-V does support up to 64 logical processors in the host, making scale-up solutions a possibility for your CPU-hungry applications.

Also, when considering the upper limits of virtual machines, if you are clustering Hyper-V hosts the maximum number of virtual machines you can run on a host stays at 384, but the total number of virtual machines for the cluster is limited to 1,000. Also, when dedicating big memory to your virtual machines, remember that the total memory available to the Hyper-V host is 1 TB, so plan your virtual machine distribution accordingly.

There are some changes coming with the newly announced Windows Server 8. Although nothing is finalized as of this article, the upgrade to Hyper-V will be expected to bump the virtual processor limit from four to 16 CPUs as well as give you the ability to assign specific workloads to specific cores. If you are planning long term, and you have processor-intensive workloads, you may want to consider how this will affect your virtual machine distribution a year from now.

Follow SearchWindowsServer on Twitter @WindowsTT.

Eric Beehler
 has been working in the IT industry since the mid-'90s, and has been playing with computer technology since well before that. He currently provides consulting and training through his co-ownership in Consortio Services, LLC.

Dig Deeper on Microsoft Hyper-V management