How to check BIOS Power management settings of ESXi hosts

Background

The performance of your workload will possibly be greatly affected by the power saving settings of your hosts. There are power saving settings both in the vSphere client¹, in the BIOS of the hosts² and inside the VMs³. This can cause much confuzion and there are a number of articles related to this issue:
Select a CPU Power Management Policy
Performance Best Practices for VMware vSphere 6.7
Virtual machine application runs slower than expected in ESXi (1018206)
ARCHITECTING MICROSOFT SQL SERVER ON VMWARE VSPHERE®

The root cause of the problem is that servers are normally shipped with a BIOS setting of Balanced power saving. This means that C states are enabled in order to make the cpus sleep whenever they are idle.

You use the vSphere client to check the settings of your BIOS (ESXi host / Configure / Hardware - Power Management) and you can also configure how ESXi should treat power savings.

Note that in vSphere 7 this option has moved to ESXi host / Configure / Hardware - Overview - Power Management.

From the example above we can see P states are also enabled on this system. P states makes turbo mode work when something requires extra performance, but doesn't need all cores. Many systems tend however to come with only C states enabled. The information seen from the vSphere client does not reveal the level of C states that are configured.  C states does not always have a severe impact, but since all systems I have seen so far come with all C states enabled it will normally affect the performance if you see it in the vSphere client.

The Performance BP doc says the following:

The SQL BP doc says:

Both these documents agree on that disabling C states is the way to go (The OS Control mode setting in BIOS typically disables C states and enables P states). Earlier versions of this document have suggested disabling saving functionality completely. 

The document Best Practices for Performance Tuning of Latency-Sensitive Workloads in vSphere Virtual Machines also suggest to set "Power Management Mode to Maximum Performance" in BIOS, disabling Power Management completely.

The the good old .Net based vSphere client it was only possible to change the settings if either C or P states were available. In the HTML5 client you can set options here even if they are disabled, which doesn't really make much sense.

Many new servers now also come with a virtualization adapted predefined power scheme that you can choose in BIOS.

When you buy servers today it's also possible to specify to make this setting the default one and then all the servers will come correctly preconfigured.

Problem

Even if all your servers at one point had the BIOS settings set to Full Performance, you may at a later point see that not all servers perform equally good. I have seen that replacement of motherboards will normally lead to a Balanced power saving setting (and degraded performance).

Solution

With Powershell you can easily identify servers that has C or P states enabled.

Get-VMHost | Sort | Select Name,

    @{ N='HW Support';

       E={$_.ExtensionData.Hardware.CpuPowerManagementInfo.HardwareSupport}}

When you run it agains your clusters it will tell you if the ESXi hosts has any of these power states enabled in the BIOS:

C:\> get-biospowersettings.ps1

Name                          HW Support

----                          ----------

esxa001.mydomain.com          ACPI C-states

esxa002.mydomain.com          

esxb001.mydomain.com          ACPI P-states

esxb002.mydomain.com          ACPI P-states

esxc044.mydomain.com          ACPI C-states

esxc047.mydomain.com          ACPI C-states

As we can see from this output one of the host has no output. This means that it has power saving disabled in the BIOS. The ones with C states probably has a default setting of Balanced (gives poor performance) and the ones with P states have probably been manually configured to take advantage of cpu Turbo modes. For most workloads (and lowest latency) you will probably want to disable power saving in the BIOS and have a blank result here.