Quick boot and hba driver updates

Background
vSphere 6.7 introduced a new feature called Quick Boot that reboots ESXi of certain server models without going into POST. This can save quite a bit of time as many servers can spend several minutes during POST. In clusters with many servers this means we can save hours or even days during a year of updates and reboots.

Problem
When using vSAN it's very important that you follow the HCL in regards to using supported firmware+driver versions. We had a newly installed vSAN cluster with servers in a non-supported state. Because of this we used Update Manager to deploy a supported driver. The driver was installed, but the server never came up again. When we checked the console of the first host we saw the error message: "LoadESX in progress".

If we reset the power of the server it would complete POST and boot up with the new driver installed.

Solution
Disabling Quick Boot (from the Update Manager configuration) solved this problem and we were able to get all servers in our cluster up to date without any problems.

"Status of other host hardware objects" on HPE Gen10 servers

Problem
On HPE Gen10 servers we have observed several error messages in hardware monitoring and VMware vCenter reports these as a critical problem.

This problem has been reported on both vSphere 6.5 and 6.7.

Solution
Update 21Nov2018: This issue has been fixed in ILO version 1.37: https://vmoller.dk/index.php/2018/11/17/lom-warning-in-vmware-on-hpe-gen10-servers/

It turned out that the servers that had this problem were using ILO version 1.30 and 1.35 while the servers that did not have this problem were using ILO 1.20. After downgrading ILO to version 1.20 the problem was resolved. Hopefully this will be fixed in a future version.


Downgrading ILO is a new feature of ILO 5:

Note that while you can upgrade ILO directly with the Update Firmware functionality, you can't downgrade it the same way. In order to be able to downgrade ILO you must upload it to the ILO Repository first. Once it has been uploaded you can downgrade the ILO firmware.

After having downgraded ILO you will need to go into each of the ESXi host's hardware status and press the reset sensors button and everything will be fine.