Caution when updating to ESXi 6.5 Update 3 HPE Custom

Windows Update[German]Brief note for users of HPE servers running VMware ESXi 6.5 Update 3 with a custom image. I just came across a problem report that there are problems with the custom image.


Advertising

ESXi 6.5 Update 3 HPE Custom

VMware ESXi is a bare-metal hypervisor, i.e. a minimal operating system that can be installed directly on your physical server. Then virtual machines can be installed on the server.

For HP servers (ProLiant), customized VMware ESXi-ISO files are offered for installation. These include all necessary drivers and management software to run ESXi on HPE servers. In addition, according to this HP announcement, this customized installation should work seamlessly with Intelligent Provisioning.

Warning, there are issues

It is currently only a single voice – I did not find any further hits during a short search. On administrator.de you can find this German article where a user describes his experiences.

I had an unpleasant encounter with the ESXI 6.5 Update 3 HPE Custom.

A clean install on two differently configured ML350 Gen10 will cause the management interface to stop responding about 2 minutes after boot-up. Neither via the WebUI, nor via the console via F2 (also not via Alt F1). You can only shut down or restart the server via F12.

Firmware is always "up-to-date" at the level of HP SPP 2019.03 (2019.06 has not yet been released). On 6.5 U2 HPE Custom the servers run normally.

Within my German blog post, other users also confirmed issues with ESXi 6.5 on HPE hardware. Anyone else who works with this constellation and has had similar experiences?

Similar articles:
VMware ESXi: Hosts crash during VM shutdown with PCI passthrough


Cookies helps to fund this blog: Cookie settings
Advertising


##1

This entry was posted in Virtualization, Windows and tagged . Bookmark the permalink.

9 Responses to Caution when updating to ESXi 6.5 Update 3 HPE Custom

  1. fr says:

    Had an issue which seems to match on a HPE gen9 server which was updated, I've had to roll it back for now.

  2. TT says:

    Exact same problem.
    HP Synergy 480 Gen10

    Currently have case open with HPE.

    During my investigation, I found that the vmkernel.log was giving warnings about the local USB (where the image is installed)

    2019-08-21T04:54:15.274Z cpu18:70978)WARNING: NFS: 1183: Invalid volume UUID mpx.vmhba32:C0:T0:L0:5
    2019-08-21T04:54:15.287Z cpu2:65741)WARNING: NFS: 1183: Invalid volume UUID mpx.vmhba32:C0:T0:L0:8
    2019-08-21T04:54:15.292Z cpu14:70977)WARNING: NFS: 1183: Invalid volume UUID mpx.vmhba32:C0:T0:L0:6

    I then compared the updated host with one I hadn't yet updated (was still at 6.5 U2, EP12).
    I noticed that the upgrade had actually renamed the identifier of the USB device.

    The identifier of the machine that HASN'T been updated = eui.00a0504658335331
    The identifier of the machine that WAS updated = mpx.vmhba32:C0:T0:L0

    I also found that after a moment of the host being in a weird hung state (even though it pings, and ssh works etc), it does eventually come to life and functions as usual.
    But after each reboot, it has the same problem.

    As I was typing this, HPE support contacted me to inform me that HPE engineering will be opening a case with VMware.

    I'll update again once I hear any more.

  3. Andre says:

    Hi @ll,

    i had the same issue with several DL380 and DL360 G10 servers.
    Workaround for me was to install the HPE 6.5 U2 custom iso and an inplace update via Update Manager to the latest build 14320405 afterwards.
    After that i added the HPE vibsdepot repositories and installed the latest driver and management components. Everything worked fine then.

    The problem seems to be caused by something buggy in the iso file of the 14320405 build.

  4. Advertising

  5. slicster says:

    Hello,
    Does anyone have an update on this? We have the same issue and have opened support tickets with VMware and HPE but no resolution in days.

    Specifically, after the host is updated to Update3 HPE customized bundle, it freezes 2-3minutes after reboot for 20-40minutes. If we use "esxcli software profile install" command to go to the VMware standard bundle, it boots and works fine. But then doing the same and going back to the HPE bundle, it freezes again. We even tried a fresh install using their ISO and same result.

    Tried re-creating the HPE custom bundle manually using the latest drivers in the HPE online depot and it works fine as well. One of the few VIBs in their bundle, our manual bundle and the VMware bundle is causing the issue but I haven't had the patience to go through them one-by-one yet. (lots of reboots!)

  6. EricH says:

    I'm having the same exact issue with VMware ESXi, 6.5.0, 14320405 on my 3 Dell R630s. I used the Dell 6.5 custom ISO to install originally. Been using Update Manager to install updates.

    Disabling cimsvc has fixed the issue, I can now do VMotion and connect to the indivudual ESXi servers.

    Any idea when VMware will have a fix for this? Are we missing anything with CIMSVC being disabled? I created a case on Saturday and will update it tomorrow after it's ran 12 hours without anymore issues.

  7. chgorges says:

    Hello everyone,

    VMware released a patch 4 days ago https://docs.vmware.com/en/VMware-vSphere/6.5/rn/esxi650-201910001.html#esxi-6-5-0-20191004001-no-tools-resolved

    The problem is descriped in the release notes.

    Tried the patch yesterday (HPE ML350 Gen10) and the hang on startup was never seen again.

Leave a Reply to Fr Cancel reply

Your email address will not be published. Required fields are marked *