Windows Server 2022: February 2023 Patchday and the ESXi VM Secure Boot Issue

Sicherheit (Pexels, allgemeine Nutzung)[German]The security update KB5022842 for Windows Server, released on February 14, 2023,  triggers collateral damage. Virtual machines can subsequently no longer start after a reboot and either can no longer find their system drives or trigger a Secure Boot error. Disabling Secure Boot helps – Microsoft and VMware have since confirmed this error.


Advertising

Boot issues with VMs

I had presented the security update KB5022842 released for Windows Server 2022 in the blog post Patchday: Windows 11/Server 2022 Updates (February 14, 2023). Shortly after the German edition of this post was published, users already came forward reporting issues (I've translated the comments).

User #1: My template Win 2022 VM does not boot up after the update once you turn it off.

User #2: Windows Server 2022 with Secure Boot / VBS enabled in the VMware are having problems after the Feb. update and will not boot.

First comes: Windows Boot Manager… Security Violation
Then Windows Boot Manager… unsuccessful

Can anyone confirm this?

User #3: Same problem, ESXi 7.0.3
After update the server is running, after further boot
comes "Security Violation
Disabling the Secure Boot solves the problem

User #4: Same problem with our Win 2022 server VMs. By disabling VBS and Secure Boot the VM boots up again. (ESXi 7.0.3 environment)

German blog reader Dennis C. also contacted me by e-mail and reported the error at the VMs:

Dear Mr. Born!

First of all, thank you for your website, it has helped me a few times.

I don't know if we are a unique case, but since today we have a problem that I would like to share with you. Maybe you can verify it and publish it:

I have the following problem with a customer (Server 2022): If there is a VM in version 19, the server member is a domain and receives the February update (KB5022842), the server does not survive the next reboot.

Using the console, you can still see the following before going into the boot options:

VMware Seciroty Violation message

If I now disable the security boot in the VM options, the server starts again as usual:

ESXi-Settings

Interestingly, the 3 requirements had to coincide in my case. A server in VM version 16 was not affected, there was no secure boot option.

I then pointed Dennis to the discussion on the blog. Within my English blog post Patchday: Windows 11/Server 2022 Updates (February 14, 2023) I've added the following warning:


Advertising

Addendum: I got several reports from German blog readers, saying, that their virtual machines can't boot either due to a "security violation" or due to a missing boot manager. Deactivating "Secure boot" should solve the issue.

The topic is also discussed on patchlist.org:

So far this is isolated to a single VM on VMware ESXI, but we have a server 2022, new install from about 2 weeks ago, installed updated Ok, rebooted OK.

Just rebooted again and it's got a "security violation."

Turning off VBS and secure boot seems to have fixed it for now.

There came the hint that on reddit.com in the patchday superthread as well as here the error was also reported. Martin noted on Facebook in an admin group to my post still:

It is certainly also important that the first restart works. So the server runs after the updates first. Only after the next regular restart, the problem then occurs. Only times as info for all those who perhaps feel safe that it is not with you.

These are probably all cases where VMware ESCi is used for virtualization. Uninstalling KB5022842 does not fix the problem according to Simon because the EFI files seem to remain in the new version.

In this comment, one refers to problems with Windows Server 2019 and Hyper-V, which is confirmed by a second user, but doesn't really correspond with above descriptions. And this comment reports Citrix PVS vdisks/machines not starting after the update.

VMware & Microsoft confirms the bug

Andi has already posted this comment on the German blog (thanks for that) and reported that a colleague has opened a support case with Microsoft:

A colleague has opened a call at MS.
MS is aware of the error, it is caused by ESXI.
ESXI 8.0 does not have the problem.
Either a patch will come from MS or from VMWare.

In the meantime VMware has published the support post Virtual Machine with Windows Server 2022 KB5022842 (OS Build 20348.1547) configured with secure boot enabled not booting up (90947) about this – thanks to Michael and other blog readers on Facebook for pointing this out. This reddit.com post summarizes the error again and provides the reference to VMware's support post. The VMware support post describes the bug and states:

Currently there is no resolution for virtual machines running on vSphere ESXi 6.7 U2/U3 and vSphere ESXi 7.0.x. However the issue doesn't exist with virtual machines running on vSphere ESXi 8.0.x. vSphere ESXi 6.7 is End of general Support.

Uninstalling the KB5022842 patch will not resolve the issue. If the Virtual machine has already been updated, then the only available options are:

  1. Upgrade the ESXi Host where the virtual machine in question is running to vSphere ESXi 8.0
  2. Disable "Secure Boot" on the VMs.

The support article then mentions upgrading to vSphere ESXi 8.0 and disabling Secure Boot in the VMs. VMware warns against the installation of the update KB5022842 on a virtual machine with Windows 2022 Server until the problem is solved. Meanwhile, Microsoft has posted the article Windows Server 2022 might not start up in the Windows Server 2022 Health Status section under Known Issues.

After installing KB5022842 on guest virtual machines (VMs) running Windows Server 2022 on some versions of VMware ESXi, Windows Server 2022 might not start up. Only Windows Server 2022 VMs with Secure Boot enabled are affected by this issue. Affected versions of VMware ESXi are versions vSphere ESXi 7.0.x and below.

Please refer to the VMware support article above. Microsoft and VMware are investigating this issue and will provide more information as it becomes available.

Addendum: The bug has been fixed for some ESXi versions, see Windows Server 2022: VMware ESXi 7.0 U3k Patch for Secure Boot Issue (Update KB5022842, Feb. 2023).

Similar articles:
Microsoft Security Update Summary (February 14, 2023)
Patchday: Windows 10 Updates (February 14, 2023)
Patchday: Windows 11/Server 2022 Updates (February 14, 2023)
Windows 7/Server 2008 R2; Server 2012 R2: Updates (February 14, 2023)
Patchday: Microsoft Office Updates (February 14, 2023)
Exchange Server Security Updates (February 14, 2023)


Cookies helps to fund this blog: Cookie settings
Advertising


##1

This entry was posted in Allgemein. Bookmark the permalink.

11 Responses to Windows Server 2022: February 2023 Patchday and the ESXi VM Secure Boot Issue

  1. Erich Signer says:

    Happened here as well.
    ESXi 7.0.3 (VCenter Build 20845200 – so Update 3i or 3j)
    Windows Server 2022 Standard with EFI/SecureBoot, after second reboot did not come back up.

  2. Aurelien says:

    thank you for the article.
    VMware ESXi, 7.0.3, 19193900 => same problem
    It saved my day.

  3. Tim Van Engeland says:

    Hello,

    Had also some clients with this issue this morning.
    Thanks for offering the solution…

    What what make me a bit angry/affraid : those customer have the Windows Update confgured as manual!

    With kind regards,
    Tim Van Engeland

  4. Advertising

  5. Hans Heiser says:

    Got the same issue on vSphere 7.0.3 and Win 2022 Server

  6. Paul Glitsch says:

    Thanks, this worked!

  7. Moel says:

    Same issue in ESX 6.7 and Windows 2022 core and gui. both of them freeze in boot manager and Secure boot deactivation solve the problem. thanks.

  8. Daniel says:

    We had this issue on every Server 2016 that ran Citrix. Disabling secure boot worked.

  9. Absolut79 says:

    20230312 VMware Win Server 2022 Secure Boot Issue

    How to fix this damn issue…

    If the VM is not booting through EFI on ESX 6.7 (u3) and the VM version is v14

    1. Open the VM settings, take screenshots of the VM settings and note the VM version will be version 14
    2. Remove (do not delete) the VM from the vcenter/esxi console
    3. Create a new VM with the same name and specs as before but with VM version 13.
    During creation, remove the default HDD and "Add exsisting HDD" and add the VMDK files from the previous VM.
    4. Before powering ON, edit the VM settings and change the boot option from "BIOS" to "EFI" and do not enable "secure boot"
    5. Power ON the VM and it will boot as normal :)

  10. Dee Tee says:

    Day/Weekend Saver!!!

    Thank you!!!!

  11. Jay K. says:

    Well Well Well
    It has happened on 100% of my MS 2019 Hyper-V servers; and every VM failed to start backup if they had been powered off; OH JOY ; and the only fix of course is to rebuild. But the funny thing is they will reboot just fine as long as they were not "shut down" after the patch. Seems it is isolated to the Hyper-Vs only nd not the VMs, thus they are the only ones need to be rebuilt.

    So it is not isolated to 2022 and or ESXi

Leave a Reply to Dee Tee Cancel reply

Your email address will not be published. Required fields are marked *