Windows Server 2022: February 2023 Patchday and the ESXi VM Secure Boot Issue

[German]The security update KB5022842 for Windows Server, released on February 14, 2023,  triggers collateral damage. Virtual machines can subsequently no longer start after a reboot and either can no longer find their system drives or trigger a Secure Boot error. Disabling Secure Boot helps – Microsoft and VMware have since confirmed this error.

Boot issues with VMs

I had presented the security update KB5022842 released for Windows Server 2022 in the blog post Patchday: Windows 11/Server 2022 Updates (February 14, 2023). Shortly after the German edition of this post was published, users already came forward reporting issues (I've translated the comments).

User #1: My template Win 2022 VM does not boot up after the update once you turn it off.

User #2: Windows Server 2022 with Secure Boot / VBS enabled in the VMware are having problems after the Feb. update and will not boot.

First comes: Windows Boot Manager… Security Violation
Then Windows Boot Manager… unsuccessful

Can anyone confirm this?

User #3: Same problem, ESXi 7.0.3
After update the server is running, after further boot
comes "Security Violation
Disabling the Secure Boot solves the problem

User #4: Same problem with our Win 2022 server VMs. By disabling VBS and Secure Boot the VM boots up again. (ESXi 7.0.3 environment)

German blog reader Dennis C. also contacted me by e-mail and reported the error at the VMs:

Dear Mr. Born!

First of all, thank you for your website, it has helped me a few times.

I don't know if we are a unique case, but since today we have a problem that I would like to share with you. Maybe you can verify it and publish it:

I have the following problem with a customer (Server 2022): If there is a VM in version 19, the server member is a domain and receives the February update (KB5022842), the server does not survive the next reboot.

Using the console, you can still see the following before going into the boot options:

If I now disable the security boot in the VM options, the server starts again as usual:

Interestingly, the 3 requirements had to coincide in my case. A server in VM version 16 was not affected, there was no secure boot option.

I then pointed Dennis to the discussion on the blog. Within my English blog post Patchday: Windows 11/Server 2022 Updates (February 14, 2023) I've added the following warning:

Addendum: I got several reports from German blog readers, saying, that their virtual machines can't boot either due to a "security violation" or due to a missing boot manager. Deactivating "Secure boot" should solve the issue.

The topic is also discussed on patchlist.org:

So far this is isolated to a single VM on VMware ESXI, but we have a server 2022, new install from about 2 weeks ago, installed updated Ok, rebooted OK.

Just rebooted again and it's got a "security violation."

Turning off VBS and secure boot seems to have fixed it for now.

There came the hint that on reddit.com in the patchday superthread as well as here the error was also reported. Martin noted on Facebook in an admin group to my post still:

It is certainly also important that the first restart works. So the server runs after the updates first. Only after the next regular restart, the problem then occurs. Only times as info for all those who perhaps feel safe that it is not with you.

These are probably all cases where VMware ESCi is used for virtualization. Uninstalling KB5022842 does not fix the problem according to Simon because the EFI files seem to remain in the new version.

In this comment, one refers to problems with Windows Server 2019 and Hyper-V, which is confirmed by a second user, but doesn't really correspond with above descriptions. And this comment reports Citrix PVS vdisks/machines not starting after the update.

VMware & Microsoft confirms the bug

Andi has already posted this comment on the German blog (thanks for that) and reported that a colleague has opened a support case with Microsoft:

A colleague has opened a call at MS.
MS is aware of the error, it is caused by ESXI.
ESXI 8.0 does not have the problem.
Either a patch will come from MS or from VMWare.

In the meantime VMware has published the support post Virtual Machine with Windows Server 2022 KB5022842 (OS Build 20348.1547) configured with secure boot enabled not booting up (90947) about this – thanks to Michael and other blog readers on Facebook for pointing this out. This reddit.com post summarizes the error again and provides the reference to VMware's support post. The VMware support post describes the bug and states:

Currently there is no resolution for virtual machines running on vSphere ESXi 6.7 U2/U3 and vSphere ESXi 7.0.x. However the issue doesn't exist with virtual machines running on vSphere ESXi 8.0.x. vSphere ESXi 6.7 is End of general Support.

Uninstalling the KB5022842 patch will not resolve the issue. If the Virtual machine has already been updated, then the only available options are:

1. Upgrade the ESXi Host where the virtual machine in question is running to vSphere ESXi 8.0
2. Disable "Secure Boot" on the VMs.

The support article then mentions upgrading to vSphere ESXi 8.0 and disabling Secure Boot in the VMs. VMware warns against the installation of the update KB5022842 on a virtual machine with Windows 2022 Server until the problem is solved. Meanwhile, Microsoft has posted the article Windows Server 2022 might not start up in the Windows Server 2022 Health Status section under Known Issues.

After installing KB5022842 on guest virtual machines (VMs) running Windows Server 2022 on some versions of VMware ESXi, Windows Server 2022 might not start up. Only Windows Server 2022 VMs with Secure Boot enabled are affected by this issue. Affected versions of VMware ESXi are versions vSphere ESXi 7.0.x and below.

Please refer to the VMware support article above. Microsoft and VMware are investigating this issue and will provide more information as it becomes available.

Addendum: The bug has been fixed for some ESXi versions, see Windows Server 2022: VMware ESXi 7.0 U3k Patch for Secure Boot Issue (Update KB5022842, Feb. 2023).

Similar articles:
Microsoft Security Update Summary (February 14, 2023)
Patchday: Windows 10 Updates (February 14, 2023)
Patchday: Windows 11/Server 2022 Updates (February 14, 2023)
Windows 7/Server 2008 R2; Server 2012 R2: Updates (February 14, 2023)
Patchday: Microsoft Office Updates (February 14, 2023)
Exchange Server Security Updates (February 14, 2023)