[German]At the beginning of December 2022, there was an outage of the Exchange instances of the US provider Rackspace that lasted several days. Now the company has confirmed that a ransomware attack on its servers was the reason for this outage. It is unclear whether customer data was affected by this incident. There is at least a suspicion that the ProxyNotShell vulnerabilities enabled this attack.
Advertising
Rackspace is an IT services provider from the USA, headquartered in San Antonio, Texas. The company is one of the largest representatives of its industry ($3 billion) and is active worldwide. Its customers include or have included GitHub, Vodafone and Mazda.
The hosted Exchange outage
I had noticed it in early December 2022, but didn't address it on the blog. Rackspace had acknowledged issues with its hosted Exchange instances in a series of tweets on December 2, 2022.
Then, on December 3, 2022, there was a message in the status area that "an issue" was affecting the vendor's Hosted Exchange environment. The Hosted Exchange environment was proactively shut down and disconnected from the Internet to investigate the extent and severity of the impact. It had already been announced there that a security incident had been identified following an analysis.
Customers who had booked Exchange hosting services from Rackspace were probably affected. Their e-mail servers were subsequently gone and the customers had to set them up again. The hoster offered customers free access to Microsoft Exchange Plan 1 licenses on Microsoft 365. On December 5, 2022, Rackspace was able to report that email services had been restored for thousands of customers.
Advertising
Ransomware infection confirmed
In its latest status update, dated Dec. 6, 2022, the vendor believes the security incident was limited to its Hosted Exchange services. The company has hired another security firm to investigate, in addition to its internal security team. The investigation is still in the early stages, it said. Therefore, no statements were made as to whether and which data was affected at all.
German blog reader Daniel K. had informed me by email that it was a ransomware incident (see here, thanks for that). This information had also come to my attention early in the morning via various security websites (e.g. this post by Bleeping Computer). In the company's status page, a ransomware incident is admitted to be the cause as of December 7, 2022.
Hosted Exchange Disruption
08:26 AM EST 12/06/22We appreciate your patience as we continue to work through the security issues that have affected our Hosted Exchange environment. As you know, on Friday, December 2nd, 2022, we became aware of suspicious activity and immediately took proactive measures to isolate the Hosted Exchange environment to contain the incident. We have since determined this suspicious activity was the result of a ransomware incident.
However, details are not really revealed. The article here on Security Week contains some additional information. In the article, security expert Kevin Beaumont is quoted as suggesting that the infection may have been possible by exploiting Microsoft Exchange vulnerabilities CVE-2022-41040 and CVE-2022-41082 (known as ProxyNotShell).
I did have some blog posts about this vulnerability, which became known as 0-day in late September 2022 and was then mitigated by Microsoft several times with inappropriate measures – see the links at the end of the article. The discovery of the vulnerability came after a Vietnamese cybersecurity firm observed an exploit in the wild (presumably by a state hacking group). Microsoft then released a security update in November 2022 (see Exchange Server security updates (November 8, 2022)).
Kevin Beaumont noted that a Rackspace Exchange server cluster that is now offline was using an August 2022 build number a few days before the incident was published. There is therefore no denying that the ransomware infection occurred via this very vulnerability – but it remains to be seen what Rackspace's investigation will uncover and whether that will then become public.
Similar articles:
Exchange Servers are attacked via 0-day exploit (Sept. 29, 2022)
Microsoft's recommendations for Exchange Server 0-day vulnerability ZDI-CAN-18333
Update on Exchange Server 0-day Vulnerability ZDI-CAN-18333: Fixes, Scripts and EMS Solution
Exchange Server: Microsoft updates it's mitigation for the 0-day ProxyNotShell vulnerability (October 5, 2022)
Exchange Server: Microsofts improves solutions for 0-day mitigation again (October 8, 2022)
Exchange Server security updates (November 8, 2022)
