[German]The hack of Microsoft's Azure cloud by the suspected Chinese group Storm-0558 from May to June 2023 was possible due to a stolen private MSA key and bugs. At the time, accounts at Exchange Online via OWA and Outlook.com had been hacked from 25 organizations. It was unclear how the attackers came into possession of a private MSA key. Now Microsoft has announced that the MSA key came from a so-called Windows crash dump, which was created on a Microsoft PC and then dumped via a compromised system.
Advertising
The Storm-0558 cloud hack
In July 2023, it became known that a Chinese hacker group named Storm-0558 by Microsoft had succeeded in gaining access to the email accounts of about 25 organizations stored in the Microsoft Cloud (Exchange Online, outlook.com). These include government agencies (U.S. State Department), as well as corresponding private accounts of individuals likely associated with these organizations.
The attackers were in possession of a private (MSA) customer key for Microsoft accounts, and were able to use this MSA key to generate fake security tokens (for OWA). Due to a verification bug, these security tokens could be misused to access private Microsoft accounts (e.g., outlook.com) as well as Azure AD accounts and probably Azure apps.
Microsoft had officially admitted this incided, but downplayed it (see also China hacker (Storm-0558) accessed Outlook accounts in Microsoft's cloud). The incident then developed into a veritable security disaster, as the attackers had been undetected in the systems since May 2023. The hack was only discovered by chance, when a customer noticed unusual activity. Security researchers from Wiz stated later that actually the entire Microsoft cloud infrastructure should be considered potentially compromised. The development of this case has been traced in numerous blog posts, linked at the end of the article. However, it was unclear how the attackers were able to obtain the private MSA customer key.
Microsoft names the circumstances of the key theft
After many weeks of analysis, Microsoft now believes it knows how the Chinese hacker group Storm-0558 was able to obtain the private MSA customer key. Redmond disclosed the whole thing on September 6, 2023 in the post Results of Major Technical Investigations for Storm-0558 Key Acquisition.
- Microsoft say that it maintains a highly isolated and restricted production environment where access is also controlled. There, the use of email, conferencing, web research and other tools is blocked.
- But outside the restricted production environment, of course, PCs are used where email, conferencing, web research and other collaboration tools are allowed and in use. These systems are then also vulnerable to attacks (via spear-pishing, etc.).
And then, so to speak, Murphy's law came around the corner. Microsoft's investigation leads to the conclusion that there was a crash of the consumer signing system within the isolated environment in April 2021. A snapshot of the crashed process ("crash dump") was generated. Normally, such crash dumps should not contain any sensitive information or these should be made unrecognizable. In plain language: This crash dump should never have contained an MSA signature key.
Advertising
In the current case, however, a so-called race condition occurred, so that the private MSA customer key was included in the crash dump (this problem has been fixed in the meantime, according to Microsoft). The private MSA customer key in the crash dump was not recognized by Microsoft's security systems (this problem has also been fixed). So at this point, more "bugs" are admitted that were only fixed after the incident.
Because no control system responded, the crash dump was then moved from the isolated production network to the debugging environment at Microsoft. There, however, the computers are connected to the Internet via the corporate network. When the private key was stored on a system in the corporate environment after April 2023, the Chinese hackers from the Storm-0558 group were able to compromise the corporate account of a Microsoft engineer. That account had access to the debugging environment with the crash dump that unintentionally contained the private MSA customer key.
While Microsoft does not have logs (for regulatory reasons involving log retention policies) with specific evidence of exfiltration by this actor. However, Redmond believes that this was the most likely mechanism by which the actor acquired the key.
Damn many coincidences, or insider job and sloppiness?
Let's take a look at Microsoft's explanations. At this point, Microsoft has admittedly made a guess as to how it probably could have been. That can't be dismissed out of hand, but a hard proof is missing. What makes me wonder, as an outside observer, is this chain of unbelievable coincidences.
- The Chinese attackers were able to compromise the very corporate account of a Microsoft engineer who happened to have access to the debug environment.
- And then there was the unlikely event that a crash dump of the incident from the production environment happened to be stored there.
- And it just so happened that this crash dump also contained this MSA key, which was not supposed to be there, but was due to a race condition.
Needless to say, the Chinese attackers also knew where to find which files, pulled off the dump, then analyzed and isolated the private MSA key in the wild sequence of hex numbers. If you asked an inspector in a movie Crime Scene, the answer would immediately be "There are not that many coincidences, never".
And there's one more fine detail, which I revealed in the blog post Microsoft as a Security Risk? U.S. senator calls for Microsoft to be held accountable over Azure cloud hack– Part 1. The MSA key used in the above hack was created by Microsoft in 2016 and expires in 2021. It is unbelievable and can't be explained that this expired MSA key could be used to generate security tokens. And it is also inexplicable that Microsoft has a single "master key" that can be used to generate tokens for access to the Azure cloud and its services. Every reader should draw his or her own conclusions here.
Why the MSA consumer key was usable
Microsoft also addressed why it was even possible to use a private MSA consumer key to create security tokens for enterprise accounts in Exchange Online and OWA in the article linked above. In September 2018, Microsoft introduced a common endpoint for publishing key metadata. This was the result of growing customer demand for support for applications that work with both consumer and enterprise applications. Here, too, many issues was outlined:
- Microsoft provided at the time, as part of an already existing library with documentation and auxiliary APIs, an API to cryptographically validate the signatures.
- However, the libraries that were supposed to perform the validation of the application scope automatically were not updated – this issue has been fixed.
- The mail systems to use the common metadata endpoint in 2022 were updated though. The developers in the mail system incorrectly assumed that the libraries would perform full validation and did not add the required issuer and scope validation.
As a consequence, the mail system accepted a request for corporate email with a security token signed with the consumer key (this issue has since been fixed with the updated libraries as well). Again, it crystallizes that a chain of failures led to the loss of a private MSA customer key resulting in these impacts.
My final 2 cents
The information now published by Microsoft is commendable. But here, too, every reader may draw his or her own conclusions about Microsoft's reliability. Personally, I always forget that Microsoft is a large company with a lot of employees – and in large companies, everything that can go wrong does go wrong.
The nimbus that Microsoft is a company with concentrated competence, with folks who know what they are doing, has definitely been vaporized with this incident and the information that has become public. And when I then see how Microsoft bullies its customers, it's time to break the company up into different parts and put it under strict regulations. Otherwise, Microsoft will drag the entire IT world with its products into a security desaster.
Similar articles:
China hacker (Storm-0558) accessed Outlook accounts in Microsoft's cloud
Follow-up to the Storm-0558 cloud hack: Microsoft is still in the dark
Stolen AAD key allowed (Storm-0558) wide-ranging access to Microsoft cloud services
Microsoft as a Security Risk? U.S. senator calls for Microsoft to be held accountable over Azure cloud hack– Part 1
Microsoft as a Security Risk? Azure vulnerability unpatched since March 2023, heavy criticism from Tenable – Part 2
