[German]You occasionally hear "we are migrating to the cloud, to Microsoft Azure, Office 365, etc., Microsoft will make sure that patching is done and vulnerabilities are closed promptly". But cloud users are, for better or worse, dependent on Microsoft's goodwill and capabilities. And in terms of timely patching of vulnerabilities, things sometimes look bleak, as a recent case on the Synapse Pwnalytics vulnerability shows, which I'll rehash here.
Advertising
The fact that vulnerability reporting often hangs and people often fall behind in bug bounty awards is well known among security experts. The following tweet suggests that companies abuse bug bounty programs to keep security vulnerabilities from becoming public, but don't close them either.
There, a security researcher was forced to delete his Github post about a vulnerability again because all bug bounty submissions are confidential until a release is made. But not everything is going smoothly at Microsoft either – as I know from various emails where I was cc'd when people reported vulnerabilities.
The Azure Synapse Pwnalytics ca
For the June 2022 patchday on 6/14/2022, Microsoft also closed vulnerabilities in Microsoft Azure. Security researcher Tzah Pahima took it up briefly on Twitter in several tweets.
Advertising
He was able to access the passwords of thousands of companies on Azure and execute code on their VMs. This includes access to Microsoft's own credentials.
Tenable criticizes Microsoft's actions
Security vendor Tenable has taken up the case again, criticizing Microsoft's tenacious approach to fixing two serious security vulnerabilities in Microsoft's cloud environment (Microsoft Azure Synapse Pwnalytics). This is because since March 10, Tenable Research has been trying to work with Microsoft to fix two serious vulnerabilities in the Azure Synapse Analytics infrastructure.
What is Synapse Analytics?
Synapse Analytics is a platform used for machine learning, data aggregation and other computing applications. The service is currently listed under "high-impact" scenarios in Microsoft's Azure Bug Bounty program. Microsoft states that products and scenarios listed under this heading "have the highest potential impact on customer security." aufgeführt.
Microsoft Azure Synapse Pwnalytics vulnerabilities
Tenable Research has discovered two serious vulnerabilities in the infrastructure on which this service runs. These vulnerabilities allow a user to escalate root user privileges within the underlying Apache Spark virtual machines or "poison" the hosts file of all nodes in an Apache Spark pool.
The keys, secrets, and services that can be accessed through these vulnerabilities are known to allow further lateral movement and compromise of Microsoft-owned infrastructures. This can potentially lead to the compromise of other customers' data, as seen in several other recent cases, such as Wiz's ChaosDB and Orca's SynLapse. However, Microsoft has claimed that cross-client access is not possible via these attack vectors.
Tenable reported these issues to Microsoft on March 10, 2022. Microsoft has already started providing a fix for the privilege escalation issue on April 30, 2022. Tenable currently believes that the issue has been successfully fixed in all regions. End users do not need to take any action to ensure their environments are no longer affected. The hosts file poisoning attack has not been patched at the time of writing. Due to the nature of these vulnerabilities and the disclosure process, Tenable does not yet have CVE reference numbers for this.
Slow processing by Microsoft
During the disclosure process, Microsoft representatives initially seemed to agree that these were critical issues. Microsoft developed and implemented a patch for the privilege escalation without further information from Tenable Research. In the final days of the disclosure process, the Microsoft Security Response Center (MSRC) attempted to downplay the severity of the privilege escalation issue, classifying it as a "best practice recommendation" rather than a security issue.
Despite clear evidence to the contrary, the MSRC declined to issue a bounty or credit for this discovery. After Microsoft was informed by Tenable to publish information about the vulnerabilities, Microsoft representatives reversed the previous decision and classified these issues as security related. This shows a clear lack of communication between the teams involved at Microsoft.
These vulnerabilities and the Tenable researchers' interaction with Microsoft show how difficult it is to address security-related issues in cloud environments. The entire process is largely out of the customer's control. Customers are completely reliant on cloud providers to fix reported issues. The good news, however, is that once a problem is fixed, it's fixed. Customers usually don't have to do anything because everything happens behind the scenes. The bad news, however, is that cloud providers rarely indicate that a security-related vulnerability was even present.
For more detailed information about the interactions with Microsoft and the technical details of these vulnerabilities, see this post on Tenable TechBlog. In a personal statement, Tenable's CEO Amit Yoran also commented on what happened last week and on the fundamental problem of vulnerabilities in cloud environments of major vendors.
Further information
- TRA-2022-19: Microsoft Azure Synapse Analytics Hosts File Poisoning
- TRA-2022-20: Microsoft Azure Synapse Analytics Privilege Escalation
Not the only flaw in Azure
Back in March 2022, security researchers from Orca Security identified a critical Azure Automation vulnerability (AutoWarp) in less than two hours. AutoWarp is a critical vulnerability in the Azure Automation service that allows unauthorized access to other Azure customer accounts using the service. Depending on the permissions assigned by the customer, this attack could mean complete control over resources and data of the target account.
Microsoft Azure Automation
Microsoft Azure Automation enables organizations to run automation code in a managed way. They can schedule jobs, provision input and output, and more. Each company's automation code runs in a sandbox, isolated from other customers' code running on the same virtual machine.
Research by Orca Security revealed that several large companies were using the service and could have accessed it. The AutoWarp vulnerability could have caused billions of dollars in damage. Orca reported the critical vulnerability in Azure Automation directly to Microsoft, it has now been fixed, and all affected customers have been notified. "We would like to thank Yanir Tsarimi of Orca Security for reporting this vulnerability and working with the Microsoft Security Response Center (MSRC) as part of the Coordinated Vulnerability Disclosure (CVD) to ensure the safety of Microsoft customers," said the Microsoft Security Response Center (MSRC). Prior to the vulnerability fix, organizations were vulnerable to AutoWarp if they used the Azure automation service and had the Managed Identity feature enabled in their automation account (which is the case by default).
Vulnerability stumbled upon by accident
The vulnerability came to light in the following way: Yanir Tsarimi, Cloud Security Researcher, at Orca Security was scrolling through the list of Azure services looking for his next service to investigate. When he saw "Automation Accounts" under the "Management & Governance" category, he was surprised. He thought it was a service that allowed Azure accounts to be controlled through automation. After creating his first automation account, he found that Azure Automation is a standard service for automation scripts. Users can use it to upload their Python or PowerShell script to Azure for execution.
Setting up a reverse shell with Python was a no-brainer. However, when Tsarimi ran some of the common commands like "tasklist," he received an error message that they were not found. Apparently, the environment variable "PathExt," which is responsible for determining which file extensions the operating system should try to execute, is set to a strange value. Normally it contains the file extension ".exe", but not in this case. Only ".CPL" was present, the file extension for Windows Control Panel items. Even when he tried to use "tasklist.exe" to list the running processes, he got a message he had never seen before. It looked like
The first two things Tsarimi noticed when he looked at the C:\ drive were the "Orchestrator" and "temp" directories. The orchestrator directory contained a lot of DLLs and EXEs. He saw filenames with "sandbox" and intuitively understood that this directory contained the sandbox he was working in. The temp directory contained another directory called "diags" and a "trace.log" file.
A look at the Azure Automation code
After downloading the files from the Orchestrator, he launched ILSpy (.NET Decompiler) and looked for the code for this "Asset Retrieval Web Service." He looked at the method for setting up the HTTP routes, and noticed that "/oauth2/token" and "/metadata/identity/oauth2/token" were assigned to "MSIController." MSI stands for "Managed Service Identity", which he found interesting.
Some software development experience in reviewing source code is very helpful here to "read between the lines" even in more complicated cases.
Tsarimi started making HTTP requests to "/oauth2/token" and customized his request to look like a metadata request. He wanted the token to be usable by the Azure management APIs, so he used "resource=https://managment.azure.com/." The request simply returned a JWT (JSON Web Token). He decoded the JWT token and saw his subscription ID, his tenant ID, and the resource ID of his automation account. He found that each automation account has a "system-assigned managed identity," which basically means that users can assign roles to their automation scripts and the identity is managed by the service.
Tsarimi now wanted to test whether his JWT token was real. He used the Azure CLI to make a simple request to retrieve all his VMs ("az vm list"), intercepted the request and replaced the JWT token. He received an error message that he did not have enough permissions. After assigning a compatible role to his managed identity, the token worked and was indeed associated with his managed identity.
A token associated with a managed identity is not a problem in itself, but there were additional ports that could be accessed locally. Random ports provided JWT tokens to Tsarimi. He ran the script a few more times, and different ports gave him different tokens again. Now he realized that he could actually access other people's identity endpoints. He had already proven that these tokens could be used to manage the Azure account if they were given enough permissions so that accessing other tenants' data was not necessary.
The Orca research team wanted to understand how far this simple vulnerability could go. The researchers used Azure Automation's Schedule feature to grab tokens from a few hundred ports and see which tenants showed up. They did not save the particular token and only extracted the metadata about the tenant (tenant ID and resource ID of the automation account). In that short period of time before the issue was patched, they were able to observe many different tenants, including several very well-known companies.
It was a relatively simple bug that evolved into a very interesting vulnerability. The root cause could not be clearly determined yet. The identity endpoint may have required some form of authentication (other endpoints on that server certainly did). However, someone may have overlooked the fact that the machine's internal network was not sandboxed as expected.
Microsoft has fixed this issue by now requiring the HTTP header "X-IDENTITY-HEADER" when requesting identities. Users must set the value to the secret value specified in their environment variables. Microsoft has also announced that it will review the entire architecture to ensure that such a bug cannot occur again.
