Ups, Microsoft deleted Azure Cloud data bases

[German]That's not cool: The issues that Microsoft have had with Microsoft 365 on January 29, 2019 also affected Azure SQL databases. Some customers have had SQL databases deleted on Azure. After restoring a snapshot, customers discovered that there were transaction data of up to 5 minutes are missing within their databases.


Microsoft 365 outage on January 29, 2019

The last days Microsoft was shaken by failures of its and Microsoft 365 services – see article links at the end of this article. Now The Register reported here that Azure's January 29, 2019 failure even claimed some unexpected victims. Customers suddenly discovered that their Azure SQL databases had been deleted or that data was missing. Here is a tweet from a victim:

Also the following user was probably caught cold by the deleted Azure SQL database. 

The issue only affected customers using Transparent Data Encryption (TDE). This is to protect the databases against malicious activities of attackers. 


Azure SQL database deleted

According to the site The Register, during the Azure instabilities that occurred on 01/29/2019, internal automatic task simply deleted the affected SQL databases. Microsoft was able to restore the databases from snapshots. But the snapshots was up to 5 minutes in the past, so customers were missing 5 minutes of data from various transactions.

The issue affected a number of Azure SQL databases that use custom KeyVault keys for transparent data encryption (TDE). The Register cited from a messages Microsoft has send to affected users:

"An automated process, designed to trigger when custom keys are removed from KeyVault, inadvertently caused these TDE databases to be dropped.

We are in the process of restoring a copy of these SQL DBs from a recovery point in time of less than 5 minutes before the database was dropped. These restored databases … are located on the same server as the original database"

Within the notification, Microsoft asks customers to check the databases for lost data and, if necessary, open a support ticket.

We ask that customers, for each database, identify if lost transactions, during this 5 minute timeframe, could impact business processes or applications outside the database. We would ask you to raise a support ticket in this instance. If the restored database is suitable, the database can be renamed to the original name to continue usage of it.

I guess, some customers are pretty pissed. But Microsoft has good news for those affected – the company wrote:

We sincerely apologize for the impact on your service. Azure usage fees are waived for all restored databases for 2 months and for all original databases for 3 months. We are continuously taking steps to improve the Microsoft Azure platform and our processes to ensure that such incidents do not occur again in the future.

That sounds good – or what's your opinion? The Register asked Microsoft what the next step would be. A spokesman said: 'We're working to restore access to resources that were unavailable to a limited subset of customers. Full access has been re-established for most of those customers already.'

Cookies helps to fund this blog: Cookie settings

This entry was posted in issue and tagged , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *