QNAP fixes critical NAS bug that may causes data loss

[German]Vendor QNAP has fixed a critical bug in its firmware for NAS drives, that may causes data loss in several RAID configurations.

QNAP is a vendor of  NAS devices (Network-attached storage), and those devices are also shipped within several OEM products. The firmware used within QNAP NAS devices have had a critical bug, that may cause corrupted user data in some scenarios. QNAP has issued a firmware update to fix this bug.

The problem was observed by an IT supporter

The bug was observed by IT supporter Wayne Small, who observed a loss of 50 TB Data on a customer's QNAP NAS in December 2016. In January 2017 the supporter observed also additional cases, where data on a QNAP NAS becomes corrupted. Not all data, but several files. The bug has been observed on all QNAP NAS devices with four and more disks, as Wayne Small writes within this blog post.

Data loss only in RAID systems after disk failure

The error is a bit exotic – it won't be observed in normal operation mode of QNAP NAS. Only if the QNAP NAS device is configured as a RAID 5 system and if one drive fails, some data files may be corrupted. This configuration was used within customer systems, where QNAP NAS units with 4 drives was configured as RAID 5. Those RAID 5 arrays was used to store backups. Within a RAID 5 system, data should survive a disk failure. But on a QNAP NAS configured as RADI 5, a disk failure causes data corruption and data loss.

QNAP's reaction was painful slow

Wayne Small observed the data loss and analyzed the issue in January 2017, because he have had several disk failures within several customer's QNAP NAS devices. Then he informed QNAP. Wayne Small wrote, that QNAP reacted with long delay, days after his report he just received a feedback via e-mail. After 2 months QNAP was willing to test the RAID 5 constellation within a lab.

But Wayne noted, that he was not alone. There is also another thread in Veeam forum dealing with data loss in NAS. In March 2017 QNAP confirmed the issue and told him that the bug will be fixed within April firmware update.

Wayne Small wrote on July 17, 2017, that he could not found any information in QNAP release notes, mentions the fix. Also there has no notification of QNAP customers via e-mail about this issue.

But it seems, that Wayne Small's blog post has triggered a movement on QNAP's side. On July 21, 2017 QNAP updated their release notes, mentioned this issue.  A German blog reader directed me to this QNAP FAQ. QNAP has released a list of NAS devices not affected by this issue and provided this advisory to check whether a drive is affected.

For affected QNAP devices with firmware version before 4.3.3.0154 20170413 or 4.2.5 20170413 should be updated. Otherwise a data los in RAID5/6 configuration may be possible. As Wayne Small wrote, QNAP started on July 25, 2017 sending e-mails to customers, informing them about the issue and the fix. I also read within the Veeam forum thread, that customers has been notified recently. (via)