HPE warns again about possible SSD failures (March 2020)

[German]Manufacturer HPE warns again of a problem with its SSDs for server storage systems (SSDs). A software bug in the firmware can cause the server storage systems to fail after 40,000 hours. A firmware update is available.

I had already pointed out a similar issue in November 2019 in the article HPE warns of SSD storage systems failure coming soon. At that time the issue was a counter in a firmware which caused a data loss after 32,768 hours of operation. Although the current failure is similar to the case from November 2019, this problem is not related to the SSD problem described last year.

Data loss after 40,000 hours

In this support document HPE now warns of a new failure scenario. Certain models of HPE SAS Solid State Drives require a critical firmware update to prevent a drive failure at 40,000 hours of operation. HPE has been notified by a Solid State Drive (SSD) manufacturer of a firmware defect affecting certain SAS SSD models:

The SSD models were built into a range of HPE server and storage products (i.e. HPE ProLiant, Synergy, Apollo 4200, Synergy storage modules, D3000 storage enclosures, StoreEasy 1000 storage). The problem affects SSDs with HPE firmware version prior to HPD7, which causes SSD failure at 40,000 hours of operation (i.e. 4 years, 206 days, 16 hours).

This issue may also affect other manufacturers and customers who purchased these drives. In this article (German), heise writes that Dell systems are also affected and that Sandisk SSDs are presumably installed. 

The solution is to immediately upgrade to the drive firmware version HPD7 released by HPE. HPE recommends performing an online firmware update on HPE Gen9 servers with minimal I/O activity. In most cases, this will not require a reboot. However, in cases where the online firmware update does not complete successfully, an offline update is required. When the flash update is complete, the Smart Component displays a message indicating whether the flash completed successfully.

IMPORTANT: This HPD7 firmware is considered a critical fix and is required to resolve the issue described below. HPE strongly recommends that this critical fix be applied immediately. Neglecting to upgrade to SSD firmware version HPD7 will result in a drive failure at 40,000 hours of operation, including data loss, and will require the recovery of data from the backup unless there is a fault tolerance (such as RAID 0 or in a fault-tolerant RAID mode). Further details, including firmware updates, can be found here.