Backblaze sees rise in hard drive failure rates

The latest quarterly report from Backblaze on hard drive reliability reveals a rise in failures among certain drives.

Backblaze is a pure storage provider; cloud storage is all they do, and they dig deep into the statistics of hard drive failure and share their data with the industry. The company currently has a massive inventory of 241,297 hard disk drives of varying capacities and from various brands. (In recent quarters, Backblaze has added SSD performance to its measurements, but SSDs are still early in their deployment lifecycle, so patterns over time have yet to fully emerge.)

In the second quarter of 2023, the firm reported an average failure rate (AFR) of 2.28%, which is a sizable increase from 1.54% in the previous quarter. But the failures didn’t occur across the board. The company noted that 8TB and 10TB drives were throwing off the numbers.

The worst offenders were a Toshiba 8TB drive with a 19.63% AFR; a Seagate 14TB drive with a 14.28% AFR; an HGST 8TB drive with a 13.53% AFR; a Seagate 10TB drive with a 12.31% AFR; and a second Seagate 14TB drive with a 10.25% AFR.

Seagate logged pretty bad failure rates overall; it accounts for the bulk of Backblaze’s inventory and thus its failures. Western Digital’s failure rates never rose above 1% AFR. Aside from the one 8TB drive, Toshiba’s drives performed well and never rose above 2% AFR.

Backblaze notes that failures are a fact of life in cloud storage:

“Of course we’d like to see them lower, but the inescapable reality of the cloud storage business is that drives fail. Over the years, we have seen a wide range of failure rates across different manufacturers, drive models, and drive sizes. If you are not prepared for that, you will fail. As part of our preparation, we use our drive stats data as one of the many inputs into understanding our environment so we can adjust when and as we need,” wrote Andy Klein, principal cloud storage storyteller at Backblaze, in a blog post about the most recent quarterly drive stats. “So, are we worried about the increase in drive failure rates? No, but we are not arrogant either. We’ll continue to monitor our systems, take action where needed, and share what we can with you along the way.”

Given the average age of Backblaze’s drives is over 60 months, it would appear that their lifespan is about as good as it gets. Five years is pretty good performance for a hard drive, especially one that is mostly read and is written to very little.

But I’d like to share another observation that may be related.

I’ve been noticing a trend among hobbyists and system builders who say newer hardware is giving out relatively fast. One system builder told me that he noticed a drop in quality, with an abnormally high failure rate of all components. What puzzled him is that the equipment wasn’t failing immediately; rather, it was failing after about a year or two of use. Hardware typically is either going to fail immediately or last for its usual lifespan, short of some kind of traumatic event like a power surge.

The system builder theorized that quality had slipped during the Covid pandemic and related supply chain problems, and that equipment that’s failing now may have been built around 2021 or so, the height of the pandemic. 

It’s clear that a gamer who’s building a PC is different from a cloud storage provider, but they are both using drives that are made in the same place. Could there be a bum batch of hard drives made about two years ago? If so, perhaps all the drives that were going to fail have failed. It will be interesting to see if Backblaze’s AFR goes back to normal next quarter.