The data in this study are collected from a large number of disk drives, deployed in several types of systems across all of Google’s services. More than one hundred thousand disk drives were used for all the results presented here. The disks are a combination of serial and parallel ATA consumer-grade hard disk drives, ranging in speed from 5400 to 7200 rpm, and in size from 80 to 400 GB. All units in this study were put into production in or after 2001. The population contains several models from many of the largest disk drive manufacturers and from at least nine different models. The data used for this study were collected between December 2005 and August 2006.

As part of this project, we have analyzed field-gathered disk replacement data from a number of large production systems, including high-performance computing sites and internet services sites. About 100,000 disks are covered by this data, some for an entire lifetime of five years. The data include drives with SCSI and FC, as well as SATA interfaces. The mean time to failure (MTTF) of those drives, as specified in their datasheets, ranges from 1,000,000 to 1,500,000 hours, suggesting a nominal annual failure rate of at most 0.88%. Below is a summary of a few of our results.

[...]

Interestingly, the replacement rates of SATA disks are not worse than the replacement rates of SCSI or FC disks (unlike commonly assumed). In Figure 1 above the blue bars and the right-most one of the cyan bars correspond to SATA disk populations, while all other bars correspond to SCSI or FC populations. This may indicate that disk-independent factors, such as operating conditions, usage and environmental factors, affect replacement rates more than component specific factors.

[...]

Figure 7 above illustrates this point by plotting the probability that a second drive in a RAID fails during reconstruction, derived in four different ways: the purple bar estimates the probability based on exponential time between failures using the datasheet MTTF; the blue bar estimates the probability based on exponential time between failures, but using the actual empirical MTTF; the orange bar uses a Weibull distribution fit to empirical data; and the green bar shows the estimates directly derived from the data. As the graph shows the estimates derived using the standard approaches (pink and blue bar) can greatly underestimate the probability of a RAID failure.

(i) During the 41-month time period, we observe more than 400,000 instances of checksum mismatches, 8% of which were discovered during RAID reconstruction, creating the possibility of real data loss. Even though the rate of corruption is small, the discovery of checksum mismatches during reconstruction illustrates that data corruption is a real problem that needs to be taken into account by storage system designers.

(ii)We find that nearline (SATA) disks and their adapters develop checksum mismatches an order of magnitude more often than enterprise class (FC) disks. Surprisingly, enterprise class disks with checksum mismatches develop more of them than nearline disks with mismatches.

(iii) Checksum mismatches are not independent occurrences – both within a disk and within different disks in the same storage system.

(iv) Checksum mismatches have tremendous spatial locality; on disks with multiple mismatches, it is often consecutive blocks that are affected.

(v) Identity discrepancies and parity inconsistencies do occur, but affect 3 to 10 times fewer disks than checksum mismatches affect.