RAID and PCs seems to be a topic that gets more complicated the more I try to understand it.

In our continuing saga, I'm toying with getting a NAS (probably a Synology DS410 - the DS411+ is overkill) or setting up a server with RAID 5. The server route seemed more appealing after thinking more about the cost, and seeing this "10TB Home Server" DIY Kit at NewEgg - http://www.newegg.co...List=Combo.569883 ($704 + shipping).

Given the horror stories we've heard, and extra expense, of Hardware RAID, it seems like Ubuntu Server 10.04 LTS with software RAID (mdadm?) (no LVM stuff), might make sense. I'm assuming there should be no issues connecting to it from OSX, Win2k, Linux, and Win7 as a file and print server.

(If I go the server route, I'd probably stick with Ubuntu rather than FreeNAS or the like.)

I'm assuming that there will be no file and print serving issues for Win2k, WinXP, Win7, OS X, and Linux clients, but haven't checked in detail.

I'm also assuming going this route will not introduce any incompatibilities even if I'm using a 32-bit version of Winders that doesn't understand partitions larger than 2TB. Will Winders freak out if it connects to a 8-10 TB partition over the network? I assume not, but ....

I assume I would have a boot drive separate from the RAID array. If I went the server route, I'd probably go with 1.5-2TB drives that are on the Synology-compatible page, or reported Ok by users, as there is at least some evidence that they've been tested in a RAID setup.

I don't care about hot-swap. I do want it to be reliable.

If one is running software RAID, are enterprise hard drives worth the cost for the peace of mind? Does software RAID care about enterprise vs. consumer grade drives? Should TLER - http://en.wikipedia....ed_Error_Recovery
- or CCTL - http://www.samsung.c...esource_CCTL.html
- be enabled or disabled for software RAID, or does it matter?

EARS and 4kB Sectors and incompatibilities and poor performance with Misaligned Sector Boundaries. Oh my! http://www.tomshardw...ector,2554-3.html

So many questions... It's easy to see why Windows Home Server and boxes like Drobo became popular. They hide the complexity. Ignorance is bliss, until something blows up. :-/

Thanks for any comments.

Cheers,
Scott.

http://labs.google.c...disk_failures.pdf (13 page .pdf)

The data in this study are collected from a large number of disk drives, deployed in several types of systems across all of Google’s services. More than one hundred thousand disk drives were used for all the results presented here. The disks are a combination of serial and parallel ATA consumer-grade hard disk drives, ranging in speed from 5400 to 7200 rpm, and in size from 80 to 400 GB. All units in this study were put into production in or after 2001. The population contains several models from many of the largest disk drive manufacturers and from at least nine different models. The data used for this study were collected between December 2005 and August 2006.

I haven't heard about the CMU study before. Hmm...

http://www.pdl.cmu.e...reData/index.html

As part of this project, we have analyzed field-gathered disk replacement data from a number of large production systems, including high-performance computing sites and internet services sites. About 100,000 disks are covered by this data, some for an entire lifetime of five years. The data include drives with SCSI and FC, as well as SATA interfaces. The mean time to failure (MTTF) of those drives, as specified in their datasheets, ranges from 1,000,000 to 1,500,000 hours, suggesting a nominal annual failure rate of at most 0.88%. Below is a summary of a few of our results.

[...]

Interestingly, the replacement rates of SATA disks are not worse than the replacement rates of SCSI or FC disks (unlike commonly assumed). In Figure 1 above the blue bars and the right-most one of the cyan bars correspond to SATA disk populations, while all other bars correspond to SCSI or FC populations. This may indicate that disk-independent factors, such as operating conditions, usage and environmental factors, affect replacement rates more than component specific factors.

[...]

Figure 7 above illustrates this point by plotting the probability that a second drive in a RAID fails during reconstruction, derived in four different ways: the purple bar estimates the probability based on exponential time between failures using the datasheet MTTF; the blue bar estimates the probability based on exponential time between failures, but using the actual empirical MTTF; the orange bar uses a Weibull distribution fit to empirical data; and the green bar shows the estimates directly derived from the data. As the graph shows the estimates derived using the standard approaches (pink and blue bar) can greatly underestimate the probability of a RAID failure.

On the other hand - http://www.pdsi-scid...roeder-fast08.pdf (16 page .pdf):

(i) During the 41-month time period, we observe more than 400,000 instances of checksum mismatches, 8% of which were discovered during RAID reconstruction, creating the possibility of real data loss. Even though the rate of corruption is small, the discovery of checksum mismatches during reconstruction illustrates that data corruption is a real problem that needs to be taken into account by storage system designers.

(ii)We find that nearline (SATA) disks and their adapters develop checksum mismatches an order of magnitude more often than enterprise class (FC) disks. Surprisingly, enterprise class disks with checksum mismatches develop more of them than nearline disks with mismatches.

(iii) Checksum mismatches are not independent occurrences – both within a disk and within different disks in the same storage system.

(iv) Checksum mismatches have tremendous spatial locality; on disks with multiple mismatches, it is often consecutive blocks that are affected.

(v) Identity discrepancies and parity inconsistencies do occur, but affect 3 to 10 times fewer disks than checksum mismatches affect.

There do seem to be some benefits from using "enterprise" class disks, but I don't know if it's worth the cost for home to use "enterprise class" SATA (FC and SAS are out of the question) - e.g. $240 for 2TB vs $120 for 2TB Hitachis.

Cheers,
Scott.
(Who probably won't go the "enterprise" route.)