As you'll need at least two: one for the data and the other for it's mirror. Oh yes, then a chunk of the I-O performance goes towards keeping the two copies in sync, so you'd better halve that figure, too.

I'm a little rusty on MTBF's

but if each module is 2000000 Hours MTBF,

then is 80 of then 1/80 th the MTBF,

which is around 2 1/2 years.

Not bad, but does that mean you need a raid or check sum then ?

> one domain can mirror another.

You must work for Microsoft/Danger.

Single point of failure?

I would expect enterprises to run these using a RAID protection. However, it doesn't need to be RAID-1 (mirroriing). A RAID-5 setup would work fine and such is the very low latency of I/Os that the RAID-5 overhead of read-modify-write cycle (so one write operation can result in 4 back-end I/Os) is not going to be an issue. (In any case, arrays with NV caches hid much of this anyway). Even when running in degraded mode with the loss of one drive out of the RAID-5 set, then such is the performance and latency of these devices that I doubt you'll even notice it in performance terms. Also, unlike hard-drive based RADI-5 sets, rebuild time should be much faster and much less intrusive.

Something like a 7+1 RAID-5 set would reduce the redundancy cost overhead and still smash the hell out of any hard disk based setup in terms of IOPs and latency.

> around 2 1/2 years

Correct. It means that half the units will have a failure in less than this time and a quarter

of them would have a failure in 15 months. To run one of these in a production environment

without having a fallback that didn't rely on any of this array's components, (and why else

would you pay such a large amount of cash for this quantity of storage if not for a front-line

production system) would be madness and / or negligence.

Steven,

The update-in-place nature of RAID-5/6 striping is about the worst kind of poisonous workload for NAND-Flash

The RAID write-penalty is unacceptable on SSD because it turns every write operation into N+1 write operations, meaning (a) you'll wear out your Flash N+1 times faster, and (b) Flash write performance is awful when you turn off the DRAM write buffer, which MUST be done for any kind of parity-based RAID.

If you lose a cached write in the RAID-5 or RAID-6 scenario, you've corrupted your data -- and will probably never know it until you try to rebuild.

Furthermore, Flash writes are so slow compared to reads that the on-disk write-buffers fill up fast, and array performance goes down the tubes. FYI, this is why IBM/STEC used mirroring on thier recent SPC-1 benchmark result.

If you want to test this yourself, build a 4+1 RAID set on Intel X25-E SLC Flash and let it run for a bit against IOmeter with a 50/50 Read/Write workload. Leave the write-cache enabled.

After a couple of minutes the SSDs are performing like cheap SATA disks.

Then, write a big zip-file out to the array and pull the plug on it just after the copy completes. Pull a disk, power up the array again and test the zip file...you will find garbage for data.

everyone agrees on that. Even the competitors.

If you use SSDs as a cache to a ZFS disk solution, then ZFS will correct errors in the SSDs. And also, the SSDs needs to "warm up", meaning that the cache needs to be populated with random data. If there are problems in the SSD, the ZFS will automatically get the correct data from disk instead and just overwrite the SSD data.

The SSDs are not important for the drives, SSDs will never write wrong data to the discs. I believe you can just pull the plug for the SSDs without any problems. You will just loose a huge performance boost. The SSDs are just an addon boosting performance.