SSDs are a new phenomenon in the datacenter. We have theories about how they should perform, but until now, little data. That's just changed.

The FAST 2016 paper Flash Reliability in Production: The Expected and the Unexpected, (the paper is not available online until Friday) by Professor Bianca Schroeder of the University of Toronto, and Raghav Lagisetty and Arif Merchant of Google, covers:

• Millions of drive days over 6 years
• 10 different drive models
• 3 different flash types: MLC, eMLC and SLC
• Enterprise and consumer drives

KEY CONCLUSIONS

• Ignore Uncorrectable Bit Error Rate (UBER) specs. A meaningless number.
• Good news: Raw Bit Error Rate (RBER) increases slower than expected from wearout and is not correlated with UBER or other failures.
• High-end SLC drives are no more reliable that MLC drives.
• Bad news: SSDs fail at a lower rate than disks, but UBER rate is higher (see below for what this means).
• SSD age, not usage, affects reliability.
• Bad blocks in new SSDs are common, and drives with a large number of bad blocks are much more likely to lose hundreds of other blocks, most likely due to die or chip failure.
• 30-80 percent of SSDs develop at least one bad block and 2-7 percent develop at least one bad chip in the first four years of deployment.

THE STORAGE BITS TAKE

Two standout conclusions from the study. First, that MLC drives are as reliable as the more costly SLC "enteprise" drives. This mirrors hard drive experience, where consumer SATA drives have been found to be as reliable as expensive SAS and Fibre Channel drives.

One of the major reasons that "enterprise" SSDs are more expensive is due to greater over-provisioning. SSDs are over-provisioned for two main reasons: to allow for ample bad block replacement caused by flash wearout; and, to ensure that garbage collection does not cause write slowdowns.

The paper's second major conclusion, that age, not use, correlates with increasing error rates, means that over-provisioning for fear of flash wearout is not needed. None of the drives in the study came anywhere near their write limits, even the 3,000 writes specified for the MLC drives.

But it isn't all good news. SSD UBER rates are higher than disk rates, which means that backing up SSDs is even more important than it is with disks. The SSD is less likely to fail during its normal life, but more likely to lose data.

Bad news: SSDs fail at a lower rate than disks, but UBER rate is higher
Ignore Uncorrectable Bit Error Rate (UBER) specs. A meaningless number.

We study a wide
range of reliability characteristics and come to a number
of unexpected conclusions. For example, raw bit error
rates (RBER) grow at a much slower rate with wear-out
than the exponential rate commonly assumed and, more
importantly, they are not predictive of uncorrectable errors
or other error modes. The widely used metric UBER
(uncorrectable bit error rate) is not a meaningful metric,
since we see no correlation between the number of reads
and the number of uncorrectable errors. We see no evidence
that higher-end SLC drives are more reliable than
MLC drives within typical drive lifetimes. Comparing
with traditional hard disk drives, flash drives have a significantly
lower replacement rate in the field, however,
they have a higher rate of uncorrectable errors.

5.1 Why UBER is meaningless

The standard metric used to report uncorrectable errors
is UBER, i.e. the number of uncorrectable bit errors per
total number of bits read. This metric makes the implicit
assumption that the number of uncorrectable errors
is in some way tied to the number of bits read, and hence
should be normalized by this number.
This assumption makes sense for correctable errors,
where we find that the number of errors observed in a
given month is strongly correlated with the number of
reads in the same time period (Spearman correlation coefficient
larger than 0.9). The reason for this strong correlation
is that one corrupted bit, as long as it is correctable
by ECC, will continue to increase the error count
with every read that accesses it, since the value of the
cell holding the corrupted bit is not immediately corrected
upon detection of the error (drives only periodically
rewrite pages with corrupted bits).
The same assumption does not hold for uncorrectable
errors. An uncorrectable error will remove the affected
block from further usage, so once encountered it will
not continue to contribute to error counts in the future.
To formally validate this intuition, we used a variety of
metrics to measure the relationship between the number
of reads in a given drive month and the number
of uncorrectable errors in the same time period, including
different correlation coefficients (Pearson, Spearman,
Kendall) as well as visual inspection. In addition to the
number of uncorrectable errors, we also looked at the incidence
of uncorrectable errors (e.g. the probability that
a drive will have at least one within a certain time period)
and their correlation with read operations.
We find no evidence for a correlation between the
number of reads and the number of uncorrectable errors.
The correlation coefficients are below 0.02 for all drive
models, and graphical inspection shows no higher UE
counts when there are more read operations.
As we will see in Section 5.4, also write and erase operations
are uncorrelated with uncorrectable errors, so an
alternative definition of UBER, which would normalize
by write or erase operations instead of read operations,
would not be any more meaningful either.
We therefore conclude that UBER is not a meaningful
metric, except maybe in controlled environments where
the number of read operations is set by the experimenter.
If used as a metric in the field, UBER will artificially
decrease the error rates for drives with high read count
and artificially inflate the rates for drives with low read
counts, as UEs occur independently of the number of
reads.

With larger capacity and higher endurance drives, some vendors quote PBW figures. This is simply the endurance expressed as petabytes written. For example, the Intel DC P4600 Series 2TB SSD has an endurance of 11.08 PBW, which translates to a DWPD of 3. The warranty of an SSD is typically expressed as either a time period or the point at which the absolute capacity has been written. Taking another example, the Samsung 850 Pro SATA III 1TB SSD has a warranty of 10 years or 300 TBW. Over the 10-year lifetime, this means a DWPD of 0.08, which seems quite low. With a 5-year period, this value naturally doubles to 0.16. The manufacturer will guarantee the drive to either the capacity written or time the drive has been in use, which ever comes first. This means DWPD is more of a useful guide than a warrantied specification.