click to visit StorageSearch.com home page
leading the way to the new storage frontier .....
top 10 SSD oems
top 20 SSD companies ..
click to see the collection of  SSD reliability articles here on StorageSearch.com
SSD reliability ..
SSD SoCs controllers
SSD controller chips
.....

Data Integrity Challenges in flash SSD Design

...

Editor's intro:- August 2013 - This page includes a definitive classic article on the theme of data integrity in SSDs and has been updated with many additional links and mini-articles on the same subject.

.
Data Integrity Challenges in flash SSD Design

by Kent Smith Senior Director, Product Marketing, SandForce

(this article was published here in October 2009)
....  SandForce logo - click to see a profile of SandForce

Solid state drives (SSDs) are similar to hard disk drives (HDDs) because both are storage devices, but due to differences in the way data is stored, managed and retrieved on each type of device, there must be a different approach to how data integrity is maintained on SSDs. While the need for a different approach to data integrity began appearing with the first memory-based SSDs, the impending release of commodity flash-based drives makes it more important than ever to implement new methods to ensure data integrity.

In this paper, we will examine the types of data integrity used in SSDs, and how the requirements for data integrity change as commodity flash memory is used in broader and more demanding enterprise storage applications.

HDDs versus SDDs

HDDs write data to free sectors and then read it from the same sector as needed. If the data is changed, the HDD controller simply overwrites the original sector, assigning a new CRC value as it is done. SSDs, on the other hand, must always erase blocks before storing new information, and they always use a newly erased block to store changed data, leaving the old data untouched in the old block. Because of this, address translation and data versioning techniques are essential to prevent the drive controller from returning stale data, e.g. data that was correct originally but has since been updated.

Given these differences in the way data is stored and changed, the designer must adapt data integrity techniques that are properly suited to each type of device. In the case of the SSD you need a much more encompassing form of protection.

Data Integrity Techniques for SSDs

The 4 methods that can be used to maintain data integrity in enterprise-class flash SSDs are:
  • Error Correction Code (ECC), which protects against read errors as a result of hardware errors in the flash memory. The drive controller monitors the read process and is able to correct hardware read errors up to a certain level. When successful the ECC will enable the drive to provide the correct data back to the user.
  • Cyclic Redundancy Check (CRC), which provides "end-to-end" protection by ensuring that the data written is the same data returned when read. As data comes in over the interface, the drive controller generates a CRC value and embeds it with the file's other metadata. When data is retrieved, the controller checks to ensure that the proper CRC value is present. However if the data does not match, CRC can only identify that an error has occurred. It cannot correct it, but it does prevent "silent data corruption."
  • Correct Address Translation via Logical Block Address (LBA), which ensures that the data is retrieved from the correct location. On an HDD this is the physical sectors on the rotating media. On an SSD this is the logical mapping of the flash memory blocks.
  • Correct Version of Data, which is used in SSDs to ensure that the current version of data is returned, rather than the stale version. This is not a problem for HDDs since they can directly overwrite the older data.
ECC

Uncorrectable Bit Error Rates (UBER) occur when a sector (HDD) or flash block or page (SSD) goes bad or there is an error reading back the data from the media. In high-end enterprise hard drives, there is less than one sector error in 1016 bits read. These rates are acceptable for hard drives because their peak data transfer performance is generally 50-100Mbps. Even if the drive runs with a typical duty cycle, this means there will be an uncorrectable sector error only once every year or so. But in an SSD, the data transfer performance is much higher at up to 250Mbps, so the UBER must be even higher - 1017 - to ensure comparable reliability.

ECC techniques are well-defined (typically using BCH or Reed Solomon algorithms), but SSDs need more sophisticated ECC protection. Each flash memory maker requires a different amount of correction depending upon the type of Flash memory and the process technology involved. As the process technology shrinks, the potential for errors increases. With SSDs, therefore, ECC must become stronger to support the increased transfer rate over HDDs.

In addition, designers must consider that ECC uses processing time and processing power on the drive controller. To optimize SSD performance, the designer must use an ECC scheme that is appropriate to the Flash memory used (sometimes spanning 4-bit to 16-bit protection). MLC flash-based SSDs will need a stronger ECC technique than SLC flash-based SSDs. The SSD controller must be also flexible enough to handle more complex ECC algorithms.

CRC

The rotating magnetic media (HDD) or flash memory (SSD) itself can introduce errors that are protected by ECC. But data passes through on-board RAM, firmware, and other elements of the drive, where a hardware or software error may cause the drive to modify the user data before the drive writes it to memory or after it reads it and sends it to the user. This is very important because these silent errors change data without the user's knowledge.

Imagine a silent error occurs in a financial firm's history log of a large stock trade. This could cost the company billions of dollars and they wouldn't be aware of any data corruption until an audit occurred, often weeks or months after the fact. This would be unacceptable, so the drive must be made aware of these "silent corruption" errors. CRC helps prevent these occurrences from happening. If the CRC is generated and stored with the user data, a simple CRC check can determine if the data about to be provided to the user matches what was originally stored. If the CRC test fails, the drive can report there was an error. It will not be able to correct the error, but it will prevent sending back wrong data without telling the user of the problem. The only way to correct such an error would be to incorporate the drive in a RAID array where the other drives would be used to regenerate the corrupted data.

LBA and Correct Data Version

Just using ECC and CRC is a particularly effective technique with HDDs because it is seldom necessary to ensure that the data address is accurate or that the data address has been translated properly. However an SSD cannot directly overwrite data in the flash memory. A process is required where the block to be modified is read by the controller, the data to be changed is modified in the buffer, and finally the entire new block is written to a new location in the flash memory. The old "stale" data is left in place.

This process requires the SSD to have a way to translate the request from the host for a particular Logical Block Address (LBA) into the correct physical location in the flash memory. This is not an issue on a hard drive (except for the few mapped out bad sectors), but it is a continuing problem with SSDs because of the need for SSDs to erase a block before writing to it. Multiple versions of data are invariably created on an SSD.

The SSD's LBA table must be continually updated to properly identify the current location of the freshest data. The stale data doesn't contain a record of its being rewritten to a new block, so if the drive controller points to the old location by mistake, the drive will return stale data. The ECC will work properly on the stale data. The CRC will decode properly from what was originally saved. In the end, the end user will get data that the drive assumes is correct. Unfortunately that data is old and was likely updated at some point, so the user is not aware of this "silent corruption."

Currently the only way to validate that the drive does not have this silent corruption problem is with explicit testing. One method would be to run a test where the LBA itself and an incremental value are stored in that LBA. The host test system would ensure that the data returned contains the correct LBA and incremental value. If the wrong value is returned the host would know the drive failed the test.

This problem can also occur if there is a power failure while the LBA is being updated, the table may contain only information for the data that is now stale. To prevent errors like this, designers can use on-board capacitors (SuperCaps) to provide temporary power during a failure so that the buffers can be flushed and the LBA table properly updated.

Lastly, there is the issue of silent corruption due to firmware bugs. The only way to guard against firmware bugs is through extensive directed testing so that products are known to be bug-free before being put into production.

Conclusion

The advantages of SSDs outweigh any challenges that impact their practical implementation in enterprise storage systems. HDD manufacturers and enterprise storage system providers have already begun to offer SSD-based storage systems, but now they are beginning to deliver systems that use commodity flash, which is denser, and has higher error rates. To ensure that MLC flash-based enterprise SSDs deliver the same or better data integrity protection as HDDs, designers must take into account the differences in the way data is written, changed, and retrieved from these devices, and they must ensure that the SSD controller has appropriate data integrity protection techniques in place.

more articles and whitepapers related to SSD data integrity

adaptive R/W flash care management IP (including DSP ECC) for SSDs - what is it? and who does it?

A Comparison of Client and Enterprise SSD Data Path Protection (pdf) - this paper by Micron describes how the company uses different strengths of end to end data integrity in SATA SSDs designed for enterprise markets and those for consumer systems.

the new SSD uncertainty principle - resetting the assumptions about enterprise MLC flash - introduction to STEC's Cellcare technology.

flash wars in the enterprise - Timeline of how flash SSDs got into the enterprise - and the market rivalry among different types of flash and related controllers (from SLC to TLC).

SSD software, SSD news, popular SSD articles
.
Error Correction in MLC Flash SSD Arrays
Editor:- October 28, 2009 - ECC Technologies has published a new article which examines data reliability issues in RAID systems using MLC flash.

In his survey of RAID and error correction related to SSDs the author Phil White said he thinks that "MLC NAND Flash memories should implement nonbinary error-correcting codes such as a Reed-Solomon (RS) codes so that all of the bits from one cell are in one symbol. The communications industry has been doing that for decades, but the Flash industry has been implementing a scheme that forces the bits from one cell to be in separate records (pages) so that one cell failure can cause multiple binary symbol failures which seems illogical."

I asked him to expand on this for our readers.

In reply - Phil said he doesn't think that most NAND Flash (SSD) companies have a high level of expertise in the field of error-correcting codes.

"Many of the NAND Flash controllers that are out in the market now have ECC Tek's ECC designs in them. None of the controller companies who have come to us have any idea how to implement binary BCH encoders and decoders in hardware. I doubt if any of the Flash manufacturers have that expertise either."

"For years the Flash manufacturers implemented a simple binary scheme that corrected only 1 bit in a page. I don't have evidence to prove this, but I believe the NAND Flash manufacturers simply decided to extend their original scheme to correct N bits in instead of 1 bit to handle higher error rate devices. I also believe that they implemented a scheme for MLC NAND Flash to "randomize" the errors when a cell fails.

"Consider 4-bits/cell. When a cell fails, 0-4 bits may be in error. In order to keep using binary error-correcting codes that only correct bits, they designed the chips so that all of the bits from that cell are in different pages.

"To the best of my knowledge, they never considered using RS codes so that all of the bits from one cell are in one RS symbol. For example, assume a RS code with 12-bit symbols. Each RS symbol can hold the data from three 4-bit cells, and if those three cells happen to fail, it will only corrupt one RS symbol. RS codes can correct t "symbol" errors and s "symbol" erasures as long as 2t+s is less than or equal to R where R is the number of "symbols" of redundancy. The most natural and powerful thing to do is to put all of the bits from one cell in one RS symbol."

storage search banner

SSD SoCs - on  storagesearch.com testing the sensitivity
of disturb errors
..
click here for more info about the Guardian SSD
highest integrity 2.5" military SATA SSDs
with SnapPurge and AES-256 encryption
TRRUST-STOR - from Microsemi
.
SSD jargon
SSD Reliability
SSD endurance
the top 20 SSD companies
SSD Data Recovery Concepts
Wear leveling and flash reliability
Surviving SSD sudden power loss
how fast can your SSD run backwards?
Does size really does matter in SSD architecture?
principles of bad block management in flash SSDs
Adaptive R/W flash care management & DSP in SSDs
.
SSD ad - click for more info
.
LSI says it pays to get a 2nd opinion from LDPC
Editor:- August 13, 2013 - in a presentation today at at the Flash Memory Summit - the Nibbles and Bits of SSD Data Integrity (pdf) - LSI explained why reserving the use of LDPC to deal mostly with read error retries (and also later in the operating life of flash cells) can be a pragmatic design choice.

And instead of applying different strengths of ECC for fixed physical block sizes - the company says another approach is to have variable sized virtual blocks - which effectively means that better cells carry lower ECC overhead.

See also:- my SSD cell care scheme is better than yours
.
"In some ways, blocks lost due to media corruption present a problem similar to recovering deleted files. If it is detected quickly enough, user analysis can be done on the cyclical journal file, and this might help determine the previous state of the file system metadata. Information about the previous state can then be used to create a replacement for that block, effectively restoring a file."
Why CRCs are important - blog by Thom Denholm Datalight (January 2013)
.
Fusion-io fast SSDs - click for more info
the standard for enterprise PCIe SSDs
by which all others are judged
ioDrives from Fusion-io
.
Intel paper - Data Integrity on 20nm flash SSDs
Editor:- August 22, 2012 - "Avoid skepticism and seek understanding" - is one of the calls to action in Intel's paper - Data Integrity on 20nm SSDs (pdf) - presented today at the Flash Memory Summit

In a bold move at the start, the author - Robert Frickey - brings to the fore the subject of flaky SSDs and firmware bugs and recalls - naming several SSD vendors in this context - including Intel.

He says "Despite datasheet metrics, it's not easy to predict behavior of SSDs in the field. Validation should be considered as part of data integrity."

Even if you've already read many other articles on SSD data integrity - this paper clearly communicates some fundamentals about flash cells and the variety of different types of disturb errors which makes it a useful educational document.

In tone with what some other leading SSD companies are saying too - the author urges you to "Understand your usage model and endurance requirements. Innovate around application needs." ...read the article (pdf)
.
image shows Z-Drive R4 f- one of the world's fastest PCIe SSDs -  designed by OC
bootable virtualized PCIe SSDs
the Z-Drive R4
from OCZ
.
New Error Technologies Required to Scale MLC SSDs
Editor:- January 18, 2010 - Forward Insights publishes a new market report this month - ECC and Signal Processing Technology for SSDs and Multi-bit per cell NAND Flash Memories.

Bit errors are becoming more severe as NAND flash memory scales below 40nm process technology and transitions to 3-bit and 4-bit per cell architectures. Increased ECC requirements will be required, however, traditional error correction codes such as BCH, RS and Hamming code suffer from increased overhead in terms of coding redundancy and read latency as the number of errors corrected increases. In addition, the number of electrons stored in the memory cell is decreasing with each generation of flash memory resulting in reduced signal/noise requiring enhanced sensing techniques.

Digital signal processing technology has been employed in the magnetic recording industry since the early 1990's when partial-response maximum-likelihood technology (PRML) was commercialized. DSP technology is now being deployed in 3-bit per cell and 4-bit per cell NAND flash memories and a concerted effort is being made by NAND flash manufacturers and a variety of startups to employ digital signal processing technology to improve the endurance and performance of next generation NAND flash memories and SSDs. Signal processing technology will be essential for the continued scaling of NAND flash memories.

This research report examines the current state of ECC methods and explores the technology, roadmap, market, cost and competitive landscape in the flash signal processing space.
.
.
SSD ad - click for more info
.
SSD ad - click for more info