this is the home page of  StorageSearch.com
leading the way to the new storage frontier .....
SSD symmetries article
SSD symmetries ..
click to see the collection of  SSD reliability articles here on StorageSearch.com
SSD reliability ..
SSD SoCs controllers
SSD controllers ..
image shows Megabyte reading a scroll - the top 50 SSD articles and recommended SSD blogs
SSD articles .....
.....
Adaptive flash care management & DSP IP in SSDs

What is it? Who does it? and why?

by Zsolt Kerekes, editor - June 19, 2012
The phrases "adaptive writes", "DSP IP in flash SSD" and "adaptive flash cell care" have appeared at various times in past SSD news stories, interviews and comments.

And while I sketched out as much explanation as was needed for each story at the time - I did promise I would eventually publish a list of SSD companies who use what I loosely called back then "adaptive DSP technologies in SSD IP" in their new designs along with a technology and market guide.

This article is it.

A year ago there were only 4 or 5 companies doing this kind of thing - but I could see this technology trend was creeping into double digits of companies and when that happens in the SSD market you know that something significant is going on.

Why are these new adaptive technologies important?

They change the rules about what SSD designers can do with cheap consumer grade MLC and TLC (x3) flash.

As I said in an earlier article - they reset the commonly held assumptions about the limitations of endurance - but that's just a small part of the new effect.

Here's a summary of what the new technologies enable in different segments of the SSD market
  • industrial SSDs - lower price, and greater density - because MLC can replace SLC in most applications
  • enterprise SSDs - changes the economics in fast-enough SSDs - because consumer grade MLC in the new SSDs can last as long as more expensive eMLC.

    But another byproduct of the new technologies is that SSDs which use it can also have significantly faster write cycles (greater IOPS) - and operate at lower power
Each of these bullet points represents a potential competitive advantage for the early adopters of these new technologies in whichever SSD segment they are applied. But the bag of magic tricks can be used to provide different characteristics for the different markets.

Currently only a small percentage of SSD makers deploy these technologies. In fact each of the many companies I spoke to about this in recent months believed they were just one of a handful of companies doing anything similar.

I told them how wrong they were and that they might be surprised when I published this article.

Driving the need to develop these extraordinarily complex SSD technologies is the certain knowledge and fear that traditional SSD controller IP will fail to deliver working SSDs with future shrinks of flash geometry.

That means a successful SSD company in generation X flash may wake up to be told by its designers that its SSDs using generation Z flash - can't be made to operate at all - let alone last for 1, 2 or 5 years. (Time for the company's VPs to create / update their bios on Linkedin.)

On the one hand - companies which have already got the new SSD magic wands - such as STEC - told me over a year ago they are confident that they may see an upturn in their businesses when their competitors (without adaptive designs) start to fall off the new flash technology cliff.

On the other hand - with the SSD market share prizes getting bigger - it's more likely that enterprising SSD IP companies will step in to sell their own maps of safe ways around the cliff - and that's the business plan of DensBits.

But I'm running ahead of myself now by mentioning companies.

Before I get onto that - it's worth asking - What's at the core of the new SSD adaptive technologies?

The simple answer has 3 parts
  • adapting the write pulse energy in the flash memory to be as low as can be - while at the same time providing usable data integrity with the attached ECC / DSP technology
  • designing a set of ECC / DSP technology which can provide usable data integrity.

    Unlike traditional SSD designs - the ECC/ DSP strength and actual choice of algorithms can be adapted to suit the flash memory according the circumstances.

    That means the same memory block may have different ECC codes wrapped around it at different times in its operating life - depending how healthy it looks. And different ECC codes may be used within the same memory chip at the same time.
  • Other techniques include adapting the spatial distribution of data - and making the data striping plans more flexible than traditional designs.
None of these individual design philosophies is entirely new.
  • Systems designers like me working with the first generation of flash chips in the early 1980s could see clearly that some locations were harder to program than others - because the write pulses were an external user designed circuit and software algorithm.

    In later generations of flash - the chipmakers embedded the write pulse circuits inside the memory chips to make it easier for digital designers to use flash - and to reduce the risk of systems designers over cooking the flash. So the awareness of this parameter may have gone away for systems designers - but it was always an important part of the memory chip design.
  • Adapting to the reliability population curve of flash memory too - with different controller technologies like wear-leveling and bad block management goes back to the early 1990s and a company called M-Systems.
  • Spatial scattering of data across multiple memory chips in SSDs with RAID goes back to the 1990s too and a company called Solid Data Systems.

    Variability in this parameter (variable size stripes) has been used in 2 generations of SSD designs already by Texas Memory Systems.
What is new about the new adaptive SSD flash care management IP is that instead of each of these parameters and design rules being fixed at the time of manufacturing the SSD according to an idea of what works best for the population of flash chips in this current generation - as with traditional controller designs - the new adaptive SSDs have smarter technologies which can each dynamically interact and learn from the chips they're connected to.

You don't have to understand the internal details of how these individual techniques work.

And with hundreds of patents already pending in this topic there's a high probability that the SSD vendor won't give you the details anyway (not even under NDA) and even if you are yourself among the rare set of people on the planet with the educational background to understand them.

It's enough to get the general idea.

Now as we're talking about the SSD market here - you don't really expect that anything - will stay clear cut for long. And so you won't be surprised to know that it isn't.

Earlier this year for example - Smart Storage launched SSDs which used the knowledge learned from tweaking its adaptive SSD controllers and then reapplied this back as a set of fixed paramters to precondition flash chips so they would run better with traditional unmodified controllers from LSI/SandForce. You can think of this as presetting the write pulse parameters in the memory chips with a better set of magic numbers than even the memory chip makers or SandForce themselves would have come up with on their own.

When that product was launched in April 2012 - I said ""SMART's trick with the SandForce controllers is like using Dolby correction with a 1980s cassette tape. Whereas SMART's trick with its Optimus controller is like having a built-in dynamic sound equalizer."

I doubt if that's the last we'll hear about hybridizing some of the IP knowledge acquired from developing these newer technologies and then reapplying them back into earlier designs to stretch their market life.

I promised you some kind of list of companies who are using these new adaptive technologies inside their SSD designs - so here goes.

The list below is my first draft - and I'll expand it next month with more detail when this article moves off the home page and gets its own permalink.

In the meantime - SSD companies which aren't mentioned below - but who think they should be - can contact me with details of supporting evidence about what they're doing in this area. (I know there are some companies in stealth mode - both the deliberate and accidental kind.)

Anobit (acquired by Apple)

DensBits (chosen by Seagate)

InnoDisk

Link_A_Media Devices (acquired by SK Hynix)

LSI

Memoright

Micron

OCZ (Indilinx)

PMC

Proton Digital Systems

SanDisk

Skyera

SMART

STEC

XLC Disk

My preliminary list above and the expanded list later is only going to include companies which have developed their own SSD controllers which use these new adaptive techniques - and not companies which simply license IP from a DSP IP controller company.

For more (serious) articles on this theme take a look at the SSD reliability papers, SSD controllers page - and for a lighter and more whimsical view of this aspect of the SSD market today you might want to see how to choose a flash health care scheme to make your SSD last longer .
....
even more stuff about adaptive R/W and DSP IP in SSD technology

Editor:- June 28, 2012 - after publishing the article above - I realized that some of you would want to know a lot more about the subject than I know or have time to write about about.

I knew that out of all the specialist SSD market analysts - the person who knows most about this aspect of the market and technology was likely to be Gregory Wong founder of Forward Insights.

So I asked Greg - what did he think about my article? And could he tell me which of his many SSD market reports is closest to this theme? and how much does it cost?

Gregory Wong said - "I think you did a pretty good job outlining these technologies. I believe the companies are mainly talking about advanced ECC and flash signal processing."

Gregory gave me the following useful add-on to this article.

For advanced ECC and flash signal processing, you need 3 things: NAND statistics, soft information and advanced decoding.

NAND statistics collection constructs the history of the NAND flash memory cell characteristics and facilitates the estimation of the reliability of each bit. To obtain soft information from the memory cell, extra read commands or test mode sequences are required. These commands are proprietary to the NAND flash manufacturer and a vendor implementing DSP would require the NAND flash manufacturer to provide these commands.

Needless to say, not all controller/SSD vendors will obtain this support.

Advanced decoding schemes employing soft decoding use the NAND statistics and soft information to determine the most probable read signal that corresponds to the actual stored data.

This allows you to obtain readable data even when the memory cell is severely degraded or there is a lot of 'noise' in cell data.

That is why you see companies like Anobit and Densbits claiming a 10x improvement in endurance. STEC and Smart Storage also claim to have similar technology.

Editor:- Greg said that his report - ECC and Signal Processing Technology for SSDs and Multi-bit per cell NAND Flash Memories 2nd Edition - which costs $6,500 - has been his best selling SSD report.
.
Since first publishing this article - as a blog on the home page of StorageSearch.com - the increasing importance of this topic has been reinforced by its many appearances as a strategic thread in these later articles.


.
"One petabyte of enterprise SSD could replace 10 to 50 petabytes of raw HDD storage in the enterprise - and still run all the apps faster and at lower cost."
meet Ken and the SSD event horizon


.
SSD ad - click for more info
.

storage search banner

....
DSP IP in SSDs - image shows Megabyte the mouse  measuring stuff
As every SSDmouse knows - measuring stuff and adapting to what you know gives you safer operating speed, better reliability and lower TCO.


.....
summary

In the future - all nand flash SSDs will have to use adaptive R/W and DSP ECC IP technologies in their controller schemes in order to be able to use newer generations of denser flash memory. Among other things these adaptive R/W techniques can magnify reliability and performance while improving SSD design efficiency and reducing cost.

As we go through the transition years - all the safe assumptions which you thought you knew about flash SSDs and suppliers will change (again).


.....
SSD ad - click for more info


.....
"The variability of the LDPC decode time is a function of how many iterations it takes to decode the data from the flash and can be upto 20 microseconds."
the latency implications of DSP ECC (May 15, 2014)


.....
4 years ago - in - SSD market history
Anobit samples 1st Memory Signal Processing flash SSDs

Editor:- June 15, 2010 - Anobit announced it is sampling SSDs based on its patented Memory Signal Processing technology which provide 20x improvement in operational life for MLC SSDs in high IOPS server environments.

Based on proprietary algorithms that compensate for the physical limitations of NAND flash, Anobit's technology (a variation of adaptive R/W and DSP ECC) extends standard MLC endurance from approximately 3K read/write cycles to over 50K cycles - to make MLC technology suitable for high-duty cycle applications.

This guarantees drive write endurance of 10 full disk writes per day, for 5 years.


.
variable pulse shapes in erase
Above - Erase Pulse Control - NAND Reliability Improvement with Controller Assisted Algorithms in SSD (pdf) - a paper by SK hynix at the Flash memory Summit (August 2013)


.
"How long before we get to clinical trials?"
...from - flash care schemes - will Brand X flash care make your SSD live longer? (Brand Y has better tv ads.)


.
LSI SandForce SSD processors - click for more info
the awards winning silicon
accelerating world's leading SSDs
from LSI


.
How adaptive is the SSD behavior to changes within itself?

All SSDs rely on processing data about the quality of the memory as part of their normal data integrity operations.

They wouldn't work without it.

But some companies have SSD IP sets in which knowledge about different parts of the SSD can be optimized and fed back to control and enhance SSD functionality over and beyond the standard accepted SSD function block boundaries.

The degree to which this passing of the intelligence (regarding the state of past and future anticipatable data flows, priorities of the application and the flash array's own readiness and healthiness condition) can impact behavior in other parts of the SSD - is what I call adaptive intelligence flow symmetry.
11 Key Symmetries in SSD design
.
LSI says it pays to get a 2nd opinion from LDPC
Editor:- August 13, 2013 - in a presentation today at at the Flash Memory Summit - the Nibbles and Bits of SSD Data Integrity (pdf) - LSI explained why reserving the use of LDPC to deal mostly with read error retries (and also later in the operating life of flash cells) can be a pragmatic design choice.

And instead of applying different strengths of ECC for fixed physical block sizes - the company says another approach is to have variable sized virtual blocks - which effectively means that better cells carry lower ECC overhead.

...Later:- In November 2013 - LSI began sampling the SF3700 SSD controller (pdf) - which included elements of adaptive DSP in its design as well as the unique ability to be configured as either a small architecture or large architecture controller.


.
"despite the bewildering range of products in the market - the performance characteristics and limitations of ALL flash SSDs are determined by a small set of of architectural parameters."
a toolkit for understanding flash SSD performance characteristics and limitations


.
"What are the main types of SSDs that we'll see in the future enterprise SSD market?...."
an introduction to enterprise SSD silos


.
SSD ad - click for more info


.
How big was the thinking in this SSD's design?
Does size really does matter in SSD design?

By that I mean how big was the mental map? - not how many inches wide is the SSD.

The novel and the short story both have their place in literature and the pages look exactly the same. But you know from experience which works best in different situations and why.

When it comes to SSDs - Big versus Small SSD architecture - is something which was in the designer's mind. Even if they didn't think about it that way at the time.
click to read the article - Big versus Small SSD  architectures For designers, integrators, end users and investors alike - understanding what follows from these simple choices predicts a lot of important consequences. ...read the article


.
flash SSD capacity - the iceberg syndrome
Have you ever wondered how the amount of flash inside a flash SSD compares to the capacity shown on the invoice?

What you see isn't always what you get.
nothing surprised the penguins - click to read  the article There can be huge variations in different designs as vendors leverage invisible internal capacity to tweak key performance and reliability parameters. ...read the article


.
SSD ad - click for more info
.
recent SSD articles

DSP in SSDs

2.5" PCIe SSDs

Consumer SSDs

the Top SSD Companies

Enterprise SSD market silos

What do enterprise SSD users want?

Where are we now with SSD software?

How fast can your SSD run backwards?

Baldness cures, diets and SSD longevity

What makes this enterprise SSD different?

Efficiency as a key internecine SSD competitive factor


.
SSD ad - click for more info


.
Surviving SSD sudden power loss
Why should you care what happens in an SSD when the power goes down?

This article will help you understand why some SSDs which (work perfectly well in one type of application) might fail in others... even when the changes in the operational environment appear to be negligible.
image shows Megabyte's hot air balloon - click to read the article SSD power down architectures and acharacteristics If you thought endurance was the end of the SSD reliability story - think again. ...read the article


.


.