this is the home page of StorageSearch.com
leading the way to the new storage frontier .....
DIMM wars - expect more
can memory do more? ....
the fastest SSDs - click to read article
the fastest SSDs ..
storage security articles and news
SSD security ..
SSD SoCs controllers
SSD controllers ..
click to see more SSD Bookmarks
the SSD Bookmarks ..
..

SSD analyzersunderstanding flash SSD
performance characteristics
and limitations - a toolkit

by Zsolt Kerekes, editor - StorageSearch.com - March 15, 2013
If you spend a lot of your time analyzing the performance characteristics and limitations of flash SSDs - this article will help you to easily predict the characteristics of any new SSDs you encounter - by leveraging the knowledge you already have and by re-interpreting all SSDs within a simple framework built around a handful of design-related concepts.

I've written about these design concepts before - and I often describe particular SSDs using these terms.

What's new about this article is that it pulls together a list of these useful ideas into a single overview for the first time.

SSD ad - click for more infoSSD design diversity

One of the interesting things about the SSD market today is the rich diversity of designs - especially in the high value enterprise market.

It means that inside the SSD there are many differences of design approach to implement them - especially in SSDs associated with big market opportunities which can leverage those differences - such as PCIe SSDs, rackmount SSDs, SAS SSDs and even in simpler products such as mission critical SATA SSDs.

But despite the bewildering range of products in the market - the performance characteristics and limitations of ALL flash SSDs are determined by a small set of of architectural parameters.

Each of these design choices predetermines some aspects of the SSD's behavior - making it easier for the designer to do some things - compared to using the alternate choice.

It's the sum effect of all those choices - that makes some SSDs better suited to some apps than others.

But - and here's the interesting thing - it's possible for competing SSDs to be viable in the market at the same time - even when those SSDs are based on a completely different permutation of these raw design ingredients.

The more you study the characteristics of different SSDs - the quicker and more easily you will start to anticipate useful behavioral characteristics of any new SSD - and assimilate new SSDs in your plans.

And you'll start to recognize symptoms of "missing technical information" too.

These are things which it's important for you to know - but which don't appear in the initial info you see about the new SSD.

In some cases you can get this missing info by asking the vendor. In other cases -when they don't understand your questions - or aren't willing to co-operate without an NDA - you may still be able to infer or deduce the missing data aspects from other things you already know.

And if you like SSDs which behave a certain way - then these descriptors make it easy for you to group similar types of SSDs together - even if they used completely different controllers inside.

The outline descriptors for flash SSD performance and characteristics discussed below are:-
  • memory type
  • host interface
  • cache sizing
  • controller sizing
  • memory population characterizing model
  • SSD symmetries
  • SSD efficiencies
  • hard wired assumptions about the SSD environment
memory type

Before 2012 - knowing the flash memory type in an SSD (and whether it's SLC, eMLC, MLC or TLC) was very useful in anticipating the SSD's behavior.

However, recent advances in SSD controllers and SSD architecture mean that the performance and reliability of some memory types can be stretched - for example multiplying endurance by upto 100x, or multiplying speed - even when using identical memory with different flash management schemes.

Consequently knowledge about the internal memory type coupled with the SSD specification tells you more about how profitable the SSD is and how well (or not) the SSD vendor is at mastering current technologies than it tells you about the SSD's likely characteristics. Memory is no longer a reliable predeterminator of total SSD characteristics.

host interface

The host interface for the SSD predertermines the best latency and throughput which you can expect from the SSD.

Using the wrong type of interface can prejudice the performance of a high performance SSD - but the intrinsic characteristics and limitations of interfaces are outside the scope of this discussion.

For a rough guide of comparative interface headroom performance in an SSD context take a look at the fastest SSDs list.

For typical performance figures of industry standard SSDs using various interface types - take a look at the directory pages of SSDs which are classified by interface - in the SSD A to Z directory on StorageSearch.com

cache sizing

The ratio of RAM-like cache in the flash SSD can determine functions such as typical latency, how easy it is for the designer to deliver endurance, and the complexity of managing data integrity in the event of SSD sudden power loss.

From the performance point of view - it makes little difference whether the RAM memory is DRAM or another RAM-like non volatile memory - although this does impact power consumption, and card footprint and efficiency of the SSD.

The primary article which discusses SSD cache sizing is RAM Cache Ratios in flash SSDs.

A different consideration of SSD cache ratios - is how much fast SSD you might need need as a cache to serve another slower storage tier (whether it's hard drive based or simply uses slower SSDs). If you're looking for articles about that and related software - take a look at the articles in this primary article - the business case for SSD ASAPs.

controller sizing

All SSD controllers - whether they are implemented by ASICs, FPGAs or software or a combination of technologies - can be usefully divided into 2 main classifications
  • big controller SSD architecture
  • small controller SSD architecture
This classification predetermines how easy it is for the SSD designer to achieve efficiency, reliability and performance from the total number of flash memory chips in a single SSD drive - or scaled up in an array of flash SSDs.

The primary article which discusses this subject is - Why size matters in SSD architecture.

memory population characterizing model

Ever since the earliest use of flash memory to design SSDs for mission critical apps (initially in industrial and military markets more than a decade ago - and more recently - since 2004 in the enterprise) SSD controller designers have tried to leverage what they understand about the intrinsic limitations of flash - and manage those devices with SSD architectures which deliver reliability and performance which is orders of magnitude better than the raw memory could deliver if assembled into a simply managed array of chips.

But the cleverest SSD companies were doing much more that.

As I revealed in the original March 2007 edition of my article SSD Myths and Legends - "write endurance" - they weren't just relying on what a particular generation of flash memory could deliver - but they were actively filtering the best flash memories for their SSD designs by choosing suppliers and process batches to get the best raw material.

For the companies involved this meant that they were writing their own data sheets for flash memory based on their own testing of batches of devices - and by careful supply management and QA they were able to rely on raw endurance which was 10x or 20x better than the industry averages.

And many SSD companies also knew from their own experience with their own controller algorithms that some parts of flash chips were better than others.

Using a variety of techniques such as over-provisioning and wear-leveling could - in combination - extract much higher reliability than indicated by extrapolating pessimistic worst case interpretations of raw endurance specifications supplied by memory makers.

Before the 2010 SSD market bubble - the special needs of SSDs were too small for memory makers to bother their heads about. But as the SSD market became bigger - some memory makers picked up on some of these tricks - and learned they could charge more for their memory by doing some of the process selections themselves. That's how we got things like eMLC - which wasn't really a new type of memory at all.

Today in 2013 - every major memory maker also wants to be a major SSD company too - but the quality of raw flash memory is declining as cell sizes get smaller - which means SSD designers have to get even cleverer to turn flash into useful products.

There are 2 simple ways to characterize all flash care management controller schemes - with respect to how they deal with these most difficult aspects of interacting with raw flash.
  • big population memory characterizing model - which for simplicity I'll call the "standard model" - because it's how 90% or more of the controllers in the market work.

    In this group - the SSD controller uses magic numbers which relate to the characteristics of flash learned from large samples of memory (hundreds of thousands to millions of memory chips).

    The idea here is to get a design which provides safe and good enough performance and reliability based on what is known about these flash memory devices in general.
  • small population memory characterizing model - doesn't simply rely on what was learned about these flash memory devices back in the corporate labs - but can adapt critical parts of its behavior - such as the exact ECC type and the exact duration of a write programming pulse - based on what it learns from the flash it's connected to.

    The cleverest of these controller schemes can adapt their behavior between different parts of the same memory chip, and also adapt to ageing or other changes within the same piece of memory - and even spread data across good and less good parts of memory to spread bet best overall SSD outcomes.

    This is a dangerous technique for novice designers - because if you get it wrong - you can reduce rather than increase the life expectancy of the flash.

    But in the hands of flash SSD masters - it's a very powerful technique.
The primary article which discusses this in more detail is - Adaptive flash care management & DSP IP in SSDs

But in 2016 we saw details of a new characterization scheme which stretched the endurance capabilities of flash SSDs (with or without DSP ECC) by another 7x.

SSD symmetries

The root cause of much complex behavior in SSDs is more easily understood and managed when analyzed from the perspective of design symmetries.

All SSDs contain implicit asymmetries due to the intrinsic characteristics of their components and architecture.

Different applications have differing needs with respect to the symmetry of SSD behaviors.

The 11 key different types of SSD symmetry - and why they are important - are discussed in this primary article - how fast can your SSD run backwards?

SSD efficiencies

Why do some SSDs use many more memory chips than others to achieve the same end result - when seen from the perspective of the SSD's headline performance and usable capacity?

Sometimes the additional memory is there - because it makes the SSD better.

Other times the additional memory is there - because the designer didn't have access to the same IP.

Or - in the case of SSD arrays - the extra memory is the summed up consequence of architectural choices (like those listed earlier in this blog) which make more sense for some types of apps rather than others.

In these latter cases - the extra memory isn't buying you a better SSD. The extra memory simply costs more and consumes more electrical power.

Does memory efficiency matter?

And what are the trade-offs in cost, performance, reliability and risk?

The 2 primary articles which discuss these issues are:- philosophical assumptions about the SSD environment

There are some arguments for placing this at the top of the list - because it can have such an enormous multiplication factor on applications performance.

This is the top level set of philosophical assumptions around which the entire software operation of the SSD is based. (It reaches down into the firmware and details of the interface architecture too.)

Almost exactly the same raw SSD hardware - operating in the same server - can have apps performance which looks OK - or upto 20x faster than OK - depending on important design assumptions about the SSD environment.

The primary article which discusses this is - Legacy vs New Dynasty - the new way of looking at Enterprise SSDs

summary

A handful of simple ways of thinking about and compartmentalizing SSDs can help you filter important SSD characteristics and shortlist partners which are important to your type of SSD requirements.

In the blog above I've outlined the main tools which I find useful on a day to basis.

PS - If you've got the time - you might want to take a look at the most popular SSD articles seen by our readers in the last month or so.
.
SSD ad - click for more info
.

storage search banner

 ....
"In an ideal world - symmetry considerations would be on page 1 of the - how to design an SSD cookbook."
11 key symmetries in SSD design architecture

.
SSD ad - click for more info

..
"In the past decade we've seen that the only technology which could realistically threaten to displace an SSD from its established market role was yet another SSD."
the Modern Era of SSDs

....
DRAM latency and why flash can replace it - article

....
Have you ever wondered how the amount of flash inside an SSD compares to the usable capacity shown on the invoice? The internal invisible capacity can be hugely different...
usable flash SSD capacity - the iceberg syndrome
....
That step - when users make the switch to newer generations of software - means not only do they need less servers - but they don't need as many SSDs as they did in an earlier phase of SSD market adoption either.
meet Ken - and the enterprise SSD software event horizon
....
SSD ad - click for more info
....
"This resource page includes news and articles about low latency, fast flash SSDs which plug into DIMM sockets and transfer data via interfaces which were originally designed for DRAM.
Memory Channel SSDs
....
SSDserver rank is a latency based architecture metric for classifying any enterprise server from an SSD software perspective
what's in a number?
....
SSD ad - click for more info
....
"Don't believe everything SSD companies tell you about the past, present or future of the SSD market."
Survivor's Guide to Enterprise SSDs
....
"The new new economics of SSD storage took the CPU designers and OS software developers completely by surprise."
the New Business Case for SSD ASAPs
(Auto-tuning SSD Accelerated Pools of storage)
....
"Knowing the memory type in the SSD doesn't tell you anything useful about the SSD's likely characteristics and limitations any more... "
Recent Strategic Transitions in SSD
....
"SSDs are only as good as the people who design and make them."
bad block management in flash SSDs
.
"Even SSD's staunchest evangelists fundamentally disagree about many aspects of SSD's future and the best ways to get there..."
the SSD Heresies
....
SSD ad - click for more info
....
"You'd think... someone should know all the answers by now. "
what do enterprise SSD users want?
....
"It's like putting the wrong tires on your car. Everything's OK on the daily commute. You see the difference only when you hit snow or drive around a sharp bend at 120 mph."
endurance issues in flash SSD operating life
.
"Little words can have with big meanings in the world of SSDs. They affect price, performance, reliability and user happiness."
flash SSD jargon explained
..
"It's clear now why it took so long
for the software industry to
wake up to the idea of SSDs."
where are we now with SSD software?
..
"In tomorrow's data-driven content factory, storage will be regarded as a profit center and not an overhead."
the future of enterprise data storage
.
"The SSD Dark Matter impact is why the SSD market will grow to be many times bigger than anyone with a sane spreadsheet could possibly expect."
The big market impact of SSD dark matter
.
"So I could immediately see 2 levels of segmentation and co-existence with PCIe SSDs - which can occur - even if we assume that every fast server has memory channel SSDs installed."
how will Memory Channel SSDs impact PCIe SSDs?
....
"7 ways to classify where all SSDs will fit in the pure SSD datacenter"
an introduction to enterprise SSD silos
.
"Why is a terabyte SSD from one company 2x, 5x, 10x or even more than 100x more expensive than another?"
Clarifying SSD Pricing - where does all the money go?
.
"If I selected 3 competing models of PCIe flash SSDs I could design 3 comparative benchmarks in which each product ranked at the top, bottom or middle of the 3 different top 3 lists. "
the 3 fastest PCIe SSDs list - or is it really lists?
.
"What I'm proposing is a simple sort filter to help you decide whether you should invest more time reading about new SSD products which fall under the enterprise SSD" umbrella."
Legacy versus New Dynasty?
.
"Over 40 companies already ship PCIe SSDs. (See partial list below.) That will rise to over 100 companies in the coming year as new IP toolkits for this market simplify the entry of newcomers to the market."
PCIe SSDs
.
"Unlike flash SSDs, RAM based SSDs never had restrictions on the number of write cycles. That made them more popular in enterprise acceleration applications in the past."
RAM SSDs
.
"An SSD customer case study in 2004 showed that a single 3.5" flash SSD could provide useful speedup in a 25,000 user server - compared to hard disk based RAID."
12 years of enterprise flash SSDs
.
DDR-4 and HMC are merely evolutionary increments in interface and packaging design and don't change data architecture.

But what if you could make RAM itself more intelligent and application aware?
Are you ready to rethink enterprise RAM?