this is the home page of
leading the way to the new storage frontier .....
pcie  SSDs - click to read article
PCIe SSDs ..
memory channel storage
memory channel SSDs ...
SSD SoCs controllers
SSD controllers ..
SSD myths - write endurance
SSD endurance ..

PCIe SSDs versus memory channel SSDs

are these really different markets? (slight return)

by Zsolt Kerekes, editor - May 2, 2014
18 months ago I published a blog called - Boundaries Analysis in SSD Market Forecasting - which is a technique I've been using for many years to help me judge how new and future SSD technologies can settle into new market uses - and whether they might create new sustainable markets which fit in alongside other competitive solutions.

And that co-existence type of eureka moment - tells you a different kind of story compared to the simpler "this will replace that - because it's cheaper and better" kind of wishful thinking which accompanies many new technology announcements.

When I wrote the original boundaries analysis article - the examples I used in it were already quite old (to me) - and I was just using them to demonstrate how you can analyze the business viability of new SSD market segments many years in the future without having to know in detail how any the specific technology problems will be solved.

(Assuming that they could be solved if it was deemed to be worthwhile - is enough in this context.)

Anyway I returned to that subject in a conversation I had this week about memory channel SSDs versus PCIe SSDs.

In some respects I was returning to some of the core judgments which I published a year ago in my blog Memory Channel Storage SSDs will the new concept fly? - should you book a seat yet?

But that was written before there were any real products based on this technology - so some of my comments were made on guesses.

Since then - we've seen the technology being shipped and adopted by IBM (who worked with MCS's creator Diablo for years and helped shape the architecture and wrote a book about its uses.

And there's an excellent faqs page which tells us almost everything we might wish to know about the technical boundaries of this product.

What about the marketing boundaries?

Here are some question from that angle.
  • if memory channel SSDs cost nothing - would there still be a market for PCIe SSDs?

    and conversely
  • if PCIe SSDs cost nothing - would there still be a market for MCS?
Let's discuss these hypothetical questions in order.

If memory channel SSDs cost nothing - would there still be a market for PCIe SSDs?

Yes - for 2 reasons.
  • PCIe SSDs fit in more readily to a PCIe fabric connection - from card to card and also between racks. This can be used to provide high availability or scalability or both.

    MCS is currently limited to short line lengths. That's part of what endows it with better latency - but is a restriction too.

    So there would still be some uses for PCIe SSDs if MCS was free (or cheaper).
  • And another reason is that PCIe SSDs can be designed to be hot pluggable - whereas MCS (like DRAM DIMMs) doesn't support hot plugability.

    So if you have a storage rack which looks like an arrays of 2.5" PCIe SSDs - you can incrementally add new modules or replace faulty modules while the system remains powered.

    Those incremental RAS features - at the drive level - are critical for some markets. So that's another reason why PCIe SSDs would still have a big market even if memory channel SSDs cost the same or less (for equivalent capacity).
And how about my other question.

If PCIe SSDs cost nothing - would there still be a market for MCS?


The main reason is latency.

In currently marketed products - MCS offers typically 5x to 10x lower latency (which can be relied on) than typical PCIe SSDs.

So if ISVs know they can rely on a different class of characteristics - and if they think the market opportunity is big enough to make it worth their while - they can differentiate their product offerings by offering new or differentiated apps functionality in MCS configured servers - which the ISVs can't support in PCIe SSD based servers.

In the same way that people talk about the economic advantages of tiering between different speeds of storage box (in classical hard drive SAN based architectures) so too - when you move inside the apps server box you can get efficiency advantages by being able to tier between different speeds and classes of SSDs.

It was after that kind of analysis that I concluded that memory channel SSDs really are a different type of product which will find their own places in the market - and shouldn't just be regarded as an alternative to some faster types of PCIe SSD.

And that's why they have their own distinct column in my architecture matrix proposal for SSDserver rank. (Which was the topic of last month's home page blog).

Here's a sanity check.

There's still a lot of software yet to be written by ISVs before you'll see some of the differences between PCIe SSDs and MCS appear in sharper focus. But the business cases are real enough to make it happen.

Update - added a new boundary condition question

A few weeks after writing the above I realized I had missed out an important key technical boundary condition which gives even better insights into the MCS vs PCIe SSDs question.

So here it is...

Whether you're a systems architect - or a sales person (who has both types of products in their catalog) - is there another hard technical boundary where one of these looks better than the other - and which I haven't mentioned before?

Remember - when it comes to boundary analysis - you are allowed to imagine extreme assumption which don't exist in reality- but if they did - help to clarify your thinking by cutting out distracting conceptual baggage.

what if (and this is a big IF) both SSD types had the same flash controller management scheme and the same flash memory?

I'm going to make this easier by starting with the assumption that you can buy MCS SSDs or PCIe SSDs which have identical offload controller management for flash and which use the same generation and type of flash memory. (This isn't a market reality at the moment - which is why some of the benchmark comparison papers you see are flawed and reach conclusions which aren't supported by the data.)

In that case - where would you draw the line?

I think the boundary condition comes when you have an application which needs low latency and high IOPs - but in which typically your servers run out of PCIe slots - so in effect you use more servers so that you can deploy more PCIe SSDs.

So the boundary condition is for fully packed SSD servers.

And the boundary condition centers around when you have more free DIMM slots than free PCIe slots in the server.

It's nearly as simple as that.

Because if you can add more MCS SSDs in DIMM slots in a server then you can add PCIe SSDs in PCIe slots - then even if you assume the latency difference between the interfaces is minimal - you get a considerable latency advantage by staying in the same server and adding another MCS device.

And in big systems which use many similar servers - each time you can use another free DIMM slot in a server - instead of expanding out to another server (or PCIe expnasion rack) - then you save the cost of another box as well.

That's a real system capability difference.

And you don't get a taste of this difference by analyzing benchmarks which simply show small numbers of PCIe SSDs compared to MCS SSDs. It's when the PCIe SSD expansion model breaks down - because you've run out slots that you get the biggest benefits of MCS.

And in intermediate cases (when you have free slots of both types) - sometimes one or sometimes the other SSD form factor will be the best.

But another design consideration in the intermediate cases (which is when you have free slots) is the scalability. Because you can scale to a higher performance in a single server with MCS than you can with PCIe SSDs (at the same or similar latency) without breaking out of the box.

Did I really say - it's as simple as that?

Of course - it's not that simple. Even in this simplified and imagined case of identical flash controllers in each type of product. Because the amount of the flash capacity in the server and not just the just raw performance also come into the mix. But if we start looking at micro-tiering inside servers - we'll get off the original track.

So my new technical boundary condition can be sumamrized as this.

If you have the type of project in which you often wish you could add many more fast low latency PCIe SSDs into each of your servers but are limited by free slots (and if you also prefer to minimize the total SSD server box count) then another option to look at is the availability of free DDR-3 DRAM DIMM slots and the use of MCS SSDs instead.

See also:- memory channel SSD news
SSD ad - click for more info
how fast can your SSD run backwards?
11 Key Symmetries in SSD design
A good way to think about what SSDs will do in the 100 per cent SSD enterprise - is to set the limits for how an enterprise can repurpose, leverage and monetize its data - and increase process efficiency by analyzing and anticipating customer demands in real-time.

Bottlenecks in the pure SSD datacenter will be much more serious than in the HDD world - because responding slowly will be equivalent to transaction failure.
will SSDs end bottlenecks?
Hmm... it looks like you're seriously interested in SSDs. So please bookmark our home page and come back again soon.
About the publisher - 22 years guiding the enterprise market

PCIe SSDs vs memory channel SSDs

SSD ad - click for more info

From 2007 to 2014 - the dominant focus and center of SSD accelerator gravity was PCIe SSDs.

It was the rapid and domino effect-like adoption of 3rd party PCIe SSD accelerators by every mainstream server manufacturer - which changed the way that SSDs were perceived from being an alienlike technology promoted by industry outsiders to being a necessary option which was expected to be supported inside every new server product line.

What we've been seeing in the enterprise PCIe SSD market in recent years, however, is that most vendors are focusing on making these products more affordable rather than trying to push the limits of performance.
DIMM wars in SSD servers - Memory1 - episode 1 (August 2015)
SSD ad - click for more info

PCIe SSDs vs memory channel SSDs

Fusion-io fast SSDs - click for more info .... .... Ulltradimm SSDs

with an update re SanDisk's agreement to buy Fusion-io

The article on the left - PCIe SSDs versus memory channel SSDs are these really different markets? (revisted) - was the home page blog on throughout most of May 2014.

As you know from storage history and as you've heard me say in other SSD articles - the proposition that one new SSD thing can replace one old SSD thing is rarely as simple as the advocates of the new thing say. Getting closer to home - when PCIe SSDs appeared in the enterprise market in 2007 - the main company advocating that technology (Fusion-io) initially asserted that PCIe SSDs would replace FC SAN rackmount SSDs.

That message was really a way of getting market attention - from the kind of people who were using the main SSD acceleration product types at that time.

As we know 7 year later - SAN SSDs are still here.

But PCIe SSDs did indeed become a new established part of the enterprise SSD landscape.

It's more complicated than that. With the right SSD software - you can emulate a legacy SAN on PCIe SSD acclerated servers. And you can build a SAN SSD - using PCIe SSDs inside.

In this article - you'll see why memory channel SSDs won't simply replace PCIe SSDs.

But it's also intended to help you figure out - at this early stage of the MCS market - where you might choose to use memory channel SSDs - if you are the type of person who has been using the nearest equivalent product type - which is PCIe SSDs.

related links update re SanDisk's agreement to buy Fusion-io

In June 2014 - SanDisk announced it had agreed to acquire Fusion-io.

I discussed - What will SanDisk really get from Fusion-io? - in an article on the main SSD news page here on

That can be summarized in this way "the ability to get more enterprise petabytes out from the same raw flash chips in - by shipping it through better architecture - is a more significant business factor in the flash memory market today than the ability to do another cell geometry shrink - or to add a few more toppings on the 3D pizza "

But in the context of the page you're reading now - PCIe vs MCS - I think it's appropriate for me to say more about the memory channel SSD angle.

Firstly - if you look at the product images at the top of this column - let's consider my choice of graphics.
  • Fusion-io fast SSDs - click for more infoThe PCIe SSD image.

    It was my deliberate choice to use a picture of a PCIe SSD from Fusion-io to represent the PCIe SSD side of the argument - because although there are over 50 other enterprise PCIe SSD oems - it was Fusion-io which created and established this form factor. And at the time of writing this - Fusion-io is still the leading vendor in the market.

    Another assumption of mine (pre merger) was that - if the MCS market develops into a sizable market - then due to its performance and latency scalability - it would be high end PCIe SSD vendors - which would be most affected. (Although not for several years due to the different maturity levels and ecosystems of the respective software offerings.)
  • Ulltradimm SSDsThe memory channel SSD image.

    I deliberately chose an image of SanDisk's UlltraDIMM to represent this side of the market. Because that's the first implementation of this type of low latency teraDIMM class of product.

    However, it's important to bear in mind that it's not SanDisk but Diablo - which pioneered this concept - and it's Diablo which supplies the RAM side of the hardware interface and the underlying software stacks.

    So unless SanDisk acquires Diablo too then it's likely that we will see other non exclusive implementations of MCS which use different controllers and different non volatile memories coming from other SSD companies - and not just SanDisk.

    And in any case the memory channel SSD concept itself is bigger than Diablo too.

    Other companies like Micron, Viking. Innodisk etc - who have DDR3 compatible flash SSD form factors (but which don't currently use the DDR3 as the data interface - but simply use it to supply power) might decide to enter the market with competing products too.
If we assume for now that SanDisk does go ahead with acquiring Fusion-io - but doesn't acquire Diablo - then some natural questions arise...
  • could SanDisk design a CPU-less DDR3 SSD using the same architecture concepts as a PCIe SSD from Fusion-io?

    And if they did - would that be worthwhile?

    I think technically it could be done.

    And in theory this would reduce the power consumption of the DIMMs compared to the current (hot) design. But it would take collaboration to do this - and the most useful way would be to bypass the higher levels of Diablo's software stack. (Which would fragment Diablo's market and make their other future products MCS less attractive.)

    The result? You'd effectively get a Fusion-io software compatible SSD in a bunch of DDR-3 slots.

    Would it be worthwhile?

    It depends on who makes the server. Maybe...
  • could some of Fusion-io's array level virtualization software be used in a useful way in a server which also has the current design if UltraDIMMs in it?

    This is a scenario I thought about in my very first discussion with SMART about the MCS concept.

    We know from other studies that when you use Fusion-io PCIe SSDs in servers which also have large memory resident databases - then the overall performance gets better by having the incremental memory resident assets.

    But I'm not confident enough to state whether you would get a significant performance benefit by having PCIe SSDs implementing one of the storage space - while also having 1st generation UlltraDIMMs implementing a large virtual memory space in the same server.

    But under new ownership - I guess it will be easy enough to try that experiment and to create a list of the types of applications and setups in which it could escalate the benefits of using each of the 2 technologies.

90% of the enterprise SSD companies which you know have no good reasons to survive.
consolidation in enterprise flash - the user revolution