this is the home page of
leading the way to the new storage frontier .....
pcie  SSDs - click to read article
PCIe SSDs ..
DIMM wars
SCM DIMM wars ..
SSD SoCs controllers
SSD controllers ..
click to read the article - Big versus Small SSD  architectures
Size in SSD architecture ..
image shows software factory - click to see storage software directory
SSD software ....

An SSD conversation with PLX

by Zsolt Kerekes, editor - - June 5, 2014
conversation ..... SSD ad - click for more info
If you don't know PLX - they're the world's biggest maker of PCIe switch chips and interfaces for the enterprise market. Their technology is used in most enterprise servers and many enterprise PCIe SSDs.

As a result they get a lot of top level visibility about what's happening in the enterprise computing market and when their company introduces something new - it can have a significant effect on many products.

Two months ago I had a conversation - about the SSD market from a PCIe interface angle - with Larry Chisvin, VP of strategic initiatives and David Hurd , Sr. Director, Corporate Communication at PLX Technology.

In this conversation I learned 2 new (to me) things which relate to the SSD market which I'm able to share with you now.
  • a correction to one of my old SSD market penetration impact effect ideas
  • and a new box - to join up the servers and SSDs in your life
I'm going to talk about the new box first.

a new PCIe fabric box - from PLX

PLX - whose primary business has been chips and related IP - is entering the systems business - with a PCIe fabric box

Anyone who has been reading articles by PLX knows that for the past 2-3 years they've been talking about the possibilities of using PCIe as a box to box fabric.

And I've talked to them about that subject in previous years too.

I said - this ExpressFabric solution idea which you talk about on your web site - I assume it's really just an outline capability of what someone could do with your chip interface and software technologies - if they choose to go in that direction them in that way...

It's not an actual product - is it?

It's just a concept diagram...

PLX said - well yes actually - that "box" you see in the diagram is a real product - and we've been sampling it under NDA to various customers.

Here's the exciting new thing I learned after probing. (Details may have changed by the time the product is launched.)

PLX started out with the intention of using the ExpressFabric box as a system design tool to help server oems and PCIe SSD designers get a feel for how they could expand their product offerings by scaling the fabric using PCIe instead of ethernet or InfiniBand.

And although the box - which has been sampling - doesn't use the next generation of silicon - which will be shipping later this year - and therefore isn't as fast as the production models will be - it provides developers with a workable platform which they can use to develop their software and test their architecture ideas.

What is the ExpressFabric box?
  • 1U 32 port PCIe fabric - cabinet top box - to interconnect racks in a single cabinet
  • internally there's a PCIe switch (3 switch chips each with 96 lanes) and a management server with software which controls the host to host communications - which supports line speed inter box RDMA
All the software and all the chips in the box are standard IPs which oems can get from PLX and use in their own designs if they want to.

But the box has been designed to be production ready. And PLX told me that if customers decide to go to market shipping the PLX box as part of their solution - PLX has done the logistics planning to support that in a way that's competitive and viable.

Pricing will be in the region $5K to $10K (those details may change). The price will be set at a level which makes it competitive compared to using other fabric alternatives - which provide similar performance and architectures.

The great thing from the point of view of PCIe SSD companies - and users - is that the PLX box will provide a standard way to incrementally add more PCIe SSDs into a system - with high throughput, high availability and low latency - without having to resort to alien (non PCIe native) bridging technologies - which add delays and support complexities.

That's it - re the box anyway.

As I said before - details may change or I may have misread my notes.

But the outline story is that the "concept diagram" which has been at the heart of PLX presentations for the past several years - will sometime soon emerge as a box which you can use for linking servers and SSD resources together to create larger, faster and more reliable versions of the technology which many of you already like.

Related links:-
  • PLX - profile page on
  • "ExpressFabric technology is initially targeted at small- to medium-sized cloud clusters with up to 1,000 nodes and 8 racks where the majority of the high-volume innovations are taking place." - ExpressFabric Wins Award (May 2014)
a correction to one of my old SSD market penetration impact effect ideas

In the first published version of this article (an SSD conversation with PLX) I placed this next section at the top of the page - before the new box. But then the next day I switched it around.

Most of you don't need to read it - as it's a marketing kind of thing.

On the other hand it just shows I can often be wrong too - in the summary ideas I carry around in my head. Even when I'm right about the lower level details. Due to not connecting the dots.

re SSD impact of unit shipments in the server market

It's common knowledge now that having SSD acceleration and the right kind of software in the enterprise impacts the nature and number of servers which are needed in the enterprise to perform workloads (in comparison to not having SSDs at all).

And hundreds of SSD companies and billions of dollars of revenue are now based on this concept.

Big cheer for the SSD companies and customers who made this happen.

But the market consequences of the idea - which I call SSD-CPU equivalence hadn't been analyzed (even by SSD companies who were enabling this trend) back in 2003 when I published one of my early SSD user value proposition based market sizing models - which looked at what would happen when the concept was more widely known.

The underlying technology concept - which was old even when I wrote about it - is that for a wide range of applications (as measured from an external reference point) - there is no little or no material performance difference between
  • a single SSD accelerated CPU, or
  • a single HDD based CPU (having the same architecture) but with a much higher clock speed (Nx) than the SSD system, or
  • (Nx) more servers or cores of an HDD based CPU at the same clock speed as the reference SSD system
Based on the measurements which I had done in the late 1980s - and from the conversations I was having with SSD pioneers in the early 2000s I felt confident enough to publish my paper which said - that as a rule of thumb - the industry should be aiming to replace 3 old style HDD servers with 1 new SSD accelerated server - while running the apps faster - and at a price point which would make adoption of SSDs a very attractive cost saving measure for users (with SSDs replacing server and software costs).

The background to the timing of my paper was that server CPU clock speeds had flatlined in the early years of the millenium due to limits imposed by architecture, the semiconductor and copper signal interface, and heat dissipation. This meant is was easier for CPU makers to add cores rather than add more GHz.

A long term consequence which I mentioned in my 2003 article - was that the world wouldn't need so many servers to do the same jobs.

And I said that would hit the server makers hard - which is why I said they would all have to become born again SSD marketers (when any one of them did) because there was no other future for them once the concept became widely known and easy to implement.

In those days - everything cost more - and the standard enterprise memory in SSDs was RAM - so the substitution effect mainly started in high end applications. And the early inroads made by SSDs in the enterprise were in customers who already owned the fastest servers - but who were still unhappy with the performance they were getting.

Anyway - what has that got to do with PLX and the market today? - you may ask.

I thought it would be a good opportunity for me to ask PLX - as they supply most of these server companies with the raw interface chips (regardless of the CPU type) - if they had seen any significant reductions in server volumes - which is what I expected from my original market model?

Zsolt - that's not exactly what we're seeing...

They didn't really want to go into too many details - about the total market for all server types - but were good enough to counter my headline suggestion - with their observation that instead of all server numbers shrinking down - they have instead been seeing high growth in a new type of server - called micro-servers.

That was a useful correction to my thinking about the server numbers - and made immediate sense to me - because this was another kind of user value proposition I had written about (later) in 2005 - where I had described the user value proposition for an ideal notebook PC. (If you're looking for that ideal SSD empowered notebook - despite the 9 years which have elapsed since I described what it should do - it hasn't appeared yet - and there's still no such product on the market.)

Anyway - the prime user value proposition was that you can run the apps on an SSD assisted notebook at desktop HDD speeds or faster - while using a slower clocked (and therefore lower power consumption) CPU.

The way the SSD notebook penetration model concepts apply to servers is this.

Instead of simply having dramatically less units of high end fast enterprise servers - the alternative (for users and cloud companies) is a mix and match approach of tactically using micro servers which have slower CPU clocks and which consumer much less electrical power.

So 1 SSD enhanced micro server can replace 1 non assisted high end server - is another alternative to - 1 SSD assisted high end server replacing 3 or more non SSD high end servers.

It depends on your workload, business and the cost of floor space - which option you'll choose. But the decline of fast clocked servers doesn't lead automatically to the decline of all servers. So in that respect I was wrong.

(And at some stage - the demand for servers will grow - because workloads don't stay the same from one decade to the next - and at some stage - new blue sky SSD enabled data industries will increase the demand for everything again.)

Anyway I found the market correction feedback from PLX useful - because I can run a long way on just a few simple new ideas each year.

Now many of you may be thinking - you already knew that (because unlike me you sometimes think about things which aren't SSDs).

In the quest to understand the SSD market (it's more like an Odyssey really - in which years of lost of time is spent visiting islands and meeting mysterious creatures which don't reappear in the story - but seem fascinatingly distracting at the time) - a lot of arcan articles get written.

Here are some related links:-

SSD ad - click for more info


storage search banner

About the publisher - 22 years guiding the enterprise market
Express fabric - click for more info

Soul of a New Interconnect
ExpressFabric - from PLX Technology

SSD news
what's RAM really?
high availability enterprise SSDs
how fast can your SSD run backwards?
should we set higher expectations for memory systems?
An important new factor for the PCIe SSD market was the series of product announcements centered around the core concept of using PCIe as an interconnection fabric between racks.
12 key SSD ideas in 2014
SSD ad - click for more info
"About a week prior to IDF2014 - NVM Express announced that it had begun work on a new specification known as NVM Express over Fabrics.

These fabrics can support large numbers of NVMe SSDs, and these SSDs can be at some distance from the host servers that need them. The goal is to make remote NVMe have no more than approximately 10 s (microseconds) of added latency compared to attaching NVMe SSDs to the local PCIe bus."
Dennis Martin, President Demartek - in his article - Demartek Comments on IDF2014 and NVMe (September 24, 2014)

SSD ad - click for more info

"The core PLX PCIe silicon business fits very well with the Avago business model and broadens Avago's portfolio serving the enterprise storage and networking end markets" said Hock Tan, President and CEO of Avago.
Avago agrees to acquire PLX (SSD news - June 2014)

"One petabyte of enterprise SSD could replace 10 to 50 petabytes of raw HDD storage in the enterprise - and still get all the apps running faster."
the enterprise SSD software event horizon

Some of the world's leading SSD marketers have confided in me they know from their own customer anecdotes that there are many segments for enterprise flash arrays which aren't listed or even hinted at in standard models of the enterprise market.

Which means they still aren't designing the right kind of SSD products and services for huge untapped markets.
Decloaking hidden segments in the enterprise

continuity of PLX's PCI Express box platform - after Avago?
Editor:- July 10, 2014 - This week a reader asked me if I thought that the PCIe fabric system - which I had written about last month in the article - An SSD conversation with PLX - would still be available as a product after the acquisition of PLX by Avago Technologies closed?

He pointed out that Avago avoids competing with its own systems customers (which is why they divested the SSD business from LSI) and also Avago isn't a systems company.

Here's what I said...

The points you make are valid and I had wondered the same thing myself.

But I think the PCIe fabric box (ExpressFabric solution) will continue in some form or other because it started out as a system design kit for demonstrating what could be done with the next generation of PCIe chips and software stacks. So its an essential sales tool for the chip business.

Before the announcement from Avago the PLX guys had already said that any of their customers would be welcome to use as much or as little as they wanted from the SDK box as PLX wasnt originally advancing this as a systems product.

But I could see from my own judgement (and PLX confirmed this) that for some customers having availability of such a box as a product would be a convenient tool.

Incidentally - thats how Intel got into the systems business - with their Multibus SBC product range in the 1970s - which was a response to customers asking if they could buy the boards which were in the early microprocessor development systems.

Returning to Avago and the PCIe fabric SDK

I agree - it's possible that Avago may decide not to get into the volume supply business of these boxes but in that case I think the boxes (and design IP for these boxes) would continue to be available in some form from a designated source otherwise lack of this integration concept tool would slow down market adoption of the new PCIe fabric chips.

It's in the interests of Avago and its oem customers to ensure that the ExpressFabric SDK remains available as a software reference architecture which is at the center of this new ecosystem.

(I'll ask for an official response about this too - but as with many acquisitions in the sensitive post announcement period - the answer may be:- no comment.)

SSD ad - click for more info

Shown above - a banner ad from 2013/14 which PLX used to promote its new fabric concept for PCIe. See also:- SSD banner advertising on

"Using PCI-E to ditch NICs..."
Thomas Ryan, Author at in his blog A Look at Avago's ExpressFabric (September 23, 2014)
image linking to article on |Semiaccurate,com

how fast can your SSD run backwards?
11 Key Symmetries in SSD design


Who are the top SSD companies?... the companies which you absolutely have to look at if you've got any new projects involving SSDs?
the Top SSD Companies


DRAM latency and why flash can replace it -  article


"You'd think... someone should know all the answers by now. "
what do enterprise SSD users want?



Can you trust SSD market data?
Heck no!


PCIe SSDs versus memory channel SSDs are these really different markets?
PCIe SSDs versus memory channel SSDs are these really different markets?