An SSD conversation with PLXby
Zsolt Kerekes,
editor -
StorageSearch.com - June
5, 2014 |
 |
..... |
 | |
If you don't know PLX - they're the world's
biggest maker of PCIe switch chips and interfaces for the enterprise market.
Their technology is used in most enterprise servers and many enterprise
PCIe SSDs.
As
a result they get a lot of top level visibility about what's happening in the
enterprise computing market and when their company introduces something new -
it can have a significant effect on many products.
Two months ago I
had a conversation - about the SSD market from a PCIe interface angle -
with Larry Chisvin,
VP of strategic initiatives and David Hurd , Sr. Director,
Corporate Communication at PLX
Technology.
In this conversation I learned 2 new (to me)
things which relate to the SSD market which I'm able to share with you now.
- a correction to one of my old SSD market penetration impact effect
ideas
- and a new box - to join up the servers and SSDs in your life
I'm
going to talk about the new box first.
a new PCIe fabric box -
from PLX
PLX - whose primary business has been chips and
related IP - is entering the systems business - with a PCIe fabric box
Anyone
who has been reading articles by PLX knows that for the past 2-3 years
they've been talking about the possibilities of using PCIe as a box to box
fabric.
And I've talked to them about that subject in previous years
too.
I said - this
ExpressFabric solution
idea which you talk about on your web site - I assume it's really just an
outline capability of what someone could do with your chip interface and
software technologies - if they choose to go in that direction them in that
way...
It's not an actual product - is it?
It's just a
concept diagram...
PLX said - well yes actually - that "box"
you see in the diagram is a real product - and we've been sampling it under NDA
to various customers.
Here's the exciting new thing I learned after
probing. (Details may have changed by the time the product is launched.)
PLX
started out with the intention of using the ExpressFabric box as a system design
tool to help server oems and PCIe SSD designers get a feel for how they could
expand their product offerings by scaling the fabric using PCIe instead of
ethernet or InfiniBand.
And although the box - which has been
sampling - doesn't use the next generation of silicon - which will be shipping
later this year - and therefore isn't as fast as the production models will be
- it provides developers with a workable platform which they can use to develop
their software and test their architecture ideas.
What is the
ExpressFabric box?
- 1U 32 port PCIe fabric - cabinet top box - to interconnect racks in a
single cabinet
- internally there's a PCIe switch (3 switch chips each with 96 lanes)
and a management server with software which controls the host to host
communications - which supports line speed inter box RDMA
All the
software and all the chips in the box are standard IPs which oems can get from
PLX and use in their own designs if they want to.
But the box has been
designed to be production ready. And PLX told me that if customers decide to go
to market shipping the PLX box as part of their solution - PLX has done the
logistics planning to support that in a way that's competitive and viable.
Pricing will be in the region $5K to $10K (those details may change).
The price will be set at a level which makes it competitive compared to using
other fabric alternatives - which provide similar performance and architectures.
The
great thing from the point of view of PCIe SSD companies - and users - is that
the PLX box will provide a standard way to incrementally add more PCIe SSDs into
a system - with high throughput, high availability and low latency - without
having to resort to alien (non PCIe native) bridging technologies - which add
delays and support complexities.
That's it - re the box anyway.
As
I said before - details may change or I may have misread my notes.
But
the outline story is that the "concept diagram" which has been at the
heart of PLX presentations for the past several years - will sometime soon
emerge as a box which you can use for linking servers and SSD resources together
to create larger, faster and more reliable versions of the technology which
many of you already like.
Related links:-
- PLX - profile page
on StorageSearch.com
- "ExpressFabric technology is initially targeted at small- to
medium-sized cloud clusters with up to 1,000 nodes and 8 racks where the
majority of the high-volume innovations are taking place." -
ExpressFabric Wins
Award (May 2014)
|
a correction to one of my old SSD market
penetration impact effect ideas
In the first published version of
this article (an SSD conversation with PLX) I placed this next section at the
top of the page - before the new box. But then the next day I switched it
around.
Most of you don't need to read it - as it's a marketing kind
of thing.
On the other hand it just shows I can often be wrong too -
in the summary ideas I carry around in my head. Even when I'm right about the
lower level details. Due to not connecting the dots.
re SSD impact
of unit shipments in the server market
It's common knowledge now
that having SSD acceleration and the right kind of software in the enterprise
impacts the nature and number of servers which are needed in the enterprise to
perform workloads (in comparison to not having SSDs at all).
And
hundreds of SSD companies and billions of dollars of revenue are now based on
this concept.
Big cheer for the SSD companies and customers who made
this happen.
But the market consequences of the idea - which I call
SSD-CPU
equivalence hadn't been analyzed (even by SSD companies who were enabling
this trend) back in 2003 when I published one of my early SSD user value
proposition based market sizing models - which looked at what would happen
when the concept was more widely known.
The underlying technology
concept - which was old even when I wrote about it - is that for a wide range
of applications (as measured from an external reference point) - there is no
little or no material performance difference between
- a single SSD accelerated CPU, or
- a single HDD based CPU (having the same architecture) but with a much
higher clock speed (Nx) than the SSD system, or
- (Nx) more servers or cores of an HDD based CPU at the same clock
speed as the reference SSD system
Based on the measurements which I
had done in the late 1980s - and from the conversations I was having with SSD
pioneers in the early 2000s I felt confident enough to publish my paper which
said - that as a rule of thumb - the industry should be aiming to replace 3 old
style HDD servers with 1 new SSD accelerated server - while running the apps
faster - and at a price
point which would make adoption of SSDs a very attractive cost saving
measure for users (with SSDs replacing server and software costs).
The
background to the timing of my paper was that server CPU clock speeds had
flatlined in the early years of the millenium due to limits imposed by
architecture, the semiconductor and copper signal interface, and heat
dissipation. This meant is was easier for CPU makers to add cores rather than
add more GHz.
A long term consequence which I mentioned in my 2003
article - was that the world wouldn't need so many servers to do the same jobs.
And I said that would hit the server makers hard - which is why I
said they would all have to become born again SSD marketers (when any one of
them did) because there was no other future for them once the concept became
widely known and easy to implement.
In those days - everything cost
more - and the standard enterprise memory in SSDs was RAM - so the substitution
effect mainly started in high end applications. And the early inroads made by
SSDs in the enterprise were in customers who already owned the fastest servers -
but who were still unhappy with the performance they were getting.
Anyway
- what has that got to do with PLX and the market today? - you may ask.
I
thought it would be a good opportunity for me to ask PLX - as they supply most
of these server companies with the raw interface chips (regardless of the CPU
type) - if they had seen
any significant reductions in server volumes - which is what I expected
from my original market model?
Zsolt - that's not exactly what we're
seeing...
They didn't really want to go into too many details - about
the total market for all server types - but were good enough to counter my
headline suggestion - with their observation that instead of all server
numbers shrinking down - they have instead been seeing high growth in a new
type of server - called micro-servers.
That was a useful correction to
my thinking about the server numbers - and made immediate sense to me - because
this was another kind of user value proposition I had written about (later) in
2005 -
where I had described the user value proposition for an ideal notebook PC. (If
you're looking for that ideal SSD empowered
notebook -
despite the 9 years which have elapsed since I described what it should do -
it hasn't appeared yet - and there's still no such product on the market.)
Anyway
- the prime user value proposition was that you can run the apps on an SSD
assisted notebook at desktop HDD speeds or faster - while using a slower clocked
(and therefore lower power consumption) CPU.
The way the SSD notebook
penetration model concepts apply to servers is this.
Instead of
simply having dramatically less units of high end fast enterprise servers -
the alternative (for users and cloud companies) is a mix and match approach
of tactically using micro servers which have slower CPU clocks and which
consumer much less electrical power.
So 1 SSD enhanced micro server can
replace 1 non assisted high end server - is another alternative to - 1 SSD
assisted high end server replacing 3 or more non SSD high end servers.
It
depends on your workload, business and the cost of floor space - which option
you'll choose. But the decline of fast clocked servers doesn't lead
automatically to the decline of all servers. So in that respect I was wrong.
(And at some stage - the demand for servers will grow - because workloads don't
stay the same from one decade to the next - and at some stage - new blue sky
SSD enabled data industries will increase the demand for everything again.)
Anyway
I found the market correction feedback from PLX useful - because I can run
a long way on just a few simple new ideas each year.
Now many of you
may be thinking - you already knew that (because unlike me you sometimes think
about things which aren't SSDs).
In the quest to understand the SSD
market (it's more like an Odyssey
really - in which years of lost of time is spent visiting islands and meeting
mysterious creatures which don't reappear in the story - but seem fascinatingly
distracting at the time) - a lot of arcan articles get written.
Here
are some related links:-
| | |
..... |
|
..... |
|
|
| . |
| About the publisher - 22
years guiding the enterprise market |
| . |
| |
.... |
|
|
SSD news
PCIe SSDs
what's RAM really?
high availability
enterprise SSDs
how fast can your SSD
run backwards?
should we set
higher expectations for memory systems? |
| .. |
| An important new factor
for the PCIe SSD market was the series of product announcements centered around
the core concept of using PCIe as an interconnection fabric between racks. |
| 12 key SSD
ideas in 2014 | | |
| .. |
 |
| .. |
"About a week prior to
IDF2014 - NVM Express announced that it had begun work on a new specification
known as NVM Express over Fabrics.
These fabrics can support large
numbers of NVMe SSDs, and these SSDs can be at some distance from the host
servers that need them. The goal is to make remote NVMe have no more than
approximately 10 s (microseconds) of added latency compared to attaching NVMe
SSDs to the local PCIe bus."
|
| Dennis Martin, President
Demartek - in
his article -
Demartek
Comments on IDF2014 and NVMe (September 24, 2014) | | |
.. |
 |
.. |
| "The core PLX PCIe
silicon business fits very well with the Avago business model and broadens
Avago's portfolio serving the enterprise storage and networking end markets"
said Hock Tan, President and CEO of Avago. |
| Avago agrees to acquire
PLX (SSD news - June 2014) | | |
.. |
|
|
.. |
Some of the world's leading
SSD marketers have confided in me they know from their own customer anecdotes
that there are many segments for enterprise flash arrays which aren't listed or
even hinted at in standard models of the enterprise market.
Which means they still aren't designing the right kind of SSD products
and services for huge untapped markets. |
| Decloaking
hidden segments in the enterprise | | |
.. |
| continuity of PLX's PCI
Express box platform - after Avago? |
Editor:- July 10, 2014 - This
week a reader asked me if I thought that the PCIe fabric system - which I
had written about last month in the article -
An SSD
conversation with PLX - would still be available as a product after the
acquisition of PLX
by Avago Technologies closed?
He
pointed out that Avago avoids competing with its own systems customers (which is
why they divested the SSD business from LSI) and also Avago isn't a systems
company.
Here's what I said...
The points you make are valid
and I had wondered the same thing myself.
But I think the PCIe fabric box (ExpressFabric solution)
will continue in some form or other because it started out as a system design
kit for demonstrating what could be done with the next generation of PCIe chips
and software stacks. So its an essential sales tool for the chip business.
Before the announcement from Avago the PLX guys had already said
that any of their customers would be welcome to use as much or as little as they
wanted from the SDK box as PLX wasnt originally advancing this as a systems
product.
But I could see from my own judgement (and PLX confirmed
this) that for some customers having availability of such a box as a product
would be a convenient tool.
Incidentally - thats how Intel got into
the systems business - with their
Multibus
SBC product range in the 1970s - which was a response to customers asking
if they could buy the boards which were in the early microprocessor development
systems.
Returning to Avago and the PCIe fabric SDK
I agree -
it's possible that Avago may decide not to get into the volume supply
business of these boxes but in that case I think the boxes (and design IP for
these boxes) would continue to be available in some form from a designated
source otherwise lack of this integration concept tool would slow down market
adoption of the new PCIe fabric chips.
It's in the interests of Avago
and its oem customers to ensure that the ExpressFabric SDK remains available
as a software reference architecture which is at the center of this new
ecosystem.
(I'll ask for an official response about this too - but as
with many acquisitions in the sensitive post announcement period - the answer
may be:- no comment.) | | |
.. |
 |
Shown above - a banner ad from 2013/14 which
PLX used to promote its new fabric concept for PCIe. See also:-
SSD banner
advertising on StorageSearch.com
.. |
|
|
.. |
|
|
..
.. |
| Who are the top SSD
companies?... the companies which you absolutely have to look at if you've got
any new projects involving SSDs? |
| the Top SSD Companies | | |
..
.. |
 |
..
.. |
|
|
..
..
.. |
|
|
..
.. |
|
| |
