designing a new enterprise PCIe SSD? |
find out more about
PCIe chips and their possible uses in SSDs
|by Zsolt Kerekes,
PCIe SSD market - the first 6½ years
StorageSearch.com has focused on enterprise SSDs since the 1990s - and
our readers were at the forefront of the PCIe SSD market - since the first
enterprise PCIe SSD products came to the open market in
At that time some people were asking - how would PCIe SSDs fit into
the enterprise SSD market landscape? - which was dominated at the top end by
proprietary rackmount systems - and a small number of fibre-channel
3.5" SSD drives.
Most SSD market experts at that time - didn't think PCIe SSDs would stick -
except as a small niche. Instead the "safe" money was on
predictable incremental changes - which were then already in progress. The
hard drive way of thinking was that the enterprise DAS SSD market would
evolve with 3.5" giving way to
2.5" form factors
and SATA giving
way to SAS. And if
servers needed access to larger amounts of fast SSD capacity this could be
easily implemented by classic
arrays of these smaller drives. SSDs were just
replacements for HDDs
- weren't they?
That's not the way the thought leaders in the SSD
market saw it, however.
The number of PCIe SSD vendors kept growing -
and the products kept getting faster and better. So much so that in September
2009 - I
announced that StorageSearch.com searches for "PCIe SSDs" and
related technology content had overtaken searches for 2.5" SSDs - marking a
decisive turning point in the direction the SSD market was taking.
the importance and size of the SSD market and the significance of PCIe SSDs
as a distinct and permanent feature of enterprise architecture have become clear
With over 100 million enterprise PCIe ports already
shipped in the market (as of January 2014) and with waiting-to-be-exploited
fabric capabilities which chipmaker
PLX calls "ExpressFabric"
(by which they mean PCIe as a fabric outside the box but still within the
cabinet) the converging PCIe SSD / server has already been a fast changing
market and is well positioned to expand into new applications.
might think that 7 years after the first PCIe SSD cards were being designed - it
would be about time for the rate of change in this technology to slow down a
bit. But far from it. New
technologies and architectures have been designed to span multiple memory
generations, technology generations and even memory types. And
2013 we started seeing more new features and form factors for PCIe SSDs.
PCIe SSDs are now at the forefront of all high performance
enterprise SSD design.
PCIe has become the incubator for testing
new SSD concepts and getting them to market quickly. Whether it's a new memory
architecture, a new non volatile memory type, a new reliability feature or a
new way of connecting clusters of servers to arrays of SSDs - this is the market
where the exciting hot ideas are seen first.
There are many reasons
you might want to learn more about PCIe interface technology and the glue chips
and IP from the world's leading supplier PLX (70% PCIe switch market
- SSD designers - if you're designing a PCIe SSD for the first time -
you might want to see the educational materials that PLX has developed for the
SSD market. The world's leading PCIe SSD companies already use their chips - but
for newcomers it's now easier to dive into this technology -with tech
support and even a system design kit.
- SSD marketers - you need to have an outline of what PCIe
interconect technology can do now - and also have a preview of new features
which you can leverage with your planned memory and controller roadmaps.
- SSD system architects - if you know the roadmap for PCIe's
capabilities- with respect to latency, throughput, topologies, fail-over and
clustering - you will be better placed to evaluate new products which you read
about and decide where they fit - compared with anticipated "best of
breed" products which haven't been launched yet.
I confess - that for
many years - like most of you - I didn't read too much about about PCIe chips
beyond the headline capabilities of what each generation could achieve in terms
of raw throughput.
- end users, investors and general readers - you don't need to
know how these chips work - but it's interesting to see how PCIe supports ways
of connecting SSDs (to servers and to each other) which are sometimes similar
and other times different to what you might expect from just knowing about
other storage interfaces and networks. And knowing more about the possibilities
enabled by PCIe technology will help you get a better feel for how it compares
to the other options you already know about.
My thinking was - isn't the idea of
standards to ensure that
all these things can connect to each other and work in the same way?
(Interoperability.) I guess that's one view. But most memory chips have
standards too. And if you look at all the
diversity in SSDs
which have been enabled by different architectures and different memory
management schemes - you'll realize that standards merely define the lowest
common denominator of what can be done.
enter the fast lane to
the future of PCIe SSDs
The art of "what is possible in SSD
design" - by using so-called standard components in new ways - which is
different for each chip vendor - is constrained only by the imagination,
market economics and physics. If you understand more about what PCIe chips
can do in SSD designs you'll be in the fast lane towards the exciting
journey to the solid state storage future.
editor's summary of key
PCIe / SSD articles, links and videos from PLX Technology
links to product data and resources
PCIe SSDs - a PCIe chipmaker's view - this article introduces you to
what PCIe can do for SSDs. It has some nice pictures which show you some of the
various connection architectures that PLX's chips support.
extract - "In its first
generation, referred to as Gen1, the speed was set at 2.5 giga-transfers per
second (GT/s) serial bi-directional interface, then later enhanced to 5GT/s
(Gen2) and eventually 8GT/s (Gen3). The PCI-SIG forum is already discussing
16GT/s for Gen4. The beauty of PCIe is that a designer can combine 2, 4, 8 or 16
of these PCIe lanes into a single data port. Equally important to those
designers is that all PCIe Gen2 and Gen3 ports are required to be
enterprise SSD designs (video) - this includes an introductory tutorial into
PCIe and its performance and architectural capabilities for SSDs including
automatic failover and multi-host capabilities. PLX's switch chips also supports
failover if the fault occurs in the PCIe switch fabric chips themselves.
extract - "...And in case one of the hosts fails
and you want to connect the SSDs - or the devices connected to that host - to
another host - that can be done automatically as well - and the surviving host
can attach the devices that were attached to the failing host to itself and
control it so that the system doesn't go down and the data stored in these
devices doesn't get isolated from the main system."
as a fabric for data center clustering (video) - PLX's PCIe extenders and
switch chips support lossless, software light, high performance CPU to CPU
memory transfers - and shared I/O - for upto about 200 nodes - as a
mini-cluster. At the next level of scalability you can interconnect these
extract - "The host to host communication can take
place directly between CPU to CPU through the PCI express fabric or through a
shared neck. ...For latency critical apps you can pretty much do memory to
memory transfers and writes using the PCIe fabric. The PCIe extenders are very
low cost, very low power, high performance. And the PCIe fabric itself
supports full peer to peer functionality. So on a 48 lane switch in Gen3 you
have 96 gigabytes per second of performance that's available to you."
|Editor:- July 17, 2013 - Taking PCIe Out of the
Box - is just one of the many different topologies discussed in a new white
paper about PCIe and enterprise storage from PLX called -
and PCI Express|