PCIe SSDs for use in enterprise server acceleration have been shipping
in the market since
one of the
most popular SSD
subjects pursued by our readers and has been (and has been since
2009 - when we
first told you that searches for PCIe SSDs on
StorageSearch.com - had overtaken
searches for 2.5"
SSDs on this site for the first time in
With over 100 million enterprise PCIe ports already shipped in
the market (upto January 2014) and with waiting-to-be-exploited fabric
capabilities which chipmaker PLX
(by which they mean PCIe as a fabric outside the box but still within the
cabinet) the fast changing and converging PCIe SSD / server market is
well positioned to expand into new applications.
Over 40 companies
already ship enterprise accelerator PCIe SSDs (see list below).
That will rise to over 100 companies as the availability of more
SSD controller chips,
other SSD related chip sets and
IP and SSD software
for this market will make it even easier than it already is for newcomers to
enter the PCIe SSDs market.
PCIe SSDs come in several shapes and
sizes. The most familiar form factor is cards, modules and racks. But a new
form factor - for
2.5" PCIe SSDs
which emerged in
2012 are opening up new applications - such as the displacement of fast
the 2nd half of 2013 - we also saw the start of another type of deployment for
the PCIe interface in the M.2
form factor which is aimed at the
consumer SSD market
notebooks. These consumer PCIe products have throughputs similar to the
enterprise products of 5-6 years earlier - but aren't rated for heavy
Nevertheless I wouldn't be surprised to see them appear as
in some future arrays in read-intensive design slots.
competitive alternative to PCIe SSDs - at the low latency end of the
applications spectrum inside (rather than
outside) the server box -
which started shipping in volume in 2014 - is a new class of low latency flash
SSDs which plug into DIMM sockets which were originally designed for
read more about these products (which go by various names such as TeraDIMMs,
ULLtraDIMM SSDs, eXFlash DIMMs etc) in the
|You could easily get the
impression that this is going to be a clone market - same host interface - same
memory chips -who cares who makes them? Only price matters - right? That's
what I expected would be the case when I started discussing the PCIe SSD market
with vendors in 2007 - but it didn't turn out that way.
|don't all PCIe SSDs
look pretty much the same?|
|how fast can your
SSD run backwards?|
|SSDs are complex devices and there's a
lot of mysterious behavior which isn't fully revealed by benchmarks, datasheets
and whitepapers. Underlying all the important aspects of SSD behavior are
which arise from the intrinsic technologies and architecture inside the SSD.
|Today's commodity MLC flash
has raw wear-out in the 2,000 to 3,000 write cycle range. And the future
direction is downwards (towards worse).|
|SSD endurance - the
oems list |
|This is a (partial list) from
past SSD news and
Mach Xtreme Technology,
Renice Technology ,
, Texas Memory
|the Problem with
Write IOPS in flash SSDs |
|Random "write IOPS"
in many of the fastest flash SSDs are now similar to "read IOPS"
- implying a performance symmetry which was once believed to be impossible.|
why are flash SSD IOPS such a poor predictor of application performance?
why are users still buying
RAM SSDs which cost an
order of magnitude more than SLC? (let alone
MLC) - even
when the IOPS specs look superficially similar?
tells you why the specs got faster - but the applications didn't.
||And why competing SSDs with
apparently identical benchmark results can perform completely differently.
|many years ago -
|BiTMICRO & CENATEK collaborate on PCI SSDs|
2002 - BiTMICRO.
announced today, a technology and marketing partnership to investigate
developing a hybrid solid state disk storage solution that brings together the
best in flashdisk storage and PCI bus-attached SSD technology.
Prices are likely to
range between $1,000 to $2,000 per gigabyte.
Editor's later comments:- the fast PCI SSDs which later
emerged from CENATEK - can be seen as immediate ancestors of the modern PCIe
But the concept of using fast bus based SSDs as storage
accelerators wasn't a new idea. It went back 3 to 4 decades.
earlier market experiments with solid state storage were always short lived -
because the expensive SSD storage in each product generation was always
competing with fast changing improvements in CPU clock speeds, bus memory
throughput or faster external magnetic storage media.
The changes in
the modern era of
SSDs - which started about 2003 - was that due to those other computer
technologies stagnating and not getting any faster - the only competitor which
killed an SSD from about that time - was another SSD.
By that time -
the new computer bus was known to be PCI express. But it was the failure of the
alternatives to solid state storage to get faster - which made the big
difference to the business viability of SSDs - rather than any inate
characteristic of PCIe SSDs.
changed in SSD year 2014?|
how fast can your SSD
PCIe SSDs versus
memory channel SSDs
SSDs - the Survive and Thrive Guide
conversation with PLX about PCIe fabric etc
what's so different about the
design of Fusion-io's PCIe SSDs?
SSDs - state of the market & technology- FMS 2014 (pdf)
hidden segments in the enterprise for rackmount SSDs
|Greenliant enters PCIe SSD
Editor:- January 12, 2015 - Greenliant Systems
- has entered the entry level enterprise
NVME PCIe SSD market -
of its new G7100 (pdf)
series MLC gen 2 x4 PCIe SSDs - upto 2.75 TB raw capacity, Full Height, Half
Length form factor, 130K / 60K
of upto 10 DWPD for 5
Internally Greenliant's new PCIe SSD has a
RAID protected array of
miniature NANDrive SSDs which use the company's own controllers.
our in-depth flash memory knowledge and volume-proven NAND controller expertise,
Greenliant is addressing the industry's increasing need for higher reliability,
higher performance and larger capacity flash-based storage solutions," said
CEO of Greenliant Systems.
Editor's comments:- ever since the
company was founded - in 2010 - Greenliant's focus has been on the
embedded SSD market. So it's surprising to see this new product aimed at
the "enterprise" market. However it's part of an
trend in the market.
Samsung mass producing gen 3 PCIe SSDs for notebooks
January 7, 2015 - 19 months after launching its first M.2 PCIe gen 2 SSDs
aimed at the
(pdf) in June
2013) - Samsung
said it is now mass producing the follow-up SM951 - which supports gen 3
Samsung says - "For ultra-slim notebooks and workstations
the SM951 can read and write sequentially at 2,150MB/s and 1,550 MB/s
The SM951 is the first SSD to adopt L1.2 low
power standby mode (the PCIe
SSD equivalent of the power saving
mode in SATA SSDs) .
When hibernating in
mode, the SM951's power consumption is less than 2mW.
Toshiba shows early version of BGA PCIe SSD
January 7, 2015 - Toshiba
it will showcase a prototype of the world's first PCIe single package SSD -
with up to 256GB in a single BGA package at
CES this week.
compatible device fits into 16mm x 20mm x 1.65mm and weighs under 1g. See
also:- BGA SSDs
What next - after PCIe SSDs everywhere?
December 2, 2014 - What next - when PCIe SSDs are already everywhere? You
know you need them. But you need more too.
Signs of interesting times ahead in 2015 are revealed in the new
edition of the
Top SSD Companies -
based on market metrics in
Q3 2014 -
researched and published by StorageSearch.com
over 5 years - Fusion-io
had occupied the top #1 spot in this list as the SSD company which
was most researched by our readers. That was inevitably going to change. And it
did. But not in the way you might have expected.
The new #1 SSD
company is Diablo
Technologies - creator of the Memory Channel Storage platform.
that's not the only sign of change.
7 years after the
2007 Year of SSD
revolutions - is the enterprise SSD market about to begin a new revolution
in server centric SSD architecture? ...read the article
the key band members - which made Fusion-io so very famous -
have reunited at Primary Data
Editor:- November 19, 2014 - "With
Woz on the
team along with Lance
we now have the band back together, and I'm amped to be reunited at Primary Data"
said Rick White,
Data today - referring to the fact that 4 of the best known people in
(including the 2 founders) were now at the helm of the new company on the day it
emerged from stealth mode. ...read
more in SSD news
why did Memblaze demonstrate 14µS latency PCIe SSDs in
Europe before US?
Editor:- October 31, 2014 - Something I saw in
a press release this week by Memblaze Technology
made me think I'd better pause to check the significance behind the words
before mentioning it here.
The story - from their
PR agency -
A3 Communications - was about
the launch of a new PCIe
SSD - the EBlaze3 - in Europe - which contained the phrase.
will be the first time the technology is available outside China."
wondered if - behind that statement - there might have been an implied message
that the new product will be sold preferentially in Europe - rather than the
US - as we have seen evolving in stories about one of the world's biggest
enterprise companies - to have emerged from China -
this wording - and the showing of Memblaze's EBlaze3 at a trade show this
week in Germany - simply due to the timing of having booked booths at trade
shows and the availability of the new product - rather than an expression of
preference about which market - the US or EU - is more important for their new
enterprise SSD to be seen in?
So I asked Ron MacLeod - who is VP
of marketing for Memblaze in Europe - to clarify these points.
- "The product will be available in the US in the same way it is available
in Europe. And with regard to timing indeed it was just coincidence that the
Powering the Cloud event was running and at which was the first time Memblaze
was public in Europe. There is no favouritism over markets except the US is
potentially larger - so no hidden messaging. We value the European business and
other potential global partners equally."
Editor's comments:- So that's why I delayed mentioning this before.
- The Memblaze EBlaze3 (upto 2.4TB in HHHL and 4.8TB in FHHL single slot) is
a new version of the
card - which is aimed at, Hyperscale and OCP (Open Compute Project
market). It offers an ultra-low latency of 14µs and enables
customers to define and alter key settings to ensure optimum performance in
their specific applications and platforms.
StorageReview.com tests Fusion-io's Atomic SSD
October 21, 2014 - Earlier this year - in
June 2014 - and
just weeks before SanDisk
announced it was acquiring the company - Fusion-io launched a
new generation of PCIe
SSDs - the Atomic Series - which was an overdue catching up exercise to
use cheaper 20nm flash.
At the time the headline product specifications seemed to me to be
adequate - rather than spectacular.
It's probably safe to assume they
were intended to restore the competitiveness of Fusion-io's products rather
than push performance boundaries.
But what are they like? And how
do they compare?
A new article -
ioMemory PX600 Review - published in StorageReview.com provides the
answers to how it performs and how it compares on popular OS platforms and
benchmarks. ...read the
how fast can your
SSD run backwards?, SSD
testing & analyzer news
Samsung mass produces 3TB 3D PCIe SSDs
September 25, 2014 - Samsung
today announced it has started mass producing 3.2 TB NVMe
PCIe SSDs (HHHL)
based on its 3D flash memory
technology, for use in enterprise systems.
new NVMe PCIe SSD, SM1715 provides a sequential R/W speeds upto 3GB/s and
2.2GB/s respectively with
rated at 10 DWPD for 5
Editor's comments:- Samsung supplied an
for the new SSD - but forgot to include the block size to which it relates - so
I've asked for clarification.
Seagate launches new improved Nytro PCIe SSDs
September 10, 2014 - Seagate
2 new PCIe SSDs -
which are based on the SSD product lines and brand assets of the recently
acquired SSD business of LSI.
XP6302 is a HHHL, gen 3 PCIe SSD - which provides up to 1.75 TB of
usable eMLC capacity with 200 microseconds average latency, and 295K/79K
(8KB) and rated for 0.9 DWPD
endurance for 5 years.
XP6210 is a FHHL gen 2 PCIe SSD with 1.86TB usable 19nm cMLC
capacity, with 50 microseconds average latency 185K/120K R/W IOPS (8KB),
and rated at 1.6 DWPD
endurance for 5 years.
HGST announces 2nd generation clustering software for FlashMAX
Editor:- September 9, 2014 - HGST today
a new improved version of the
clustering capability previously available in the
PCIe SSD product line
acquired last year from Virident.
allows clustering of up to 128 servers and 16 PCIe storage devices to deliver
one or more shared volumes of high performance flash storage with a total usable
capacity of more than 38TB.
HGST says its Virident HA provides a "high-throughput,
low-latency synchronous replication across servers for data residing on FlashMAX
PCIe devices. If the primary server fails, the secondary server can
automatically start a standby copy of your application using the secondary
replica of the data."
For more details see -
Virident Software 2.0 (pdf)
Editor's comments:- This
capability had already been demonstrated last year - and
ESG reported on the
technology in January
But at that time - the clustering product called vShare -
was restricted to a small number of servers - and the data access fabric was
restricted to Infiniband
With the rev 2.0 software - the number of connected devices has
increased - and users also have the lower cost option of using
Ethernet as an alternative
OCZ samples hot swap, fast 2.5" NVMe SSDs
September 9, 2014 - OCZ
that this month it will begin sampling a new
swappable enterprise PCIe SSD - the Z-Drive 6000 - is a native PCIe
3.0 NVMe 1.1 solution - which the company says "provides industry-leading
IOPS per dollar". It has a SFF-8639 connector, internal RAID, power
loss data protection, "consistent low latency", and encryption.
comments:- Although OCZ demonstrated the SSD industry's first working 3.5"
PCIe SSD prototype 4 years ago - in
August 2010 - the
company didn't follow through to establish an early lead in its natural
successor - the 2.5" enterprise PCIe market - where current leaders today
are Micron and
Samsung. The main
reason for that was financial problems in OCZ in the years which
immediately followed - which mitigated against products which didn't have a
short term guaranteed market.
Now, however, with OCZ having been
almost a year as a Toshiba group company - the small form factor enterprise
NVMe market looks like a natural fit for OCZ - as an extension of its long
running conventional form factor PCIe SSD accelerator business.
StorageSearch.com updates 10 key SSD ideas in 2014
September 5, 2014 - Are SSD years getting faster? It's not just the SSDs.
Anyway it seemed to me that an end of year SSD market update was due
even though 1/3 of this year hasn't happened yet.
That's why StorageSearch.com
this week published a new home page blog -
10 key SSD ideas in 2014.
As you might expect - PCIe SSDs come into the picture. But there's
more to it than that. If you're impatient - you don't have you don't have to
wait till January 2015. ...read
Plextor's M.2 PCIe SSD wins award at FMS
August 7, 2014 - Plextor
- an M.2 PCIe SSD - has
won Best of Show for most Innovative flash memory technology at the Flash Memory Summit.
M6e SSD combines a multi-core
Marvell PCIe 9183
controller and Toshiba
toggle NAND flash with firmware developed by Plextor's in-house team.
say hello to Shannon Systems
Editor:- August 6, 2014
- I hadn't heard of Shannon
Systems before. But I got a nice email this morning from Xueshi Yang,
CEO and co-founder who said he has been reading StorageSearch.com "for
quite a number of years now" and also said that his company is showing
their products at Flash Memory
Among other things - Xueshi Yang said - "Shannon
System is a startup I co-founded in 2011 in China after I left
company is dedicated to the enterprise flash storage market. Currently, we focus
on the high performance PCIe market with our proprietary controllers and
software systems. In April this year, we
the industry first 6.4TB PCIe SSD with a single controller, which boasts 67us
read access latency and 9 us write access latency (all in 4KB, random). While in
June, we introduced a PCIe SSD with SFF-8639 interface, which is hot-pluggable.
We currently serve over 100 customers in China, including Tier 1 internet
companies, as well as other named customers such as China Mobile, China Telecomm
eASIC supports Mobiveil's NVMe platform
August 6, 2014 - eASIC
announced support for Mobiveil's
NVMe platform (pdf)
implemented in eASIC devices.
"eASIC is enabling the rapid
deployment of SSD technology at substantially lower cost and up to 70% lower
power than alternative solutions", said Jasbinder Bhoot,
VP of Worldwide Marketing at eASIC. "By working with Mobiveil, customers
will have access to a complete NVMe solution running in cost, power and
performance optimized eASIC devices."
HGST rekindles concept of a PCM based PCIe SSD
August 4, 2014 - HGST
it will demonstrate a PCM PCIe SSD concept at the Flash Memory Summit. HGST says
the demonstration model delivers 3 million random read IOPS (512 Bytes) and
a random read access latency of 1.5 microseconds.
funded the world's first enterprise PCM PCIe SSD demo 3 years ago (in
June 2011). The
storage density of PCM resulted in an SSD which had pitifully low capacity
compared to flash memory at that time - and earlier this year (in January 2014)
that Micron had temporarily abandoned this idea.
Is HGST really
going to wander into memory space where even the memory makers don't want to go?
Or is this just a market signal that HGST isn't just looking at short term SSD
Lite-On enters enterprise M.2 PCIe SSD market
July 30, 2014 - Lite-On
will be showing a new M.2 PCIe
SSD for the enterprise market next week at the Flash Memory Summit.
The P1P is a
PCIe SSD with capacities of 1TB and
protection circuit all in a small form factor.
The small form
factor and high capacity allow enterprise customers to pack more storage in a
smaller footprint. The P1P Series can deliver sequential R/W speeds up to
610MB/s and 520MB/s while random R/W speeds can be upto 95K/15K
drive has a MTBF
of 2 million hours and an
rating of up to 1 drive write
per day for 5 years.
OCZ announces availability of ZD-XL SQL Accelerator 1.5
July 15, 2014 - OCZ
this week announced availability of
1.5 of its ZD-XL SQL Accelerator (PCIe SSD and caching software bundle) the
beta version of which was announced in
SanDisk to buy Fusion-io
Editor:- June 16, 2014 -
a definitive agreement to acquire Fusion-io in
all-cash transaction valued at approximately $1.1 billion.
comments:- What will SanDisk really get from Fusion-io?
the short version from me.
"The ability to get more enterprise
petabytes out from the same raw flash chips in - by shipping it through better
architecture - is a more significant business factor in the flash memory market
today than the ability to do another cell geometry shrink - or to add a few more
toppings on the 3D pizza. "
For the much longer version(s) - see
my article in SSD news
PLDA enters the PCIe SSD controller market with 6.6 Gb/s NVMe
Editor:- June 9, 2014 - PLDA today
immediate availability of a new
SSD controller aimed
at the PCIe SSD market.
IP is a high performance, low latency, highly-configurable PCI Express 3.0
soft IP supporting endpoint, root port, switch and bridge configurations. The
certified design has been engineered for both ASIC/SoC and FPGA implementations.
comments:- PLDA has written a product description -
challenges of PCIe with NVMe in order to deliver highly competitive Enterprise
PCIe SSDs (pdf) - which contains an overview of the design. Here are some
- power saving features? - PLDA implements L1 PM Sub-states, reducing power
consumption without affecting performance.
Here are some questions I put to the
- performance? - 270K write IOPS, 12µS write latency write (4KB) using
the FPGA-based reference design.
- is there a minimum order quantity or cost associated with licensing your
No minimum quantity.
- What is your deliverable?
An Encrypted Source code file that can
be downloaded or sent on CD
- How does a customer evaluate your IP?
We provide 1 month Free Evaluation IP
new article re PCIe and SSDs
Editor:- June 5, 2014 -
published a new article -
conversation with PLX.
That concept diagram they've been talking about for the past 3 years -
may soon become a 1U PCIe fabric box reality which you can use to glue your
servers and SSDs together.
new PCIe SSDs from Fusion-io
Editor:- June 5, 2014 -
2 new gen 2.0 x8 standard height, half length PCIe SSD product lines - which
they call the Atomic Series
- which are available in 2 different price and application categories.
series - upto 5.2TB usable capacity, with 92µS / 15µS R/W
latency, 2.7GB/s / 2.1GB /s R/W throughput and 375K write IOPS - designed for
high end servers - with 5 years warranty.
Fusion-io says that more than 7,000 customers in
over 80 countries use its products to reduce latency and speed the flow of
- upto 6.4TB usable capacity - with similar latency to PX600 - but lower
throughput (2.6GB/s / 1.2GB/s R/W throughput) and 16% lower power consumption -
designed and priced for scale-out architectures and hyperscale environments -
with 3 years warranty.
Hynix buys Violin's PCIe SSD product line
May 29, 2014 - Today in line with previous guidance signals - Violin Memory (which in
January 2013 had
been observed to tentatively dip its toes as a late entrant into the raging
torrents of the PCIe SSD
market - but which had not been regarded by me as very likely to survive
a rigorous swim)
that it has sold its PCIe SSD product line to Hynix which paid $23
million in cash and assumed certain liabilities totaling $0.5 million.
Marvell samples controller for SATAe SSDs
May 21, 2014 - If you're designing SATA Express SSDs then Marvell today
it is sampling the 1st SSD
controller specifically designed for the SATAe market - which will
enable the design of 2.5"
PCIe SSDs at costs which could be
fast SATA SSDs.
88SS1083 is a 2 lane PCIe Gen2 SSD controller with transfer rates up to
1GB/s. It supports DevSleep and L1.2 PCIe low power state - to minimize power
consumption in notebook
and enterprise array
EMC acquires (memory channel SSD in a box but connected via PCIe
Editor:- May 5, 2014 - EMC has acquired a stealth
mode rackmount SSD company - DSSD
- it was announced
Products based on the new DSSD architecture - which connects
to servers via PCIe - are expected to be available in 2015.
comments:- for a competitive analysis and technology perspective see
rackmount SSD news
the Top SSD Companies in Q1 2014
Editor:- April 30,
2014 - StorageSearch.com
today published the 28th
quarterly edition of the
Top SSD Companies List
based on metrics in Q1 2014. Newcomers to the list included Maxta and A3CUBE and there were
significant movements in the top 10 companies. ...read the article
LSI's Nytro works with BPE in SQL Server 2014
April 24, 2014 - A lot of SSD companies have announced that their products
are compatible with Microsoft's SQL Server 2014. Another noteworthy company in
that category is LSI
- who today
flash accelerator cards (PCIe SSDs) work with the new
Pool Extension feature.
OCZ launches new desktop PCIe SSD
Editor:- April 24,
2014 - OCZ today
a new gen 2 PCIe SSD
for high performance desktop uses such as gaming and workstation. The Linux and
Windows compatible RevoDrive
350 (240GB, 480GB, and 960GB capacity models) has R/W speeds upto 1.8 / 1.7
GB/s and 140,000 4K random write IOPS.
Editor's comments:- a
key difference between this type of prosumer desktop SSD and the kind you need
in intensive 24x7 server slots is
OCZ says the RevoDrive 350 is rated at "upto 50GB of host writes
per day for 3 years". That may be OK for a desktop - but if you translate
that into enterprise jargon - DWPD
for 5 years - it translates into 0.1 - which is 40x lower than a typical
enterprise PCIe drive.
Fusion-io says - we don't regard NVM DIMMs as a threat to our
PCIe SSD business
Editor:- April 23, 2014 - Fusion-io today
revenue up 14% yoy to $100 million but losses have grown too - to $30
million for the recent quarter.
Editor's comments:- Here are
some technology points which emerged from the investor conference call.
- Threat to FIO's PCIe SSD business from
said - he thought NVM DIMMs were a validation of what FIO had already done -
but he regarded NVM DIMMs (in their current form) as a niche market. He went
on to say - "We don't believe that DDR-3 and the memory interface is the
bottleneck. It's about managing the nand and getting the best performance out of
- Re 3rd generation iodrives? - These are being qualified. FIO's 3rd
generation products - and the 5th generation of nand flash geometry used by the
company - will be available in the next month or so.
if memory channel SSDs cost nothing - would there still be a
market for PCIe SSDs?
Editor:- April 17, 2014 - StorageSearch.com today published
a new blog - memory channel SSDs versus
PCIe SSDs are these really different markets? - which revisits (from a
current perspective) some of the key questions I posed a year ago about
Memory Channel SSDs.
Are these really different markets? And - at
of price - what are
the different application roles - where one type of SSD is better than the
other? ...read the article
Editor:- April 16, 2014 - BiTMICRO recently
announced that Bharadwaj
Pudipeddi has joined the company as VP of Engineering and Lead
Architect. Pudipeddi's past design roles in notable SSD companies include 2
years at Violin Memory.
what's the market for Plextor's PCIe SSD?
April 7, 2014 - I asked Plextor's virtual
marketing representative in the US - Andrew Erickson at
Alaniz Marketing to tell me more
about Plextor's thinking about routes to market for their entry level (Gen2)
PCIe SSD market
offering - the
said - "The M6e is marketed directly to consumers as an end-user upgrade
and mostly via newegg and major e-tailers."
Andrew went on to say - "Plextor is definitely moving into the
enterprise space and we're expecting to see a couple of read intensive and
multi-use ssd products later this year. It sounds like some of the enterprise
firmware advances may already be evident in the M6e - though it's not being
marketed in that way. I have seen some Plextor drives sold in high end gaming
rigs, but I'm not aware of current plans to sell OEM at this time."
Samsung's new 2.5" NVMe PCIe SSD
March 25, 2014 - Although Samsung's own
meat navigation inspired
SSD selection page doesn't list this product yet (at the time of writing
this) Samsung is engaged in the
2.5" PCIe SSD
market - and is shipping a 1.6TB NVMe PCIe SSD rated at
7 DWPD for 5
years to Dell for use
in its PowerEdge
PCIe Switching - new article in EnterpriseTech.com
March 13, 2014 - A new article in EnterpriseTech.com
Switching Takes on Ethernet, InfiniBand - reviews the antecedents and
current state of the PCIe fabric market.
This should be of interest
to anyone thinking about the emerging architectural influences which may
impact their plans within the
PCIe SSD market.
of the many vendors discussed in the article - PLX - says it's "not
targeting warehouse-scale datacenters... but is rather thinking on a smaller
scale, from hundreds to thousands of nodes." ...read
a new SSDserver nomenclature?
Editor:- March 5, 2014
- In a new home page blog today - StorageSearch.com
published proposals for a new shorthand for SSD enhanced enterprise
servers in the context of complex architectural dialogs. ...read the article
(You won't need this if all your projects look the same, or if your plans only
include small numbers of SSDservers.)
OCZ launches Z-Drive 4500 - 19nm enterprise PCIe SSD
March 4, 2014 - OCZ
is still using LSI's
SandForce SSD controllers
SF-2582 enterprise SATA (pdf)) in its newest PCIe SSD - the
Z-Drive 4500 Series
today - which has upto 3.2TB of usable
19nm flash, R/W bandwidth
of 2.9GB/s and 2.2 GB/s respectively, and 252K / 76 K R/W
in a FHHL form factor and is integrated with OCZ's new WXL caching software.
4500 briefing notes (pdf)
Editor's comments:- As I
reported in an interview
last November - OCZ's VXL bundles have been very successful in small to
medium scale enterprise deployments. The evolution of this product line -
supporting as it does another new generation of (lower cost) memory - will
further extend its reach.
A3CUBE unveils PCIe memory fabric for
PCIe SSD architectures
Editor:- February 25, 2014 -
PCIe SSDs can now
access a true PCIe connected shared memory fabric designed by A3CUBE - which exited
stealth today with the
of their remote shared broadcast memory network -
RONNIEE Express -
which provides 700nS (nanoseconds) raw latency (4 byte message) and which
enables message throughput - via standard PCIe - which is 8x better
Editor's comments:- I spoke to the
recently - who say they intend to make this an affordable mainstream
The idea of using PCIe as a fabric to share data at low
latency and with fast throughput across a set of closely located servers
isn't a new one.
The world's leading PCIe chipmaker
PLX started educating
designers and systems architects about these possibilities
few years ago - as a way to elegantly answer a new set of scalability
problems caused by the increasing adoption of PCIe SSDs. These questions
- how do you make this expensive resource available to more servers?
the least year or so - we've seen most of the leading vendors in the enterprise
PCIe SSD market leverage some of the new features in PCIe chips - to
implement high availability SSDs with low latency.
- how do you enable a simple to implement failover mechanism - so that data
remains accessible in the event of either a server or SSD fault?
But although there
are many ways of doing this - the details are different for each vendor.
- until now - if you wanted to share data at PCIe-like latency across a bunch
of PCIe SSDs from different companies - located in different boxes - the
simplest way to do that was to bridge across ethernet or infiniband. - And even
though it has been technically possible with standard software packages - the
integration, education and support issues - compared to legacy SAN or NAS
techniques would be extremely daunting.
That's where A3CUBE comes into
the picture. Their concept is to provide a box which enables any supported PCIe
device to connect to any other - at low latency and with high throughput -
in an architecture which scales to many thousands of nodes.
heart of this is a shared broadcast memory window - of 128Mbytes - which can be
viewed simultaneously by any of the attached ports.
ever used shared remote memory in a supercomputer style of system design at
any time in the past 20 years or so - you'll know that the critical thing is how
the latency grows as you add more ports. So that was one of the questions I
Here's what I was told - "The latency is related to the
dimension of the packet for example: In a real application using a range of
64-256 bytes of messages the 3D torus latency doubled after 1,000 nodes.
With larger packets, the number of nodes to double the latency becomes grater.
But the real point is that the latency of a simple p2p in a standard 10GE is
reached after 29,000 nodes.
"A more clear example of the scalability of the system is this.
Imagine that an application experiences a max latency of 4 us with 64 nodes, now
we want to scale to 1,000 nodes the max latency that the same application
experience will became 4.9 us. 0.9 us of extra latency for 936 more nodes."
Editor again:- Those are very impressive examples - and demonstrates that the
"scalability" is inherent in the original product design.
didn't want to say publicly what the costs of the nodes and the box are at this
stage. But they answered the question a different way.
Their aim is to
price the architecture so that it works out cheaper to run than the legacy
(pre-PCIe SSD era) alternatives - and they're hoping that server oems and fast
SSD oems will find A3CUBE's way of doing this PCIe fabric scalability stuff -
is the ideal way they want to go.
There's a lot more we have to learn
- and a lot of testing to be done and software to be written - but for users
whose nightmare questions have been - how do I easily scale up to a 10,000
PCIe SSD resource - and when I've got it - how can I simplify changing
suppliers? - there's a new safety net being woven. Here are the
PLX is ready with 1st Gen3 compliant PCIe switches
February 4, 2014 - PLX
announced it is
the 1st PCIe switch vendor to have achieved Gen3 compliance having passed the
compliance testing procedures of PCI-SIG.
Editor's comments:- the new standard doubles the maximum
data bandwidth of PCIe SSDs to 1GB/s per lane in each direction - which enable
32GB/s total throughput for a x16 link.
SSD interface chips,
Express 3.0 Integrators List
ESG reports on clustering Virident's PCIe SSDs
January 23, 2014 - ESG
today published a
report for Virident
/ HGST's FlashMAX
II (PCIe SSDs)
which validated the ability of this product family and its
(vShare ) to be configured for useful operation in an Oracle RAC environment
availability configurations which were clustered via
ESG said performance was good they also commented on some current
limitations of the product suite for this type of application. In particular:-
- the lack of a graphical interface for setup and performance mintoring,, and
- the lack of support for other supported fabrics such as 10GbE (mentioned
in the report as a future option), and PCIe fabric (which was not mentioned at
all in this report).
This is what it takes to be #2 in PCIe SSDs
January 22, 2014 - LSI
said today in its
financial report it has shipped over 100,000
PCIe SSDs since it
entered this market in April 2012.
Editor's comments:- LSI
shipped 60,000 units in the 8 months from May to December 2013 (a figure
calculated by subtracting the numbers contained in an
Despite those impressive numbers - LSI itself
estimates that it is still the #2 company in this market. Which gives you an
idea of how large this market has grown.
IBM launches first memory channel SSD based servers
January 16, 2014 - IBM
today disclosed preliminary test results comparing the latency of
SSDs to PCIe SSDs - in the
press release of its new X6 architecture for X86 servers.
- "Our evaluators are seeing 5-10 microseconds write latency for eXFlash
DIMMs in preliminary testing vs. 15-19 microseconds latency for PCIe-based flash
storage from Fusion IO,
Virident, and 65
microseconds latency for Intel
S3500 and S3700 SSDs."
IBM also announced enhancements and sales
figures for the rackmount
SSD product line it acquired from
Texas Memory Systems a
year ago. See SSD news for
more about the latter.
reasons to expect more PCIe SSDs in 2014 PC Market
December 16, 2013 - A new
in DRAMeXchange discusses the issue of
SATA 3 SSDs being
replaced by PCIe SSDs
in the consumer SSD
market in future designs of PCs. ...read
how to avoid hot pluggable PCIe SSD failures
December 3, 2013 - What happens if you test
PCIe SSDs for their
sensitivity to data
corruption or even failure
- in the event of
You'd think that with more hot pluggable products coming
into the market -especially in the
2.5" form factor
- that the experimental outcomes would be known by the designers and problems
debugged so that users wouldn't have to worry.
In August 2013 Quarch Technology
launched some special
test equipemnt to inject power related faults into PCIe SSDs - and the
company today announced
it has extended this range to automate power line error testing of PCIe SFF
Technical Director at Quarch Technology told me today that "Almost every
combination of test kit we have tried in Quarch (a number of friendly customers
lent us kit and eval drives to get the new module up and running) has failed in
some way. Sometimes failing to come back up again nicely, sometimes with a full
BSOD which will almost
certainly have risked data loss."
estimating the size of the server-side SSD market etc...
October 8, 2013 - Before you make that next presentation about what's happening
in the business world of enterprise flash, or before you commit to any future
datelines for hard drives being sold into the enterprise - you'd be well
meet Ken and the
enterprise SSD software event horizon - the (long anticipated) new home
page blog on StorageSearch.com.
|archived versions of this PCIe SSDs news page
|the SSD Buyers Guide|
where are we now with SSD software?
SSDs for newbies - what are they etc
|the 3 fastest flash
PCIe SSDs - list / lists|
|Are you tied up in
knots trying to shortlist flash SSD accelerators ranked according to
published comparative benchmarks?|
You know the sort of thing I mean -
where a magazine compares 10 SSDs or a blogger compares 2 SSDs against each
other. It would be nice to have a shortlist so that you don't have to waste too
much of your own valuable time testing unsuitable candidates wouldn't it?
StorageSearch's long running
fastest SSDs directory
typically indicates 1 main product in each form factor category but those
examples may not be compatible with your own ecosystem.
If so a
new article -
the 3 fastest PCIe
SSDs list (or is it really lists?) may help you cut that Gordian
knot. Hmm... you may be thinking that StorageSearch's editor never gives easy
answers to SSD questions if more complicated ones are available.
||But in this case you'd be
wrong. (I didn't say you'd like the answers, though.) ...read the article|
|How are fault tolerant
PCIe SSD designs supported in chips?|
enterprise SSD designs - this video by PLX includes an
introductory tutorial into PCIe and its performance and architectural
capabilities for SSDs including automatic failover and multi-host capabilities.|
|PLX's switch chips also supports failover if
the fault occurs in the PCIe switch fabric chips themselves.
extract - "...And in case one of the hosts fails
and you want to connect the SSDs - or the devices connected to that host - to
another host - that can be done automatically as well - and the surviving host
can attach the devices that were attached to the failing host to itself and
control it so that the system doesn't go down and the data stored in these
devices doesn't get isolated from the main system."
|How Big is the Market for
|In 2010 I wrote here that I confidently
expect that PCIe SSDs will become a multi-billion dollar a year market.|
now the standard SSD
Since 2010 most of
the top 10 SSD companies
also sold PCIe SSDs.
For more about PCIe SSDs fit into the enterprise SSD landscape see -
introduction to SSD
|"This is a tsunami warning
event for SSD vendors addressing the enterprise server acceleration market."|
|...editor's comments (September 24, 2009 ) when
I alerted readers and vendors to the fact that search volume for PCIe SSDs
had surpassed that for 2.5" SSDs for the first time. |
type of search spike had been a reliable advance
predictor for new
interfaces - such as SATA
and iSCSI - in earlier
phases of storage
history - and search volume has also been a good predictor for
successful SSD companies
SSDs Technical Pros and Cons|
|The great attraction of PCIe for SSD
oems is that it can support a wide range of performance options with throughput
upto 16GB/s, and much lower attachment costs than the
older busses like PCI and cPCI also provide performance which is adequate for
Bus connected SSDs have been around since the
of the SSD market.
The advantage of this approach is high
throughput and low latency compared to SSDs connected via traditional hard disk
style interfaces like
there are disadvantages too which include:-
1 - Bus style
interfaces reduce the available market for the SSD oem. Because older servers
may not have the interface, or perhaps the interface (for example Sun's SBus) is
proprietary and is only available in a small range of models.
2 - Bus
interfaces tend to have shorter permissable cable lengths - which restrict how
such SSDs can be connected.
3 - Bus interfaces usually don't include
intrinsic end to end error detection and correction. If the physical arrangement
of the SSD pushes the speed and cable lengths too far - then errors can arise in
the bus connect - which have to be dealt with in the associated driver.
May 13, 2009 - Dolphin's
CTO, Venkat Krishnan emailed this article correction.
StorExpress addresses concerns of PCIe direct attached SSDs raised in (2) above.
It includes support for different types of PCIe interfaces (ExpressModule, AMC,
etc.). Multiple PCIe SSD cards can be used without requiring multiple PCIe slots
in the server. The storage can be collocated at distances of up to 300m from the
server and can also be potentially shared by more than one server."