what's so very different about the design of Fusion-io's ioDrives / PCIe
SSDs?re Fusion-io's ioDrive and ioMemory
editor - December 2011 - a classic article
If you ask
most people who know
about the SSD market - which company do you think of when you think about the
concept of PCIe SSDs? - the answer is
Fusion-io - the
company which evangelized the concept of using
MLC flash in
enterprise PCIe SSD accelerators - in new ways which break away from
decades of performance sapping legacy
hard disk software.
Fusion-io ignited the interest of the world's leading server makers
too - transforming accelerator SSDs from a hard to find after-thought into a
"must have" factory shipped option.
Fusion-io has lots of
different models which span the range from products which accelerate graphics
effects workstations, scalable value based accelerators which accelerate some
of the world's largest online websites and high-end products for supercomputer
modeling and analysis apps.
In this this article - which was prompted
by many questions from readers - I try to answer the question - what's so very
different about the design of Fusion-io's ioDrives / PCIe SSDs?
begin with taking a step back and asking - Why would you want to install an
The obvious answer is - to make your storage and
applications run faster.
But if you have a preference - which should
This is a case of where the raw numbers -
without the narrative - fail to tell the full story.
- Speeding up the storage? - or
SSDs are unique in today's enterprise flash accelerator market in that
they don't include a microprocessor on-board the SSD card to implement the
functions of the
SSD controller. Doing
the flash management isn't as big a load as you might suppose for the ioDrive
host - because it's still less than the traditional packeting and handshaking
workload of talking to legacy storage interfaces. In other respects FIO's SSDs
fit into in my classified groupings of SSD architecture types as follows:-
What are the consequences of the "host"
CPU also being the flash controller?
- ioDrives are -
This was a bold move by the
fledgling company when it started. Throughout
all past SSD
history - the designers of SSDs had gone to great lengths to hide their
memory handling intellectual property inside chips like FPGAs and ASICs. One
business result of this was that in the early days Fusion-io had to be careful
about picking its customers - so they made sure they chose big name reputable
companies - whether they were end users (like Facebook) or server oems (like HP
etc) - who would not reverse engineer the algorithms - or if they did - would
be worth suing. Of course nowadays Fusion-io is a big enough company itself that
it has volume based cost and brand advantages - so even if another company were
to copy some of their SSD technology - why would anyone buy it. But when
Fusion-io's founders started down this track it was a very risky move - so
why do it?
The consequences of this architectural decison are:-
- less hardware in the SSD card - so much better intrinsic
potentially the lowest cost to build product
- faster access to flash data by wrap around software APIs
- greater stickiness with customers - once customers start seeing the
benefits from leveraging the features - competing products with bland raw
speed don't look so attractive
back to my question at the start of this piece. Because ioDrive's don't have
on-board offload processors - in a benchmark which simply looks at raw storage
performance - ioDrive's (which are very fast) don't always look the fastest.
Because it's possible to
in which the offload plays a bigger part - so other competing SSDs will
sometimes look faster.
- speed of the SSD family is predictably scalable over future product
That's important for customers who don't want the risk of
being locked into a design which looks fast this year - but might not be able to
keep up with future CPU advances. If you look at the philosophy of the Fusion-io
designs - one view you can take is that the SSD makes the host CPU work faster
- because it speeds up access to the data. Another view - equally valid - is
that because the host CPU is doing work for the flash memory - the faster
the user's CPU - the faster the SSD.
A practical consequence for
server oems is this. If they have a bunch of ioDrives in stock and they put a
faster CPU in their new server - some of that speedup also makes the ioDrives
work a bit faster too - in a way which is different to products which use
on-board microprocessors. And unlike SSD companies which use microprocessors -
as long as host CPUs get faster and flash gets faster - then it's easy to
predict that future ioDrives can be faster too (this is what I call
But if you look an application speed
- and if the app uses Fusion-io's APIs - which they call VSL - you see a
Suppose your software does a lot of twiddly bits very
fast and very often. No matter how fast or clever the offload processor is on
another type of PCIe SSD card - there is still a level of inefficiency that
comes in - because your software is talking to other software which has to go
and look at the flash somewhere else. But in the ioDrive - the APIs are inside
the flash controller loop. You can access (cleaned up and
managed) data almost at the same latency as if you had designed the SSD
yourself for this particular app.
That's why with the ioMemory
architecture the application can speed up at a faster rate than you
might think by simply looking at comparative raw storage benchmarks. But you
only have this level of control and software gain factor in a new
installation - or a new design of storage appliance. That's something I
discussed in my article
Legacy vs New
Dynasty a new way of looking at enterprise SSDs. You can probably guess -
that Fusion-io is New Dynasty - and that's a description they liked when they
read my article. (And their competitors on the Legacy side of the fence liked
their own appellation too.)
If you're already in a distressed
environment with servers overloaded and legacy apps which are running slow -
adding an SSD accelerator may help - but the advantages and disadvantages of the
ioMemory compared to alternative products are difficult or impossible to
predict. I always say in that situation - it's best to try before you buy.
Cautious minded users may even think about looking at much higher-cost add-on
rackmount SSDs in
those distressed situations - to avoid the risk of doing bad things to their
already suffering old servers.
Another thing to think about in the
legacy scenario is that - simply adding a fast SSD to a server running heavy
duty apps isn't guaranteed to speed up your app.
That's something I
learned in 1990 - when I had a customer with an Oracle app I wanted to speed up
with an SSD. (The data set wasn't I/O bound - it was already small enough to
be sitting entirely in main memory.)
More recently an architect from a
leading software company told me he was surprised to discover that their core
software didn't speed up as much as he expected when they evaluated the impact
of ioDrive PCIe SSDs. When he analyzed the cause - it lay in their own source
code and data architecture. Their software had been written for
hard disk arrays and had
inbuilt assumptions which meant they weren't running enough parallel threads to
take advantage of the greater bandwidth and lower latency of SSDs. They would
have got the same result with any other brand of PCIe SSD - but it made him
realize that there was a lot more to being ready for the solid state storage
future. Of course when they rewrote that software it ran faster.
to success in the SSD market is complicated. I see my job here at the mouse
site as trying to understanding all this complex stuff to help the SSD
industry get better, faster. I hope you've found this helped you too. It's just
one of thousands of SSD related peices I've written in the past 20 years. You
can see the top
100 most popular SSD articles here.
I've spoken to the founders of
Fusion-io many times in the past 4 years and also to systems integrators and
ISVs who have used these products. I also talk to all their main SSD
competitors too to learn how competing products and businesses work. My
notes above are designed to help you quickly understand why this product from
Fusion-io is different - and whether those are differences which matter to you.
noticed that Fusion-io has written a white paper re
Fusion-io Difference (SSD Differentiator). It has a different perspective -
but has nice pictures. For some of you that may be worth seeing too.
still or should it be PPS?
Take a look at this quote - "Hard
drive emulation appeals to silicon providers because it offers the shortest
route to market and requires the least innovation. By emulating disks, SSDs
can be wedged into existing software stacks. Problem is - the native
features of solid state are hidden by emulation. Performance gains are blunted,
protection at the silicon level is ignored, and software interfaces at the cell
level are simply not considered."
It's from a white paper
Going Beyond SSD -
to Software Defined Flash - which Fusion-io probably wrote a long time ago -
but which I only saw recently (in April 2013). The article usefully
encapsulates the key principles which make FIO's SSDs different. (I'm still
going to call them SSDs.)
and now? - rackmounts tooFusion-io was one of the companies which I
discussed in my home page blog - (June 2013) -
inside the box will lead to better enterprise flash arrays
news story (September 2013) I thought it would be useful to recall the history
of FIO's SSDs and technologies inside the rackmount SSD market. For the sake of
speed and simplicity I've cut and pasted it here below. I may tidy it up later
into a separate article.
Fusion-io - upcoming webcast on rackmount
Editor:- September 11, 2013 - The other thing which Fusion-io would like
you to recognize them for - is
The company yesterday
that it will be talking about this in
a webcast tomorrow
(Thursday 9am PT).
Editor's comments:- Using
PCIe SSDs as
components within rackmount
SSDs is already a well established concept in
past - and expected to remain one of the
key uses in enterprise
SSD's future too.
A company called
Dolphin launched the
first such systems back in
Fusion-io's own ioDrives have been integrated by various companies within
rackmount storage systems in the past starting with :-
NextIO - as fast but
software-less storage (April
2010), and notably followed in a very different way (as the raw flash in
availability FC SAN SSDs) by
Kaminario (September 2011).
But for anyone who thought this might be a good idea - but didn't see
why they should have to buy this kind of solution from another new vendor -
they only had to wait till
August 2012 -
which is when Fusion-io launched its
- a software bundle which enabled any user to build their own legacy software
compatible fast FC SAN
compatible SSD rack using a bunch of iodrives and almost any customer
preferred standard server.
That created the possibility of a new
competitive choice for those in the
But for those interested in SSD acceleration whose needs for
performance and cost were more modest - there would soon be another way they
could use FIO's PCIe SSDs in a different way (as the
flash cache in
iSCSI hybrid HDD
An early example of this in
March 2011 - was
iSCSI hybrid systems by StoneFly
and then later we heard about iSCSI systems by
NexGen Storage (which
Fusion-io acquired in
- for most people in the enterprise SSD market - the mere mortals who haven't
already got inside Fusion-io's priviledged big customer gatekeeper orbit -
it has been exceedingly difficult to get a coherent picture of
whether these systems products are relevant - and if so - how to buy them.
Delays in getting that information were undoubtedly not helped by the
need to assimilate the NexGen products into a business culture in which
hard drives had
previously been anathema - coupled with an ultra competitive market outlook (the
quality and diversity of competing options facing users in this market is
very high) and a reorganization due to the precipitous change in leadership
4 months ago.
So despite the title (Flash First for Hybrid and All Flash Storage)
I'm sure that even more of you than usual - will be interested to learn
what FIO has to say on this particular subject.
more links to FIO
rackmount info - added later
FIO's aggressively priced
hybrid rackmount for the iSCSI
intro) on SlideShare.net
rackmount offering for the FC
SAN SSD market
presentation slides on SlideShare.net
Data Accelerator Data Sheet
|A the start of 2014 I
thought it curious that I hadn't seen a DWPD rating for Fusion-io's ioDrives.
When I did find the information - it prompted me to publish a regularly
updated list of DWPD numbers for leading SSDs.|
the state of DWPD?|
|NexGen decouples from
Fusion-io accelerator juice|
|Editor:- June 30, 2015 - As previously signaled -
has decoupled itself from relying on SanDisk's PCIe SSD
product line in its hybrid storage arrays with the
that it has introduced NVMe readiness as an update in its software
seeing excitement build for a new class of memory:- persistent memory - which
has the persistence capabilities of storage and access performance similar to
"Given this richness of media technologies, we now have the
ability to create systems and data center solutions which combine a variety of
memory types to accelerate applications, reduce power, improve server
consolidation, and more.
"We believe these trends will drive a
new set of software abstractions for these systems which will emerge as
software-defined memory a software driven approach to optimizing memory of all
types in the data center."
Fellow at SanDisk
- in her blog -
Emergence of Software-Defined Memory (October 1, 2014)|
enterprise DRAM architecture?
DIMM wars in
SSD servers - the Memory1 episode
|PCIe SSDs |
changed in SSD year 2014?
guide to enterprise SSDs
how fast can your SSD
directions in rackmount SSDs
Legacy versus New
Dynasty Enterprise SSDs
Efficiency - making the
same SSD - with less flash
conversation with PLX about PCIe fabric etc
you've solved the problem of making SSDs reliable and fast - it's tempting to
create an SSD instruction set which which focuses on application layer needs too
- and not just those of dumb storage"|
promises in-situ SSD processing (July 29, 2014)|
|This is Fusion-io's 22nd
straight quarter at the top of this list
It was a momemtous quarter for Fusion-io as aside from product and
partner announcements (FIO shipped its long anticipated 20nm flash generation
PCIe SSDs - the Atomic Series - and also announced that Dell was rebranding
FIO's fast rackmount SSD IP set - the ION) the market learned that SanDisk had
agreed to acquire Fusion-io.
|the Top SSD Companies in
|"The winners in SSD
software could be as important for data infrastructure as Microsoft was for
PCs, or Oracle was for databases, or Google was for search."|
|all enterprise data
will touch an SSD |
|The result of combining the
product lines from SanDisk and Fusion-io will be an enterprise SSD product
offering which is unmatched in the industry in a broad range of enterprise SSD
SanDisk really get from Fusion-io?|