Memory Channel SSDs - 2013 to 2017This resource page includes
news and articles about low latency, fast flash SSDs which plug into DIMM
sockets and transfer data via interfaces which were originally designed for
This scope includes but is not limited to:- TeraDIMMs,
low latency flash SSDs,
SSDs, DDR-4 SSDs, flash
DIMMs, NVDIMMs (some
types), Storage Class Memory, NVDIMM-X, etc
The scope excludes:-
flash backed DRAM
DIMMs, hybrid DIMMs, NVDIMM-N.
why's Intel so
infatuated with 3DXPoint?
reasons for fading out DRAM
what's RAM really? - RAM in an SSD
where are we
heading with memory intensive systems?
are we ready for
infinitely faster RAM? - (and what would it be worth)
|whatever did really happen
to ULLtraDIMM? |
|Editor:- August 1, 2017 - The recent history
and market adoption of NVDIMMs is similar to the early
of the SSD market in that fascinating products appear at one time and then
fail to get traction to remain in the market in successive memory
generations.The reasons are similar:-
- Competition from other ways of getting similar work done.
case of NVDIMMs not just other types of raw native memory but SSDs in other form
factors too. Such as PCIe
which can be deployed to give approachingly similar performance.
A new blog -
an NVDIMM Primer
(Part 1 of 2) (July 25, 2017) by Jim Handy - founder Objective Analysis offers
this explanation for market demise of the ULLtraDIMM.
- Software support
which is meaningful.
Without a competitive and capacble software base
which can recognize the latent strengths of the new memory technology - the
results you get are never as good as the raw technology can deliver. Or if the
early software is good enough but the capability is single sourced that
deters market growth due to fears of being locked into a proprietary
Jim says "Both
SanDisk and IBM later abandoned the technology, which I have heard was due to
performance issues stemming from the lack of an interrupt pin on the DDR3 bus."
In my own contemporaneous coverage of that product I
wrote about other factors which I thought at the time indicated weaknesses in
that first generation (of its kind) product. These were:-
- there was a SATA bridge inside the DIMMs between the flash and the DDR3
logic. The result was system level performance which was not as great as you
might expect - compared to native enterprise PCIe SSDs.
- for about a year there were legal wrangles surrounding patents associated
with the design which scared off other wouldbe adopters and at one stage a court
order which stopped shipments.
- the ULLtraDIMM guzzled power - so you couldn't just drop it into a standard
motherboard socket without checking that the power tracks had sufficient
As you can see there were certainly enough bullets
to wound (if not kill) the first generation
But its unsung (less sung about) creator had learned the lessons and produced a
superior follow up product.
- the ULLtraDIMM product was not the "reason to buy the company"
product line in 2 successive company acquisitions of its flash technology
parents SMART then SanDisk - so it was just one of several SSD products lines
which were let go.
related blogs on StorageSearch.com
after AFAs -
what's the next box?
NV DIMMs - the flash
backed DRAM kind
the road to DIMM
wars and Diablo's Memory1
where are we
heading with memory intensive systems and software?
and risk reward ratios with big memory "flash as RAM"
|"We are morphing from
a storage hierarchy to a memory hierarchy. This is why I choose to work where I
do. Memory rules."|
Peglar, Senior VP & CTO, Symbolic IO in a
LinkedIn (February 2, 2017).|
prototype RapidIO DRAM controller|
|Editor:- November 17, 2016 - Western Digital
today said in a
it will show a prototype memory controller aimed at big data which uses
RapidIO as the interconnect at SC16.|
new approach to incorporating SCM into the memory hierarchy can potentially
provide the flexibility to deploy ultra-low latency, coherent SCM across
high-performance computing and hyperscale environments, without disrupting
compatibility with the existing infrastructure, scaling up to petabytes of data
generated by applications and analytics in the age of "Big Data". ...read
Editor's comments:- The first mention of RapidIO
here on StorageSearch.com was in
March 2005 .
More recently in May
2014 - I drew attention to a blog -
You Really Know RapidIO? - by Eric Esteve , founder of
IPnest who said - "Maybe
it's time for the server/storage industry to give a second chance for the
|Rambus and Xilinx partner
on FPGA in DRAM array technology|
|Editor:- October 4, 2016 - Rambus today
a license agreement with Xilinx
that covers Rambus patented memory controller, SerDes and security technologies.
Rambus is also exploring the use of Xilinx FPGAs in its
Data Acceleration research program. The SDA - powered by an FPGA paired
with 24 DIMMS - offers high DRAM memory densities and has potential uses as a
CPU offload agent (in-situ
|Xitore's NVDIMM-X -
comparison white paper|
|Editor:- September 27, 2016 - If you're
interested in a single document which summarizes most of the DIMM wars products
in the market today - take a look at this -
NVDIMM-X (pdf) - an architectural white paper by Xitore which is
creating this "-X" technology.|
This paper was brought to my
attention today by Xitore's CEO - Mike Amidi, CEO - who
said in his email - "This can be a good article to explain the main
difference between all non-traditional DRAM based solution sitting on DRAM
memory bus. Either persistence, non-persistence, or SSD-on-A-DIMM." ...read
the article (pdf)
Editor's comments:- This confirms
that Xitore has ambitions in the
SSD market .
My initial impression of how their "-X"
product looks - is that it's something similar to what you'd get if you
combined the separate DIMMs in
Diablo Memory1 solution
(the mandatory motherboard DRAM and the Diablo flash DIMM and multilayered
controller software) and repackaged all the pieces into a single DIMM with
some external components to provide
power hold up
for the DRAM.
Xitore does say - "If a future version of
NVDIMM-X were to also use a non-volatile memory technology for the cache, then
no temporary backup power will be needed."
As a business
proposition - placing everything into a single component has advantages -
because it provides a lower cost threshold of application viability.
the other hand - like every other product in the memory market it still has to
pass the sanity test of - is there a distinct application for which this would
be the lowest cost solution? And how well does it scale up?
retrospect the first generation memory channel SSDs from Diablo weren't
compelling solutions and (aside from legal problems) had a weak internal bridge
between the flash and the memory bus.
Nobody owns that market space
today and if it looked like an interesting idea 2-3 years ago - the market
is more receptive to looking at such products now.
The risks are
lower because we can all see a competitive ecosystem developing in which this
type of architecture will prevail in the market regardless of the fortunes of
any single vendor
|Court of Appeals affirms
jury decision on DIMM wars trade secrets|
|Editor:- July 14, 2016 - One of the factors which
stalled the anticipated business growth in the
SSD market in 2014 and the first half of 2015 was uncertainty about the
outcome of a long running patent and trade secret dispute between Diablo and Netlist related to
how they had solved the line loading and latency problems associated with
connecting significant amounts of control logic and memory in standard
interfaces which had been originally designed to support low numbers of
DRAM chips. |
external observers stopped fretting about that in
March 2015 when
a court verdict seemed to clarify the situation.
But Netlist appealed
that verdict and so there was still an element of uncertainty pending the
outcome of the appeal hearing scheduled this month.
release by McDermott Will & Emery LLP (the law firm acting for Diablo)
seems to close the book on this matter.
Diablo at the appellate briefing and oral argument on July 7, 2016. Two business
days after the argument, the Federal Circuit panel unanimously affirmed the
judgment in favor of Diablo."
|Diablo invites ISVs to play
at new experience center|
|Editor:- June 28, 2016 - Diablo Technologies
today announced the opening of its ISV Development and Customer Experience
Center at the company's Silicon Valley office in San Jose, California. This
will be the main hub for hands-on demonstrations and customer engagement
activities related to Diablo's Memory Channel Storage and Memory1 products.|
|Xitore unstealths into
Memory Channel SSD market|
|Editor:- February 1, 2016 - Another new name
coming into in the SSD
DIMM wars saga
is Xitore which
exited stealth mode today with an announcement about their
NVM-X technology - which
promises "sub-2 microsecond latency" and 25GB/s bandwidth.|
comments:- Xitore's web site currently has almost no information about its
product details beyond the headline claims.
|new ideas in the SSD
|Editor:- December 15, 2015 - Memory Channel SSDs
were one of the new big SSD ideas of
They're now part of the furniture. |
year-end review on the home page of StorageSearch.com I discuss...
...read the article
- new ideas to assimilate
- big ideas to unlearn and forget
- the trends which will dominate our strategic SSD thinking in 2016
|Intel will enter Memory
Channel SSD market|
|Editor:- August 24, 2015 - Back in
July Intel and Micron
a new bulk material based resistive memory nvRAM platform which they called
XPoint technology (later branded as Optane). At that time - the
technical information about the memory technology were vague and lacking in
More details emerged during the shows which immediately
followed (FMS and IDF) and here's a link with the
says cost per bit is likely to be somewhere between DRAM and nand flash.
is said to be 1,000x faster than nand but slower than DRAM.
density? A single chip can store 128Gb.
Sampling? Later this year with
production in 2016.
Some of the many form factors and attach points
which might benefit from this new technology are PCIe SSDs and Memory Channel
As with any new memory technology it will take time and
experience to prove whether Optane memory has enterprise grade reliability. For
this reason and due to the need to establish a new software ecosystem - early
uses of the memory will probably be in experimental cloud appliances and
consumer gaming devices.
|Viking aims to design ReRAM
|Editor:- August 10, 2015 - Viking Technology and
a collaboration agreement to develop the next generation of Non-Volatile Dual
in-line Memory Module (NVDIMM) products leveraging ReRAM Storage Class Memory
from Sony Corporation. |
Diablo resumes shipments of MCS as injunction dissolved|
|Editor:- April 27, 2015 - Diablo has resumed
business as usual in the shipment and development of its
channel storage technology following months of impediments related to
legal wrangles. Among other things the company today
that the US District Court for the Northern District of California has
ruled to completely dissolve a preliminary injunction enacted in
Diablo's roadmap back on track following jury verdict|
|Editor:- March 25, 2015 - Diablo today
it has won a "decisive victory" in its critical court battle with
jury unanimously concluded that there was no breach of contract and that there
was no misuse of trade secrets. Further, the jury confirmed Diablos sole
ownership and inventorship of the 917 patent.
We are extremely
pleased with the jurys verdict today, said Riccardo
Badalone, CEO and Co-Founder of Diablo Technologies. We look forward to
getting back to serving our customers and delivering on our exciting Memory
Channel Storage roadmap.
Editor's comments:- Diablo replaced
Fusion-io as the #1
most searched SSD company by the readers StorageSearch.com in
Q4 2014 - because
you all know a disruptive SSD technology platform when you see it.
long running legal tangles got to the point where Diablo was prevented by an
injunction from making more products - pending a trial outcome.
verdict means the enterprise application acceleration industry can resume
its onwards progress by being able to count on the availability of a
flash latency asset
and software platform for deployments inside the server box.
Diablo updates position re UlltraDIMM sanctions|
|Editor:- February 24, 2015 - If - like me -
you've been following with interest the development of true SSD acceleration
technologies packaged in RAM DIMMs (aka
SSDs and similar names) then you may have been wondering - what's the
current state of the play in the
SanDisk patent and
implied rights to IP legal wrangle?|
The last furious clash of
legally related press releases - from both sides - in mid January - ended with a
lot of smoke in the air - and dire expectations regarding body count. In
particular the impression was that - until the next court session on these
matters - further shipments of SanDisk's ULLtraDIMM SSDs would be suspended.
is an update sent to me yesterday from a spokesperson communicating the Diablo
side of things. So "we" and "our" in the text below means
from the perspective of "Diablo".
Injunction - Diablo says
- SanDisk has been granted a stay on their preliminary injunction, meaning
that they can ship their inventory to Lenovo, Supermicro, and Huawei.
- The preliminary injunction on Diablo is still in effect while we await the
standard appeal process.
At the center of the dispute is the idea that our technologies
compete. Our technologies do not compete. There is a long list of reasons why
they dont. Here are some of them:
- It continues to be our belief that the standard appeal process will find in
- The Netlist NVvault is memory. The OS and applications see it and treat it
as DRAM, which is why no OS drivers are necessary. Ours is storage. Ours is
seen by the OS, hypervisors and applications as a block storage device and this
is why MCS does require OS drivers.
- They are used differently. For example, you typically would not put a
whole database on a DRAM NV-DIMM but you would on an MCS-based device.
- A DRAM DIMM can be used in place of DRAM, an MCS-based device cannot. An
MCS-based device, because it is storage, requires separate DRAM in the system
for execution. DRAM based devices are required to make the server run and are
complimentary to MCS-based devices.
- A DRAM NV-DIMM cannot be removed from the system and replaced with an
MCS-based device and be expected to perform the same function.
- The NVvault product is an 8GB device because it is a memory device. Since
ours does not use DRAM and instead interfaces directly to flash, it is capable
of being hundreds of gigabytes in capacity.
- JEDEC has defined a DRAM-based
NV-DIMM (NVDIMM-N) as a completely different category from an MCS-based device
(NVDIMM-F) because they operate differently and service different applications
in very different ways. There are several other companies building NVDIMM-N
devices including Netlist,
Diablo is the only company we know of that is building an NVDIMM-F device.
Editor's comments:- I think it's important for the SSD
industry to know whether it can count on seeing a competitive market for memory
channel SSDs being developed. For that to happen it is essential for Diablo to
establish in the courts or by agreement as soon as possible that the roadmap
for its kind of technology has a future.
- Simply because they both fit into the same slot and use a similar interface
does not mean that they compete. Most PCIe cards serve completely different
functions and do not compete, even though they use the same physical interface
(examples are graphics, audio, networking, and storage cards).
If this doesn't happen quickly
- and if the whole issue is left unresolved for another year - then the window
of opportunity for this class of enterprise SSD may close. Because - as far as
I know - Netlist doesn't have a Diablo like product in a similar state of market
So if Netlist were to succeed in preventing Diablo's
product roadmap - there isn't a similar product which architects could fall back
to. And even if Netlist chose to pursue that kind of product opportunity - which
it can't do on its own - the SSD market isn't going to wait idly by for
another 2 years waiting for that to happen.
Other ways of adding
applications intelligence into
PCIe SSDs - and other
alternatives to RAM cached to flash are already in development. And the software
market has to judge - which new markets are most likely to return value on
their developer investment.
remind you - the bullet points above - came from Diablo and whether you agree or
disagree with them or not - the reason so many lawyers are involved in this
right now is more to do with the fact that 2 companies (Netlist and Diablo)
have a different recollection of what they once agreed in a past collaborative
project and they disagree on what rights that past agreement confers on what
they're doing now.
If I get more updates I'll let you know.
key things for now are:-
- if you've got a design which uses 1st generation UlltraDIMM style memory
channel SSDs - then you can still get products to fill those slots.
- But - if you've been planning around the preannounced 2nd generation
products - your projects are probably on hold.
Diablo appeals shipment injunction - says court was misled|
|Editor:- January 14, 2015 - Diablo Technologies
that it has appealed the court ruling (reported earlier this week -
and initiated by Netlist)
which had granted a preliminary injunction to halt Diablo's shipments of Memory
Channel Storage based chipsets. |
Diablo's appeal explains that the
ruling is based on an erroneous interpretation of the contract and a failure to
recognize the technology differences among the products involved.
importantly, however, the court did not find that Diablo MCS uses
Netlist trade secrets.
Diablo says - to support the judgment, the
order effectively rewrites the language in the contract signed by the parties in
2008: the additional words included in the order changed the terms of the
contract significantly and imposed a new obligation that was not agreed between
the parties. In other words, there was no violation by Diablo of the original
contract. The court was misled about important technology distinctions:
court relied on Netlist's representation that their HyperCloud and Diablo MCS "are
used to perform the same function" which is not the case because the
HyperCloud is DRAM (memory) and Diablo MCS is a block storage device (disk).
The court also relied on Netlist's representation that the products
are competitive because they both "attach to the same memory channel."
Diablo also says "Netlist equates the 2 devices simply because
they use the same location and i/o channel; extending that logic would equate
all devices that reside in PCIe slots, which would be a similarly erroneous
"It is important that the facts of the case are well understood;
it should also be stressed that the court did not determine that Diablo uses
Netlist trade secrets," stated Riccardo
Badalone, CEO and co-founder of Diablo Technologies. "We offer an
innovative storage device that gives customers great performance advantages, but
with this injunction, the court is putting our company and our customers at
risk. With this appeal, we expect to reverse this decision and get back to
Editor's comments:- are Netlist's products functionally
different to those using Diablo's architecture?
Yes! The differences
are so great that the 2 products are listed in different directories here on
Channel Storage SSDs - which encompasses low latency, fast flash SSDs which
plug into DIMM sockets and transfer data via interfaces which were originally
designed for DRAM.
The fact that standards organizations and some vendors have
historically used the term "flash DIMM" in the context of both types
of products - has contributed to industry confusion.
- hybrid DIMMs, NV
DIMMs, flash backed DRAM DIMMs - which includes DRAM modules which
automatically save their contents when electrical power drops to an integrated
non volatile memory from which the data is reloaded after normal power is
But the term -
flash DIMM - is a description of the physical form factor - and tells you
nothing about the operation and functionality of the device from a data
architecture point of view.
The differences are vast and immediately
obvious to anyone who's technical. But I was wondering how would I explain the
gulf of difference to someone who doesn't know anything about computer design.
analogy goes like this... Suppose someone offered you the choice of 2 types of
backpack when you started a balloon ride.
One is a parachute, the
other is a jetpack.
They both perform different functions - although -
until you activate their functions they both might look the same.
I know you need a higher skillset to operate the jetpack.
also need a much more developed
SSD software support
ecosystem to deploy memory channel SSDs too.
California Court halts sales of ULLtraDIMM SSDs|
|Editor:- January 13, 2015 - Netlist today
that the US District Court for the Northern District of California has granted
Netlist's motion for a preliminary injunction against Diablo Technologies for
controller chips used by SanDisk
in its high-speed ULLtraDIMM SSD product line. |
Under the court's
order, Diablo and SanDisk are prohibited from manufacturing and selling the
controller chipset used by SanDisk in the ULLtraDIMM and as a result, from
further sale or distribution of the ULLtraDIMM itself.
Rogers - also rejected SanDisk's motion for reconsideration, asking
that it be allowed to sell existing inventory of the enjoined products.
The court advanced the trial date to March 9, 2015, for Netlist's claims upon
which the motion was decided, including claims against Diablo for trade secret
misappropriation, breach of contract, and other causes of action related to the
components supplied by Diablo for the ULLtraDIMM.
The court's order
specifically identifies the ULLtraDIMM as well as the eXFlash modules from
IBM, although the
injunction affects all modules containing Diablo components.
comments:- the possibility of such a injunction has been discussed in
before. At the heart of the dispute are whether an earlier design collaboration
between Netlist and Diablo included the rights for both companies to use a
critical interface design in the DIMM bus facing part of Diablo's memory channel
storage design. A patent ruling on December 29, 2014 (see below) upheld
Netlist's patents related to this.
The court case related to the
injunction stems from Netlist's argument that if Diablo did not have the right
to use the interface technology - then it follows that any design - such as the
ULLtraDIMM SSD - which relies on such internal technology - should only be
disallowed - pending any future agreement about licensing such a technology.
(Which it may not choose to do.)
Netlist wants to create a low latency
memory channel SSD product line of its own.
And even though Netlist
is more than a year behind the productization of this integrated technology
(compared to the first generation ULLtraDIMM designed Diablo and
SMART Storage -
which was acquired by SanDisk) the case from Netlist is that the market must
wait for its own design or any designs for which it grants licenses.
the Diablo side - its argument has been that they thought they had an
agreement which allowed them to use the DIMM interface technology (in whose
implementation they participated) in products which were different to Netlist's
flash backed DIMMs.
stake is a future market for server based accelerators which could be worth a
low double digit percentage of the entire
enterprise PCIe SSD market.
inconceivable that a small company like Netlist or Diablo would be able to
satisfy such demands on their own - especially given the fact that neither has
any core IP related to
But future licensing partners (or wouldbe
acquirers) need to be satisfied that the core technology they're using - is
patent troll proof.
|Netlist revalidates core
patent related to ULLtraDIMM's core technology|
|Editor:- December 29, 2014 - Netlist today -announced
that the US Patent and Trademark Office has denied petitions requesting
Inter Partes Review of Netlist patents asserted against the
The petitions were filed by
SanDisk who partnered
with Diablo Technologies
to produce the ULLtraDIMM. Both are co-defendants in Netlist's patent
infringement action pending in the US District Court of the Northern District
of California. Of the 5 petitions filed by SanDisk, 3 were denied in their
entirety and a 4th was denied as to some of the challenged claims.
The Patent Trial and Appeal Board denied institution as to all claims of U.S.
Pat. No. 8,516,185 which relates to the fundamental distributed buffer
architecture created by Netlist and is integral to the architecture of the
ULLtraDIMM. Among other functions, the distributed buffer architecture allows a
proprietary DIMM containing various types of memory to appear to the system as
standard DRAM memory.
Netlist's CEO said, "We are very pleased with this outcome and the overall
progress in our multi-year legal proceedings against ULLtraDIMM and SanDisk. The
technologies covered by our patents are critical for the enterprise computing
and storage space. We believe this ruling by the PTAB is a clear validation of
our IP in this area and a testament to the years of seminal development work and
Editor's comments:- One of the things which I had expected to
see in 2014
- but didn't - was the appearance of more vendors competing in the low latency
Although one reason for this could be that developing
any new server based ecosystem requires more software investment than
introducing new products in already established markets (thereby making this a
riskier investment choice compared to new form factors of
PCIe SSD - for
example) another reason may be that the likely suspects to enter the future
memory channel market are waiting to get clarification from the courts as to
which companies they should be seeking to get patent licenses from (if any).
|flash backed DIMMs - new
directory on StorageSearch.com|
|Editor:- October 21, 2014 - Although StorageSearch.com has been writing
about flash backed
DRAM DIMMs since the first products appeared in the market - I didn't think
that subject was important enough before to rate a specific article or market
That's unlike this page you're seeing now -
SSDs - which has become 1 of the top 10
viewed by readers after having had its own directory page here since
Despite my lack of initial enthusiasm for bybrid DIMMs (or
for that matter too) I realize that sometimes a
market is defined as
much by what it isn't as by what it is. Which is why I have relented.
so - to help clarify the differences between these 2 types of similar looking
storage devices (one of which I think is much more significant than the other
- but both of which are important for their respective customers) I have today
created a directory page for
hybrid DIMMs etc -
which will act as the future launch pad for related articles.
|Netlist asks court to shut
down SanDisk's ULLtraDIMM production|
|Editor:- October 10, 2014 - Netlist recently
it has filed a motion for
injunction which seems to be intended to restrain Diablo and its
flash-side SSD integration partner SanDisk from any
further manufacture or sale of the ULLtraDIMM (memory channel SSDs).|
Court has set a tentative date of December 2, 2014, for a hearing on Netlist's
motion. If granted this would immediately shut down any further manufacture and
sale of the ULLtraDIMM. Netlist has further asked the Court to order the recall
of any ULLtraDIMM products previously sold.
This series of legal disputes has been going on since last year. If you haven't
read those stories - the essential story is something like this.
some time in the past Netlist and Diablo had a technology supplier agreement -
as a result of which - Diablo had access to Netlist's IP related to
minimizing the capacitive load of complex circuits susch as controllers when
they sit on a DRAM style of bus. Netlist doesn't have any products which are
similar to SanDisk's ULLtraDIMM - but Netlist claims that the ULLtraDIMM
design has used its patented interface technology without a license. Countering
that - Diablo says it did get the rights to use some of the DRAM load
interface technology - and that in any case - this aspect of the design is
not the essential defining characteristic of their flash SSD architecture.
Diablo's CEO - Riccardo
Badalone retorted to Netlist's latest legal move saying this - "After
a year of court proceedings and months of discovery, Netlist still cannot
decipher how Memory Channel Storage works, much less substantiate that it
infringes on any of their IP."
|Diablo countersues Netlist
|Editor:- September 25, 2014 - Diablo announced today
that is has filed a lawsuit against Netlist for unfair
business practices that violate Diablos IP rights.|
This appears to be
a countermeasure to 2 earlier lawsuits initiated by Netlist against Diablo -
which were widely reported by the SSD related press in
- Diablo reiterates that its
Channel Storage (DDR3/4 form factor
and interface compatible flash SSD) is a new and innovative architecture
that neither infringes upon, nor misappropriates any Netlist IP rights. And
Diablo argues that its MCS-based products and the
HyperCloud DIMM (high
density DRAM) - which were the cited products in Netlist's earlier legal
moves - are designed to serve different purposes and are not
Diablo says the contract between the 2 companies (which have been
mentioned in the press) clearly assigns legal ownership of the implementation
IP in the HyperCloud chipset to Diablo. As a result, Diablo is seeking damages
for breach of contract for Netlists attempt to usurp the companys IP rights.
We have been very patient throughout this entire process and it is
now time for us to share our side of the story" said Riccardo
Badalone, CEO and Co-founder of Diablo Technologies. "We will
demonstrate definitively that products based on the Memory Channel Storage
architecture do not use any Netlist IP.
| another auspicious
design win for ULLtraDIMM|
|Editor:- September 16, 2014 - SanDisk today
that its ULLtraDIMM (memory channel SSD)
has been selected by Huawei
for use in its RH8100 V3 servers. |
Huawei is ranked the top server
supplier for cloud and
mobility in China - by Sino-Bridge
Editor's comments:- Since the
announcement that IBM was
using ULLtraDIMM SSDs in some high end servers - there haven't been many
conspicuously auspicious design win announcements like today's Huawei story.
One reason is that IBM had a head start on the market - having worked
with Diablo for years to
refine the MCS architecture and software APIs.
Another reason is that
the 1st generation ULLtraDIMMs apparently guzzled more electrical power than
modern RAM DIMMs even
though they were still within the permitted power envelope according to industry
standards. This means that in order to support arrays of them in a server design
(and indeed you do need arrays to get
performance beyond the PCIe SSD level) requires a redesign of the copper
power tracking on the motherboard. You can't just plug large numbers of
ULLtraDIMMs into any old server without analyzing the thermal consequences.
| Diablo unveils DDR-4
flash DIMM SSDs|
|Editor:- August 7, 2014 - Diablo yesterday
announced details of a new 2nd generation
SSD - low latency flash SSD accelerators in DDR-4 sockets - which will
sample to oems in the first half of 2015.|
Along with the new hardware
technology there will be an improved software platform - with features like
NanoCommit - which Diable says will enable hundreds of millions of
transactions per second, with nanosecond latency.
Storage DDR4 solutions represent the next evolution of Server Acceleration
technology," said Riccardo
Badalone, CEO and Co-founder of Diablo Technologies. "In addition
to supporting a faster memory interface, the Carbon2 platform delivers
unprecedented levels of hardware acceleration for new software innovations like
NanoCommit. Converged Memory, where the best of Flash and DRAM are combined,
will rely on this type of technology to give applications the ability to
transparently persist updates to main memory."
comments:- After FMS - Diablo sent me
more info (pdf)
about their FMS
presentation (pdf) from which I have extracted these key features.
- Diablo's converged memory architecture (flash tiered with DRAM) is planned
to support 700 million random cachelines / sec.
- Latency of each cacheline is about 48 nanoseconds.
- Diablo's NanoCommit supports byte addressable small writes to flash with
high transaction rates and the ability to mirror the DRAM contents to
- The combination of technologies would enable something like a 1U server
with 25TB of converged memory.
|Diablo secures patent
related to MCS technologies|
|Editor:- July 31, 2014 - Diablo today
that the US Patent and Trademark Office (USPTO) awarded U.S. patent No.
8,713,379, entitled a System and method of interfacing co-processors and
input/output devices via a main memory system, to the company. The 379 patent
- A method for connecting non-volatile memory directly to the memory
controllers of a processor.
- A learning machine to handle data interleaving/de-interleaving and data
scrambling/de-scrambling algorithms for DDR3/4-based memory
- A method to remap the non-linear DIMM address space back to linear address
space used by the driver
|PCIe SSDs versus memory
channel SSDs are these really different markets?|
|Editor:- June 10, 2014 - Throughout
we've seen many claims that a new upstart storage technology will replace
It doesn't always work out to be as simple as that.
the enterprise SSD market of today -
PCIe SSDs are the safe
standard legacy server acceleration SSD technology and
SSDs with akas like UlltraDIMM, TeraDIMM, MCS etc are the upstart
Each of these 2 SSD types can do some very
similar things - such as make server apps run faster - but they aren't the
It's reasonable to ask - are there any hard technical rules
where you can say for your new projects either:-
- I don't need to know about the new thing, or
Without getting mired in contentious
product by product benchmark comparison claims and counter claims?
- maybe I should look at it?
my blog - memory
channel SSDs versus PCIe SSDs (slight return) - which was on the home page
of StorageSearch.com throughout May 2014 and which now has a new permalink
location. I've updated it with some new notes and contextual comments. ...read the article
|EMC acquires memory channel
SSD in a box company DSSD|
|Editor:- May 5, 2014 - EMC has acquired a stealth
mode rackmount SSD company - DSSD
- it was announced
today. Products based on the new DSSD architecture are expected to be
available in 2015.|
Editor's comments:- for a competitive
analysis and technology perspective see
rackmount SSD news
|1st generation ULLtraDIMM
SSDs are limited by having "SATA inside" - so imagine what a next
generation flash DIMM might perform like|
|Editor:- May 8, 2014 - Today I learned some new
details about the architecture in SanDisk's
SSDs by watching a video -
Architecture and Design Technology - which was recorded at the recent
Storage Field Day event
(April 25, 2014).|
Inside each flash DIMM are 2x SATA CloudSpeed SSDs
which connect to the DDR3 interface via a Diablo designed bridge
chip which has something on the order of "megabytes" of buffer memory
That gives a much better basis for
performance limits of these devices.
And it confirms why you only
start to see noteworthy performance from them after installing 3, 4 or
|However - despite the "SATA
inside" limited nature of this 1st generation memory channel storage
design - the performance - aggregated at a block level - results in a usable
write latency envelope which IBM
has verified to be in a
latency category than most leading
- if you're thinking long term - and roadmaps - it's reasonable to assume that
if a 2nd or 3rd generation of MCS flash DIMM SSD could abandon the "SATA
inside" and use instead a
architecture flash controller with a native memory array optimized type of
bridge interface - then the gaps between what MCS can offer - compared to PCIe
latency - could widen.
By the way in this 52 minute video I also
learned 2 interesting details about
SanDisk's CloudSpeed SSDs
which I didn't know before:-
SSD news and articles
- SanDisk's Guardian technology doesn't perform scheduled data integrity
checks to detect silent errors for data at rest.
The stated reason
being that this would eat into
And the implied assumption being that SanDisk thinks this type of activity is
best determined by the system designer. So - if you design these SATA SSDs
into a storage array - then you'll have to take responsibility for that aspect
of long term data health in your own software.
|Fusion-io says - we
don't regard SanDisk's NVM DIMMs as a threat to our PCIe SSD business|
|Editor:- April 24, 2014 - Fusion-io
revenue up 14% yoy to $100 million but losses have grown too - to $30
million for the recent quarter. |
Editor's comments:- In the
investor conference call an analyst asked - in effect - whether
should be viewed as a threat to Fusion-io's PCIe SSD business?
President and COO at Fusion-io said - he thought NVM DIMMs were a validation
of what FIO had already done - but he regarded NVM DIMMs (in their current
form) as a niche market.
He went on to say - "We don't believe
that DDR-3 and the memory interface is the bottleneck. It's about managing the
nand and getting the best performance out of the nand."
Editor comments:- Whereas I agree with Fusion-io that the
hardware latency difference in itself between MCS and PCIe is less
significant than the combined effects of flash controller architecture and FTL
software interplaying with data in an SSD-aware software stack - nevertheless I
don't agree with Fusion's downplaying and dismissal of the market and
technology significance of MCS.
I think that in maybe 2 to 6
quarters - this comment could return to haunt them.
Because it's precisely in the kind of high end use cases which
Fusion-io currently dominates - that the MCS technology will be most economic
It will take at least that long - because Fusion-io already has
very broad, deep and proven software surrounding their flash and an
entrenched 7 year established hold in the server market.
In contrast -
SanDisk's best selling enterprise SSDs are
SAS SSDs - and even
before acquiring the MCS product line last year - SanDisk's penetration of
the server side caching market in the words of its own CEO - had been
| new whitepaper from
Diablo looks at the "drawbacks of traditional PCIe SSDs"|
|Editor:- April 22, 2014 - Diablo Technologies
today published a white paper
Storage Performance: it's all about the Architecture (and not just the
|This paper presents Diablo's perspective - as
the creator of
Storage and looks at the measured performance weaknesses of some specific
(but unnamed) PCIe SSDs
with respect to their measured performance
under various loads and then attempts to generalize those weaknesses as part
of an argument that MCS is a better solution than PCIe SSDs - because MCS is
more scalabe and doesn't suffer from the same kind of architectural
comments:- This paper from Diablo reminds me of the type of comparisons
between different types of PCIe SSDs (and their sensitivities to different
workloads) which was one of the pivotal marketing points of difference which
Virident used to
hang their reputation on in the years leading up to its acquisition by
HGST. (Although as I
often reminded readers at the time - Virident wasn't the only company with that
kind of array scalability or no-compromise performance.)
Going back to
Diablo's white paper - for me - like many vendors written papers - it's good and
bad in different parts.
The best bit is the middle - in which you get
a reminder - from measured results that fast SSDs with similar capacities
The worst part in my view is the attempt to link
these comparison results to a general conclusion about the merits of MCS
versus PCIe SSDs.
Because I think the architecture of the
controllers on the
flash side of the SSDs plays such an important part as too does the
software. And that's
still just a small part of the picture.
My considered view is that
Diablo's extrapolation towards a general market conclusion from one
selected comparison example is not merited by the evidence presented in this
And while I am convinced of the benefits of Diablo's MCS
architecture - and what it can do differently and better compared to PCIe SSDs
- I think this particular paper isn't the best case they could argue.
|what's in a number? - for
MCS add 4 |
|Editor:- March 4, 2014 - The presence (or
absence) of Memory Channel SSDs in a server is one of the factors which go
towards calculating the
SSDserver rank -
a new latency based configuration metric - which is proposed as a standard
|Diablo comments on legal
barbs cast by Netlist|
|Editor:- February 14, 2014 - Diablo's CEO and
Badalone was reported to have said in an
blog by Willem ter Harmsel today - re the lawsuit launched against
them by Netlist...|
(Netlist) basically have yet to prove that:- they understand the product, and
also that they have found anything Diablo is actually infringing on. We are
just going to let this run through the (legal) process and we are confident
about the outcome." ...read
|Micron doesn't currently
have a low latency, high capacity, flash DIMM SSD technology platform in its
product line yet. But I think it's inevitable that Micron will have to publicly
address this product gap to maintain confidence in its server customer base.
Clearing the patent and IP decks in readiness for this may have been one of
the factors in the December 2013 patent deal with Rambus.|
|the Top SSD Companies in
Q4 2013 - February 4, 2014|
|"We're seeing 5-10
microseconds write latency for eXFlash DIMMs in preliminary testing vs. 15-19
microseconds latency for PCIe-based flash storage from Fusion IO, Micron, and
Virident, and 65 microseconds latency for Intel S3500 and S3700 SSDs"|
January 16, 2014 -
Architecture press release|
|"One of the most
significant new form factors introduced in the enterprise SSD market in 2013
was memory channel storage."|
changed in SSD year 2013? - December 9, 2013|
|McObject shows in-memory
database resilience in NVDIMM|
|Editor:- October 9, 2013 - what happens if you
pull out the power plug during intensive in-memory database transactions?
For those who don't want to rely on batteries - but who also need ultimate
speed - this is more than just an academic question.|
Recently on these
pages I've been talking a lot about a new type of
SSDs which are hoping to break into the application space owned by
PCIe SSDs. But another
solution in this area has always been DRAM with power fail features which save
data to flash in the event of
loss. (The only disadvantages being that the memory density and cost are
constrained by the nature of DRAM.)
products include in-memory database software) yesterday
published the results of
benchmarks using AGIGA
Tech's NVDIMM in which
they did some unthinkable things which you would never wish to try out for
yourself - like rebooting the server while it was running... The result?
Everything was OK.
"The idea that there must be a tradeoff
between performance and persistence/durability has become so ingrained in the
database field that it is rarely questioned. This test shows that mission
critical applications needn't accept latency as the price for recoverability.
Developers working in a variety of application categories will view this as a
breakthrough" said Steve Graves,
Here's a quote from the whitepaper -
Persistence, Without The Performance Penalty (pdf) - "In these tests
eXtremeDB's inserts and updates with AGIGA's NVDIMM for main memory storage
were 2x as fast as using the same IMDS with transaction logging, and
approximately 5x faster for database updates (and this with the
transaction log stored on RAM-disk, a solution that is (even) faster than
storing the log on an SSD).
The possibility of gaining so much speed
while giving up nothing in terms of data durability or recoverability makes the
IMDS with NVDIMM combination impossible to ignore in many application
categories, including capital markets, telecom/networking, aerospace and
Editor's comments:- last year
McObject published a paper showing the benefits of using PCIe SSDs for the
transaction log too. They seem to have all angles covered for mission critical
ultrafast databases that can be squeezed into memory.
|It looks like you're seriously interested
in SSDs. So please bookmark this page and come back again soon. |
you've got the time - you might want to take a look at
the top 100 most
popular SSD articles seen by our readers in recent months.
|About the publisher - 21
years guiding the enterprise market||....
when you're in a desperate hurry and can't afford to keep hanging around
|Fred Moore, who was
Manager, WW Systems Engineering for StorageTek at the time told me (for this
article) - "The original STK 4305 in 1978 was built using
CCD. We built
less than 25 CCD units before shifting to DRAM. I believe CCD supplier issues
were a key factor for the change. As soon as that change was made, efforts to
add battery backup options gained momentum." |
Rise of the SSD Market|
"I (still) don't think we've reached stability in reference enterprise
SSD designs and use cases. Zsolt Kerekes,
editor - StorageSearch.com
All the systems in the market today are
implementations of transient architectures..."
comment above (without the word "still") in an outlook blog -
how fast can your SSD
Big versus Small in
SSD controller architecture
utilization and the
enterprise SSD event horizon
care management & DSP IP in SSDs
will SSDs end
bottlenecks? - and cure all my problems?
memory channel SSDs
versus PCIe SSDs - are these really different markets?
|One of the challenges for
the enterprise SSD market when designing new products is to understand complex
customer needs and decision criteria - which go beyond the traditional bullet
segmentation models are needed because the enterprise SSD market is moving
into uncharted territories and use cases where a considerable proportion of the
customer needs which affect buying behavior are still formally unrecognized as
hidden segments in the enterprise|
|"See how optimizing
processors for SSD can gain a 2x to 250x speed-up on popular functions as well
as reduce the energy consumed by a similar amount! "|
|SSD Bookmarks - from
|IBM Redbook places memory
channel SSDs in onboard server apps contexts and hierarchy|
|Editor:- February 18, 2014 - IBM today published a new
free 28 page ebook (aka Redbook) -
IBM eXFlash Memory-Channel Storage in Enterprise Solutions (abstract) / (pdf) -
which describes how
SSDs fit into the concept of servers relative to the other types of SSDs
already available. |
Editor's comments:- I've been writing
about this technology since the time it was being developed and have been well
briefed by the original developers - so this paper didn't have any great
surprises to me - but I think this document presents a balanced introduction
to this technology and a contextualized analysis of how it compares to the
other well established SSD acceleration options which are available for use
key takeway - in my view is table 2 - in which you can see a hierarchy of write
latencies which are approximately 5x longer in each case as you
progress up the flash SSD options from
SSDs, PCIe SSDs
and SAS SSDs.
bearing in mind that SSD data write latency is not the same as apps
performance latency (because the integration of R/W data flow patterns
with the software
plays a significant part too) and also remembering that
some products in
the market will blur the ratio of the latency boundaries for these 3
different SSD types - you can, nevertheless see why memory channel has a
distinct slot within the onboard SSD acceleration options which you need to
|Netlist says ULLtraDIMM
SSDs infringe patents|
|Editor:- January 29, 2014 - Netlist today
it had filed motions to add two additional patents to the lawsuits against the
ULLtraDIMM memory module from
The 2 newly added patents (U.S. Patent Nos. 7,881,150
and 8,081,536) are generally related to load reduction, a critical feature in
low latency memory modules. Netlist has now asserted a total of 7 patents
against the ULLtraDIMM, in addition to trade secret theft, trademark
infringement and other claims.
"We have spent years developing
our industry-leading technology, and are encouraged by the progress we've made
defending our intellectual property in these lawsuits," said Netlist
President and CEO, C.K. Hong. "We
will continue to vigorously defend our IP, uncovering any and all theft of our
technology and infringement of our patents, and will not allow others to profit
from this unlawful activity."
in its 8-K last December, Netlist says it received a whistleblower letter
describing in detail how Diablo "stole Netlist's detailed architecture and
design" of its flagship product, HyperCloud, to create the ULLtraDIMM.
comments:- SanDisk's ULLtraDIMM is a
new class of
product in which cheap high capacity
MLC flash emulates many
DRAMs in a module which
contains little or no DRAM itself but which nevertheless plugs into a DDR3
In contrast - Netlist's "similar sounding but very
different" product families include real DRAMs which accomplish power
fail data protection aka non volatility - by means of fast backup and
restore to onboard flash - all in the same DDR3 module.
doesn't appear to be saying that it could design a product like the ULLtraDIMM -
because it doesn't to my knowledge have the
controller IP to do anything remotely similar.
Instead - what
Netlist appears to be saying is that techniques in DDR3 design - which enable
lots of circuitry to be placed behind a RAM interface - without placing too
much load on it and slowing it down - which Netlist has itself patented - are
suspected to have been used within the design of the ULLtraDIMM.
|SMART samples first
|Editor:- August 8, 2013 - SMART Storage Systems
it has begun sampling the first memory channel SSDs compatible with the
interface and reference architecture created by Diablo Technologies.|
|SMART's first generation enterprise
(ULL = ultra-low latency) can be deployed via any existing DIMM slot and
provides 200GB or 400GB of enterprise class flash SSD memory with upto 1GB/s and
760MB/s of sustained read/write performance, with 5 microseconds write latency.
Throughput, IOPS and memory capacity all scale with the number of ULLtraDIMM
deployed in each server. |
Editor's comments:- With the current
design -only one DIMM slot in each server has to be reserved for conventional
DRAM. Apart from that constraint any DIMM slot can be used for either flash or
DRAM as deemed necessary for the application.
For more about the
potential of this technology, the thinking behind it and the competitive
landscape relative to PCIe
SSDs etc see my earlier articles on the
|Diablo discusses details of
Memory Channel Storage|
|Editor:- July 30, 2013 - Diablo Technologies
today did the
public launch of
its its new technology -
Memory Channel Storage
- which repurposes the interface and form factor of server DRAM into a new
architecture for ultrafast flash SSDs which the company positions as a
competitive alternative to very fast
PCIe SSDs. |
comments:- I spoke recently to Diablo's new VP of Marketing - Kevin Wagner about the
company's new MCS.
I had already gleaned a heads up on the boundary
capabilities and market potential of the new technology in
interview with Diablo's flash partner SMART. So it was natural to ask how
Diablo viewed the impending acquisition of
surprisingly Kevin indicated that whereas Diablo already thought it had made the
best choice before - the upside potential of having SanDisk as a partner for
this memory technology made it 10x better.
Here's what I
- Diablo's TeraDIMMs are designed to fit standard sockets designed for DDR-3.
They are electrically, form factor and power compatible. But instead of RAM - a
typical TeraDIMM using today's technology might have 400GB of flash.
- Diablo's controller architecture means that the host CPU can read and
write from memory in the same transparent way as it would talk to RAM.
- TeraDIMMs can be installed in every set of slots where you'd normally
insert RAM. The only limiting factor in the current architecture is that at
least one pair of slots has to be RAM. All the rest can be flash.
- Diablo claims that MCS has better write latency than most PCIe SSDs.
Specifically MCS has a write latency of 3 to 5 microseconds. This is
really a write to the controller.
TeraDIMMs have a
architecture - which means that apart from a small amount of register
memory in the MCS controller itself - no
RAM is used in the SSD.
The MCS design includes
hold up capacitors which guarantee that all data which has been written to
the TeraDIMM gets completely saved to flash.
- From the applications point of view MCS looks like a massive amount of
persistent RAM - but with terabytes rather than tens of gigabytes of memory
space and with a cost structure closer to the
market price of flash
the product is not yet available - Kevin Wagner told me that Diablo has been
getting a lot of interest from server companies. Diablo has been validating
software with their ASIC based implementation for several months following good
earlier characterizations of the design with FPGA. From the software point of
view Diablo's aim was to prioritize a usable design which would work for the
market as soon as possible. Obviously many possibilities for leveraging the
basic technology spring to mind. I asked for example about preferred models of
- Is it bootable? No. Not yet that wasn't regarded as a priority.
Kevin said that so many companies are interested in
what they're doing that they don't have enough resources to talk to them all
right now. It won't be long before the company publishes more details of its
reference architecture - and conversations have already begun with ISVs and
other companies which could be the seeds of a new ecosystem.
current design has been designed to work in virtualized and non-virtualized
environments and the company has got already got a good idea of how headline
big data apps would perform in benchmarks using their technology.
raised the question of inter-operability with
PCIe SSDs (in the same
MCS resident servers) and it looks like the guesses I had made about
segmentation and collaboration and competition with other SSD types which I've
about in my earlier article about memory channel SSDs - are still valid. So
I won't repeat those points here.
We still have to wait for firm
product pricing and configuration details. But if you had any doubts about where
the money will be inside servers - MCS provides another new way of packing
even more flash SSD capacity in.
|"SSDs, have an
identity crisis. They are clearly not drives in the conventional sense of slow,
spinning hard disk drives. Yet SSDs look like drives to system designers because
they retain legacy disk interfaces and rely on disk protection schemes designed
in the 1980s specifically for high-capacity, slow I/O mechanical disks."|
SSD: The Fusion Software Defined Flash Memory Approach - is a paper by Fusion-io. It was
probably written a long time ago - but I only saw it recently.|
innovation in the host-attached flash storage marketplace today comes from
products that not only access flash through a PCIe connection, but also bypass
storage protocols to drive new levels of performance and enable new
functionality not previously imagined. To achieve this, existing technologies
must be left behind..."|
and FlashMAX with Virident's vFAS (July 2012) - blog by Jeff Sosa Director of
|"The risk of data
corruption from power cycling isn't a random, unforseeable event. It's a direct
result of choices made (or not made) when that SSD was designed... "|
sudden power loss|