this is the home page of  StorageSearch.com
leading the way to the new storage frontier .....
the fastest SSDs - click to read article
the fastest SSDs ..
DIMM wars - expect more
can memory do more? ....
SSDs over 163  current & past oems profiled
SSD news ..
DRAM latency
DRAM's latency secret ....
hybrid DIMMs
hybrid DIMMs ..
..

Memory Channel Storage SSDs

This resource page includes news and articles about low latency, fast flash SSDs which plug into DIMM sockets and transfer data via interfaces which were originally designed for DRAM.

This scope includes but is not limited to:- TeraDIMMs, ULLtraDIMM SSDs, eXFlash DIMMs, memory channel storage, ultra low latency flash SSDs, DDR-3 SSDs, DDR-4 SSDs, flash DIMMs, NVDIMMs (some types), Storage Class Memory, NVDIMM-X, etc

The scope excludes:- flash backed DRAM DIMMs, hybrid DIMMs, NVDIMM-N.

latency loving reasons for fading out DRAM
where are we heading with memory intensive systems?
SSD lesson 2016? - No more deference to other markets.
..
"We are morphing from a storage hierarchy to a memory hierarchy. This is why I choose to work where I do. Memory rules."
Rob Peglar, Senior VP & CTO, Symbolic IO in a comment on LinkedIn (February 2, 2017).
..
3DXpoint revenue
"3D cross point we've said is a very de minimis amount of revenue in -- fiscal 2017. We will ship for revenue, but it's actually a fairly small amount and then we've set the expectation for somewhere around 5% of company revenues in 2018."

Ernie Maddock, CFO - Micron (January 10, 2017) in Micron Presents at Needham Growth Conference (transcript - by SeekingAlpha.com)
..
WDC shows prototype RapidIO DRAM controller
Editor:- November 17, 2016 - Western Digital today said in a blog it will show a prototype memory controller aimed at big data which uses RapidIO as the interconnect at SC16.

"This new approach to incorporating SCM into the memory hierarchy can potentially provide the flexibility to deploy ultra-low latency, coherent SCM across high-performance computing and hyperscale environments, without disrupting compatibility with the existing infrastructure, scaling up to petabytes of data generated by applications and analytics in the age of "Big Data". ...read the article

Editor's comments:- The first mention of RapidIO here on StorageSearch.com was in March 2005 . More recently in May 2014 - I drew attention to a blog - Do You Really Know RapidIO? - by Eric Esteve , founder of IPnest who said - "Maybe it's time for the server/storage industry to give a second chance for the RapidIO protocol."
..
Rambus and Xilinx partner on FPGA in DRAM array technology
Editor:- October 4, 2016 - Rambus today announced a license agreement with Xilinx that covers Rambus patented memory controller, SerDes and security technologies.

Rambus is also exploring the use of Xilinx FPGAs in its Smart Data Acceleration research program. The SDA - powered by an FPGA paired with 24 DIMMS - offers high DRAM memory densities and has potential uses as a CPU offload agent (in-situ memory computing).
..
Xitore's NVDIMM-X - comparison white paper
Editor:- September 27, 2016 - If you're interested in a single document which summarizes most of the DIMM wars products in the market today - take a look at this - Introducing NVDIMM-X (pdf) - an architectural white paper by Xitore which is creating this "-X" technology.

This paper was brought to my attention today by Xitore's CEO - Mike Amidi, CEO - who said in his email - "This can be a good article to explain the main difference between all non-traditional DRAM based solution sitting on DRAM memory bus. Either persistence, non-persistence, or SSD-on-A-DIMM." ...read the article (pdf)

Editor's comments:- This confirms that Xitore has ambitions in the memory channel SSD market .

My initial impression of how their "-X" product looks - is that it's something similar to what you'd get if you combined the separate DIMMs in Diablo Memory1 solution (the mandatory motherboard DRAM and the Diablo flash DIMM and multilayered controller software) and repackaged all the pieces into a single DIMM with some external components to provide power hold up for the DRAM.

Xitore does say - "If a future version of NVDIMM-X were to also use a non-volatile memory technology for the cache, then no temporary backup power will be needed."

As a business proposition - placing everything into a single component has advantages - because it provides a lower cost threshold of application viability.

On the other hand - like every other product in the memory market it still has to pass the sanity test of - is there a distinct application for which this would be the lowest cost solution? And how well does it scale up?

In retrospect the first generation memory channel SSDs from Diablo weren't compelling solutions and (aside from legal problems) had a weak internal bridge between the flash and the memory bus.

Nobody owns that market space today and if it looked like an interesting idea 2-3 years ago - the market is more receptive to looking at such products now.

The risks are lower because we can all see a competitive ecosystem developing in which this type of architecture will prevail in the market regardless of the fortunes of any single vendor
..
Court of Appeals affirms jury decision on DIMM wars trade secrets
Editor:- July 14, 2016 - One of the factors which stalled the anticipated business growth in the memory channel SSD market in 2014 and the first half of 2015 was uncertainty about the outcome of a long running patent and trade secret dispute between Diablo and Netlist related to how they had solved the line loading and latency problems associated with connecting significant amounts of control logic and memory in standard interfaces which had been originally designed to support low numbers of DRAM chips.

Most external observers stopped fretting about that in March 2015 when a court verdict seemed to clarify the situation.

But Netlist appealed that verdict and so there was still an element of uncertainty pending the outcome of the appeal hearing scheduled this month.

Now a press release by McDermott Will & Emery LLP (the law firm acting for Diablo) seems to close the book on this matter.

"McDermott represented Diablo at the appellate briefing and oral argument on July 7, 2016. Two business days after the argument, the Federal Circuit panel unanimously affirmed the judgment in favor of Diablo."
..
Diablo invites ISVs to play at new experience center
Editor:- June 28, 2016 - Diablo Technologies today announced the opening of its ISV Development and Customer Experience Center at the company's Silicon Valley office in San Jose, California. This will be the main hub for hands-on demonstrations and customer engagement activities related to Diablo's Memory Channel Storage and Memory1 products.
..
Xitore unstealths into Memory Channel SSD market
Editor:- February 1, 2016 - Another new name coming into in the SSD DIMM wars saga is Xitore which exited stealth mode today with an announcement about their NVM-X technology - which promises "sub-2 microsecond latency" and 25GB/s bandwidth.

Editor's comments:- Xitore's web site currently has almost no information about its product details beyond the headline claims.
..
new ideas in the SSD market?
Editor:- December 15, 2015 - Memory Channel SSDs were one of the new big SSD ideas of 2013. They're now part of the furniture.

In my 2015 SSD year-end review on the home page of StorageSearch.com I discuss...
  • new ideas to assimilate
  • big ideas to unlearn and forget
  • the trends which will dominate our strategic SSD thinking in 2016
...read the article
..
Intel will enter Memory Channel SSD market
Editor:- August 24, 2015 - Back in July Intel and Micron unveiled a new bulk material based resistive memory nvRAM platform which they called 3D XPoint™ technology (later branded as Optane). At that time - the technical information about the memory technology were vague and lacking in detail.

More details emerged during the shows which immediately followed (FMS and IDF) and here's a link with the webcast.

Intel says cost per bit is likely to be somewhere between DRAM and nand flash.

Latency is said to be 1,000x faster than nand but slower than DRAM.

Storage density? A single chip can store 128Gb.

Sampling? Later this year with production in 2016.

Some of the many form factors and attach points which might benefit from this new technology are PCIe SSDs and Memory Channel SSDs.

As with any new memory technology it will take time and experience to prove whether Optane memory has enterprise grade reliability. For this reason and due to the need to establish a new software ecosystem - early uses of the memory will probably be in experimental cloud appliances and consumer gaming devices.
..
Diablo aims to shrink enterprise DRAM market with flash
Editor:- August 12, 2015 - Surprise! - Diablo's long anticipated 2nd flash based DDR4 compatible product called called "Memory1" isn't an SSD.

What it is and what it isn't and the possible market impacts are discussed in a new blog on the home page of StorageSearch.com
..
Viking aims to design ReRAM NVDIMMs
Editor:- August 10, 2015 - Viking Technology and Sony today announced a collaboration agreement to develop the next generation of Non-Volatile Dual in-line Memory Module (NVDIMM) products leveraging ReRAM Storage Class Memory from Sony Corporation.
..
Diablo resumes shipments of MCS as injunction dissolved
Editor:- April 27, 2015 - Diablo has resumed business as usual in the shipment and development of its memory channel storage technology following months of impediments related to legal wrangles. Among other things the company today announced that the US District Court for the Northern District of California has ruled to completely dissolve a preliminary injunction enacted in January 2015.
..
Diablo's roadmap back on track following jury verdict
Editor:- March 25, 2015 - Diablo today announced it has won a "decisive victory" in its critical court battle with Netlist.

The jury unanimously concluded that there was no breach of contract and that there was no misuse of trade secrets. Further, the jury confirmed Diablos sole ownership and inventorship of the 917 patent.

We are extremely pleased with the jurys verdict today, said Riccardo Badalone, CEO and Co-Founder of Diablo Technologies. We look forward to getting back to serving our customers and delivering on our exciting Memory Channel Storage roadmap.

Editor's comments:- Diablo replaced Fusion-io as the #1 most searched SSD company by the readers StorageSearch.com in Q3 and Q4 2014 - because you all know a disruptive SSD technology platform when you see it.

The long running legal tangles got to the point where Diablo was prevented by an injunction from making more products - pending a trial outcome.

This verdict means the enterprise application acceleration industry can resume its onwards progress by being able to count on the availability of a significantly different flash latency asset and software platform for deployments inside the server box.
..
Diablo updates position re UlltraDIMM sanctions
Editor:- February 24, 2015 - If - like me - you've been following with interest the development of true SSD acceleration technologies packaged in RAM DIMMs (aka memory channel SSDs and similar names) then you may have been wondering - what's the current state of the play in the Netlist versus Diablo and SanDisk patent and implied rights to IP legal wrangle?

The last furious clash of legally related press releases - from both sides - in mid January - ended with a lot of smoke in the air - and dire expectations regarding body count. In particular the impression was that - until the next court session on these matters - further shipments of SanDisk's ULLtraDIMM SSDs would be suspended.

This is an update sent to me yesterday from a spokesperson communicating the Diablo side of things. So "we" and "our" in the text below means from the perspective of "Diablo".

re Preliminary Injunction - Diablo says
  • SanDisk has been granted a stay on their preliminary injunction, meaning that they can ship their inventory to Lenovo, Supermicro, and Huawei.
  • The preliminary injunction on Diablo is still in effect while we await the standard appeal process.
  • It continues to be our belief that the standard appeal process will find in our favor.
At the center of the dispute is the idea that our technologies compete. Our technologies do not compete. There is a long list of reasons why they dont. Here are some of them:
  • The Netlist NVvault is memory. The OS and applications see it and treat it as DRAM, which is why no OS drivers are necessary. Ours is storage. Ours is seen by the OS, hypervisors and applications as a block storage device and this is why MCS does require OS drivers.
  • They are used differently. For example, you typically would not put a whole database on a DRAM NV-DIMM but you would on an MCS-based device.
  • A DRAM DIMM can be used in place of DRAM, an MCS-based device cannot. An MCS-based device, because it is storage, requires separate DRAM in the system for execution. DRAM based devices are required to make the server run and are complimentary to MCS-based devices.
  • A DRAM NV-DIMM cannot be removed from the system and replaced with an MCS-based device and be expected to perform the same function.
  • The NVvault product is an 8GB device because it is a memory device. Since ours does not use DRAM and instead interfaces directly to flash, it is capable of being hundreds of gigabytes in capacity.
  • JEDEC has defined a DRAM-based NV-DIMM (NVDIMM-N) as a completely different category from an MCS-based device (NVDIMM-F) because they operate differently and service different applications in very different ways. There are several other companies building NVDIMM-N devices including Netlist, Viking and others, while Diablo is the only company we know of that is building an NVDIMM-F device.
  • Simply because they both fit into the same slot and use a similar interface does not mean that they compete. Most PCIe cards serve completely different functions and do not compete, even though they use the same physical interface (examples are graphics, audio, networking, and storage cards).
Editor's comments:- I think it's important for the SSD industry to know whether it can count on seeing a competitive market for memory channel SSDs being developed. For that to happen it is essential for Diablo to establish in the courts or by agreement as soon as possible that the roadmap for its kind of technology has a future.

If this doesn't happen quickly - and if the whole issue is left unresolved for another year - then the window of opportunity for this class of enterprise SSD may close. Because - as far as I know - Netlist doesn't have a Diablo like product in a similar state of market readiness.

So if Netlist were to succeed in preventing Diablo's product roadmap - there isn't a similar product which architects could fall back to. And even if Netlist chose to pursue that kind of product opportunity - which it can't do on its own - the SSD market isn't going to wait idly by for another 2 years waiting for that to happen.

Other ways of adding applications intelligence into PCIe SSDs - and other alternatives to RAM cached to flash are already in development. And the software market has to judge - which new markets are most likely to return value on their developer investment.

Sanity check

Just to remind you - the bullet points above - came from Diablo and whether you agree or disagree with them or not - the reason so many lawyers are involved in this right now is more to do with the fact that 2 companies (Netlist and Diablo) have a different recollection of what they once agreed in a past collaborative project and they disagree on what rights that past agreement confers on what they're doing now.

If I get more updates I'll let you know.

The key things for now are:-
  • if you've got a design which uses 1st generation UlltraDIMM style memory channel SSDs - then you can still get products to fill those slots.
  • But - if you've been planning around the preannounced 2nd generation products - your projects are probably on hold.
..
Diablo appeals shipment injunction - says court was misled
Editor:- January 14, 2015 - Diablo Technologies today announced that it has appealed the court ruling (reported earlier this week - and initiated by Netlist) which had granted a preliminary injunction to halt Diablo's shipments of Memory Channel Storage based chipsets.

Diablo's appeal explains that the ruling is based on an erroneous interpretation of the contract and a failure to recognize the technology differences among the products involved.

Most importantly, however, the court did not find that Diablo MCS uses Netlist trade secrets.

Diablo says - to support the judgment, the order effectively rewrites the language in the contract signed by the parties in 2008: the additional words included in the order changed the terms of the contract significantly and imposed a new obligation that was not agreed between the parties. In other words, there was no violation by Diablo of the original contract. The court was misled about important technology distinctions:

The court relied on Netlist's representation that their HyperCloud and Diablo MCS "are used to perform the same function" which is not the case because the HyperCloud is DRAM (memory) and Diablo MCS is a block storage device (disk).

The court also relied on Netlist's representation that the products are competitive because they both "attach to the same memory channel."

Diablo also says "Netlist equates the 2 devices simply because they use the same location and i/o channel; extending that logic would equate all devices that reside in PCIe slots, which would be a similarly erroneous claim."

"It is important that the facts of the case are well understood; it should also be stressed that the court did not determine that Diablo uses Netlist trade secrets," stated Riccardo Badalone, CEO and co-founder of Diablo Technologies. "We offer an innovative storage device that gives customers great performance advantages, but with this injunction, the court is putting our company and our customers at risk. With this appeal, we expect to reverse this decision and get back to work."

Editor's comments:- are Netlist's products functionally different to those using Diablo's architecture?

Yes! The differences are so great that the 2 products are listed in different directories here on StorageSearch.com
  • Memory Channel Storage SSDs - which encompasses low latency, fast flash SSDs which plug into DIMM sockets and transfer data via interfaces which were originally designed for DRAM.
  • hybrid DIMMs, NV DIMMs, flash backed DRAM DIMMs - which includes DRAM modules which automatically save their contents when electrical power drops to an integrated non volatile memory from which the data is reloaded after normal power is restored.
The fact that standards organizations and some vendors have historically used the term "flash DIMM" in the context of both types of products - has contributed to industry confusion.

But the term - flash DIMM - is a description of the physical form factor - and tells you nothing about the operation and functionality of the device from a data architecture point of view.

The differences are vast and immediately obvious to anyone who's technical. But I was wondering how would I explain the gulf of difference to someone who doesn't know anything about computer design.

My analogy goes like this... Suppose someone offered you the choice of 2 types of backpack when you started a balloon ride.

One is a parachute, the other is a jetpack.

They both perform different functions - although - until you activate their functions they both might look the same.

OK I know you need a higher skillset to operate the jetpack.

And you also need a much more developed SSD software support ecosystem to deploy memory channel SSDs too.
..
California Court halts sales of ULLtraDIMM SSDs
Editor:- January 13, 2015 - Netlist today announced that the US District Court for the Northern District of California has granted Netlist's motion for a preliminary injunction against Diablo Technologies for controller chips used by SanDisk in its high-speed ULLtraDIMM SSD product line.

Under the court's order, Diablo and SanDisk are prohibited from manufacturing and selling the controller chipset used by SanDisk in the ULLtraDIMM and as a result, from further sale or distribution of the ULLtraDIMM itself.

Judge Gonzalez Rogers - also rejected SanDisk's motion for reconsideration, asking that it be allowed to sell existing inventory of the enjoined products.

The court advanced the trial date to March 9, 2015, for Netlist's claims upon which the motion was decided, including claims against Diablo for trade secret misappropriation, breach of contract, and other causes of action related to the components supplied by Diablo for the ULLtraDIMM.

The court's order specifically identifies the ULLtraDIMM as well as the eXFlash modules from IBM, although the injunction affects all modules containing Diablo components.

Editor's comments:- the possibility of such a injunction has been discussed in these pages before. At the heart of the dispute are whether an earlier design collaboration between Netlist and Diablo included the rights for both companies to use a critical interface design in the DIMM bus facing part of Diablo's memory channel storage design. A patent ruling on December 29, 2014 (see below) upheld Netlist's patents related to this.

The court case related to the injunction stems from Netlist's argument that if Diablo did not have the right to use the interface technology - then it follows that any design - such as the ULLtraDIMM SSD - which relies on such internal technology - should only be disallowed - pending any future agreement about licensing such a technology. (Which it may not choose to do.)

Netlist wants to create a low latency memory channel SSD product line of its own.

And even though Netlist is more than a year behind the productization of this integrated technology (compared to the first generation ULLtraDIMM designed Diablo and SMART Storage - which was acquired by SanDisk) the case from Netlist is that the market must wait for its own design or any designs for which it grants licenses.

From the Diablo side - its argument has been that they thought they had an agreement which allowed them to use the DIMM interface technology (in whose implementation they participated) in products which were different to Netlist's flash backed DIMMs.

At stake is a future market for server based accelerators which could be worth a low double digit percentage of the entire enterprise PCIe SSD market.

It's inconceivable that a small company like Netlist or Diablo would be able to satisfy such demands on their own - especially given the fact that neither has any core IP related to enterprise flash.

But future licensing partners (or wouldbe acquirers) need to be satisfied that the core technology they're using - is patent troll proof.
..
Netlist revalidates core patent related to ULLtraDIMM's core technology
Editor:- December 29, 2014 - Netlist today -announced that the US Patent and Trademark Office has denied petitions requesting Inter Partes Review of Netlist patents asserted against the ULLtraDIMM.

The petitions were filed by SanDisk who partnered with Diablo Technologies to produce the ULLtraDIMM. Both are co-defendants in Netlist's patent infringement action pending in the US District Court of the Northern District of California. Of the 5 petitions filed by SanDisk, 3 were denied in their entirety and a 4th was denied as to some of the challenged claims.

The Patent Trial and Appeal Board denied institution as to all claims of U.S. Pat. No. 8,516,185 which relates to the fundamental distributed buffer architecture created by Netlist and is integral to the architecture of the ULLtraDIMM. Among other functions, the distributed buffer architecture allows a proprietary DIMM containing various types of memory to appear to the system as standard DRAM memory.

Chuck Hong, Netlist's CEO said, "We are very pleased with this outcome and the overall progress in our multi-year legal proceedings against ULLtraDIMM and SanDisk. The technologies covered by our patents are critical for the enterprise computing and storage space. We believe this ruling by the PTAB is a clear validation of our IP in this area and a testament to the years of seminal development work and ongoing investment."

Editor's comments:- One of the things which I had expected to see in 2014 - but didn't - was the appearance of more vendors competing in the low latency memory channel SSD market.

Although one reason for this could be that developing any new server based ecosystem requires more software investment than introducing new products in already established markets (thereby making this a riskier investment choice compared to new form factors of PCIe SSD - for example) another reason may be that the likely suspects to enter the future memory channel market are waiting to get clarification from the courts as to which companies they should be seeking to get patent licenses from (if any).
..
flash backed DIMMs - new directory on StorageSearch.com
Editor:- October 21, 2014 - Although StorageSearch.com has been writing about flash backed DRAM DIMMs since the first products appeared in the market - I didn't think that subject was important enough before to rate a specific article or market timeline page.

That's unlike this page you're seeing now - memory channel SSDs - which has become 1 of the top 10 SSD subjects viewed by readers after having had its own directory page here since April 2013.

Despite my lack of initial enthusiasm for bybrid DIMMs (or hybrid drives for that matter too) I realize that sometimes a market is defined as much by what it isn't as by what it is. Which is why I have relented.

And so - to help clarify the differences between these 2 types of similar looking storage devices (one of which I think is much more significant than the other - but both of which are important for their respective customers) I have today created a directory page for hybrid DIMMs etc - which will act as the future launch pad for related articles.
..
Netlist asks court to shut down SanDisk's ULLtraDIMM production
Editor:- October 10, 2014 - Netlist recently announced it has filed a motion for preliminary injunction which seems to be intended to restrain Diablo and its flash-side SSD integration partner SanDisk from any further manufacture or sale of the ULLtraDIMM (memory channel SSDs).

The Court has set a tentative date of December 2, 2014, for a hearing on Netlist's motion. If granted this would immediately shut down any further manufacture and sale of the ULLtraDIMM. Netlist has further asked the Court to order the recall of any ULLtraDIMM products previously sold.

Editor's comments:- This series of legal disputes has been going on since last year. If you haven't read those stories - the essential story is something like this.

At some time in the past Netlist and Diablo had a technology supplier agreement - as a result of which - Diablo had access to Netlist's IP related to minimizing the capacitive load of complex circuits susch as controllers when they sit on a DRAM style of bus. Netlist doesn't have any products which are similar to SanDisk's ULLtraDIMM - but Netlist claims that the ULLtraDIMM design has used its patented interface technology without a license. Countering that - Diablo says it did get the rights to use some of the DRAM load interface technology - and that in any case - this aspect of the design is not the essential defining characteristic of their flash SSD architecture.

Yesterday Diablo's CEO - Riccardo Badalone retorted to Netlist's latest legal move saying this - "After a year of court proceedings and months of discovery, Netlist still cannot decipher how Memory Channel Storage works, much less substantiate that it infringes on any of their IP."
..
"Having SSDs located in a DIMM socket in one server - no longer precludes that very same data being accessed by another server as if it were just a locally installed PCIe SSD."
Zsolt Kerekes, editor - StorageSearch.com in an SSD news story linking Diablo to A3CUBE (September 23, 2014)
..
Diablo countersues Netlist
Editor:- September 25, 2014 - Diablo announced today that is has filed a lawsuit against Netlist for unfair business practices that violate Diablos IP rights.

This appears to be a countermeasure to 2 earlier lawsuits initiated by Netlist against Diablo - which were widely reported by the SSD related press in January 2014.

Today - Diablo reiterates that its Memory Channel Storage (DDR3/4 form factor and interface compatible flash SSD) is a new and innovative architecture that neither infringes upon, nor misappropriates any Netlist IP rights. And Diablo argues that its MCS-based products and the Netlist HyperCloud DIMM (high density DRAM) - which were the cited products in Netlist's earlier legal moves - are designed to serve different purposes and are not interchangeable.

Diablo says the contract between the 2 companies (which have been mentioned in the press) clearly assigns legal ownership of the implementation IP in the HyperCloud chipset to Diablo. As a result, Diablo is seeking damages for breach of contract for Netlists attempt to usurp the companys IP rights.

We have been very patient throughout this entire process and it is now time for us to share our side of the story" said Riccardo Badalone, CEO and Co-founder of Diablo Technologies. "We will demonstrate definitively that products based on the Memory Channel Storage architecture do not use any Netlist IP.
..
another auspicious design win for ULLtraDIMM
Editor:- September 16, 2014 - SanDisk today announced that its ULLtraDIMM (memory channel SSD) has been selected by Huawei for use in its RH8100 V3 servers.

Huawei is ranked the top server supplier for cloud and mobility in China - by Sino-Bridge Consulting.

Editor's comments:- Since the January 2014 announcement that IBM was using ULLtraDIMM SSDs in some high end servers - there haven't been many conspicuously auspicious design win announcements like today's Huawei story.

One reason is that IBM had a head start on the market - having worked with Diablo for years to refine the MCS architecture and software APIs.

Another reason is that the 1st generation ULLtraDIMMs apparently guzzled more electrical power than modern RAM DIMMs even though they were still within the permitted power envelope according to industry standards. This means that in order to support arrays of them in a server design (and indeed you do need arrays to get meaningful performance beyond the PCIe SSD level) requires a redesign of the copper power tracking on the motherboard. You can't just plug large numbers of ULLtraDIMMs into any old server without analyzing the thermal consequences.
.....
Diablo unveils DDR-4 flash DIMM SSDs
Editor:- August 7, 2014 - Diablo yesterday announced details of a new 2nd generation memory channel SSD - low latency flash SSD accelerators in DDR-4 sockets - which will sample to oems in the first half of 2015.

Along with the new hardware technology there will be an improved software platform - with features like NanoCommit - which Diable says will enable hundreds of millions of transactions per second, with nanosecond latency.

"Memory Channel Storage DDR4 solutions represent the next evolution of Server Acceleration technology," said Riccardo Badalone, CEO and Co-founder of Diablo Technologies. "In addition to supporting a faster memory interface, the Carbon2 platform delivers unprecedented levels of hardware acceleration for new software innovations like NanoCommit. Converged Memory, where the best of Flash and DRAM are combined, will rely on this type of technology to give applications the ability to transparently persist updates to main memory."

Editor's comments:- After FMS - Diablo sent me more info (pdf) about their FMS presentation (pdf) from which I have extracted these key features.
  • Diablo's converged memory architecture (flash tiered with DRAM) is planned to support 700 million random cachelines / sec.
  • Latency of each cacheline is about 48 nanoseconds.
  • Diablo's NanoCommit supports byte addressable small writes to flash with high transaction rates and the ability to mirror the DRAM contents to persistent storage.
  • The combination of technologies would enable something like a 1U server with 25TB of converged memory.
Diablo MCS - nanocommit - click for pdf
.....
Diablo secures patent related to MCS technologies
Editor:- July 31, 2014 - Diablo today announced that the US Patent and Trademark Office (USPTO) awarded U.S. patent No. 8,713,379, entitled a System and method of interfacing co-processors and input/output devices via a main memory system, to the company. The 379 patent describes:
  • A method for connecting non-volatile memory directly to the memory controllers of a processor.
  • A learning machine to handle data interleaving/de-interleaving and data scrambling/de-scrambling algorithms for DDR3/4-based memory controllers.
  • A method to remap the non-linear DIMM address space back to linear address space used by the driver
.....
PCIe SSDs versus memory channel SSDs are these really different markets?
Editor:- June 10, 2014 - Throughout storage history we've seen many claims that a new upstart storage technology will replace another.

It doesn't always work out to be as simple as that.

In the enterprise SSD market of today - PCIe SSDs are the safe standard legacy server acceleration SSD technology and memory channel SSDs with akas like UlltraDIMM, TeraDIMM, MCS etc are the upstart unproven wannabies.

Each of these 2 SSD types can do some very similar things - such as make server apps run faster - but they aren't the same.

It's reasonable to ask - are there any hard technical rules where you can say for your new projects either:-
  • I don't need to know about the new thing, or
  • maybe I should look at it?
Without getting mired in contentious product by product benchmark comparison claims and counter claims?

See my blog - memory channel SSDs versus PCIe SSDs (slight return) - which was on the home page of StorageSearch.com throughout May 2014 and which now has a new permalink location. I've updated it with some new notes and contextual comments. ...read the article
.....
EMC acquires memory channel SSD in a box company DSSD
Editor:- May 5, 2014 - EMC has acquired a stealth mode rackmount SSD company - DSSD - it was announced today. Products based on the new DSSD architecture are expected to be available in 2015.

Editor's comments:- for a competitive analysis and technology perspective see rackmount SSD news
.....
1st generation ULLtraDIMM SSDs are limited by having "SATA inside" - so imagine what a next generation flash DIMM might perform like
Editor:- May 8, 2014 - Today I learned some new details about the architecture in SanDisk's implementation of memory channel SSDs by watching a video - SanDisk ULLtraDIMM Architecture and Design Technology - which was recorded at the recent Storage Field Day event (April 25, 2014).

Inside each flash DIMM are 2x SATA CloudSpeed SSDs which connect to the DDR3 interface via a Diablo designed bridge chip which has something on the order of "megabytes" of buffer memory inside.

That gives a much better basis for understanding the performance limits of these devices.

And it confirms why you only start to see noteworthy performance from them after installing 3, 4 or more modules.
inside  ultradimm - architecture video
However - despite the "SATA inside" limited nature of this 1st generation memory channel storage design - the performance - aggregated at a block level - results in a usable write latency envelope which IBM has verified to be in a consistently lower latency category than most leading PCIe SSDs.

So - if you're thinking long term - and roadmaps - it's reasonable to assume that if a 2nd or 3rd generation of MCS flash DIMM SSD could abandon the "SATA inside" and use instead a large architecture flash controller with a native memory array optimized type of bridge interface - then the gaps between what MCS can offer - compared to PCIe latency - could widen.

By the way in this 52 minute video I also learned 2 interesting details about SanDisk's CloudSpeed SSDs which I didn't know before:-
  • SanDisk's Guardian technology doesn't perform scheduled data integrity checks to detect silent errors for data at rest.

    The stated reason being that this would eat into endurance. - And the implied assumption being that SanDisk thinks this type of activity is best determined by the system designer. So - if you design these SATA SSDs into a storage array - then you'll have to take responsibility for that aspect of long term data health in your own software.
See also:- memory channel SSD news and articles
.....
Fusion-io says - we don't regard SanDisk's NVM DIMMs as a threat to our PCIe SSD business
Editor:- April 24, 2014 - Fusion-io recently reported revenue up 14% yoy to $100 million but losses have grown too - to $30 million for the recent quarter.

Editor's comments:- In the investor conference call an analyst asked - in effect - whether SanDisk's ULLtraDIMMs should be viewed as a threat to Fusion-io's PCIe SSD business?

Lance Smith, President and COO at Fusion-io said - he thought NVM DIMMs were a validation of what FIO had already done - but he regarded NVM DIMMs (in their current form) as a niche market.

He went on to say - "We don't believe that DDR-3 and the memory interface is the bottleneck. It's about managing the nand and getting the best performance out of the nand."

Editor comments:- Whereas I agree with Fusion-io that the hardware latency difference in itself between MCS and PCIe is less significant than the combined effects of flash controller architecture and FTL software interplaying with data in an SSD-aware software stack - nevertheless I don't agree with Fusion's downplaying and dismissal of the market and technology significance of MCS.

I think that in maybe 2 to 6 quarters - this comment could return to haunt them.

Because it's precisely in the kind of high end use cases which Fusion-io currently dominates - that the MCS technology will be most economic too.

It will take at least that long - because Fusion-io already has very broad, deep and proven software surrounding their flash and an entrenched 7 year established hold in the server market.

In contrast - SanDisk's best selling enterprise SSDs are SAS SSDs - and even before acquiring the MCS product line last year - SanDisk's penetration of the server side caching market in the words of its own CEO - had been negligible.
.....
new whitepaper from Diablo looks at the "drawbacks of traditional PCIe SSDs"
Editor:- April 22, 2014 - Diablo Technologies today published a white paper Enterprise Storage Performance: it's all about the Architecture (and not just the interface) (pdf)
graph from the white paper
This paper presents Diablo's perspective - as the creator of Memory Channel Storage and looks at the measured performance weaknesses of some specific (but unnamed) PCIe SSDs with respect to their measured performance asymmetries under various loads and then attempts to generalize those weaknesses as part of an argument that MCS is a better solution than PCIe SSDs - because MCS is more scalabe and doesn't suffer from the same kind of architectural bottlenecks.

Editor's comments:- This paper from Diablo reminds me of the type of comparisons between different types of PCIe SSDs (and their sensitivities to different workloads) which was one of the pivotal marketing points of difference which Virident used to hang their reputation on in the years leading up to its acquisition by HGST. (Although as I often reminded readers at the time - Virident wasn't the only company with that kind of array scalability or no-compromise performance.)

Going back to Diablo's white paper - for me - like many vendors written papers - it's good and bad in different parts.

The best bit is the middle - in which you get a reminder - from measured results that fast SSDs with similar capacities can behave differently.

The worst part in my view is the attempt to link these comparison results to a general conclusion about the merits of MCS versus PCIe SSDs.

Because I think the architecture of the controllers on the flash side of the SSDs plays such an important part as too does the software. And that's still just a small part of the picture.

My considered view is that Diablo's extrapolation towards a general market conclusion from one selected comparison example is not merited by the evidence presented in this article.

And while I am convinced of the benefits of Diablo's MCS architecture - and what it can do differently and better compared to PCIe SSDs - I think this particular paper isn't the best case they could argue.

related articles
.....
market boundaries analysis for memory channel SSDs
Editor:- April 17, 2014 - StorageSearch.com today published a new blog - memory channel SSDs versus PCIe SSDs are these really different markets? - which revisits (from a current perspective) some of the key questions I posed a year ago about Memory Channel SSDs.

Are these really different markets? And - at the limits of price - what are the different application roles - where one type of SSD is better than the other? ...read the article
.....
what's in a number? - for MCS add 4
Editor:- March 4, 2014 - The presence (or absence) of Memory Channel SSDs in a server is one of the factors which go towards calculating the SSDserver rank - a new latency based configuration metric - which is proposed as a standard by StorageSearch.com. ...read the article
.....
Memory Channel Storage Demystified webex
Editor:- SNIA hosted an open call to discuss memory channel SSDs on March 10, 2014.
.....
Diablo comments on legal barbs cast by Netlist
Editor:- February 14, 2014 - Diablo's CEO and founder - Riccardo Badalone was reported to have said in an interview blog by Willem ter Harmsel today - re the lawsuit launched against them by Netlist...

"They (Netlist) basically have yet to prove that:- they understand the product, and also that they have found anything Diablo is actually infringing on. We are just going to let this run through the (legal) process and we are confident about the outcome." ...read the article
..
Micron doesn't currently have a low latency, high capacity, flash DIMM SSD technology platform in its product line yet. But I think it's inevitable that Micron will have to publicly address this product gap to maintain confidence in its server customer base. Clearing the patent and IP decks in readiness for this may have been one of the factors in the December 2013 patent deal with Rambus.
the Top SSD Companies in Q4 2013 - February 4, 2014
..
"That leads on to the question of - how much revenue will this ULLtraDIMM SSD bring to SanDisk in the next 18 months?"
sizing the scale of market for ULLtraDIMM SSDs - January 21, 2014
.....
"We're seeing 5-10 microseconds write latency for eXFlash DIMMs in preliminary testing vs. 15-19 microseconds latency for PCIe-based flash storage from Fusion IO, Micron, and Virident, and 65 microseconds latency for Intel S3500 and S3700 SSDs"
IBM:- January 16, 2014 - X6 Architecture press release
.....
"One of the most significant new form factors introduced in the enterprise SSD market in 2013 was memory channel storage."
what changed in SSD year 2013? - December 9, 2013
.....
McObject shows in-memory database resilience in NVDIMM
Editor:- October 9, 2013 - what happens if you pull out the power plug during intensive in-memory database transactions? For those who don't want to rely on batteries - but who also need ultimate speed - this is more than just an academic question.

Recently on these pages I've been talking a lot about a new type of memory channel SSDs which are hoping to break into the application space owned by PCIe SSDs. But another solution in this area has always been DRAM with power fail features which save data to flash in the event of sudden power loss. (The only disadvantages being that the memory density and cost are constrained by the nature of DRAM.)

McObject (whose products include in-memory database software) yesterday published the results of benchmarks using AGIGA Tech's NVDIMM in which they did some unthinkable things which you would never wish to try out for yourself - like rebooting the server while it was running... The result? Everything was OK.

"The idea that there must be a tradeoff between performance and persistence/durability has become so ingrained in the database field that it is rarely questioned. This test shows that mission critical applications needn't accept latency as the price for recoverability. Developers working in a variety of application categories will view this as a breakthrough" said Steve Graves, CEO McObject.

Here's a quote from the whitepaper - Database Persistence, Without The Performance Penalty (pdf) - "In these tests eXtremeDB's inserts and updates with AGIGA's NVDIMM for main memory storage were 2x as fast as using the same IMDS with transaction logging, and approximately 5x faster for database updates (and this with the transaction log stored on RAM-disk, a solution that is (even) faster than storing the log on an SSD).

The possibility of gaining so much speed while giving up nothing in terms of data durability or recoverability makes the IMDS with NVDIMM combination impossible to ignore in many application categories, including capital markets, telecom/networking, aerospace and industrial systems."

Editor's comments:- last year McObject published a paper showing the benefits of using PCIe SSDs for the transaction log too. They seem to have all angles covered for mission critical ultrafast databases that can be squeezed into memory.
..

Memory Channel Storage SSDs

will the new concept fly? - should you book a seat yet?

by Zsolt Kerekes, editor - April 29, 2013

Recently we've been hearing more details about the implementation plans for a new type of fast enterprise SSD.

Memory channel storage was devised by Diablo Technologies.

While in stealth mode Diablo had hinted it was planning to use its assets and expertise in DRAM interface chips and datacomms to design a new type of enterprise SSD which would couple - the capacity cost advantages of flash with the low latency characteristics of a RAM "bus" to create a new type of fast storage device which would be an order of magnitude faster at accelerating apps than good old PCIe SSDs.

To make a product like this viable will require world leading skills in flash controller and management expertise - with particular emphasis on expertise in enterprise SSD markets.

Diablo last week announced that it has chosen SMART Storage Systems to exclusively supply the necessary flash SSD expertise.

The 2 companies will work together to bring the first products to market in the coming year. The products will be sold by SMART.

At this point you may well ask...

What does a datacomms rooted company like Diablo (because that's where the founders came from) know about SSDs?

We'll have to see as details emerge later. However, in past conversations I've had with people in enterprise SSD array pioneer Texas Memory Systems (acquired by IBM) - TMS had characterized their big controller SSD architecture as being primarily "a router" which could move data reliably from a SAN into a memory array with low latency, fast throughput and good data integrity.

And another place we've been seeing datacomms concepts entering the SSD market recently is in the field of adaptive DSP and ECC techniques - most explicitly in the nomenclature of controller and IP products from DensBits - whose "memory modem" approach uses the philosophical idea that charge based R/W errors in flash cells can be modeled and managed in similar ways to signal noise in comms circuits.

Who's SMART? (Diablo's flash partner).

If you're not already familiar with SMART - a quick summary is to say they are a Top 20 SSD Company, which is a leading supplier in the SAS SSD market, and has its own adaptive R/W flash ECC IP and its own enterprise SSD controllers. SMART doesn't have a PCIe SSD product line. So you might think that was a strange partner choice - but I'll be returning to this question later in this article.

I spoke recently about the market opportunity and technical challenges posed by memory channel SSDs with John Scaramuzzo, President of SMART.

A small part of the article which follows here - is based on that discussion. But most of the article below is based on my own thoughts - inspired by the kind of questions which I know many of you would ask. These speculations are based on my own analysis of the SSD market.

what's different between this and past flash SSDs in DRAM packages?

John and I briefly discussed why you shouldn't confuse memory channel SSDs with past attempts by memory makers to place flash SSDs in DRAM style DIMM packages.

The two best known companies who have gone down that route are Viking and Micron - in whose DRAM compatible flash products the flash is used as a backup for DRAM which triggered into operation by a drop in the power supply. There have also been DIMM flash SSDs which were mounted on the motherboard in a DRAM package - but where the data was connected via another interface such as SATA - and not through the DRAM bus.

Another phrase which SMART likes to use in relation to the Diable technology is "ultra low latency" SSD. John told me the purpose of the ull SSDs was to provide latency an order of magnitude faster than existing PCIe SSDs.

ULL SSDs - PCIe SSD killers or collabrative co-workers?

One question I asked gave me a picture of how the PCIe SSD market might look - even if we rushed ahead and assumed that ULL SSDs were successful beyond the wildest dreans of their creators (which as we'll see later - stretches a lot of assumptions).

I asked - how many inches in physical distance can the memory channel SSD bus be routed? And specifically - can it hop off the motherboard?

John said - no that's not the idea. The current concept is to be operate the same as a DRAM bus.

So I could immediately see 2 levels of segmentation and co-existence with PCIe SSDs - which can occur - even if we assume that every fast server has memory channel SSDs installed.
  • 1st level of market co-existence:- is high performance server clusters - where PCIe SSDs effectively provide the next level of SAN-like data connectivity and fault tolerance functionality using PCIe fabric technology.

    PCIe fabric - sharing data between different physical servers at low latency - is something which PCIe SSDs can do which DRAM memory technology can't (yet).

    One way to think about this is - what's the best way for one memory channel storage accelerated server to talk to another? The answer is via a switched PCIe SSD fabric.
  • 2nd level of market co-existence - is entry level to mid-range server performance.

    For example - in cost sensitive server markets like iSCSI SSDs.

    That's because the memory management technology needed to implement a Diablo-style ull SSD will be more expensive to implement than entry level fast-enough PCIe SSDs - which don't need such fast SSD controllers and don't need RAM caches or nvRAM caches - each of which escalates upwards the floor price of the ull SSD.

    The cheapest fast-enough PCIe SSDs will always be cheaper than the cheapest memory channel SSDs.
That's my analysis of what would happen - even if the new technology was wildly successful.

John also confirmed that he saw these as different markets. And for example - at some future date - when there's an adequate set of industry standards for PCIe SSDs - you can still expect that SMART might bring to market its own family of PCIe SSDs.

For many of you - that's the key message from this article - and you don't need to read any further. I'll say it simply here...

Memory channel SSDs will not remove the need or market desireability of PCIe SSDs.

In extreme cases it's possible to imagine servers in which both types of SSDs are operating (in different functional roles) at the same time. And in my view - the market opportunity for PCIe SSDs will remain much bigger - because it can be adapted more economically for a wider spectrum of performance.

This brings us to the next 2 big questions and problems about bringing ull SSDs to market - flash technology and controller technology

flash technology and ull SSDs

Without giving too many details away John told me that getting the memory technology optimized for the low latency and high IOPS of ull SSDs was going to be a tough challenge.

If you look at the endurance problem for a very fast SSD of this type - the demand pushes you in the direction of SLC like characteristics.

The latency can only be met by true DRAM or nvRAM like memory.

John told me - they weren't in this venture with Diablo only to design a product which worked. The product had to be affordable too. Therefore it has to be flash rich.

If I step back and speculate what performance might be needed in this new type of SSD - it's way beyond what SMART implements in its SAS SSDs.

On the other hand - adaptive R/W flash IP (of the type which SMART has) can be tweaked to deliver SLC-like performance.

And if you look at the rackmount SSD market - Skyera shows you can get very high system performance and reliability by using adaptive R/W alongside nvRAM cache.

In fact - if you look at a technology roadmap for future flash memory and SSDs - it would be a mistake to launch the new memory channel SSDs using an SLC based implementation - because apart from the cost penalties - SLC doesn't have a geometry scalable future. When you shrink it - it picks up all the baggage of problems of MLC. So MLC and adaptive R/W is the only way to go. (That's true for industrial SSDs too - and not just in the enterprise flash market.)

My guess is that Diablo must have realized that when they were looking at the long range memory future. And that's one of the reasons they picked SMART.

SMART is currently one of the leading companies which spans both adaptive R/W technology and has enterprise SSD market experience but doesn't yet have the inconvenient distraction of having a PCIe SSD product line.

SMART's controller performance problem

SMART's current SSD controller technology (as publicly revealed) can simply be described as fast enough for the purposes of 2.5" SSDs - which it was designed for - but nowhere near fast enough compared say to fast PCIe SSDs such as those from Fusion-io, Virident or Micron.

So how's SMART doing to fill the controller gap to deliver fast ull SSDs?

This is my speculation here and not based on anything said to me by anyone in SMART.

But there are several ways you can look at this problem....

Scaling and bundling the technology which aleady exists.

For example OCZ was the first company which demonstrated that you can design very fast throughput PCIe SSDs by using arrays of controllers which had been designed for 2.5" SSDs. In OCZ's case their original PCIe SSDs included arrays of SandForce controllers - and are now available using arrays of their own OCZ controllers.

This approach (using multiple Guardian controllers) would be feasible for SMART to use in new ull SSD too. It would need a fat RAM cache to deal with latency and endurance peaks. But it's not the most efficient solution.

When you get into the fast enterprise SSD club you start seeing controller designs which have similar characteristics.

These big controller designs tend to split the controller design into different partitions - an intelligence and host / apps leaning side and a distributed flash management I/O side.

My guess is that new ull products from SMART could be eASIC or FPGA heavy implementations rather than simply arrays of their Guardian controllers. That's because when you own the algorithms and IP - you aren't tied to past implementations of how you packaged it. You can implement the functions in any way that makes best sense for the market you're going for.

It doesn't really matter which way SMART does it as long as it's fast enough and is affordable.

The host interface and high end memory controller / CPU IP side of things is where it would be natural to assume that Diablo will be contriibuting their own IP.

how successful will memory channel SSDs / ull SSDs be?

This is where we can return to some important strategic speculation.

First of all - in looking at what problem these new SSDs might solve which isn't being solved today - an important thing to realize is that the competitive environment isn't the same as when Fusion-io started shipping PCIe SSDs back in 2007.

The new ull SSDs have to fight for a viable toehold in an already pre-existing and very sophisticated market for enterprise SSDs.

And even though the state of the SSD software leaves many questions open to debate the importance of software in the SSD market as a sales accelerator is well established and well understood - even if mostly by the high sums that such companies have been acquired for - relative to their modest revenue achevements as standalone ISVs.

You can get some interesting insights into this by using the mental trick I call SSD market boundaries analysis and rephrasing some of the assumptions about the new technology to get these sanity checks.
  • if Fusion-io was the flash technology partner for Diablo (instead of SMART - and forgetting about the very material facts that FIO doesn't admit publicly to having the same flash technology and also has a very different approach to everyone else when it comes to SSD controller design) what could Fusion-io deliver with a memory channel connected SSD that it can't already deliver with a PCIe SSD?

    FIO has demonstrated in market shipping products that shrinking down latency requires more than hardware design.

    You need to remove layers of hard drive related junk which is hard coded into many layers of enterprise systems software. But when you do this - sometimes by introducing new side-stepping APIs - you can get apps performance which is 10x faster than raw speedup which you get from simply running legacy software faster on SSDs.

    This is something which academic researches at the Non Volatile Systems Lab UCSD have also written about too.

    Fusion-io is already well down this learning curve - and if they had the benefits of a lower latency to memory technology - my guess is they would be best placed to further exploit it. Having said that, however, until we know what the Diablo/SMART memory channel controller latency delivers - we can't be sure how it will compare to what FIO already delivers with the worse latency of PCIe but better latency of a flash translation layer in which the apps server is the same CPU that manages the flash.
  • if SMART were simply to come to market with a new fast PCIe SSD (instead of a new type of SSD) it would struggle and probably fail to establish a market for such a product. If and when SMART does introduce its own PCIe SSD my guess is it would be a fast-enough product optimized for cost and efficiency rather than speed. But that's another story.

    Without a software base - simply having a lower latency PCIe SSD wouldn't get you many customers today.
The memory channel SSD concept - if the product is fast enough and affordable - will find a market if it can exceed the kind of server cost / performance which you can only get today by using Fusion-io's PCIe SSDs with Fusion-io's APIs - but in a business model which is independent of Fusion-io.

One of the problems for the Diablo / SMART ull SSD market is the lack of a spohisticated SSD ecosystem. That's a problem which faces any new type of SSD. My guess is that a critical part of Diablo's business model will have to be invested in showing how the new SSD type can side step those difficult "SSD" type integration questions - by delivering a product which looks to applications transparently like a bigger cheaper type of DRAM.

The idea of selling flash as a memory tier has long been on the agenda of leading PCIe SSD companies like Fusion-io and Virident - but the physics of flash and the characteristics of the the interface have meant that it's not been possible to do this without a lot of software being introduced into the mix. And the enterprise market is traditionally cautious about relying on single source solutions.

So - who are the first customers for these new ull SSDs likely to be?

Until the first products come to market we have to guess at their possible characteristics.

If you've been following my flow until now - my guess is the first memory channel SSDs will need to be fast SSD with apps acceleration in a similar class to using Fusion-io PCIe SSDs (and APIs).

But the memory channel SSDs will have a completely different software architecture. The possibilities range from a very light software support set in which the new SSDs are initiated in the server to look like a big RAM. Or maybe some SSD-like software in which the modules look like a traditional SAS drive.

Or the details may be completely different.

Who would be the ideal customers for such products? - It's going to be very tech savvie customers for whom pushing the boundaries of performance is worth the risk of investing in new technology because they use or sell thousands of servers. So it will be dark matter SSD users, SSD appliance makers and traditional server oems.

conclusion

It's always fun to speculate about the likely impact of new SSD technologies and changes in market directions.

As the above article is mostly based on speculating about a single new product line it's almost certain that in many detailed respects it will be wrong.

Nevertheless I think that the safe conclusion - which is almost where I started - is that even if memory channel / ull SSDs are successful in the market there are many ways to see how in the next 2-3 years they can still coexist with several different types of PCIe SSDs.

For related info see these articles and guides:-

PCIe SSDs news
RAM market news
SSD controllers and flash IP
PCIe SSDs aren't all the same
glue chips and IP in the SSD market
an introduction to enterprise SSD silos

.
SSD ad - click for more info
.
Beware the distorting lens of viewpoint
as in - "SSDs are similar to..."
Can you trust SSD market data?
.
"You'd think... someone should know all the answers by now. "
what do enterprise SSD users want?
.
"Don't believe everything SSD companies tell you about the past, present or future of the SSD market."
Survivor's Guide to Enterprise SSDs
.
"In an ideal world - symmetry considerations would be on page 1 of the - how to design an SSD cookbook."
11 key symmetries in SSD design architecture
.
It looks like you're seriously interested in SSDs. So please bookmark this page and come back again soon.

If you've got the time - you might want to take a look at the top 100 most popular SSD articles seen by our readers in recent months.
.
About the publisher - 21 years guiding the enterprise market
....
image for memory channel SSDs ..

when you're in a desperate hurry and can't afford to keep hanging around

..
If you're trying to understand the SSD and memory systems markets and looking for a single big idea which gives you a confident feel for what's going on - then you'll be disappointed if you stare too hard and look in the expected places.
1 big market lesson and 4 shining technology companies
..
SSD ad - click for more info
..
In 2015 there were significant and tangible product announcements around the ideas of rethinking enterprise RAM architecture - which have the impact to change the balance of memory types used in enterprise systems in as fundamental a way as SSDs themselves were predicted to change the server, storage and software markets in my 2005 article - 5 User Value Propositions for buying SSDs.
retiring and retiering enterprise DRAM and other ideas
..
SSD ad - click for more info
..

"I (still) don't think we've reached stability in reference enterprise SSD designs and use cases.

All the systems in the market today are implementations of transient architectures..."

Zsolt Kerekes, editor - StorageSearch.com

Original comment above (without the word "still") in an outlook blog - January 2015.

how fast can your SSD run backwards?
Big versus Small in SSD controller architecture
utilization and the enterprise SSD event horizon
Adaptive flash care management & DSP IP in SSDs
will SSDs end bottlenecks? - and cure all my problems?
memory channel SSDs versus PCIe SSDs - are these really different markets?
..
SSD ad - click for more info
..
One of the challenges for the enterprise SSD market when designing new products is to understand complex customer needs and decision criteria - which go beyond the traditional bullet points.

New segmentation models are needed because the enterprise SSD market is moving into uncharted territories and use cases where a considerable proportion of the customer needs which affect buying behavior are still formally unrecognized as being significant...
Decloaking hidden segments in the enterprise
..
SSD ad - click for more info
..
"See how optimizing processors for SSD can gain a 2x to 250x speed-up on popular functions as well as reduce the energy consumed by a similar amount! "
SSD Bookmarks - from Cadence
..
IBM Redbook places memory channel SSDs in onboard server apps contexts and hierarchy
Editor:- February 18, 2014 - IBM today published a new free 28 page ebook (aka Redbook) - Benefits of IBM eXFlash Memory-Channel Storage in Enterprise Solutions (abstract) / (pdf) - which describes how memory channel SSDs fit into the concept of servers relative to the other types of SSDs already available.

Editor's comments:- I've been writing about this technology since the time it was being developed and have been well briefed by the original developers - so this paper didn't have any great surprises to me - but I think this document presents a balanced introduction to this technology and a contextualized analysis of how it compares to the other well established SSD acceleration options which are available for use inside servers.

comparison table - click to see article

The key takeway - in my view is table 2 - in which you can see a hierarchy of write latencies which are approximately 5x longer in each case as you progress up the flash SSD options from memory channel SSDs, PCIe SSDs and SAS SSDs.

While bearing in mind that SSD data write latency is not the same as apps performance latency (because the integration of R/W data flow patterns with the software plays a significant part too) and also remembering that some products in the market will blur the ratio of the latency boundaries for these 3 different SSD types - you can, nevertheless see why memory channel has a distinct slot within the onboard SSD acceleration options which you need to think about.



..
Netlist says ULLtraDIMM SSDs infringe patents
Editor:- January 29, 2014 - Netlist today announced it had filed motions to add two additional patents to the lawsuits against the previously announced ULLtraDIMM memory module from Diablo Technologies and Smart Storage Systems.

The 2 newly added patents (U.S. Patent Nos. 7,881,150 and 8,081,536) are generally related to load reduction, a critical feature in low latency memory modules. Netlist has now asserted a total of 7 patents against the ULLtraDIMM, in addition to trade secret theft, trademark infringement and other claims.

"We have spent years developing our industry-leading technology, and are encouraged by the progress we've made defending our intellectual property in these lawsuits," said Netlist President and CEO, C.K. Hong. "We will continue to vigorously defend our IP, uncovering any and all theft of our technology and infringement of our patents, and will not allow others to profit from this unlawful activity."

As reported in its 8-K last December, Netlist says it received a whistleblower letter describing in detail how Diablo "stole Netlist's detailed architecture and design" of its flagship product, HyperCloud, to create the ULLtraDIMM.

Editor's comments:- SanDisk's ULLtraDIMM is a new class of product in which cheap high capacity MLC flash emulates many DRAMs in a module which contains little or no DRAM itself but which nevertheless plugs into a DDR3 module.

In contrast - Netlist's "similar sounding but very different" product families include real DRAMs which accomplish power fail data protection aka non volatility - by means of fast backup and restore to onboard flash - all in the same DDR3 module.

Netlist doesn't appear to be saying that it could design a product like the ULLtraDIMM - because it doesn't to my knowledge have the flash controller IP to do anything remotely similar.

Instead - what Netlist appears to be saying is that techniques in DDR3 design - which enable lots of circuitry to be placed behind a RAM interface - without placing too much load on it and slowing it down - which Netlist has itself patented - are suspected to have been used within the design of the ULLtraDIMM.
..
image for ULLtraDIMM IBM article
..
SMART samples first ULLtraDIMM SSDs
Editor:- August 8, 2013 - SMART Storage Systems today announced it has begun sampling the first memory channel SSDs compatible with the interface and reference architecture created by Diablo Technologies.
ultra low latency memory channel SSD
SMART's first generation enterprise ULLtraDIMM SSD (ULL = ultra-low latency) can be deployed via any existing DIMM slot and provides 200GB or 400GB of enterprise class flash SSD memory with upto 1GB/s and 760MB/s of sustained read/write performance, with 5 microseconds write latency. Throughput, IOPS and memory capacity all scale with the number of ULLtraDIMM deployed in each server.

Editor's comments:- With the current design -only one DIMM slot in each server has to be reserved for conventional DRAM. Apart from that constraint any DIMM slot can be used for either flash or DRAM as deemed necessary for the application.

For more about the potential of this technology, the thinking behind it and the competitive landscape relative to PCIe SSDs etc see my earlier articles on the Memory Channel SSDs page.
..
Diablo discusses details of Memory Channel Storage
Editor:- July 30, 2013 - Diablo Technologies today did the public launch of its its new technology - Memory Channel Storage - which repurposes the interface and form factor of server DRAM into a new architecture for ultrafast flash SSDs which the company positions as a competitive alternative to very fast PCIe SSDs.

Editor's comments:- I spoke recently to Diablo's new VP of Marketing - Kevin Wagner about the company's new MCS.

I had already gleaned a heads up on the boundary capabilities and market potential of the new technology in an earlier interview with Diablo's flash partner SMART. So it was natural to ask how Diablo viewed the impending acquisition of SMART by SanDisk?

Not surprisingly Kevin indicated that whereas Diablo already thought it had made the best choice before - the upside potential of having SanDisk as a partner for this memory technology made it 10x better.

Here's what I learned.
  • Diablo's TeraDIMMs are designed to fit standard sockets designed for DDR-3. They are electrically, form factor and power compatible. But instead of RAM - a typical TeraDIMM using today's technology might have 400GB of flash.
  • Diablo's controller architecture means that the host CPU can read and write from memory in the same transparent way as it would talk to RAM.
  • TeraDIMMs can be installed in every set of slots where you'd normally insert RAM. The only limiting factor in the current architecture is that at least one pair of slots has to be RAM. All the rest can be flash.
  • Diablo claims that MCS has better write latency than most PCIe SSDs.

    Specifically MCS has a write latency of 3 to 5 microseconds. This is really a write to the controller.

    TeraDIMMs have a skinny flash cache architecture - which means that apart from a small amount of register memory in the MCS controller itself - no RAM is used in the SSD. The MCS design includes power fail hold up capacitors which guarantee that all data which has been written to the TeraDIMM gets completely saved to flash.
  • From the applications point of view MCS looks like a massive amount of persistent RAM - but with terabytes rather than tens of gigabytes of memory space and with a cost structure closer to the market price of flash than DRAM.
  • Is it bootable? No. Not yet that wasn't regarded as a priority.
Although the product is not yet available - Kevin Wagner told me that Diablo has been getting a lot of interest from server companies. Diablo has been validating software with their ASIC based implementation for several months following good earlier characterizations of the design with FPGA. From the software point of view Diablo's aim was to prioritize a usable design which would work for the market as soon as possible. Obviously many possibilities for leveraging the basic technology spring to mind. I asked for example about preferred models of high availability?

Kevin said that so many companies are interested in what they're doing that they don't have enough resources to talk to them all right now. It won't be long before the company publishes more details of its reference architecture - and conversations have already begun with ISVs and other companies which could be the seeds of a new ecosystem.

But the current design has been designed to work in virtualized and non-virtualized environments and the company has got already got a good idea of how headline big data apps would perform in benchmarks using their technology.

I raised the question of inter-operability with PCIe SSDs (in the same MCS resident servers) and it looks like the guesses I had made about segmentation and collaboration and competition with other SSD types which I've already written about in my earlier article about memory channel SSDs - are still valid. So I won't repeat those points here.

We still have to wait for firm product pricing and configuration details. But if you had any doubts about where the money will be inside servers - MCS provides another new way of packing even more flash SSD capacity in.

..
"Much of the prior work in SSDs has treated SSDs as fast disks. Diablo's MCS takes a fundamentally different approach, and enables SSDs to look more like memory."
Our Flash Memory Investment in Diablo (November 2012) - blog by Alex Benik, Battery Ventures (whose many other investments also included adaptive R/W SSD controller trailblazer - Anobit)


.
"This really changes everything. The convergence of Diablo's groundbreaking MCS technology and our award-winning Guardian Technology is not just about re-architecting storage, but re-architecting the integration of servers, storage and Flash acceleration.
Diablo names SMART as exclusive flash partner to pioneer memory channel SSDs - (April 24, 2013)


.
"Fast non-volatile memories will soon make their appearance on the processor memory bus. They offer the potential for extremely low-latency, high-bandwidth access to persistent files. However, existing interfaces will impose large system call and file system overheads on those accesses, squandering the memories performance."
Quill: Exploiting Fast Non-Volatile Memory by Transparently Bypassing the File System (pdf) - Part of the growing SSD literature output coming out from - NVSL ( the Non-Volatile Systems Lab at UCSD).


.
"SSDs, have an identity crisis. They are clearly not drives in the conventional sense of slow, spinning hard disk drives. Yet SSDs look like drives to system designers because they retain legacy disk interfaces and rely on disk protection schemes designed in the 1980s specifically for high-capacity, slow I/O mechanical disks."
Going Beyond SSD: The Fusion Software Defined Flash Memory Approach - is a paper by Fusion-io. It was probably written a long time ago - but I only saw it recently.


.
"The 'radical' innovation in the host-attached flash storage marketplace today comes from products that not only access flash through a PCIe connection, but also bypass storage protocols to drive new levels of performance and enable new functionality not previously imagined. To achieve this, existing technologies must be left behind..."
Innovation and FlashMAX with Virident's vFAS (July 2012) - blog by Jeff Sosa Director of Product Management, Virident Systems


.
"Up until now, volatile DRAM was the only media that could communicate directly with the microprocessors memory controller. This partnership (between SMART and Diablo) allows us to connect Flash as persistent storage directly to the memory controller."
SMART's blog - re Diablo partnership and New Era of Flash Storage (April 25, 2013) - blog by John Scaramuzzo, President, SMART Storage Systems


.
Have you ever wondered how the amount of flash inside an SSD compares to the usable capacity shown on the invoice? And what is it used for?..
usable flash SSD capacity - the iceberg syndrome


.
"The risk of data corruption from power cycling isn't a random, unforseeable event. It's a direct result of choices made (or not made) when that SSD was designed... "
Surviving SSD sudden power loss


.
"...the SSD market will be bigger in revenue than the hard drive market ever was."
How will hard drives fare in an SSD world?


.
"Knowing the memory type in the SSD doesn't tell you anything useful about the SSD's likely characteristics and limitations any more... "
Recent Strategic Transitions in SSD


.
"despite the bewildering range of products in the market - the performance characteristics and limitations of ALL flash SSDs are determined by a small set of of architectural parameters."
a toolkit for understanding flash SSD performance characteristics and limitations