click to visit home page
leading the way to the new storage frontier .....
the fastest SSDs - click to read article
the fastest SSDs
Rackmount SSDs click for news and directory
rackmount SSDs ..
hybrid DIMMs
hybrid DIMMs ..
image shows megabyte waving the winners trophy - there are over 200 SSD oems - which ones matter? - click to read article
top SSD companies ..
SSDs over 163  current & past oems profiled
SSD news ..

InfiniBand - storage and SSDs

after AFAs - what's the next box?
how fast can your SSD run backwards?
latency loving reasons for fading out DRAM
where are we heading with memory intensive systems?
miscellaneous consequences of the 2017 memory shortages
Controllernomics and risk reward ratios with big memory "flash as RAM"
Gen-Z 1.0 now available to view

Editor:- February 13, 2018 - The Gen-Z Consortium today announced that the Gen-Z Core Specification 1.0 is publicly available on its website. Its memory media independence and high bandwidth coupled with low latency is intended to enable advanced workloads and technologies for end-to-end secure connectivity from node level to rack scale.

Editor's comments:- Gen-Z follows in the multidecade footsteps of Infiniband and then recently PCIe memory fabrics - but Gen-Z was born with with confident expectations that because of SCM DIMM wars andthe memoryfication of the enterprise there is a real business potential that memory boxes could become the mainstream instead of sitting in the side lines of HPC.

The rev 1.0 specification is a 980 plus page pdf - which I haven't read through yet - but which shows definite promise of a being a thought provoking page turner.

From what I've seen so far the thinking looks like a modern, secure and media agnostic way to request or respond to byte and block addressable (or addressed by default as the next component) data packets between a huge scalable population of components which can be memory devices, processors or controllers.

U.S. Department of Veteran's Affairs uses RAID Inc. in 5PB SDS

Editor:- December 7, 2016 - A customer news story from RAID Inc. - Department of Veterans Affairs Selects RAID Inc. for Multi-Petabyte Microsoft Storage Spaces Deployment - includes the useful ratio metric that "400TB of flash-accelerated hot tier storage" were used to support 5PB of cold tier data in an InfiniBand attached fault tolerant array which supports over 460,000 devices in more than 280 sites.

Nimbus re-emerges from stealth with 1PB / U raw HA SSD

Editor:- August 9, 2016 - Nimbus Data Systems has emerged from its self imposed exit into marcomms stealth mode with the announcement of a new range of Ethernet/FC/Infiniband attached rackmount SSDs based on its new ExaFlash OS with GA in Q4 2016.

Entry level products start in a 2U box with 50TB raw capacity for under $50K and for larger configurations Nimbus says its ExaFlash offers an effective price point as low as $0.19 per effective gigabyte (including all software and hardware).

Higher density boxes in this product line - D-series models - will have 4.5 PB raw capacity in 4U (12 PB effective).

AccelStor adds 16G FC to 1U AFA range

Editor:- May 19, 2016 - AccelStor today announced its has added dual port 16Gbps FC connectivity to its NeoSapphire 1U rackmount SSD product line which was previously available with InfiniBand or 10GbE iSCSI.

Themis ships rugged mobile datacenter platform

Editor:- November 17, 2015 - Themis Computer today announced immediate availability of a rugged Infiniband connected rackmount SSD system - called Hyper-Unity which integrates scalable SDS architecture software from Atlantis Computing with 4 ruggedized, 8 hot pluggable SSD drive, RES-XR5-1U rack mounted servers (MIL-STD-461), with a high-speed, 12-port Mellanox Infiniband switch to run virtualized applications for rugged mobile datacenter applications.

HGST's IB fabric leverages PCM

Editor:- August 11, 2015 - The 3 strongest contenders for ultra-low latency rack to rack memory fabric have been FC SAN (traditionally popular in traditional business sites), InfiniBand (popular in HPC and research sites) and (emerging recently in product form since 2014 - due to the widespread penetration of PCIe SSDs) - PCIe fabric.

Until recently the memories in these solutions were predominantly mainstream RAM or flash or combinations.

Now after more than a decade of crying wolf by alternative non volatile memories - there are indications from several SSD companies that the gaps in the market represented by the application spectrum (latency combined with capavity and cost) are seen as big enough business opportunities to justify the introduction of new memory types.

Fitting into this category - a thoughtful preview article re new PCM IB fabric from HGST is discussed in this recent article - HGST To Display PCM Fabric at FMS 2015 on Tom's IT Pro

Diablo's MCS supported by more slots in new Lenovo servers

Editor:- May 6, 2015 - Diablo Technologies today announced that Lenovo's new 8U x3950 X6 servers now support upto 32 eXFlash DIMMs (memory channel SSDs) per system.
SSD ad - click for more info
A3CUBE shows shape of R/W in remote shared memory fabric

Editor:- April 14, 2015 - There was a disproportionately high amount of reader interest in A3CUBE in 2014.

A3CUBE was one of those rare companies which entered the Top SSD Companies list within a single quarter of exiting stealth mode or launching their first product. At that time they hadn't shipped any production products so we had to make some guesses about how the architecture would work with different R/W demands.

R/W performance of 4 node remote PCIe shared memoryWith any remote memory caching system there are always some types of R/W activities which work better than others and now we can get an idea of the headroom in A3CUBE's remote PCIe shared memory from a new slidedeck released by the company (Fortissimo Foundation - all NVMe solution some benchmarks) which is based on a 4 server node configuration.

In this 13 slide presentation - the most interesting for me was #12 - which shows random writes. A3CUBE says "This test measures the performance of writing a file with accesses being made to random locations within the file."

The throughput range is typically 700MB/s to 8GB/s. The low end is more impressive than it first appears - when you consider that it's a 4KB record changed within a remote 64KB file. ...see the presentation

You don't need to worry about the endurance of our FlashSystems - says IBM

Editor:- October 7, 2014 - Worried about endurance?

"None of the thousands of FlashSystem products (fast rackmount SSDs) which IBM has shipped has ever worn out yet! - says Erik Eyberg, Flash Strategy & Business Development at IBM - in his new blog - Flash storage reliability: Aligning technology and marketing. "And our metrics suggest that will remain true in almost all cases for many, many years (certainly well beyond any normal and expected data center life cycle)"

Erik goes on to explain that's the reason IBM can now officially cover flash storage media wear-out as part of its standard IBM FlashSystem warranty and maintenance policies - without changing the prices for these services.

And his blog has a link to a white paper about the reliability architecture underlying this product (although it's behind a sign-up wall - which seems counter productive to me.)

Editor's comments:- Don't expect all other flash array vendors to follow suit (with no cost endurance guarantees) - because this product range from IBM is based on design rules and memory reliability architectures experience in FC SAN compatible enterprise SSD racks which have evolved since the 1st generation RamSan from TMS (in 2000). And for more than a decade before that using other popular enterprise storage interfaces.

Holly Frost - who founded Texas Memory Systems - and who was the CEO when TMS was acquired - told me a revealing story about TMS's policies concerning the reliability of their SSD systems and customer care procedures.

This conversation took place in December 2011 - when the company was launching its first high availability SSD - which became the basis of IBM's FlashSystem.

It still makes interesting reading today. You can see it in this article - in the right hand column - scroll down to the box titled - "no single point of failure - except..."

See also:- high availability enterprise SSDs

Violin is brand leader in InfiniBand SSDs

Editor:- July 17, 2014 - Violin Memory was voted the SSD brand leader in InfiniBand SSDs in a new report published today by IT Brand Pulse. See also:- SSD market analysts, Can you trust SSD market data?

OCZ's SAS SSDs in InfiniBand benchmark configuration

Editor:- June 12, 2013 - Mellanox today announced details of a benchmark demonstration it did this week showing its FDR 56Gb/s InfiniBand running on Windows Server 2012 in a system which uses OCZ's Talos 2R SSDs (2.5" SAS SSDs) working with LSI's Fast Path I/O acceleration software and RAID controllers - getting over 10GB/s throughput to a remote file system while consuming under 5% of CPU overhead.

Whiptail expands ambitions up and down the SSD needs spectrum

Editor:- June 7, 2013 - Whiptail recently announced imminent availability of its Infiniband based INFINITY high end controller architecture. A 30 node INFINITY cluster - consisting 6 x 72TB INVICTAs - can provide a 360TB, 4 million IOPS, 40GB/s system.

The company also announced details of a new entry level ($20K floor price) fast-enough 1U iSCSI SSD - which is aimed at the branch office environment. The WT-1100 offers 100K IOPS with upto 4TB capacity and will be available via resellers and systems integrators.

sharpen your SSD R/W grid latency weapons to 5µS

Editor:- May 9, 2012 - Kove has published some new record latency numbers for its fast RAM SSD - the XPD L2 - which has achieved continuous and sustained 5 microsecond random storage read and write when connected via 40Gb/s InfiniBand adapters from Mellanox .

Kove's system has good R/W symmetry which the company says - is not subject to periodic performance jitter or "periodicity". Even under constantly changing disk utilization, it delivers uniform, predictable, and deterministic performance.

"The Kove XPD L2... allows high performance applications to use storage as a weapon rather than accept it as a handicap," said Kove's CEO, John Overton. "We are pleased to set a new bar height for storage latency."

how fast can your SSD run backwards?

Editor:- April 20, 2012 - today published a new article which looks at the 11 key symmetries in SSD design.

Whether you're a buyer, designer, marketer or investor in SSDs - this new conceptual framework will help you to comparatively evaluate any SSD compared to competitive offerings. It's helpful whether you're looking at raw SSD IP and controller chips right up to the most complex datacenter SSD racks.

SSDs are complex devices and there's a lot of mysterious behavior which isn't fully revealed by benchmarks and vendor's product datasheets and whitepapers. Underlying all the important aspects of SSD behavior are asymmetries which arise from the intrinsic technologies and architecture inside the SSD.

Which symmetries are most important in an SSD? - That depends on your application. But knowing that these symmetries exist, what they are, and judging how your selected SSD compares will give you new insights into SSD performance, cost and reliability.

There's no such thing as - the perfect SSD - existing in the market today - but the SSD symmetry list helps you to understand where any SSD in any memory technology stands relative to the ideal. And it explains why deviations from the ideal can matter.

The new article unifies all SSD architectures and technologies in a simple to understand way. Now that I've spent 20 years thinking about it - it all seems really obvious now. This is the most important article about SSDs that I've written in the past few years. I couldn't have written it before. I hope you like it. to read the article

PS - I'll be using the new symmetry terminology from now on in news stories and in SSD company profiles - so you'll soon get used to it - just as you've got used to lots of other SSD jargon.

Intel buys InfiniBand line from QLogic

Editor:- January 24, 2012 - Intel yesterday announced an agreement to acquire the InfiniBand 40Gbps (pdf) related product lines, IP and business assets of QLogic.

Editor's comments:- if you're not familiar with InfiniBand - it was originally proposed in 2000 as a standard for remote CPU R/W with small packet sizes and ultra low latency to support arrays of CPUs over many cards and racks. In the early days - InfiniBand evangelists and some storage analysts believed the standard would go into the commercial server mainstream.

Instead what happened was that fatter multi-core CPU chips, and faster GbE wiped out the volume market need for IB technology - because they could do the same job cheaper and incrementally for smaller clusters of CPUs. So the IB market nowadays is mainly a niche market for scientific research and high performance computing.

Some of the fastest SSD benchmarks have been recorded in IB environments. And at one time (before 2008) I thought that IB might be a significant and natural upward path for high performance SSDs. However, PCIe SSD systems also support remote array connections - so IB's role remains that of occupying the narrow turf of clustering hundreds to thousands more CPUs than Intel or others can pack into a single chip.

Another way to think about it is this. You can't have viable HPC without SSD. But you can have a healthy SSD market where HPC is a small niche.

There's no doubt that SSDs are an enabling technology which make it realistic for CPU designers to think about what they could do with hundreds of cores on a single chip and over 1,000 cores on a single server card. I discussed that blue sky concept with processor designers nearly 10 years ago. But does the mainstream market need such servers?

In the SSD data driven factories of the future - the answer is yes. But that could be another 5 years in the future - because there are still closely related standards to firm up - such as Hybrid Memory Cubes. And storage history shows that new standards take years to get into the market. In the meantime - if you're not in the HPC market - but still need very fast CPU performance - keep an eye on what the leading PCIe SSD makers do - and you won't go far wrong.

SSD system beats $ / GB in record setting benchmark

Editor:- May 10, 2011 - Texas Memory Systems today announced record-setting audited SPC-1 and SPC-2 performance results produced by its flagship rackmount flash SSD - the RamSan-630.

One of the new results includes a first appearance of an InfiniBand product in SPC-2 category. Whereas the other is for a fibre-channel system.

Editor's comments:- as I said in an article 3 years ago - results which simply show that an SSD is faster than a traditional over provisioned hard disk system rarely deserve news mention status - no matter how high the ratio.

What TMS says is different about these results - is that its SLC flash SSD system also cost less per gigabyte than the previous record holder's HDD based system.

Kove launches 20 Gigabytes/s RAM SSD

Editor:- November 22, 2010 - Kove recently demonstrated a 4U InfiniBand & FC compatible terabyte class RAM SSD product line (with under 25 µS latency) called Xpress Disk - which can sustain 20GB/s throughput via 6x InfiniBand ports and 600,000 read IOPS and 500,000 write IOPS.

Editor's comments:- despite costing an order of magnitude more - the market for RAM SSDs hasn't been killed by flash.

On the contrary - all the vendors of high end RAM SSDs that I've spoken to in the past year say they have been pleasantly suprised to see demand for this type of product growing. The reason? - When a bunch of flash SSD accelerated servers hits a storage performance bottleneck - the only way to go faster is to interpose RAM SSDs. And unlike the old days when the first terabyte RAM SSDs became commercially available (2003) - users today are already amenable to the concept of SSD acceleration.

SSDs ensure smooth running in Penguin's new IceBreaker

Editor:- November 15, 2010 - Penguin Computing recently launched a new family of fast 160TB DAS RAID storage systems which the company says provides the performance and scalability of 40Gbps InfiniBand environments.

Called IceBreaker - it's designed for high workflow apps like scientific modeling - which don't need the "frills" (and cost) associated with similar capacity SAN systems.

Editor's comments:- I asked the question - does solid state storage feature in any significant way in the new product architecture? If so - at what level and managed how? To see what they said - see Penguin's profile page

QLogic reports new InfiniBand switch record

Editor:- November 9, 2010 - QLogic announced that its InfiniBand adapters and switches have achieved cluster message rate performance of over 86 million messages per second - a new record.

The system configuration was a 14 node cluster with 2 switches.

InfiniBand option doubles speed of RamSan-630 (3U SLC SSD)

Editor:- October 4, 2010 -Texas Memory Systems today announced a Quad Data Rate InfiniBand card interface option for its RamSan-630 3U SSD - which effectively doubles the IOPS to 1 Million (compared to the performance previously obtainable with the fibre-channel) connection option.

Fusion-io SSDs deployed in nuclear safety program

Editor:- June 15, 2010 - today it was announced that Lawrence Livermore National Laboratory has deployed over 100TB of Fusion-io's dual 320GB enterprise MLC ioMemory modules deployed in ioSAN carrier cards that connect over Infiniband in a testbed project to develop high-performance simulation computing capabilities needed to ensure the safety, security and reliability of the nation's aging nuclear deterrent without underground testing.

With the addition of Fusion-Powered I/O, the Hyperion testbed will deliver over 40 million IOPS and 320GB/s of bandwidth from just 80 1U appliances (2 racks) compared to 43 racks and more power if implemented in an HDD based system.

"This project demonstrates how flash that performs like memory, rather than disk, can scale to the highest levels of performance," said Neil Carson, CTO of Fusion-io. "Equally important, it demonstrates that it is possible to accomplish this with a fraction of the infrastructure. Our customers are already capitalizing on the ioMemory's ability to let them do far more with far less, and Lawrence Livermore has just taken it to the next level."

TMS ships 10TB 500K IOPS 3U SLC SSD

Editor:- April 8, 2010 - Texas Memory Systems today announced the availability of the RamSan-630 an FC / InfiniBand ible 3U SLC SSD with 4 to 10TB capacity, 500,000 IOPS, 8GB/s bandwidth, and R/W latency of 250 / 80 microseconds in a 450W power budget.

Levi Norman, Director of Marketing and OEM for Texas Memory Systems explained the rationale behind the new product - "We developed it in response to observing how customers were struggling to boost performance without adding to their data center footprint. The explosive growth in IT and storage over the years is resulting in many data centers reaching their limits for space and power draw."

QLogic Ships 40Gbps InfiniBand Switches

Editor:- June 8, 2009 - QLogic today announced general availability of its 12000 Series 40Gb/sec QDR InfiniBand switches.

"With 864 ports and bandwidth of 51 terabits per second, the QLogic 12000 series is the highest capacity general purpose QDR InfiniBand switch on the market today," said David Vellante, co-founder of "The huge bandwidth of this solution brings HPC customers better scale-out performance, lower latencies, simpler management and reduced costs."

See also:- SAN switches, InfiniBand

New Module Aims at "must-have 100 terabytes SSD" Users

Editor:- April 21, 2009 - Texas Memory Systems announced the RamSan-620 - a 2U rackmount SLC Flash SSD with 2TB to 5TB capacity and 2 to 8 FC or InfiniBand ports.

Throughput is 3GB/s. R/W latency is 250µS and 80µS respectively. Transactional performance is 250,000 random IOPS. Power consumption is 325W. Multiple RamSan-620s can scale to higher capacities. Upto 100TB can fit in a single 40U rack.

"The IT community is looking for ways to increase storage efficiency while boosting productivity," said Greg Schulz, founding analyst at StorageIO and author of "The Green and Virtual Data Center".. "It is time to stop moving around I/O or other bottlenecks and start enabling storage efficiency via performance optimised storage that does more work, in a smaller footprint (power, cooling, floor-space, economic) while boosting productivity. Anyone can attach flash SSD to a computer or storage system; however the real trick and business benefit is when a storage system or applications server can fully utilise the technology without introduction of, or moving I/O and performance bottlenecks elsewhere. The RamSan-620 is an example of a new breed of storage solutions that have been optimised to leverage the capabilities of flash SSD while preserving application QoS and service level objectives."

Editor's comments:- there has been a lot of debate in the fastest lanes of the SSD accelerator market about whether it's better for users to deploy this technology inside the server box (as PCIe cards) - or outside the box (on the SAN). This is reminiscent of the old CISC vs RISC processor debates of the mid 1980s.

Entertaining as it is to analyze these polarized approaches I explained in my 2009 - Year of SSD Market Confusion and rackmount SSD articles why I believe that users will, in fact, do both.

Texas Memory Systems has in the past told me, that whenever they launch a new rackmount SSD they have some customers who just fill up a complete cabinet with the new model and use that as their basic unit of solid state storage until the next new model comes around. They'll only need 6.5kW for the 100TB SSD enabled by this model - and they'll get the transactional performance of 10,000 hard drives.

Fusion-io Unveils InfiniBand flash SSD

San Diego, Calif. - September 8, 2008 - Fusion-io unveiled the ioSAN - a 10GbE or Infiniband attached flash SSD on PCIe form factor.

Using a standards-based, memory-speed protocol over either 10GigE or 40GBps QDR InfiniBand, the ioSAN shares ioMemory capacity between servers. With latencies of less than 2 microseconds, the ioSAN incorporates an integrated network interface that can dynamically alternate between 10Gb/s Ethernet or 40Gb/s quad data rate InfiniBand. The built-in network interface makes it easy to create networked storage across servers with increased performance and flexibility, and with zero footprint. This networked storage is extremely easy to integrate and manage within existing server infrastructure. Prior to general availability of the ioSAN, Fusion-io is inviting innovative and visionary companies to join them as they launch their 3rd party development program at the beginning of next year.

"With this development, everything you thought you knew about SSD and storage networking is no longer true," said David Flynn, CTO of Fusion-io. "The ioSAN fuses SSD with storage networking, combining the best of direct-attached and storage networking with the best of SSD and traditional storage. With this revolutionary advancement, Fusion-io has commoditized high-performance network storage in the same way that companies like NVIDIA and ATI commoditized high-performance graphics processing. Fantastic applications of this technology are now beginning to emerge." ...Fusion-io profile, PCIe SSDs

Editor's comments:-
Fusion-io's full press release text says this is "the world's first networked enterprise SSD."

That's not strictly correct because fiber-channel SSDs have been available from many oems for over a decade. And it's not the first "ethernet NAS SSD" either. We ran an ad here in 2002 for the NAS-168F from IEI.

And it's not the first "enterprise NAS SSD". In April 2008 Nimbus Data Systems announced an SSD acceleration option for its Breeze Hybrid series of multi-protocol 10GbE IP storage systems.

And it's not the first InfiniBand SSD either. That was Texas Memory Systems in 2005.

What is special about Fusion-io's new product is that it could become the first high performance native NAS SSD to cross the chasm between PR-ware and success in the market. Although high speed NAS SSDs have been announced before on these storage news pages by various companies in years gone by none of them left a deep impression in the market. The reason? - the benefits of SSD acceleration were not cost effective when channeled through slow ethernet networks. The low cost of Fusion-io's flash array coupled with its high speed and 10GbE interface could place this product in new territory in the SSD roadmap.

DataDirect Networks Expands Japanese Operations

TOKYO - July 1, 2008 - DataDirect Networks, Inc. today announced it is investing 0.5 billion yen (approx $5 million) to expand its sales and support operations throughout Japan.

This investment will fuel the company's expansion in these key sectors and enable it to accelerate its growth trajectory in the world's 2nd largest economy.

Analog TV broadcasting will end in Japan in July 2011, and the Association for the Promotion of Digital Broadcasting Japan recently announced that as of March, more than 32 million digital terrestrial television sets are already in use. With the transition to 100% digital environments, DataDirect Networks Japan is partnering with major broadcasters, post production and digital intermediate facilities to ensure a seamless transition to all digital content. ...DataDirect Networks profile, Storage Resellers in Japan

LSI and QLogic Cross Certify InfiniBand Products

MILPITAS, Calif - February 12, 2008 - QLogic Corp. and LSI Corp today announced that the 2 companies will be working together to cross-certify their InfiniBand based products.

LSI is actively involved in the QLogic HPCtrack Program in which InfiniBand companies collaborate to optimize the performance of multi-vendor solutions. ...LSI profile, ...QLogic profile

DataDirect Scales to X00 Gigabytes/ Second

Chatsworth, Calif - November 14, 2007 - DataDirect Networks Inc. today announced that the company's 8th generation S2A appliance will enable CFS/SUN's Lustre I/O to reach several hundred gigabytes / second.

DataDirect Networks manufactures the fastest, highest capacity, most scalable Lustre-based storage systems on the planet. Lustre, taken from the words "Linux" and "cluster," is a distributed file system used for large scale cluster computing. The file system, which was acquired by Sun Microsystems in October, was designed to scale to tens of thousands of nodes that are attached to petabytes of storage. The main advantage of the file system is that it does not compromise a storage system's speed or security.

New features built into the latest S2A appliance include support for 8Gbps Fibre Channel and 20Gbps Infiniband DDR host connections. The system leverages the serial attached SCSI protocol to communicate to the drives it manages, providing a future-proof roadmap to the latest disk drive technologies, speeds and capacities. ...Data Direct Networks profile, SAN switches, Record Breaking Storage

Violin will Demo InfiniBand Memory Array at SC07

Iselin, NJ - October 24, 2007 - Violin Memory, Inc. will exhibit the Violin 1010 Memory Appliance at SC07 in Reno, NV next month.

The Violin1010 will be network attached to the SCinet InfiniBand network during the show. The network technologies and partners to enable this attachment will be announced at the tradeshow. ...Violin Memory profile, Storage Events

Voltaire Reports 194% Revenue Growth

HERZELIYA, Israel - August 8, 2007 - Voltaire Ltd. today announced financial results for the 3 month period ended June 30, 2007.

Revenue increased by 194% to $11.7 million compared to Q206.

"This has been another strong quarter for Voltaire" said Ronnie Kenneth, Chairman and CEO. "During the quarter we shipped our new 20Gbps switch platform and continued to penetrate growth-oriented vertical markets..." ...Voltaire profile

Mellanox Announces InfiniBand Market Milestone

editor- May 21, 2007 - Mellanox Technologies, Ltd. today announced that it has shipped over 2 million InfiniBand (10 and 20Gb/s) ports to OEMs.

More than 6 years after launched a dedicated directory page for InfiniBand in May 2001 most users have forgotten just how much this technology was once hyped.

For example IDC was once quoted as saying "75% of all servers shipped in 2004 will be shipped with InfiniBand connectivity." In fact that estimate was 100x too high. The need for InfiniBand functionality didn't go away - instead it was mostly implemented by on chip memory to memory access in multi-core processors. 10GbE also took a swipe out of the inter blade connection market. So today, InfiniBand is a small niche market instead of being the dominant technology which was once foretold.

Mellanox Demos 40Gb/s Connectivity Over Copper

SANTA CLARA, CA – April 16, 2007 – Mellanox Technologies, Ltd. announced the demonstration of a 40Gb/s InfiniBand server-to-server connection over a copper cable using advanced versions of the recently announced ConnectX IB Host Channel Adapter.

These are the first I/O adapters to support the increased performance of PCI Express 2.0 – a doubling of bus speeds expected to be integrated in server and storage platforms over the next year.

"This 40Gb/s demonstration is strong evidence that InfiniBand will continue to provide connectivity performance leadership for the industry's most demanding computing and storage applications, and it's software compatible with the InfiniBand installed base," said Shai Cohen, Mellanox's VP of operations and engineering at Technologies. ...Mellanox profile, Record Breaking Storage

InfiniBand Storage Powers Moviemakers

Los Angeles, CA - March 27, 2007 - DataDirect Networks announced today that Academy Award winning Ascent Media Creative Services has selected its technology (incorporated in SGI's RM660 and Infinite 6700) as the primary storage solution supporting 4K, 2K and HD post production and digital intermediate workflow.

SGI's RM660 and Infinite 6700 systems are based on DataDirect Networks' Silicon Storage Appliance technology and is an ideal storage solution for creative environments. The S2A technology, with up to 3GB/s of sustained real-time throughput and up to 960TB of storage capacity, can easily scale to handle the bandwidth and capacity intensive requirements needed in the digital intermediate space, empowering true real-time collaborative workflow in digital post-production. ...Data Direct Networks profile, ...SGI profile, SSDs in tv and movie markets
SSD ad - click for more info

storage search banner

InfiniB page
Megabyte knew how to go fast over a short distance
SSD ad - click for more info
Charting the Rise of the InfiniBand Market
This timeline tracks key InfiniBand events and market milestones, as they were recorded here on
  • June 2000 - first mention of InfiniBand. Hubert Yoshida, Vice President of Data Network Solutions at Hitachi Data Systems " ...As very large-capacity disk drives come to market for future generations of storage subsystems, these disk drives themselves may become bottlenecks to system performance... Adopting dual-ported Fibre Channel interface technology is the way to get faster back-end performance out of systems using the large disk drives... Infiniband standards, which are now being defined, bring the concept of switching into the processors to replace the shared PCI bus."
  • January 2001 - Intel announced it was shipping samples of its InfiniBand silicon and software products, critical components that will help implement the InfiniBand I/O industry specification and enable a key advance in Internet servers.
  • February 2001 - Mellanox Technologies introduced its InfiniBridge family of devices supporting the new InfiniBand architecture which includes Switches, Host Channel Adapters (HCAs), and Target Channel Adapters
  • February 2001 - first STORAGEsearch editor article mentioning InfiniBand. Re: Does a SAN need to include fibre-channel? - "To my way of thinking, circa 2001, "SAN" now encompasses the idea of any kind of significant storage system which can connect over some distance to a network, whether that connection is by fibre-channel, ethernet, the internet, or in the future InfiniBand... I think the SAN market today is like the PC market before Microsoft, and the networking market before Cisco. Nobody really knows who is going to create the standards which are going to be the winners."
  • April 2001 - DAFS - a new file access protocol, specifically designed to take advantage of standard memory-to-memory interconnect technologies such as InfiniBand in high-performance clustered data center environments - was 75% defined.
  • April 2001 - Paceline Systems became the first of many hopeful InfiniBand focused startups - and raised $20 million in financing from top-tier venture capitalists. "IDC predicts the InfiniBand switch port opportunity will exceed $1 billion by 2004."
  • May 2001 - STORAGEsearch started a dedicated directory for InfiniBand (this page). "Infiniband was one of the top 5 words or phrases searched using the on-site search-engine during the month preceding."
  • June 2001 - InfiniCon Systems published first of a series of educational articles about InfiniBand.
  • June 2001 - InfiniSwitch quoted a market projection "Research from IDC estimates that more than 75% of all servers shipped in 2004 will be shipped with InfiniBand connectivity." - Most of IDC's projections about storage interface technologies at that time were much too optimistic. This resulted in many startup companies going bust in later years - because the markets in InfiniBand (and iSCSI - the other big analyst hyped subject) weren't big enough to feed them.
  • August 2001 - Intel announced several interoperability demos including the world's largest InfiniBand fabric to date, which featured 24 vendors and over 100 InfiniBand devices.
  • October 2001 - InfiniCon Systems announced that it will work with IBM to integrate IBM's InfiniBand chip technology - known as InfiniBlue - into sharable I/O solutions to be offered by InfiniCon Systems.
  • February 2002 - Voltaire demonstrated its TCP Termination product architecture - which connected InfiniBand to existing IP networks
  • March 2002 - we started seeing the first of the InfiniBand startups go bust. The assets of Essential Communications were acquired by SBS Technologies

    Also in March 28, 2002 - Overland Data, Inc announced a strategic alliance with InfiniCon Systems to collaborate on InfiniBand-based solutions for Overland's Neo series tape libraries.
  • May 2002 - On its first anniversary - the InfiniBand directory on was ranked #24 out of the pages viewed by readers. Just below disk to disk backup.
  • July 2002 - view this InfiniBand page and news back in time.
  • September 2002 - InfiniCon Systems announced general availability of its InfinIO 7000 Shared I/O Systems - the first commercially available I/O system based on InfiniBand using 10Gbps channel speed.
  • May 2003 - On its second anniversary - the InfiniBand directory on was ranked #39 out of the pages viewed by readers. In the same month SATA was #1.
  • May 2003 - first vendor independent InfiniBand training workshops launched by CATC and Dashcourses.
  • August 2003 - InfiniCon Systems announced that its InfinIO family of products - leveraging InfiniBand architecture - using the industry-standard IP-over-InfiniBand (IPoIB) protocol, applications drove data in excess of 350MB/second for a single link, compared to an average of 65MB/second for a Gigabit Ethernet link.
  • November 2003 - SBS Technologies introduced the IB4X-PMC-2 Host Channel Adapter - the first InfiniBand HCA to be provided on a PCI mezzanine card (PMC).
  • February 2004 - Mellanox announced that more than 200,000 InfiniBand ports have been shipped to its customers.
  • March 2004 - Topspin Communications announced the industry's first intelligent Boot over InfiniBand solution.
  • May 2004 - On its third anniversary - the InfiniBand directory on was ranked #36 out of the pages viewed by readers. In the same month solid state disks was #1.

    Also in May 2004 wannabe InfiniBand startup Banderacom (which had exited the InfiniBand market due to its small size and poor prospects) renamed itself and re-emerged as NetEffect - with $22 million to pursue the high-performance, multi-gigabit Ethernet market.
  • June 2004 - IWILL selected Mellanox's InfiniHost host channel adapter to provide 10Gb/s Landed on Motherboard connections for the DK8S2-IB server platform - the world's first server with 10Gb/s network connectivity shipped standard with every system.
  • December 2004 - Infiniband was identified as one of the storage market "Dogs Which Didn't Bark" in's end of year review.
  • January 2005 -'s Solid State Disk Buyer Market Survey showed that InfiniBand came in 9th - cited by just 6% of SSD buyers in response to the question - "Which interfaces best suit your current or future SSD requirements?"
  • May 2005 - On its fourth anniversary - the InfiniBand directory on was ranked #31 out of the pages viewed by readers. In the same month NAS was #1.
  • June 2005 - Voltaire announced $15 million in further financing taking its total to $65 million investment. STORAGEsearch commented on InfiniBand's slower than predicted adoption in the market. "The original benefit of a faster storage connection has got more fuzzy as other technologies have also speeded up. It's also possible that new technologies like multi-core processors in which CPUs share access to the same memory bus inside the same chip have eroded some of the multiprocessing benefits which InfiniBand's proponents originally had in mind."
  • October 2005 - InfiniBand is 33rd most popular subject viewed by's readers.
  • June 2006 - Voltaire ships InfiniBand-based switching solution and software stack to support Microsoft Windows Compute Cluster Server 2003 (Microsoft's official new entry into the world of HPC).
  • October 2006 - InfiniBand is 28th most popular subject viewed by's readers.
  • November 2006 - InfiniBand (re)entered the top 20 storage searches by readers - at #18
More recent events in InfiniBand history can be seen in the left hand column of this page.

See also:- SSD market history and SSD news.
InfiniBand's 1st business decade

InfiniBand, first mentioned here on in June 2000, was a graveyard for many startups which came into being to support this technology in the first 3 to 4 years of this interface's market life.

The original idea behind InfiniBand was that it would offer an industry standard alternative to the many high speed proprietary busses which server manufacturers used to cluster their most powerful servers.

The technology delivers 10 to 40Gps remote RAM access with very low latency.

The server recession in 2001-2003 slowed down the pace of new server developments and provided a disincentive for manufacturers to end of life their most profitable products.

2 other factors have reduced the potential market size for InfiniBand.

(1) - The availability of processor chips with multiple central processing units on the same chip has reduced the need for motherboard to motherboard memory access of the type provided by a factor of 2, 4 or 8 - for different chip implementations.

(2) - The availability of 10Gbps Ethernet, and the imminent 20GbE, provides a workable alternative in many blade to blade and box to box interconnect applications which would have looked like natural slots for InfiniBand when it was first proposed.

How Big is the InfiniBand Market?

In September 2006 the InfiniBand Trade Association estimated that over 500 end-user sites had deployed InfiniBand products in production applications.

2010 perspective

IBTA says it expects the IB market in 2011 to be worth around $800 million (adapters plus switches).

In the datacenter of the future InfiniBand may be used commonly alongside PCIe SSDs. (Although some users are doing that already today - it's a rarity - and mostly occurs in scientific modeling applications - rather than commercial database processing).

To get the best results from the low latency - which InfiniBand offers for repeated remote writes - requires RAM SSDs or fat flash SSDs (which have large RAM caches). Without a RAM cache - much of the performance benefits of InfiniBand in write intensive apps would be wasted - or costly to achieve.
SSD ad - click for more info

the SSD Buyers Guide
Storage Architecture Guide
this way to the Petabyte SSD
What is the InfiniBand Ecosystem?
Cutting Through the Infiniband Buzz
The new way of looking at Enterprise SSDs
Has Infiniband Established Itself in the Market?
InfiniBand FAQs - 40Gbps / $800 million market
Centralizing Server I/O Infrastructure with InfiniBand
InfiniBand - editor mentions on
how fast can your SSD run backwards?
11 Key Symmetries in SSD design

STORAGEsearch is published by ACSL