the SSD Buyers Guide - click to see article
SSD buyers guide
click to see the top 10 SSD oems
top 10 SSD oems ..
hard disk drives
hard disk drives ..

enterprise buyers guides since 1991

storage search
"leading the way to the new storage frontier"

Has Infiniband Established Itself in the Market?

article by Engenio
November 28, 2005

by Steve Gardner, Director, Product Marketing Engenio

Editor's intro:- This article looks at the state of the Infiniband market at the end of 2005. After 5 years stirring in the emerging market cauldron the Infiniband market hasn't turned out to be the popular flavor which was originally anticipated. But it's finally starting to get served up in some important markets. An Infiniband port now costs half as much as a fibre-channel port and delivers many times the performance rate. According to the author, Infiniband is now ready to take its place on the mainstream technology menu.
click to learn more about Fusion-io's SSD products PCIe, Infiniband etc -  and company
Not much has been heard for a while from high-speed interconnect Infiniband. Yet this technology is very much alive, it has firmly established itself in the sector of high-performance computing and is now venturing into the mainstream.

2000 was an exciting year for the IT industry. The boom phase of the internet was in full swing, dot-com firms catapulted themselves in a few months from unknowns to market movers.

A new high-speed interconnect called Infiniband was announced that could replace all other rapid internal and external system interconnects, yet went largely unnoticed. Yet in contrast to most of the technologies and companies that boomed in the dot-com phase, Infiniband has single-mindedly worked its way forward to the present and is in excellent health, even if some of its previous claims have been scaled back.

Infiniband was presented at the time as the alternative to practically all high-speed interconnects. Today it is becoming widely deployed concentrating on rapid external interconnects between servers and - to a lesser extent - storage systems. Here Infiniband can make full use of its advantages, such as extremely low latency periods, simple implementation and low costs.

Rough start, robust growth

The Infiniband standard has had to overcome several difficult periods in its short history. It was developed in 1999 as a fusion of two competing procedures: Next Generation I/O and Future I/O. The first products went under during the post-internet bubble era, in which firms had higher priorities than to convert to a new high-speed interconnect. Furthermore, several major manufacturers such as Intel and Microsoft 2002 revised their plans for Infiniband support and put their own products on hold for a while.

Andrey Semin, Technical Program Manager for HPC EMEA at Intel, explains this lack of a substantial market for busses at the time: "Intel was concentrating on its core competencies then, and had cancelled its activities in other fields. But we were, and still are, strong supporters of an Infiniband standard."

Thus, although Intel put aside plans for its own products, the chip giant continued its efforts on another level. Many small start-ups which had emerged in the area of Infiniband technology, worked intensively together with Intel on this development, among them Infinicon (now Silverstream Technologies), Mellanox and Voltaire.

The result: as the hype about the technology receded, a whole series of vendors had developed marketable solutions. In the meantime, the InfiniBand Trade Association (IBTA) has grown to more than 70 members. An effective steering committee ensures quick decisions concerning standards are taken. So for example, extensions were decided on in autumn 2004, whose results will have an impact far into the future. Transmission rates of up to 60 gigabits per second will then be possible. At the moment Infiniband is already firing 10 gigabits per second down cable lines.

Rapid, inexpensive, low latency periods

At present, Infiniband beats almost all other connections relating to current and planned transmission rates. The technology achieves such high speeds on the basis of its architecture. It sets up virtual channels, which are similar to the mainframes of yester year, and transmits information on the basis of parallel individual messages.

An Infiniband system, consisting of host-channel adapters (HCA) in the servers, target-channel adapters (TCAs) in memory systems or gateways and the connecting switches is called "fabric", as with Fibre Channel technology. The (HCA) constitutes the interface between the internal bus in the server and the external bus fabric in the same way as a network card.

The original idea was to integrate Infiniband chips inside the servers as well and thus be able to send data at maximum speed between the central memory, processor and external devices without converting it or requiring an additional card.

At present however, the task of intermediary between the internal bus and external interconnect is almost always carried out by a plug-in card with PCI-X or PCI Express bus. However, blade servers are making noticeable efforts to integrate Infiniband chips directly into the boards and reduce cost and complexity. It is thus clear that the host need not be a complete computer in the usual sense of the term.

With Infiniband, a processor can be coupled with a central memory and rudimentary extensions together with other, identical elements to build a functioning server system. Such an end node can communicate with every other node in the same subnet. A subnet manager configures the adapter with local addresses for every individual port. What is missing in each individual device, namely the I/O interfaces, is provided for by the Infiniband fabric.

The connection through switches is the foundation of the fabric. Physically, Infiniband can be routed through a whole series of interfaces, including copper and optical cables. The advantages of this distribution of computing power in individual nodes and a common I/O network are obvious: scalability and redundancy. Greater computing performance can be attained by adding further processor nodes, and guarantees against breakdowns by the distribution of active processes to other nodes. Small wonder then that above all users of high-performance computing (HPC), where it is a question of parallel processing and maximal performance, are currently among the most enthusiastic users of Infiniband and large-scale clusters.

While you still can't buy clustered systems based on Infiniband at the local retailer, the last three years have seen cluster products developed which are quite stable and which can be implemented with little effort. Linux first and foremost has carved out a portion of this segment for itself based on its early support of Infiniband in core systems. Sun Microsystems announced a while ago that its Utility Computing Grid will be based on Infiniband. IBM has presented a solution for its blade center together with Topspin and SGI, one of the pioneers and most important participants in the HPC market, has also integrated Infiniband in its Altix servers.

Advantages and applications

Examining the technical features and the Infiniband concept, it's advantages all the way. Separating the I/O interfaces from the processor is music to the ears of server developers and computing centre providers. Fewer parallel bus systems in the server mean less clutter, less electricity consumption and, naturally, greater concentrations of devices with, simultaneously, less need to cool them. Furthermore since I/O use normally comes in waves, the busses inside the servers often function with reduced workloads. Infiniband addresses this problem, since all I/O interfaces in the fabric are accessible to all processing nodes and can be dynamically distributed. This sounds ideal for blade servers, and manufacturers such as IBM are already building Infiniband into their blade systems.

This is good news for Infiniband manufacturers too, since the growth rates for blades are impressive. IDC speaks of a growth rate of 44% in units sold between the third quarters of 2003 and 2004. When its lower prices are taken into account, Infiniband should actually have replaced Fibre Channel as a storage interconnect already. Today an Infiniband port costs half as much as a fibre-channel port, with double or even quintuple the transmission rate. Gigabit Ethernet would be cheaper, but in return attains only a tenth of Infiniband's speed, and 10GB-Ethernet is significantly more expensive, leaving aside the fact that its standardisation has hardly been completed.

Yet on the one hand, Infiniband was brought too late onto the market. Fibre Channel had already established itself, and these investments had to be amortised. On the other hand, firms in sector are conservative when it comes to introducing new infrastructures. Without a thorough battery of tests and guaranteed availability, not only of a technical, but also of a logistical sort, no purchasing decision will occur. Research institutes are more open in this regard and more likely to agree to a trial run, which explains the high interest in Infiniband in the academic HPC sector.

The analyst William Hurley goes along with this: "I don't expect that in near future Infiniband is going to make a breakthrough among firms as a backplane network." He is however convinced: "The principle makes sense, the prices are competitive and the effort for implementation minimal."

This is also reflected in then number of ports sold: in 2004 a total of some 500,000 Infiniband ports were marketed, the same figure for fibre channel at around 20 million.

Engenio: Integrated storage and Infiniband

One problem which has often come up in Infiniband fabrics up to now has been connecting them to mass storage devices. A gateway, which functions as the interface between Infiniband and the fibre-channel or SCSI, almost always has to be integrated. Performance suffers as a result of the conversion, and what is more, a broader storage network has to be available. Engenio has now shown that Infiniband can not only function as a server interconnect, but also represents a very good medium for connecting data storage.

Connecting native IB storage not only makes sense on grounds of performance, but also reduces costs and administrative effort. A native hook-up of storage systems into Infiniband fabrics could enable the Infiniband standard to move out of the smaller HPC environment. The analysts at IDC think so too. A recent IDC survey confirmed that there is growing interest in Infiniband in different branches. The use of Infiniband in a great many current supercomputer installations and growing support from manufacturers should thus help to convince users of the usefulness and feasibility of employing Infiniband. ...Engenio profile

Here are some other popular articles which you may be interested in reading:-

click for more info

Infiniband on
Megabyte was going to catapult himself
into faster motion using infinitely strong
rubber band technology.

the RamSan-440 is a 4U RAM SSD delivering 600,000 random IOPS - click for more info
RamSan-440 Enterprise Solid State Disk
512GB RAM SSD, 600,000 IOPS
from Texas Memory Systems
Squeak! - Why are Most Analysts Wrong About Solid State Disks?
read the article - Why are Most Analysts  Wrong About Solid State Disks?
Most analysts and editors of other computer publications don't really understand the solid state disk market. They show their ignorance and naivete by prefacing every discussion of SSDs with a superficial analysis which compares the cost per byte of storage between flash and hard disk drives. That's the wrong answer to the wrong question. And it's far removed from why the SSD market is racing to become a multi billion dollar market seemingly in blithe ignorance of the cost per byte proposition.

This new article tells you what's important to users and the main applications in which SSDs are already being used and new applications where they will be used in the next 3 years. the article, Solid State Disks
read article by Plasmon the Impact of Compliance  on Archival Storage Strategies
the Impact of Compliance on Archival Storage Strategies - article by Plasmon

It's difficult enough protecting and archiving your data so that it's available to the right people at the right time (and cost). But now that's only part of the problem. With so many new rules and regulations which prescribe how you should destroy data records at the appropriate time - how do you guarantee that they stay deleted?

Archiving data on the wrong kind of media could mean you run the risk of breaking the law. Advances in the data recovery industry, and the future cohabitation of storage search-engines both mean that Compliance Officers have to pay much more attention to the ways in which data is dispersed and disposed of in different types of media.

This article summarizes the strengths and weaknesses of currently available market technologies. ... read the article, ...Plasmon profile, Optical Libraries
What is Data Recovery? - article by  Disklabs
What is Data Recovery? - article by Disklabs

The first time you try any new technology service it can be a bit daunting. Who do you choose? What if it goes wrong? The pressure really turns up when you are in the situation of having lost your data due to a backup failure, hardware fault, operator error or physical disaster. That's usually the first time you look at the data recovery market. and it's when you're in a high state of anxiety and can't take the risk of anything else going wrong. This article is a simple guide to how the data recovery industry works and how it can help you in this frightening situation. the article,...Disklabs profile, Data Recovery, backup software, Hard disk drives
article:-  Virtual Tape: Can You Afford to Ignore It?
Virtual Tape: Can You Afford to Ignore It? - article by MaXXan Systems

Network connected disk to disk backup systems for the enterprise have come a long way since the first pioneering products started to appear in the pages of in the late 1990s.

Some of the growing sophistication in the market can be seen by the way that the marketing terminology has morphed from the early D2d (let's kill tape backup), via D2D2T (let's be friends with tape / peaceful coexistence) to the current VTL (Virtual Tape Library - let's just see if they notice that it's more reliable and works faster - and don't tell them that there isn't a tape in the box) type of approaches.

But if you think that speed, reliability and cost are the only things you need to know about the "virtual" versus "real" tape library argument - take a look at this comprehensive article from MaXXan Systems which shows there are a lot more benefits than that. the article, ...MaXXan profile, Disk to disk backup
article by Engenio
Disk to Disk Backup versus Tape - War or Truce? - article by Engenio

Will disk to disk backup make tape backup obsolete? That's a question that's been debated hotly here on STORAGEsearch for many years.

At the extreme polarized ends of the argument are tape media makers like Sony, who in an article here made a case for the long term survival of tape, and at the other end of the argument are disk to disk supporters like STORAGEsearch whose editorial view has been that tape doesn't have a viable role the midsize market any more. In the middle of this argument are the moderates who say that maybe tape and D2d can co-exist. This article by Steve Gardner at Engenio takes the middle course line - and says why he thinks there's still a place for both. See if you agree. the article, ...Engenio profile, RAID systems, Disk to disk backup
click to read article by Xtore
NAS, DAS or SAN? - Choosing the Right Storage Technology for Your Organization - article by Xtore

It's 5 years since we published the Storage Architecture Guide a classic reference written by the world's first network storage company Auspex. The new overview article from Xtore places the main storage connection strategies in a current context. Here's an extract.

"Another important consideration for a medium sized business or large enterprise is heterogeneous data sharing. With DAS, each server is running its own operating platform, so there is no common storage in an environment that may include a mix of Windows, Mac and Linux workstations. NAS systems can integrate into any environment and serve files across all operating platforms. On the network, a NAS system appears like a native file server to each of its different clients. That means that files are saved on the NAS system, as well as retrieved from the NAS system, in their native file formats. NAS is also based on industry standard network protocols such as TCP/IP, FC and CIFS. " ... read the article, ...Xtore profile

storage search banner

STORAGEsearch storage manufacturers SAN History Solid state disks Flash Memory Backup software
STORAGEsearch is published by ACSL