 |
Not much has been heard
for a while from high-speed interconnect Infiniband. Yet this technology is very
much alive, it has firmly established itself in the sector of high-performance
computing and is now venturing into the mainstream.
2000 was an exciting year for the IT industry. The boom phase of the
internet was in full swing, dot-com firms catapulted themselves in a few months
from unknowns to market movers.
A new high-speed interconnect called Infiniband was announced that
could replace all other rapid internal and external system interconnects, yet
went largely unnoticed. Yet in contrast to most of the technologies and
companies that boomed in the dot-com phase, Infiniband has single-mindedly
worked its way forward to the present and is in excellent health, even if some
of its previous claims have been scaled back.
Infiniband was presented at the time as the alternative to practically
all high-speed interconnects. Today it is becoming widely deployed
concentrating on rapid external interconnects between servers and - to a lesser
extent - storage systems. Here Infiniband can make full use of its advantages,
such as extremely low latency periods, simple implementation and low costs.
Rough
start, robust growth
The Infiniband standard has had to overcome several difficult periods
in its short history. It was developed in 1999 as a fusion of two competing
procedures: Next Generation I/O and Future I/O. The first products went under
during the post-internet bubble era, in which firms had higher priorities than
to convert to a new high-speed interconnect. Furthermore, several major
manufacturers such as Intel and Microsoft 2002 revised their plans for
Infiniband support and put their own products on hold for a while.
Andrey
Semin, Technical Program Manager for HPC EMEA at Intel, explains this lack of a
substantial market for busses at the time: "Intel was concentrating on its
core competencies then, and had cancelled its activities in other fields. But we
were, and still are, strong supporters of an Infiniband standard."
Thus, although Intel put aside plans for its own products, the chip
giant continued its efforts on another level. Many small start-ups which had
emerged in the area of Infiniband technology, worked intensively together with
Intel on this development, among them Infinicon (now Silverstream Technologies),
Mellanox and Voltaire.
The result: as the hype about the technology receded, a whole series
of vendors had developed marketable solutions. In the meantime, the
InfiniBand
Trade Association (IBTA) has grown to more than 70 members. An effective
steering committee ensures quick decisions concerning standards are taken. So
for example, extensions were decided on in autumn 2004, whose results will have
an impact far into the future. Transmission rates of up to 60 gigabits per
second will then be possible. At the moment Infiniband is already firing 10
gigabits per second down cable lines.
Rapid, inexpensive, low
latency periods
At present, Infiniband beats almost all other connections relating
to current and planned transmission rates. The technology achieves such high
speeds on the basis of its architecture. It sets up virtual channels, which are
similar to the mainframes of yester year, and transmits information on the basis
of parallel individual messages.
An Infiniband system, consisting of host-channel adapters (HCA) in the
servers, target-channel adapters (TCAs) in memory systems or gateways and the
connecting switches is called "fabric", as with
Fibre Channel technology.
The (HCA) constitutes the interface between the internal bus in the server and
the external bus fabric in the same way as a network card.
The original idea was to integrate Infiniband chips inside the servers
as well and thus be able to send data at maximum speed between the central
memory, processor and external devices without converting it or requiring an
additional card.
At present however, the task of intermediary between the internal bus
and external interconnect is almost always carried out by a plug-in card with
PCI-X or PCI Express bus. However, blade servers are making noticeable efforts
to integrate Infiniband chips directly into the boards and reduce cost and
complexity. It is thus clear that the host need not be a complete computer in
the usual sense of the term.
With Infiniband, a processor can be
coupled with a central memory and rudimentary extensions together with other,
identical elements to build a functioning server system. Such an end node can
communicate with every other node in the same subnet. A subnet manager
configures the adapter with local addresses for every individual port. What is
missing in each individual device, namely the I/O interfaces, is provided for by
the Infiniband fabric.
The connection through switches is the
foundation of the fabric. Physically, Infiniband can be routed through a whole
series of interfaces, including copper and optical
cables. The advantages
of this distribution of computing power in individual nodes and a common I/O
network are obvious: scalability and redundancy. Greater computing performance
can be attained by adding further processor nodes, and guarantees against
breakdowns by the distribution of active processes to other nodes. Small wonder
then that above all users of high-performance computing (HPC), where it is a
question of parallel processing and maximal performance, are currently among the
most enthusiastic users of Infiniband and large-scale clusters.
While
you still can't buy clustered systems based on Infiniband at the local retailer,
the last three years have seen cluster products developed which are quite stable
and which can be implemented with little effort. Linux first and foremost has
carved out a portion of this segment for itself based on its early support of
Infiniband in core systems. Sun
Microsystems announced a while ago that its Utility Computing Grid will be
based on Infiniband. IBM has presented a solution for its blade center together
with Topspin and SGI,
one of the pioneers and most important participants in the HPC market, has also
integrated Infiniband in its Altix servers.
Advantages and
applications
Examining the technical features and the Infiniband concept, it's
advantages all the way. Separating the I/O interfaces from the processor is
music to the ears of server developers and computing centre providers. Fewer
parallel bus systems in the server mean less clutter, less electricity
consumption and, naturally, greater concentrations of devices with,
simultaneously, less need to cool them. Furthermore since I/O use normally comes
in waves, the busses inside the servers often function with reduced workloads.
Infiniband addresses this problem, since all I/O interfaces in the fabric are
accessible to all processing nodes and can be dynamically distributed. This
sounds ideal for blade servers, and manufacturers such as IBM are already
building Infiniband into their blade systems.
This is good news for Infiniband manufacturers too, since the growth
rates for blades are impressive. IDC speaks of a growth rate of 44% in units
sold between the third quarters of 2003 and 2004. When its lower prices are
taken into account, Infiniband should actually have replaced Fibre Channel as a
storage interconnect already. Today an Infiniband port costs half as much
as a fibre-channel port, with double or even quintuple the transmission rate.
Gigabit Ethernet would be cheaper, but in return attains only a tenth of
Infiniband's speed, and 10GB-Ethernet is significantly more expensive, leaving
aside the fact that its standardisation has hardly been completed.
Yet
on the one hand, Infiniband was brought too late onto the market. Fibre Channel
had already established itself, and these investments had to be amortised. On
the other hand, firms in sector are conservative when it comes to introducing
new infrastructures. Without a thorough battery of tests and guaranteed
availability, not only of a technical, but also of a logistical sort, no
purchasing decision will occur. Research institutes are more open in this regard
and more likely to agree to a trial run, which explains the high interest in
Infiniband in the academic HPC sector.
The analyst William Hurley goes
along with this: "I don't expect that in near future Infiniband is going to
make a breakthrough among firms as a backplane network." He is however
convinced: "The principle makes sense, the prices are competitive and the
effort for implementation minimal."
This is also reflected in
then number of ports sold: in 2004 a total of some 500,000 Infiniband ports were
marketed, the same figure for fibre channel at around 20 million.
Engenio:
Integrated storage and Infiniband
One problem which has often come up in Infiniband fabrics up to
now has been connecting them to mass storage devices. A gateway, which functions
as the interface between Infiniband and the fibre-channel or
SCSI, almost always has to
be integrated. Performance suffers as a result of the conversion, and what is
more, a broader storage network has to be available. Engenio has now shown that
Infiniband can not only function as a server interconnect, but also represents a
very good medium for connecting data storage.
Connecting native IB storage not only makes sense on grounds of
performance, but also reduces costs and administrative effort. A native hook-up
of storage systems into Infiniband fabrics could enable the Infiniband standard
to move out of the smaller HPC environment. The analysts at IDC think so too. A
recent IDC survey confirmed that there is growing interest in Infiniband in
different branches. The use of Infiniband in a great many current supercomputer
installations and growing support from manufacturers should thus help to
convince users of the usefulness and feasibility of employing Infiniband.
...Engenio profile |
|
| Here are some other popular
articles which you may be interested in reading:-
|
|
| |
|
|
|
|
|
| . |
Squeak! -
Why are Most Analysts Wrong About Solid State Disks?
Most
analysts and editors of other computer publications don't really understand the
solid state disk market. They show their ignorance and naivete by prefacing
every discussion of SSDs with a superficial analysis which compares the cost per
byte of storage between flash and hard disk drives. That's the wrong answer to
the wrong question. And it's far removed from why the SSD market is racing to
become a multi billion dollar market seemingly in blithe ignorance of the cost
per byte proposition.
This new article tells you what's important to
users and the main applications in which SSDs are already being used and new
applications where they will be used in the next 3 years. ...read the
article, Solid State Disks | |
| . |
 |
the Impact
of Compliance on Archival Storage Strategies - article by Plasmon
It's
difficult enough protecting and archiving your data so that it's available to
the right people at the right time (and cost). But now that's only part of the
problem. With so many new rules and regulations which prescribe how you should
destroy data records at the appropriate time - how do you guarantee that they
stay deleted?
Archiving data on the wrong kind of media could mean you
run the risk of breaking the law. Advances in the
data recovery
industry, and the future cohabitation of storage search-engines both mean that
Compliance Officers have to pay much more attention to the ways in which data is
dispersed and disposed of in different types of media.
This article
summarizes the strengths and weaknesses of currently available market
technologies. ...
read the article,
...Plasmon profile,
Optical Libraries | |
| . |
 |
What is
Data Recovery? - article by Disklabs
The first time you
try any new technology service it can be a bit daunting. Who do you choose?
What if it goes wrong? The pressure really turns up when you are in the
situation of having lost your data due to a backup failure, hardware fault,
operator error or physical disaster. That's usually the first time you look at
the data recovery market. and it's when you're in a high state of anxiety and
can't take the risk of anything else going wrong. This article is a simple guide
to how the data recovery industry works and how it can help you in this
frightening situation. ...read the article,...Disklabs profile,
Data Recovery,
backup software,
Hard disk drives | |
| . |
 |
Virtual
Tape: Can You Afford to Ignore It? - article by MaXXan Systems
Network
connected disk to disk backup systems for the enterprise have come a long way
since the first pioneering products started to appear in the pages of
STORAGEsearch.com in the late 1990s.
Some of the growing
sophistication in the market can be seen by the way that the marketing
terminology has morphed from the early D2d (let's kill tape backup), via D2D2T
(let's be friends with tape / peaceful coexistence) to the current VTL (Virtual
Tape Library - let's just see if they notice that it's more reliable and works
faster - and don't tell them that there isn't a tape in the box) type of
approaches.
But if you think that speed, reliability and cost are the
only things you need to know about the "virtual" versus "real"
tape library argument - take a look at this comprehensive article from MaXXan
Systems which shows there are a lot more benefits than that. ...read the article,
...MaXXan profile,
Disk to disk backup | |
| . |
 |
Disk to
Disk Backup versus Tape - War or Truce? - article by Engenio
Will
disk to disk backup make tape backup obsolete? That's a question that's been
debated hotly here on STORAGEsearch for many years.
At the extreme
polarized ends of the argument are tape media makers like Sony, who in
an article here made a
case for the long term survival of tape, and at the other end of the argument
are disk to disk supporters like STORAGEsearch whose
editorial view has been
that tape doesn't have a viable role the midsize market any more. In the
middle of this argument are the moderates who say that maybe tape and D2d can
co-exist. This article by Steve Gardner at Engenio takes the middle course line
- and says why he thinks there's still a place for both. See if you agree. ...read the article,
...Engenio profile,
RAID systems,
Disk to disk backup | |
| . |
 |
NAS, DAS
or SAN? - Choosing the Right Storage Technology for Your Organization -
article by Xtore
It's 5 years since we published the
Storage Architecture
Guide a classic reference written by the world's first network storage
company Auspex. The new overview article from Xtore places the main storage
connection strategies in a current context. Here's an extract.
"Another
important consideration for a medium sized business or large enterprise is
heterogeneous data sharing. With DAS, each server is running its own operating
platform, so there is no common storage in an environment that may include a mix
of Windows, Mac and Linux workstations. NAS systems can integrate into any
environment and serve files across all operating platforms. On the network, a
NAS system appears like a native file server to each of its different clients.
That means that files are saved on the NAS system, as well as retrieved from the
NAS system, in their native file formats. NAS is also based on industry standard
network protocols such as TCP/IP, FC and CIFS. " ... read the article,
...Xtore profile | |
| . | |
