|
"RAID
was one of the unsung heroes of the digital economy in the past 20 years.
Without RAID - we wouldn't have the big data (and reliability) that we have
gotten used to today. But don't get too hung up on learning the internal
details of traditional RAID. With today's fast links and processors there are
many other new ways to stripe data across drives and memory chips too." -
Zsolt Kerekes,
editor |
RAID is
an acronym for
Redundant Array of Inexpensive Disks. |
In the mid 1980's when
this term first entered public awareness, you could buy 2 types of
disk drives, either low
cost drives such as used in the average PC, or high speed high performance
mainframe drives as used by Quantel
in its digital video effects systems.
The huge market for PC disks soon became the leading edge technology
drivers for disk storage and overtook the larger minicomputer and mainframe
form factor disks in speed,
reliability and
capacity.
By the late 1990s RAID systems using PC form factor disks
had become the most common form of bulk storage in enterprise servers and even
some (Unix) mainframes.
Today the original RAID concept remains valid even though hard disks
have changed form factors many times in the past 20 years (8", 5.25",
3.5", 2.5", 1.8" and below 1") and the concept may be useful
in the near future when the "disks" in the array could actually be
flash solid state
disks and not traditional
hard disks.
You
can create a virtual disk array which looks electronically just like a bigger
ordinary disk, by attaching a bunch of disks working in parallel and connected
to a
RAID controller
interface.
The combined system can be programmed to provide desirable
characteristics such as faster data throughput (for example a 4 disk
wide system could have a data throughput capability 4 times faster than a single
disk).
RAID can also provide
fault tolerance, because redundant disks can be added into the array and
the data split up in such a way with redundant error bits that there is no loss
of data if any single disk fails (or if 2 disks fail in some RAID
configurations) - provided the dead disk(s) is replaced and the data rebuilt
before the next failure occurs.
RAID doesn't always result in an
application speedup. It can slow down the access time in some types of
application in which the data sets are small and randomly located - because the
latency of the RAID controller is additional to the disk's own access time. ...from
Megabyte's Storage
Dictionary | |
| . |
|
| . |
| Nibbles Re:
RAID History |
IBM received the first patent
for a disk array subsystem in
1978, and
co-sponsored the research by the University of California at Berkeley that led
to the initial definition of RAID levels in 1987.
IBM launched the
first modern style RAID systems in
1990
It wasn't until the late 1990's that RAID technology became a "must-have"
building block in commercial Unix servers.
In the future RAID
technology will be already integrated in most home PC's and entertainment
systems, because home users don't do backups, but they will have large digital
entertainment libraries which won't fit neatly onto a single disk. | |
|
| . |
|
Nibble:- RAID and Me
I first came
across the concept of RAID in 1986 when I joined a company called Databasix
as their hardware engineering manager.
Databasix, based in Newbury
in the UK, was founded by John Golding from the nucleus of an earlier company
he had been in called Microconsultants.
Microconsultants had spawned
rich sister company Quantel.
Peter Michael, who had been the business and technical brains in both these
earlier companies, gave Databasix a very generous start in life from what I
could see by the pool of talented people and money which had mostly been already
spent when I joined the 70 people or so in the new "start-up".
Among
the many things on my to-do list was to build a working RAID controller and RAID
array demonstration system.
"I don't know much about hard drives"
- I said. I could afford to be honest - because these guys had already seen the
worst when they first met me. My VC backed networked data acquisition company
was going bust and they had been a potential buyer.
"There's not
much too it." My boss said. "Just read the manuals that come with the
disk drives. We want to see if RAID will give us fast real-time disks at a cost
that's significantly less than the video disks from Japan used by our sister
company Quantel."
"OK" I said. "I don't know much
about RAID either."
"Nobody does. Here's a bunch of
articles. They tell you all you need to know. We'd like the demonstration ready
in 3 weeks."
"That sounds like a very short time to me."
"We've
already ordered the disks to save time. You order whatever chips you need, and
use some of the software guys to help on this."
From memory, I
think I got the demo deadline pushed out to about 4 to 5 weeks.
We
also had an Artificial Intelligence demo being worked on at the same time by
about 50 software engineers, and a parallel computing demo, but the RAID
functionality was the "must have" thing which could not be easily
dropped from the sales plan.
We did build a working 4 drive RAID
system. One of our biggest problems had been the high rate of Dead On Arrival
disk drives. That caused a lot of problems which we initially blamed on the
software. Hard drives were a lot more sensitive in those days and could be
killed just by putting them down on your desk.
But by then I knew a
lot more about disks and realised that the Inexpensive Disks we used in our
demonstrator weren't anywhere near as fast as they could have been, because they
were the wrong standard. Then, as now, there were many interface standards for
disk drives. If you're going to build a fast system then you might as well use
fast building blocks. Dataquest was telling me that we should probably be using
SCSI instead.
As my imposed wish lists started to pile up and
commercial reality started hitting my new employer, I decided that it would be a
heck of lot easier to partner with a disk controller company which was already
down this part of the curve, and later we became a beta site for dozens of
manufacturers of processor cards, array processors, hardware interfaces, memory
and disk drives as we tried to make a business out of selling the technology
curve to military buyers almost before it was really there. That was great fun,
but a different story.
It was about 10 years after that before RAID
systems next appeared in my life - when RAID companies like DEC (acquired by
Compaq and now part of HP) and Data General (acquired by EMC) started promoting
their RAID systems to readers of my Sun foused
SPARC Directory.
Nearly 12 years after my first acquaintance with RAID it became one
of the first 4 product categories here on STORAGEsearch.com. And although the
interface patterns have changed over the years, from DAS SCSI, then
Fibre-channel SAN, then Ethernet NAS, and then iSCSI, the ideas inside the box
have remained the same.
When you get to be an old guy like me, it's a
lot easier if some of the new stuff which hits your brain, is actually a rehash
of old stuff.
...more about Databasix.
I didn't want to interrupt
the narrative flow above - but the core Directors at Databasix when I joined
were:- John Golding, Andrew Bruce, Ray Potter and Dan Boxshall.
| | |
..... |
 |
| ..... |
| RAID, RAIC,
RAISE etc - RAID's Many Aliases |
The long established word "RAID"
has been stretched by marketers into many other disguises to make their
products sound better.
The RAID system concept itself is simple, being
a box of disk drives with self healing properties which can be run in parallel
for faster data throughput. But that's not sexy enough for most storage product
marketers. So we now have the following refinements.
- RAID connected by SCSI, Firewire or IDE can be called a
DAS (Directly Attached Storage). The RAID has to be connected by
something, but "DAS" sounds more modern, and indicates that you chose
this method of connection in preference to all the others.
- RAID connected by Fibre-channel can be called a SAN (Storage Area
Network). That sounds better already.
- RAID connected by Ethernet can be called a NAS (Network Attached
Storage). Latterly the term IP-SAN has been used to add new freshness to
this word-washing.
- RAID connected by both Fibre-channel and Ethernet can be called a NUS
(Network Unified Storage).
- Even better than plain old vanilla NUS,apparently, is
SUS, or Scalable Unified Storage, coined by the short lived startup
Broadband Storage
- More likely to endure than either NUS or SUS, is market research company
Gartner's term FAS
for Fabric Attached Storage which also lumps NAS and SAN together.
- RAIN (Redundant Array of Independent Nodes) is an
Adaptec creation
- MAID (Massive Array of Idle Disks) is a whimsical term from
COPAN Systems used for
disk to disk backup systems.
- RAIGE - (RAID Across Independent Gigabit
Ethernet) is a creation of Pivot3
although some of concepts sound similar to how
Google implements its
internal storage infrastructure.
- RAISE - (Redundant Array of Independent Silicon Elements) is a
term invented by SandForce.
Their SSD controller uses this protection scheme inside
1.8" and
2.5" SSDs.
- RAIC - redundant array of independent chips - was coined in the
1990s by Solid Data
Systems
- DVRAID - is a proprietary RAID technology from
ATTO Technology that is "optimized
for digital content creation environments that require protection in the event
of a disk failure without the performance penalty traditionally seen with parity
RAID."
- SAID - Self-maintaining Array of Identical Disks - a possibly
overambitious term from Atrato
- RAIDn
- is an algorithm launched in 2003 by InoStor. It never achieved wide
currency.
- Finally, a RAID not connected to anything at all
can be called a
LUS (Lonely Unloved Storage)...
No, I just made this one up.
But you can see the basic principle at work here. And no doubt there will be
other terms later for RAID connected by the Internet or Infiniband. See also:- Megabyte's
Storage Glossary which
includes definitions of the many other strange terms which appear from time to
time in these pages. | | |
| ..... |
|
|
iSCSI /
NAS /
SAN RAID controller cards SATA Raids the
Datacenter A
Storage Architecture Guide RAID
Levels Outlive Their Usefulness RAID
storage basics guide for beginners Which RAID Manufacturers
will Survive? New Market for RAID - Flash
SSD Arrays RAID
- editor mentions on StorageSearch.com Market Trends in the
Rackmount SSD Market 10 Ten Tips for a
Successful RAID Implementation
RAID
- what it is, why we need it and how it works (pdf) Using Solid State Disks
to Boost Legacy RAID Performance NAS, DAS or SAN? -
Choosing the Right Storage Technology |
RAID news |
Pure Storage has amassed
$55 million for bulk FC SAN SSD storage
Editor:- August 24, 2011 -
Pure Storage
yesterday unveiled its first SSD product line and announced it had received
$30 million in series C funding bringing its total capital funding up to $55
million.
Pure Storage 's
FlashArray
provides bulk / utility SSD storage for
FC SAN enviroments - which
by using inline dedupe and compression - can in some applications (25TB and 50K
IOPS per U) offer lower cost and yet still deliver higher performance than
classic hard drive disk arrays.
Editor's comments:- This
looks like a spreadsheet based value proposition rather than a disruptive new
product - and follows a market groove already established by
WhipTail Technologies
and Nimbus Data Systems.
The market for this type of SSD market will be huge - but along the way to
proving itself will have to fight off competition from
auto-tieing SSDs and
white box SSD RAID which
will nibble away at the same customer SSD budgets.
Pushing data reliability up hard drive hill
Editor:-
July 4, 2011 - Why didn't
hard drives get more
reliable?
Enterprise users are still replacing hard drives according to cycles that have
haven't changed much since RAID
became common in the 1990s. So why didn't HDD makers do something to make their
drives better?
Error correction code inventor Phil White -
founder of ECC
Technologies has recently published a
rant
/ blog in which he describes the 25 years of rejections he's had from
leading HDD makers - and the reaons they said they didn't want to use his
patented algorithm - which he says could increase data integrity and the life
of hard drives (and maybe SSDs too.) It makes interesting reading for any other
wannabe inventors out there too. ...read
Phil White's article
But I think another reason for past
rejections might simply have been market economics.
The capacity
versus the cost of HDDs has improved so much throughout that period - and at
the same time data capacity needs have grown - maybe the user value proposition
didn't make sense.
If you (RAID user) find that all your 5 year old
drives are still working (instead of being replaced) - how much is that really
worth? By now those 5 year old drives might only represent 3% to
10% of the new storage capacity you need anyway. (The reliability
value proposition is different outside service engineer frequented zone - but I
don't want to get side-tracked into
SSD market
models here.)
Looking ahead at the future of the HDD market my own
view is that whatever the industry does with respect to reliability won't tip
the balance against
SSDs
in the enterprise.
The best bet for the future of hard drive
makers is in consumer products where fashion ranks higher up the reason to
buy list than longevity. Most people I know replace their notebook pcs, tvs
and phones not because the old ones have stopped working - but because the new
ones have lifestyle features which make them more desirable.
the missing link?
software which sits between HDD RAID
and SSDs
Editor:- July 1, 2011 - earlier this week I spent an
interesting hour talking to
FlashSoft's
CEO - Ted Sanford
about the company's business plans and technology.
The company
recently launched software which enables almost any SSD to act as a cache
accelerator front end for hard disk storage arrays in enterprise servers. By
automatically learning data hot spots as little as 15 minutes after being
installed - the new software speeds up SQL queries for example 4x - and
enables users to use less servers. ...read more
SANBlaze ships PCIe to 1.8" SSD RAID adapter
June
13, 2011 - SANBlaze
Technology is shipping a new
rear
transition module which connects upto 8x
1.8" SSDs to
PCIe with
RAID options.
optimizing SSD architecture to cope with flash plane errors
Editor:-
May 24, 2011 - a new slant on
SSD reliability
architectures is revealed today by Texas Memory Systems
who explained how their patented Variable Stripe RAID technology is used in
their recently launched PCIe SSD card - the
RamSan-70.
TMS
does a 1 month burn-in of flash memory prior to shipment. (One of the
reasons cited for its use
of SLC rather than
MLC BTW.)
Through its QA processes the company has acquired real-world failure data
for several generations of flash
memory and used this to model and characterize the failure modes which
occur in high IOPs SSDs.
Most enterprise SSDs use a simple type of
classic RAID which groups
flash media into "stripes" containing equal numbers of chips. RAID
technology can reconstruct data from a failed Flash chip. Typically, when a chip
or part of a chip fails, the RAID algorithm uses a spare chip as a virtual
replacement for the broken chip. But once the SSD is out of spare chips, it
needs to be replaced.
VSR technology allows the number of chips to
vary among stripes, so bad chips can simply be bypassed using a smaller stripe
size. Additionally, VSR provides greater stripe size granularity, so a stripe
could exclude a small part of a chip rather than having to exclude an
entire chip if only part of it failed - "plane error". With VSR
technology, TMS says its SSD products will continue operating longer in the
installed base.
Dan Scheel, President of Texas Memory Systems explained why their
technology increases reliability.
"...Consider a hypothetical
SSD made up of 25 individual flash chips. If a plane failure occurs that
disables 1/8 of one chip, a traditional RAID system would remove a full 4% of
the raw Flash capacity. TMS VSR technology bypasses the failure and only reduces
the raw flash capacity by 0.5%, an 8x improvement. TMS tests show that plane
failures are the 2nd most common kind of flash device failures, so it is very
important to be able to handle them without wasting working flash."
Editor's
comments:- by wasting less capacity than simpler RAID solutions - more
usable capacity remains available for traditional
bad block
management. This extra capacity comes from the over provisioning budget
which figure varies according to each SSD design (as discussed in my recent
flash iceberg syndrome article) but
is 30% for TMS.
STEC's MLC SSDs used in IBM RAID
Editor:- December
15, 2010 - STEC
announced that MLC versions of its
ZeusIOPS
SSDs are being used in
IBM's
Storwize V7000 (RAID
systems).
One of the advantages of flash SSDs which IBM exploits in
this system is fast data replication.
SSDs accelerating the backup
process was also mentioned in a
recent
interview with Fuion-io's CEO.
Penguin launches IceBreaker HPC RAID
Editor:-
November 15, 2010 -
Penguin Computing
recently
launched
a new family of fast 160TB DAS
RAID storage systems which
the company says provides the performance and scalability of 40Gbps
InfiniBand
environments.
Called
IceBreaker -
it's designed for high workflow apps like scientific modeling - which don't
need the "frills" (and cost) associated with similar capacity
SAN systems.
Editor's
comments:- I asked the question - does
solid state storage feature
in any significant way in the new product architecture? If so - at what
level and managed how? ...click
to read more
How many disks - and why?
Editor:- November 10, 2010
- StorageSearch.com
- today published a new article called -
How many disks does it
take to store a disk-full of data?
Sometimes you can learn
something very useful by asking silly questions which initially seem to
have trivial and obvious answers.
...And where do the SSDs creep in?
They always seem to sneak into my articles somewhere... And RAID? -
naturally that gets mentioned in passing too.
You don't need a
calculator or spreadsheet for this one. ...read the article
SSDs accelerate disaster recovery
Editor:- September
24, 2010 - an update to Intel's SSD Bookmarks
- published today on StorageSearch.com
- includes links to a case study in which RAID rebuild times for a real-time
education server were reduced from 12 hours to 40 minutes, while response times
became 25x faster. ...read the articles
10,000x more reliable than RAID?
Editor:- August 26,
2010 - Amplidata
claims that its
BitSpread
technology is 10,000x more reliable than current
RAID based technologies
and requires 3x less storage.
Is another new way of fixing
reliability
problems in hard disk
arrays worth the effort just as we approach the end of the
hard disk market's life?
- I doubt it. See why in -
this way to the
petabyte SSD.
Infortrend joins the STEC inside club
Editor:- July
22, 2010 - Infortrend
today
announced
it will use STEC's
ZeusIOPS (SAS SSDs) in
its ESVA F60 product line (FC
RAID systems).
PMC-Sierra acquires Adaptec's RAID business
Editor:-
May 10, 2010 - PMC-Sierra
announced a
definitive
agreement to acquire Adaptec's channel
storage business for approximately $34 million in cash.
This deal
includes Adaptec's RAID storage product line, its global VAR customer base,
board logistics capabilities, and
SSD cache performance
solutions.
Editor's comments:- I had heard that
Adaptec's storage business was up for sale a few months ago.
In my
storage market outlook
2010 to 2015 article - published last year - I explained why I thought that
the RAID controller
market couldn't stay as it was.
These companies have to get into
offering complete SSD
solutions in the long term. In the short term PMC-Sierra may be able to do a
better job aggregating a bigger percentage of whatever remains of the untied
RAID controller business.
I expect the RAID business (for
hard disks) will
eventually become a consumer / SMB market - while the enterprise storage array
part of this market will morph through an
SSD ASAP phase -
while users struggle to redefine
new storage
architectures for the datacenter.
Themis launches 1U rugged RAID box for its military servers
Editor:-
May 4, 2010 -
Themis Computer
has
launched
the RES-XStore -
a 1U (17" deep) storage system for mission-critical applications in harsh
environments.
The RES-XStore includes 4 hot pluggable canisters,
each containing 3x 2.5" drives. The RES-XStore communicates with the host
server, via an add-in HBA and PCIe x8. The RES-XStore includes a
RAID controller that
supports RAID levels from
0 to 60. Each canister is hot pluggable. As with all products in the
Rugged Enterprise Servers
family - the new storage enclosure is designed to operate in the most
challenging environments.
SAS / SATA drive array tester reviewed by Demartek
Editor:-
March 22, 2010 - Quarch
Technology announced that
Demartek
had published a report (pdf) which reviews the usability and benefits of
its SAS/SATA disk array test system.
It's useful for integrators
qualifying components and modules for use in new
RAID systems. | |
|
|
|
| ..................................................................................................................... |
| |
..... |
are you looking for an elusive shiny needle
- in an over towering RAID haystack?
My lists of RAID
systems vendors started with just a handful of companies in 1991 and grew to
hundreds of companies.
When lists are that large they are boring. And
RAID is now a commodity. So I've zapped the RAID oems list which used to be on
this page.
Instead - you'll get better results using the site search
box below - where you can add qualifiers to your RAID search - such a "military"
or "broadcast" or whatever else you're looking for. |
|
| | |
|
 |
Megabyte
found that tying lots of barrels together to cross the data stream worked
well. And if one of them got punctured, the raft didn't sink. | |
|
| SSDs in
RAID's past and future |
SSDs have been used in
RAID configurations since the
1990s.
But there are problems with this approach when using conventional
RAID controllers and
2.5" SSDs.
Although
SSD IOPS
(and throughput) does
scale
in arrays - latency is slugged by slow controllers which were originally
designed for HDDs. This approach is also wasteful of flash capacity compared
with other redundancy approaches designed into proprietary
rackmount SSDs.
And finally - the RAID approach - when applied to
2.5" SSD modules
- doesn't address the problem of end to end
data integrity -
which has been mentioned in this
article.
Some enterprise RAID companies, recognizing that hard disk based
RAID in the enterprise will
be a dead market in a few years, have moved into the emerging
SSD ASAPs market
which blends the advantages of HDD (cheap capacity) and SSD (fast IOPS)
into Auto-tuning SSD Accelerated Pools of storage.. | | |
| ..... |
|
|
| . |
|
| |