|
"RAID
was one of the unsung heroes of the digital economy in the past 20 years.
Without RAID - we wouldn't have the big data (and reliability) that we have
gotten used to today. But don't get too hung up on learning the internal
details of traditional RAID. With today's fast links and processors there are
many other new ways to stripe data across drives and memory chips too - with
lower latency and better efficiency." -
Zsolt Kerekes,
editor |
RAID is
an acronym for
Redundant Array of Inexpensive Disks. |
In the mid 1980's when
this term first entered public awareness, you could buy 2 types of
disk drives, either low
cost drives such as used in the average PC, or high speed high performance
mainframe drives as used by Quantel
in its digital video effects systems.
The huge market for PC disks soon became the leading edge technology
drivers for disk storage and overtook the larger minicomputer and mainframe
form factor disks in speed,
reliability and
capacity.
By the late 1990s RAID systems using PC form factor disks
had become the most common form of bulk storage in enterprise servers and even
some (Unix) mainframes.
Today the original RAID concept remains valid even though hard disks
have changed form factors many times in the past 20 years (8", 5.25",
3.5", 2.5", 1.8" and below 1") and the concept may be useful
in the near future when the "disks" in the array could actually be
flash solid state
disks and not traditional
hard disks.
You
can create a virtual disk array which looks electronically just like a bigger
ordinary disk, by attaching a bunch of disks working in parallel and connected
to a
RAID controller
interface.
The combined system can be programmed to provide desirable
characteristics such as faster data throughput (for example a 4 disk
wide system could have a data throughput capability 4 times faster than a single
disk).
RAID can also provide
fault tolerance, because redundant disks can be added into the array and
the data split up in such a way with redundant error bits that there is no loss
of data if any single disk fails (or if 2 disks fail in some RAID
configurations) - provided the dead disk(s) is replaced and the data rebuilt
before the next failure occurs.
RAID doesn't always result in an
application speedup. It can slow down the access time in some types of
application in which the data sets are small and randomly located - because the
latency of the RAID controller is additional to the disk's own access time. ...from
Megabyte's Storage
Dictionary | |
| . |
|
| . |
| Nibbles Re:
RAID History |
IBM received the first patent
for a disk array subsystem in
1978, and
co-sponsored the research by the University of California at Berkeley that led
to the initial definition of RAID levels in 1987.
IBM launched the
first modern style RAID systems in
1990
It wasn't until the late 1990's that RAID technology became a "must-have"
building block in commercial Unix servers.
In the future RAID
technology will be already integrated in most home PC's and entertainment
systems, because home users don't do backups, but they will have large digital
entertainment libraries which won't fit neatly onto a single disk. | |
|
| . |
|
Nibble:- RAID and Me
I first came
across the concept of RAID in 1986 when I joined a company called Databasix
as their hardware engineering manager.
Databasix, based in Newbury
in the UK, was founded by John Golding from the nucleus of an earlier company
he had been in called Microconsultants.
Microconsultants had spawned
rich sister company Quantel.
Peter Michael, who had been the business and technical brains in both these
earlier companies, gave Databasix a very generous start in life from what I
could see by the pool of talented people and money which had mostly been already
spent when I joined the 70 people or so in the new "start-up".
Among
the many things on my to-do list was to build a working RAID controller and RAID
array demonstration system.
"I don't know much about hard drives"
- I said. I could afford to be honest - because these guys had already seen the
worst when they first met me. My VC backed networked data acquisition company
was going bust and they had been a potential buyer.
"There's not
much too it." My boss said. "Just read the manuals that come with the
disk drives. We want to see if RAID will give us fast real-time disks at a cost
that's significantly less than the video disks from Japan used by our sister
company Quantel."
"OK" I said. "I don't know much
about RAID either."
"Nobody does. Here's a bunch of
articles. They tell you all you need to know. We'd like the demonstration ready
in 3 weeks."
"That sounds like a very short time to me."
"We've
already ordered the disks to save time. You order whatever chips you need, and
use some of the software guys to help on this."
From memory, I
think I got the demo deadline pushed out to about 4 to 5 weeks.
We
also had an Artificial Intelligence demo being worked on at the same time by
about 50 software engineers, and a parallel computing demo, but the RAID
functionality was the "must have" thing which could not be easily
dropped from the sales plan.
We did build a working 4 drive RAID
system. One of our biggest problems had been the high rate of Dead On Arrival
disk drives. That caused a lot of problems which we initially blamed on the
software. Hard drives were a lot more sensitive in those days and could be
killed just by putting them down on your desk.
But by then I knew a
lot more about disks and realised that the Inexpensive Disks we used in our
demonstrator weren't anywhere near as fast as they could have been, because they
were the wrong standard. Then, as now, there were many interface standards for
disk drives. If you're going to build a fast system then you might as well use
fast building blocks. Dataquest was telling me that we should probably be using
SCSI instead.
As my imposed wish lists started to pile up and
commercial reality started hitting my new employer, I decided that it would be a
heck of lot easier to partner with a disk controller company which was already
down this part of the curve, and later we became a beta site for dozens of
manufacturers of processor cards, array processors, hardware interfaces, memory
and disk drives as we tried to make a business out of selling the technology
curve to military buyers almost before it was really there. That was great fun,
but a different story.
It was about 10 years after that before RAID
systems next appeared in my life - when RAID companies like DEC (acquired by
Compaq and now part of HP) and Data General (acquired by EMC) started promoting
their RAID systems to readers of my Sun foused
SPARC Directory.
Nearly 12 years after my first acquaintance with RAID it became one
of the first 4 product categories here on STORAGEsearch.com. And although the
interface patterns have changed over the years, from DAS SCSI, then
Fibre-channel SAN, then Ethernet NAS, and then iSCSI, the ideas inside the box
have remained the same.
When you get to be an old guy like me, it's a
lot easier if some of the new stuff which hits your brain, is actually a rehash
of old stuff.
...more about Databasix.
I didn't want to interrupt
the narrative flow above - but the core Directors at Databasix when I joined
were:- John Golding, Andrew Bruce, Ray Potter and Dan Boxshall.
| | |
..... |
 |
| ..... |
|
| ..... |
| RAID, RAIC,
RAISE etc - RAID's Many Aliases |
The long established word "RAID"
has been stretched by marketers into many other disguises to make their
products sound better.
The RAID system concept itself is simple, being
a box of disk drives with self healing properties which can be run in parallel
for faster data throughput. But that's not sexy enough for most storage product
marketers. So we now have the following refinements.
- RAID connected by SCSI, Firewire or IDE can be called a
DAS (Directly Attached Storage). The RAID has to be connected by
something, but "DAS" sounds more modern, and indicates that you chose
this method of connection in preference to all the others.
- RAID connected by Fibre-channel can be called a SAN (Storage Area
Network). That sounds better already.
- RAID connected by Ethernet can be called a NAS (Network Attached
Storage). Latterly the term IP-SAN has been used to add new freshness to
this word-washing.
- RAID connected by both Fibre-channel and Ethernet can be called a NUS
(Network Unified Storage).
- Even better than plain old vanilla NUS,apparently, is
SUS, or Scalable Unified Storage, coined by the short lived startup
Broadband Storage
- More likely to endure than either NUS or SUS, is market research company
Gartner's term FAS
for Fabric Attached Storage which also lumps NAS and SAN together.
- RAIN (Redundant Array of Independent Nodes) is an
Adaptec creation
- MAID (Massive Array of Idle Disks) is a whimsical term from
COPAN Systems used for
disk to disk backup systems.
- RAIGE - (RAID Across Independent Gigabit
Ethernet) is a creation of Pivot3
although some of concepts sound similar to how
Google implements its
internal storage infrastructure.
- RAISE - (Redundant Array of Independent Silicon Elements) is a
term invented by SandForce.
Their SSD controller uses this protection scheme inside
1.8" and
2.5" SSDs.
- RAIC - redundant array of independent chips - was coined in the
1990s by Solid Data
Systems
- DVRAID - is a proprietary RAID technology from
ATTO Technology that is "optimized
for digital content creation environments that require protection in the event
of a disk failure without the performance penalty traditionally seen with parity
RAID."
- SAID - Self-maintaining Array of Identical Disks - a possibly
overambitious term from Atrato
- RAIDn
- is an algorithm launched in 2003 by InoStor. It never achieved wide
currency.
- Finally, a RAID not connected to anything at all
can be called a
LUS (Lonely Unloved Storage)...
No, I just made this one up.
But you can see the basic principle at work here. And no doubt there will be
other terms later for RAID connected by the Internet or Infiniband. See also:- Megabyte's
Storage Glossary which
includes definitions of the many other strange terms which appear from time to
time in these pages. | | |
| ..... |
iSCSI /
NAS /
SAN
RAID controller cards
SATA Raids the
Datacenter
A
Storage Architecture Guide
RAID
Levels Outlive Their Usefulness
What do
enterprise SSD users want?
Which RAID Manufacturers
will Survive?
Where are we now
with SSD software?
How fast can your SSD
run backwards?
RAID
storage basics guide for beginners
Market Trends in the
Rackmount SSD Market
10 Ten Tips for a
Successful RAID Implementation
RAID
- editor mentions on StorageSearch.com
RAID
- what it is, why we need it and how it works (pdf)
Using Solid State Disks
to Boost Legacy RAID Performance (2004) |
RAID news |
Intels oems LSI's RAID
caching SSD technology
Editor:- April 8, 2013 - Intel - which already
uses LSI's
SandForce controllers in some SSDs - will oem LSI's dual-core RAID-on-Chip
flash caching technology it was announced today.
LSI says their
caching technology can double the number of VDI sessions supported in the same
sever and flash environment.
“Intel’s selection of
LSI
Nytro MegaRAID technology is another significant validation of our strategic
focus and investments in flash-based server acceleration technology,” said
Gary Smerdon,
senior VP and GM, Accelerated Solutions, LSI.
Old 3ware RAID 10 kills Intel SSD array performance
Editor:-
February 11, 2013 - It's not really news that anything which wastes latency and
raw CPU performance in the host interface and related controllers - also
prevents and wastes the possible performance gain from SSDs. But sometimes
users are the victim of circumstances outside their control.
This
point is vividly demonstrated in the article -
How
slow can SSD be or why is testing a new server performance important? -
which describes the problems someone had when using SSDs in an array with a
slow controller designed for hard drives which was the default offered by
their ISP company.
Amplidata gets $8 million C funding for - (don't call it RAID)
Editor:- February 29, 2012 - apparently you can still raise
VC funding for
HDD based storage array
companies.
Amplidata
today said it got
$8
million in a Series C round of funding for its multi-petabyte scalable,
efficiently data protecting and healing
BitDynamics technology.
The
idea is to implement the cheapest possible and easily expandable bulk
storage capacity - what they call
unbreakable
object storage for petabyte-scale unstructured data (pdf) - so the storage
nodes are stuffed with low power hard drives. But the controller racks can be
internally accelerated by SSDs.
Pure Storage has amassed $55 million for bulk FC SAN SSD storage
Editor:- August 24, 2011 - Pure Storage
yesterday unveiled its first SSD product line and announced it had received
$30 million in series C funding bringing its total capital funding up to $55
million.
Pure Storage 's
FlashArray
provides bulk / utility SSD storage for
FC SAN enviroments - which
by using inline dedupe and compression - can in some applications (25TB and 50K
IOPS per U) offer lower cost and yet still deliver higher performance than
classic hard drive disk arrays.
Editor's comments:- This
looks like a spreadsheet based value proposition rather than a disruptive new
product - and follows a market groove already established by
WhipTail Technologies
and Nimbus Data Systems.
The market for this type of SSD market will be huge - but along the way to
proving itself will have to fight off competition from
auto-tieing SSDs and
white box SSD RAID which
will nibble away at the same customer SSD budgets.
Pushing data reliability up hard drive hill
Editor:-
July 4, 2011 - Why didn't
hard drives get more
reliable?
Enterprise users are still replacing hard drives according to cycles that have
haven't changed much since RAID
became common in the 1990s. So why didn't HDD makers do something to make their
drives better?
Error correction code inventor Phil White -
founder of ECC
Technologies has recently published a
rant
/ blog in which he describes the 25 years of rejections he's had from
leading HDD makers - and the reaons they said they didn't want to use his
patented algorithm - which he says could increase data integrity and the life
of hard drives (and maybe SSDs too.) It makes interesting reading for any other
wannabe inventors out there too. ...read
Phil White's article
But I think another reason for past
rejections might simply have been market economics.
The capacity
versus the cost of HDDs has improved so much throughout that period - and at
the same time data capacity needs have grown - maybe the user value proposition
didn't make sense.
If you (RAID user) find that all your 5 year old
drives are still working (instead of being replaced) - how much is that really
worth? By now those 5 year old drives might only represent 3% to
10% of the new storage capacity you need anyway. (The reliability
value proposition is different outside service engineer frequented zone - but I
don't want to get side-tracked into
SSD market
models here.)
Looking ahead at the future of the HDD market my own
view is that whatever the industry does with respect to reliability won't tip
the balance against
SSDs
in the enterprise.
The best bet for the future of hard drive
makers is in consumer products where fashion ranks higher up the reason to
buy list than longevity. Most people I know replace their notebook pcs, tvs
and phones not because the old ones have stopped working - but because the new
ones have lifestyle features which make them more desirable.
the missing link?
software which sits between HDD RAID
and SSDs
Editor:- July 1, 2011 - earlier this week I spent an
interesting hour talking to
FlashSoft's
CEO - Ted Sanford
about the company's business plans and technology.
The company
recently launched software which enables almost any SSD to act as a cache
accelerator front end for hard disk storage arrays in enterprise servers. By
automatically learning data hot spots as little as 15 minutes after being
installed - the new software speeds up SQL queries for example 4x - and
enables users to use less servers. ...read more
SANBlaze ships PCIe to 1.8" SSD RAID adapter
June
13, 2011 - SANBlaze
Technology is shipping a new
rear
transition module which connects upto 8x
1.8" SSDs to
PCIe with
RAID options.
optimizing SSD architecture to cope with flash plane errors
Editor:-
May 24, 2011 - a new slant on
SSD reliability
architectures is revealed today by Texas Memory Systems
who explained how their patented Variable Stripe RAID technology is used in
their recently launched PCIe SSD card - the
RamSan-70.
TMS
does a 1 month burn-in of flash memory prior to shipment. (One of the
reasons cited for its use
of SLC rather than
MLC BTW.)
Through its QA processes the company has acquired real-world failure data
for several generations of flash
memory and used this to model and characterize the failure modes which
occur in high IOPs SSDs.
Most enterprise SSDs use a simple type of
classic RAID which groups
flash media into "stripes" containing equal numbers of chips. RAID
technology can reconstruct data from a failed Flash chip. Typically, when a chip
or part of a chip fails, the RAID algorithm uses a spare chip as a virtual
replacement for the broken chip. But once the SSD is out of spare chips, it
needs to be replaced.
VSR technology allows the number of chips to
vary among stripes, so bad chips can simply be bypassed using a smaller stripe
size. Additionally, VSR provides greater stripe size granularity, so a stripe
could exclude a small part of a chip rather than having to exclude an
entire chip if only part of it failed - "plane error". With VSR
technology, TMS says its SSD products will continue operating longer in the
installed base.
Dan Scheel, President of Texas Memory Systems explained why their
technology increases reliability.
"...Consider a hypothetical
SSD made up of 25 individual flash chips. If a plane failure occurs that
disables 1/8 of one chip, a traditional RAID system would remove a full 4% of
the raw Flash capacity. TMS VSR technology bypasses the failure and only reduces
the raw flash capacity by 0.5%, an 8x improvement. TMS tests show that plane
failures are the 2nd most common kind of flash device failures, so it is very
important to be able to handle them without wasting working flash."
Editor's
comments:- by wasting less capacity than simpler RAID solutions - more
usable capacity remains available for traditional
bad block
management. This extra capacity comes from the over provisioning budget
which figure varies according to each SSD design (as discussed in my recent
flash iceberg syndrome article) but
is 30% for TMS.
SSDs accelerate disaster recovery
Editor:- September
24, 2010 - an update to Intel's SSD Bookmarks
- published today on StorageSearch.com
- includes links to a case study in which RAID rebuild times for a real-time
education server were reduced from 12 hours to 40 minutes, while response times
became 25x faster. ...read the articles | |
|
|
|
| ..................................................................................................................... |
| |
..... |
 |
Megabyte
found that tying lots of barrels
together to cross the data stream worked
well. And if one of them got punctured,
the raft didn't sink. | |
| .. |
| "In 1998
StorageSearch.com's RAID page (this page you're seeing now) listed 51
manufacturers." |
| ...from -
storage history | | |
| ..... |
| SSDs in
RAID's past and future |
SSDs have been used in
RAID configurations since the
1990s.
But there are problems with this approach when using conventional
RAID controllers and
2.5" SSDs.
Although
SSD IOPS
(and throughput) does
scale
in arrays - latency is slugged by slow controllers which were originally
designed for HDDs. This approach is also wasteful of flash capacity compared
with other redundancy approaches designed into proprietary
rackmount SSDs.
And finally - the RAID approach - when applied to
2.5" SSD modules
- doesn't address the problem of end to end
data integrity -
which has been mentioned in this
article.
Some enterprise RAID companies, recognizing that hard disk based
RAID in the enterprise
will be a dead
market in a few years, have moved into the emerging
SSD ASAPs market
which blends the advantages of HDD (cheap capacity) and SSD (fast IOPS)
into Auto-tuning SSD Accelerated Pools of storage.. | | |
| .. |
| "The performance
impact from RAID rebuilds becomes compounded with long rebuild times incurred by
mutli-terabyte drives. Since traditional RAID rebuilds entirely into a new spare
drive, there is a massive bottleneck of the write speed of that single drive
combined with the read bottleneck of the few other drives in the RAID set." |
| Dave Wright,
CEO - SolidFire
- in his blog -
Say farewell
to RAID storage (March 2013) | | |
| .. |
| "When it comes to
hard drives RAID was better than what it replaced in the 1980s but if you
started again with the internet connectivity and processors we have today and
started to design big arrays of disks from first principles then you wouldn't
see just the RAID systems you see today. That's because RAID is small
controller architecture..." |
| Editor:-
in the article - Why
size matters in SSD controller architecture (June 2011) | | |
| ..... |
| "What we need to
do now is recognize that in very large systems, with lots of components
failures are no longer a rare anomalous event - they are a frequent normal
condition - and we need to quote performance and availability and
reliability assuming that there is always some repair going on and we need to
acknowledge that all these redundancy schemes reduce the probability of data
loss they don't eliminate data loss..." |
| Garth Gibson,
founder and chief scientist,
Panasas
Long Live RAID
(video) (May 2012) | | |
| . |
| "...Cached RAID
solutions are starting to run out of gas in high-performance,
transaction-intensive applications: while the performance demands on e-business
infrastructures are being driven to new levels by exploding demand,
unpredictable peak loads, and an increasingly impatient population of on-line
customers. It's time for another architectural innovation." |
| Michael Casey, Solid Data Systems
(November 2000) - in his article -
Solid State
File-Caching for Performance and Scalability | | |
| . |
 |
| . |
|
|
| . |
are you looking for an elusive shiny needle
- in an over towering RAID haystack?
My lists of RAID
systems vendors started with just a handful of companies in 1991 and grew to
hundreds of companies.
When lists are that large they are boring. And
RAID is now a commodity. So I've zapped the RAID oems list which used to be on
this page.
Instead - you'll get better results using site search -
with whatever is your favorite search-engine.
| | | |
