top 10 SSD oems
top 10 SSD oems

storage search
10 years - "leading the way to the new storage frontier"

Storage Virtualization Means More Than One Media

Pegasus Disk Technologies

Article by Roy Slicker and Jim Wheeler Pegasus Disk Technologies

See also:- optical storage drives
Storage Software ISVs
Can you trust your flash SSD's specs?
History of Enterprise Disk to Disk Backup
the Benefits of SAS for External Subsystems
Editor's intro Storage Virtualization - is a term you're seeing in a lot of news stories nowadays. The reason you need virtualization is because systems change over the lifetime of your data, and this makes managing change easier. Once you become familiar with the concept you see it's a great idea, and may ask yourself "why hasn't it been around forever?" This article discusses virtualization generally and looks in detail at the important aspect of archiving.
storage ad click for more info
Aberdeen Reports on Storage Virtualization
BOSTON, MA – April 19, 2007 – more customers are turning to storage and server virtualization to ease data center headaches according to an end-user survey recently conducted by Aberdeen.

Among the benchmark report's key findings is that 50% of the overall respondents either are evaluating or plan to adopt server virtualization, while 47% either are evaluating or plan to adopt storage virtualization within 6 to 18 months.

Furthermore, results concluded that 31% of the overall respondents have deployed server virtualization and 28% have installed storage virtualization. ...Aberdeen Group profile, Market research
Backup software on
Megabyte's uncle Spellerbyte was a wizard when it came to software. He'd given Megabyte a magic potion for copying critical data.

Virtualization: According to Boston, Mass.-based Aberdeen Group, Inc., virtualization is technology that enables the separation of representation of storage to the server operating the system from actual physical storage. This division of physical storage devices from the logical storage space presented to users and applications turns storage into a generally available utility pool. Virtualization fulfills a roll for storage similar to that which an operating system does for a computer system—namely making programming and operation simpler by automating resource management "housekeeping". When this process occurs, computer users are said to be "viewing resources at a higher level of abstraction." Thus in short, virtualization is the abstraction of storage. This definition of virtualization facilitates in the ability for customers to pool a wide range of storage technologies from different vendors and formats, to easily add capacity, readily move data, or automate moving data among devices independent of server operating systems or network infrastructure.

In a report issued earlier this year by Aberdeen Group, Inc., Analyst Dan Tanner advised IT customers to consider the potential benefits of storage virtualization which, he said, include: creating virtual volumes that can span multiple storage units, enabling heterogeneous "mix and match" server and storage device usage, serving as a platform for cross-device storage services, and enabling secure storage sharing and efficient storage utilization.

The general storage market, particularly the Storage Area Network (SAN) and Storage Service Provider (SSP) segments, has been very active with several new entries being funded on a large scale in a relatively short period of time. While this is good news for users looking for new products and services for growing data bases, increasing backup windows and short term storage applications, these systems do not address the specific needs of the large and growing archive and compliance segment of the market.

While current SAN and SSPs battle for market share, the need to store strategic and compliance or regulatory records has baffled providers because of the reluctance of enterprise customers to out source storage, while on the other hand, they continue to use magnetic based RAID for the storage of regulatory records. All the energy and money being pumped into the marketing of SAN and SSP systems is having a positive and wider spread effect on other storage companies in general by raising the awareness of storage products. It appears that a growing number of users are more interested in the secure and reliable storage aspect rather than just adding massive internal or off-site storage capabilities.

Lately, there have been more definitions of virtualization than there are storage company start-ups, or perhaps shutdowns. There's a lot of hype around what "it" is. SAN vendors have been saying that SANs are what virtualization is all about. The problem is that the current SAN systems are not including software to handle all storage media types, thus applications that use or should use archive-based media do not benefit from this new "virtualized" world of storage. Almost everyone talking about or offering storage virtualization products limit their offering to magnetic based media, however, magnetic media only addresses part of the storage puzzle. Magnetic media does not address users that have historically used WORM based media, including CD and DVD for archiving and continue to do so. Storage virtualization software must be able to address most, if not all a users need for storage in order to deliver on the promise of ease of use, scalability and cost effectiveness. SANs basically solve a wiring issue, but they don't provide software that virtualizes all storage media available. One thing is very clear: A limitation of virtualization techniques to only magnetic media will limit user functionality and market opportunities for this must-have technology.

Storage Virtualization must include secondary storage devices.

Secondary Storage is a multi-billion dollar business, characterized by its ability to remove the storage medium (cartridge) from the device and store it away from the computer, if necessary e.g. off-site for security.

How valuable is your data?

"One third of businesses that lose all their data fold within 2 years" ...Infocorp

"Two out of five enterprises that experience a disaster will go out of business in five years." ...Gartner Group

"50% of companies suffer data loss through human error" ...Financial Times

"Even with RAID, there is still a single point of failure" ...FORTEL

the RamSan-440 is a 4U RAM SSD delivering 600,000 random IOPS - click for more info
RamSan-440 Enterprise Solid State Disk
512GB RAM SSD, 600,000 IOPS
from Texas Memory Systems

Recent Gartner Dataquest research on application downtime shows that an average of 40% of downtime is caused by application failures (e.g., performance issues or "bugs"), 40% by operator error, and approximately 20% by system or environmental failures. The majority of the failures in the "system or environmental" segment — 60% — is caused by hardware problems. Overall, less than 5% of application downtime is attributable to disasters.

If the following statistic is widely known, few people do much about it: 80% of on-line information is static and unused once it has been stored. Storage analyst and guru Jon Toigo estimates "Corporate information technology departments will spend between 75 and 90 cents of every hardware dollar over the next five years on data storage products".

In addition Toigo goes on to quote an IDC estimate that states 55% of distributed storage management costs are administrative—a number that could be significantly reduced through logical consolidation of storage resources for more effective management. If the static data were migrated to lower management cost near-line or off-line storage technology, then an IT (Information Technology) Manager would gain the benefit of re-claiming a vast amount of their current on-line storage investment.

Gartner Dataquest predicts a move towards managing larger amounts of storage, quicker and in remote site based on recent events. The five-year outlook remains strong for data storage growth — driven by the use of Web-based applications, multimedia data and data warehouse/business intelligence implementations — demands better storage administration tools. Gartner Dataquest forecasts the market will grow at a CAGR of 26.1% through 2005 to reach $16.7 billion.

Primary Data vs. Archive Data

Maybe you've seen this: "It's taken 300,000 years for humans to accumulate 12 exabytes (billion gigabytes) of information. It will take just 2.5 more years to create the next 12 exabytes" – according to an EMC Corp funded study "How Much Information?" by a team of faculty and students at University of California at Berkeley's School of Information Management and Systems.

It is important to point out that the "information" being created is made up of several different types of data and not just data that is stored solely on magnetic disks. There are two types of data that should be addressed here: data that is used for a relatively short period of time (3 to 5 years), such as databases, backups, application programs, day-to-day data, etc; and data that must be stored for long periods of time – archived and compliance data based on self imposed or government regulations.

Here's another interesting fact. Currently there are quite a number of users that were sold expensive magnetic-based systems. They are using these systems to store critical compliance-based or archive data, even though there are regulations requiring the storage of information on archive based (non-erasable) media, and they are paying the fines or penalties for doing so. Many users are told that adding archive media to the system, which was extremely expensive in the first place, is too expensive to do or that the capabilities simply don't exist. They get the advice to add more SAN based storage and back it (up) with off-site systems. This may increase storage, but still doesn't fulfill the requirements for properly storing regulatory data. In addition, as the costs continue to rise, users are more and more reluctant to add the right storage types to the system, which in reality would be more cost and data efficient. Many rationalize these decisions by saying that the system enhancement costs are more than the fines or penalties. This calculation is only true, if the data is never lost or manipulated. If the data is compromised, then the cost to the company could be significant. Doing the math, magnetic is the most cost effective media for data that is changing on a regular basis. Contrary to what many with a specific mission have said, optical media is less expensive than magnetic considering how long certain data has to be kept and maintained. Optical media can be stored off-line and the cost of data expansion is merely the cost of additional media.

Application solution providers that address storage needs using magnetic based media systems have difficulties selling to archive applications. Those offering technologies such as optical (MO, DVD and CD) based media solutions for compliance-based and archive applications tend not to meet the needs of applications where magnetic media is best suited. As a result, users that utilize RAID within a SAN for databases, application or other data end up utilizing separate systems to address workflow, back office records management or compliance based document or data management needs. In a survey of archive and compliance based application users and developers, one thing became very clear: There is a critical need for software to centralize access to a broader range of media types. Storage virtualization should mean total storage virtualization, not just magnetic storage virtualization. The impression is that storage offerings remain disjointed, resulting in less interoperability and greater expense for storage solutions. Users want a system that can search the entire network or enterprise from a single console and map all the available storage resources, whether from one vendor or not, and map those into a single share point or volume.

statutary regulations dictate what media you can use for digital archives
All data is not created equal and should not be treated equally, unless of course you don't mind the thought of losing critical information that is stored on media that is prone to a 3 to 5 year life span. Data lost to vandalism, unwanted changes, accidental loss or loss due to a disaster or user error is increasingly more evident. There are installations that use magnetic based media to store everything, even though there are inherent risks associated with that approach. There are also those applications in records management environments, document imaging and government applications, such as those mentioned above that use media, such as optical (MO, CD or DVD), and even tape to store things for longer periods of time. The fact is that, applications where magnetic based storage solutions are best suited do exist, and there are applications where optical media or some form or tape is the best approach. This is not a battle between the two storage media "camps", although one side may see more value in a single media solution than in offering a true virtualized approach. What's needed is a common sense approach to solving the problems of speed of access, overall cost of the system, accessibility of long-term data, and ease of access, above all, while abstracting out the various layers (media types) and providing a common user interface.

The intention of this article is not to promote one storage technology over the other. It is simply pointing out that users should not have to choose between one or the other. In fact a user does not care how the data is stored so long as it is readily accessible when it is needed. Virtualization of the storage devices should provide a system that handles both categories of data (and probably back-up as well) in a cost effective way and with a single-user interface that is transparent to the user.

Many users are asking for a blending of the various storage technologies, a virtualization. Integrators, developers and users are asking the system integrators, "Let me decide, what goes where, based on my data access requirements and my legal exposure for storing data". There is also a concern about the cost of the overall system and its effectiveness in tracking information and recovering from failures. Certainly there are users that want to be told what is best. That is fine, as long as they are given choices, as each customer requirement is unique to each market they address. A single media solution for all data is not a choice and can be a disaster, if critical information is lost. The fact is that there are solutions available that best fit different data storage requirements, and the "blending" of these technologies would provide the user with the highest levels of performance, security, longevity and cost effectiveness.

A solution may be at hand with the integration and virtualization of the three most common storage categories: primary storage -- magnetic disk (including RAID), secondary storage -- optical disk (MO, DVD and CD), and back-up-- tape. To date, it seems to be more of a battle between magnetic-based disk vendors and the "other" technologies. Everyone is fighting for market share. What's needed is an augmentation of both feature and media benefits that allow s a comprehensive approach to expansion, rather than replacement. Virtualization provides a secure, strategic process for the full spectrum of storage consumer needs, merging security with simplicity, while using an existing company infrastructure more efficiently than it is being used today.
all data is not created equal

Compliance and Archive Data

"Why should I treat this data any different than general day-to-day information?" There are many regulations placed on the storage of data, either self-imposed or government-based. These regulations specifically state what data needs to be stored how. Data that is considered critical is also often stored for long periods of time, such as medical, financial, insurance, securities transactions, customer transactions that may be effected by legal action and so on.

Workflow operations such as banking or medical environments that require maintenance of information for 10 to 15 years or however long the patients or customers are alive are examples of compliancy of regulatory data. Data is brought in via computerized forms based system or scanned and indexed by capture software. Once the data has entered the system – it's critical data. There is a need to store and access that data over a number of years, as much as 50 years or more in some cases. Magnetic media is just not able to keep data readable for that period of time, especially contact magnetic media such as tape, however, many IT managers have decided to "take the risk" and store archive data on tape. Some have discovered through either a restore requirement or just through a database search that magnetic tape does not have a long shelf life and files have gone missing.

A Virtual Solution

Storage virtualization is about providing software tools and connection strategies that automate which particular storage technology should be employed to hold a piece of information according to how it's accessed. Pegasus Disk Technologies provides single software interfaces that can be used to scan the enterprise for storage resources and or volume to facilitate, particularly those used in applications that require storage of data/documents for long periods of time. This type of data, being a significant portion of the data stored globally, is often referred to as compliance based or archive data. It includes medical records, stock transactions, insurance records, banking information, e-mail storage and a wide range of government records. Pegasus offers a simple no-nonsense definition of virtualization of storage hardware for information stored in these systems, showing how such a virtualized storage system can work and how the user or developer benefits from both cost effectiveness and ease-of use-standpoints.

What are the benefits of software virtualization of all these media types?

One of the major benefits of virtualization of the three media types is the cost associated with the various storage resources within the mixed and shared hardware and media pool.

Data that is used and/or changed frequently on a hour-by-hour, day-to-day, or even month-to-month basis belongs on magnetic disk or RAID. Speed of access is critical here, and magnetic-based media is the best fit in this case. However as storage requirements increase, storage costs can skyrocket while all data remains on non-removable magnetic media. It is not uncommon to see SAN based storage systems utilizing RAID storage costing anywhere from a few hundred thousand dollars to well over a million. Of course these systems offer terabytes of capacity, and many are scalable based on storage needs. But when the typical RAID-based system is analyzed, it can be quickly determined that the vast majority of data on the RAID is not accessed at all, much less frequently. This static data can be off-loaded to much more cost effective scalable storage media such as optical, thus cleaning up the primary storage system, and reducing the amount of time it takes to back-up the data on the RAID system. This will facilitate in the use of a scalable removable media device that is much more cost effective to expand while the data in the archives continues to grow.

you might need to archive data for the lifetime of your customer...

...or even longer for medical applications!

Other benefits are that a logical consolidation of storage technologies from a single volume facilitates in fewer IT personnel to manage growing data needs, and a complete use of storage technologies would facilitate in a decrease in operating downtime and the reduction of lost data. It is a fallacy that hard drives do not fail in array products. In fact, RAID exists because hard drives do fail. Simple statistics show that a 50-device array will loose on average three drives per year. If the system is a RAID device, chances are the data can be reconstructed or is mirrored on another drive, but if the device is a JBOD (Just a Bunch of Disks) the data will be lost.

hard drives fail, even in a RAID system
.......... Editor's comments... many readers may incorrectly extrapolate the high reliability of disk drives they see in their desktop PC's, but actually the failure rate of hard drive in a RAID can be much higher, because the head gets more demand from a multi-user operating system, and is probably running in a hotter ambient environment within the RAID enclosure if all the disk slots are occupied. ..........

Secondary storage is more resilient than non –removable storage technologies by design. Should a drive fail in a removable media library, the data will not be lost. The media must only be moved to an operational drive, and voila, the data will be back on-line. Using multiple media types reduces back-up windows. Given the previous statement that removable media is inherently fault tolerant, only primary data need be backed-up. Data moved off to optical media solutions would not need to be part of the back-up strategy, thus reducing the back-up window by as much as 80%.

Storage System Monitoring – Virtualized Resources

Virtualization is abstracting out the specific hardware as well as individual media surfaces and/or types, resulting in a single view or volume for user interaction. The diagram below shows what storage virtualization must be in order to serve on an enterprise, utilizing a multiple or single server-attach approach.

All the different physical storage types on the network can create a significant storage management dilemma. A single logical view of storage using virtualization techniques can streamline data access and speed overall data management. Within an organization, data storage devices or resources of different types can be located either in different locations or centrally located within a controlled environment. With software-based virtualization such as this, storage devices and sizes, locations and methods used to attach them to the network become irrelevant to the user.

At some point, an enlightened storage provider will offer the market a virtualization solution that includes the option of using any or all of these specific storage levels. Appropriate and reliable physical media types will automatically be chosen through the user's application. Based on parameters previously set, the location of the stored data will be transparent to the user. This kind of media blending or virtualization will facilitate in a cleaner SAN that complies with the various regulatory requirements of the appropriate agencies for the specific user market, and will provide a scalable solution over time that is more cost efficient to manage and grows as the data explosion continues.

...Pegasus Disk Technologies profile

Pegasus article image

storage ad click for more info

storage search banner

home storage manufacturers news articles Jukeboxes Backup software, HSM & SRM
STORAGEsearch is published by ACSL