How will hard drives fare in an SSD world? - article on StorageSearch.com


leading the way to the new storage frontier	.....


SSD news	..


HDD news	..


top SSD companies	..


PCIe SSDs	..


2.5" PCIe SSDs	.....

How will the hard drive market fare...

in a solid state storage world?

by Zsolt Kerekes, editor - November 13, 2012

This article looks at an evolving set of storage market adoption models which - in the past - had to explain why SSDs would find a place in an HDD dominated world.

And it joins the dots from - where we have been - and are now - to project ahead to a time when it's the hard drives which will struggle to find viable market gaps in a storage world dominated by SSDs.

How will the hard drive market fare...
in an SSD storage world?

I only write about SSD related topics that I'm thinking about or am interested in.

So the last thing I ever thought I'd be doing in the last quarter of 2012 is writing yet another article about hard drives - especially in this slot - the home page of StorageSearch.com - which has been reserved in recent years for thought leading SSD articles.

It's nearly 3 years since I last wrote anything substantial about HDDs - and the reason is simple. My analysis had led me to the conclusion that whatever products or technologies might come out of the hard drive market in future years - would have no significant impact on slowing down or preventing the ultimate growth of the solid state storage market. And I've only got one brain - which defies all attempts at overclocking and parallelization. Struggling to understand and make sense of the SSD market is complicated enough - so I could see no more reason to waste time thinking about or writing about HDDs - except occasionally to note when some particular event demonstrated another phase in the technology's displacement.

the past of HDD vs SSD sophistry

Although the idea of technically replacing HDDs by SSDs went back decades - 2005 was a bloom year for this type of article. Apart from some rare exceptions - most of the published "analysis" on this theme I had seen was transfixed on the moving relative target of HDD versus flash memory price / capacity curves - even though it was known to some business experts in the SSD industry that other factors could be just as important.

I didn't claim any originality for my 2003 article about SSD-CPU equivalence - I merely used it to signal to readers that the total available market for enterprise accelerator SSDs could about 100x bigger than the actual revenue of the market at that time. In fact it was a surprise to me that others hadn't done the same kind of calculations. And that was a special case of SSDs replacing servers - and not - SSDs replacing hard drives.

My own thoughts on the subject of SSD adoption and HDD displacement evolved as a dialog with readers over a period of many years and articles.

2005 - why buy SSDs? - 5 user value propositions - the 2nd generation SSD market adoption model from StorageSearch.com - which analyzed and described the different reasons for SSDs use - and which is still a good outline analysis today.

A key conclusion of this analysis was that in most applications for SSDs the price per gigabyte of SSD compared to HDD was irrelevant to understanding SSD market adoption and was not the right competitive comparison to make.

2007 - How Solid is Hard Disk's Future? - suggested a capacity ratio based formula for anticipating where SSDs would replace HDDs in consumer markets. (It still stands the test of time today.)

The article also said that due to the small size of the SSD market relative to the HDD market at that point - it was likely that both markets could grow in revenue at the same time for many years as this was "not a zero sum game" (even if the growth rate of the SSD market was very high).

2010 - this way to the petabyte SSD - looked ahead to a time (which I guessed to be around 2016) when I anticipate that SSDs would significantly replace HDDs in datacenters (in over 50% of new systems) in "the last bastions of hard drives in the datacenter" - as bulk online storage.

My analysis showed that even in this most cost sensitive application - the changeover would occur (based on extrapolating known technology and market trends) even if the cost of hard drives continued to drop rapidly - and even if hard drive capacity was free!

That stunning conclusion - based on SSD market boundary analysis - was the clincher for me. And after the article was published some SSD company CEOs told me it was a Eureka moment for them too.

You can argue about whether that 2016 date still looks appropriate from today's perspective. There are arguments for saying it will come sooner in some markets and later in others.

But I didn't entirely stop thinking about hard drives in an SSD world.

From time to time HDDs have sometimes popped up in a supporting role in 3 other types of news story and article.

2005 to 2012 - Hybrid storage drives (modules / components) were seized on by hard drive makers as an attractive tactic for carrying on business as usual and staving off the threat of SSDs by treating flash as just another special type of buffer or cache in a hard drive.

But this delusion meant that HDD makers delayed far too long in recognizing and grabbing the real market opportunities of SSDs.

While the hard drive makers wasted their talent on designing expensive, complicated and undesirable rotating magnetic disk/flash hybrids - the true SSD companies just got better at developing native SSD technologies and markets which evolved out of the reach and ken of HDD companies - except by way of SSD company and IP acquisitions and licensing.

Customer case studies of the - shooting HDD fish in an SSD barrel stories variety - by which I mean the kind of story put out by SSD accelerator companies about how one of their customers replaced short stroked hard drives with an SSD rack (or two).

This is a marketing communications practice I grew so tired of that I bemoaned it in a 2008 article - Calling for an End to Unrealistic SSD vs HDD IOPS Comparisons in which I bluntly stated that the real competition for fast enterprise SSDs is other fast SSDs.

When SSD vendors publish case studies about replacing old rotating storage arrays - as indeed they sometimes still do - it doesn't impress anyone. And it demonstrates that even SSD companies don't always understand the mood of users in the SSD market.

2009 to 2012 - Auto tiering / caching appliances (SSD ASAPs) come in many shapes and sizes.

In 2010 - I realized that these SSD ASAPs would always be needed - even in a pure SSD storage world.

From that time onwards - in my editorial, talking to readers and in my thinking too - I consciously and sometimes unconcsiously - segmented these companies into 2 types.
- those with an HDD weighted outlook - either because they leveraged a lot of HDD specific IP, or because they had an HDD-oriented center of gravity in their marketing and business development thinking.
  
  In my SSD elitist view I think of this category of SSD caching/tiering vendors as the rotationally-fated / doomed to crash when the HDD market spins down. They will find it hard to adjust to a zero RPM storage future.
- those with an SSD weighted outlook - whose architecture and marketing can work just as well in the context of a pure solid state storage market as in the short lived market operating alongside enterprise hard drives.
  
  Although they'll still find it tough competing to establish their place among the several leading SSD software platforms of the future - they could become the entry point through which millions of users view the rest of the SSD market.

So - when I say that in the last 2-3 years:- "I stopped thinking and writing about hard drives" - what I mean is that compared to the many hundreds of articles I was writing about SSDs - and content with my view that hard drives posed no new or insurmountable challenges for the SSD market it was rare that I gave them any thought.

But in the 2nd half of 2012 - a strange thing happened.

I was getting more reader emails asking me to clarify the phenomenon which they loosely called "HDD's displacement by SSDs".

As you know - I would never call it that - because I expect that in the long term the SSD market will be bigger in revenue than the HDD market ever was.

SSDs do more than replace hard drives and servers. - SSDs enable the creation of new data markets which could never have worked in an HDD only world - because the new apps rely on latency and data artchitecture processing assumptions which are on a performance scale which is impossible or uneconomic to achieve with storage which isn't solid state.

Anyway - when readers asked for my opinion about the demise of HDDs I knew what they meant.

As I learned more about where these readers were coming from I got less surprised at getting these questions..

investors in hard drive storage makers are wondering what impact all those billions of SSD dollars might have on HDD purchases.

readers who are new to the SSD market probably haven't seen my older articles.

even readers who had read my earlier SSD adoption articles felt uncomfortable with making big leaps - across the chasms from where my old HDD narratives had stopped - to the next place. Whatever that next place might be.

here's the new stuff

So, without wishing to repeat too much of what I've already written before here's a set of condensed notes (extracted from recent emails) in which I have described how I see the HDD-SSD thing working out in the next 2-3 years. As with any technical clairvoyancy - I caution you that it could be wrong. But these are the best guesses I have to offer - until better guesses or reality replace them.

will HDD use migrate to SSD?

No.

It's not exactly like that. In the long term the SSD market will be bigger in revenue than the HDD market ever was.

Until recently most of the use of SSDs in enterprise servers have been replacing server budgets rather than storage.

enterprise

We're at a turning point where SSDs are starting to be used by early adopters in the enterprise as pure bulk storage devices. But we're still 3-4 years away from that becoming mainstream. That's because we are also in a market today where if an oem has a choice of how to repurpose 10,000 flash chips in a box they can make much more money by packing those flash chips into a fast SSD rather than a low power consumption bulk storage SSD.

In the next 3 to 4 years - some vendors will move closer to the fast-enough and cloud / bulk SSD silo because they cant do the fast stuff and others will choose it deliberately. The technologies already exist to widen and clarify these roles. But as long as enterprise SSD makers have the easy ride up the tornado they will defer making difficult choices.

When some SSD market leadership positions become entrenched - in the next few years and when competition between SSD rack makers becomes a really much tougher issue than it is now (when it's so easy to find new customers who don't already have legacy SSD allegiences) - the vacant lot of - "doing economic HDD replacement" - without also having to bundle in massive performance too will appear more attractive as a product marketing option - for those seeking new markets. That's the market model for the bulk storage / 1U petabyte SSD.

consumer

In the notebook market HDDs currently have a role in entertainment pcs where the device is holding a significant amount of video, music etc and where performance (speed or battery life) are not prime requirements.

The entertainment PC will fragment into users who have different preferences and options re cloud vs having it in their hands storage.

Do users need to have hundreds or thousands of movies on their PCs? or will most of them decide its OK to have 50, 100, 200.

I think the convenience of streaming video and movie services means most consumers will be satifisfied with a hybrid approach which is having enough local storage for the legacy stuff theyve already got (and like to watch in the hut or listen to in the car) while relying mostly on the cloud for the newer stuff. That would lead to a glass ceiling for HDD capacity in PC like systems - above which the capacity wasn't necessary. Which means the SSD can creep up and take over.

As for the cloud that will eventually all be solid state because of cost of ownership.

The HDD market isn't going to disappear in the next 5 years because hard disk makers will find new markets. Its just that the most profitable markets they used to have in the datacenter will fade away.

will HDD revenue decline gradually or abruptly?

This depends how markets behave - and not just the potential of the respective technologies.

As I predicted in my 2003 article about SSD-CPU equivalence once a single major server oem adopted factory fitted SSD acceleration all the others would follow very soon after or become uncompetitive. That turned out to be FIO being adopted by HP, IBM, Dell etc It was the switch from seeing SSDs as a threat to CPU sales and instead a necessary and better product for server makers to sell.

When HDD sales collapse in the server market I anticipate it will be a catastrophic fall on the order of 30% to 50% within a single year.

The precise placement of that year could be 2015/16 which is what Ive been saying in earlier articles published in 2009 and 2010.

A lot of servers are already going out the factory door with SSDs fitted where previously there would have been SAS HDDs. When server vendors have proved for a year or so that SSDs as singles or in arrays get a good customer reaction and when the new value engineered efficient architecture enterprise SSDs have reached compelling price advantages compared to classic enterprise SSDs and when a single array maker like IBM discloses it has shipped billions of dollars of RamSan SSD racks confidence in the SSD market will increase.

Another factor is apps.

Before long all enterprises will rely on software elements which can only work with SSDs. HDD arrays wont be able to run the new generation of killer enterprise software because HDD IOPS are too slow. Its the combination of these several factors which will lead to collapse of demand in enterprise SSDs. Any one factor can be mitigated.

is cost per terabyte the best way to think about SSD cost?

No. Capacity is only one dimension of SSD cost - and not always the most important. In my 3rd generation enterprise SSD market model (SSD silos) makes a good applications and architecture based case for upto 4 distinct speeds of SSDs having sustainably different applications roles within an all solid state datacenter. (With another 3 specialist types of functionally different SSDs inter-operating among them.)

The value propositions within these distinct SSD silos varies - according to the role that these specialist SSDs will have.

If you step back and look across the whole SSD infrastructure investment in the enterprise - a useful top level cost metric might $ per user who gets a satisfactory apps experience.

That's going to be a different number in different markets and in different applications.

At the extreme capacity end of the SSD market - archive storage - for example - getting the lowest power consumption and maximum capacity per rack unit will be the main factors - and speed will almost be irrelevant. Whereas - in the ultra-fast end of the apps acceleration architecture - the prime determinants of price will be speed factors such as low latency and high IOPS rather than capacity. And that's just the enterprise market.

The same SSD capacity will have a different market price in military, industrial and consumer markets too. Because the SSDs are very different products with different market specific attributes (such as reliability, ruggedness, security) and ease of integration into the required apps / use cases. And even within the consumer gadget market - we could start to see different levels of acceptable price / performance SSDs in different types of product.

This is a completely different world view than for hard drives - which have been fossilized into a small RPM based differentiated model for more than 10 years.

A better analogy for SSDs is comparisons with processor pricing. But SSDs are more complex than CPUs and have more dimensions of apps related functionality.

conclusion

There's a lot more to the SSD story than its role in displacing hard drives in storage. SSDs enable new types of data processing architecture and are one of the 3 most disruptive electronics based business enabling technologies in the past 40 years.

It looks like you're seriously interested in SSDs. So please bookmark this page and come back again soon.

Also - if you've got the time - you might want to take a look at the top 100 most popular SSD articles seen by our readers in the last month or so.

....

the top SSD companies
11 key SSD symmetries
7 apps silos for enterprise SSDs
where are we now with SSD software?
the survivors guide to enterprise SSDs
8 years of flash SSDs in the enterprise
2012 in SSD - month by month summary
Efficiency - making the same SSD - while using less flash

....

"SSDs are only as good as
the people who design
and make them."

bad block management in flash SSDs

....

"Even SSD's staunchest evangelists
fundamentally disagree about
many aspects of SSD's future
and the best ways to get there..."

the SSD Heresies

....

"Don't place too much faith in what SSD companies tell you about the present or the future of the SSD market."

Survivor's Guide to Enterprise SSDs

....

"You'd think... someone should know all the answers by now. "

what do enterprise SSD users want?

....

flash SSD capacity - the iceberg syndrome

Have you ever wondered how the amount of flash inside a flash SSD compares to the capacity shown on the invoice? What you see isn't always what you get.

nothing surprised the penguins - click to read the article

There can be huge variations in different designs as vendors leverage invisible internal capacity to tweak key performance and reliability parameters. ...read the article

....

"Like cosmic dark matter - the mass of SSD dark matter will be bigger than anything which we can currently see or foresee."

The big market impact of SSD dark matter

....

"It's like putting the wrong tires on your car. Everything's OK on the daily commute. You see the difference only when you hit snow or drive around a sharp bend at 120 mph."

endurance issues in flash SSD operating life

....

"Little words can have with big meanings in the world of SSDs. They affect price, performance, reliability and user happiness."

flash SSD jargon explained

....

"nice vs naughty flash"

sugaring MLC for the enterprise

....

"In tomorrow's data-driven content factory, storage will be regarded as a profit center and not an overhead."

the future of enterprise data storage

....

"When it comes to fault tolerance - you have to analyze SSDs as data systems devices and not as electronic components."

FITs - reliability abstraction levels in SSD modeling

....

"7 ways to classify where all SSDs will fit in the pure SSD datacenter"

an introduction to enterprise SSD silos

....

When thinking about SSD market boundary conditions the starting point is often... this is what we expect most people to do. But what if we change some of the assumptions? Maybe stretch them to breaking point. Is there a point where the market would behave in a completely different way? And what can we learn from that?"

Boundaries Analysis in SSD Market Forecasting

....

"Why is a terabyte SSD from one company 2x, 5x, 10x or even more than 100x more expensive than another?"

Clarifying SSD Pricing - where does all the money go?

....

"Understanding those differences tells you (nearly) everything you need to know about the strengths and weaknesses of these products in different markets and applications."

Why size matters in SSD architecture

....

"The new new economics of SSD storage took the CPU designers and OS software developers by surprise."

the New Business Case for SSD ASAPs

....

"Is Lenin dead yet?"

MLC flash lives longer in my SSD care program

....

"If I selected 3 competing models of PCIe flash SSDs I could design 3 comparative benchmarks in which each product ranked at the top, bottom or middle of the 3 different top 3 fastest lists. "

the 3 fastest PCIe SSDs list(s)

....

Beware the distorting lens of viewpoint as in - "SSDs are similar to..."

Can you trust SSD market data?

....

new LSI blog on the value of enterprise flash

Editor:- March 14, 2013 - You won't be surprised to see me mentioning a recently published blog by Robert Ober, System and Processor Architect, LSI - about the value of PCIe SSDs in big datacenters - which includes these statements:-

"Work/$ is the correct metric (and not crazy expensive $/bit)."

"when users say - $8k PCIe card in a $4k server really? - I am always stunned by this"

I'm guessing that the title of Robert's blog - What are the driving forces behind going diskless? Will 100% flash storage make sense in enterprise? - was either inspired by SEO considerations (stuffing the title with value-loaded words for search-engines) or was predetermined before the blog was written.

I prefer this alternative title - suggested by a banner graphic in the blog itself - An $8K PCIe card in an $4k server - huh!?!

....

"It's clear now why it took so long
for the software industry to
wake up to the idea of SSDs."

where are we now with SSD software?

....

"What I'm proposing is a simple sort filter to help you decide whether you should invest more time reading about new SSD products which fall under the enterprise SSD" umbrella."

Legacy versus New Dynasty?

....

SSD HDD ratios in big data? - blog by Panasas

Editor:- December 11, 2012 - "An important piece of our research involved determining how much SSD capacity customers would need...

" and whether it would make a big enough difference in system performance to be worth the incremental cost of including SSD storage in the system. To do this, we extracted key data from production file systems in the field..." - says Geoffrey Noer, Sr. Director of Product Marketing at Panasas in his recent blog SSDs and parallel storage (part 3) - in which he also says that in HPC workloads users can get good enough results by using as little as 1.5% ratio of SSD to HDD - compared to (read his blog if you want to see the exact ratio) needed for financial and other markets.

Editor's comments:- Although I was told about the earlier episodes in this blog series - which includes an introduction to SSDs - I didn't mention them before - because I assume if you don't already have a good idea of what's an SSD? before you get to these pages - then you won't linger here very long.

Another reason for my earlier reticence about these Panasas blogs - is I think that a description of SSDs written from the viewpoint of an SSD ASAP company whose boxes average over 90% hard drives - may be subtly tilted to a perspective which I consider to be a sideline along the enterprise SSD adoption road rather than the straightest way to the final destination.

Nevertheless - to give credit where it's due - some of the other past blogs I've seen from Panasas have included valuable insights which are storage media agnostic.

I've said to a couple of readers recently (including a Panasas customer) - that if I had to make a list of the last 5 companies on this planet which I thought would still be shipping hard drives in arrays into the enterprise - then Panasas would be one of them. That's because a small group of companies have invested enough talent into new ways of managing large HDD populations in a more effective and efficient way than the small controller architecture of classical RAID systems.

It will be lonely for the HDD array sales guys when that happens though - just as it has become quiet lately in the tape library industry conferences.