this is the home page of  StorageSearch.com
leading the way to the new storage frontier .....
image shows software factory - click to see storage software directory
SSD software ....
SSD symmetries article
SSD symmetries ..
SSDs over 163  current & past oems profiled
SSD news ..
read the article on SSD ASAPs
SSD ASAPs ..
high availabaility SSD arrays
HA SSDs ..
....

Where are we now with SSD software?

(And how did we get into this mess?)
Editor:- June 11, 2012 - I'm currently reading a book called - the Mystery of the Missing Antimatter - which explores the notion - that if there were equal amounts of matter and antimatter in the universe shortly after the Big Bang - then how is it that we're not seeing more antimatter everywhere now?

I like a good mystery. But I'm still stuck in the early chapters. Who would have believed physicists could make up so many bit part characters? Don't email me to tell me who stole it - or how it all ends.

For many years the SSD software industry was like that too. And it would be reasonable to pose the question...

"If the SSD market is so big and important - then why haven't we seen more SSD software?"

I've been thinking about that topic for a long time. And in the early 2000s when something called HSM was the fashion in storage software - I asked some of these new HSM companies - where's the SSD layer in your standard model?

It seemed to be missing from all their pyramidy diagrams.

It soon became clear it was missing from their thinking too.

Tape, optical and different rotation speeds of hard drives - they could manage those OK in their storage hierarchies - but SSDs weren't part of the picture.

I thought - it's amazing how ignorant storage software companies are about SSDs. But they weren't alone in that respect.

Then a few years later I thought maybe the operating system would be a good place to locate the SSD software.

In 2004 I suggested to a company called Sun Microsystems they could put SSD support in their Solaris OS to make their servers appear faster. They didn't do it.

And a few years after that when Microsoft started talking about putting support for balancing the load between HDDs and SSDs in Vista it seemed like something useful might come from that direction. But it didn't. Later Microsoft sold off its SSD caching technology and it wasn't good enough to make any of its subsequent owners rich.

It was still a mystery to me how we ended up in a situation - where throughout most of the SSD market's history - there was precious little software apart from drivers and some analysis tools?

I would date the start of the true SSD software market at around 2009.

That's when we started to see shipments of products which promised auto-caching and auto-tiering - which I lump together as SSD ASAPs. But there's a lot more to SSD software than that.

With the benefit of hindsight and having talked to many founders of leading SSD software companies I think there were several reasons the software industry seemed to take so long to wake up to the idea of SSDs.
  • the SSD market was too small and fragmented.

    If you're selling software software at somewhere between $100 to $2,000 per license - you need a big established hardware market to fund a software company.
  • everyone thought that someone else was going to do it.

    If you think that SSD management is going to appear in a future release of the big operating systems - then it's not worth investing the effort yourself. Instead you wait to see what the OS companies do - and then hope to create products to fill in the gaps.

    But the big OS companies didn't do anything. They were used to getting technology roadmaps from the processor makers - which told them exactly how much memory and what type of interfaces they would see in a 5 to 10 year lookahead timeframe. And SSDs weren't in those plans.

    Server makers too discovered out they weren't in control of this new SSD market either. The best they could hope for was to buy in SSDs from the market until they could figure out any better plans.
  • SSDs were more complicated than other types of computer hardware - and a fast moving target.

    It's clear from conversations I've had with software developers that many people invested time in understanding complicated aspects of SSDs which later turned out to be irrelevant or short lived.

    It turns out they were mostly chasing phantom problems like block size performance sensitivities and endurance. Good SSDs would deal with those problems internally.

    They should have been thinking at a different architectural level and instead of just asking - how can we make SSDs fit into our old HDD software schemes? - asking better questions like - what can we do with SSDs that we couldn't do before? Or - what new opportunities will SSDs everywhere create? (Some did ask that question.)
For several years - it seemed like the most productive thing that traditional storage software developers ever did for SSDs was to search for and remove countless embedded delay loops from their legacy code which had originally been inserted by performance tuning geniuses (who had since moved on to higher management) to prevent thrashing HDD controllers with too many I/O requests.

But the SSD software market is making up for lost time now.

And in the past year or so - a good marker of that progress has been - by how many SSD software companies have been acquired. The acquirers have been almost always been SSD hardware companies.

Why would they do that?

A simple business answer - is that if the software makes it easier to use SSDs - or easier to use more SSDs - then making it compatible with your own hardware is a better investment than recruiting more sales people or spending more on advertising.

For SSD hardware makers - leveraging software around their products is probably the highest added value activity they can do.

Which is why we've seen small software companies gobbled up for amounts of money which just don't make sense when you look at how little they would be worth valued on license sales.

We're now in an exciting time for SSD software. In the next few years we're all going to hear about companies we've never heard of before - doing stuff - some of which (already) looks predictable - but some of which is entirely new.

What's the role of systems software?

In the past it has always been about optimizing whatever was seen as the most valuable resource.

At various times and in various places that "most valuable resource" has meant different things:-
  • feed the processor,
  • make many processors work as one,
  • make them talk to each other,
  • show us some pretty pictures,
  • manage this unruly population of hard drives.
and managing the data was mostly
  • backup the data (tonight please)
  • stop the bad guys getting to the data (how come those malware guys are so smart at writing software?)
  • unback the data - (who would have thought it would be so difficult).
In the SSD era the most valuable part of the infrastructure is the SSDs.

With SSDs systems designers have the freedom to ask fundamental questions like - if data is so valuable - how can I automatically create even more valuable data and then ensure that it's in the right place at the right time to do good for my business? Maybe selling data can be my business?

Good SSD software will change the rules about everything you can do with SSDs.

Where will you find out about it?

There's always been coverage of SSD software - here on StorageSearch.com - it's just that for many years there wasn't much substance to write about.

The interplays between the memory chips, the controllers, interfaces and applications are poised to branch out in many new directions.

There are many questions still remaining to be answered.
  • where's the best place to put the SSD software?

    In the OS? In the cloud? In the app? In the SSD controller? Or a little bit everywhere?
  • What are the boundaries of where the SSD software should reach?

    Should it have boundaries? Or should it - like the internet - reach everywhere?
  • If the SSD software is so important - then should we be more careful about whose software we buy and what features we use?
  • Will SSD software stabilize into being a commodity - where everyone's software does nearly the same? - or will it be like databases and OS's - where you started out using it because it was convenient - but then 20 years down the road you're locked into a solution that costs a fortune to migrate .
Right now... no-one knows the answers.

But the key people who will create those answers - or choose which casts of the SSD software dice will succeed in the market - like you - also read the pages of StorageSearch.com.
image shows software factory - click to see storage software directory And when the time is right - we'll all find out together. Stay tuned to this channel.
.
re - Your article on "Where are we now with SSD software?"
Hello Zsolt,

I just read your article today on "Where are we now with SSD software?". I found it to be a very interesting read.

In particular, you have touched upon several important items, such as reasons why the software industry has taken so long to wake up to the idea of SSDs - and moreover, the "many questions still remaining to be answered" that you listed at the end of your article.

Indeed, it does appear that we are entering into an exciting time for SSD software. Thanks again for being at the forefront on this important topic.

Best, Tom West , President hyperI/O (May 29, 2012)
....
storage search banner
....
image shows  Spellabyte's software factory
..
SSD ad - click for more info
...
SSD ad - click for more info
...
suggested SSD articles

SSD backup
SSD recovery
the Top 20 SSD companies
Aren't all PCIe SSDs very similar?
Does size matter in SSD design?
What do enterprise SSD users want?
the New Business Case for SSD ASAPs
How fast can your SSD run backwards?
Revamping the IO software stack for SSD
Where does all the money go? (inside SSDs)
What's the future of enterprise data storage?
How hard is it to recover data from dead SSDs?
7 SSD types will satisfy all future enterprise needs
who are Fusion-io's toughest sofware competitors?
.
SSD ad - click for more info
.
How will 2.5 inch PCIe express SSDs change things?
Rackmount SSDs - open vs proprietary architectures
How can enterprise SSD market revenue keep growing?
what do I need to know about any new rackmount SSD?
Why can't SSD's true believers agree about architecture?
if Fusion-io sells more does that mean Violin will sell less?
is eMLC the true successor to SLC in enterprise flash SSD?
Bottlenecks in the pure SSD datacenter will be more serious
Why is the raw to usable capacity in flash SSDs so different?

.
"What you're looking at is the virtual storage which is needed to make things work in an SSD world - which is very different to the capacity needed in the HDD world."
meet Ken - and the enterprise SSD software event horizon

.
SSD ad - click for more info


.
"The decades old architectures of SAN and NAS, originally devised for the physical world, are still the dominant enterprise deployment architectures and are not particularly well suited for the virtual world. These architectures are still based on the notion of a storage box or storage array... The constructs used for storage abstraction, such as LUNs, volumes, files, etc., do not align well with virtual machine constructs."
Enterprise storage is broken! - blog by Maxta (2013)


.
"In 2014 we will see the battle lines for the SSD platform being drawn up - as ISVs and SSD drive makers and SSD systems companies all try to convince you that any long term plans you make will be more future-proof if you use their software."
the big SSD idea changes in 2013?