new SSD thinking inside the box

will lead to better enterprise flash arrays

• SSD software:- 11 companies

• SSD controller IP:- 5 companies

• nv-memory IP (excluding flash):- 2 companies

• SSD systems vendors:- 5 companies acquired - out of which no more than 2 had product lines which were good enough to stay on the SSD race track while changing T-shirts - so they effectively continued to be marketed seamlessly without intense modification or adaptation by their new owners.

• to open up new markets for SSD rackmount arrays by leveraging efficiency and thereby enhancing the reliability and performanceof the whole array - and doing more for less cost.

• to start laying the foundations for an SSD software platform.
  
  Although I've discussed this aspect before in earlier news stories - I won't say too much more of this aspect here because it will be covered in a later article.
  
  OK - I will say a bit more - otherwise you'll be wondering - what's the SSDmouse squeaking about with these tantalizing hints?
  
  One way to look at the evolution of industry standard operating systems is they started as way to virtualize commonly required functionality for processors, and then by virtue of their market popularity - became more important than the original processors - because it was easier to incrementally inrtoduce new apps into markets by co-existing with other popular apps which already did other stuff.
  
  If you fast forward in time 3-4 years when the biggest investment in hardware assets in the datacenter are the SSDs (and not the servers or rotating storage) then the whole world of new systems software will revolve around the view and reach of the SSD software platform.
  
  That means for new apps to work better it will more important to know the SSD software than to know the OS.
  
  A handful of SSD software founders that I've spoken to in recent years have been fully aware of what possibilities that represents for their companies. (Better than being the next Microsoft.)
  
  But to get to the future they still have to cope with and survive the market trepidations of the present.
  
  My guess is that we'll end up with several major SSD software platforms - which will co-exist in some kind of oligopolistic tension. So maybe more like Oracle than Microsoft.
  
  But these platforns will be there because of the de-facto market status of the SSD companies which started them and the sizes of their customer bases.

• what should it do?

• what will it be used for?, and

• how can I be sure that my new design is the one which users prefer to buy - instead of all those others - which have been designed by my competitors who have access to exactly the same memory chips?

• also - if I want to maintain control of my company and my present position and don't want to spend more time talking to investors than I do talking to customers - how can I be sure that my design will be more profitable than all the others?

• the enterprise SSD market used to be simple to characterize.
  
  Users bought SSDs for a limited set of reasons centered around speed. And those users could afford to identify that problem - they could often afford to solve it too.

• the enterprise SSD market used to be small.
  
  So there was little or no incentive to design a different kind of product for a niche.

• the number of design dimensions available to play with - within the IP of a typical SSD rack used to be restricted.
  
  It was either mostly custom hardware - with little or no software playing inside,
  
  or it was mostly me-too hardware - with arrays based on commonly available (commercial off the shelf) SSD drives - with some cute SSD software tricks to spice up the reliability and ease of integration with legacy installations.

• Market Trends in the Rackmount SSD Market (2009).
  
  The main idea to take away from this article is that there will continue to be diversity in the internal architectures and detailed features of rackmount SSDs due to valid different preferences among end-users.
  
  This is something which often mystifies SSD marketers who are new to the enterprise markets and come from electronics or semiconductor components backgrounds so it needs repeating.
  
  It means users will choose SSD systems which are technically very different on the inside - even when their apps requirements (for performance and capacity and compatibility) look superficially the same from the outside.
  
  That's because other factors outweigh these top level headline tech specs - such as
  • completely different operating assumptions about the need for upgradeability or fault replacement at the module level (ranging from mandatory to irrelevant depending on whether the atomic change unit is a drive or the whole rack),
    
    
  • different assumptions about the needs for the SSD to be a good datacenter citizen.
    
    Is it one of few SSDs in the customer's population - so it'salready better than the hard drive RAID which was there before - or it one of thousands of similar SSD racks - so the user has to analyze small differences - such as density and power consumption - because they will add up to big effects
    
    
  • the usual differences - which have always been factors in the enterprise market - which involve judging whether it's worth taking a risk to use a newer cheaper solution from a less well known supplier - or pay more and wait for the new technology to trickle down through established suppliers

• an introduction to enterprise SSD silos (March 2012).
  
  This article provides a useful framework for grouping all the enterprise SSDs which exist now or which will exist in the next 5 years into 7 sets - based on what roles they will have in a pure solid state storage datacenter.
  
  The categorization scheme is based on relativistic characteristics - and it describes a heirarchy of different types of distinctly different SSD system types.
  
  It's not a simple up down heirarchy. It goes sideways too.
  
  My silos system is based on the firm belief that no single enterprise SSD product will be technically able to meet all forseeable enterprise market needs at a price which is affordable to all the people who need and can afford SSDs. SSDs will be segmented and optimized for different roles. Most large end user sites will use many of these differing SSD types.
  
  The differences in these SSD classifications - which is still hard to spot in some of the systems being offered today unless you're an expert - will clarify as these product segments grow into bigger markets.
  
  That's because vendors who don't understand where they fit into these silos will go bust or exit the market due to the inefficiency of offering products which don't satisfy needs economically - by trying to satisfy too many needs unrealistically.
  
  If you're making supplier shortlists it's a great aid.
  
  I've already heard from readers who are big consumers of SSDs they find it useful. I also know that many SSD company CEOs and VPs of marketing find it useful too as a way to map where they are in the competitive landscape and who their real competitors are.

• the guide to SSD guides - among other things includes a short month by month summary of key product news in the past 12 months or so.
  
  If you're interested in companies mentioned in that article - clicking on their name will give you a more detailed SSD-centric profile. But the guide also includes lists to top level directories - such as extracts from news about high availability SSDs, iSCSI SSDs etc.
  
  It's far from being perfectly complete - because if a company isn't at the leading edge of competitiveness - or if their marketing is dozy - they may not get mentioned at all.
  
  But diving into these pages and reading the comments etc will give you a taste of what's available in these product groups and issues which surround them.

• faster performance - 3x to 10x
  
  There are many different ways of doing this.
  
  The leading advocate of ditching legacy APIs - Fusion-io - has always said that customers who go down this road with their solutions often see speedups in the 10x and upwards range. Having said that - the company also has also been working on parallel routes to market in legacy compatible rackmount storage markets too which are more oriented towards cost savings than simply speed.
  
  Skyera says its way of doing RAID striping involves 3x less writes than tradional RAID-6. (There are many other companies who also do something similar too.) This technique means better peak performance loonger operating life too.

• using less chips - while providing the same usable capacity at the same speed and operating life.
  
  How many less chips?
  
  That percentage can range from 30% to 50%.
  
  Violin often used to say that was a competitive advantage which came from their architecture.
  
  But although it's easier to achieve in big SSD controller architecture in hardware - it can be accumulated by incremental differences in software too.
  
  These kinds of efficiencies can be gained in many parts of the array.
  
  Hot spots for gains for all array vendors are in the areas related to fault managment - particularly in the way that data is striped across memory devices to provide RAID like protection.
  
  More about this and other factors such as over-provisioning rations and ECC efficiences are discussed in the SSD capacity iceberg syndrome.

• better reliability.
  
  Whiptail says that a combination of RAM caching and software efficiencies means that logs from its legacy SSD customers are indicating usable operational lives - from a flash endurance viewpoint of 7 to 8 years.
  
  Skyera says that the combination of the techniques used in their boxes effectively deliver SSD array life which is 100x better than suggested by the raw life of the flash chips they use. They aren't suggesting that customers will use their systems any longer - instead it 's one of the factors which enable them to build an SSD box with one of the lowest costs per TB in its class.
  
  Nimbus uses high level software in its Halo OS as part of the endurance management. I learned that their SSD racks support 50x full capacity writes each day for 5 years - which is significantly better than the raw endurance of the SAS SSDs in their arrays.

• lower system cost - upto 10x lower cost - to do the same things
  
  In addition to the efficiency gains mentioned above which reduce the number of raw memory chips needed to deliver each terabyte of usable physical capacity - some vendors also include real-time dedupe and compression as functions which can be turned on or off, or application specific optimizations.
  
  In Astute Networks new system which is heavily optimized for iSCSI environments - the combination of IP accelerator hardware coupled with dedupe enables them to offer fast-enough virtualized storage at under $2,000 / TB.
  
  Another vendor - still in stealth mode has been telling me recently about software technologies which will enable million IOPS class SSD racks with several GB/s throughput using a combination of dedupe, compression and other techniques to deliver systems that will cost in the region of $100 to $200 per network virtualized terabyte. We'll have to wait and see how that looks in a real product.

• doing much more than the sum of the parts

• doing much better with less
  • less memory,
  • less resources,
  • less useless thrashing around,
  • and much lower costs too