Enterprise SSDs - the Survive and Thrive Guide

• Don't believe everything SSD companies tell you about the past, present or future of the SSD market.
  
  If SSDs are going to be a big part of your business future wouldn't it be nice to have a world view or future outlook for the type of products you're looking at?
  
  It's been a tradition in the computer market that users and integrators look to see what their suppliers say about the future of the market and defer to those visions. Other good places to look for answers were universities (theoretical concepts) and market research companies (numbers). None of that works with enterprise SSDs.
  
  This is disruptive market where what happens next doesn't follow on the smooth curve trajectory of what happened before.
  
  I talk to a lot of SSD business leaders about the market. There are a lot of different views out there and they aren't always compatible. But - right or wrong - an SSD vendor's world view of the SSD market does determine where they focus their resources and also the types of SSDs they make.
  
  Here are my paraphrased shortened versions of what some companies have said in the past
  
  
  • PCIe SSDs will make the SAN obsolete - Fusion-io
    
    
  • there won't be enough flash manufacturing capacity to replace hard drives - Seagate
    
    
  • because we make the memory - we can design better SSDs - Micron
    
    
  • PCIe SSDs don't give us any performance advantages in our racks compared to SAS SSDs - Nimbus
    
    
  • SSDs will coexist in harmony with HDDs - many vendors
    
    
  • SSDs will make HDDs obsolete - many vendors
    
    
  • SSD have already rendered HDDs obsolete in arrays - Pure Storage
    
    

• Software used to be SSD's enemy. Now it can be SSD's best friend.
  
  Legacy storage software used contained a lot of hard wired assumptions about the best way to optimize performance - which were based on the intrinsic characteristics of hard drives:- your grandfather's generation of random disk access time, but oversprayed with modern tones of streaming R/W thoughput once you got to the right cylinder. That meant when users connected modern SSDs to traditional software they often got performance which was less than they expected - due to many hidden slugs and choke points buried in the code.
  
  Modern SSD software makes it much easier to extract the maximum performance from SSD hardware (when using legacy apps and legacy storage hierarchies) and - if you're building new SSD apps for which performance is the business game-changer - then using new SSD APIs can deliver latencies and IOPS which are 20x faster than the best you can get with legacy compatible APIs.
  
  Whether your SSD app is legacy or new dynasty (and some of you will have both in your organizations) you'll find that new generation of SSD-aware software will get you SSD acceleration which is faster and sooner.
  
  Is there a risk of vendor lock-in with new API's? Sure there is. Some SSD products are very sticky.
  
  That's good news for some - like investors in sticky leading SSD companies - bad news for competitors.
  
  If you're a user who's worried about vendor lock-in then you can stay on the legacy side of the SSD software fence. There's plenty of support for this choice. You won't get the ultimate performance - but you'll get more performance than you get now.
  
  But in the next 5 years all enterprise systems software will migrate to being an SSD and cloud biased model instead of just HDD biased. And all cloud storage infrastructure will become SSD - even if it's being marketed as "slow HDD type." For more see - where are we now with SSD software?

• The memory inside most enterprise SSDs is irrelevant.
  
  What I mean by this is - If you're a user or integrator you don't need to worry much about 1X, 2X, 3X nm and SLC, MLC, TLC in most well designed enterprise SSDs (upto and including fast-enough speeds).
  
  Good design is more important than good memory. A clever designer with generation X memory can craft an SSD that competes with generation Y. And the other way around. It's the surrounding SSD IP sets which matter more.

• In SSDs - rules are made to be broken.
  
  Don't get too attached to any particular set of safe, handy rules to guide you.
  
  In recent years we've seen that each new significant generation of flash memory was only made possible - not simply by shrinking chip geometries - but by changing the accepted rules of what was safe in a memory chip design. Supporting those changes were assumptions about the lengths to which flash controller designers would go in order to live with the new memory defects and hide them in order to get increased storage density at lower price points.

• The most important technology factors in the enterprise SSD market today and near future are:-
  
  
  • the ability to leverage software
    
    
  • the ability to customize commodity flash by techniques evolved from adaptive write cycles coupled with DSP IP
    
    
  • unifying high availability and high performance into a single seamless competitive architecture (or plausible segmented marketing model).
    
    
  • the ability to remarket and productize big or small controller architectures outside their original comfort zones.
    
    The comparative weaknesses of small controller architecture are inefficiencies in raw to usable capacity, reliability and performance. DSP IP - which is shipping in smaller drives first - may be a competitive equalizer at the rack array level. (I discussed the impact using STEC and FIO as examples in the recently published Top SSD Companies.)
    
    Vendors with large controller architectures will have to remarket more of their IP in smaller form factors - such as PCIe SSDs and 2.5" PCIe SSDs - in order to get better economies of scale. Texas Memory Systems already does that. Violin doesn't yet. Bitmicro - will probably launch their new big controller as a module or card - instead of a rack.

• Flash will remain the highest capacity memory element in SSDs for another 3 to 5 years
  
  While it's true that flash shrinkability will slow down - so you won't see so much more usable capacity coming from the raw flash each year - compared to the gains seen before - many of these physics limited scaling issues will be more than outweighed in the balance by better architecture, packaging and software.
  
  For example:- in many of today's SSD racks - upto 50% of the raw capacity isn't usable capacity - because it's allocated to reliability and performance roles. Those inefficiences can be reduced by application specific technologies. That's as much benefit to users as a significant die shrink.
  
  And we'll see power management, TLC (3 bits per cell), real-time compression and dedupe all play their part in helping to keep flash density competitive compared to other storage media in most roles.
  
  That's not to say that RAM and other types of nv memory won't find their own markets too. But they'll have niche roles in the SSD market such as fastest SSDs and inside flash SSDs as regular caches.

• The enterprise SSD market will become more complex and confusing - before it becomes any simpler.
  
  If I thought it was going to become boring and predictable - I would've hung up my keyboard and written my last article about the SSD market a long time ago.
  
  That's not going to happen anytime soon. Instead as more people become aware that the transition to solid state storage in computer architecture is 1 of the 3 most important and exciting things which have happened in the digital electronics market in the past 40 years - we will see even more mind boggling shifts and turns in these titanic struggles.
  
  At stake is what can you do with data?
  
  The data factories of the future will be as significant to world economies as the auto factories of the past. SSD is the pivotal factor which will move us from the iron age of data processing to the thought age - in which much more will be possible - if you can code it - at the speed of a click. (And sometimes - anticipating what you will probably decide to click on - will give you the data before you press any buttons at all.)

• what do enterprise SSD users want?

• an introduction to enterprise SSD market silos

• How Bad is - choosing the wrong SSD supplier?