this way to the petabyte SSD - 2016 to 2020 roadmaps

by Zsolt Kerekes, editor - StorageSearch.com - March 2010

• SSD market adoption model (2005 edition) - in which I described why users would buy SSDs. This analyzed the user value propositions for the 4 main markets in which SSDs would be adopted in the decade following publication.

• How solid is hard disk's future? (2007) - in which I explained that the growth of the SSD market wouldn't result in any sizable reductions in overall hard disk market revenue till about 2011 - because the HDD market was itself gaining revenue from new consumer markets such as video recorders almost as fast as it was losing revenue to SSDs in notebooks and embedded markets.

• MLC flash SSDs (the best looking horses in the theoretical race to achieve HDD and SSD price parity) will not play a significant part in dislodging hard drives from their use in archives for reasons explained later in this article connected with data integrity.
  
  Later note added in 2012 - I was wrong about this. But the fact that other nand flash can do fast-boot reliably - doesn't change the conclusions in the article. The use of adaptive R/W DSP in MLC flash will enable TLC / QLC to play the same market replacement role which I originally anticipated would require mainly powered down SLC with fast boot. But apart from this memory type difference - the market impact and architecture assumptions below remain unchanged.

• New emerging applications and market conditions (in the 2011 to 2019 decade) will put much greater stress on archived data.
  
  These new search-engine centric applications will accelerate the growth of data capacity (by creating automatically generated data flow patterns akin to ant-virus strings matching - which will speed up the anticipation and delivery of appropriate customer data services).
  
  At the same time these new apps will demand greater IOPS access into regions of storage which have hitherto been regarded as offline or nearline data. Those factors would - on their own - simply increase storage costs beyond a level which most enterprises can afford.

• The ability to leverage the data harvest will create new added value opportunities in the biggest data use markets - which means that backup will no longer be seen as an overhead cost. Instead archived data will be seen as a potential money making resource or profit center. Following the Google experience - that analyzing more data makes the product derived from that data even better. So more data is good rather than bad. (Even if it's expensive.)

• Physical space and electrical power will continue playing their part as pressure constraining corsets on these systems. That's why new SSD architectures will be needed - because current designs are incapable of being scaled into these tight spaces.

The 4 propositions discussed in this article are:-

• what kind of animal will the PB SSD be?

• who's going to buy it - and why?

• where will it fit in the datacenter storage architecture?

• what are the technical problems which need to be solved?

• looking back at the last bastions of magnetic storage

• acceleration SSD - close to the application server - as a DAS connection in the same or adjacent racks (via PCIe, SATA, SAS etc.)

• auxilary acceleration SSD - on the SAN / NAS.

• bulk storage archive SSD - whose primary purpose is affordable bulk storage which is accessible mainly to SAN / NAS - but in some data farms - may connect directly to the fastest servers.

technical problems which need to be solved

...introducing a new species of storage device - the bulk archive SSD

This is a very strange storage animal which - although internally uses nv memory such as flash - and externally looks like a fat 2.5" SSD - looks as alien architecturally to a conventional notebook or server SSD as does a tape library to a hard drive. The differences come from the need to manage data accesses in a way which optimize power use (and avoids the SSD melting) rather than optimizing the performance of data accesses.

The requirements of the controllers for SSDs in bulk store applications differs from those in today's SSDs (2010) in these important respects:-

• optimzation of electric power - the need to power manage memory blocks within the SSD so that at any point in time 98% are in sleep mode (in the powered off state). I'm assuming that the controller itself is always in the mostly awake state.

• architecture - internally each SSD controller is managing perhaps 20 to 50 independently power sequenceable SSDs. In this respect the 2.5" SSD architecture resembles some aspects of a mini auto MAID system.

• endurance - data writes to the library chips are nearly always in large sequential blocks (because it's a bulk storage appliance) therefore write amplification effects are a lesser concern than with conventional SSDs. Also the main memory is SLC - not MLC due to the need for data integrity. That's partly because the thousands of power cycles which occur during the life of the product - which can be triggered by reads (not just by writes) would lead to too many disturb errors - and also because the logic error bands in MLC thresholds are too small to cope with the electrical noise in these systems.
  
  ...Later:- I was wrong about the need for these bulk storage SSDs to be built around SLC. A few years after writing this article developments in adaptive SSD controller design and DSP - showed that MLC, TLC (x3) and even QLC (x4) can be managed to produce adequate operating life and data integrity.
• power up / power down - for the controller this is a different environment than a notebook or conventional server acceleration SSD. It lives in a datacenter rack with a short term battery hold-up and in no other type of location - ever. The power cycling must be optimized to reduce the time taken for the 1st data accesses from the sleep state. (Fast boot to ready SSDs.)

• performance - R/W throughput and IOPS are secondary considerations for this type of device and likely to lag 2 to 3 years behind the best specs seen for other types of rackmount SSDs. In the awake state sequential R/W throughput - for large data blocks - has to be compatible with rewriting the whole SSD memory space in approx 24 hours or less.

• refresh cycle - you know about refresh cycles in DRAMs - why need one for flash? The answer is that seldom accessed data inside the SiliconLibrary could spend years in the unpowered (or powered but static data) state. That would be a bad thing - because the data retention of the memory block can decline in certain conditions increasing the risk of data loss. So to guarantee integrity in the SSD is a house-keeping task which ensures that ALL memory blocks in the SSD are powered up and refreshed at regular intervals - maybe once every 3 months - for example. If you're familiar with tape library management - think of it as "spooling the tape."

• surges and ground bounce in the SSD. If uncontrolled - these could be a real threat to data integrity. That's one of the reasons why the wake access time has been specificied as a number like 20 milliseconds - instead of an arbitrary number like 200 micro-seconds. Soft starting the power up (by current limiting and shaping the slope) will reduce noise spikes in adjecent powered SSDs and also minimize disturb errors.

• awake duty cycle. I haven't said much about the nature of the duty cycle for the powered up (awake) state for memory blocks. To achieve good power efficiency I've assumed that in the long term this will be just a few per cent of the time. But how will it look on any given day? That depends on the HSM scheme used in the SSD library. My working assumption is that once an SSD block is woken - it stays in use for a period ranging from seconds to minutes. The controller which woke it needs to have a way of anticipating this. As the SSD library is the slowest tier in the SSD storage world - it would be reasonable to assume that the device which originated the request (an online SSD) can do some buffering and pack or unpack data requests into multiple GB chunks. Something for the "hello world!" members of the design team to think about.

Footnotes about the companies mentioned in this article

Some of the companies mentioned in the "fictional" part of this article - to illustrate the 2016 press release - are real companies. Thiese are the reasons why I chose the companies whose names I used to illustrate certain concepts. And I hope they won't be offended.

• WD Solid State Storage (SiliconLibrary) - WD's SSD business unit has been intensively testing SSD data integrity by running individual SSDs through thousands of power cycles - since 2005 - as part of verifying its PowerArmor data protection architecture. These are the longest running such test programs I know - and they have already harvested data from millions of hours of device tests (by 2010).
  
  Understanding what happens to MLC SSDs when subjected to these stresses is a key factor in the confidence to design bulk storage SSDs - in which each memory block may undergo upwards of 5,000 power cycles in its operating life.
  
  Although many other industrial flash SSD companies also have experience in this area - WD also has experience with designing hard disks which have good power performance in sleep mode. These features have not been widely deployed in MAID systems because of the long wake access time. Will the 20mS wake time - which I've proposed for archive SSD prevent its acceptance? We will see.

• Fusion-io (SSD port) - although SSDs currently use standard interfaces within and between racks - I speculate that in some markets there will be a cost / performance advantage to creating a new proprietary interface to "get the job done." I'm proposing that Fusion-io - already the best known brand in the PCIe SSD market (2010) - is a likely company to adapt its products for new markets when the need arises - and create a new defacto industry standard for inter-rack SSD ports.

• Google - It's not unreasonable to expect that in the 6 years following publication of this article Google (who already markets search appliances, is the #1 search company, and is working on an SSD based OS for notebooks) will be setting many key standards for the manipulation of large data sets within the enterprise. Google APIs will be as important to CIOs in the future as Oracle and other SQL compatible databases have been in the past as tools which support data driven businesses.

"Business thinking will change - towards valuing more data. But physical space and electrical power will continue playing their part as pressure constraining corsets on these systems. That's why new SSD architectures will be needed - because current designs are incapable of being scaled into these tight spaces"