click to visit StorageSearch.com home page
leading the way to the new storage frontier .....
SSD symmetries article
SSD symmetries ..
image shows megabyte waving the winners trophy - there are over 200 SSD oems - which ones matter? - click to read article
top SSD companies

SSDs in IPTV, movie creation and tv

PCIe SSDs
the Fastest SSDs
rackmount SSDs
SSD Market History
Can you trust SSD market data?
The enterprise SSD story... why's the plot so complicated?
SSD ad - click for more info
new Digital Storage for Media and Entertainment Report

Editor:- August 27, 2018 - Coughlin Associates today announced the availability of its new (14th annual) Digital Storage for Media and Entertainment Report - 2018 - (254 pages, $7,500).

Tom Coughlin (author and President Coughlin Associates) says...

"The report benefited from input from many experts in the industry including end users and storage suppliers, which along with economic analysis and industry publications and announcements, was used to create the data including in the report. We have made modifications to earlier reports to better model current market conditions. As a result of changes in the economics of storage devices higher performance solid-state storage will play a bigger role in the future. The cloud and hybrid storage including the cloud will be a bigger part of the media and entertainment storage market going forward."

Editor's comments:- Among other things the press release about the new report includes these interesting observations:-
  • By our estimates, professional media and entertainment storage capacity represents about 4.5% of total shipped storage capacity in 2017
  • In 2017 we estimate that 71% of the total storage media capacity shipped for all the digital entertainment content segments was in HDDs, with digital tape at 22.7%, 4.3% optical discs and flash at 2.0%. Flash memory dominates cameras and is finding wider use in post production and content distribution systems.
  • Overall cloud storage capacity for media and entertainment is expected to grow about 13.3X between 2017 and 2023 (5.1 EB to 68.2 EB)
See also:- other storage market research companies


Netflix - the latencies behind flash caching

Editor:- July 24, 2018 - A recent article - Evolution of Application Data Caching : From RAM to SSD - on Netflix Technology blog - discusses their experience of using SSDs instead of pure RAM for caching data (which in their case is mostly streaming videos).

This trend of flash replacing RAM in enterprise caches was a hot topic with my readers in 2007 when flash SSDs were approaching the tipping point of replacing RAM SSDs.

As we know - flash won.

The PCIe SSD market and demands for server based flash caching were the key enablers in that victory and for the formation of an independent SSD software market. Notable ISV pioneers in the flash caching applications arena were FlashSoft and IO Turbine - both founded in 2009.

The technique of using flash to replace RAM is now standard practice and thrives in many SSD form factors.

Expectations that the low latency portion of this market could become plug compatible with DRAM (and replace most of the DRAM market) - were a factor in the flare up of industry wide SCM DIMM wars in 2015.

The business lessons learned from the fizzling out of "SCM DIMM wars 1" - despite the memory shortages of 2016/17 which should have assisted the flash tiered as RAM concept - were that were that although "flash tiered as RAM" does indeed provide useful outcomes it is in fact usable across a much broader range of latencies than just the ultra-low latencies which the pure play NVDIMM-focused marketers had hoped would dominate these design-win conversations.

That's another reason I find the recent Netflix technology blog interesting - because (although our industry talks much about single digit microsecond memory fabric latencies) the Netflix case study shows that high double digit microsecond latencies can be good enough to replace most RAM.

Netflix says...

"We observed during experimentation that RAM random read latencies were rarely higher than 1 microsecond whereas typical SSD random read speeds are between 100500 microseconds. For EVCache our typical SLA is around 1 millisecond with a default timeout of 20 milliseconds while serving around 100K RPS. During our testing using the storage optimized EC2 instances (I3.2xlarge) we noticed that we were able to perform over 200K IOPS of 1K byte items thus meeting our throughput goals with latency rarely exceeding 1 millisecond. This meant that by using SSD (NVMe) we were able to meet our SLA and throughput requirements at a significantly lower cost."

Starting from that base - Netflix then went a stage further in their software implementation and were able to achieve typical latencies (with SSD rich caches) under 100 microseconds. ...read the article


Tegile's flash goes to Hollywood

Editor:- March 10, 2017 - A new blog - Flash Storage Goes to Hollywood by Brandon Farris, Director of Marketing Tegile Systems says among other things...

"The MLB Network, for example, uses Tegile flash storage in their post-production environment. During the regular season, they need to record all of the games and produce content for shows like MLB Tonight, The Rundown, Intentional Talk, MLB Now, and Quick Pitch, which focus on the day's activities and give a snapshot of what's going on around the league. In the off-season, they produce several countdown shows, individual player profiles, and other programming that goes behind the daily game and into more of the storytelling about baseball. That's over 500,000 hours of digital content!" ...read the article


memory intensive data streaming / rematerialization architecture - unstealthed by Symbolic IO

Editor:- May 25, 2016 - 1 petabyte usable storage in 2U along with a flash backed RAM rich server family which uses patented CPU level aware cache-centric data reduction to deliver high compute performance are among the new offerings unveiled today by Symbolic IO which has emerged from stealth mode.

..
the fastest SSDs - click to read article
.. the fastest SSDs
..
Giving a clue to performance - Founder & CEO, Symbolic IO - Brian Ignomirello said - "One of our early tests, allows us to run a full cable class content delivery network over 80+ nodes, while streaming 80+ full-featured movies simultaneously on one channel and requires less than 8% of the CPU capacity and we had plenty of headroom to run more. IRIS (Intensified RAM Intelligent Server) is 10,000 times faster than today's flash."

Editor's comments:- I hadn't spoken with Symbolic IO (when I wrote this) but my first impression was that the company is in line with at least 3 strategic trends that you've been reading about on StorageSearch.com in recent years:- Their company profile summarizes their capability like this...

"Symbolic IO is the first computational defined storage solution solely focused on advanced computational algorithmic compute engine, which materializes and dematerializes data effectively becoming the fastest, most dense, portable and secure, media and hardware agnostic storage solution."

For more about the company's background see this article - Symbolic IO Rewrites Rules For Storage on Information Week.

From the marketing point of view it's interesting to see that in its launch press release Symbolic IO positions itself in the DIMM Wars context in this way "IRIS... is 10 times faster than 3D XPoint."

Symbolic IO says the new systems will be start to become generally available in late Q4 2016.

From an enterprise segmentation viewpoint the IRIS systems will be proprietary. There is space for such approaches in the future market consolidation roadmap because not everyone needs the fastest performance. But many webscale SSD companies are already using data reduction techniques for their own utilizations and acceleration purposes.

The new thing - if there is a new thing - is that Symbolic IO will make available boxes which incorporate modern data architectures from a single source.

Although like all new systems companies they'll have to wade their way through the apps accreditation and compatibility lists before their revenues create any ripples - an adoption dampening factor I wrote about in my 2013 article Scary Skyera.

...Later:- in March 2018 - Symbolic IO exited the hardware supply business and changed its name to to Formulus Black.


what were the new ideas in the SSD market in 2015?

Editor:- December 15, 2015 - Memory Channel SSDs were one of the new big SSD ideas of 2013. They're now part of the establishment. (Even judges know what they are and why they're different.)

In my 2015 SSD year-end review on StorageSearch.com I discuss...
  • new ideas to assimilate
  • big ideas to unlearn and forget
  • the trends which will dominate our strategic SSD thinking in 2016
...read the article


Video footage accounts for 100 Exabytes per year of new storage

Editor:- September 17, 2015 - "Video footage accounts for 7% of the total storage sold worldwide for any reason" - that factoid is from a paper - Taming the firehose of media files (pdf) by a media management company called axle Video


Efficiency is important for web scale users - says Coho

Editor:- October 9, 2014 - Facebook as a file system - a web scale case study - a new blog by Andy Warfield , cofounder and CTO - Coho Data - made very interesting reading for me - as much for revealing the authoritative approach taken in Andy's systematic analysis - as for the object of his discussion (Facebook's storage architecture).

It reveals useful insights into the architectural thinking and value judgments of Coho's technology - and is not simply another retelling of the Facebook infrastructure story.

When you read it you may get different things out of it - because it's rich in raw enterprise ideas related to architecture, software, and dark matter users. All of which makes it hard to pick out any single quote. But here are 2.
  • re - the miss match between enterprise products and user needs

    Andy Warfield says - "In the past, enterprise hardware has had a pretty hands-off relationship with the vendor that sells it and the development team that builds it once it's been sold. The result is that systems evolve slowly, and must be built for the general case, with little understanding of the actual workloads that run on them."
  • re efficiency and utilization

    Andy Warfield says - "Efficiency is important. As a rough approximation, a server in your datacenter costs as much to power and cool over 3 years as it does to buy up front. It is important to get every ounce of utility that you can out of it while it is in production."
There are many more I could have chosen. ... read the article


new article on enterprise SSD pricing

Editor:- July 18, 2014 - in a new article on StorageSearch.com - Exiting the Astrological Age of Enterprise SSD Pricing - I explain why I think that 2014 will be seen as the start of a new phase of creativity in the enterprise SSD market on the subject of pricing and affordability and I name 3 companies leading this charge.

It shouldn't come as any surprise that the dominant form factor leading this new market trend is rackmount SSDs. because as I told you in this article last year - exciting new directions in rackmount SSDs - that's the most productive form factor for doing something efficiently different with technology.

And the simplest way for vendors to signal to the world that they are masters and commanders of the solid state storage high seas - rather than merely floating barges of nand flash which can be swept along in any direction by the latest technology gust - is to hoist new colors of SSD pricing. ...read the article.


say hello to CacheIO

Editor:- June 10, 2014 - CacheIO today announced results of a benchmark which is described by their collaborator Orange Silicon Valley (a telco) as - "One of the top tpm benchmark results accelerating low cost iSCSI SATA storage."

CacheIO says that the 2 million tpm benchmark on CacheIO accelerated commodity servers and storage shows that users can deploy its flash cache to accelerate their database performance without replacing or disrupting their existing servers and storage.

Editor's comments:- The only reason I mention this otherwise me-too sounding benchmark is because although I've known about CacheIO and what they've been doing with various organizations in the broadcast and telco markets for over a year - I didn't list them on StorageSearch.com before.

That was partly because they didn't want me to name the customers they were working with at that time - but also because with SSD caching companies becoming almost as numerous as tv stations on a satellite dish - I wanted to wait and see if they would be worth a repeat viewing. (And now I think they they are.)


SanDisk ships 350MB/s write CFast for cameras


Editor:- September 13, 2013 -SanDisk today announced it is shipping one of the first SSDs based on the CompactFlash Association's CFast 2.0 standard. Aimed at applications such as cameras - SanDisk's Extreme Pro CFast 2.0 card has upto 120GB capacity (MSRP $1,809), R/W speeds upto 450MB/s and 350MB/s respectively.


SSDs in TV

Editor:- November 21, 2011 - An article in TVTechnology.com discusses the state of play for SSDs in the tv industry.

Unfortunately one of the statements in the article - about the readiness of SSDs for the enterprise market is historically incorrect in suggesting that consumer SSDs came first. I'd like to know which consumers were spending $40K for an entry level SSD back in the 1990s? Nevertheless the article shows what some people in the tv equipment market are thinking.


AMP launches CineDrive

Editor:- June 29, 2011 - AMP launched the CineDrive a low power, rugged 2.5" SATA SSD optimized for the video recording industry.


the future of data storage

Editor:- January 23, 2011 - the future of data storage is the lofty sounding but aptly chosen title of a new article published online today in Broadcast Engineering - written by Zsolt Kerekes editor of StorageSearch.com (that's me).

It's a completely new article which synthesizes and integrates concepts from several futuristic articles which have already appeared here on the mouse site and wraps them into a cohesive whole. Anyone who reads it will get a clear idea of where the incremental changes they read about in storage news pages (like this one) are likely to end up. ...read the article


Who Needs Smart TV?

Editor:- January 13, 2011 - Who Needs Smart TV? is the subject of a new blog by Michell Prunty at Semico Research.


SSDs cheaper than HDDs in some new DVRs

Editor:- September 9, 2010 - SanDisk today announced that NDS (a tv set top box designer with with over 30 million DVR units deployed) has successfully has designed SanDisk SSDs into a new range of lower cost set-top DVRs.

The new solution allows for the deployment of energy-efficient STBs with decreased power consumption, small form factors and virtually silent operation.

SanDisk says its SanDisk P4 is substantially less expensive in 4GB, 8GB and 16GB capacities than hard disk drives, which are often only available in higher capacities. Editor's note - This floor price SSD penetration model was 1st proposed in the 2005 article by Jim Handy - Flash vs. Hard Disks - Which Will Win?

"For the first time, SSDs are now a cost-effective alternative to hard drives in the STB market," said Doron Myersdorf, senior director, SSDs, SanDisk.

Editor's comments:- flash storage has been used in STBs for a long time. Samsung shipped 117 million units of its 0.5GB OneNAND devices in 2006 into various markets including STB.

Low acoustic noise was mentioned as a desirable factor in the world's 1st SSD user preferences survey (2004). In those days - because of the cost of SSDs - we were talking about silent running in submarines.

For DVRs with terabytes of storage - HDDs will remain a lower cost option for a while. And you will probably continue seeing HDDs in new consumer products long after HDDs have disappeared from the datacenter - because the cost of ownership and performance envelopes will diverge.


Fusion-io accelerates 3d movie releases

Editor:- July 28, 2010 - Fusion-io today revealed how it's been helping Hollywood's 3D movie makers.

Fusion-io's SSDs have been used in the production of:- Alice in Wonderland, Avatar, How to Train a Dragon and Clash of the Titans.

Utilizing Fusion-io, the S3D conversion of Clash of the Titans took under 8 weeks instead of typically 6 months using conventional systems.

"Fusion's ioDrives really helped us meet a tough deadline" said Sean Konrad, Project Manager for Stereoscopic Conversion on this project at Prime Focus. "With typical VFX projects, we complete about 20-30 shots per day. The ioDrives (from Fusion-io) scaled our data load capabilities by about 20x. Using Fusion-io drives, each review session covered far more shots and we could continue cycling work in between sessions. We don't have to worry anymore about defragmenting disks, rebuilding RAIDs, or swapping out disks. We are buying many more ioDrives to implement throughout our different locations."


Nexenta streams online tv

Editor:- May 20, 2010 - Nexenta Systems announced that its products (which include SSD ASAP features) are being used by the Dutch Public Broadcasting Agency NPO for storing and delivering online tv in a configuration which includes 192TB of hard disk drives and a 1.9TB SSD read cache.

The broadcaster's website has approximately 80TB of video available to online users who want to watch previously broadcasted television programs. During an average evening, between 10 and 20,000 people stream data, adding up to 25GB in capacity. The customer (who evaluated multiple vendors ) says that important selection criteria were:- performance, price, support and power consumption.

older articles and whitepapers - also worth reading
 click here for larger DVD Megabyte image
After another hard day working on
the web Megabyte liked to sit back
and view someone else's efforts.

see also:- SSD videos
.
SSD ad - click for more info
.
flash SSD capacity - the iceberg syndrome
Have you ever wondered how the amount of flash inside a flash SSD compares to the capacity shown on the invoice?

What you see isn't always what you get.
nothing surprised the penguins - click to read  the article There can be huge variations in different designs as vendors leverage invisible internal capacity to tweak key performance and reliability parameters. ...read the article
.
SSD Pricing - where does all the money go?
SSDs are among the most expensive computer hardware products you will ever buy.

Understanding the factors which determine SSD costs is often a confusing and irritating process...
Clarifying SSD Pricing - where does all the money go? - click to read the article ...not made any easier when market prices for identical capacity SSDs can vary more than 100x to 1! Why is that? ...read the article
.
SSD ad - click for more info
.
this way to the Petabyte SSD
In 2016 there will be just 3 types of SSD in the datacenter.

One of them doesn't exist yet - the bulk storage SSD.

It will replace the last remaining strongholds of hard drives in the datacenter due to its unique combination of characteristics, low running costs and operational advantages.
click to read the article -  reaching for the petabyte SSD - not as scary as you may think ... The new model of the datacenter - how we get from here to there - and the technical problems which will need to be solved - are just some of the ideas explored in this visionary article.
.
Who makes the fastest SSDs?

Speed isn't everything, and it comes at a price.........
But if you do need the speediest SSD (chip, card, module or rackmount) then wading through the web sites of hundreds of SSD oems to shortlist products slows you down.

And the SSD search problem will get even worse as we head towards a market with over 1,000 SSD oems.
the fastest SSDs  sorted by interface and form factor - click to read article ... Relax - I've done the research. And this whizzy wish list is updated daily from storage news and direct contacts from oems. ...read the article,

storage search banner

STORAGEsearch is published by ACSL