Can you trust SSD market data? by Zsolt Kerekes,
editor - StorageSearch.com,
February 12, 2013 |
This article looks
at problems in the collection, analysis and interpretation of market data and
reports about the SSD market.
In one way - a fast growing and
disruptive market like SSDs is no different to any other technology business
market - in that you need a lot of data to help guide you in rational decision
making.
But the SSD market presents unique challenges in the areas of
data collection, validation of interpretation, raw complexity, and interplay
between different internecine competitive approaches which apparently compete
for the same short term customers. Added together these factors make
market models problematic.
Can you trust market reports and the
handed down wisdom from analysts, bloggers and so-called "industry experts"
any more than you can trust
SSD benchmarks to tell you which product is best?
The short answer
is - heck no! - whatever gave you that silly idea?
For a longer answer
read my article below.
why go down this difficult road?
Alternative
subtitle:- plowing the deep SSD market furrows - bad days in SSD - and where
did that rock come from?
In this article I'm going to assume
you have more than a passing curiosity and more likely a serious professional
interest in the SSD market - otherwise you wouldn't still be here.
Understanding
what's going on, and why, identifying the main SSD trends and companies which
might affect your decisions and honing your skills at anticipating what
might happen next (to get better control and results in your SSD projects) is
what you do.
For many of you that serious interest in the world of SSDs
means you may already have spent thousand of dollars on market reports from the
many companies which
offer such products.
Or it may mean you've spent hundreds of hours
reading about SSDs online.
You spend a lot of time thinking about SSDs
or talking about them.
Maybe you've gone to some conferences too.
Compared to where you started - there's no doubt that you've now
advanced your thinking about the SSD market a considerable distance compared
to where you started. And and compared to the world at large and in the
circles in which you move - you are already an expert in many aspects of this
SSD thing.
And some of you - I know - are running SSD businesses.
You
eat drink and breathe SSD market data.
And a small number of you - are
the people who compile those SSD market reports or write SSD market blogs
and articles...
But one thing you've all got in common - in this
context - is that from time to time - you encounter something from someone
you regard as being a trusted SSD expert - which just doesn't sound right.
And while mostly - their content follows the tone and line of
arguments you would be inclined to agree with - there's this bunch of
assertions or ideas or comments or data - dropped in like a bad cut and
paste job - which you feel in your bones is unsettling. It jars. It's a rock
in your data furrow - which upto now had been looking pretty neat and tidy and
heading slowly but surely in a generally forwards kind of direction.
And
you wonder - as you step down to assess the damage from your heavily invested
SSD data harvester:-
What's going wrong?
Is there
more SSD stuff you've got to read?
Worse - is there more you've got to
understand?
You wonder - is there another strand of SSD thinking -
which the expert author of this report or blog - implicitly assumes you
should already know?
What if - everyone else knows this - except
you?
And if the said expert is right about this point (which is not
exactly what you were anticipating from what you had been reading in the
lead up to the bump) how confident can you be about all those other SSD related
ideas which you thought you had already safely tucked in the ticked part of the
SSD assimilation list? Do you have to go back to re-examine them too?
Just
when you thought you had all the bases covered - you've hit something which
dents your confidence. Just like when you started to learn about SSDs.
For
anyone intensely involved in the SSD market - this kind of thing happens a lot.
bad SSD data day
And on a particularly bad SSD data
day you may be inclined to ask yourself:-
- what do I really know about the SSD market?
- what are my safe assumptions?
- in the event of major conflicts of opinion, market data and differences of
interpretation - who can I trust?
- how do I decide which way to go from here?
- and - in extremis - is it time to update my profile on linkedin?
If
it's any consolation - I see those same conflicting and contradictory SSD
ideas and market data trends myself too. And while there can be valid
market and technology directions
which leading SSD companies themselves disagree about - many of these
dissonances I encounter in the raw market stream of data I wade through every
day (stats, reports, blogs, articles, opinion pieces, analysis and
conversations) have a simple cause.
Someone got things wrong.
And
the rest of this article will discuss the most likely reasons.
a
brief note re - SSD market research philosophy
When you see the
length of words which still flows below this point - you may be relieved to
hear that to save time - I've decided to skip a "philosophical"
discussion of why market reports and analysis can generally be wrong which
might have included such questions as:-
- why was the report written in the first place?
- what was the raw data which influenced the findings?
- how much did the report's authors understand the issues they were looking
at in the first place? and
- how closely did all those above choices and preconditioned biases
align with your own unique needs for getting answers
In the real
world you mostly have to rely on reports which were written for a general
purpose - whereas each of you has a particular set of your own reasons why
you're going through this stuff.
The original aims, interests,
sentiments, aspirations, frameworks and methodologies of the report's
authors are just as important to the validity of what you extract from
reports and articles as any of the content - because they predefine the
shape of what you see.
As a professional who leverages your
understanding of the SSD market - you're continuously making judgements
about how you weigh the significance of things you read from various SSD
sources.
The acid test is this.
When you read something about
the SSD market which you didn't know before - or which seems to contradict
something which you previously held as a working hypothesis - what do you do
next?
- Accept the new statement for now?
- Embark on more data searches - or dialog - to learn more one way or the
other?
- Disregard the offending data or observations - because you judge the
author doesn't know as much about this as you do - or is writing from a
viewpoint which isn't relevant to what you do.
It's the kind of thing
you need to know if you're assessing a PhD - but we're not going to stretch
this home page blog into 80,000 words. Not today - at any rate. (I'll do a
wordcount check at the end - just to make sure I've kept my promise on this.)
Instead
I'm going to list and talk about the 5 main practical reasons why things
go wrong in the SSD market data collection, interpretation, modeling and
analysis business.
- the distorting lens of viewpoint
- the brain busting factor of complexity
- under counting and under sampling the vendor population
- sampling the wrong people for customer related opinions
- can you believe what real customers say? - sometimes yes - often no
the
distorting lens of viewpoint
as in - "SSDs are similar to..."
When
you first approach SSDs having a background in something else - there's a
temptation to think - that some of what you see in the SSD market is like
something you've seen or read about before - maybe in semiconductors, memories,
computer hardware design, software or storage systems. But carrying over
thinking from previous familiar and apparently closely related subjects - can
lead you to conclusions and analysis which - in the world of SSDs - doesn't lead
you to the right place.
That's one of the reasons that long
established leading companies in hard drives and storage systems - such as
Seagate and
EMC didn't ignite the SSD
market and were late adapting to the SSD market. SSDs are similar to HDDs and
storage racks - but different.
And even chip companies like
Intel,
Micron and
Samsung didn't drive
the sharp end of SSD architecture innovation and high end customer adoption.
SSDs are mostly made from chips - but being successful at designing and selling
memory chips or processors didn't lead to automatic success in SSDs for these
chipmakers.
For several years each of the above companies
interpreted the SSD market's potential through the distorting lens of its
similarity to what they had done before. This meant that for years they were
doing the wrongs things (including nothing at all) even when seeing the same
raw market data as other companies which got ahead of them in some aspects of
the market.
- Seagate initially saw SSDs as specialized - but more expensive versions of
hard drives. And failing to understand that SSDs really were different HDDs-
and might one day become a
big market
compared to hard drives - led the company to cling to the illusion for many
years that a coping strategy would look something like
hybrid drives.
- EMC initially saw SSDs as just another kind of drive it could put into a
JBOD or RAID. Even when
sales people from the upstart
Fusion-io invited EMC
to the PCIe SSD water trough and gave them early versions of their PCIe SSD
products to evaluate - EMC failed to understand the benefits of drinking this
form of flash juice until many years later.
- Samsung - earlier than other similar sized companies - said it
recognized the potential of SSDs - but it learned the hard way that simply
installing me-too SSDs in its own
notebooks
wasn't going to transform it into an SSD leader.
- Intel assumed that because the world had followed its processor and
interface standards for 30 years that it didn't have anything to learn from the
new SSD market. But it did in every important respect - particularly in
reliability and
architecture.
- Micron initially assumed that making
flash and other non volatile
memory meant it would automatically have advantages in SSDs. Instead - in
the modern market - the supplier of the raw memory in SSDs has become almost
irrelevant - as SSD designers have become more adept at virtualizing and
managing the inherent design and process characteristics of memory and
adapting to whatever the memory market can supply.
All the above named
companies have since changed the way they engage with the SSD market - even
though for many of them a key ingredient in their new emerging coping
strategies has involved
licensing and
acquiring SSD technology from native SSD companies. You can see more details
in their profile pages.
This part of the discussion is simply to show
how strong can be the distorting effect of viewing the same SSD market reports
and projections that were available to all companies - through their own lenses.
And everyone - whatever type of company they're in - including
analysts - has to fight a distorting gravity well.
But that's not the
end of the story - because - "SSDs are similar to..." - is just one
of many possible market distortion lenses.
Other market filters and
stances can be just as influential in altering and distorting how market data
is reacted to:-
- the lens of SSD IP - these are the limits set by our technology (unless
we buy more)
- the lens of investment - these are the limitations of what we're interested
in based on the limits set by our funding
- the lens of positioning - these are the (different) options we are
interested in as an established leader, follower, or newcomer to the market
- the lens of how much we (as a user) understand our own technology
infrastructure and how much (or little) business advantage we think we can get
by changing it
the brain busting factor of complexity
the
SSD market is more complicated than previous technology markets
No
one really understands the SSD market.
And the SSD market isn't a
single cohesive entity.
Instead - what we're referring to - when we
talk about the SSD market - is now a whole load of complicated markets and use
cases with variable degrees of independence and interdependence which create
the illusion of a looking like a big market which is called SSD (for
convenience) - because the SSD sub markets use similar raw ingredients.
And
some of the SSD products which satisfy some market segments - particularly
some types of 2.5"
SSDs and controllers
- superficially look like they span a much wider range of use cases than they
actually do. (Technically they can. Competitively they can't.)
I'm not
going to list all the significant differences between different types of SSDs
and application specific strengths and weaknesses here - because if you scroll
down the home page of StorageSearch.com you'll see that I have identified more
than enough to keep you busy reading for the next few weeks.
My point
is this. It's possible to be an expert in one of the SSD market and be
successful in that area - and yet be clueless or not really care about other
significant parts of the SSD market at all.
One of the other reasons
people get things wrong when analyzing what's happening in the SSD market is
that it's really complicated. To understand the raw ingredients in the SSD
market pot requires an acute awareness about the limitations of semiconductors,
processors, interfaces, computer architecture, system software, apps software
and economics - and how these factors interact in disruptive ways at internet
market speeds.
At the extreme this is a market where almost
anything is possible and a lot of different technology and business ideas
are still being thrown at the wall in the hope they will stick. That means there
are few real enduring certainties or rules which can be safely relied on.
As if that wasn't complicated enough - there's strong disagreement
among different types of SSD companies themselves about the best way of doing
things. (Among the many discussed in previous articles are disagreements
things like
- to what extent hard drives should be a part of the SSD ecosystem? -
heavily weighted, some, none
So
you could ask - how are analysts and other market commentators to make sense of
all this? and how can people create a sense of order - out of what are in fact
different chaotic SSD markets - which have similar internal technologies - and
compete with each other as well as competing with traditional ways of doing
things?
Even with the best intentions and starting points - and taking
into account the distorting effects of what your starting position and long term
interests may be - there are still other factors which can muddy the waters -
and which at heart are very simple.
Under counting and under
sampling data and influence from vendors
In traditional high
tech markets (computers, storage, chips, comms, software etc) the custom and
practice for compiling reports was - to collect data from a short list of what
were assumed to be the top 6 to 10 companies in the market and extrapolate
market trends from that.
It only works if those companies have been
correctly identified in the first place - and if the shape of who those industry
leaders are is relatively static (the usual suspects). But this approach fails
in disruptive markets like SSD because the significant pressures which will
change the market may be coming from companies which aren't in the sample
list. It also results in an underestimate of the size of the market and risks
missing entirely the identification of new use cases and market segments.
This
isn't a new problem. I identified the same risks a decade ago in commenting on
reports about storage arrays based on hard drives. In that case the only problem
was undercalling the vendor headcount. But it didn't affect the shape of the
market because every vendor was drawing from the same population of hard drives
- and therefore the scope for doing something significantly different and
innovative was minimal.
But this factor is a material risk in
innovative markets - like SSD - in which the leading companies list is fluid.
And I warned about the risks in my (2005)
SSD market
adoption article
More recently - in
2011 -
commenting on a news story about an SSD market report from
IDC - I said...
"What
these types of reports can do is understate the market size of fast growing
markets - because of sampling errors (such as excluding key players - who
may not wish to participate in the data collection process) and also because
of financial reporting time lags (which may provide indicators several quarters
behind search-volume based reports).
"In the growth phase of a
market - any type of consistently collated info is more useful than none. But
perfect information about markets only exists when the markets are dead or have
reached a steady state. Marketers who use these types of reports have to be
aware that all market reports from all vendors have in-built flaws and biases."
This
weakness in methodology - in the context of SSDs - is something which IDC
has recently publicly acknowledged - and presumably will correct.
Lists
such as StorageSearch.com's 6 year running
Top SSD Companies List
- means researchers have easy ways to speedily shortlist emerging disruptive
companies based on leading adopter search volume - while the scrutiny offered
by the growing financial value of established SSD companies makes it easier than
ever before to identify candidates at the other end of the size scale too.
Sampling
the wrong people for customer related opinions about the SSD market
The
symptom goes like this. XYX market research company asked 300 people (or 50 or
1,000) what they think about the SSD market and what they aim to do about it?
I
read the report - and see a lot of extrapolated nonsense which doesn't sound
like any version of reality with which I'm familiar - yet because the raw data
is based on a questionnaire and is reporting what real people think - the
results are publicized as having a special validity in their predictive
assertions.
Yeah - that's just about as valid and useful as when the
banks in the pre-credit crunch era rolled up a load of inappropriately sold
mortgages, renamed them as high grade
CLOs
and sold them on to other banks.
Freedom of speech and thought are
sacrosanct values - but just because you've asked a bunch of people what they
think about something doesn't mean that's what's happening.
- Who are these people you sampled?
- Where did they come from?
- How engaged are they - in fact with the topic of SSDs?
- Are they in fact the people who have been doing things in the SSD market?
Although
many of these reports are entertaining, well written and stylishly presented
- the market-wide validity of inferences drawn by researchers from such
reports are questionable - because the sample came from some trade show or
conference or online publication where the core draw subject was something else
- and not SSDs.
can you believe what customers say?
Traditional
market research theory says that there's nothing better you can do to
understand what's going on than talk to your customers.
But how true
is this for the SSD market?
Sometimes it works - and you can get some
valuable insights which change the products you offer.
That's what
happened when Fusion-io did market research into its own base of enterprise
customers. The result - as recounted to me by
Fusion-io's CMO Rick White - was they
identified a special group of super-user SSD customers - who were using FIO's
PCIe SSDs - but whose detailed needs for product functionality were materially
different to their other SSD customers.
These super-users already had
installed thousands of the company's SSDs (or had the potential to do so) and
they were willing to invest technical expertise to tweak the products to
optimize them for their main apps. Many of the so-called enterprise wraparound
features offered in FIO's standard offering to the market either weren't needed
or weren't valued by these users who had their own way of managing things - and
were less interested in following legacy enterprise models than getting the best
results from their massive SSD investments.
It's these super-user
customers which have been catered to with the release of special products like
ioScale (low cost PCIe
SSDs evolved from Fusion-io's consumer / workstation family) and also the
ION software (which converts a bunch of PCIe SSDs and a server into a SAN
compatible SSD rack).
I was dismayed to read analysts talking
seriously about the ION software as if it was something aimed at mainstream
SAN SSD customers. These commentators missed the point. The value to FIO of
this product line wasn't in thousands of customers who might buy a small number
of systems each. It was in a small number of customers who each had the
potential to buy thousands of SSD racks.
Another interesting thing
happened when SanDisk started talking to enterprise SSD customers using
the channel of their FlashSoft
acquisition - which as Rich
Petersen (former VP of Marketing at FlashSoft and now Director,
Marketing Management at SanDisk) told me is now the company's center of market
intelligence in strategic enterprise SSD related stuff.
SanDisk
learned that despite their position in the flash market - their biggest
potential customers had already made investments in other people's enterprise
SSDs - and they could see apps in which that would continue to be a better
option for them. So - whereas they liked SanDisk SSDs for some use cases and
they liked the potential of SanDisk's
VMware vSphere
- they weren't going to invest in more than one core
SSD software platform.
The result? - SanDisk learned it could sell more enterprise SSDs if
its software also supported competing hardware. That's not what they originally
had in mind - but was a necessary condition to being accepted by large
enterprise users who have diverse needs and already have their own SSD history.
But
lest you fall into the trap of thinking you've now found the key to getting
reliable SSD market - this is the point where I say that - SSD customers don't
always know best.
The incorrect identification of SSD competitors
occurs commonly in reader emails and SSD investment blogs. And the roots go
right back to customers themselves - as learned when I asked Thomas Isakovich,
CEO of Nimbus Data Systems
why he had listed a set of vendors in a presentation by the company as their
main competitors.
"Because those are the competitors mentioned
most often in customer feedback of which other vendors users were looking at"
- was the answer.
But as I pointed out - users themselves aren't
always the most reliable source of competitive information - because in the
case of systems purchases - because the users are dipping into the market and
sampling what's available for only as long as it takes to satisfy their
immediate needs.
It's all too easy for users to compile short lists of
SSD vendors which aren't ideal for their needs - but it doesn't matter as long
as at least one or more of the vendors in the list can meet their needs.
That's
not the same as saying that all those other vendors are natural competitors.
Often they are just companies which make SSDs with a similar form
factor - but which are optimized for different
apps and architecture
silos.
Being more aware that these problems exist - and are
widespread will arm you better in your search for SSD knowledge and ongoing
data accumulation and sifting activities and may help you to better recognize
which conflicting information nuggets you can trust more and which you can
trust less - or safely disregard entirely.
Summary
I
could have written much more - but this is where I'm going to stop. And as
promised above - the wordcount was considerably less than 80,000 words.
A
rational and pro-active defense strategy for leveraging SSD market data is
to become your own expert in the area of expertise where you spend most of
your time.
Learn to make your own market models and predictions and
then review them regularly.
As time passes you'll increase your
confidence and maybe even make decisions with better outcomes. Good luck.
Can
you trust market reports and the handed down wisdom from analysts, bloggers and
so-called "industry experts" any more than you can trust SSD
benchmarks to tell you which product is best?
You've seen what I think.
If you've got your own blog or twitter channel and feel like this
subject needs an airing with even more words than I used already - I look
forward to seeing what you say.
Thanks for your time. |
further suggested reading
SSD history
the SSD
Bookmarks
after AFAs -
what's next?
Are
we there yet? - SSD market in 2017
about the publisher of
StorageSearch.com
Historical
Perspectives - on the SSD market
SSDs - list of
recommended market analysts
Can you tell me the best
way to SSD Street?
exploring the
limits of the market in your head
Another list of storage
market research companies
A simple way of looking
at the Enterprise SSD market
The big market
gravitational impact of SSD dark matter
| | |
| . |
 |
|
|
| |
.... |
| "Using revenue based
reports to guide your way ahead in a fast growing market like SSDs is about as
sensible as driving fast down the highway and steering ahead by what you see in
the rearview mirror." |
| Editor's comments (July 2008) in the
the Top 10 SSD OEMs -
2008 Q2 edition | | |
|
.. |
|
 |
|
.. |
|
"The real power of the
Gartner Group is the illusion of what the behavioural psychologist Robert
Cialdini calls “social proof”, which plays to our subconscious need to
conform with the crowd.
In his book - Influence – The Psychology
of Persuasion - Cialdini gives an example of this that totally captures the
essence of Gartner’s power; Canned Laughter." |
| Kevin Hunt, MicroStrategy in his blog -
Does
Gartner Group Matter? (March 2, 2016) | | |
|
.. |
|
We can't afford NOT to be
in the SSD market...
But don't ask how we're going to make money out
of it yet. (Or this year or next...)
Coz we're going to do it
anyway. |
| hostage to the
fortunes of SSD | | |
|
.....
..... |
|
 |
|
. |
|
One of the challenges for
the enterprise SSD market when designing new rackmount products is to understand
complex customer needs and decision criteria - which go beyond the traditional
marketing bullet points.
New segmentation models are needed because the enterprise SSD market
is moving into uncharted territories and use cases where a considerable
proportion of the customer needs which affect buying behavior are still formally
unrecognized as being significant (in market research data). |
| Decloaking
hidden segments in the enterprise | | |
|
. |
|
 |
|
. |
|
| When you include hundreds
of products in such a guide from all known vendors - then the sampling process
is transparent (and those not in the guide - need to make better efforts to
communicate with their market) but when you have a guide which samples only a
small percentage of vendors then inevitably questions get asked about how those
in the sample were chosen. |
| re -
DCIG's AFA
Buyers Guide (September 2015 ) | | |
|
. |
|
| Jim Handy discusses the
paradox of many competing vendors all claiming to be #1 in flash arrays |
Editor:- May 29, 2015 - A new blog -
Who's #1 in Flash
Arrays? - by Jim Handy
- founder Objective Analysis
- discusses a "puzzling set of claims" by various competing
rackmount SSD
companies - who all claim to be #1 in the flash array market. ...read the blog
Editor's
comments:- Jim's observations provides fresh examples and confirmation of 2
things I warned you about in earlier articles:-
- "The distorting lens of viewpoint" etc - discussed in the
article on the left of this page.
| | |
|
.
. |
|
I was talking to an end
user whose organization has spent hundreds of millions of dollars on EMC
storage.
They'd love to decouple themselves and benefit from modern lower cost
flash. But the flash marketers in startups aren't doing those kinds of
conversations.
For many of them a single customer like that is bigger than their
whole business plan.
|
| the unreal
positioning of many flash array "startups" | | |
|
.
. |
|
Brand awareness, financial
reports and online search data - intrinsically provide different numerically
weighted views of the same market.
And which methodology you prefer
depends on whether your priority is in understanding the past, present or future
- and what it is you're trying to decide. |
| re - SSD brand leaders -
from IT Brand Pulse (July 2014 ) | | |
|
.
. |
|
"Gartner 's Magic
Quadrant for all-flash storage arrays is based on criteria that hamstring
vendors and reduce customer choice...
"In talking to senior
executives at several storage vendors, I've discovered that these frankly stupid
criteria are not just keeping products off the Magic Quadrant, but also keeping
more flexible products off the market." |
| Howard Marks
in his blog -
Gartner
All-Flash Array Magic Quadrant: A Bad Influence (June 2015) | | |
|
.
. |
|
|
|
|
.
. |
|
"Theres a natural
business reaction to want to see a new entrant through the lens of a subset of
your existing market. Once you can do that you get more comfortable doing battle
in a small way rather than head-on. You feel your market size will trump a niche
player.
"The problem is that such perspective assumes a static view of
the market. Youre assuming that all the other attributes of your implementation
will remain advantaged and the new competitor will fail to translate that single
advantage into a broader attack." |
| Steven Sinofsky, Board
Partner - Andreessen Horowitz- in his
blog -
Blackberry
and the Lesson of "Don't forget all the parts move" (May 23,
2015 ) | | |
|
.
. |
|
|
|
|
.
. |
|
|
|
|
.
.
.
.
. |
|
 |
|
.
.
..
.
. |
|
| Every year I learn 2 new
important new ideas about SSDs. But every year I also have to remember to forget
or discard 1 old idea which was vital to know before because it's no longer
useful, valid or true |
| what
changed in SSD year 2013? /
2014?
/ 2015? /
2016? | | |
| |
