re SSD data recovery|
|by Zsolt Kerekes,
It's 2016 - but as I warned 9 years ago in
2007 (at the
bottom of this page) don't count on always being able to recover data from a
failed flash SSD.
Warning to readers!
Anyone can in a
few hours create a plausible looking website which claims their company can
perform data recovery on flash SSDs.
Many of the sites I've seen
in this market segment make claims which are unsupportable and few of the
so-called SSD recovery companies I've queried in this market have any clear
idea of the complexity of the task involved.
Many SSD failures are in
fact unrecoverable - because if the remapping tables get trashed - the media
data is effectively randomized - and mixed up with blocks which were marked as
corrupted and unusable even before the SSD failed. Worse still if the SSD is
advice at this stage in the SSD market expansion is to look at data recovery
information on your SSD vendor's original site and (mostly) disregard any data
recovery sites you may see which have been in operation for less than 5 years.
You can verify age and content using the
advice I can give is - do your best to minimize the risk of needing data
recovery in the first place. Do
regular backups - using multiple backup media - onsite and offsite.
|"Data recovery of
solid state drives is inherently complex. Due to advances in NAND storage
concepts older drives (Pre 2009) tend to be more recoverable than newer (2010
and later) SSDs."|
|SSD Data Recovery
Concepts and Methodologies|
|OCZ ends guessing games for
consumer DWPD |
|Editor:- February 3, 2016 -
TLC was originally
intended as a consumer SSD technology (not that you'd realize this from reading
about all the enterprise arrays which have assimilated it). |
OCZ recently announced
availability of a 15nm TLC based consumer range of 2.5" SATA SSDs - the
One of the interesting things about how the marketing of consumer
SSDs has evolved is that these new SSDs come with
DWPD guidance ratings
which are 0.25 DWPD.
Be aware, however, when comparing DWPD ratings
for consumer, enterprise and industrial SSDs that the warranty periods for these
different classes of drives - are different.
The Trion 150
warranty is 3 years -
which is typical for client SSDs - rather than 5 years (as for enterprise
related marketing messages have come a long way in the past 12 years or so.
2014 IBM said (in
effect) "You don't need to worry about the endurance of our FlashSystems."
That was my summary of an IBM blog at the time.
Nowadays OCZ says this
about their Trion SSDs...
"Never Fear, OCZ Endurance is Here."
one way I've got to admire the reckless implied simplicity of OCZ's endurance
message. But I also groan in anticipation of how other vendors will retaliate
with similar endurance messages of their own.
I think OCZ's "never
fear" tagline may have been around since last summer (for the earlier
Trion 100 - which OCZ says "quickly became a top seller for us")
but as I don't visit consumer SSD pages any more than I have to (even
my own) I didn't
see it until today.
in the SSD Market
|Data Recovery for
flash SSDs? - don't count on it|
|by Zsolt Kerekes -
September 24, 2007
flash SSD market
reaches the levels of penetration predicted by many
analysts - then in a
handful of years nearly half of all new
PCs will use flash SSDs instead of
hard disk drives.
happens when those SSDs inevitably fail - and there's no
consumers don't do regular backups - and most small businesses don't either.
hard drives fail, get
water or get damaged in fires - the solution of last resort - is to call a
data recovery company.
These superheroes can often recover a lot of data - even if the pcbs
and chips in the disk drive have been damaged. Superheroes don't come cheap.
The cost for a difficult recovery can run into thousands of dollars (for a
single disk) but for many satisfied customers that's a much better result than
being left with no business or months of lost time rewriting reports, novels
Although flash SSDs are new to the consumer market - they've been
around for many
years in markets which absolutely needed their levels of ruggedness (and
could bear the high cost). So you may be thinking that there's a well
established industry already out there ready to process your flash SSD - if you
are unlucky enough to need a data recovery service today.
You would be
The reason is that the biggest traditional customers of flash
SSDs have been the military
or industrial users who didn't want enemies / competitors stealing their
Erstwhile flash SSD manufacturers like
STEC specialised in
of various forms to make sure that that the data is never recovered by the wrong
So there isn't an established data recovery market track
record for flash SSDs in those applications which have been around the
The nearest that the market has to offer - is experience
with recovering data from simple
flash memory storage
(like USB keyring style
devices or camera memory cards). Unlike SSDs - those devices aren't designed for
intensive write applications - and there is nothing very complicated between the
interface controller and the flash chips themselves. So if the controller gets
zapped by static - or crunched by your car driving over it - the data is
relatively easy for experts to recover from the flash chips.
the case with most flash SSDs - which use complicated
to extend the reliability
and speed of storage. The architecture inside a high performance SSD is more
complicated than that in most
RAID systems. The
algorithms which map addresses to physical media locations vary from
manufacturer to manufacturer - and in many cases - like the
formula for making
Coke or Pepsi - the details are closely guarded commercial secrets.
at the server market and data recovery (at the single SSD level) is not a
burning issue for datacenter applications - because most often the SSDs operate
in some kind of RAID
protected array - and are also backed up (internally or externally) to other
missing in the consumer
market is a clear signal by oems - that data in their devices can be easily
recovered - if there is no backup - or the backup failed. Maybe the next
generation of products will address that issue. It would be another way of
segmenting the consumer flash SSD market - and a market need and opportunity
which hasn't been understood by SSD product marketers at the close of 2008.
flash SSDs are inherently much more
reliable than hard
drives - that's no consolation for the customers who will be the pioneers in SSD
|data recovery from server
Editor:- August 4, 2016 - data recovery from memory... is
that even a real thing? - some of you may be asking.
And why would you
want to do it?
In the past its viability depended on several factors
- including - how long was it since power was turned off.
Now in many
markets (web scale server and embedded gadgets) we're seeing a crossover of
SSD and SCM architecture concepts.
And due to the past success of
PCIe SSDs in servers -
we're also seeing new memories
It was those thoughts which led to my blog -
is data remanence a new risk factor in
Apple and FBI case demonstrate difficulties of SSD data recovery
March 3, 2016 - If anyone still had doubts about how difficult it is to recover
data from an encrypted SSD in the absence of a universal back-door key - the
proposition has been lent weight by the recent story rippling around the
world's news media about the FBI's efforts to force Apple to assist in
unlocking iphones. In the unlikely event you don't know what I'm talking about
here to see summaries of the unfolding story.
techniques have multiple uses and many of them originated as part of
intelligence and law enforcement data gathering activities.
data recoverability is a primary objective of
destruction design techniques used in many
SNIA gets interested in SSD data recovery
August 10, 2015 - SNIA
a new initiative aimed at the
SSD data recovery
market . This is the formation of a new Data Recovery/Erase Special
Interest Group (DR/E SIG).
One of the aims of the new group was said to
be - "Collaborating with solid state storage manufacturers to incorporate
capabilities needed to perform data recovery and erase in product design for
future device models."
Editor's comments:- I've commented
before on the inevitably conflicting interests between
SSD data purging
and data recovery and I don't think that the formation of this new group will
make any material difference to this.
For example - in the
military market - it's
not in the interests of secure
SSD designers to provide architectural information to 3rd party data
recovery companies which could simplify the reverse engineering efforts to
read such data.
Exactly the same kind of data can also be helpful to
competitors who want to know how they can improve their designs. So for those
reasons I think that collaboration in data recovery techniques will remain
limited to a subset of SSD suppliers.
statistical indicators re SSD data recovery market in 2014
January 22, 2015 - Data recovery concepts in flash drives were touched on in
some of the papers presented at last summer's Flash Memory Summit.
are some of the links and highlights.
drive recovery from a manufacturer perspective (pdf) - by Intel.
How do SSDs compare with
hard drives in notebook failure rates?
Intel's own experience -
based on a its employee population of around 100,000 notebooks - is that SSDs
are 5x to 10x less likely to fail. When it comes to data recovery - Intel is
seeing the need for about 1 SSD recovery each day based on 100,000 SSD notebooks
under its control.
SSD company attitudes can increase data recovery costs
March 20, 2014 -
Associates has written a new blog -
Tools Will Reduce The Costs of SSD Data Recovery.
Commenting on the
5x higher relative cost of
SSD recovery in
situations where the DR company has to do in its own research to understand the
distribution of the data - Tom says...
devices are a valuable method to protect consumer data and privacy but they can
also prevent recovering that data if parts of the SSD become corrupted. There
needs to be a greater degree of cooperation between the SSD manufacturers and
legitimate data recovery companies to make recovering data for an SSD customer
easier, while maintaining protection of SSD company intellectual property."
SSDs or hard drives? - the data forensics differences
October 23, 2012 - When you need to retrieve critical unbacked up data from a
damaged notebook (which you left in the car when you clambered out the
window after realizing that the puddle across the road was much deeper than you
first thought) you call the process "data recovery" - but
when a court seizes a suspect's notebook to try and retrieve data which may have
been deliberately "deleted" - they call it "data forensics"
- either way - in the most demanding cases the experts who work on these tasks
are the same.
SSD Data Recovery
(as opposed to dumb flash memory recovery) is a relatively new market which
didn't exist 5 years ago.
A recent article
SSD Drives Destroy Court Evidence - on a site called
ForensicFocus.com - discusses how
techniques which are essential to the operation of flash SSDs (such as
and wear leveling)
mean that from the forensic viewpoint SSDs yield up potentially much less
deliberately deleted recoverable data than hard drives.
adaptive R/W poses new complexities for independent SSD data
Editor:-October 22, 2012 - Earlier this year I wrote an
and DSP ECC flash techniques - an important new set of technologies working
its way into all SSD markets (except
The new technology can improve
flash SSDs - and adaptive R/W DSP techniques are an essential
prerequisitie for designing reliable TLC (x3) SSDs and all future generations of
One of the characteristics of adaptive DSP is that the ECC
coding and even the size of raw data blocks within the same SSD vary. But the
IP set - which lies behind these technologies is extremely valuable, tightly
controlled and the subject of hundreds of patents.
Where am I going
I think - at this time- data recovery of SSDs which use
adaptive R/W is only feasible by the original manufacturers of the SSDs. It
will be impossible for independent data recovery companies to reverse analyse
the data - because the exact pattern of coding in the flash translation layer
is unique to each SSD and is a mixture of many different coding schemes.
SandForce names trusted partner for SSD data recovery
May 5, 2011 - SandForce
today named DriveSavers
as a member of its trusted partners program.
"DriveSavers was our
first choice to expand the SandForce Trusted program to include
data recovery services,"
said Kent Smith, Senior Director of Product Marketing for SandForce. "While
SandForce SSD Processors eliminate the most common data loss scenarios through
DuraClass NAND flash management features, DriveSavers can provide the SSD a
safety net should the unexpected SSD failure happen and data loss occurs."
Editor's comments:- although many data recovery companies have
to deal with SSDs - some SSDs are difficult or impossible to recover without
the support of the original
This is a significant announcement because it makes SandForce SSDs more
attractive in consumer
markets. Over 90% of consumers don't do reliable
new article - SSD Data Recovery Concepts and Technologies
December 1, 2010 -
published a new article -
Introduction to SSD
Data Recovery Concepts and Technologies - written by Jeremy Brock,
It's hard enough understanding the
design of any single SSD. And there are so many different designs in the
If you've ever wondered what it looks like at the other end of
the SSD supply chain - when a user has a damaged SSD which contains priceless
data with no usable backup - this article - written by one of a rare new
breed of SSD recovery
experts will give you some idea. I've waited more than 3 years to find
someone to write an article on this subject for you. And now it's only a click
away - read the
bad block management in flash SSDs
26, 2010 - StorageSearch.com
today published a new article -
bad block management in flash SSDs.
It's a non technical
introduction to the thinking behind one of the many vital functions inside a
flash SSD controller.
The new article - started out life this morning as a long email reply to one of
my readers with whom I have been discussing various aspects of SSD data
Hyperstone will use Toshiba tech in new SSD controllers
October 12, 2010 -
(Europe) has agreed to provide the company with a variety of
ASIC design and
controllers based on Toshiba semiconductor process technology will sample
in Q1, 2011.
Editor's comments:- Some of Hyperstone's products appear in
consumer SSDs - and this is a market which is noted for its lack of effective
backups (unlike the enterprise
SSD market) - so I asked the company if they had any views about the
emerging SSD data
Axel Mehnert, VP
Marketing at Hyperstone told me - "Regarding data recovery, we do not
really have any particular statement or policy. It depends on the firmware.
Customers (SSD oems) use different implementations. Depending on the deletion
process or errors' root causes data recovery might be possible or, in other
cases, not even desired. Our customers specify their requirements and we adopt
our firmware accordingly. Quite possibly, you will find different types of
implementations when looking at the same controller ID."
SandForce shows x2 SSD controller
Editor:- October 7,
2010 - SandForce
availability of its next generation
family SSD processors - for oems designing
SAS 3 class (6Gbps)
The SF-2000 supports 500MB/s sequential R/W,
60,000 sustained random IOPS, wire speed encryption, end to end
data integrity checks
and industrial temperature operation in a
Also new in this controller generation is support
for sector sizes additional to 512-bytes e.g., 520, 524, 528, 4K, etc., with
Data Integrity Field (DIF) for true enterprise-class SAS drive behavior and
Editor's comments:- one simple way of looking at
the SF-2000 would be as an incremental x2 version of what SandForce has
done before - which also demonstrates that the glass ceiling for their
architecture is much higher than some people might have thought.
briefing yesterday I asked about the data recoverability of the SSDs based on
the new controllers - while acknowledging that the market it was aimed at - the
datacenter- does adequate backups so DR shouldn't be necessary.
Smith, Director of Product Marketing, SandForce told me that in this family of
SSD controllers - the company would be moving even closer towards what already
exists in military SSDs
- and offering the option of having on board
The data in SF-2000 driven SSDs is double encrypted (encrypted on the way in
from the SATA controller and then encrypted again as it is written to the flash
array. The company's view is that it would be impossible for a DR company to
reconstruct data from the flash chips in the SSD without having access to
the SSD oem's unique key generation technology. (The oem has the ability to do
this as a one time programmable function.) Without that data - even SandForce
would be unable to read the contents of the SSD.
These technologies are
designed to make customer data secure. It would be possible for SSD oems to
select DR partners to whom they entrusted their own keys - but that was a
matter for the SSD maker. Proliferation of such data is likely to be restricted
- because otherwise it defeats the security of the product.
SSD Data Recovery - update
Editor:- September 15,
2010 - regular readers of
StorageSearch.com know that
I'm skeptical about the claims which most data recovery companies put on their
web sites about their abilities to
recover data from
General Manager of Memofix
emailed me to say - "I was reading what you were saying about SSD drives
and data recovery. I agree 100% with your view that most recovery companies
cannot deal with most SSD hardware issues. But please remember a large portion
of the cases any recovery company sees are the result of file system damaged or
corruption .. and these cases are easily handled by any half descent DR
"Memofix has only ever seen 3 SSD drives for data
recovery and 2 cases were file system damage including a simple deletion case.
In the other case we were able to replace a non-memory component and make the
device accessible again.
"Additionally we do dozens of
USB flash drives with
anywhere from 1-4 actual memory chips onboard, so we do intimately understand
the intricies of translation tables and putting all the pieces back together."
Editor:- it's good to hear from people who know what they're doing in
this new area of SSD data recovery. David Foster also writes a
blog in which he
reliability and recovery in a more informative way than many others I've
seen - based on his long experience in the industry.
For example - did
you know that 2.5"
drives are more recoverable than 3.5" drives? - I didn't - and would
have expected it to be the other way round.
Recalibrating Consumer Assumptions about SSD Data Recovery
December 7, 2009 - this is an update on the theme of
Data Recovery for
The ability to recover data (or not) from a damaged
flash SSD could become an important way of segmenting SSD products. In this
context (as always) an
defined as a device which has internal
wear-leveling - as
opposed to simpler flash
drives which don't. The loss of data which maps logical to physical
addresses inside the SSD
controller presents a tough challenge for recovery.
Recoverability segments can be broadly defined as
- easily recoverable. This includes devices which have internal
support to facilitate data recovery - designed into the controller architecture.
Although such products are in the design stage - they are not yet widely
available. This type of SSD could be as economic to recover as a current
notebook hard drive.
- recoverable at high cost. This is the case for nearly all flash
SSDs currently shipping. (See comments from a
data recovery expert
- non recoverable (unlucky). This is one step beyond the category
above. Most flash SSDs with internal encryption would not be economic to recover
- if the internal translation tables were corrupted. This includes many new
- and will come as an unwelcome surprise to their owners should they be unlucky
enough to need data recovery services.
been talking to data recovery experts about SSD recovery for many years - but
it's only recently that the market has reached the size where this is starting
to become part of their daily experience.
- non recoverable (intrinsic). These are SSDs which have been
to thwart any prospects for data recovery. In these SSDs - data destruction
circuits are part of the product design - and you pay more for this feature.
Andy Butler, founder
of ABC Data Recovery
today told me - "I have 3 technicians who all trained on NAND
readers. On average we do about 30 per week but can handle more. It gets
more time consuming - therefore costly - as you move into larger SSDs. A
64GB PCIe SSD unit
could take a technician over a week of nonstop work.
" It's more complex than a
RAID recovery, but consumers
assume their data is safe and a recovery will be cheap because it's not a
mechanical repair. As the recovery tools / technology develops we should be
able to speed the process up, but for the time being any SSD over 8GB are
charged on a case by case basis. Anyone with an encrypted SSD should be
warned to backup, if the
controller gets damaged, it's most likely we would only recover encrypted data."
|Data Recovery for flash
SSDs - overview|
|by Zsolt Kerekes,
- editor StorageSearch.com
- May 29, 2015|
|Throughout most of its 7 years short
history - which isn't all that long - compared to the
(admitted) age of the SSD market itself - the market for SSD data
recovery - which is largely aimed at
consumer style SSDs
in notebooks - had been a sterile zone when it came to hard facts and
statistics. But at last - some useful hard data did finally begin to emerge
As previously expected - SSDs are more
hard drives when used
in notebooks - and less likely to need data recovery. (If your interest is
data recovery for hard drives, tape or optical drives - there's
another guide which I
started publishing in the 1990s which has more to help you on those subjects.)
much better are SSDs? About 5x. (See the article below by Intel for
Why do SSDs need data recovery? - The main problem
found in SSDs which were being sent for professional data recovery was data
corruption due to
mechanisms. (See the article below by Gillware for more details.)
related problems were 20x more likely to lead to SSD data recovery than
all other component failures combined.
You might conclude that
endurance is the key driver of the SSD data recovery industry.
when you examine the impact of endurance in the complexity of
SSD controllers and
software - the reality
is that endurance (and its management) underpins everything to do with flash
The idea of the SSD market is to make
flash memory usable as
storage drives - despite endurance.
It's only when all these
mechanisms fail - and when you don't have any other way of restoring your data -
that you have to resort to forensic data recovery.
|"I had to make sure
that I filtered out spammy SSD data recovery companies - who just want your
business - but don't know as much as they should. To be fair - nobody knows all
the answers today (in 2010)."
|Footnotes and disclaimer to the article
SSD Data Recovery
Concepts and Methodologies by the editor of StorageSearch.com|
|"More often than not,
files that were deleted from an SSD drive will not be recoverable. This may not
be what you wanted to hear, but is due to the TRIM command which causes the
SSD controller to physically clear data blocks that were used to store deleted
files. Granted, the actual data wipe doesn't occur immediately..."|
Information from SSDs - blog by Hetman
Software (September 2012)|
Self-Encrypting SSDs (if you think you might need a future data recovery)..."
|That's the "advice" in a blog
SSDs: Flash Technology with
Risks and Side-Effects (August 2013) - by Kroll Ontrack - which
goes on to say - |
"This type of encryption is very secure, but
ensures total data loss in the event of a failure. With SEDs, the encryption
keys are only known to the hardware manufacturers and will not be released.
What this means is in the event of a failure, the data is no longer accessible
to professional data
how data is mapped into the flash memory in a failed unknown
USB flash drive using
reverse engineering methods and poking data into a similar working drive
monitored by a logic analyzer is demonstrated in a
(May 2015) by Gillware Data Recovery|
|"In some ways, blocks
lost due to media corruption present a problem similar to recovering deleted
files. If it is detected quickly enough, user analysis can be done on the
cyclical journal file, and this might help determine the previous state of the
file system metadata. Information about the previous state can then be used to
create a replacement for that block, effectively restoring a file."|
CRCs are important - blog by Thom Denholm Datalight (January
WARNING! - CONSUMER SSD
contents liable to
change without notice
|Editor:- June 13, 2014 - it seems that the risk
of preplanned component substitutions by the original branded SSD maker (rather
than merely the supply chain risk of counterfeits by persons unknown) is another
uncertainty which readers in the consumer SSD market may now have to contend
sudden power loss|
|Why should you care
what happens in an SSD when the power goes down? |
This article will
help you understand why some SSDs which (work perfectly well in one type of
application) might fail in others... even when the changes in the operational
environment appear to be negligible.
|First you learned about SLC
(the good flash). |
|Then you learned about MLC (naughty
flash when it played in the enterprise - but good enough for the short
attention span of consumers). |
Then MLC SSDs learned how to be good.
Now some MLC is much nicer than others. - When it's preceded by an "e"
(extra-good). But it costs more.
But other people say you don't need
the expensive "e" - because their controllers empathize better
with naughty flash. (They really care about naughty flash being sent to bad
block jail too soon.)
Is your head ready to explode yet?
It's going to get even more complicated.
sugaring MLC for
|Can you believe the
word "reliability" in a 2.5" SSD ad?|
Reliability is an
important factor in many applications which use
SSDs.... but can you trust
an SSD brand just because it claims to be reliable?|
As we've seen in
recent years - in the rush for the
SSD market bubble -
many design teams which previously had little or no experience of SSDs were
tasked with designing such products - and the result has been successive waves
of flaky SSDs and
SSDs whose specifications
couldn't be relied on to remain stable and in many products quickly
degraded in customer sites.
||As part of an education
series for SSD product marketers - this case study describes how one company -
which didn't have the conventional background to start off with - managed to
equate their brand of SSD with reliability in the minds of designers in the
embedded systems market. ...read the article|
|What is Data Recovery?|
a Data Recovery Provider
an introduction to SSD
Data Recovery Concepts
the Fastest SSDs
the SSD Heresies
Who's who in SSD?
the SSD Buyers Guide
SSD Jargon Explained
SSD Reliability Papers
Tuning SANs with SSDs
After SSDs... What Next?
Flash SSDs /
the Top 20 SSD Companies
this way to the petabyte
the 1" SSD Market
SSDs - the big
Imprinting the brain of
Flash SSD Reliability
Storage Market Outlook to
MLC for the enterprise
Hard way ahead for
sudden power loss
animal brands in the
I Tire of "Tier Zero Storage"
Data Recovery from Flash
Cache Ratios in flash SSDs
Big versus small -
What's the best /
cheapest PC SSD?
Is the SSD Market
Branding Strategies in
the SSD Market
3 Easy Ways to Enter
the SSD Market
- impacts in notebook SSDs
Overview of the
Notebook SSD Market
35 Years of
SSDs - SSD Market History
a new way of looking
at Enterprise SSDs
capacity - the iceberg syndrome
Power, Speed and
Strength in SSD brands
the Problem with
Write IOPS - in flash SSDs
SSD Myths and
Legends - "write endurance"
Challenges in flash SSD Design
why the notebook
SSD crystal ball is still murky
Market Trends in the
Rackmount SSD Market