|Editor's intro:- In
the data storage industry, testing is generally considered a neccessary evil in
bringing a product to market and as a result often companies will do the bare
minimum neccessary in order to cross the item off its checklist. This whitepaper
(published here in April 2005) attempts to explain why testing is so important
to product validation and also to provide a concise methodology to testing by
focusing on the three main components of proper test design, test execution, and
analysis of the test results.|
Test to Fail
often, testing groups tend to run tests intended to pass a storage peripheral
without understanding what a passing result means.
As a storage
community, we need to commit to designing tests that push the limits of the
technology. This means continually evaluating your test plan and increasing the
intensity and/or complexity of a test until failures can consistelntly be
generated and real watermarks for pass/fail criteria can be established.
Creating tests that are too lenient or that circumvent known issues only leads
to poor product quality and customer dissatisfaction which is the main reason we
test in the first place. Testing to fail also requires the intentional injection
of errors to test the robustness and recovery capabilities of your device. The
more failure scenarios that you test for the better prepared your data storage
solution will be for the real world, where the unexpected will happen due to the
uncontrolled environment. Being able to gracefully manage these scenarios is a
sign of quality that will be appreciated by your customers.
One of the key ingredients in knowing where to begin in your
test design is know the limitations of both the products being tested and the
tools available for test. Depending on the peripheral being tested these
limitations could include maturity, interoperability, speed or a host of other
issues. Identify these limitations in the peripheral to create a clear
understanding of the test objectives and a realistic expectation of the results.
Limitations exist in test solutions as well such as multi-threading, tagged
queuing, remote control access, automation capabilities and low level protocol
control. Understand which features are necessary in a testing solution for your
particular testing application and make an informed decision on which solution
is right for you.
Another important facet of "smart" testing is creating test
plans. Test plans should be created for each stage of your peripheral's
development from Engineering Verification Testing all the way through to
Manufacturing and Field Service. This ensures that no matter which stage of the
peripheral's life cycle you are in, there is a process in place to ensure that
it is functioning to the level expected. In qualifying a storage peripheral, it
is important to note that without statistical significance, there is no
significance to your results. This means that you need to test a relatively
large sample of product (when applicable) to ensure that once testing is
complete, enough data, variety of data, mechanical motion etc. has been
exercised through each component to draw a significant, confident conclusion.
Managing vendors is sometimes a forgotten art. I can't
even begin to count the number of instances where a call to a vendor application
engineer could have quickly solved problems that had been dragging on at various
customer sites. Developing a good working relationship with vendor support
engineers / application engineers is key to receiving timely updates and
avoiding days of needless debugging.
Thorough testing accompanied by the proper documentation to back up
your assertions of problems is the key to establishing a rational discussion
with a vendor and getting the problems taken seriously and resolved quickly.
Provide descriptions of what was being tested at the time, a copy of your test
(for reproduction), any error logs from the resulting failure and of course
protocol traces if applicable. This enables your vendor to do their job more
quickly yielding in more rapid root cause failure analysis and problem
Besides product updates, vendor engineers can also provide useful tips
and tricks for testing their products as well as lists of outstanding and
resolved issues which can help to pinpoint problems in the field and in the lab.
Remember that vendor data for MTBF (Mean Time Between Failures), read/write
error rates and all other statistics presented should be used as temporary
watermarks until real results can be ascertained in your own test
As we have discussed in this
paper, testing is an interactive, multifaceted process that requires an
understanding of the quality required and determines step by step processes to
achieve and maintain that quality in all stages of a products life cycle. It
should be a dynamic process that evolves as the technology being tested
necessitates. It must push the test subject to its limits and ensure that the
quality customers expect is found in each and every product sold. Testing should
not be viewed as a necessary evil but rather a means of achieving quality,
confidence, and ultimately customer satisfaction. ...Extreme Protocol Solutions
sudden power loss|
|Why should you care
what happens in an SSD when the power goes down? |
This important design
feature - which barely rates a mention in most SSD datasheets and press releases
- has a strong impact on
SSD data integrity
This article will help you understand why some
SSDs which (work perfectly well in one type of application) might fail in
others... even when the changes in the operational environment appear to be
|How big was the
thinking in this SSD's design?|
|Does size really does matter in SSD
By that I mean how big was the mental map? - not how many
inches wide is the SSD.
The novel and the short story both have their
place in literature and the pages look exactly the same. But you know from
experience which works best in different situations and why.
it comes to SSDs - Big versus Small SSD architecture - is something which was
in the designer's mind. Even if they didn't think about it that way at the time.
||For designers, integrators,
end users and investors alike - understanding what follows from these simple
choices predicts a lot of important consequences. ...read the article|