|
|
|
|
|
|
| Editor's intro:- In the data storage industry, testing is generally considered a neccessary evil in bringing a product to market and as a result often companies will do the bare minimum neccessary in order to cross the item off its checklist. This whitepaper (published here in April 2005) attempts to explain why testing is so important to product validation and also to provide a concise methodology to testing by focusing on the three main components of proper test design, test execution, and analysis of the test results. | |||
| |||
Test to Fail Too often, testing groups tend to run tests intended to pass a storage peripheral without understanding what a passing result means. As a storage community, we need to commit to designing tests that push the limits of the technology. This means continually evaluating your test plan and increasing the intensity and/or complexity of a test until failures can consistelntly be generated and real watermarks for pass/fail criteria can be established. Creating tests that are too lenient or that circumvent known issues only leads to poor product quality and customer dissatisfaction which is the main reason we test in the first place. Testing to fail also requires the intentional injection of errors to test the robustness and recovery capabilities of your device. The more failure scenarios that you test for the better prepared your data storage solution will be for the real world, where the unexpected will happen due to the uncontrolled environment. Being able to gracefully manage these scenarios is a sign of quality that will be appreciated by your customers. Test Smart One of the key ingredients in knowing where to begin in your test design is know the limitations of both the products being tested and the tools available for test. Depending on the peripheral being tested these limitations could include maturity, interoperability, speed or a host of other issues. Identify these limitations in the peripheral to create a clear understanding of the test objectives and a realistic expectation of the results. Limitations exist in test solutions as well such as multi-threading, tagged queuing, remote control access, automation capabilities and low level protocol control. Understand which features are necessary in a testing solution for your particular testing application and make an informed decision on which solution is right for you. Another important facet of "smart" testing is creating test plans. Test plans should be created for each stage of your peripheral's development from Engineering Verification Testing all the way through to Manufacturing and Field Service. This ensures that no matter which stage of the peripheral's life cycle you are in, there is a process in place to ensure that it is functioning to the level expected. In qualifying a storage peripheral, it is important to note that without statistical significance, there is no significance to your results. This means that you need to test a relatively large sample of product (when applicable) to ensure that once testing is complete, enough data, variety of data, mechanical motion etc. has been exercised through each component to draw a significant, confident conclusion. Manage Your Vendors Managing vendors is sometimes a forgotten art. I can't even begin to count the number of instances where a call to a vendor application engineer could have quickly solved problems that had been dragging on at various customer sites. Developing a good working relationship with vendor support engineers / application engineers is key to receiving timely updates and avoiding days of needless debugging. Thorough testing accompanied by the proper documentation to back up your assertions of problems is the key to establishing a rational discussion with a vendor and getting the problems taken seriously and resolved quickly. Provide descriptions of what was being tested at the time, a copy of your test (for reproduction), any error logs from the resulting failure and of course protocol traces if applicable. This enables your vendor to do their job more quickly yielding in more rapid root cause failure analysis and problem resolution. Besides product updates, vendor engineers can also provide useful tips and tricks for testing their products as well as lists of outstanding and resolved issues which can help to pinpoint problems in the field and in the lab. Remember that vendor data for MTBF (Mean Time Between Failures), read/write error rates and all other statistics presented should be used as temporary watermarks until real results can be ascertained in your own test environments. Summary As we have discussed in this paper, testing is an interactive, multifaceted process that requires an understanding of the quality required and determines step by step processes to achieve and maintain that quality in all stages of a products life cycle. It should be a dynamic process that evolves as the technology being tested necessitates. It must push the test subject to its limits and ensure that the quality customers expect is found in each and every product sold. Testing should not be viewed as a necessary evil but rather a means of achieving quality, confidence, and ultimately customer satisfaction. ...Extreme Protocol Solutions profile | |||
|
|
 | ||||||
| ..... | ||||||
| ||||||
| ..... | ||||||
| ||||||
| ..... | ||||||
| ||||||
| ... | ||||||
