Overlooked Configuration; Software Test
by Leo Clark
Every software development organization has people and equipment dedicated to "quality." Sometimes the measure of quality is suspect. In some dysfunctional organizations, the measure of quality is "two hours of testing without finding any significant bugs." If it meets that test, SHIP IT! or GO LIVE!
More reasonable organizations may set up a test station in the conference room. Every employee is expected to hammer on the code during their latte break and see if they can break it. The theory states that this additional user pressure must surely find all the problems.
More mature organizations pay people to provide "quality assurance." This definition of quality is like positioning at the end of the assembly line. Not a bad improvement but certainly not evidence of a truly mature organization.
Software QA needs dedicated resources, professionals and tools, even in the most chaotic shops. However, these tools and procedures should be subject to the same positive management control that world class organizations exhibit. Unfortunately, they are woefully underserved. The software test environment is a configuration, like any other solution. It contains a platform, operating system, utilities, automated testing tools, test data sets, metric gathering and reporting applications and other necessary components. A world class organization would know explicitly each component of the environment and measure to see if the environment conforms before testing.
Most less-than-mature companies have no idea what the configuration of the environment should be. They don't even know what they have. So much software has been installed and deleted, without control, that the software test environment, itself, is a mystery. The resulting measure of quality should also be suspect.
Some software test automation tools, such as WinRunner or SQAtest, are an essential part of the environment configuration. However, the QA department may have switched tools, applied patches, deleted configuration files, or made any number of unauthorized modifications to the tool or the test sets. Without positive management control, why is anyone surprised when customers complain about products which were tested and passed. The correct fix is to change the process.
The correct software test environment is the result of proper analysis, design and construction. Analyze the requirements that the software must exhibit. Design tests that cause the software to demonstrate correct behavior. Then construct a test environment to perform those tests.
The CMII model provides the best way to allocate resources for each step in the process. The software test environment is a physical item hierarchy. Use the CMII model to define the hierarchy and then build one that conforms to the design. The software test environment is a product. Manage it that way. Your organization's reputation is a direct reflection of your ability to manage the delivery systems you employ. If you or your customers are not happy with that reflection, its time to use CMII to change.
Institute of Configuration Management Scottsdale, AZ 85261-5656 Tel: (480) 998-8600 Fax: (480) 998-8923 Email: info@icmhq.com