Business Value of Testing

Testing excessively does not deliver good business value because the testing will impose unreasonable delays and cost more than it saves. Testing too little does not deliver good business value because too many defects will be delivered to users. The optimum lies between those two extremes. The Test Manager must help testing stakeholders understand this optimum and the business value of testing.

While most organizations consider software testing valuable in some sense, few managers, including Test Managers, can quantify, describe, or articulate that value. In addition, many Test Managers, test leads, and testers focus on the tactical details of testing while ignoring the larger strategic issues related to testing that other project participants, especially managers, care about.

Software testing delivers value to the organization, project, and/or operation in both quantitative and qualitative ways:

Quantitative values include finding defects that are prevented or fixed before release, finding defects that are known before release, reducing risk by running tests, and delivering information on the project, process, and product status.

Qualitative values include an improved reputation for quality, smoother and more predictable releases, increased confidence, protection from legal liability, and reduced risk of loss of whole missions or even lives.

Test Managers should understand which of these values apply to their organization, project, and/or operation and be able to communicate about software testing in terms of these values.

A well-established method for measuring the quantitative value and efficiency of testing is called cost of quality. Cost of quality involves classifying project and operational costs into four categories related to product defect costs:

Costs of prevention, e.g., training developers to write more maintainable or secure code

Costs of detection, e.g., writing test cases, configuring test environments, and reviewing requirements

Costs of internal failure, e.g., fixing defects detected during testing or reviews before delivery

Costs of external failure, e.g., support costs associated with defective software delivered to customers

A portion of the testing budget is the cost of detection. At the same time, the remainder is a cost of internal failure. The total costs of detection and internal failure are typically well below the costs of external failure, which makes testing an excellent value. By determining the costs in these four categories, Test Managers can create a convincing business case for testing.