Risk-Based Testing

Risk-Based Testing

Risk is the possibility of a negative or undesirable outcome or event. Risks exist whenever some problems may occur which would decrease customer, user, participant, or stakeholder perceptions of product quality or project success. Where the primary effect of the potential problem is on product quality, potential problems are referred to as quality risks, product risks, or product quality risks. Where the primary effect of the potential problem is on project success, potential problems are referred to as project risks or planning risks.

In risk-based testing, quality risks are identified and assessed during a product quality risk analysis with the stakeholders. Our test team then designs, implements, and executes tests to mitigate quality risks. Quality includes the totality of features, behaviors, characteristics, and attributes that affect customer, user, and stakeholder satisfaction. Therefore, a quality risk is a potential situation where quality problems might exist in a product. Examples of quality risks for a system include incorrect calculations in reports (a functional risk related to accuracy), slow response to user input (a non-functional risk related to efficiency and response time), and difficulty in understanding screens and fields (a non-functional risk related to usability and understandability). When tests reveal defects, testing has mitigated quality risk by providing the awareness of defects and opportunities to deal with them before release. When tests do not find defects, testing has mitigated quality risk by ensuring that, under the tested conditions, the system operates correctly.

Risk-based testing uses product quality risks to select test conditions, to allocate test effort for those conditions, and to prioritize the resulting test cases. A variety of techniques exists for risk-based testing, with significant variation both in the type and level of documentation gathered and in the level of formality applied. Whether explicitly or implicitly, risk-based testing has the objective of using testing to reduce the overall level of quality risk, and, specifically to reduce that level of risk to an acceptable level.
Risk-based testing consists of four main activities:

  • Risk identification
  • Risk assessment
  • Risk mitigation
  • Risk management

These activities overlap. The following subsections will discuss each of these activities.

To be most effective, risk identification and assessment include representatives of all project and product stakeholder groups, though sometimes project realities result in some stakeholders acting as surrogates for other stakeholders. For example, in mass-market software development, a small sample of potential customers may be asked to help identify potential defects that would impact their use of the software most heavily; in this case, the sample of potential customers serves as a surrogate for the entire eventual customer base. Because of their particular expertise with product quality risks and failures, our testers are actively involved in the risk identification and assessment process.