Ideally, risk management occurs throughout the entire lifecycle. If an organization has a test policy document and/or test strategy document, then these will describe the general process by which product and project risks are managed in testing, and show how risk management is integrated into and affects all stages of testing.
In a mature organization where risk awareness pervades the project team, risk management takes place at many levels, and not just for testing. Important risks are not only addressed earlier in particular test levels, but also in earlier test levels. (For example, if performance is identified as a key quality risk area, not only will performance testing begin early in system test, but performance tests will be run during unit and integration testing.) We do not only identify risks, but also the sources of risk and the consequence of those risks will they become outcomes. For those defects that do occur, root cause analysis is used to understand sources of risk more deeply and to implement process improvements that prevent defects in the first place. Mitigation exists throughout the software development lifecycle. Risk analysis is fully informed, considering related work activity, analysis of system behavior, cost-based risk assessment, product risk analysis, end-user risk analysis, and liability risk analysis. Risk analysis transcends testing, with the test team participating in and influencing a program-wide risk analysis.
Most risk-based testing methods also include techniques for using the level of risk to sequence and prioritize the testing, thus ensuring early coverage of the most important areas and discovery of the most important defects during test execution. In some cases, all of the highest-risk tests are run before any lower-risk tests, and tests are run in strict risk order (often called “depth-first”); in other cases, a sampling approach is used to select a sample of tests across all the identified risk items using risk to weight the selection while at the same time ensuring coverage of every risk item at least once (often called “breadth-first”).
Whether risk-based testing proceeds depth-first or breadth-first, it is possible that the time allocated for testing might be consumed without all tests being run. Risk-based testing allows our testers to report to management in terms of the remaining level of risk at this point and allows management to decide whether to extend testing or to transfer the remaining risk onto the users, customers, help desk/technical support, and/or operational staff.
During test execution, the most sophisticated risk-based testing techniques—which need not be the most formal or heavy-weight—allow project participants, project and product managers, executives, senior managers, and project stakeholders, to monitor and control the software development lifecycle, including making release decisions, based on the residual level of risk. This requires our Test Manager to report test results in terms of risk in a way each test stakeholder can understand.