Test Improvement Process

Test Improvement Process

Jlust as organizations use testing to improve software, process improvement techniques can be selected and used to improve the process of developing software and the resulting software deliverables. Process improvement can also be applied to the testing processes. Different ways and methods are available to improve the testing of software and of systems containing software. These methods aim at improving the process, and hence the deliverables, by providing guidelines and areas for improvement.

Testing often accounts for a major part of the total project costs. However, only limited attention is given to the test process in the various software process improvement models, such as CMMI®.

Test improvement models such as the Test Maturity Model integration (TMMi®), Systematic Test and Evaluation Process (STEP), Critical Testing Processes (CTP) and TPI Next® were developed to address the lack of attention to testing in most software process improvement models. Properly used, these models can provide a degree of cross-organization metrics that can be used for benchmark comparisons.

The models presented in this syllabus are not intended to be a recommendation for use, but are presented here to provide a representative view of how the models work and what they include.

Introduction to Process Improvement

Process improvements are relevant to the software development process as well as to the testing process. Learning from one’s own mistakes makes it possible to improve the process that organizations are using to develop and test software. The Deming improvement cycle: Plan, Do, Check, Act, has been used for many decades, and is still relevant when testers need to improve the process in use today.

One premise for process improvement is the belief that the quality of a system is highly influenced by the quality of the process used to develop the software. Improved quality in the software industry reduces the need for resources to maintain the software and thus provides more time for creating more and better solutions in the future. Process models provide a place to start improving, by measuring the organization’s process capabilities against the model. The models also provide a framework for improving the organization’s processes based on the outcome of an assessment.

A process assessment leads to a process capability determination, which motivates a process improvement. This may invoke a subsequent process assessment to measure the effect of the improvement.

Types of Process Improvement

The use of assessment models is a common method that ensures a standardized approach to improving test processes using tried and trusted practices.

Process improvement models are categorized into two types:

  1. The process reference model which provides a maturity measurement as part of the assessment in order to evaluate an organization’s capability compared with the model, to evaluate the organization within the framework, and to provide a roadmap for improving the process.
  2. The content reference model which provides business-driven evaluations of an organization’s opportunities to improve including, in some cases, benchmarking against industry averages using objective measurements. This evaluation can be used to create a roadmap for improving the process.

Improving the Testing Process

The IT industry can work with test process improvement models to reach a higher level of maturity and professionalism. Industry standard models are helping to develop cross-organization metrics and measures that can be used for comparison. Out of the need for process improvement in the testing industry, several sets of recommended processes have materialized. These include STEP, TMMi, TPI Next and CTP. The staged models, such as TMMi and CMMI, provide standards for comparison across different companies and organizations. The continuous models, such as CTP, STEP and TPI Next, allow an organization to address its highest priority issues with more freedom in the order of implementation. These are each discussed further in this section.

All of these models allow an organization to determine where it stands in terms of its current test processes. Once an assessment is performed, TMMi and TPI Next suggest a roadmap for improving the test process. Alternatively, STEP and CTP provide the organization with means to determine where its greatest process improvement return on investment will come from and leave it to the organization to select the appropriate roadmap.

Once it has been agreed that test processes should be reviewed and improved, the process improvement implementation steps to be adopted for this activity could be as defined in the IDEALSM model:

  • Initiating the improvement process
  • Diagnosing the current situation
  • Establishing a test process improvement plan
  • Acting to implement improvement
  • Learning from the improvement program

Initiating the improvement process

Before the process improvement activities start, the objectives, goals, scope and coverage of the process improvements are agreed on by the stakeholders. The choice of the process improvement model is also made at this time. The model may either be selected from publically available options (such as CTP, STEP, TMMi, and TPI Next) or developed internally. In addition, success criteria should be defined and a method by which they will be measured throughout the improvement activity should be determined.

Diagnosing the current situation

The agreed assessment approach is undertaken and a test assessment report is created which contains an appraisal of current testing practice and a list of possible process improvements.

Establishing a test process improvement plan

The list of possible process improvements is prioritized. The prioritization could be based on return on investment, risks, alignment with organizational strategy, and/or measurable quantitative or qualitative benefits. Having established the priority order, a plan for the delivery of the improvements is developed.

Acting to implement improvement

The test process improvement plan for the delivery of the improvements is implemented. This could include any training or mentoring required, piloting of processes and ultimately their full deployment.

Learning from the improvement program

Having fully deployed the process improvements, it is essential to verify which benefits (of those established earlier in addition to potentially unexpected benefits) were received. It is also important to check which of the success criteria for the process improvement activity have been met.

Depending on the process model used, this stage of the process is where monitoring of the next level of maturity starts and a decision is made to either start the improvement process again, or to stop the activity at this point.