Testing tasks may be done by people in a specific testing role, or by people in another role (e.g., customers). A certain degree of independence often makes the tester more effective at finding defects due to differences between the author’s and the tester’s cognitive biases. Independence is not, however, a replacement for familiarity, and developers can efficiently find many defects in their own code.
Degrees of independence in testing include the following (from low level of independence to high level):
- No independent testers; the only form of testing available is developers testing their own code
- Independent developers or testers within the development teams or the project team; this could be developers testing their colleagues’ products
- Independent test team or group within the organization, reporting to project management or executive management
- Independent testers from the business organization or user community, or with specializations in specific test types such as usability, security, performance, regulatory/compliance, or portability
- Independent testers external to the organization, either working on-site (in-house) or off-site (outsourcing).
For most types of projects, it is usually best to have multiple test levels, with some of these levels handled by independent testers. Developers should participate in testing, especially at the lower levels, so as to exercise control over the quality of their own work.
The way in which independence of testing is implemented varies depending on the software development lifecycle model. For example, in Agile development, testers may be part of a development team. In some organizations using Agile methods, these testers may be considered part of a larger independent test team as well. In addition, in such organizations, product owners may perform acceptance testing to validate user stories at the end of each iteration.
Having a software testing team on site has its pros and cons. Here, we are going to compare local vs independent QA teams.
Disadvantages of working with an in-house team:
- A chef should not certify the dish they made. Developers who coded the product should not also be testing it. Businesses hire objective testing teams for a reason; the most notable is that a developer shouldn’t certify their code. If they do, that just outrightly infringes the central principle of software testing.
- Independent testers are likely to recognize different kinds of failures compared to developers because of their different backgrounds, technical perspectives, and biases
- An independent tester can verify, challenge, or disprove assumptions made by stakeholders during the specification and implementation of the system
- Independent testers of a vendor can report in an upright and objective manner about the system under test without (political) pressure of the company that hired them
- The organization that created the software (development firm) can be biased toward testing the same software. The unbiased and fresh view reveals defects that can’t be discovered otherwise but can be essential for the market success of the software product
- We regularly see testers reaching to the development manager – that’s harmful to the project. You cannot get the best out of your QA leads and QA engineers if they report issues to the person whose other team is responsible for it. The mindset of a QA is a lot different in this case, and they are more worried about what to present in the defect report than whether they tested the product thoroughly. In my personal experience, I have seen Project Managers walking over to the QA team and asking (ordering) not to reject the build as that would mean a poor show by the development team. In such scenarios, the metrics might be manipulated, and so is the quality. When a tester is entirely free from inhibitions, they can seek out more defects or discrepancies than those already working within the developing team.
- Developers sometimes don’t like the idea of a QA team member in the same office. Since developers expect their code to be correct, they have a confirmation bias that makes it difficult to accept that the system is incorrect. In addition to confirmation bias, other cognitive biases may make it difficult for people to understand or accept information produced by testing. Further, it is a universal human trait to blame the bearer of bad news, and reports produced by testing often contain bad news.
- Limited staff pool. Having a personal QA team will require finding talents from a limited pool of candidates in your surrounding area, along with their management and training expenses.
Advantages of working with an unbiased remote team:
- Less management work. The hassle of hiring and training testers is eliminated.
- We are always more attentive and careful when we are told that someone is going to review our work. Knowing that there is an independent software testing team that is going to find bugs in the software brings that new discipline within the development team.
- A pure QA firm would align the quality goals to that of their customer and not the development firm. The SLAs are decided between the customer and the QA firm. There are entry and exit criteria, but working under the same roof and for the same organization puts all that on the back burner.
- 3rd Party Independent software testing brings in impartiality to the table. The metrics, data, and reports are often correct and not cooked up. You get a fair assessment of the quality of code. You can hold QA accountable for any bugs that get leaked into production. And you can have the development team responsible for not meeting the desired code quality.
- Customers can rely on one vendor to take care of all their QA needs. You are more likely to find all skills under one roof if you work with an independent QA firm than if you were to outsource QA to the development firm. It is much easier for a QA company to attract and retain QA talent than it is for a software development firm.
- Testing and reporting bugs to developers require confidence, self-assurance, and the ability to think creatively. Good testers become familiar with all the nuances of an entire application system and are efficient at testing. They have a broader knowledge of the application’s regression history than any developer.
- Testers perform non-obvious functions that push an application in different directions, often where it was never intended to go. They believe in defects and don’t accept that bugs are fixed unless they have proof. They are unafraid to try and fail. Failing to find a bug the first time around means they need to execute tests with additional creativity.
- QA testers work best within a team of developers because both groups are more productive when they work closely together. Pure code-based testing fails because it lacks the human factor. Humans do interesting things to applications in ways that are often surprising. QA testers enhance the success of coded tests by providing a human eye, and a human element, to help anticipate that.
Note: the information provided on this document can also appear in the ISTQB documentation too.