Is manual testing dead, or:
1.Is automation testing enough for your testing needs?
2.Is there something else a QA engineers can do except write automation scripts?
First, let’s break it down – what can we do to obtain and maintain high-quality software products?
I have a saying. We don’t just do mindless testing. My goal for my QA team is to become the binding force, the driving vector that pushes the product quality forward, by becoming the glue between the development team (front-end/back-end), DevOps, project management, product owners, and stakeholders.
How do we do that? We can do this by using quality assurance, which includes quality control and software testing.
But what do we mean when considering QA, QC, and software testing? Well, let’s break it down.
Quality Assurance
- Subset of SDLC
- Process-oriented
- Ensure that processes and procedures are in place to achieve quality
- Focus on the process to achieve the required quality
- Prevent defects
- Whole team approach
- Proactive process
Quality Control
- Subset of QA
- Product-oriented
- Activities to ensure the product quality
- Focus on the product to check for the required quality
- Find and fix defects
- Reactive process
Software Testing
- Subset of QC
- Product-oriented
- Validate the product against specifications
- Focus on actual testing of the product
- Find and fix defects
- Reactive process
And then, what are the responsibilities of a Quality Assurance Engineer? Well, that’s what you see above!
Now let’s think about automation. Can everything be automated out of all the responsibilities we identified above? You got it! I would also say just the latter part, within the software testing column.
Validating the product against the specifications. This is great – we can automate the product’s functionality!
But wait – what happens if the automation script finds a bug – is that a bug? Has that bug already been reported? What’s the ramification of that defect? How critical is that bug? Does that feature already exist in production? Do we need to do a hotfix? Discuss this with the project manager and the product owner.
And this is before the bug is fixed…
Pass it along to the development team, and you can get to a point with them saying that the problem is not happening for them. It doesn’t happen often, but it can, and it did happen.
Once the ticket is fixed, we need to figure out whether the code has been deployed to the server under test.
When the fixed code has been deployed to the testing environment, we need to figure out if something else broke – related to the modified/edited functionality. This happens all too often – fix it in one place, break it in another.
Now let’s think about the steps above where automation can help you. Once the fix is deployed, we have confirmed the problem is no longer happening, and that related functionality has not been affected by regression testing. It’s a great idea to run the automation script related to that feature to ensure no hidden problems are happening on the almost infinite combinations of browsers and operating systems the product requires to work correctly.
Oh, this brings us to another part of the responsibilities of a QA engineer – Identifying what browser and operating systems the product should work on.
Returning to the above table, I hope you will better understand where it makes sense within the software development life cycle.
Brad, why do we still need manual testing in this AI-driven world? It's 2023!
Before the ever-changing trends in software testing, more and more professionals are choosing automation! It is important to note that even if automation is a good approach for your project, it cannot be the only one you use.
Taking a look over the product with the eyes of a QA Engineer, without directly writing automation scripts, you get the human perspective of the product. For example, you cannot automate documentation reviews and product improvements based on reviewing the design and functional requirements before anything gets implemented.
Documentation review, by the way, is the part in which you can get the most cost-effective ways to find and fix bugs – before the development team implements it. Who doesn’t want products shipped earlier? Who doesn’t wish to lower cost per bug fix? I’m sure you know the price per bug fix, depending on where you got that bug fix.
Defect Discovery Cost
Requirements / Architecture
1x
Coding / Implementation
6.5x
Integration / Component Testing
15x
Production / Post-release
100x
Moving forward.
You cannot automate “the need to have a bug fixed” before shipping to production. Often, people think that a tester’s job is finished once we find the existent bugs in a feature.
Now that’s a false statement!
A good QA engineer is the one who makes sure the “needed bugs that need to get fixed” are getting fixed before we ship the new feature to production. Sometimes, the product owner or project manager doesn’t fully understand a ticket and ends up in the backlog. As a result, it’s assigned to be fixed in a faraway sprint – rather than being addressed before the product is released.
The "Holistic Approach".
A good QA Engineer within an SDLC team will use the “holistic approach”; as time goes by, they will understand more about the project. What happens on the other side of the project if you “plant” a new feature right in the middle of it? Wait, Brad, what does this mean?.
Ok, here we go:
We might have web, backend, or mobile development. Each is done by different teams. The backend development team will likely not know how the mobile app looks or works; it won’t know all its front-end features and OS-related behavior. The same for the web development team.
In this “holistic approach,” the QA team will consider all of the project knowledge when new features are integrated. Kind of how we had the Oracle from the Matrix? The QA Engineer’s role will play a big part when they need to figure out how this new feature will integrate within the existing product – with the existing user roles and functionality – be it web, mobile, or backend.
However, once the features are thoroughly tested and the needed bugs to be fixed get squashed, it is a great time to get a happy flow path of the feature and automate it. You don’t want to repeatedly re-run the same test on all iOS or Android OS versions. Or on all desktop browser / OS versions. That’s where the shine and power of automation kick in. Find a repetitive process and automate the hell out of it.
In order to fully grasp the implications of a bug discovered by automation scripts, QA Engineers must determine which product components and features are affected as well as the severity of the issue. Is it a critical one, or is it negligible?
What does a QA engineer do on a project?
As a QA Engineer on a project, you write test cases, find and report bugs, and discuss the gaps within the documentation with the product owner. You talk with the development team about what can be done and why they could not reproduce a certain problem. You push the project forward and ensure that before a release, the product owner and stakeholders understand the severity and implications of certain defects left unfixed – in case they want those problems to get sorted out. You talk with the DevOps team and ensure that the deployment process will actually deploy all the needed resources in the right places, and so on.
Then, however, we use automation testing when running multiple tests on several different devices and/or browsers, which would be very time-consuming to do manually.
For example, a software product might require you to run several tests on iOS 13, iOS 14, iOS 15, and several browsers such as Safari, Chrome, Edge, and Firefox. Testing this manually would take a significant amount of time which is not feasible with the project’s timeline. In that case, you would choose to automate.
As an automation QA, you mostly write test scripts, report bugs, run the tests and adjust the test scripts based on needs/requirements. Anything unable to automate is sent to the manual QA to be tested manually. Suppose the automation QA finds an issue during the automation run. In that case, they will announce the manual QA so they can investigate the issue, reproduce it and create a ticket if it’s a bug.
Let's look at it using an existing project.
Context: A platform that allows patients to build their profiles and transmit them to the public health system. The platform includes explicitly an app that provides users with online forms to fill out manually. After each field has been completed, the app automatically generates a pdf file and a code unique to each file which is then sent to the relevant authorities.
Because the categories in each form contain sensitive data, QA Engineers also need to verify if the generated data in the pdf file is error-free. So that’s it, this is the project – simply put.
This project uses manual and automated testing to endure product quality. Yet, manual testing has proven to be more efficient in detecting bugs. Why, do you ask? Let me explain.
The following examples will showcase the importance of extensive manual testing in favor of automation when required and why it depends on manual testing!
Bug Examples
One of the fields within the .pdf has only eight boxes on the .pdf file, while in the form, you can populate with eight or nine characters. The steps to reproduce this bug are as follows: the tester opens the page, populates the field with nine characters then proceeds to generate the PDF. As expected, with only eight boxes available, the digits don’t populate the boxes accordingly – they show up crowded for the bigger string to fit the text still.
By comparison, QA engineers can trace an automated found bug by checking several fields at once. For example, a delay has been registered when editing each field from a page. After completing a few fields, the editing takes more time with every saving. The expected behavior should save the edited content at approximately the same time for each field. The automation tests show the actual behavior: after editing a few fields, the next ones gradually take more time to save, each longer than the last. This issue was reported to the QA team for further investigation, followed by its reproduction and a ticket if the results are considered a bug.
Conclusion
QA Engineers performing other non-automated “testing-related” activities are still relevant in 2023 because their manual testing approach gives a human perspective of the product and a broader perspective that automation might miss. Also performing manual tests, QA Engineers can observe issues related to UI and UX and other visual components, which can help improve user experience.
Moreover, they become the positive vector that keeps and pushes the standards of quality at the level required for the product to be successful while already released out in the wild.
Automation testing provides swift solutions and high accuracy by employing tools to execute more tests at the same time. Despite automation being time and cost-effective, the part of doing QA without having automation involved in it is still a very much necessary approach.
Stay Updated with the Latest in QA
The world of software testing and quality assurance is ever-evolving. To stay abreast of the latest methodologies, tools, and best practices, bookmark our blog. We’re committed to providing in-depth insights, expert opinions, and trend analysis that can help you refine your software quality processes.
Delve deeper into a range of specialized services we offer, tailored to meet the diverse needs of modern businesses. As well, hear what our clients have to say about us on Clutch!