Why validation is not the same as testing

Every software team tests. Almost none validate. That’s why audits fail.

When regulatory inspectors walk into your facility, they don’t ask “Did your tests pass?” They ask “Can you prove this software does what users need it to do?” Most development teams can’t answer that question – because they’ve confused testing with validation.

Verification vs validation: the critical distinction

Testing asks: Does the software work as coded?

Validation asks: Does the software meet its intended use and user needs?

These are fundamentally different questions. Testing is verification -confirming you built the product correctly. Validation is validation -confirming you built the correct product.

The classic formulation:

• Verification: “Did we build it right?”
• Validation: “Did we build the right thing?”

A system can pass every test in your suite and still fail validation. Here’s how: imagine building a login form that perfectly implements password hashing, input sanitization, and rate limiting. All tests pass. But if the actual requirement was for single sign-on integration with Azure AD, your perfectly tested code completely fails validation. You built the wrong thing correctly.

Why 100% test coverage doesn’t mean validation

Test coverage measures how much of your code is executed during testing. Validation measures whether your software fulfills its intended purpose for actual users.

Consider these scenarios where high test coverage fails validation:

1. Misunderstood requirements: Tests verify code behavior, but the behavior itself doesn’t match what users need
2. Incomplete requirements: You tested everything you built, but you didn’t build everything required
3. Environmental assumptions: Tests pass in development but the software fails in the actual user environment
4. Usability failures: Functionality works but is unusable for the intended user population

The failure mode is well known in regulated industries: software arrives at validation review with very high test coverage and is rejected, not because any test failed but because nothing in the suite demonstrates that the software meets the user requirements recorded in the Device Master Record. The tests were real. The traceability was missing.

The validation process

Validation isn’t a single event – it’s a documented chain of evidence linking user needs to delivered functionality:

1. User requirements specification (URS)

Document what users actually need the software to do. Not technical specifications – genuine user needs. “Laboratory technicians must be able to record sample test results within 30 seconds per sample” is a user requirement.

2. Functional specifications (FS)

Translate user requirements into functional specifications. “The system shall provide a barcode scanner input field on the sample results screen” translates the user need into a functional requirement.

3. Design specifications (DS)

Define how the software will implement functional requirements. “The barcode input component will use Web Bluetooth API to connect to Honeywell scanners” specifies the technical approach.

4. Code implementation

Build what was designed.

5. Testing (verification)

Confirm the code works as designed. Unit tests, integration tests, system tests – these verify you built it right.

6. Validation testing

Confirm the delivered system meets original user requirements. This requires traceability: for each user requirement, you must demonstrate which tests prove that requirement is satisfied.

7. Traceability matrix

The validation artifact regulators demand: a matrix showing:

• User Requirement ID → Functional Spec ID → Design Spec ID → Test Case ID → Test Evidence

Without this traceability, you cannot prove validation. You can only prove you tested code.

Common validation failures

Calling unit tests “validation”

Unit tests verify code modules work in isolation. Validation confirms the integrated system serves user needs. These aren’t interchangeable.

No traceability to requirements

Teams write excellent tests but can’t map them back to original requirements. When auditors ask “Which test validates requirement URS-047?” there’s no answer.

Validating in production

Validation happens before deployment, not after. If you’re validating in production, you’re using real users as test subjects – which is unacceptable in regulated industries and poor practice everywhere else.

Self-validation

The team that built the software cannot independently validate it. Validation requires independent review – someone who wasn’t involved in development evaluating whether the delivered system meets requirements. This prevents confirmation bias.

Missing environmental validation

Testing in a development environment doesn’t validate software for production use. You must validate in an environment that mirrors actual user conditions – same hardware, same network constraints, same data volumes.

Industries where this distinction is mandatory

Healthcare and pharmaceuticals (FDA)

The FDA requires Computer System Validation (CSV) for any software used in drug manufacturing, medical devices, or clinical trials. 21 CFR Part 11 mandates documented evidence that systems do what they’re supposed to do – validation, not just testing.

Finance (SOX)

Sarbanes-Oxley compliance requires validation of financial reporting systems. You must prove the system accurately implements business rules and calculations – not just that code executes without errors.

Aerospace (DO-178C)

Aviation software certification requires objective evidence that software requirements are validated against system requirements and verified through testing. Both validation and verification are mandatory, separately documented processes.

Automotive (ISO 26262)

Functional safety standards require validation that safety requirements are appropriate and complete, plus verification that implementation meets those requirements. You can’t substitute one for the other.

Why non-regulated teams should care

Even outside regulated industries, the validation mindset prevents expensive failures:

• Reduced rework: Validating requirements early catches misunderstandings before you build the wrong thing
• Better alignment: Traceability ensures what you’re testing actually relates to what users need
• Audit readiness: If you ever need SOC 2, ISO 27001, or customer security reviews, validation documentation answers their questions
• Risk management: Validation identifies gaps between assumptions and reality before users encounter them

How BetterQA approaches validation

At BetterQA, we separate verification from validation in every engagement:

We don’t just run your tests. We prove your software does what it’s supposed to do.

Practical steps to start validating

1. Document user requirements separately from technical specs: Create a URS that describes what users need in their language, not developer language
2. Build traceability from day one: Use requirements management tools (Jira, Azure DevOps, or specialized tools like Jama or Polarion) to link requirements → specs → tests
3. Define acceptance criteria per requirement: Each user requirement needs measurable acceptance criteria that validation testing will prove
4. Plan validation testing separately: Don’t assume your unit/integration tests constitute validation. Design specific validation test cases mapped to requirements
5. Document everything: Validation requires objective evidence. Screenshots, logs, test results, and traceability matrices aren’t overhead – they’re proof
6. Get independent review: Have someone who didn’t build the software review validation evidence to catch gaps and biases

The bottom line

Testing proves your code works. Validation proves your software serves its intended purpose.

Most teams do the first. Regulated industries demand both. Smart teams – regulated or not – recognize that validation catches the expensive mistakes testing never finds: building the wrong thing, missing critical requirements, and deploying software that technically works but fails to deliver user value.

If you can’t trace every test back to a user requirement, and every user requirement forward to test evidence, you haven’t validated. You’ve just tested. And when the audit comes – or worse, when users reject your software – that distinction becomes painfully expensive.

Related reading

• What breaks FDA 21 CFR Part 11 software in real audits – The 7 Part 11 compliance gaps that trigger Form 483 observations
• Why dev-owned testing is not independent QA – Why validation requires organizational independence
• Why feature velocity doesn’t matter without independent QA – How validation prevents building the wrong thing fast

Built by BetterQA