End-to-end testing checks a complete user journey through the real, assembled system, from the first click to the last side effect, the way a customer would go through it. An end-to-end test signs up, pays or books something, then confirms that the thing actually happened: the order exists, the email was sent, the account is on the new plan.
That part is simple. The harder question, and the one most explainers skip, is how many of these tests you should own, because they are the most expensive tests there are. This guide covers what end-to-end tests are for, where they stop paying for themselves, a worked example, and the ways they fail in practice.
What end-to-end testing means
An end-to-end (E2E) test drives the product through its real interfaces, usually the browser or the mobile UI, against a deployed build with its real database, services and integrations wired together. Nothing on the path is mocked except what you cannot or should not touch, such as a live card network, which is replaced by the payment provider’s sandbox.
Three properties separate it from the other test types:
- Its scope is a journey, not a component. A unit test checks one function. An integration test checks that two parts agree with each other. An end-to-end test checks that a person can get from intent to outcome.
- It runs against an assembled environment. Staging, a preview environment or a dedicated test environment. If half the system is stubbed, what you have is an integration test with a browser attached.
- Its assertion is a business outcome. “The confirmation page appeared” is a weak end-to-end assertion. “The order is stored with the right total, and the warehouse system received it” is the real one.
You will also see E2E testing split into horizontal and vertical. A horizontal test follows a user across several applications: a customer orders on the web shop, and a warehouse user picks the order in a separate back-office tool. A vertical test goes down through the layers of one application, from UI to API to database, for a single feature. Most products need a few of each.
How it compares with other test types
| Test type | What it proves | Typical run time | Maintenance cost |
|---|---|---|---|
| Unit | One function or class behaves correctly | Milliseconds | Low |
| Integration or API | Two components or services agree on a contract | Seconds | Medium |
| System | The whole application meets its stated requirements | Minutes to hours | Medium |
| End-to-end | A real user journey produces the right outcome across everything it touches | Minutes | High |
The run time and maintenance columns are why nobody builds a whole test strategy out of E2E tests. Google’s testing team made the argument in a 2015 post titled Just Say No to More End-to-End Tests, and suggested a 70/20/10 split between unit, integration and end-to-end tests “as a good first guess”. It is a starting ratio, not a law. Martin Fowler’s test pyramid makes the same point in a picture: many small, fast tests at the bottom, a handful of broad ones at the top.
When end-to-end testing is worth it
Write an end-to-end test when a failure would cost real money or trust and no cheaper test can see it. In practice that means:
- Revenue and sign-up paths. Registration, checkout, subscription upgrade, booking. If these break, the business finds out from its bank balance.
- Journeys that cross systems you do not own. Single sign-on, a payment provider, an email or SMS gateway, a CRM sync. Each hand-off is a place where two correct components can still produce a wrong result together.
- Behaviour that only exists in the assembled product. Permissions that depend on configuration, feature flags, caching, the order in which services start.
- The short list you would check by hand before a release anyway. If a tester already clicks through the same five journeys every Friday, those five are your first automated E2E tests.
A good E2E suite is small enough that one person can name every journey in it and say why it is there. A suite of hundreds usually means lower-level checks have been pushed up to the UI because that is where the tooling happened to be.
When it is not
Skip the end-to-end test when a smaller test can give the same answer:
- Validation rules and calculations. Whether a VAT number is rejected, or a discount rounds correctly, belongs in a unit or API test that runs in milliseconds.
- Every variation of a form. One E2E test that the form submits, plus API tests for the fifty input combinations.
- Screens still changing every sprint. The test will be rewritten more often than it runs green. Test the API underneath until the UI settles.
- Anything that needs a human judgement. Whether a page is confusing or an error message is helpful. That is exploratory testing, and it is not going away.
A worked example: a plan upgrade in a SaaS product
Take a B2B invoicing product with a free and a paid plan. The journey worth an E2E test is “a free customer upgrades to the paid plan and can immediately use a paid feature”. It crosses the web app, the billing service, the payment provider’s sandbox, the invoice generator and the permissions check.
Written as steps a tester, or a script, would follow:
- Arrange the data through the API, not the UI. Create a fresh free-plan account with a unique email for this run. Creating it through the sign-up screen would make this test depend on sign-up working, which has its own test.
- Log in as that user through the real login page.
- Open Billing, choose the paid plan, and pay with the payment provider’s documented test card.
- Check the UI outcome. The plan badge reads “Paid”, and the paid-only “Recurring invoices” menu item is now visible.
- Check the business outcome behind the UI. Query the billing API: the subscription is active, the amount matches the price list, and exactly one invoice exists for this account. This is the step most E2E tests leave out, and it is the one that catches the defect where the page says “Paid” and the backend never recorded it.
- Use the paid feature once. Create one recurring invoice and confirm it saves. An upgrade that unlocks a menu item without unlocking the permission behind it is a real and common bug.
- Clean up. Cancel the subscription and delete the account through the API, so the next run starts from the same state.
Notice what is not in it: no check of every price, no test of card declines, no copy review. Declines and prices are API tests against the billing service. Seven steps, one journey, and a clear answer if it goes red: customers can or cannot pay you.
Recording tools can produce a first draft of a journey like this. Playwright’s code generator, Cypress Studio and our own browser recorder, Flows, all do it. What they cannot do is step 5 or step 7, which is why a recording should be treated as a starting point that a tester edits, not a finished test.
Why E2E suites fail in practice
They go flaky, and then nobody believes them
A flaky test passes and fails on the same code. Google measured this across its own test corpus in 2016: about 1.5% of all test runs reported a flaky result, almost 16% of its tests had some level of flakiness, and about 84% of the pass-to-fail transitions it observed involved a flaky test. The last figure is the expensive one. Most of the time a test turned red, someone first had to work out whether the test or the code was at fault, and Google notes that real failures in flaky tests commonly get ignored because of all the false alarms.
End-to-end tests are the most exposed, because they touch the most moving parts: network calls, animations, third-party scripts, data left behind by the previous run.
They wait for the wrong thing
The most common root cause of flakiness is timing. In a study of 201 commits that likely fix flaky tests across 51 open-source projects, Luo and colleagues at the University of Illinois found that 45% of the categorised fixes (74 of 161) were for “async wait” problems: the test did not properly wait for an asynchronous result before checking it. Concurrency and test order dependency came next, and the three together covered 77%.
In E2E terms, that is the fixed sleep(3) that works on a developer’s laptop and fails on a loaded CI runner. Modern frameworks wait for elements to be ready before acting on them, as the Playwright auto-waiting and Cypress retry-ability docs describe. That removes a lot of the problem but not all of it: the framework knows when a button is clickable, not when your background job has finished writing the invoice. Wait for the outcome you are about to assert, never for a number of seconds.
Tests share data
Two tests that use the same account will eventually run in parallel and change each other’s state. Each E2E test should create what it needs, with unique identifiers, and remove it afterwards, as in the worked example. A suite that only passes in a fixed order is a suite that will fail the first time someone turns on parallel execution.
The environment is the real subject
A staging environment with an expired certificate, a sandbox that rate-limits, or a feature flag set differently from production will fail tests that are correct. Keep a short health check that runs before the suite and fails loudly with “environment not ready”, so an environment problem is never reported as a product bug.
Red runs get retried until they are green
Automatic retries hide real intermittent defects, the kind customers do hit. If you retry, record every retry, and treat a test that needed one as a finding to investigate, not a pass.
Keeping an E2E suite useful
Set a rule for flaky tests before you need it: a test that flakes twice in a week is moved to a quarantine list with a named owner and a fix-by date, and it stops blocking releases while it is there. If it is not fixed by the date, it is rewritten at a lower level or deleted. A quarantine without an expiry date is where tests go to be forgotten.
Review the suite against the regression testing question of what is worth running on every change, and keep the fastest few journeys as the smoke test that gates each deployment. Write the list of journeys, the environment and the exit criteria into your test plan so that the scope is a decision, not an accident. And push every check that does not need the whole journey down to API testing, where it runs in a fraction of the time.
For the wider picture of what to automate and at which level, see our guide to test automation.
Frequently asked questions
Is end-to-end testing the same as system testing? Not quite. System testing checks the whole application against its requirements, often one feature at a time. End-to-end testing follows a user journey, which may cross several systems, and asks whether the outcome is right. The two overlap, and many teams use the terms loosely.
Should end-to-end tests be automated? The stable, high-value journeys, yes, and they should run on every deployment to a shared environment. Keep some manual end-to-end passes before major releases, because a person notices things an assertion was never written for.
Who should own end-to-end tests? Whoever will fix them when they break. In most teams that is QA, with developers responsible for testability: stable selectors, a way to create test data through an API, and an environment that stays up.
Where BetterQA fits
BetterQA is an independent software testing company in Cluj-Napoca. We build and maintain end-to-end suites as part of our test automation services, and we take over existing flaky suites and make them trustworthy again. If you would rather hand the whole QA function to a dedicated team, see QA outsourcing.
Sources
- Google Testing Blog, Just Say No to More End-to-End Tests, 2015.
- Google Testing Blog, Flaky Tests at Google and How We Mitigate Them, 2016.
- Luo, Hariri, Eloussi and Marinov, An Empirical Analysis of Flaky Tests, FSE 2014.
- Martin Fowler, TestPyramid.
Need help with software testing?
BetterQA provides independent QA services across manual testing, automation, security audits, and performance testing. ISO 27001, 9001, 14001 and 13485 certified.
