The new store was ready to launch. The basket was still on the old platform.
For COOs and e-commerce leads about to move a store to a new platform.
An online retailer was moving its storefront from a legacy commerce platform to a new headless stack. We automated every user acceptance case against the staging site.
- Retail
- Online store migration to a headless stack
- Money
- Clients
- Reputation
What was at risk
Customers lost at the last step before checkout
The basket, the last step before checkout, sat on a different platform from the rest of the journey.
No safety net after launch
None of the retailer's 158 acceptance cases was automated before. Every later release would have depended on manual re-checking.
A go-live decided because pages load
A migration looks finished when every page loads. 65 bugs were filed: 1 critical, 21 high, 33 medium and 10 low.
Standards and rules in play
- The acceptance cases are the contractAll 158 user acceptance cases were automated, as 200 test cases, and run against the migration's staging site.
- Go-live ruleOur report recommended that critical findings block go-live unless the business explicitly accepts them.
Context: retailers that take card payments work under PCI DSS.
How we tested it
- Test level
- Acceptance testing: the retailer's user acceptance cases, automated and run through the user interface of the staging site.
- Test types
- Functional testing, end to end across the legacy and the new platform, as a regression suite.
- Test basis
- The retailer's 158 user acceptance cases.
- How we workedScenario-based testingWhere it showed upCases follow what a customer does from the first page to the basket, not one page at a time. That is how the split basket showed.
- How we workedTest automation for regressionWhere it showed up200 automated test cases are mapped to the 158 acceptance cases, so every later release can re-run them.
Terms as in the ISTQB CTFL 4.0.1 syllabus and glossary.
What we found, and what it costs
- FindingCriticalThe basket ran on the legacy platform while the rest of the site ran on the new oneWhat it costs youThe last step before checkout on a different platform from the journey. We marked it a go-live blocker.
- FindingRanked65 bugs filed: 1 critical, 21 high, 33 medium and 10 lowWhat it costs youOne critical finding outweighs a long tail of small ones. The ranking tells the team what to fix before launch.
- FindingFailing11 of the 200 flows failed, and none of them was a flakeWhat it costs youEach failure traces to a specific, real condition recorded in the bug tracker, not to test instability.
189 of the 200 flows passed at the time of the report. Before the status report went out we also re-checked the 40 open bugs against the staging site as it was that day: 1 had been silently fixed, 3 partly fixed, 24 were still real and 12 needed a human or screen-reader pass. A stale bug costs the reader's trust in the fresh ones.
What changed
The retailer went from none of its 158 acceptance cases automated to all of them, as 200 test cases. That gives it regression coverage for every release after this one.
Migrating a store or a platform?
We automate your acceptance cases and test the seams between the old and new. The first month is a trial.
