How to test an Unreal Engine or cloud-streamed 3D application
Short answer
Standard web automation cannot see inside a 3D scene, and it cannot see inside a video stream at all. Test these applications in three layers: the engine itself with Unreal's own Automation System and Gauntlet, the stream and the web portal around it with browser tools, and what the user sees by driving input at screen positions or through console commands and comparing screenshots against approved ones. People still check what tools cannot judge, such as whether a scene is right.
This guide is for teams building training, simulation and visualisation products in game engines, often streamed to a browser so users need no powerful hardware. These products sit between game testing and web testing, and the usual approach of each one misses part of the risk.
Why normal web testing does not work here
Web test automation finds buttons and fields in the page's structure and checks their text. A 3D scene drawn by an engine has no such structure: to a browser tool, the whole scene is one canvas. When the engine runs in the cloud and is streamed to the browser, as with Unreal's Pixel Streaming, the browser receives video frames over WebRTC and sends back mouse and keyboard events. There is nothing inside the frame for a tool to inspect.
So a product like this has three different things to test, each needing its own approach:
- The engine application: logic, physics, scenarios, scoring.
- The delivery: the web portal, login, the stream, its quality and its recovery from bad connections.
- The experience: whether a user sees the right thing and can complete the task.
Test the engine with its own tools
Unreal Engine ships with an Automation System. Developers and testers can write automation tests in code, and functional tests placed in a level that run a scenario and report pass or fail: load the scene, perform an action, check that an object ended up where it should or that a score changed. These run without anyone watching the screen, which makes them the right home for logic checks such as quiz scoring, scenario branching and physics outcomes.
Gauntlet, also part of Unreal, runs these sessions on target machines and collects the results, so the same tests can run on each build. The Session Frontend includes screenshot comparison: a test captures an image at a known point and compares it with an approved one, flagging visual changes for a person to review.
These need access to the project and work best when testers and developers agree early which scenarios get functional tests. They are the cheapest automated checks you will have, so it pays to build them into the project rather than add them at the end.
Test the portal and the stream like a web product
Everything around the stream is ordinary web software and should be tested that way, with browser automation and security and accessibility checks: sign-up and login (often provided by a third party, which needs its own test accounts), course or session selection, progress and results, and data deletion requests under GDPR.
The stream itself needs checks a game rarely gets:
- Start time: how long from clicking "start" to a usable scene, including when cloud capacity is cold.
- Input delay: the time between a click and the scene responding, on a fast connection and a weak one.
- Quality changes: what happens when bandwidth drops: does resolution degrade gracefully or does the session freeze?
- Reconnection: after a short drop, does the user return to the same point, or lose their progress?
- Browsers and devices: the stream runs in the browser, so test the browsers and screen sizes your users actually have, including managed hospital or corporate laptops with restricted settings.
- Capacity: how many simultaneous sessions the cloud set-up supports before start times grow.
Test what the user sees
For end-to-end automation through the stream, two techniques work, usually together:
- Input at screen positions. The test sends clicks and key presses at known coordinates of the streamed frame, the way a user would, then captures a screenshot and compares it with an approved image within a tolerance. It is simple and works through any stream, but breaks when the layout or camera changes, so keep these tests few and focused on critical journeys.
- Console commands. Many engine applications accept commands that set state directly: load a scenario, jump to a step, set a value. A test uses them to reach the point of interest fast and reliably, then checks the result with a screenshot or a value the application reports back. This is far more stable than clicking through every step.
Then there is what no tool can judge. A simulation can run perfectly and still be wrong: an anatomical model positioned slightly off, a quiz that marks a defensible answer as incorrect, a scenario that a practitioner would find unrealistic. Content in medical, industrial or safety training needs review by a domain expert, and functional testing by people who understand what the user is trying to learn. Automation tells you the build behaves the same as yesterday; people tell you whether it is right.
Plan QA so it is not squeezed at the end
Teams that come from game development often run discovery, then production, then bring QA in once a skeleton build exists, and the release date compresses that last phase. Two habits help without changing your whole process:
- Stagger features. Finish one feature early so testers can work on it while the others are completed, rather than receiving everything at once.
- Involve QA when scenarios are designed. The functional tests in layer 1 are cheapest to plan alongside the scenario, and a tester reading the design catches ambiguities before they are built.
How BetterQA tests 3D and streamed products
Some of our engineers came to us from a game studio, where they tested commercial game titles, and now work on simulation and training products alongside web and mobile testers. We calibrate the approach to each product: Unreal's own automation where we have project access, coordinate and screenshot checks through the stream for the critical journeys, browser automation, security and accessibility testing for the portal, and manual exploratory testing for what only a person can judge. Everything we write stays in your repository and your tracker.
Frequently asked questions
Building a 3D, simulation or streamed product?
Tell us how your product is delivered and what your next release covers. We will propose a test approach for the engine, the stream and the portal.
Book a call Our testing servicesNeed help with software testing?
BetterQA provides independent QA services across manual testing, automation, security audits, and performance testing. ISO 27001, 9001, 14001 and 13485 certified.
