What is smoke testing? The build check that runs first

Smoke testing is a short, shallow set of checks run against a new build to decide one thing: is this build stable enough to be worth testing properly? It is a gate, not an assessment. The output is go or no-go, and a build that fails goes straight back to development without anyone running the other four hundred tests.

The name comes from hardware. You power up the board, and if smoke comes out you stop and do not bother with the rest of the test plan. Software borrowed the idea and the logic is the same: find the catastrophic failure in the first fifteen minutes rather than the fourth hour.

You will also see it called a build verification test or a confidence test. Same thing.

What smoke testing actually checks

Broad and shallow is the whole design. The suite touches many areas of the product and goes one step into each, rather than testing any of them thoroughly. If your smoke suite validates edge cases, it has stopped being a smoke suite.

For a typical web application, the suite answers questions like these. Does the application respond at all, and is it the build you meant to deploy? Can a user log in, and do the main roles land on the right page? Does the primary navigation load without errors? Can the central object of the product be created, opened and saved, whether that is an order, a patient record or a project? Do the critical integrations answer, meaning the database, the payment provider, the identity service?

That is ten to thirty checks. Not a hundred. The single most common way teams ruin a smoke suite is by adding to it every time a bug escapes, until it takes ninety minutes and nobody runs it on every build, which removes the only property that made it valuable.

How long it should take

Fifteen minutes is the working target for a full run, and under five is better if the product allows it. The constraint is not a rule about testing, it is a fact about human behaviour: a check that takes longer than a coffee break gets skipped under deadline pressure, and a smoke suite that is skipped on the busy releases is skipped exactly when it would have paid for itself.

If the suite has grown past that, the fix is to move cases out into the regression suite rather than to accept the longer runtime. Ask of each case: if this failed, would we stop everything and reject the build? If the honest answer is no, it does not belong here.

Running it on every build

Smoke testing is the clearest candidate for automation in the whole test process. It runs on every build, the steps never change, and the result is binary, which is the exact profile automation handles well and humans handle badly.

The first check should always be that the deployed artefact is the one you think it is. This is unglamorous and it catches a failure mode that is otherwise very hard to see, because a stale build produces test results that look completely plausible:

#!/usr/bin/env bash
set -euo pipefail

BASE="${1:?usage: smoke.sh https://staging.example.com}"
EXPECTED_SHA="${GITHUB_SHA:?expected commit not set}"

# 1. Is it up, and is it the build we meant to test?
deployed=$(curl -fsS --max-time 10 "$BASE/api/version" | jq -r .commit)
[ "$deployed" = "$EXPECTED_SHA" ] || {
  echo "FAIL: deployed $deployed, expected $EXPECTED_SHA"; exit 1; }

# 2. Do the critical dependencies answer?
curl -fsS --max-time 10 "$BASE/health/ready" | jq -e '.database == "ok" and .payments == "ok"'

echo "smoke: infrastructure OK"

Then the user-facing half, which needs a browser. Keep it to the paths that would make the build worthless if broken:

login as each role          -> lands on the expected dashboard
open the main list view     -> renders rows, no console errors
create the core object      -> saves and is retrievable after reload
open checkout / submit form -> reaches the confirmation step

Wire it to run automatically on deploy to the test environment and to fail the pipeline loudly. A smoke suite whose failures arrive as an email nobody reads is decorative. If the tests are flaky, fix or delete them, because a suite that cries wolf twice gets ignored on the third occasion, which will be the real one.

What a failure means

A failed smoke test rejects the build. That is the entire point, and it is where the discipline usually collapses, because the pressure to "just test around it" is considerable when a release date is close.

Testing around a failed smoke check is a bad trade for a reason worth stating plainly: every result you produce afterwards is about a build that is already known to be broken, so you cannot tell which failures are the original defect and which are new. You will spend the afternoon investigating symptoms of a cause you already found. Reject it, get a fixed build, start again.

The one legitimate exception is a failure in an area that is genuinely isolated from what you need to test, and even then it is a decision to record rather than a judgement to make silently.

Where it sits next to the other checks

Smoke testing is one of three short checks that get confused with each other, and the distinction is about when they run and who runs them.

The sequence matters and is often run out of order. Sanity testing assumes a stable build, so running it before smoke means a failure tells you nothing: you cannot distinguish a bad fix from a bad build. Smoke first, always.

If you are trying to decide between smoke and sanity for a particular situation, the longer comparison with worked examples is in our guide to smoke testing vs sanity testing. For the formal sign-off process at the end, see acceptance testing.

The mistakes that come up most

The dominant one is letting the suite grow until nobody runs it, which is covered above. Put a runtime budget in place and hold to it: if a new case pushes the run past fifteen minutes, an old one leaves.

Testing the wrong build is next, and it is the quietest. If the version check is not the first thing the suite does, you will eventually spend a day on defects that were fixed before you started.

Then there is the data argument. Teams hold off building a smoke suite because the test data is not realistic enough. It does not need to be. A smoke suite is not looking for performance problems, so what it needs is a known, seeded state, so that a failure means the build is broken rather than the data being unusual.

Skipping it for small changes is the one that shows up in incident reviews. The one-line change that cannot possibly break anything breaks things: a CSS specificity change can kill a login button, a config change can point the build at the wrong database. Fifteen minutes is cheap next to finding that out in the fourth hour.

And tolerating a flaky smoke suite is worse than tolerating flakiness anywhere else, because being believed is this suite's entire function.

If you want a smoke suite like this built and kept running on every build without hiring for it, our managed testing services can run it for you.

Sources

The ISTQB glossary carries the standard definitions of smoke testing, sanity testing and acceptance testing, and ISO/IEC/IEEE 29119 covers the documentation side.

Smoke testing is the first activity in the execution phase of the software testing life cycle, and it is also the check that confirms an environment is genuinely ready to test against.

BetterQA builds and maintains smoke suites as part of embedded QA work, usually starting by cutting an existing suite down to something that runs in under fifteen minutes and can be trusted.

Built by BetterQA