QA glossary
What is an LLM-powered agent?
An LLM-powered agent is software that uses a large language model to decide what to do next and then does it, by calling tools such as a browser, an API or a test runner. The ISTQB glossary defines it as "An application that integrates LLM reasoning, decision-making, and memory, using tools to perform tasks."
An example from QA practice
A tester gives an agent a user story and the address of a staging site. The agent reads the story, opens the site in a browser, finds the login form, writes a test that signs in and checks the dashboard, runs it and reports the result. A chatbot would only have described those steps. The agent took them, which is the difference, and it is also why its mistakes cost more: a wrong step is executed, not just written down.
Autonomous or semi-autonomous
- Autonomous agents work with minimal human intervention once they are given a goal.
- Semi-autonomous agents work with periodic human oversight, so a person checks the output against the goal.
- Multi-agent systems split a job between several agents with different roles that coordinate with each other.
The ISTQB CT-GenAI v1.1 syllabus (section 4.1.3) notes that agents inherit the hallucinations, reasoning errors and biases of the LLM behind them, and recommends automated checks on their results or semi-autonomous agents for critical tasks.
Why it matters when you buy testing
Many testing offers now say an agent writes and runs the tests. The question to ask is the one the syllabus raises: who checks what the agent did before a result is reported as a pass? An agent that marks its own work is grading its own exam.
How BetterQA does it
Our own tools run on AI agents. The AI Security Toolkit coordinates AI agents on every scan, and Flows, our browser test tool, repairs broken selectors by itself (self-healing). In both, the agents are semi-autonomous in the syllabus's sense: an engineer reviews every AI-generated result, including each selector repair, before it counts as a pass or a finding.