An agent decides, calls tools and acts. That makes a wrong step costly and hard to see. We test whether your agent picks the right tool, stays inside its permissions, stops when it should, and whether your MCP servers hold up when an agent or an attacker uses them.
Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value or inadequate risk controls (Gartner press release, 25 June 2025). Testing does not remove those risks. It makes them visible while changing course is still cheap.
Agents are hard to test the usual way. The same task can take a different path each run, and the final answer can look right when the steps behind it were not. So we check the steps and the result.
Did it choose the right tool, pass valid arguments and use the result correctly?
Duplicate orders, repeated emails, edits to the wrong record and loops that do not stop.
What the agent can reach, and whether a user or a document can talk it into reaching further.
We review the sequence of steps and the answer against what a sensible path would be.
Caps on steps, tokens and calls, and what happens when the cap is hit.
The same tasks rerun when the model, prompt or tools change, so differences show.
The Model Context Protocol lets agents call tools exposed by a server. The server is code like any other, and an agent is an unusual client for it. We test:
Tool schemas. Names, descriptions and parameters that are clear enough for a model to use correctly, and that reject invalid input.
Error handling. What the agent sees when a call fails, times out or returns something unexpected, and whether it recovers or loops.
Authentication and authorisation. Whether each tool checks who is calling and what they may do.
Tool poisoning and injection via tool output. Text in a tool description or a tool result that tries to give the agent new instructions. That is prompt injection through the tool channel.
BetterQA builds and runs its own MCP servers for its own QA tools, including BugBoard and Flows. We test MCP servers every day, so we know where they tend to break. We still only test for clients: we do not build your product.
Its goal, its tools, its permissions and what a mistake would cost.
A set of tasks with the outcome and a sensible path for each, agreed with you.
Each task run several times, with every step and tool call recorded.
Hostile input through the user, documents and tool output, to test the boundaries.
Findings with the recorded trajectory, then a rerun when you change the model, prompt or tools.
We build and run MCP servers for our own tools (BugBoard, Flows, JRNY, Hireo, a site scanner, the AI Security Toolkit, Maestro and others), and AI agents use them every day. Every surprise goes into a feedback journal: 221 entries across 13 tools, and 819 issues filed from it, of which 697 are closed and 122 open. These failure classes came up most.
With several per-project keys, a tool silently acted on the wrong project. We now test every tool with more than one key and check where each write lands.
A tool answered "Updated" and dropped the fields. Security scanners reported success while scanning nothing because one interpreter was dead. Now we re-read after every write, and AI assertions that default to optional are refused at creation.
A list capped at 100 was read as the total, and a health check reported 0 dead links after stopping at its cap. We check a count against the source, not against a capped list.
A server reported a current version while running month-old code, and two probes gave a false green. We check the running code, not the reported version.
A scan in parse-only mode echoed plaintext credentials from request bodies into the agent's context. We read what a tool returns as well as whether it ran.
The lesson. Most of these were not crashes. The tool answered, and the answer was wrong or empty. A test for an agent tool has to check what changed, not what the tool said.
Findings ranked by risk, each with the recorded steps that led to it.
Tasks, expected outcomes and attack cases, so your team can rerun them on each change.
Each finding logged in BugBoard or your tracker, ready to fix and retest.
Agents that face users also need AI red teaming. For AI doing the testing instead, see agentic QA testing. The overall approach is on our AI quality assurance page.
Testing a system that plans, calls tools and acts on its own. We check the steps it took and the final answer.
A chatbot talks. An agent also acts, so wrong steps have side effects: orders, emails, changed records. That needs checks on tools, permissions and loops.
Testing a Model Context Protocol server: its tool schemas, error handling, authentication, and resistance to tool poisoning and injection through tool output.
There is no single standard. We build the test set around your tasks and tools, and hand it over so you can run it in your own pipeline.
We set step and token limits on the test runs, and we test the limits themselves, since runaway loops are a failure we look for.
Tell us which tools it can call. In a complimentary call we will outline what to test first.
