Your chatbot will meet people who try to break it. We play those people first. Our engineers attack your chatbot, LLM feature or agent by hand, the way a hostile or curious user would, and report what gave way, with the exact conversation that did it.
Functional tests ask whether the bot answers the questions you expect. Red teaming asks what it does with the questions you did not expect. ISTQB defines red teaming for AI as deliberately attacking a system to provoke harmful or unintended output (CT-AI 4.2.2), and it is a human skill: the attacker adapts to each answer.
Two public cases show the cost. In February 2024 the British Columbia Civil Resolution Tribunal held Air Canada responsible for wrong refund information its website chatbot gave a customer. In December 2023 a Chevrolet dealer's chatbot was talked into agreeing to sell a car for one dollar, and the conversation spread widely online. Neither case needed advanced hacking, only a conversation.
Role play, hypotheticals and slow persuasion over many turns that wear down a rule the bot holds on turn one.
Instructions typed by the user, and instructions hidden in a web page, document or email the bot reads.
Can a user pull out the hidden prompt, other customers' data or internal documents?
Refunds, discounts, promises or advice the company never authorised, made in your name.
For agents: calls to tools or systems the user should not reach, or actions beyond the task.
Not sure where your risk is?
Tell us what the bot can say and do. We will rank the areas above for you.
We organise findings against the OWASP Top 10 for LLM Applications, so your security team can read the report in a framework it already knows. OWASP also publishes a separate AI Testing Guide (version 1, November 2025), a useful reference for what to cover.
| Our attack area | OWASP LLM category it maps to |
|---|---|
| Jailbreaks, direct and indirect injection | Prompt injection |
| Data and system prompt leakage | Sensitive information disclosure; system prompt leakage |
| Tool and permission abuse | Excessive agency |
| Wrong or invented answers that cause harm | Misinformation |
| Runaway use of tokens or calls | Unbounded consumption |
This organises our findings. It is not a certification and we do not claim one. Category names follow the OWASP list as we read it; check them against the current edition when you cite the report.
Which bot, which channels, which data and tools it can reach, and what is off limits. Signed NDA first.
Who would attack it and why: a curious customer, a competitor, someone chasing a refund.
Our engineers run manual conversations, then repeat the successful ones to see how reliably they work.
Each finding with the conversation, the risk, the matching OWASP category and a suggested fix.
After your team changes the prompt, filters or permissions, we run the attacks again.
LLM red teaming is a capability we offer. We have specialist agents for it: one for prompt injection, and one for jailbreak and data exfiltration testing. We do not cite a past engagement against a client's LLM feature, because we have none to show. What we can show is how the AI Security Toolkit behaved on real applications and on itself. Client work is anonymised to the type of business and described at the level of the class of problem.
An access-control weakness on sensitive records, reported to the client with a fix. Cross-tenant access, injection and admin escalation were each tried and ruled out, with a negative control to prove the checks could fail.
Two authorisation gaps, one on a sensitive account action, chained into an account takeover path. The permission model held elsewhere in the same app, so the problem was route coverage, not design. Reported to the client with a fix.
The app ignored the usual interception setup, so our first captures were empty. We set up custom test tooling before we could read its traffic. The finding that mattered was a negative one: a cross-account access test between two throwaway accounts failed on every attempt.
Custom checks that never ran were recorded as clean. A secrets scanner reported success without reading its output: on a fixture with 6 planted secrets it found 0 before the fix and 6 after. Did-not-run is now its own result.
The lesson. An attack tool is a system to test too, and a negative result needs proof that the check could have found something. On a public vulnerable-by-design benchmark the toolkit produced 27 findings and 6 attack chains where the other tools produced none. We bring the same habits to a chatbot or agent.
Findings ranked by risk, each with the full conversation so your team can reproduce it.
Every prompt we used, in a form you can rerun after each model or prompt change.
Each finding logged in BugBoard or your own tracker, so it can be fixed and retested.
Two layers, two methods. The AI Security Toolkit scans the application around your AI: it runs 18 AI agents on every scan, plus up to 3 more when the scan needs them (mobile apps, SOC2 mapping, role-based access testing). Human red teaming covers the language model layer, where the weakness is what the bot can be persuaded to say and do.
Most clients need both. For the broader picture of testing AI, see AI quality assurance. For a bot you already run, start with chatbot testing.
Deliberately attacking an AI system to provoke harmful or unintended output, then reporting what worked so it can be fixed. ISTQB covers it in the CT-AI syllabus (4.2.2).
Penetration testing of an LLM application looks at the application and its infrastructure. Red teaming focuses on the model's behaviour under hostile conversation. Many engagements use both, which is why we pair the AI Security Toolkit with manual red teaming.
Checking whether text from a user, or text inside a document or web page the bot reads, can override the instructions you gave it.
No. Testing shows the presence of problems, not their absence. We report what we found, and what we tried and did not break.
Yes. We check which tools the agent can call, what a user can make it call, and whether it acts beyond the task. See also AI agent and MCP testing.
Tell us what it can say and do. In a complimentary call we will scope the first round of attacks.
