Our AI sales agent picked the wrong person 11 times in 47
It looked up the right company name and unlocked someone at a different company, often with a verified email.
AI SDR is our sales agent. It finds the right person at a target company and enriches the contact. On a run of 47 small software firms it picked someone at a different company 11 times. A second step, guessing email addresses, had never worked at all.
The agent asked by name and did not check
AI SDR asks a contact database for people by the company's name. The database matches loosely. The agent then ranked the people on that page by job title alone and unlocked the top one, without checking where that person works. It had been given the company's website and never used it.
On the run of 47, 11 picks were at a different company, mostly with a verified email that a caller would have mailed. A firm was matched with an accounting firm of a similar name. Another was matched with a firm one letter away. In one case the real chief executive was on the same page, ranked below the look-alike firm's founder.
The fix checks equality, not similarity
A person is now eligible only if the employer the database states equals the requested name, ignoring case, punctuation, accents and legal forms such as Ltd, SRL or GmbH. We chose equality on purpose. Similarity is how "Neo Vision" and a similar name end up in the same bucket.
- Old wrong answers are searched again. The caches serve a saved pick only if it passes the same rule.
- No match is not a miss. A title that matches only at other employers is "inconclusive", and an inconclusive answer is never cached as "no contacts".
The price, measured on the same 47
The rule refused all 11 wrong picks and lost 3 of the 26 right ones, where the database added a word to the company name. Those become a miss for a human to look up. We would rather lose a lead than email a stranger.
Three adversarial review rounds each found one Important defect, all fixed, and every fix was run against the code from before it so we saw the new tests fail first.
The second lesson: a step that never worked
When the database finds a person but no email, AI SDR guesses addresses and checks each with the company's mail server. That step needs the company's domain, and it had never worked: in every completed enrichment on record, no guessed email was ever produced.
The domain lookup used a web search, which answered with an object, while the code accepted only a bare list and silently discarded every answer. The tests handed it a bare list, which is why they never noticed.
Why we did not just fix the parsing
Review showed that looking up a domain by company name cannot be made safe. A firm with a similar name resolved to a different company's domain. A wrong domain whose mail server answers honestly turns a stranger's real address into a "verified" guess that a sales system may then email.
So the search was removed. Guessing now happens only at a domain the caller supplies, and a guessed email is never stored in the cache where it could be mistaken for a confirmed one.
What to take to your own team
- Check the thing you were given. If an input never gets used, ask why it was passed.
- Prefer a miss to a wrong match. Measure the trade-off on real data and say the number.
- Feed tests the real shape. A hand-built fixture hid a bug that real responses would have shown on the first run.
Using an AI agent that touches real people?
We test agents on real inputs and count what they get wrong before you do. The first month is a trial.
