Our AI research put news on the wrong company, and our own guard approved it
The guard checked that the link was real. It was. It belonged to someone else.
JRNY researches companies for our sales team. An unrelated firm with the same short name had its careers page fed in as the main evidence about a French software company. The check built to stop invented links passed it, because the link existed.
What went wrong
A French software firm had almost nothing to fetch on its own website, so the spare research slots filled with other results. The careers page of an unrelated Thai interior-design firm with a similar name was fetched and handed to the model as primary evidence. Five interior-design jobs were written against a French software company.
Why our own guard approved it
The fetched page was labelled "live-fetched content, most reliable". Our link-provenance check passes any source link that appears among the fetched pages. So admitting a foreign page manufactured authoritative evidence, and the guard that exists to catch fabricated links certified it, because the link was real. It just belonged to someone else.
The earlier fix was prompt text
This happened 17 days after we had told the model, in its instructions, to anchor on the company's own domain. The model did not obey, and nothing verified whether it had. Instructions are requests. A page that should not be fetched has to be refused by code.
The fix: fewer pages, on purpose
Only two kinds of host can be tied back to the target company, so only those are fetched: the company's own domain and its subdomains, and known job-board platforms where the page names its own employer. Everything else is dropped and counted in a log line.
Fetching nothing is acceptable. No evidence beats confident evidence about the wrong company, and it keeps "we found nothing" distinguishable from a bad guess.
The trade-off is real: news signals lose full-page fetch on third-party press sites and keep only search snippets, which already go through an earlier filter.
How we proved it
The new test failed against the old code on exactly the two cases that describe the incident, and the correct-behaviour cases already passed, so the suite pins the defect and not the new code. We then reverted each of the three mechanisms alone: company domain, job-board allowlist and the refusal of unattributable hosts. Each one turned named tests red against a clean control run. In the same investigation we removed the wrong-company signals already in the system, each with a reason naming the real organisation.
What to take to your own team
- A check that the source exists is not a check that it is about the right entity.
- Do not rely on instructions to an AI for a hard rule. Enforce it in code.
- Say "no evidence" when there is none. It is a better answer than a plausible wrong one.
Using AI research in your sales or compliance work?
We test whether the evidence is about the right company, not only that it exists. The first month is a trial.
