We built and run our own MCP servers. Here is what we learned the hard way.

Why we write every surprise down

An agent cannot tell a tool that is right from a tool that sounds right. If a tool says "updated", the agent moves on. So the defects that hurt are rarely crashes. They are answers that look fine.

Each time one of our tools surprised an engineer or an agent, it went into a journal, and from there into an issue in the tool's own repository. More than 800 issues so far, and more than 700 of them closed.

The ten ways our tools misled an agent

Most frequent first. Every one of these happened to us, in our own tools.

1. Wrong project. With several per-project keys, a tool acted on the wrong project without a word. A suite landed in the default project; reads crossed project boundaries while writes did not.
2. Silent success. A tool answered "Updated" and dropped the fields; reading the record back showed nothing had changed. Security scanners reported success while scanning nothing, because one interpreter was dead.
3. Caps read as totals. A list capped at 100 was read as the full count. A health check reported 0 dead links when it had simply stopped at its limit. One result read "18/14 passed".
4. Schema says one thing, behaviour another. A field required at runtime was marked optional. A tool advertised a status the database rejects.
5. Version drift. A server reported its current version while running month-old code. A fix landed and a restart did not pick it up.
6. Errors nobody can act on. A bare "Failed", raw database codes, and a hint that pointed the wrong way.
7. AI text stored as fact. Email summaries that inverted the meaning of the email were saved to the CRM. Enrichment attached contacts from a different company with the same name.
8. Sensitive data in tool output. One scan mode echoed credentials from captured request bodies straight into the agent's context.
9. Output too large. A single call overflowed the agent's context window.
10. False liveness. Status said "connected" while nothing was listening, or "not connected" for a healthy extension.

What we did not find

We looked for prompt injection arriving through tool output, the attack most articles about agents lead with. Across our journals we found none. That does not mean it cannot happen. It means the failures that actually cost us time were plainer: tools that were confidently wrong.

What we changed

• A write is checked by reading it back. "Updated" with nothing changed is now an error, not a success.
• Every count says whether it is complete. A capped list says it is capped, and a zero says whether anything was actually checked.
• Project scope is explicit. A tool that holds keys for several projects must be told which one, and it says which one it used.
• Unsafe settings are refused at creation. For example, AI checks in our mobile engine defaulted to "optional", so a failed check still passed. Creating one without an explicit choice is now refused.
• Values are redacted by default. Tools return the shape of sensitive data, never the data.

If you are connecting agents to your own tools

Five questions we now ask of every MCP server, ours or a client's:

• What does the tool return when it did nothing? Is that different from success?
• Is every list either complete or labelled as cut?
• Which project, tenant or account did the call act on, and does the answer say so?
• Does any secret or personal data ever appear in the output?
• Does the reported version match the code that is actually running?

We can connect our own tools to your agents during an engagement, and we test the MCP servers and agents you build.