Tools, providers and the supply chain

Where your evidence stops when the agent calls something you do not run.

My agent calls third-party tools and MCP servers. Where does my evidence stop?

Your evidence stops at the call. Responsibility does not. You can evidence which servers the agent may reach, what credential it presents, what it may send, and what happens to the reply. What goes on beyond that call is not yours to evidence. A claim otherwise is one somebody will check. Treat every tool response as untrusted input. A server that can put text in front of a model can try to instruct it. So the boundary worth documenting is where that text becomes an action. What is the agent permitted to do because a tool said so?

How do I stop my agent being tricked into acting by content it reads?

Constrain what it can do. You cannot reliably constrain what it believes. Any text your agent reads is input from whoever wrote it. A web page. A ticket. A tool response. Treat all of it as untrusted, because a model has no reliable way to separate your instructions from text shaped like instructions. Filtering helps and does not hold. Your permission boundary holds. If the agent cannot delete the record, no sentence on a web page makes it delete the record. Design so your worst case is a wasted call.

My model provider changed their terms. What do I actually need to check?

Retention, training use, subprocessing and location. Those four move, and each is a statement you may have made to a customer. Retention decides how long your data sits with them, and whether your deletion request reaches it. Training use decides whether your customer's content improves somebody else's model. Subprocessing decides who else is in the chain. Location decides which regime applies. Check what you told your customers against what your provider now says. That gap is your exposure. Set a reminder to re-read, since nobody tells you when a page changes.

How should I evaluate an agent framework for security?

Look at what it does with credentials, and what it lets a tool response do. Frameworks differ enormously here and the marketing does not cover it. Three questions get you a long way. Where do your credentials live when a tool is called, and can the model see them? Can a tool response trigger another tool call with nothing checking? And is the permission decision the framework's, or yours? A framework owning your authorisation logic is a dependency for a control you have to evidence. Prefer one that stays out of that decision.

How do I know my tool definitions match what the tools actually do?

Check the endpoint, not the description. A tool definition is a promise written by whoever added the tool, and your model plans against that promise. When the endpoint gains a parameter or widens what it returns, the description keeps saying the old thing. Now your agent has a capability nobody documented, and your evidence describes a system that no longer exists. Generate definitions from the interface where you can. Where you cannot, review them when the endpoint changes and give that review an owner. This drift is silent.