One door for every tool and every agent
Not long ago an agent at work did one thing over the network. It sent a prompt to a model and read the answer. Today the same agent calls internal tools, asks Semantic Graph where a method is called, and hands work to other agents.
Every one of those calls carries company data. Until recently, our gateway only looked after the first one.
This post is about closing that gap. The gateway is now the only way in to our internal MCP servers and to our own agents. The core of it landed in two days at the end of September.
That was only possible because almost everything it needed was already there.
Three kinds of traffic
An agent talks to three kinds of things, and each one has its own protocol.
| Talks to | Protocol | Example |
|---|---|---|
| A model | OpenAI or Anthropic APIs | “Write a test for this class” |
| A tool | MCP | “Search the docs for the release process” |
| Another agent | A2A | “Fix this ticket and open a merge request” |
The model traffic already had everything: tenant keys, policies, PII masking, audit, metrics. Tool calls and agent calls carry the same data, so they need the same protection.
So the idea was simple. Treat a tool call and a call to another agent like a prompt. Same door, same key, same rules.
One URL for every tool
The first thing a developer notices is the setup. You add the gateway once, and you see every tool your team is allowed to use.
claude mcp add --transport http gateway https://gateway.example/mcp/ \
--header "Authorization: Bearer $TENANT_KEY"Behind that one URL, the gateway opens a session on every server the tenant can reach. It joins their tool lists into one, with each server’s name in front of its tools. The name alone tells the gateway where to send each call.
The session the client gets back carries everything needed to route the next call. So any replica can take it, and the replicas share no state for it.
The agent never holds the key
This is the part I like most. The agent has one credential, its tenant key. It never sees the credential of the server it is calling.
The gateway checks the tenant and sends the call on with its own credential. It also tells the server which tenant is calling. A client cannot set that value itself, so it cannot claim to be someone else.
Everything the server does not need stays at the gateway. The server gets a clean request from one caller it trusts.
Some servers want more than a value they have to trust. For those, the gateway signs a short-lived token that says who is calling and who it is for. The server checks it against a public key, so the two share no secret. A copied token is useless a minute later.
Nothing is allowed until someone says so
A new tenant reaches no tool at all. Access is denied by default. Each tenant has a policy that names the servers, the methods and the tools it may use.
A small detail makes a big difference here. When the agent asks for the list of tools, the gateway cuts it down to the ones the policy allows. The model never learns that the other tools exist, so it cannot try them and does not waste context on them.
The policy is managed like everything else on a tenant, and every change is audited.
Signing in with a browser
Tenant keys are good for headless agents. People should not have to paste keys around, though.
So the gateway also speaks OAuth 2.1, as the MCP spec describes. A client like Claude Code needs only the URL. Its first call is refused, and the answer tells it where to sign in.
The client opens a browser, and the person signs in with the company identity provider, like they do for everything else. The gateway stores no people. Access is granted and removed where it already lives.
The rest is the boring, careful kind of work that a sign-in needs. Short-lived codes that work once, tokens that rotate, and nothing stored in clear. A sign-in can start on one replica and finish on another.
Guardrails in both directions
This is what made the whole thing cheap to build. The gateway already had a guardrail pipeline for prompts, with PII masking, banned terms and prompt injection detection. MCP and A2A messages are JSON-RPC, so the work was mostly teaching the pipeline where the text is.
On the way in, it scans what the agent sends. On the way back, it scans every message the server sends, and every event on a stream.
Values it masked on the way in are restored on the way out. So the agent sees the real data, and the server sees only what it needs.
When a tool goes down
A single door has an obvious risk. If one thing behind it breaks, it should not take the others with it.
So every server and every agent behind the gateway gets its own timeout and its own circuit breaker. When one of them stops answering, its breaker opens and its calls fail fast. The calls to everything else carry on as if nothing happened.
Retries needed more thought than I expected. With a model, sending the same prompt twice costs some money and nothing else.
A tool is different. A tool call can create a ticket, push a branch or delete a file, and running it twice is a bug.
So the gateway only retries a call that never reached the server, such as a connection that failed to open. Once the server has the request, the gateway never sends it again, even if the answer gets lost. The agent gets the error and decides what to do, because only the agent knows whether the call is safe to repeat.
A server can also run as several instances. If one cannot be reached, the call moves to the next.
Agents talking to agents
MCP is for tools. A2A is the protocol for one agent asking another to do a job, with tasks, conversations and streaming updates. The gateway serves it the same way: one URL per agent, one tenant key, the gateway’s own credential downstream.
A2A brought one problem that MCP did not. Every tenant reaches an agent through the gateway, so the agent sees a single caller. It cannot tell one tenant’s tasks from another’s.
So the gateway keeps track of who owns what. When an agent starts a task, the gateway records the tenant before the answer reaches the client. Every later call that names a task checks the owner first.
Another tenant’s task gets the same “task not found” the agent itself would send. The caller cannot even tell whether the task exists. That is the point.
The agent’s card, the document that says what it can do, is rewritten for each caller. It points at the gateway and shows only the skills that tenant may use.
What it costs
None of this is free, so I measured it. The benchmark ran the full stack, with the database, the cache, auth, limits, budgets and audit. A fifth of the traffic was MCP tool calls of 2 to 100 KB, with personal data to mask.
| What the gateway adds | p50 | p95 | p99 |
|---|---|---|---|
MCP tools/call only |
3.5 ms | 5.8 ms | 23.8 ms |
| All traffic, 32 sessions | 8.4 ms | 19.6 ms | 33.0 ms |
The whole stack served about 500 requests per second with zero errors, and peaked at 145 MiB of memory. It all ran on one laptop, so read the tails with some care. A tool call that takes seconds on the server does not notice a few milliseconds at the gateway.
Where it is now
Both sides are running. Developers add one URL and get the tools they are allowed to use. Agents call other agents through the same door, with the same rules.
Adding a new server or agent is a few lines of configuration and a line in each tenant’s policy. No code, no keys to hand out, no new audit to build.
What I took from it
The protocols were the easy part. MCP and A2A are both JSON-RPC, and a proxy for JSON-RPC is not hard to write.
What made this work in two days was everything around the proxy: tenants, keys, policies, masking, audit, metrics and circuit breakers. All of it already existed for model traffic. The new code mostly said “do the same here”.
That is the real argument for putting a gateway in front of agents early. Every new thing an agent learns to talk to is one more door. If you already have a good one, you just make it a bit wider.