One recipe instead of sixteen calls

Semantic Graph has a REST API, and for the first few months people used it the way you use any REST API: one endpoint per question. Find the method. Get its callers. Get the callers of those. Find the endpoints. Look up the table. Sixteen calls to answer “what breaks if I change this”, and the caller was usually not a person. It was a coding agent.

Watching an agent do that taught me more about API design than any human user ever did. This post is about what changed.

What an agent does badly

An agent chaining calls has three problems a script does not.

It guesses parameters. If an endpoint takes fqn and the previous one took name, the agent will send name to both, get a 422, and either retry with a different guess or give up. Every guess costs a round trip and a chunk of context.

It stops early. Give it a long response and it will read until it thinks it has enough, then move on. The remaining callers on page three never make it into the answer, and the answer looks complete.

And it pays for every hop. Each call is a request, a response and all of that text sitting in the context window for the rest of the conversation. Sixteen calls to answer one question is sixteen responses the agent carries around afterwards.

None of that is the agent being stupid. It is a caller with a small working memory, a strong desire to finish, and no way to know what it does not know. The API has to be shaped for that caller.

Markdown, pages, and a catalog

Three decisions came first, before recipes.

Responses are markdown by default. JSON is there if you send an Accept header for it, and services use it. But the default is a document the model reads well: headings, tables, code fences with the real source. Formatting for the reader turned out to matter more than formatting for the parser.

Long responses are paginated, and pages are cut on headings. The default page is twenty thousand characters. When a ## heading falls in the last ten percent of a page, the cut moves to the heading, so a table is not sliced in half. Every page ends with Page 2 of 6, and the instructions the model gets say, in capitals, fetch every page before you write a conclusion. The docstring in the pagination module is blunt about why: the model tends to stop reading once it thinks it has enough, and the page footer keeps it honest.

The API describes itself. There is a discover endpoint that renders the OpenAPI schema as a compact catalog: method, path, one-line summary, required parameters, optional parameters with defaults, and the type of each. The rule given to the model is discover first, no exceptions, and copy the parameter names verbatim. A 422 means you should have discovered first. It sounds strict. It removed almost all of the guessing.

What a recipe is

A recipe is a single endpoint that runs a whole question server-side and returns one document. The impact one is the most used, and the first post walks through what it does internally. Resolve the seed. Batch the callers. Fan out the blast radius under a semaphore. One reverse scan for endpoints, one lookup for HTTP and Kafka boundaries, and consolidate.

Structurally it is nothing special. A FastAPI route, a Pydantic request model, and a function that calls the same store methods the primitive endpoints call. The difference is who does the chaining. When the chaining is in Python, it can be batched, throttled and tested. When it is in the agent, it is none of those things.

There are a handful of them. impact for “what breaks”. table-analysis for a table: columns, keys, indexes, triggers, views and the code that touches it, which crosses the SQL graph and the code graph in one call. status-snapshot for “what is indexed”, across every product in parallel. jira-history for the incidents that mention a service, with aggregations by category and priority. Adding one is a code change and a pull request, not configuration, and that is fine. A recipe encodes judgement about which calls belong together, and judgement should be reviewed.

Every recipe shows its work

The part of a recipe I would not give up is the trace. Each step reports what it did and how long it took. The report is the first section of the response:

## Steps executed
- retrieve → 10 targets (2.98s)
- callers (batched, 10 targets) (0.01s)
- blast-radius×21 (targets + public callers) (2.85s)
- reverse-scan (REST+servlets, 2537 callers) (0.52s)
- boundaries (60 methods) (2.27s)

The implementation is a small queue that the recipe writes into while it runs:

class TraceStream:
    async def step(self, msg, elapsed=None):
        if elapsed is None:
            elapsed = time.perf_counter() - self._t_start
        await self._queue.put(f"- {msg} ({elapsed:.2f}s)\n")

This started as a debugging aid for me. It stayed because the agent reads it too. When a step says WARN: 3/21 blast-radius calls failed, the model knows the answer is partial and says so. When retrieve returns zero targets, the recipe says so in plain text and the model falls back to the primitive tools instead of inventing an answer. The trace is how the API tells the caller what it does not know.

There is also a recipe_version at the top. Golden tests snapshot the whole markdown a recipe produces for a fixed fixture repository, so any change in output has to be intentional. The version tells a consumer when the shape changed.

Six tools in front

For the first months people connected through skill files: markdown playbooks that told the agent which curl to run for which question. It worked for the team. It did not scale. Every new user had to edit the file to point at the server and add credentials, and credentials in a markdown file that people copy around is not a distribution mechanism.

So the API grew an MCP server, mounted in-process at /mcp. It exposes six tools, one per domain: discover, code, db, config, wiki and execute. Each domain tool is a thin proxy. code(endpoint="callers", params={...}) becomes a request to /api/v1/code/callers through an in-process HTTP client against the same FastAPI app. No query logic is duplicated. The MCP layer adds transport and auth, nothing else.

Six tools is the front door, not the size of the surface. Behind them are around forty endpoints and the recipes, and the model finds them through discover, the same way a REST client would.

What moved into the MCP server was the judgement. The skill files used to say which endpoint answers which question, in what order, and what not to do. That text is now the tool docstrings and the server instructions the client receives on connect. Discover first. Pass a scope only if the user named one. Fetch every page. If a domain is not indexed, say so, never fabricate. The model gets the playbook without installing anything.

Structure for everyone, content for your area

One design point in the MCP layer took longer than the rest. Structure is shared across products: you can ask what calls a method in any repository, because a library drift in one product breaks another. Content is not: the source bodies, the wiki pages, the SQL, the config values stay inside the area you belong to.

The gateway in front of the server stamps the caller’s area into a reserved parameter on every tool call, overwriting whatever the model put there. The MCP layer lifts it out of the parameters before they reach the API and re-presents it as a header. That way the entitlement never travels in the same place as scope, which is a search filter the model legitimately controls, and there is no way to confuse one for the other. A 403 from a content endpoint is a permission boundary, and the instructions tell the model to say that plainly and answer with structure instead.

What it costs now

The old way to give an LLM context about code was to put files in the prompt. Whole files. Often whole directories, because you did not know which file mattered.

The agent now asks one question and gets back a few pages of markdown with the twenty lines that matter and their structure. It never sees the other four thousand files. Another internal tool already runs its pull-request review gates on top of these recipes, calling only the ones each gate needs. Facts from the graph, decisions from the reviewer.

I do not have a clean before-and-after number for tokens, because nobody was measuring before. What I can say is that the sixteen-call conversation, with its guesses and retries and half-read pages, does not happen any more. One call in, one document out, and the document tells you how sure it is.

The next post is about the day the indexer ran out of memory and what the fix taught me about the parts I had not questioned.