Edu Sanchez

Why a code graph needs two databases

The previous post was about what Semantic Graph does and why we built it. This one starts on how it works, and what I got wrong along the way.

It started in February, and the first usable version took about two months. Today it reads almost five hundred repositories in eight languages, plus SQL, Spring config and a Confluence wiki, and turns all of it into something an LLM can ask questions about. The decision that shaped everything else is that it uses two databases, not one.

The question everybody asks

Before you touch a method in a big codebase, you ask the same question every time: what breaks if I change this?

Today you answer it by hand. You open the file, you search for the name, you follow a few call sites, and at some point you stop and hope you did not miss one. In a single repository this is annoying. Across a few hundred repositories that call each other through shared libraries, HTTP and Kafka, it is not possible. Nobody has that in their head.

An LLM is not better at this than you are. If you give it the file, it will read the file. It will not know that a shared library with no HTTP endpoints at all is called from hundreds of places in other repositories, because that fact is not in the file.

So the tool has to know the structure before the question arrives.

Two kinds of context

When I started, I thought this was a search problem. Embed the code, put it in a vector store, ask questions. That works for one kind of question and fails completely for another.

There are two kinds of context an LLM needs about code:

  • Structure. Who calls this method. What implements this interface. Which REST endpoints end up reaching this line. What sits downstream if it changes. This is a graph, and you traverse it.
  • Meaning. “The code that validates an order before it is saved”, when you have no idea what the class is called. This is vector search, and you rank by similarity.

A graph cannot help you when you do not know the name of the thing. Cypher needs a starting node, and “the code that does X” is not a node.

A vector index cannot help you once you have found it. It will happily give you the ten most similar chunks. It has no idea that one of them is called from sixty endpoints and the other nine are dead code. Similarity is not impact.

I tried to make one store do both. The graph databases with vector support were young, and the vector stores with graph features were not really graphs. The full story of that evaluation is the next post. The short version is that I stopped fighting it and kept both.

What is in each store

Memgraph holds the structure. One property graph per product. The nodes are what you would expect: files, classes, interfaces, methods, fields. The edges are where the value is: CALLS, EXTENDS, IMPLEMENTS, INJECTS, USES_TYPE. Then a second family for crossing service boundaries: EXPOSES for an HTTP endpoint, CALLS_HTTP for a client call, PRODUCES and CONSUMES for Kafka. Without those four, impact analysis stops at the edge of the repository, which is exactly where the interesting problems start.

Memgraph speaks Bolt, the same wire protocol as Neo4j, so the client is the normal Python neo4j driver. Right now the graphs hold over a million relationships in total.

Elasticsearch holds the meaning. One shared instance, with an index per product and per kind of content: code, database schema, wiki pages, incidents, config properties. Each chunk is a method, a class or a file section, with its real source text and a 384-dimension embedding.

The search is hybrid. Dense kNN over the embeddings, plus BM25 over the text with a custom analyzer that splits camelCase, snake_case and dotted names into tokens. The BM25 side boosts a match on the method name three times, and a match on the fully qualified name twice. An exact identifier beats a paragraph that happens to talk about the same thing.

This matters more than it sounds. Half the questions people ask contain an identifier, and an embedding model is bad at exact identifiers. BM25 with a code-aware tokenizer is very good at them. Together they cover each other’s blind spot.

The embedding model is BGE-small, quantised to ONNX and committed to the repo. It runs inside the indexer with fastembed. It is a small model, and it is enough for code, and it means the index is built without calling any external service. That was the first question security asked, so it is worth saying early.

The thing in the middle

Two stores are only useful if something makes them act as one. In Semantic Graph that is a class called QueryEngine, and the core of it is short enough to show.

async def retrieve(self, query, n_results=5):
    # 1. Ask Elasticsearch for candidates, by meaning and by keywords.
    hits = await self.vectors.search(query, n_results=n_results)
    if not hits:
        return []

    # 2. One trip to the graph for all of them, not one per hit.
    fqns = [h["metadata"]["fqn"] for h in hits]
    contexts = await self.graph.get_graph_contexts_batch(fqns)

    # 3. Hand back the source and the structure together.
    return [
        {
            "fqn": h["metadata"]["fqn"],
            "file_path": h["metadata"]["file_path"],
            "source": h["document"],
            "graph": contexts[h["metadata"]["fqn"]],
        }
        for h in hits
    ]

There is one more line in the real version. If Elasticsearch returns a chunk and Memgraph has no node for that name, the hit is dropped. The two stores are written by different phases of the indexer, and a chunk without a node is a symptom of a half-finished run, not something to show a user.

Vector first, then the graph, then one answer. The order is deliberate. The vector store is the only one that can turn a vague question into a concrete set of names. Once you have names, the graph is the only one that can tell you what they are connected to.

The batch in step two is the part I would defend the hardest. The first version did one graph query per hit. With ten hits it was fine. With the fan-out that happens in a real impact query, where each hit has callers and each caller has callers, it became hundreds of round trips and the whole thing crawled. Everything in the graph layer now takes a list and returns a dictionary.

So let’s ask it something

The question people ask most is “what breaks if I change this?”, so it has its own endpoint. I will use placeOrder as the example, which is not the real name but has the same shape: an interface, two implementations and a couple of overloads.

The first step is the retrieval above. The name goes into the hybrid search and comes back as a list of about ten concrete methods, because placeOrder, placeOrders and the two implementations all match. That wider net is on purpose. If you only take the top hit you get the interface and miss the code that actually runs.

Then the graph takes over. One batched query returns the direct callers of every target. Each caller is reduced to its short name, with the parameter list cut off. The blast radius matcher works on names, and keeping the signature would find one overload and miss the others.

From there it walks upwards from every target and every caller, up to five hops, until it reaches something with an HTTP annotation on it. This fan-out is bounded by a semaphore of sixteen. That number is not random. Memgraph runs four Bolt workers. The first version fired sixty blast-radius calls at once, each of them a dozen queries, and six hundred queries fought over four workers. Sixteen keeps the workers busy without the queue.

The last two steps are batched too. One reverse scan over the union of every caller found attaches REST endpoints and servlets. One lookup returns the HTTP and Kafka edges of every method touched. Both used to run once per target. Now they run once.

For a real method of that shape the answer takes under ten seconds and comes back as a few pages of markdown. Around sixty REST endpoints can reach it, and only a handful call it directly. The rest arrive through a generic save, a handleRequest or an execute, three or four hops away. grep placeOrder gives you the handful. It will never give you the sixty, because the other fifty do not contain the string.

Every response ends with a section called Steps executed, with the time each step took. When something is slow, you know which part, and so does the agent.

Why this saves tokens

The usual way to give an LLM context about code is to put files in the prompt. Whole files. Sometimes whole directories, because you do not know which file matters.

With the structure already indexed, the agent asks a precise question and gets back the twenty lines that matter: the method, its callers, the endpoints, the table it writes to. It never sees the other four thousand files in the repository. The graph works as a cache for reasoning. The expensive part, understanding how everything connects, was done once at index time, and every question after that is cheap.

In practice this is what changed how people use it. Not the search. The fact that you stop feeding whole repositories into a context window.

What it took to trust it

A knowledge graph is only useful if it is right, and “right” is hard to check when the input is a few hundred repositories you did not write.

Most of the work was iteration. I ran the parsers against our real repositories again and again. Then I went into the graph and checked by hand. Were the calls there? Did the interface resolve to the right implementation? Did the endpoint three hops up show up in the blast radius? Every time something was missing, it became a fixture and a test.

Today the suite has well over a thousand tests in four tiers:

  • unit tests on the parsers, with no infrastructure at all
  • integration tests that spin up Memgraph and Elasticsearch in testcontainers
  • end-to-end tests against a small fake repository
  • golden tests that snapshot the markdown a recipe returns, so a change in the output has to be intentional

The other thing that changed is speed. The first full index of our biggest repository took almost an hour and a half. The same repository now takes about eight minutes. A good part of that gain came from a spike that failed, which is a story for the next post.

Where it runs

Everything runs on an internal Kubernetes cluster, deployed with Helm. One Memgraph and one indexer per product, because Memgraph keeps the graph in RAM and splitting by product keeps each instance a manageable size. One Elasticsearch for everyone. A FastAPI service in front that exposes the REST API and an MCP server, so people ask from their coding agent and never see the stores.

The next post is about how the two databases were chosen. It includes the week I spent trying to replace both of them with one, and why that failed in a useful way.

All posts