What we built at the hackathon: agents as a service in two and a half days

Last week was our internal hackathon. It started on Monday after lunch and ended on Wednesday, so two and a half days. Fifteen teams, four judges, and we were one of the winners.

This post is different from the others on this blog. It is not about one component. It is about how a few pieces fit together into one service, and what we learned making them work.

The idea

Everybody at work already runs coding agents. They run them on their own laptops, one at a time, while they watch. Close the lid and the work stops.

We wanted the opposite. You define an agent once: what it takes as input, and the steps it goes through. Then anyone can run it, from a page or from another system, and it works in a remote sandbox while nobody watches. If it needs a decision, it asks a person and waits. If it should run every morning, you give it a cron expression.

So, agents as a service. The pitch was one sentence: you describe the job, and the platform does the rest.

The pieces

The service has five parts, and every request goes through all of them.

Piece	What it does
LLM gateway	Every model call goes through the gateway: keys, masking, costs, logs
Sandbox	Boots a microVM, runs one agent, destroys the VM
Toolkit	Skills, subagents and company standards the agent loads
Scheduler	Defines agents, runs their steps, tracks what they did
UI	Create agents, run them, watch them, answer them

The sandbox

The sandbox is a close relative of the ones on a Mac and on a server. Those are for a person who works inside the sandbox with the agent.

Here nobody attaches a terminal. A caller sends instructions and a task over HTTP, gets back a stream of events, and the VM is gone at the end. One run, one microVM, nothing kept.

It uses Firecracker, and not Kata like the server version, for a boring reason. The shared host where this runs has KVM, but we have no root on it. We cannot change the Docker daemon config or restart it. So Firecracker runs inside a normal container with access to /dev/kvm, and that container is the outer wall.

Egress is default-deny, with a short allowlist: the gateway, Jira, our Git servers and the package mirrors. The VM has no DNS at all. The service resolves the allowed hosts and sends in a hosts file.

Inside the VM there is opencode, the toolkit with its skills and MCP servers, Java, Node, Python and a Docker daemon for integration tests. Every model call goes through the gateway, like any other agent at the company.

One small thing broke on the first real run. The gateway masks hostnames in the traffic to the model, so the agent never saw the real address of our Git server. It invented one. The fix was to give the agent the addresses as environment variables and tell it never to type a host.

Agents in stages

The new piece in the middle is the scheduler, a small Python service with Postgres. An agent is a JSON document with a name, some arguments and a list of stages.

{
  "name": "Jira ticket implementer",
  "arguments": [{ "name": "jira_ticket", "type": "string", "description": "The ticket key" }],
  "stages": [
    { "index": 0, "name": "developer", "prompt": "...",
      "inputs": [{ "name": "jira_ticket", "type": "argument" }],
      "artefacts": [{ "name": "merge_request", "description": "The link to the MR" }] },
    { "index": 1, "name": "reviewer", "prompt": "...",
      "inputs": [{ "name": "merge_request", "type": "artefact" }] }
  ]
}

Each stage becomes one sandbox run. Stages run in order, and each one can read what earlier stages left behind. Separate executions run at the same time.

Every five seconds the scheduler looks for a stage that is ready and sends it to the sandbox. It follows the event stream, writes the logs, and judges the stage when the stream closes.

The agent does not decide when it is done

This was the first design decision, and probably the best one. An agent that says “I finished” is not evidence of anything. Agents are very happy to tell you they finished.

So each stage declares the artefacts it must produce, each with a description. When the run ends, the scheduler checks they exist. If they do, the stage is done. If one is missing, the stage failed and the ones after it never start.

The agents write those artefacts themselves, through a Postgres tool inside the sandbox. The scheduler never writes them and never reads what is inside. It only checks they are there.

An artefact is not only a merge request link. It can be a report, a file or a whole PowerPoint deck. Whatever the stage is meant to produce, the next stage or the person who ran the agent can open it.

Asking a person, with no memory

The hardest feature was the simplest to describe. An agent should be able to stop, ask a person something, and continue when they answer.

The problem is that a sandbox run does not wait. It runs once and the VM is destroyed. The agent that continues is a brand new one that remembers nothing.

So the agent writes down a report and a question before its run ends. The stage moves to waiting, and the UI shows the question. When someone answers, the scheduler starts a fresh run. That run’s task opens with the whole history of the stage: what each earlier run did, what it asked, and what the person said.

We also tell every agent, before anything else, how this works. It runs once. Nothing it says reaches a person. The next run is a stranger whose only memory is what this one writes down. Without that, a prompt that reads like a chat makes the agent behave as if someone were listening.

Rock, paper, scissors

To test this, we built the least serious agent of the week. It plays rock, paper, scissors with you.

It is a good test because every move needs a person. The agent asks for your move, its run ends, and the VM is gone. You answer from the UI. A new agent starts, reads the history, plays its move and asks again.

If the history is wrong, the game makes it obvious straight away. The agent forgets the score, or plays a round that already happened.

Four agents, in production

We ran four agents end to end, on the real stack, with the UI talking to the real backend.

Dependencies. It reads the product health data for a service, finds outdated libraries and ones with known exploits, upgrades them and opens a merge request with the changes.
Bugfix. It takes a Jira ticket that describes a bug, finds the code, fixes it and opens a merge request.
Presentations. It takes a topic and the material behind it, and produces a PowerPoint deck as its artefact.
Rock, paper, scissors. For the reason above.

All four worked. The dependencies agent is also where the last commits of Wednesday came from. It needed to reach GitHub releases, the vulnerability scanner and the NVD database, so the egress allowlist grew three times that afternoon.

That is the allowlist doing its job, by the way. Every new host was a decision and a commit, not something the agent found on its own.

What changed in practice

The numbers here are from our own tests, not from a benchmark. Even so, they were the reason the idea felt real.

For the work we tried, the total time came down by three to four times. Part of that is the work itself. The bigger part is that nobody has to sit and watch it.

Some of our processes took more than ten hours on a work laptop. That does not fit in a working day, so we had to split them into pieces and run one piece per day. Now we start the job on the service, close the laptop and look at the result the next morning.

The screen

We built the UI in the same two days, as a team, like the rest. You can define a new agent visually: add its arguments, then its stages and steps, and see the agent drawn as you go. No JSON by hand. Then you run it with its arguments, and watch the stages move. You read the logs live, open the artefacts, and answer the questions.

It has two skins. One follows our backoffice design system. The other is a deep-sea scene for the stage, where an octopus is the scheduler and each tentacle holds a run. A key switches between them during the demo, and the same code serves both.

It talks to the real scheduler through the gateway, which now forwards both the HTTP API and an A2A endpoint. Other agents can start and inspect work the same way people do.

What is not there

It is a hackathon project, and some of it shows.

The scheduler has no authentication. It is only safe on a trusted network.
The sandbox keeps its state in memory. A restart loses the runs in flight.
The agent is root inside the VM, and the Git and Jira tokens are in its environment. The VM is the boundary, so the tokens should be scoped to what the runs need.
The database has no backups.

None of these stopped the demo. All of them would stop a real rollout.

What we took from it

Two and a half days is not much. What made it enough was spending none of it on isolation, keys or cost. The gateway and the sandbox took care of those, so the whole team could work on the new parts.

Those parts were deciding when a stage is finished, and letting a stateless agent hold a conversation. Both turned out to be less about code and more about not trusting the agent. Check the artefacts, not its word, and rebuild its memory, because it has none.

And if you ever need to test that an agent can wait for a person, make it play rock, paper, scissors. It is the cheapest integration test we know.