Why Replay Matters for AI Agents That Take a Different Path Every Run
Most programs are boring in a good way. Give them the same input and they do the same thing. If one fails, you can usually run it again and watch it fail again.
An AI agent isn’t like that. The model’s answer varies from run to run, so the agent picks different tools, calls them in a different order, reads different pages and ends somewhere else. Ask it the same thing three times and you may get three different runs, and only one of them may go wrong. Run it again to see what happened, and you’re watching a different run.
So with agents, the kept run is the only record there is of what a particular run did. Logs help, but they hold what someone thought to log. A recording holds what actually went over the wire.
What a Recording Gives You
Most agents talk to the world over HTTP: the model API, search, web pages, other APIs. CodeInstances records all of it from outside the program, whatever language the agent is written in, as long as it uses the usual proxy settings. That includes the model’s own replies, because they come back as HTTP responses too.
Replay then hands the agent the same replies in the same order. The model “says” exactly what it said the first time, the tools return what they returned, and the run goes down the same path. You can step through the bad run as often as you like, offline, without paying for a single model call.
And you can replay a changed agent against it. Fix the prompt handling or the tool code, replay the recorded run, and see whether the new version still makes the same calls. If it asks for something the recording doesn’t have, the replay says so, and that tells you exactly where the new version took a different turn.
What v0.1.0 Can and Can’t Do Here
Be clear about the limits. A replay only makes sense while the agent makes the same requests. Once a change sends it somewhere new, the recording has no answer for that, and the replay reports the divergence instead of guessing. That’s the honest behaviour, and it’s also the useful one, because the first divergence is usually the interesting moment.
Streaming model responses are buffered whole in v0.1.0, so a streamed reply reaches the agent all at once on replay. Agents that use WebSocket aren’t recorded. Clock and random capture is for Go programs only. The reference has the whole list.
Where This Is Heading
A kept run is also evidence. When an agent does something that matters (sends money, changes a record, writes to a customer) someone may later need to show what it was asked, what it was told and what it did. Regulators are heading the same way. The EU AI Act asks high-risk AI systems to log events automatically.
Today an instance carries a SHA-256 id, so an edit made without rewriting the id shows up at once. It doesn’t prove who recorded the file, since anyone can change one and write a new id. Signed instances, which would, are on the project’s plan along with comparing runs and shrinking a failing run to its smallest form. Agents are one use for all of that, and a good one. They’re not the only one.
The quick start records a first run in a couple of minutes.