Intro
Engineering teams and their agents are pushing more code than ever. As teams grow and onboard new engineers (and with them a new fleet of agents), the codebase grows, the product grows, and therefore collaboration, organization, and consistency matter the most. Think the approach you tried in March and rolled back when it doubled read times, the file someone is halfway through rewriting that your agent should not copy yet, the reason you said no to the obvious fix last year and whether it still holds. Quirks like these are rarely important enough to document, but they hurt the most when you have to remind your agent of them every single time. Engineers no longer want to tweak their skills for every task they do, they want the platform they instrument their agents on to feel native, almost like an extension of themselves. This led us to build a memory system that works across every harness, and gives every fleet of agents two things, your organization’s shared standards and the personal preferences unique to each and every engineer using Replicas.
How Memory Works
After a workspace sleeps or gets archived, we asynchronously run workers that parse the chat, including tool calls, reasoning, and the back and forth between the agent and the user.
The workers use semantic judging to create rollout summaries for the elapsed session. What the user asked for and what the tools returned outweigh what the agent said about its own work. Every summary is checked for credentials and secrets on the way out, including base64, hex, and reversed encodings of anything we already know is a secret.
From there, new workers pick those up, compile the findings, and write them into two different MEMORY.md files, organization and personal.
Each scope keeps its own index and its own rollout summaries.
- Your organization’s memory. The decisions, procedures, and preferences safe to share with the whole team, so a new teammate’s agent already knows what the team tried and dropped, and why.
- Your personal memory. Your preferences and working history, the things you would otherwise repeat in every new workspace. It follows you, so it installs into the workspaces you start, not into the rest of the team’s.
Most if not all coding agents are trained to grep and read filesystems extremely efficiently. Thus, we thought it made sense to build our memory system as Markdown files, nothing fancy. Your agent greps those indexes first and only opens a rollout summary when it needs more detail.
Because memory is just a file installed on every workspace, it works across every harness and inference provider.
Benchmarking Memory
A memory system is easy to demo but hard to trust, so we benchmarked ours against the leading industry harnesses, Claude Code and Codex. Determining the correctness of code an agent produces is highly inconsistent and impossible to guarantee, so we put agents with and without memory up against reputable open source repositories, Next.js, ClickHouse, Grafana, Supabase and PostHog.
We chose 6 tasks from each repo that we thought would be good tests. 5 of them build up the memories and 1 is held back as a final exam that tests retrieval. Every task runs under three conditions per harness, no memory at all, the harness’s own native memory, and Replicas memory.
From there we gave our judge agent a Markdown file of the correct patterns and best practices from each repo and told it to use that as its reference for correctness. This way we judge agent output against what was actually merged into production for these repos.
Early results are promising. Memory raised correctness against the no memory counterparts, and landed on par with or slightly ahead of the native implementations. The difference is that ours is harness agnostic and splits organization from personal, so the same gains follow your team everywhere. We aim to post full results on benchmarking in the near future.
Where Memory Goes From Here
Think back to what you find yourself explaining twice. That’s what memory is for, and your agents now pick it up from the work your team already did.
Memory is not about recording everything your team has ever done. It is about making sure what you already worked out, and what has changed since, does not have to be explained again every single time. The same correction stops coming back, and your agents show up already working the way you do.