· Diar Shakimov · 5 min

memory is capped by what it can gather. for a team, that's the whole problem.

shlok khemani spent a year taking apart chatgpt's and claude's memory and landed on one limit: the context a memory system can gather. for a coding agent on a team, that limit is the problem.

case 06

shlok khemani spent a year reverse-engineering how chatgpt, claude, gemini and poke remember their users. his talk at the ai engineer world's fair ends on one limit. the best architecture, infinite compute and even continual learning are all "Still capped by the context it can gather about you."

i think he's right, and for a coding agent on a team it's the whole problem.

the decision happened somewhere else

his example is a trip. chatgpt saw two conversations about two possible trips, thailand and turkey. he and his partner picked thailand in person, and the assistant never heard that. the bookings were in his email, and in his experience connecting email didn't get them into the profile. so both trips stayed in memory as if they'd happened.

swap the partner for a reviewer and that's every team i've worked on. the decision gets made in a review thread, a call or another engineer's agent session. the person asking the agent wasn't there, and nothing tells the agent it happened.

where i disagree: stale matters more at work

in his june post on chatgpt's memory he writes "Staleness isn't a bug in the memory system." his reason is that people rarely tell an assistant when plans change, and a model can update mid-conversation when you correct it. for one person's assistant that holds, because you know what changed in your own life.

at work the person asking often doesn't know. i hit this the first time rema ran inside a real agent. the question was about a team's uv config. rema handed the agent a uv error report from may 24 2024, 07:59 utc. at 13:25 a maintainer answered in the next comment that the old value was a deprecated alias, and the alias merged six minutes later. rema had the report and not the reply, and the agent told the user the error still happens. the user had no way to correct it mid-conversation, because they didn't know either.

so a hit in rema now brings the next comment in its thread with it. that fixed this case. a correction two comments down, or in a different thread, still gets missed.

a team has no profile

his talk shows chatgpt and claude landing on the same shape. there's a running profile that's always in context, plus search over past chats. he reports chatgpt's profile at about 4,000 tokens refreshed every few days, and claude's at about 1,000 tokens refreshed every 24 hours.

a team doesn't have a profile like that. it has decisions, each written down in one place by one person on one day. so rema keeps nothing always loaded. on every prompt it searches what the team wrote down, its issues, review threads, docs and agent sessions, and hands the agent the passages that match.

his other point is about trust: "the surest way to earn a user's trust is to let them see" what the product believes, and correct it. his june post is about chatgpt's memory summary, which he says isn't what goes into the model's context. in rema what the agent is told is the passage itself, and what you can install today puts the source and file date on every one. you can click through and see who wrote it and when.

assistant memoryrema
whose contextone person'sa team's
always in contexta running profile (~4,000 or ~1,000 tokens)nothing; searched on each prompt
where it gathers fromyour chats, some connected appsissues, review threads, docs, agent sessions
what you can seea summary that isn't what the model gets (chatgpt)the passages themselves, each with its source
knowing it has nothingkeeps confident guesses (his trip example)no reliable signal yet
Table 1. assistant memory as shlok khemani describes it in his talk and june post, next to rema. the last row is where rema loses.

that last row is real. in september i asked rema six questions about the uv threads, three it could answer and three it couldn't. two of the unanswerable ones scored higher than one answerable one, so the search alone can't tell when it has nothing. inside an agent the model reads the passages and judges them, and the hook tells it to say when the team's sources don't cover the question. that helps, and it isn't solved.

if you hit this

pick one decision your team made last month in a review thread or a call, and ask your agent about it cold. if it doesn't know, that's the gap, and a bigger model won't close it.

what this changed in rema

rema gathers from where a team actually decides (issues, review threads, docs and agent sessions) and puts the source on every passage, so a person can see what the agent was told and fix it.

Sources