Do Agents Dream of Electric Sheep? How Claude Revisits Its Memory (Series #AI)

When I got to the word dreaming in Anthropic’s “The components of agent memory”, I immediately thought of one of my favorite books, Philip K. Dick’s Do Androids Dream of Electric Sheep? I’ve mentioned the novel in Book Cube before, when I wrote about Dick’s biography in comics and the Blade Runner 2049 art book. Here the dreams had an engineering job. For example, they could check the log to see whether you had actually made it out of a cave in Pokémon Red.

In the November 6 article, Anthropic’s Lance Martin looks at what Claude can carry from one session to the next. As it works, the agent leaves a log and can write down an important finding. Later, a separate background pass, the dreaming step, checks notes against the original data, fixes errors, and assembles related wiki pages. The most useful items can go into a short index that the agent will see at the start of its next session.

Anthropic tested different parts of this approach. In CL-Bench, Claude answered database questions, received the correct answer, and could leave a note for later. In Pokémon Red, it wrote that it had got through the cave, then jumped off a ledge three times and landed back on a route it had already covered. With a note corrected against the log, Claude returned to the cave and reached the next town. On 25 questions about Stainless documents, wiki plus search met 61% of the criteria, compared with 53% for search alone. The wiki alone reached 48% at best. These are results from Anthropic’s own tests.

Now to my margin notes.

🔸 How a hunch turns into a rule In my write-up of Richmond Alake’s talk, I wrote about procedural memory and how an agent saves successful ways of working. Claude had a note about changing the pause between button presses after a series of unsuccessful throws and then catching a Magikarp. I’d keep it as an idea for the next attempt. To record the rule “a pause improves the chance of catching it,” you need to compare a series of attempts with and without the pause. In the database test, Opus 5 recorded the rule and how it verified it. There’s another good twist in CL-Bench. After a database schema change, even a verified rule may need to be checked again.

🔸 What to tell an agent about memory I wrote “what, not how” next to this part. Telling Sonnet 4.6 explicitly to read its notes first and verify its guesses helped. Opus 5 lost a substantial share of the benefit from memory when told to record only errors. To me, those are different decisions: reminding the agent to consult past experience and deciding for it in advance which experience counts as useful.

🔸 What could search do? This reminds me of RepoSwarm, which I’ve written about before. It assembles architecture documentation from repositories. In both cases the agent gets a ready-made map and can use it to reach the sources. To evaluate those eight points, though, I’d like to know what the wiki was compared with. Was it text search, semantic search, or a hybrid? In Anthropic’s public example, the agent searches with grep, but that run used SEC documents about a deal. The article doesn’t say how search worked on Stainless.

I’d like to see the same questions run with several search methods, each with and without the wiki. Anthropic reports saving roughly 20 thousand tokens per answer. What does it cost to build the wiki and keep it current? With those figures you could estimate how many questions it takes for the dreams to pay off.

#AI #Agents #Research #Architecture #Evals