Memory vs no-memory on a five-session simulated project
Protocol pre-registered 2026-07-07. Results publish when all runs complete.
Hypothesis
An agent with an indexed-fact-file memory system will repeat fewer mistakes and reconstruct context faster across a five-session project than an identical agent without memory, with the gap widening by session.
Method
| Tasks | One project evolved across 5 scripted sessions, each session seeding facts the later sessions need (ports, conventions, decisions, gotchas). |
|---|---|
| Conditions | Condition A: no persistent memory. Condition B: indexed fact files with a session-start index load. Identical prompts otherwise. |
| Runs | N=5 project runs per condition (50 sessions total). |
| Scoring | Repeated-mistake count per session, context-reconstruction time (measured to first correct use of a prior fact), drift incidents (acting on stale assumptions). |
| Pinned | Session scripts, fact inventory, and memory-system configuration pinned at registration. |
Results
Empty until runs complete. This section is the only part of the page that will change, and the change will be logged in the changelog.
Limitations
Scripted sessions compress real project time; drift dynamics over weeks are simulated, not lived. Memory helps most when facts recur, and the script guarantees recurrence: results bound the best case.