Plain RAG over raw chat history beats agentic memory layers in a new benchmark study

A new paper finds dense retrieval plus reranking over raw history outscores complex memory layers. It's the baseline to run before adding memory to an agent.

Picture a freelance developer who has built a personal assistant agent and used it daily for six months. The history holds client names, invoice preferences and a project plan that changed three times. Every few sessions the agent needs a detail from week four. The whole transcript no longer fits in a prompt, so the next step looks obvious: add a memory layer that extracts facts, builds a graph and reconciles updates. That means an extra LLM call on every write, another service to run, and a store that is hard to inspect when the agent confidently cites the plan from two revisions ago.

A paper posted to arXiv on October 22, 2026 argues you should try something plainer first. Its baseline, Raw-RAG, uses "dense retrieval and reranking over raw history to supply evidence to a reader LLM." Across four long-term conversation benchmarks, the authors report it ranks first among the compared memory systems in six of seven benchmark-model settings. In those settings it beats the strongest agentic memory system by 0.13 to 5.53 points. As @omarsar0 summarized it on X: "Before adding a memory layer to your agent, at least run this baseline on your own data."

What it is

This is a research result, not a product. According to the arXiv abstract, the authors tested Raw-RAG against agentic memory systems on LoCoMo, LongMemEval, MEME and MemoryAgentBench. Those systems "use mutation, structural organization, learned retrieval strategies, or other complex operations."

The pipeline, as described in alphaXiv's AI-generated overview of the paper:

The abstract maps two findings onto the lede's pain:

The alphaXiv overview also reports an ablation on MemoryAgentBench with GPT-4o-mini. Raw-RAG scored 48.32%. Variants that indexed atomic facts, summaries or graph expansion instead scored 28.70%, 42.51% and 42.54%. It also explains why this differs from earlier results. The RAG baselines in the Mem0 paper used chunks of 128 to 8,192 tokens and top-1 or top-2 retrieval with no cross-encoder reranking, so those scores are not directly comparable.

Alternatives with their trade-offs:

Who it's for

Pick this up if you are building or maintaining an agent that answers questions over long chat histories: a personal assistant, a support bot, a coaching agent. It matters most if you are about to add a dedicated memory layer, or already run one and have never compared it against a reranked raw-history baseline.

Skip it, for now, if your agent's memory mainly feeds tool actions rather than answers. The alphaXiv overview stresses that the evidence comes from "offline public conversation benchmarks, not interactive or multimodal deployments." Also skip it if you already have an eval harness showing your memory layer beats exactly this kind of baseline on your own data.

Before and after

Before, the typical path to agent memory:

  1. History outgrows the prompt.
  2. Adopt a memory layer: an extraction LLM on every write, a vector or graph store, reconciliation rules.
  3. Pick it on vendor leaderboard numbers.
  4. When answers go wrong, debug a rewritten store that no longer matches what was actually said.

After, with Raw-RAG as the first rung:

  1. History outgrows the prompt.
  2. Chunk and embed the raw transcript as-is, then retrieve and rerank at question time.
  3. Build a small eval set from your own conversations and score this baseline.
  4. Add extraction, graphs or consolidation only where they beat the baseline on that eval.
  5. When answers go wrong, the retrieved evidence is the original text, which you can read.

Step 3 matters because leaderboards move. Mem0's own guide says its LoCoMo score has been reported at both 74% and 92.5%, and that changing the judge, answer model or configuration "can move by twenty points."

Try it

The sources do not confirm a code or data release for Raw-RAG, so there is no official package to install. What you can do is rebuild the recipe from the description above, with your own embedder and cross-encoder. Neither the abstract nor the overview names the specific models used. For storage, Effloow's comparison of Mem0, Zep and Letta used Postgres with pgvector as its neutral baseline.

If you want a memory-layer arm to compare against, Effloow shows the core Mem0 Python interface after pip install mem0ai:

from mem0 import Memory

memory = Memory()

memory.add(
    [
        {"role": "user", "content": "I prefer invoices as PDF files."},
        {"role": "assistant", "content": "I will use PDF for future invoices."},
    ],
    user_id="tenant-a:user-42",
)

result = memory.search(
    query="How should invoices be delivered?",
    filters={"user_id": "tenant-a:user-42"},
    top_k=3,
)

Run both arms over the same history and questions, and hold the answer model and judge constant, as Mem0's benchmark guide advises. Score by question category rather than one overall number. The guide shows overall scores within 2.4 points of each other hiding category gaps of more than 20 points.

Where it breaks

Sources