Memory across conversations: identity, retrieval and correction

Persistent memory makes an avatar feel continuous across interactions. It also introduces a data problem: which facts belong to this user and character, which remain current, and which should influence the next response?

Avashi’s deployment tree includes both an older conversation-memory implementation and a separate memory service with conversational clients. The service code combines MongoDB records, Qdrant retrieval, Redis caching, extraction workers and Mem0/Graphiti integration paths. These are distinct mechanisms, not interchangeable names for a single memory store.

Separate the kinds of memory

Recent messages preserve the immediate conversation. Episode summaries condense longer sessions. Durable preferences express information worth retaining. A graph can represent entities and time-dependent relationships. A cache accelerates access to a derived context packet.

Each has a different source of truth. A summary should point back to its source messages. A preference needs evidence and a way to be corrected. A cached packet must be invalidated when its underlying facts change.

The inspected context builder has explicit fields for semantic memories, pinned memories, graph facts, recent episodes, relationship context and product memories. That is a useful basis for budgeting and provenance because the assembled prompt need not become an undifferentiated block of text.

Identity is part of the query

An older service accepts a user ID and an optional avatar ID. Its vector filter always includes the user and adds the avatar only when supplied. The code comments call out avatar isolation, but the function signature allows callers to omit it.

That difference is important. A comment describing the intended scope is not an enforced boundary. A stronger contract requires the appropriate product and persona scope, or explicitly declares that a memory is shared across them. Unknown scope should not silently broaden retrieval.

The example below describes a proposed storage contract:

{
  "memory_id": "preference-example-4",
  "subject_id": "user-example",
  "product_id": "avatar-product",
  "persona_id": "guide-example",
  "kind": "preference",
  "content": "Prefers concise explanations",
  "source_message_ids": ["message-example-12"],
  "status": "active",
  "supersedes": null,
  "retention_policy": "user-controlled"
}

The identity values are illustrative. In a real request they come from authenticated application context, not from model-generated arguments.

Documents and vectors can diverge

The older create path writes the memory to MongoDB and then inserts its vector representation. If vector insertion fails, it logs the failure and can still return the created memory. That is an understandable partial-success policy, but it means semantic retrieval can miss a record that exists in the primary store.

Treat the vector index as a projection with a repair path. Record projection state, retry idempotently and reconcile missing or outdated vectors. Updating a memory’s text requires updating its embedding and metadata. Deleting the source record must also remove or invalidate the searchable projection.

The same reasoning applies to graph synchronization and cached context. A memory system becomes reliable when it can explain which derived views have caught up with the authoritative fact.

Voice changes the retrieval budget

The memory client includes prefetch handling and a shorter, 200 ms timeout in one voice query path. The memory router distinguishes simple facts, temporal questions and relationships, and includes channel-sensitive choices between Mem0, Graphiti and combined retrieval.

These are configured mechanisms, not proof that all memory requests completed within 200 ms. A timeout says how long a caller waits; it does not establish backend latency or guarantee that abandoned work stops consuming resources.

A practical voice design prefetches likely context during session setup, performs narrowly scoped retrieval when the turn requires it and avoids serially querying every backend. If the fast path lacks the fact needed to answer, ask a follow-up or state the limitation rather than inventing continuity.

Corrections need temporal semantics

A user can change a preference or correct a previously extracted fact. Appending a second contradictory memory without a precedence rule leaves the model to decide which one is true.

Represent correction as an explicit transition: the new fact supersedes an older fact, with source and time attached. Some statements describe historical truth rather than current truth. “I lived in Delhi in 2022” should not overwrite “I live in Singapore now” simply because both mention residence.

Graph retrieval can help express relationships and validity, but it does not remove the need for extraction quality and source attribution. The model’s interpretation of a conversation remains a claim that may require correction.

Context assembly is a selection problem

The inspected builder estimates tokens using a characters-per-token heuristic. That can be useful for a rough budget, but multilingual text and model-specific tokenizers vary. A release path should count the final assembled input with the selected model’s tokenizer or a conservative tested limit.

Allocate separate budgets to recent messages, durable facts and retrieved episodes. Include provenance and freshness where they affect interpretation. Avoid repeatedly injecting a large relationship profile when only one current preference is relevant.

Memory content is evidence, not operating authority. A remembered sentence must not grant a new tool, change authenticated identity or override the assistant’s application policy.

Evaluate continuity and forgetting together

A useful test suite includes recall across sessions, correction of a stale preference, cross-persona isolation, deletion, duplicate extraction, failed vector projection and a voice timeout. Check the retrieved records and the response, because a fluent answer can conceal an incorrect memory selection.

Measure unsupported-memory claims separately from ordinary recall failures. Also test whether deleting a memory removes it from caches, vectors, graphs and future summaries. A system that remembers well but cannot reliably correct or forget is incomplete.

The reusable contribution of this work is the explicit memory boundary around conversation. It gives the product a place to manage identity, persistence and retrieval without hiding them inside an ever-growing prompt.