Persistent AI memory is the part that keeps your details on hand, so you don't have to repeat your job, your stack, or your preferences every single time you talk.
Ours was clever, but it wasn't right. We tore it all down in March 2026, taking out nearly every piece of retrieval we'd built, even the vector search. And it worked better.
This covers the removals we made and why.
What does persistent AI memory actually store?
It holds onto specific details you share, not the full conversation. We're talking about things like your company's name, what language you use, your preferred style for answers, or a particular rule you set one time. These pieces are all concise, and they're designed to be stable over time, applicable to all your chats.
This information gets pulled from your chats and goes into a Postgres database. You have full control; every single memory is visible in your profile settings, and you can change or remove any of them. The system also catches direct instructions, so if you tell it to remember, forget, or update a fact, it handles that explicitly.
How does ChatFuse create a memory in the first place?
ChatFuse builds these in 2 distinct methods. The majority are pulled out by a system that analyzes the chat and suggests structured options; these candidates get checked against the existing data store before any new entry is made.
The other method is a direct command from you. When you tell the platform to remember, update, or remove a fact, that instruction is caught by a separate detector. It doesn't rely on the extraction model to spot it. We keep these paths separate because an automated inference is a guess, while a clear user instruction is not, and we never want the 2 to conflict.
Each piece of memory has a priority setting that determines its chance of being recalled. You can mark one as 'always', 'context', or 'rare'. If you pin it, that memory gets pulled first every time.
Why did the original design use vector search?
Vector search was the clear solution, and in theory it is the right one. First you encode every memory, then you encode the query, and finally you grab the most similar ones. That way, only the useful memories get sent to the model, so the prompt stays compact.
But we didn't stop there. We also built an LLM layer to check the results, a keyword backup, a graph system to grab connected memories, a scoring fallback, and a feature that pushed one memory at random 15% of the time. We added each one to fix a problem the last layer created.
What went wrong with it?
The search was pulling up user memories that didn't match the query at all, and each safety layer just hid the problem more. We had the similarity cutoff set to 0.3, and that's so low most searches will grab nearly any stored text, letting unrelated details through.
The language model's own filter was set to allow anything even loosely connected, so it rarely removed anything; if it failed, it just sent every possible result through. 2 different data insertion paths weren't working the same way anymore. One of them prefixed the memory with the tag "Relevant user memory:", and the model treated that text as a command, so it forced private information into responses where it didn't belong.
Request a simple poem and it would include your employer's name. That's what users saw. Every component was functioning as designed, which is what made this so difficult to catch.
What did you replace it with?
A single ChatFuse query handles the entire thing: it pulls every active memory, sorts by whether it's pinned first, then by its importance score, and then by how recent it is, and it stops at 20 results. The system doesn't do any embedding matching when you ask a question, there's no relevance filtering, and no backup plans. The model that writes the answer just figures out what's useful on its own.
We don't use embeddings for retrieval anymore. They still get made and saved when data comes in, since that's cheap and they might be useful for something else later. But pulling answers doesn't go near them.
Our model picks what's relevant right when it's generating a response. That's the same way ChatGPT and Claude handle it. Turns out, the big model reading your prompt is just better at spotting what matters than a precalculated cosine score ever was.
Does injecting every memory not waste context?
It's cheaper than the old way. A hard cutoff at 20 items for simple facts keeps the memory block tight, and it ends up smaller than what our last system would push through once its filters stopped working.
This is a real constraint, we aren't hiding that. It only works since most users don't have that many personal memories. If someone built up thousands, then the limit would cut useful things and we'd need to pick what stays. We'd rather build that selection later, when we actually need it, than keep a complex retrieval system that never paid for itself. AI that manages its own memory is a separate issue, which we cover in self maintaining AI memory.
How do you stop memory leaking into creative work?
We now instruct the model to use memory only when it's necessary for a full and correct response, and we put that same rule directly in the system prompt for both pipelines. It isn't a retrieval rule anymore.
For creative tasks like making images, writing poems or stories, or brainstorming, we start with the standard output. Then we can show a personalized version if it helps. We also wrapped the memory block in <user-memory> tags instead of using a phrase like "Relevant user memory:". That stopped the model from misreading the label as a command. This tag change fixed more problems than any adjustment we made to retrieval.
How does memory work when the model changes mid conversation?
Your memory lives at the account level, so it follows you no matter which model handles your next request. Since ChatFuse can send different parts of a conversation to different models, the third model in a chain receives the exact same memory as the first one did. Nothing gets dropped along the way.
We explain how the routing works in our post on AI model orchestration. That consistent memory block is why you can switch AI models mid conversation without having to start over. This kind of continuity is a big reason we keep memory in the platform itself instead of inside a single provider's account. We cover that in more detail at own your AI context.
Who can see my ChatFuse memories?
You and the AI that responds to you. A system guard enforces incognito mode and a kill switch, which means an incognito chat can't read from or write to memory.
All memories are viewable and adjustable in your profile settings. If you delete one, it stops showing up in responses right away. Our full approach to access control is detailed in our zero trust approach to AI data security.
What happens when two memories contradict each other?
A conflict check looks for conflicting memories at the moment they are saved and sorts them out right there. If you don't do this, the model gets 2 facts that can't both be true and has to guess which one is correct.
Is persistent AI memory the same as a long context window?
No. A memory system holds onto things from one chat to the next, even between different models. A huge context window only works for a single session; it always starts from zero. But memory brings what it knows about you into a brand new conversation right away. Here's how they stack up:
What did ChatFuse learn from the memory rebuild?
We'd been adding all these extra steps because our initial filter was set way too loose. Each new patch just buried the real problem deeper. So we cut out 8 different mechanisms and put in a single, ordered query instead. Accuracy improved.
The big lesson? If your system keeps needing backup plans for its backup plans, the real issue is probably right at the start. You don't need a third fix; you need to fix the first one.
How do I try persistent AI memory in ChatFuse?
Memory is automatically enabled for all accounts. You can see everything it saves right in your profile, and you're free to change or remove anything in there. A good way to test it is to give it a piece of information you want it to hold, then open a fresh chat and see if it recalls that detail only when it's useful.
Start free and try it.
Comments
Loading comments…