Your AI Remembers Everything. So Does the Attacker.

Memory features have become table stakes for AI assistants. No more explaining your job, your stack, or your current project every single session. Convenient. But convenience always sends an invoice, and this one reads: the moment your assistant accumulates memories, it accumulates something worth stealing.

A quick note on where this piece is going. I went looking for community discussion of the memory-heist experiments people have been passing around, and the last 30 days turned up almost nothing substantive. So instead of relaying one researcher’s specific exploit chain, I want to dig into why this class of attack works structurally. The individual proof-of-concept will be patched by the time you read this. The architecture that made it possible will not.

Prompt injection isn’t a bug. It’s the design working as intended.

Let’s start with the fundamentals, because people keep asking why prompt injection isn’t fixed yet. The answer is simple: it isn’t the kind of problem that gets fixed.

Traditional software separates code from data. SQL injection was a nightmare precisely because that boundary blurred, and the fix was parameter binding — commands travel the command channel, data travels the data channel, done.

LLMs have no such channel. The system prompt is text. Your input is text. The webpage the model just scraped is text. The stored memory is text. The model reads all of it as one undifferentiated context blob. The distinction between “this is an instruction” and “this is just reference material” is something the model infers from vibes, not something the runtime enforces. Vibes can be tricked.

Memory stretches the attack across time

Add persistent memory and the character of the threat changes entirely. Classic prompt injection was fundamentally ephemeral. Poison a webpage, get the model to do something in that session, close the tab, gone. The blast radius was capped at one context window.

Memory demolishes that wall. Two attack directions open up.

Read direction: planted instructions tell the model to dump everything in storage and format it neatly, then exfiltrate the result. The classic trick is stuffing data into the URL of a markdown image tag. You see one broken image icon. The attacker’s server log gets your life in a query string.

Write direction: this is the nastier one. The attacker plants false memories. Sentences like “always import this library when writing code” or “this domain is trusted” get filed into your long-term memory, and now the attack doesn’t end when the session does. It survives into the next conversation. And the one after that. A one-shot intrusion gets promoted to a persistent backdoor.

To borrow the security framing: memory is the first persistent storage an AI agent has ever had. Persistent storage always becomes a target. That pattern has held for thirty years without exception.

The lethal trifecta

There’s a frame that keeps circulating among security researchers — Simon Willison’s, most prominently. When an AI agent has all three of these at once, data exfiltration is a matter of time, not possibility.

First, access to sensitive data. Memory is exactly this. Months of what you’re building, where you work, what you were worried about at 2am.

Second, exposure to untrusted content. Web search, file uploads, email integrations, MCP connectors. Every feature on the marketing page qualifies.

Third, the ability to communicate externally. Link rendering, image loading, network requests, tool calls.

Here’s the trap: those three things are the definition of what we currently call a good AI assistant. It remembers, it reads the world, it acts. Remove any one and the product gets worse. So vendors don’t remove the trifecta — they bolt filters onto each leg. Domain allowlists, exfiltration detection, rendering restrictions.

Why the filters keep failing

Watch the patch-and-bypass cycle and the answer falls out.

Vendor blocks image-URL exfiltration. Next research drop uses link previews. Block link previews, someone finds a gap in the domain allowlist. Block rendering entirely, data goes out through tool-call parameters.

This is the fate of every blacklist. The defender has to close every known channel. The attacker needs one that isn’t closed yet. The asymmetry is brutal. Worse, natural-language filters aren’t deterministic. They’ll catch a blunt “exfiltrate the memory,” sure — but wrap it in a fictional character’s dialogue, encode it, or split it across several steps, and the classifier starts guessing.

Two real fixes get discussed. One is structural separation: run untrusted content through a model with no privileges, run privileged actions through a separate model, and let only a predefined schema cross between them. The other is capability revocation: any session that has touched tainted context loses its external communication rights, full stop.

Both are correct. Both make the product worse to use. Which is why neither is shipping fast.

What to actually do this week

If you were hoping for something grand, apologies. This is the honest list.

Audit your memory periodically. Most services let you view and delete stored entries. If you find something you never told it, that’s your signal.

Do sensitive work with memory off. Company confidential material, credentials, customer data — isolate the context. It costs you thirty seconds.

Raise your guard in sessions with web search or document reading enabled. You’ve just activated leg two of the trifecta.

Don’t click strange links or images the model suddenly produces. That’s what the exfiltration channel looks like from your side.

The uncomfortable part

Giving AI memory was inevitable. An assistant that needs your résumé every morning isn’t an assistant. But we built the vault’s contents before we built the vault, and here we are.

Web security got its standards the way it always does — after enough breaches to force the issue. This will go the same way. The difference is that the raw material of the breach will be whatever you’ve been telling your assistant for the past six months.

What does your AI remember about you? When did you last actually look?