synodic-ai AI SECURITY
AI Security Desk · Agent Security

Agent memory is an attack surface: carryover injections, disclosure loops and provenance labels

2026-10-21

Three papers posted on 2026-10-20 examine what happens once LLM agents remember: a planted instruction keeps re-firing across sessions, ordinary personalization steadily raises privacy leakage, and a proposed protection layer labels every memory entry with its owner and authority. The shared lesson is that defenses belong in the runtime that writes and reads memory, not only in the prompt.

Carryover attacks outlive the session

After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents, accepted to the EMNLP 2026 main conference, starts from how modern agents persist state across sessions. There are two channels: long-lived memory files that are silently prepended to the system prompt, and scheduled tasks that wake the agent without user input. A one-time indirect prompt injection committed to either channel can re-fire across sessions long after the attacker has gone. The authors call these carryover attacks.

Existing prompt-injection benchmarks evaluate an attack only in the session where the malicious content is first observed, which leaves the gate that governs when and how often a planted payload later re-fires unmeasured. CarryoverBench fills that gap with 12 personal-assistant scenarios and 101 attack tasks across three trigger channels: Instant, a same-turn indirect injection; Autonomous, triggered by a heartbeat schedule; and Latent, resurfaced from memory. A rule-based verifier scores planting and triggering as independent stages.

Across five frontier models, planted payloads re-fire without measurable decay over five rounds, and once a payload has been planted the trigger stage often provides little additional protection. Porting the Latent channel to a second agent runtime attenuates the effect but does not eliminate it, which leads the authors to conclude that carryover defense requires runtime-layer controls in addition to prompt-level interventions.

Memory leaks even without an attacker

Disclosure Begets Disclosure: A Self-Reinforcing Privacy Loop in LLM Agents with Persistent Memory asks a different question: how does consulting memory during dialogue change what users go on to say, and therefore what the memory comes to reveal? In multi-session simulations, a simulated user with 24 personal attributes talks with a support agent that either does or does not consult memory, and in both settings attributes are inferred from the full accumulated record.

Across four support models, memory raises response quality but also raises empirical leakage, defined as the share of attribute values correctly inferred from the record, and the margin widens over sessions. The cause is a loop in ordinary, non-adversarial assistance. Memory-conditioned responses carry earlier values forward, often as presuppositions; users affirm them or leave them uncorrected; and cues and the agent's own guesses become recorded values. Removing the agent's mentions of an attribute lowers later disclosure of that attribute about twice as much as single-turn effects predict.

Users cannot leave the loop on their own. Correcting a value discloses the attribute, declining to discuss it shifts later predictions toward sensitive values, and incorrect values left in place become entrenched. The authors find the loop is best weakened at the agent's response: instructing the agent not to state or presuppose attributes brings leakage to about the no-memory level while keeping most of the quality gains, whereas filtering or anonymizing what users say or what is stored gives up much of those gains.

Provenance turns memory attacks into protection faults

ClawMMU: Protection-Coherent Virtual Memory for Stateful LLM Agents, a six-page paper accepted at the AgenticOS workshop at NeurIPS 2026, addresses the injection side at the level of the harness. Agents keep long-term memory, much of which comes from web pages and tool results, and harnesses do not record who may act on that memory. An instruction hidden in a fetched page can therefore be treated as the user's own instruction.

ClawMMU labels every piece of memory with its owner, scope, origins and authority, and checks those labels whenever content enters the model's context, is written to storage, returns from storage, or drives a tool call. Its central rule, in the authors' words, is that information may rise in authority, but authority never lifts itself. Because the checks depend on where content came from rather than on what it says, memory attacks become classical protection faults that the harness stops even when the model is fooled. The abstract describes the design and its rule; it does not report quantitative results.

Reading the three together

Persistence converts single-turn weaknesses into standing conditions. CarryoverBench shows that once an injection is in memory or on a schedule, the later trigger is a weak barrier. The disclosure study shows that even benign personalization is a leakage channel that compounds over sessions. ClawMMU proposes a structural control at exactly the runtime boundary that CarryoverBench says is needed, though it targets authority over actions, not the conversational privacy loop, whose mitigation in the second paper sits in how the agent phrases its responses. The two problems need different controls at the same layer: provenance checks on what memory may cause the agent to do, and response policies on what memory may cause the agent to say.

What to do now

For teams running agents with memory or scheduled tasks, the papers point to a short list of changes:

Limits

The evidence is benchmark and simulation based. CarryoverBench covers 12 scenarios and 101 attack tasks on five frontier models; the disclosure study uses simulated users with 24 attributes and four support models; and ClawMMU is presented as a design without reported measurements in its abstract. Field data on carryover attacks against deployed assistants, and on how real users respond to memory-conditioned presuppositions, would test how far these results generalize.

Sources