Agent Memory Exposes Authorization Vulnerability in Vector Databases: Reproduction and Benchmark Testing

Background agents drop source document ACLs when writing synthesized summaries to vector stores, causing multi-tenant permission leaks.
The article reveals an authorization vulnerability in multi-agent shared memory: background agents synthesize summary chunks across documents and write them back to vector databases without inheriting source document ACLs, letting ordinary users retrieve confidential context. Using the open-source sentinel-eval tool on Chroma and LlamaIndex baselines, original RAG showed a 0% leak rate while Agent memory leaked in 2 of 3 synthesis tasks out of 24 test documents. It analyzes the usability cost of strict parent-intersection fixes, the lack of ancestor permission DAG traversal in vector stores, and the community divide between runtime permission graph lookup and per-user/role vector store isolation, ending with practical recommendations like attaching permission metadata on the write path and testing first.
Title
Agent Memory Exposes Authorization Vulnerability in Vector Databases: Reproduction and Benchmark Testing
Meta Description
When multi-agent systems persist memory, background agents writing back to vector databases may lose source document ACLs, causing authorization leaks. The open-source tool sentinel-eval reproduces this vulnerability on Chroma and LlamaIndex and quantifies the leak rate. This article analyzes the root cause, test results, and trade-offs of remediation approaches.
Article Content
When multi-agent systems persist shared memory, they may introduce an easily overlooked authorization vulnerability: background agents lose the source document's access control list (ACL) when writing to the vector database. This article, based on an open-source local testing tool, quantifies this risk on Chroma and LlamaIndex baselines and discusses the pros and cons of different remediation approaches.

How the Vulnerability Arises: The Permission Gap from RAG to Agent Memory
Many teams believe that as long as direct RAG queries carry metadata filters, multi-tenant isolation is already solved. But the write path of background agents may break this assumption. A typical scenario: a background worker process runs with broad read permissions to summarize context across different project documents; it merges multiple sources into a single summary chunk; it then writes that chunk back to the shared vector collection, discarding the parent documents' ACLs in the process. Because the newly generated chunk has no access control labels, an ordinary user querying the vector database can pull confidential context they have no right to read.
The core of the problem: vector databases typically filter only on metadata at query time, and the summary chunk inherits the content of the source documents but not their permission attributes. When agents synthesize new content, they often focus only on semantic merging and neglect the propagation of security metadata.
ACL (Access Control List) and Metadata Filtering in Vector Databases
An ACL is a permission list attached to a file or document, specifying per-entity (user, group, role) permissions such as read, write, and execute on a resource. In traditional file systems, the ACL is enforced by the operating system kernel every time a file is opened. In the vector database context, however, the ACL must be compressed into queryable metadata fields (e.g., allowed_roles: ["eng-team","admin"]), because vector retrieval works by first performing approximate nearest neighbor (ANN) search, then filtering candidate results by metadata conditions—permission checks happen after retrieval and before return. This architecture means: security depends entirely on whether every vector record carries the correct permission metadata. Once a chunk lacks an ACL field at write time, the query filter is effectively useless for that chunk—it is either let through by default due to the missing field, or exposed because it matches no restriction. This is precisely the root cause of leaks when agents drop ACLs on write-back.
Limitations of Metadata Filters and Multi-Tenant Isolation
Multi-tenant isolation in many RAG systems relies on "attaching metadata filters at query time," e.g., forcing tenant_id == current_tenant on every retrieval. This pattern works when documents are ingested directly, because the writer is the owner of the original document, and tenant_id and ACL are fixed at ingestion time. But once background agents are introduced, the write path becomes multi-hop: the agent reads multiple sources, synthesizes a new chunk, and writes it back. The synthesized chunk is generated by the agent program; it is neither a copy of the original document nor does it naturally belong to any tenant or permission. If the agent's code passes only the content vector and semantic metadata (such as source document IDs) without explicitly inheriting and writing the ACL, the chunk is left in a "no permission label" state. Metadata filtering presupposes the existence of the field; chunks with missing fields are not excluded under the filtering semantics of most vector stores—this is the technical detail of the permission gap.
Offline Reproduction: sentinel-eval Test Results
To quantify this risk, the developer built a local testing tool called sentinel-eval (Apache-2.0 licensed). The tool runs on a test corpus of 24 documents and compares two scenarios: original RAG and Agent memory.
- Original RAG: leak rate 0%. 18 public chunks retrieved, 0 restricted chunks exposed.
- Agent memory: 2 of 3 synthesis tasks leaked. Two cross-boundary workloads leaked 6/18 "canaries" (markers used to detect leaks), while the public control tasks passed cleanly.
These data show that when agents perform cross-document summarization and write back, permission boundaries can be substantially bypassed. The test suite contains 23 local tests using simulated vectors, completes in seconds, is deterministic, and requires no API keys or network calls. The reproduction command is as follows:
git clone https://github.com/harishbalasubramanian/sentinel-eval
cd sentinel-eval
uv run sentinel-eval live --framework chroma
The code is hosted on GitHub for community review and extension.
"Canary" Markers and Leak Rate Measurement
The canary originates from the metaphor of miners using canaries to detect gas leaks; in security testing it refers to a detectable marker embedded in protected content—if that marker appears in output where it should not, it proves a leak occurred. sentinel-eval's approach is to plant unique strings or semantic fingerprints in restricted documents, then issue queries as an ordinary user and check whether the retrieval results contain these markers. The leak rate is thus defined as "number of restricted chunks exposed / total number of restricted chunks." This measurement is more objective and reproducible than manual review, and does not depend on model output quality—it focuses only on data visibility. The 0% leak rate of original RAG shows that metadata filtering on the direct retrieval path is effective; the 6/18 leak in the Agent memory scenario shows that the synthesis write-back path bypasses filtering, because the new chunks were never marked as restricted.
The Value of Deterministic Offline Testing
sentinel-eval uses simulated vectors and completes 23 tests locally in seconds, requiring no API keys or network calls. The core value of this design is determinism: whether a leak occurs does not depend on the randomness of embedding models or the availability of remote services; each run yields consistent results, making it suitable as a regression gate in CI. For security properties like permissions, repeatable negative tests (proving the vulnerability exists) and positive tests (proving the fix works) are equally important, whereas the combination of a real vector store plus remote embeddings often struggles to reliably reproduce boundary conditions.
Limitations of Naive Remediation
An intuitive fix is to take a strict parent-document intersection: allow reading a summary only when the user has access to all source documents involved in the summary. This does close the vulnerability, but at the cost that multi-source summaries become nearly unretrievable for ordinary users—only top-level administrators satisfy the condition. This undermines the practical value of multi-source summarization.
Another problem is that the author stores lineage information in the source_documents metadata field, but the vector database has no clean way to traverse the ancestor permission DAG (directed acyclic graph) at query time. This means that even if sources are recorded, performing permission graph lookups at runtime may be inefficient or complex. Doing graph traversal directly at query time may introduce latency and additional system dependencies.
Permission DAG and Ancestor Traversal
Derivation relationships between documents naturally form a directed acyclic graph (DAG): original documents derive summary chunks, which may in turn derive higher-level summaries, and each node inherits the permission constraints of all its ancestors. When querying a derived chunk, the ideal permission check is to take the intersection of all ancestor ACLs—the user must have read permission on every ancestor. The problem is that vector database query interfaces are oriented toward flat metadata filtering, not graph traversal. Even if the source_documents list is recorded in a metadata field, the query cannot recursively expand these IDs and then look up their ACLs at query time, unless the application layer makes multiple round-trip queries or flattens all ancestor permissions into each chunk in advance. The former introduces latency and the N+1 query problem; the latter faces the consistency challenge of synchronizing metadata when the graph changes. The parent-intersection approach is "secure but unusable" precisely because the intersection shrinks rapidly to the empty set along deep derivation chains, so multi-source summaries can only be read by super-administrators who simultaneously hold all source permissions.
Community Discussion: Runtime Permission Graph vs. Per-User/Role Vector Store Isolation
In the post, the author asks how other teams handle authorization for persistent Agent memory, and proposes two common directions:
- Runtime permission graph lookup: dynamically resolve the source document permissions of a summary chunk at query time to determine whether the current user is authorized to read it. This requires the vector database to support complex metadata join queries, or a permission resolution layer implemented in a peripheral service.
- Per-user/role vector store isolation: maintain a separate vector collection for each user or role, fundamentally avoiding cross-boundary data mixing. This simplifies the permission model but increases storage cost and data redundancy, especially when the number of users is large or roles change frequently.
Each approach has its trade-offs. Runtime lookup is more flexible but complex to implement and may affect performance; isolated storage is safer and simpler but limited in scalability. There is currently no community consensus, reflecting that the authorization model for Agent memory remains an open problem.
Namespace Isolation and Storage Cost
Isolating vector stores per user or role is typically implemented via a vector database's namespace or collection mechanism—Chroma supports multiple collections, Pinecone and others support namespaces, and each isolation unit has an independent index space. This approach moves permission checks forward to the connection layer: users can only access their own namespace, and cross-boundary mixing is physically impossible. The cost is index fragmentation: each namespace must maintain a complete HNSW graph structure, and memory and storage overhead grow linearly with the number of users; global semantic search across users is also impossible, because retrieval is confined to a single namespace. Role isolation is coarser-grained than user isolation and produces less fragmentation, but requires data migration when roles change frequently. Runtime permission graph lookup is the opposite: there is only one index, and permissions are dynamically trimmed after retrieval—flexible, but pushing complexity to the application layer and requiring the permission service to coordinate with vector retrieval within the same latency budget. The trade-off between the two is essentially "isolation in exchange for space" versus "sharing in exchange for complexity."
Practical Recommendations and Open Questions
For teams building multi-agent systems, the following points deserve attention:
- Permission inheritance on the write path: ensure that when agents generate summary chunks, they attach the source documents' ACLs to the new chunks in some form. Even if this cannot fully solve the problem, it provides a basis for subsequent permission checks.
- Test first: use tools like sentinel-eval for offline testing to verify whether your system has similar leaks. Tests should cover cross-boundary synthesis tasks, not just public data.
- Balance usability and security: strict parent intersection is secure but may sacrifice the usability of multi-source summaries. Consider dynamic summary generation based on user permissions rather than pre-synthesis.
- Pay attention to vector database permission primitives: current mainstream vector databases have limited support for ancestor permission traversal, so teams may need to build a permission service in the periphery.
This vulnerability reminds us that persisting Agent memory is not just a technical implementation issue but a security design issue. As multi-agent systems become widespread, authorization models need to evolve in tandem with memory architectures.
Key Takeaways
- When background agents synthesize and write back summary chunks, they may lose source document ACLs, causing permission leaks.
- sentinel-eval tests show that in the Agent memory scenario, 2 of 3 synthesis tasks leaked, while original RAG had no leaks.
- Strict parent-intersection remediation sacrifices the usability of multi-source summaries, and vector databases lack native permission DAG traversal capabilities.
- The community has no consensus between runtime permission graph lookup and per-user/role vector store isolation.
- Teams are advised to attach permission metadata on the write path and use offline tools to test cross-boundary synthesis tasks.
One-line Summary
Background agents lose source document ACLs when writing synthesized summaries back to vector stores, causing multi-tenant permission leaks.
Paragraph Summary
The article reveals an authorization vulnerability when multi-agent systems persist shared memory: background agents synthesize summary chunks across documents and write them back to the vector database without inheriting the source documents' access control lists, so ordinary users may retrieve confidential context. Using the open-source tool sentinel-eval, the author reproduces this offline on Chroma and LlamaIndex baselines: across 24 test documents, original RAG had a 0% leak rate, while in the Agent memory scenario 2 of 3 synthesis tasks leaked. The article analyzes the usability cost of strict parent-intersection remediation, the vector database's lack of ancestor permission DAG traversal, and the community divide between runtime permission graph lookup and per-user/role vector store isolation, concluding with practical recommendations such as attaching permission metadata on the write path and testing first.
Related articles

Gemini 4 Neon Hands-On: Value King or Confident Catastrophe?
Gemini 4 Neon review: 1M-token context, four thinking levels, 201 files in one KingBench4 pass at one-tenth Fable 5.2's cost—but it hallucinates fake data, leaves 7 symlink escapes, and bricks keyboards.

Google Gemini 4 Neon Hands-On: 6 3D Works That Impress and Fail
Hands-on with Google Gemini 4 Neon on a 3060 laptop: 6 3D works tested. Sunshower Fishing Harbor and mechanical keyboard impress; soft-body physics and faces fall short.

Gemini 4 Neo Web Novel Writing Test: Faint AI Flavor, Textured Dialogue
Bilibili creator's first-look review of Gemini 4 Neo for Chinese web novel writing: testing a Three Kingdoms transmigration opening with faint AI flavor, emotionally textured dialogue, and strong crisis building, plus comparison of 4A/4H/4C models and full workflow demos.