RAG Access Control: 3 Ways to Enforce Document Permissions in AI Applications

RAG (Retrieval-Augmented Generation) is becoming the standard architecture for enterprise AI assistants, but it also introduces an easily overlooked yet critical problem: permission overreach. When a user asks the AI assistant a question, the retrieval layer may bring back documents the user has no right to access—contracts, financial reports, HR records—and then the model generates these contents "naturally" into the answer. The stronger RAG's retrieval capability, the greater this leakage risk becomes.

According to industry technical analysis, current mainstream RAG access control approaches can be grouped into three routes: On-behalf-of tokens, Live ACL re-checks, and Parent-level fine-grained authorization. They are not substitutes for each other, but rather apply to different system boundaries and trust models.

1. Why RAG's permission problem is trickier than traditional search

In traditional search, permission control happens at the moment of "returning results"—what the user sees is a filtered list of links. But RAG adds a layer: the retrieved content enters the large model's context window, and the model may summarize, infer, or rewrite it, ultimately outputting it in natural language. Even if the user never directly sees the original document, the model can still leak key information from it.

What's more troublesome is that embeddings in vector databases naturally "aggregate content." After a permission-restricted document is split into multiple chunks, a user may hit one of those fragments through a semantically similar query. Permission filtering must happen at the retrieval stage, not after generation—reviewing the output afterward is already too late.

This is also why the core difference among the three approaches lies in: at which stage permissions are verified, and how fine-grained the verification is.

Vector databases and embeddings: A vector database is a database specifically designed to store and retrieve high-dimensional vectors. Common products include Pinecone, Weaviate, and Milvus. In RAG, a document is first split into several chunks, and then an embedding model converts each chunk into a high-dimensional vector (i.e., an embedding). Mathematically, this vector encodes the semantic information of that fragment, allowing the system to find the fragments semantically closest to the user's query by computing similarity between vectors (such as cosine similarity).

It is worth noting that an embedding itself is a semantic compression of the original text and does not naturally carry any permission labels. Therefore, if no filtering conditions are attached to vector queries at retrieval time, the vector database will recall all semantically related fragments indiscriminately, regardless of whether the user has the right to view them. This is precisely the technical root of RAG permission risk, and why permission filtering must be embedded into the retrieval process rather than relying on post hoc output review.

2. Method One: On-behalf-of tokens

The on-behalf-of token model promoted by Databricks has the core idea of making retrieval issue requests as the user, rather than through a high-privilege service account. When a user asks the AI assistant a question, the system passes the user's identity credentials to downstream data sources, and the data sources themselves determine which documents that user can read.

The advantage of this approach is that permission logic does not need to be reimplemented at the RAG layer. Data platforms such as Unity Catalog already maintain complete ACLs, and on-behalf-of tokens directly reuse this authoritative determination. For organizations that have already centralized data governance on a certain platform, this is the path with the smallest changes and the most consistent semantics.

The cost is that it places high demands on identity system integration. If the retrieval pipeline involves multiple heterogeneous data sources, each source must support token passing, and engineering complexity rises quickly.

OAuth 2.0 and token passing: In engineering terms, the on-behalf-of token model usually relies on OAuth 2.0 Token Exchange or the On-Behalf-Of (OBO) authorization flow. OAuth 2.0 is the industry-standard authorization framework, originally used to let third-party applications access resources under user authorization without exposing the user's password. The OBO flow goes one step further: after the intermediary service (such as the RAG retrieval layer) receives the user's access token, it exchanges that token for a new token that accesses downstream data sources "as that user."

In this way, requests received by downstream data sources (such as Unity Catalog or Snowflake) look as if they were initiated by the user themselves, allowing precise determination using the data source's existing ACLs. The benefit of this mechanism is that the "single source of truth" for permission verification remains on the data platform side, and the RAG layer does not need to maintain a potentially outdated or inconsistent copy of permissions. The cost, however, is that every link in the chain must support token passthrough and exchange, requiring strong integration capability for heterogeneous systems.

3. Method Two: Live ACL re-checks

The idea advocated by AWS is: after each retrieval result is returned, immediately re-query the permission system and filter out documents the user has no right to access. Compared with static permission labeling at the indexing stage, live re-checks can reflect dynamic permission changes—employee transfers, project completion, authorization revocation—all take effect immediately.

The key to this type of approach lies in the word "live." Permissions are not a snapshot at write time, but a state at query time. For scenarios with frequent permission changes or strict compliance requirements (finance, healthcare, legal), live re-checks can avoid the awkward situation where "permission has been revoked but the AI can still read it."

The cost is latency. Each retrieval requires an additional call to the permission service, which may become a bottleneck under high concurrency. In engineering practice, permission caching and invalidation strategies are usually needed to balance consistency and performance.

ACLs and permission caching: ACL (Access Control List) is a classic permission model that maintains a list for each resource, recording which users or roles can perform which operations on that resource (read, write, execute, etc.). In traditional systems, ACLs are usually stored in relational databases or directory services, and querying them itself incurs some overhead.

Live ACL re-checks require querying the permission system after every retrieval, which introduces significant latency and load in high-concurrency scenarios. To balance performance and consistency, engineering practice often introduces permission caching: caching the user's most recent permission decisions in memory or Redis, with a relatively short TTL (time to live). When permissions change, event notifications or active invalidation mechanisms quickly expire the cache, striking a compromise between "revocation takes effect immediately" and "avoid hitting the permission service on every request." Understanding this helps evaluate the feasibility and tuning space of live re-check solutions in real systems.

4. Method Three: Parent-level fine-grained authorization

The third approach focuses on permission inheritance within document hierarchies. Enterprise documents are often not isolated files, but exist in containers such as folders, project spaces, and knowledge bases. Parent-level fine-grained authorization applies rules at the container level, and child documents automatically inherit them, avoiding the need to configure permissions separately for every document.

This method is especially practical in knowledge management scenarios. When a document is referenced at multiple levels, permission determination traces upward along the hierarchy to find the nearest authorization node. It reduces the cost of permission maintenance, but it also requires the system's document model to have clear parent-child relationships—flat vector storage often does not satisfy this prerequisite.

5. How to choose among the three methods

The consensus from multi-source analysis is: there is no universally optimal approach among the three; the choice depends on where your trust boundary lies.

• If data governance is already centralized on one platform (such as Databricks or Snowflake), on-behalf-of tokens are the most natural, reusing existing ACLs with the best consistency.
• If permissions change frequently and compliance pressure is high, live ACL re-checks better satisfy the requirement that "revocation takes effect immediately."
• If documents naturally have a containerized hierarchical structure, parent-level fine-grained authorization can greatly reduce configuration complexity.

In practice, many teams combine them: using parent-level authorization for coarse-grained fallback, live re-checks for dynamic changes, and on-behalf-of tokens to connect the identity chain. The key is that permission determination must happen at the retrieval stage, rather than being left to the model to "voluntarily comply."

One often overlooked point is that even if the retrieval layer filters correctly, the model's output should still undergo secondary validation. This is because the model may infer the existence or content outline of filtered documents from context—especially dangerous in multi-turn conversations. RAG's security boundary is ultimately a complete defense-in-depth system spanning identity, retrieval, and generation.

One-line Summary

RAG permission overreach risk originates at the retrieval stage, and the three access control approaches should be combined according to trust boundaries.

Paragraph Summary

In RAG architectures, the retrieval layer may recall documents a user has no right to access, and the large model then generates output from them, causing permission overreach. Traditional search filters at return time, while RAG adds a layer of model summarization and inference, making the risk greater. There are three mainstream approaches: on-behalf-of tokens (Databricks), which let retrieval initiate requests as the user and reuse data platform ACLs; live ACL re-checks (AWS), which immediately re-query permissions after retrieval and support dynamic changes but add latency; and parent-level fine-grained authorization, which uses document hierarchy inheritance to reduce configuration costs. None is universally optimal, and the choice depends on trust boundaries, with combinations often used in practice. The core consensus is that permission determination must occur at the retrieval stage, supplemented by secondary validation of output to form defense in depth.