[KongchangAI]
Product

sentinel-eval

An open-source local testing tool for quantifying authorization vulnerabilities and leakage risks in vector databases within RAG and agent memory scenarios. It supports deterministic offline testing.

Timeline (last 90 days)

Nov 9

sentinel-eval is a local testing tool, licensed under Apache-2.0, for quantifying authorization leakage risk in Agent memory scenarios.

Unverified50%
Nov 9

In sentinel-eval's tests, the original RAG scenario had a leakage rate of 0% (18 public chunks retrieved, 0 restricted chunks exposed); in the Agent memory scenario, 2 of 3 synthetic tasks leaked, and the cross-boundary workload leaked 6/18 canary markers.

Unverified50%
Nov 9

sentinel-eval's test suite includes 23 local tests using mock vectors, completing in seconds, deterministic and requiring no API keys or network calls.

Unverified50%
Nov 9

The reproduction commands for sentinel-eval are: git clone https://github.com/harishbalasubramanian/sentinel-eval, then run uv run sentinel-eval live --framework chroma.

Unverified50%

All Facts (4)

Source Articles