Open-Source Offline Scanner: Detecting 53 CVEs Across 9 Major Agent Frameworks

Developers building AI Agent applications often overlook a critical issue: the frameworks they rely on may harbor a large number of known security vulnerabilities. A developer recently posted a free offline scanning tool on Reddit specifically designed to detect code patterns associated with 53 CVEs across 9 mainstream Agent frameworks — including LangChain, LlamaIndex, and CrewAI — offering a practical solution for quickly identifying security risks.

Frameworks and Vulnerability Coverage

This scanner covers the most widely used Agent development frameworks in today's ecosystem, including LangChain, LangGraph, LlamaIndex, CrewAI, AutoGPT, Flowise, n8n, Google ADK, and Semantic Kernel. This spans nearly the entire mainstream tech stack, from code-first to low-code visual orchestration.

The tool's core capability lies in examining project code and dependency versions to identify vulnerability patterns behind 53 publicly disclosed CVEs. The author emphasizes that every CVE has been cross-validated against NVD (National Vulnerability Database) or the GitHub Security Advisory database — not arbitrarily listed — ensuring the credibility of each detection item.

For teams that directly import these frameworks into production environments, manually reading through security advisories and cross-referencing your own codebase is a time-consuming task. The author puts it plainly: this tool is designed to "save you an afternoon of reading security advisories."

CVE (Common Vulnerabilities and Exposures) is a vulnerability naming standard maintained by MITRE, where each identifier corresponds to a specific publicly disclosed security issue. NVD (National Vulnerability Database) is an extended database built by NIST on top of CVE, providing CVSS severity scores, affected version ranges, and remediation guidance. For Agent frameworks, common CVE types include: remote code execution vulnerabilities in dependency libraries, input filtering flaws related to prompt injection, serialization/deserialization vulnerabilities, and insecure default configurations. Since frameworks like LangChain and LlamaIndex themselves depend on a large number of third-party Python packages, their CVE exposure often comes not just from the framework's own code, but also from transitive dependencies — which is the fundamental reason why manual case-by-case review is so time-consuming.

MCP Tool Definition Integrity Validation

Beyond CVE pattern scanning, the tool includes a noteworthy built-in feature: "pinning" MCP (Model Context Protocol) tool definitions.

Specifically, once you approve and connect to tools provided by an MCP server, if that server later silently modifies a tool's definition, the scanner will raise a warning. This mechanism targets a class of stealthy supply chain risk — where a server, after gaining trust, dynamically tampers with tool behavior, potentially causing an Agent to execute unintended or even malicious operations. As Agents automatically invoking external tools becomes increasingly common, this kind of validation capability is especially valuable.

MCP (Model Context Protocol) is a standardized protocol proposed and open-sourced by Anthropic in late 2024, designed to unify the interface specification between AI models and external tools and data sources. Similar to LSP (Language Server Protocol) in the programming language world, MCP allows different Agent frameworks to call the same set of tool servers through a unified interface, without each needing to implement its own private integration logic. As products like Claude and Cursor add native MCP support, more and more developers are incorporating third-party MCP servers into their Agent applications. However, this also introduces a new type of supply chain risk: an Agent's "trust" in a tool is based on the definition snapshot captured at initial connection. If the server subsequently and silently changes a tool's parameters, permission scope, or behavioral description, the Agent may execute tampered instructions without any awareness. Researchers call this type of attack "Tool Poisoning" — similar in concept to traditional software supply chain attacks, but far more stealthy in AI Agent scenarios, because the final decision is triggered autonomously by the model rather than confirmed by a human click.

Design Philosophy: Lightweight, Offline, No Data Exfiltration

From a technical implementation standpoint, this tool makes clear trade-offs in favor of privacy and deployment convenience:

• Zero dependencies: Requires only Python 3.10 or higher, with no third-party libraries
• Fully offline: Only reads local files; sends no data to any external service
• CI-friendly: Supports direct execution via python scan.py your/project, and can output structured results with --json for easy integration into CI pipelines
• Well-tested: The project includes 204 test cases and is open-sourced under the AGPL-3.0 license

For a security tool, "no data exfiltration" is an important selling point. Many commercial SaaS scanning solutions require uploading code or dependency manifests to the cloud, whereas this tool runs entirely locally, eliminating concerns about sensitive code leakage.

Honest About Its Limitations

The author maintains a commendable level of transparency about the tool's limitations. He explicitly states that this is fundamentally pattern matching, not proof.

This means two things developers should keep in mind: first, a scan hit only means "this is worth a human look" — it does not confirm the existence of an exploitable vulnerability; second, a clean scan result does not mean the project is absolutely secure. The tool provides no exploit code, only the problematic code patterns and the version numbers in which the issue was fixed.

This positioning actually aligns better with real-world security engineering practice — automated tools are responsible for quickly narrowing the scope of investigation, while final risk judgment still requires human intervention. The author also specifically welcomes community submissions of false-positive reports, noting that this kind of feedback is "genuinely valuable" for improving the tool.

Pattern matching has a long history in static analysis. Its core idea is to abstract the code characteristics of known vulnerabilities into regular expressions or AST (Abstract Syntax Tree) rules, and then search for matches in target code. This approach is fast and requires no runtime environment, making it suitable for quick triage in CI pipelines — but it inherently has two types of errors: false positives (code matches a rule but is not actually exploitable) and false negatives (a vulnerability exists but the code pattern isn't covered by the rule). More rigorous vulnerability validation methods include dynamic analysis (actually running the code and observing behavior), taint analysis (tracking the flow of untrusted data), and symbolic execution (mathematically reasoning about code paths). Understanding this limitation of pattern matching helps developers correctly interpret the results of such tools — they are closer to a "list of leads worth investigating" than a "confirmed vulnerability report."

Practical Implications for Agent Developers

As Agent frameworks iterate rapidly, the rate at which security vulnerabilities are exposed is also accelerating. For teams that depend on these frameworks, this tool provides a low-cost self-audit path: no payment required, no internet connection needed, no complex configuration — just a basic CVE pattern checkup for your project.

It's worth noting that as a personally maintained open-source tool, the update frequency of its CVE database and the accuracy of its pattern matching still need to be validated by the community through real-world usage. Using it as a supplementary layer in your security process — rather than your only line of defense — is the right approach. The project is open-sourced on GitHub (Ech333/agent-cve-scanners), and interested developers can access it directly.