How Can AI Agents Be Both Secure and Fast? Two Sigma's Five-Layer Defense in Practice

When AI agents can execute operations at lightning speed in production environments, enterprises face a core dilemma: how do you ensure security without constraining agent capabilities? Two Sigma engineer John Feminella shared their systematic thinking and practices in this area.

The Nature of the Problem: Balancing Security and Innovation

In regulated financial institutions, insufficient security is an existential risk, while insufficient innovation is a slow death. Feminella cited John Shedd's famous quote to illustrate the core philosophy: "A ship in harbor is safe, but that is not what ships are built for."

Two Sigma's approach starts with five key elements to build an agent governance system that ensures security without stifling innovation. This framework is not one-size-fits-all—each enterprise needs to adapt it based on its own risk exposure and business objectives.

Five-Layer Defense System Explained

Agent Registry: Eliminating Duplicate Efforts

The agent registry is the first step in the entire system. It records each agent's tools, endpoints, owners, and associated products, enabling agents to understand and collaborate with each other and with humans.

Two Sigma's registry design prioritizes minimal intrusiveness: agents created through approval templates are automatically registered, while customized agents are detected through heuristics and automatically registered, with their owners notified. Without a registry, teams waste enormous amounts of duplicate time, effort, and tokens.

Risk Scoring: Differentiating Agent Risk Levels

Mainstream agent frameworks (such as Claude Code, Codex, Pi) are fundamentally different from most enterprise-deployed agents. The former are open-ended—at startup, they don't know what requests they'll receive or what purpose they'll be assigned, which poses enormous challenges for risk assessment. Enterprise agents, on the other hand, almost always have specific and dedicated purposes, making assessment dimensions much clearer.

The registry scores each agent's risk profile based on its declared tools, purpose, and systems, updating with every change. Low-risk agents with limited capabilities can be published without human intervention, while high-risk agents are flagged. This differentiation mechanism concentrates review efforts where they're most needed.

Egress Control: Zero Trust by Default

Part of an agent's value comes from its ability to combine and remix information to produce output. But Two Sigma's agent runtime environment trusts no agent by default—all agents run behind a proxy that only allows approved destinations.

This proxy doesn't trust agent code, adopts a fail-closed strategy, and pulls each agent's policy directly from the registry. This approach eliminates the need to submit firewall tickets or maintain fragile configurations.

Fail closed is a default strategy in security design that means when a system encounters an anomaly, missing policy, or uncertain judgment, it denies the operation by default rather than allowing it through. The opposite is "fail open," which allows passage by default during anomalies. For AI agents running in production environments, fail closed means that if the registry is temporarily unreachable, policy parsing fails, or the classifier times out, the proxy will directly block that egress request rather than degrading to allow it. This sacrifices some availability but prevents agents from executing potentially dangerous operations in an uncertain state—a common choice in heavily regulated scenarios like finance.

Shared Evaluation Framework: Making Performance Comparable

Without a unified evaluation approach, every team would have to build its own custom solution to understand how models perform on their agents. Feminella argues this is as wasteful as requiring every team to invent its own unit testing framework.

Two Sigma's approach is to establish a shared evaluation system, giving every team writing agents a consistent way to compare performance. Individual model performance can be aggregated across teams to understand how models perform in the enterprise's specific context. The shared framework also handles trace collection and pipeline work—authors only need to write regular agent logic and test cases, and evaluations run automatically on every change like a CI/CD pipeline.

The disconnect between public benchmarks and actual enterprise performance is a common pitfall in agent deployment. Public leaderboards (such as various LLM rankings) are typically based on general tasks and static datasets, while enterprise agents face internal tools, proprietary data, and specific business rules. The same model's performance in an enterprise context can differ significantly from its public benchmark ranking. The value of a shared evaluation framework lies in anchoring evaluation to the enterprise's own task distribution and enabling aggregation of use cases across teams, so you can determine whether a model upgrade is a net gain for the entire business line or introduces regressions.

Automatic Mode Classifier: Avoiding Human Approval Fatigue

If every agent operation required human review and approval, it would not only slow down agents but also make people careless due to complacency and alert fatigue. Two Sigma adopted an "auto mode" classifier familiar to agent framework users.

When an agent wants to perform an operation that isn't explicitly prohibited, a two-stage classifier decides how to handle it: first, a fast triage that leans toward blocking, then a deeper review for unclear cases.

Alert fatigue refers to how reviewers gradually reduce their attention when alerts or approval requests come too frequently, tending to mechanically click "approve," rendering security controls ineffective. In agent scenarios, since agents initiate operations at machine speed, requiring human approval for every operation not only causes severe queue backlogs but also breeds complacency among approvers. The two-stage classifier design is precisely meant to mitigate this: fast triage filters out large volumes of obviously safe or obviously dangerous operations, leaving only a small number of borderline cases for more resource-intensive deep review, thus balancing speed and security.

Implications for Enterprises

The core insight of this framework is: agents will run at machine speed regardless of whether their controls can keep up. Successful enterprises need to build control systems that can keep pace, so that no one needs to slow down.

Feminella acknowledges this is a rapidly evolving field, and what's correct today may no longer apply in six months. Two Sigma continuously engineers its platform to stay ahead of internal demand for these capabilities. For enterprises exploring agent deployment, these five dimensions provide a governance framework starting point worth referencing—the key is to adapt based on your own risk exposure and business objectives, not to copy it wholesale.