How Do You Build Legal AI Agents When You Can't See the Data? Legora CTO Explains

In the legal industry, data privacy is not just a compliance requirement—it's the foundation of professional practice. As AI agents begin to take part in legal work, a sharp contradiction emerges: agents need data to learn and optimize, but law firm clients' data is highly sensitive—even the AI company itself cannot peek at it. Legora CTO Jacob Lauritzen said plainly in his talk: "Legora cannot read any trace of what the agent does for lawyers, because the data belongs to the law firm's clients." This statement captures the core challenge of building legal AI agents—how do you develop, test, and ensure the reliability of an agent when you cannot see the data at all?

The "Black Box" Dilemma of Legal AI: Data Invisibility

Traditional AI model iteration relies on observing training data and inference logs, but in legal scenarios this path is blocked. Legora's agents handle case analysis, document review, and other tasks for lawyers, and all data generated throughout the process belongs to the law firm's clients. As the technology provider, Legora cannot directly access this content. This means that every "thought" and "action" of the agent is invisible to the developers.

This invisibility is not a technical flaw but a consequence of the legal industry's privacy barriers. Lawyers owe strict confidentiality obligations to their clients, and no third-party AI service provider can touch the raw data. Therefore, Legora must ensure the agent behaves as expected without observing the specific content. Jacob Lauritzen treats this constraint as a design premise rather than an obstacle—it forces the team to rethink how AI agents are built.

Building Within Invisibility: Legora's Engineering Strategy

If you can't look at the data, what can you look at? Legora's answer turns to metadata, user feedback, and synthetic testing. According to Jacob Lauritzen, the team designed a highly structured framework for agent interactions so that every step the agent takes generates auditable abstract signals rather than raw content. For example, whether the agent invoked the correct tool, whether it returned results within the expected time, and whether the lawyer modified the output—these behavioral patterns can be captured and analyzed without exposing data.

Another key strategy is building synthetic datasets that simulate legal scenarios. Legora uses public legal documents and internally constructed cases to test the agent's reasoning ability and ensure stable performance on real data. Although synthetic data cannot fully replace real scenarios, it provides a safety net that lets the team catch obvious logical flaws before deployment. Jacob Lauritzen emphasizes that this "data-invisible" testing approach has become central to Legora's engineering culture, requiring developers to rely more on contract-based design and observability metrics rather than directly debugging data.

Design by Contract is a software engineering approach whose core idea is that components agree on their respective responsibilities and obligations through explicit "contracts," including preconditions (conditions the caller must satisfy), postconditions (results the callee must guarantee), and invariants (properties that always hold). In Legora's case, because developers cannot observe the specific data the agent processes, the interaction interfaces between the agent and tools, and between the agent and lawyers, must be strictly defined—under what conditions the agent may call which tool, what structure of result must be returned after the call, and which behaviors count as violations are all written into code and test cases in a formal or semi-formal way. In this way, even without seeing the data, developers can judge the correctness of the agent's behavior by verifying "whether the contract is honored," thereby maintaining system reliability under privacy constraints.

Observability Metrics originate from the field of distributed systems operations and refer to the ability to infer a system's internal state from its external outputs (such as logs, metrics, and traces). In traditional AI development, developers are used to directly examining inference logs and input-output pairs to debug models; but under the privacy constraints of legal AI, raw content is invisible, so observability must be built on abstract signals. What Legora captures—"whether the agent invoked the correct tool," "whether it returned results within the expected time," and "whether the lawyer modified the output"—are exactly typical observability metrics: they contain no sensitive content yet reflect the agent's behavioral patterns and health. By continuously monitoring these metrics, the team can detect anomalies, locate problems, and evaluate iteration results without touching client data—this is also the technical cornerstone that makes a privacy-first architecture work.

Design Lessons from a Privacy-First Architecture

Legora's practice offers a model for all AI applications that handle sensitive data. In healthcare, finance, and other domains, the same constraint of data invisibility exists. Jacob Lauritzen's talk reveals a trend: future AI agent development will increasingly rely on indirect signals and privacy-enhancing technologies rather than centralized observation of raw data.

The challenge this architecture brings is higher testing costs and slower iteration, but what it buys is client trust and compliance safety. For legal tech companies, being able to prove "we cannot see your data" is itself a competitive advantage. As a fast-growing company, Legora's technical choices show that privacy protection and AI capability can go hand in hand—the key is to turn constraints into design principles.

Privacy-Enhancing Technologies (PETs) are a set of technologies designed to enable data processing and value extraction without exposing raw data. Common methods include differential privacy, federated learning, secure multi-party computation, homomorphic encryption, and trusted execution environments. Although Legora's practice mainly relies on metadata and synthetic data, its approach is in line with PETs: decoupling "data visibility" from "data usability" so that developers optimize the system only through abstract signals and controlled interfaces. For industries under similarly strict regulation, such as healthcare and finance, PETs offer a feasible path to implement compliance requirements at the technical level—for example, training models without centralizing raw medical records, or completing cross-institutional risk calculations in an encrypted state. Understanding this technology spectrum helps clarify the position and evolution of legal AI privacy architectures within the broader field of sensitive data.

Key Takeaways

• When Legora's AI agents handle legal tasks, the data belongs entirely to the law firm's clients, and the company itself cannot read any trace of the operations.
• Faced with data invisibility, Legora builds and evaluates agents through metadata monitoring, user feedback, and synthetic data testing.
• This privacy-first engineering approach requires greater reliance on contract-based design and observability metrics rather than directly debugging data.
• This model offers a reference path for AI agent development in sensitive data domains such as healthcare and finance.

One-line Summary

Without being able to read client data, Legora builds reliable legal AI agents using metadata, feedback, and synthetic data.