Teaching Code Agents to Ask Questions Without Leaking Private Code

Local code agents often turn to more powerful cloud models when they encounter complex problems. But this process frequently exposes private identifiers, file paths, and even source code snippets from the repository. How to protect code privacy while preserving the ability to ask for help has become an urgent problem to solve.

Limitations of Existing Solutions

According to the research paper "Teaching Code Agents to Ask Without Leaking Private Code," there are currently two common approaches: host-side masking and training agents to reduce information disclosure. The former can only hide already-identified identifiers, but the surrounding code and the agent's question phrasing remain unchanged, so attackers may still infer sensitive information from context. The latter may go too far, causing agents to be afraid to ask questions or to ask questions so vague that they cannot get effective help.

Neither approach fundamentally teaches the agent how to express questions safely.

Host-side masking is a privacy protection technique in which sensitive information is identified and replaced on the host side before data leaves the local environment. Common practices include replacing identifiers such as variable names, function names, and file paths with placeholders, or redacting code snippets. The advantage of this method is that it is simple to implement and low-cost, but its limitations are also obvious: it can only handle sensitive entities with predefined rules, and it is powerless against semantic-level information leakage (such as business logic inferred from code structure, comments, or use cases). In addition, masked code may lose readability, making it difficult for cloud models to understand the problem context and thus reducing the effectiveness of consultation.

Training agents to reduce information disclosure is a learning-based approach that uses reinforcement learning or fine-tuning to make agents actively avoid including sensitive information when asking questions. However, the training objective of this method is difficult to control precisely: excessive suppression causes the agent to become "afraid to ask" or to ask overly general questions, thereby losing the value of consulting cloud models; insufficient suppression still leaves leakage risk. Both approaches remain at the level of "hiding" or "suppressing" and do not teach the agent how to express questions in a safe and effective way, so it is difficult to satisfy both privacy and utility requirements simultaneously.

Utility-Verified Abstraction (UVA) Method

The paper proposes a new method called "Utility-Verified Abstraction" (UVA). Its core idea is to have the agent describe private entities by their roles and types while preserving the technical details needed by the expert.

In the implementation, researchers rewrite each problem from solved repair trajectories into this abstract form. A rewrite is retained only when the rewritten problem discloses fewer private entities and the repair process can be verified through repair trajectory replay under the same execution state. This ensures both that the problem does not leak privacy and that it remains effective.

Repair trajectory replay verification is a key verification step in the UVA method. Specifically, researchers extract problem rewrite samples from solved repair trajectories. Each trajectory records a series of operations and state changes executed by the agent from the initial state to repair completion. When verifying whether a rewritten problem is effective, the system simulates rerunning the key steps in the repair trajectory under the same execution state using only the abstracted problem description, and checks whether the same repair result can be achieved. If replay succeeds, it means the abstracted problem retains enough technical detail to guide the expert model in reproducing the repair process; if it fails, it means the abstraction is excessive and has lost necessary information, so the rewrite is discarded. This mechanism ensures that privacy protection does not come at the expense of problem utility, achieving dual verification of privacy and utility.

Experimental Results: More Consultation and Higher Resolution Rate

In two benchmarks, the trained agent initiated consultation more frequently than before training. On repository-level issue resolution tasks, it solved 37.6% of problems, compared with a 25.3% resolution rate when the same model worked alone. This improvement came from training as a whole, not merely from the consultation behavior itself.

More importantly, the privacy protection effect: compared with questions asked by an untrained agent to the same expert, questions after training disclosed 3.7 times fewer private entities and 1.4 times less verbatim code.

Repository-level issue resolution is a type of benchmark for evaluating code agent capabilities. It requires the agent to understand the problem, locate defects, and generate repair patches in the context of a complete code repository. Compared with function-level or file-level tasks, repository-level tasks involve complex factors such as cross-file dependencies, project structure understanding, and build configuration, placing higher demands on the agent's context awareness and reasoning ability. In the experiment, the agent's resolution rate on repository-level tasks increased from 25.3% to 37.6%. This improvement is relatively significant among similar studies, indicating that UVA training not only improves privacy protection but also genuinely enhances the agent's problem-solving ability in real development scenarios.

Significance and Outlook

The UVA method provides a new direction for privacy protection in code agents. It does not rely on simple masking or suppressing questions, but instead trains the agent to learn a safe way of expression. This both improves the agent's practicality and reduces the risk of privacy leakage. In the future, this method may be extended to more types of sensitive information protection scenarios.

One-line Summary

UVA trains code agents to ask questions safely, reducing privacy leakage by 3.7x while raising the repair rate to 37.6%.

Paragraph Summary

This article introduces a new method called "Utility-Verified Abstraction" (UVA), aimed at solving the problem of private code leakage when local code agents consult cloud models. Existing solutions such as host-side masking and suppression training either cannot prevent contextual inference or cause agents to be afraid to ask questions or to ask vague ones. The core of UVA is to have the agent describe private entities by their roles and types while preserving technical details, and to verify through repair trajectory replay whether the rewritten question remains effective. Experiments show that agents trained with UVA consult more frequently, raise the resolution rate on repository-level issue resolution tasks from 25.3% to 37.6%, and disclose 3.7 times fewer private entities and 1.4 times less verbatim code. This method provides a new direction for privacy protection in code agents and can be extended to more sensitive information protection scenarios in the future.