From Tokens to Files: A New Framework for Multi-Granularity Detection of AI Involvement in Code

Large language models generating code has become commonplace, but distinguishing purely human-written code from code with AI involvement—that is, any part of a file, however small, that was generated or substantially modified by a model—remains a challenge. What makes it trickier is that we don't just need to flag such files; we also need to pinpoint exactly which lines or code blocks were contributed by AI. Existing methods tend to compress an entire file into a single score, relying on surface-level stylistic features or global code representations. When only a few code blocks involve AI, this averaging dilutes the evidence. A recent paper proposes a new framework that models AI authorship at the token level and can make predictions at three levels—line, block, and file—offering a more fine-grained perspective on code provenance.

Limitations of Existing Methods: Why Global Scores Fall Short

Traditional AI code detection methods typically take an entire file as input and output a binary result or probability score. This "file-level" judgment may work when AI involvement is high, but when only a few lines of a file are AI-generated (for example, via completion or fill-in-the-middle), global features get overwhelmed by the large amount of human code. The paper points out that compressing to a single score "averages out" crucial evidence, leading to missed detections. Moreover, methods based on surface stylistic features (such as variable naming or comment style) are easily disrupted by code formatters or developer habits, making them insufficiently robust.

Another problem is that many detectors are trained on files that are entirely AI-generated or entirely human-written, lacking samples of mixed authorship, which leads to poor performance in real-world collaborative editing. The authors argue that a method capable of capturing local AI traces is needed, rather than relying on global summaries.

Core Idea: Token-Level Evidence and Multi-Granularity Aggregation

The core of the framework is "modeling AI authorship at the token level." Specifically, the researchers use a frozen code language model and equip it with two low-rank adapters (LoRA), trained separately on human-written files and machine-written files. For each token, they compute the log-likelihood ratio under the two adapters; this ratio provides local evidence of whether that position was generated or modified by AI.

Once token-level signals are obtained, a lightweight prediction head aggregates this evidence at the line and block levels and uses surrounding code context for correction. Finally, local predictions are aggregated into a file-level decision. This design allows the model to output results at multiple granularities: whether a file involves AI, which lines involve AI, and which code blocks involve AI. The paper emphasizes that this hierarchical aggregation avoids the pitfalls of global averaging and is especially suited to mixed-authorship files.

For training data, the researchers constructed mixed-authorship files: starting from original human code, they generated corresponding AI-generated or AI-modified versions, covering scenarios such as continuation and fill-in-the-middle. This data construction approach more closely reflects the form of AI-assisted programming in real development.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique that freezes most of a pretrained model's weights and injects only a small number of trainable low-rank matrices into each layer. Specifically, it decomposes the weight update into the product of two small matrices, reducing the number of trainable parameters to roughly one-thousandth of the original. In this framework, two LoRA adapters are trained on human-written code and machine-written code respectively, sharing the same frozen base model, so two sets of "author style" parameterized representations can be obtained at low cost. During inference, the log-likelihood under each adapter is computed for every token, and their difference constitutes the local evidence for that position. Compared with training two independent models, the dual-LoRA scheme uses less GPU memory, switches flexibly, and facilitates controlled comparisons on the same base model.

Log-likelihood ratio is a classic quantity in statistical hypothesis testing, equal to "the log of the token's probability under the AI model" minus "the log of the token's probability under the human model." A positive value indicates the token fits the AI-generated distribution better, while a negative value indicates it fits the human distribution better. The advantage of using a ratio rather than absolute probability is that it cancels out the baseline probability of high-frequency tokens in code (such as brackets and keywords), thereby highlighting the positions that are genuinely discriminative. The likelihood ratios of multiple tokens can be accumulated into line-level or block-level scores, which is also the mathematical basis for the subsequent lightweight prediction head's multi-granularity aggregation.

Experimental Performance: Internal Tests and External Benchmarks

On the internal test set, the method was evaluated on 11,773 AI-involved files (covering six generation modes) and 2,541 human files, achieving a macro-F1 of 72.4 at the 1% false positive rate (FPR) operating point. This result shows that even under strict control of false positives, the model maintains strong overall detection capability.

More noteworthy is cross-dataset generalization. On the HybridCodeAuthorship external human-machine collaboration benchmark (which has no training set), the method achieves a line-level F1 of 62.5 without any adaptation, surpassing the best line-level result of 56 reported in the original paper. This indicates that token-level evidence has strong transferability and does not depend on training on a specific dataset.

False positive control on real human code is equally critical. The researchers selected 19,445 human-written files from repositories created before 2022, and at the transferred 1% threshold, the false positive rate remained between 0.55% and 0.81%. This result shows that the operating point can be stably maintained on real human code and will not fluctuate dramatically due to domain shift.

Macro-F1 is computed by first calculating the F1 score for each class (here, "AI-involved" and "human-written") and then taking the arithmetic mean, so it is insensitive to class imbalance and will not be dominated by the majority class. The paper reports a macro-F1 of 72.4 at the 1% FPR operating point, meaning that while keeping the proportion of human code misclassified as AI at 1%, the combined harmonic mean of the two-class discrimination still reaches 72.4 (on a 100-point scale). False positive rate (FPR) refers to the proportion of human-written code judged as AI-involved; in code provenance scenarios, it directly relates to the risk of falsely accusing developers, so the industry commonly adopts an evaluation approach of fixing a low FPR and then looking at recall or F1. The external benchmark HybridCodeAuthorship has no training set and is a zero-shot transfer evaluation; its line-level F1 of 62.5 indicates that the token-level signals learn relatively general author traces rather than overfitting to a specific dataset.

The Significance and Challenges of Multi-Granularity Detection

The value of multi-granularity detection lies in the fact that it can answer not only "does this file involve AI" but also "where does it involve AI." For scenarios such as code review, academic integrity, and open-source license compliance, locating specific lines or blocks is more useful than file-level flags. For example, in code review, reviewers can focus on the AI-involved parts to improve efficiency; in academic assignments, teachers can more precisely judge whether students over-relied on AI.

However, the method also has limitations. First, it relies on a frozen code language model and two adapters, making training and inference more costly than a simple classifier. Second, token-level likelihood ratio signals may be affected by code context and biases in the model itself, and may perform worse for certain programming languages or domains (such as low-resource languages). In addition, the paper's evaluation is mainly conducted on mainstream languages such as Python, and cross-language generalization still needs verification.

Another challenge is distinguishing AI modification from AI generation. The paper defines "AI involvement" as any part generated or substantially modified by a model, but in practice AI may only perform minor refactoring (such as renaming variables), and the traces of such modifications are weaker and harder to detect. The paper's mixed-authorship data construction covers continuation and fill-in-the-middle, but does not explicitly discuss minor-modification scenarios.

Future Directions and Industry Impact

This research offers a new approach to AI code provenance: moving from global scores to local evidence, and from a single granularity to multi-granularity aggregation. With the spread of AI programming assistants (such as GitHub Copilot and Cursor), transparency of code authorship will attract increasing attention. Multi-granularity detection frameworks are expected to be integrated into code hosting platforms or CI/CD pipelines to automatically flag AI-involved parts and assist developers, reviewers, and compliance teams.

Future work may include: extending to more programming languages and domains; reducing computational overhead to make it suitable for large-scale codebases; combining other signals (such as edit history and commit patterns) to improve robustness; and exploring adversarial scenarios, where developers deliberately obfuscate AI-generated traces.

The paper and code have not yet been made public, but the proposal of this framework undoubtedly advances AI code detection from "file-level binary classification" toward "fine-grained localization." For researchers and engineers concerned with the governance of AI programming tools, this direction is worth continued attention.

One-line Summary

Dual LoRA computes likelihood ratios at the token level, enabling multi-granularity AI code provenance at the line, block, and file levels.

Paragraph Summary

The article introduces a code provenance framework that models AI authorship at the token level. Traditional methods compress an entire file into a single score, so evidence is diluted when only a few code blocks are AI-generated, and they rely on surface stylistic features that are easily disrupted by formatting. The new framework freezes a code language model and equips it with two LoRA adapters trained on human-written and machine-written data respectively, computing a log-likelihood ratio for each token as local evidence; a lightweight prediction head then aggregates this at the line, block, and file levels and uses context for correction. Training data is constructed via continuation and fill-in-the-middle to create mixed-authorship files. Internal tests achieve a macro-F1 of 72.4 at a 1% false positive rate; the external HybridCodeAuthorship benchmark reaches a zero-shot line-level F1 of 62.5, surpassing the previous best; and on 19,445 real human files the false positive rate remains stable at 0.55%-0.81%. The article also discusses limitations such as computational overhead, cross-language generalization, and detection of minor refactoring.