Tracing LoRA Fine-Tuning Signatures: Black-Box Detection of Diffusion Model Variants

Diffusion models have revolutionized the generation of realistic artificial images, and efficient fine-tuning techniques such as LoRA (Low-Rank Adaptation) make it possible to adapt these models to specific use cases, making them more diverse and widely available. However, open foundation models can also be misused in online services to generate biased, harmful, or illegal images. To address this issue, a recent paper proposes a protocol for detecting whether a service is using a LoRA variant of a standard model. Unlike prior work that identifies models within a known candidate list, this study focuses on real-world open-world scenarios—where the set of possible variants and alternative models is entirely unknown.

Background: LoRA Misuse and the Need for Provenance

LoRA efficiently fine-tunes pre-trained diffusion models through low-rank decomposition, greatly reducing adaptation costs and making it easy for individuals or organizations to customize image generation services. But this also introduces risks: malicious operators may fine-tune variants that violate content policies based on open-source foundation models (such as Stable Diffusion) and deploy them as online services. Because fine-tuned models behave similarly to the base model, it is difficult to determine solely by observing outputs whether a service uses a LoRA variant of a known model. Existing model fingerprinting or watermarking methods typically assume a known candidate model set and cannot handle the challenges of unknown variants and unknown model families in the open world.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique whose core idea is not to modify the original weights of the pre-trained model, but to inject trainable low-rank decomposition matrices into the Transformer's attention layers. Specifically, for a weight matrix $W \in \mathbb{R}^{d \times k}$, LoRA represents its update as $\Delta W = BA$, where $B \in \mathbb{R}^{d \times r}$, $A \in \mathbb{R}^{r \times k}$, and the rank $r \ll \min(d,k)$. During training, only $A$ and $B$ are updated while the original model parameters are frozen, so the number of trainable parameters often drops to less than one-thousandth of the original model. This makes it possible to complete stylistic customization of large models such as Stable Diffusion on a single consumer-grade GPU. Precisely because the barrier is so low, the risk of LoRA variants being misused to build non-compliant image generation services rises significantly.

Threat Model and Two Provenance Problems

The paper clarifies the precise setup and threat model of the provenance problem: given a black-box image generation service, determine whether it is backed by a standard foundation model, a LoRA adaptation of that model, or an entirely different model family. Because there are multiple model families and multiple adaptation versions, the authors define two distinct problems: first, distinguishing the base model from its LoRA variants (within the same family), and second, distinguishing that family from other unknown families. This dual challenge requires the protocol to generalize and not rely on a predefined candidate list.

Black-box model provenance refers to inferring a model's identity or origin solely through input-output behavior, without access to model weights, architecture, or training data. Unlike white-box fingerprinting methods, the black-box setting can only send queries (such as text prompts) to the service and collect returned images, so it is necessary to design statistical features invisible to the model's internals as "signatures." The open-world setting further increases the difficulty: the candidate set is unknown, and the signature must have generalization discriminative power for unseen variants and model families, rather than performing classification on a closed set.

Black-Box Signatures Based on Hypothesis Testing

The research adopts a hypothesis testing approach to build the protocol. The core innovation is designing a black-box signature that depends only on the intrinsic dimension of the set of images generated by the model. Intrinsic dimension reflects the effective degrees of freedom of image data during the generation process, and different models (base models, LoRA variants, other families) differ in the intrinsic dimension distribution of their generated images. Through statistical testing, the protocol can determine with high confidence whether a service uses a LoRA adaptation of the target base model. This method requires no access to model weights or internal structure—only querying the service and analyzing output images—which fits real-world black-box scenarios.

Intrinsic dimension originally refers to the dimension of the low-dimensional manifold actually occupied by the distribution of high-dimensional data, and is commonly used to measure the effective degrees of freedom of a representation. In generated image analysis, one can estimate local intrinsic dimensionality in pixel or feature space (e.g., MLE estimation based on nearest-neighbor distances) to obtain a scalar for each image. Due to differences in training objectives, fine-tuning data, and architecture, different models exhibit distinguishable statistical properties in the intrinsic dimension distribution of their generated images. The paper uses exactly this distribution difference as a signature and applies hypothesis testing (e.g., two-sample tests) to decide, at a given significance level, which model category backs the service—thereby turning the provenance problem into a statistical inference problem.

Experiments and Significance

The paper experimentally validates the effectiveness of the protocol in open-world scenarios, being able to distinguish base models, LoRA variants, and different model families. The significance of this work lies in providing a new approach for tracing AI-generated content, especially targeting the misuse risks brought by LoRA fine-tuning. In the future, this technology can assist platform regulation and content moderation by helping identify the model source of non-compliant services. However, the paper also notes that the protocol relies on the stability of image intrinsic dimension and may fail for services that apply post-processing or adversarial attacks—an area that still requires continued exploration in open-world provenance.