Testing Conditional Distribution Equality with Generative Models: A New Method for High-Dimensional Settings

Testing conditional distribution equality is a central problem in causal inference, model diagnostics, and transfer learning. Traditional methods often lose testing power when the dimensionality of the conditioning variables is high. A new study proposes leveraging generative models to compare conditional distributions, using both observed data and synthetic samples, and controlling estimation error through sample splitting and studentization, demonstrating superior testing performance in high-dimensional settings.

Background: Why Conditional Distribution Testing Is So Difficult in High Dimensions

The goal of conditional distribution equality testing is to determine whether the conditional distributions P(Y|X) of two groups of data are identical given conditioning variables X. This problem is used in causal inference to test whether intervention effects vary with covariates, in model diagnostics to assess whether a model performs consistently across different subgroups, and in transfer learning to measure conditional shift between source and target domains.

The difficulty lies in the fact that as the dimensionality of the conditioning variables X increases, non-parametric estimation of conditional distributions suffers from the curse of dimensionality. For kernel-based methods—whether conditional two-sample tests or joint two-sample tests—the convergence rate of the test statistic drops sharply as dimensionality increases, making it nearly impossible to detect true differences when the sample size is limited.

The starting point of this research is precisely to bypass the direct non-parametric estimation problem: if a generative model can learn the conditional distribution, then synthetic samples can be used to help construct the test statistic.

Curse of dimensionality is the key background for understanding the motivation of this paper. It refers to the fact that as the covariate dimension d increases, the sample size required for non-parametric estimation grows exponentially: to estimate a function with fixed accuracy in each dimension, the sample size roughly needs to expand exponentially with d. This directly causes the convergence rates of methods such as kernel density estimation and kernel mean embedding to become impractical. For conditional distribution testing, this means that the variance of kernel-based test statistics under the null hypothesis inflates as dimensionality increases, while the signal under the alternative hypothesis relatively weakens, ultimately manifesting as a sharp decline in testing power. This paper's choice to "bypass direct non-parametric estimation" is essentially an acknowledgment that P(Y|X) cannot be stably estimated in high dimensions using traditional approaches, and instead turns to generative models to implicitly learn the conditional distribution, transforming the statistical challenge into a problem of generative modeling and sample splitting—a paradigm shift in test method design under high-dimensional conditions.

Method Core: Generative Models + Sample Splitting + Studentization

The new method has three key design layers.

The first layer is the dual role of the generative model. Researchers use generative models to simultaneously fit the conditional distributions of two groups of data and draw synthetic samples from them. The test statistic depends on both the observed data and these synthetic samples, thereby "borrowing" the expressive power of the generative model in the process of learning the conditional distribution.

The second layer is sample splitting and studentization. To avoid the overfitting of the generative model on the training set affecting the asymptotic properties of the test, the method adopts sample splitting: one portion of the data is used to train the generative model, and another portion is used to compute the test statistic. Studentization is then used to correct estimation error, so that under the null hypothesis the test statistic asymptotically follows a normal distribution.

The third layer is the lenient conditions for theoretical guarantees. Under the null hypothesis and fixed alternatives, the theoretical analysis only requires that the learned conditional distribution be consistent, without requiring the generative model to have a specific parametric form or convergence rate. This means that a broad range of generative models—including diffusion models, conditional GANs, and conditional VAEs—can be incorporated into this framework.

The research also establishes local power against continuous alternatives, an important indicator in testing theory for measuring whether a method can distinguish "weak but real differences."

Sample splitting is a classical technique in statistical inference, with the core purpose of cutting off the overfitting bias that arises when "data are used both to estimate nuisance parameters and to construct the test statistic." If the same batch of data is used directly to train the generative model and compute the test statistic, the fitting error of the generative model on the training set becomes entangled with the test statistic, causing the distribution under the null hypothesis to deviate from the theoretical distribution and thereby breaking asymptotic normality. The approach in this paper is to divide samples into a training subset and an inference subset, so that the distribution of the test statistic can be treated as independent conditional on the training subset, thereby restoring theoretical guarantees. The cost is a reduction in the sample size used to compute the statistic, which is also the reason why the power of this paper's method may be limited in small-sample settings. Paired with sample splitting, studentization refers to dividing the centered bias of the statistic by its estimated standard deviation, so that the standardized quantity asymptotically follows a standard normal distribution under the null hypothesis, making it convenient to set rejection regions and p-values uniformly without recalibrating critical values for different dimensions.

Theoretical Properties: Asymptotic Normality and Local Power

Under the null hypothesis, the test statistic asymptotically follows a standard normal distribution, which provides a direct basis for setting rejection regions and computing p-values. More critically, the research proves that under continuous alternatives, the test has non-trivial local power—that is, when the difference between the two conditional distributions shrinks as sample size increases, the test can still detect this difference with probability higher than random guessing.

The conditions for this theoretical result are quite lenient: it only requires that the conditional distribution learned by the generative model be consistent, without requiring its convergence rate to reach a certain threshold. In contrast, many kernel-based tests require conditional distribution estimation to achieve a specific convergence rate to guarantee testing power, which is often difficult to satisfy in high dimensions.

Local power is a core concept in testing theory for evaluating method sensitivity. Unlike fixed alternatives, local alternatives let the difference between two distributions shrink at a rate of n^{-1/2} as the sample size n increases, examining whether the test can still detect this "increasingly weak" difference with probability above the significance level. If a test's power tends to 1 under local alternatives, it is said to have non-trivial local power, typically indicating that the test is close to optimal in the information-theoretic sense. This paper proves that its method possesses this property under continuous alternatives, and only requires the conditional distribution learned by the generative model to be consistent, without requiring a specific convergence rate. The importance of this lenient condition lies in the fact that many kernel-based tests require conditional distribution estimation to achieve a certain rate (such as n^{-β}) to guarantee local power, but this rate is often defeated by the curse of dimensionality in high dimensions, so the local power of kernel methods may degenerate to random guessing levels in high dimensions.

Experimental Performance: Size Control and Testing Power in High-Dimensional Settings

Empirical results validate the new method from two dimensions. In terms of size control—that is, whether the rejection rate is close to the preset significance level when the null hypothesis is true—the new method exhibits accurate size control, without over-rejection or excessive conservatism.

In terms of testing power, the research compares the new method with representative kernel-based conditional two-sample tests and joint two-sample tests. In high-dimensional settings, the new method demonstrates stronger testing power, while kernel methods show obvious power decay due to the curse of dimensionality.

This comparison result is consistent with the theoretical analysis: the introduction of generative models enables the method to adapt to the complex structure of conditional distributions, without relying on kernel distance metrics that rapidly degrade in high-dimensional spaces.

Applicability Boundaries and Open Questions

Despite the lenient theoretical conditions, the practical performance of the method still depends on whether the generative model can accurately learn the conditional distribution. In scenarios where the conditional distribution itself is difficult to learn—for example, when the relationship between conditioning variables and response variables is extremely complex or data are extremely scarce—the reliability of the test will be affected.

Furthermore, although sample splitting guarantees theoretical properties, it also reduces the sample size used to compute the test statistic, which may weaken testing power in small-sample settings. How to use data more efficiently while maintaining theoretical guarantees is a direction worth exploring.

This method provides a new paradigm for conditional distribution testing: transforming generative models from "data generation tools" into "statistical inference components." As the capabilities of generative models continue to improve, this approach is expected to play a greater role in downstream tasks such as causal inference and model diagnostics.

Consistency refers to the property that as sample size tends to infinity, an estimator converges in probability to the true parameter or true function. In the context of this paper, it requires that the conditional distribution \hat{P}(Y|X) learned by the generative model approaches the true conditional distribution P(Y|X) as sample size increases, but does not specify how fast it approaches. Compared with "convergence rate" conditions, consistency is a weaker asymptotic requirement, which allows the theoretical results of this paper to cover modern generative models such as diffusion models, conditional GANs, and conditional VAEs for which explicit convergence rates are difficult to provide. However, consistency itself is an asymptotic property and does not guarantee performance under finite samples. When data are scarce or the structure of P(Y|X) is extremely complex, the generative model may be far from a practically meaningful approximation, thereby affecting test reliability. In addition, consistency is usually proved under specific assumptions, such as model capacity growing with sample size and optimization algorithms finding global or sufficiently good solutions, which do not necessarily hold in the actual training of deep generative models. Therefore, lenient theory does not mean unconditional practical applicability.

One-line Summary

A new study uses generative models to assist conditional distribution testing, achieving better size control and testing power in high dimensions.

Paragraph Summary

The article introduces a new testing method that uses generative models to compare conditional distributions P(Y|X), aiming to bypass the curse of dimensionality in high-dimensional non-parametric estimation. The core design has three layers: using generative models to simultaneously fit two conditional distributions and draw synthetic samples, so that the test statistic depends on both observed and synthetic data; adopting sample splitting and studentization to control estimation error caused by generative model overfitting, so that under the null hypothesis the statistic asymptotically follows a standard normal distribution; and requiring theoretically only that the learned conditional distribution be consistent, making the method applicable to a broad range of models such as diffusion models, conditional GANs, and conditional VAEs. The research also proves that the method has non-trivial local power against continuous alternatives, capable of detecting weak differences that shrink as sample size increases. Experiments show that in high-dimensional settings, compared with kernel methods, the new method has more accurate size control and stronger testing power, but practical performance still depends on whether the generative model can accurately learn the conditional distribution, and sample splitting may also weaken power in small samples.