Non-Reversible Anchored Langevin Dynamics: Accelerating Sampling for Non-Smooth and Heavy-Tailed Distributions

Background: When Sampling Meets Non-Smooth and Heavy-Tailed Distributions

Efficient sampling from complex probability distributions has long been a central challenge in Bayesian inference, statistical physics, and machine learning. Traditional Langevin dynamics relies on gradient information from the target distribution's density function, which works well when the density is smooth and differentiable. But real-world problems are rarely so cooperative — the target density may be non-differentiable, heavy-tailed, or constrained to a bounded domain.

Anchored Langevin dynamics (ALD) was designed precisely for such non-smooth sampling scenarios, capable of handling target distributions that are both non-differentiable and heavy-tailed. Reflected Anchored Langevin dynamics (RALD) further extends this capability to sampling problems on constrained domains. The new methods proposed in this paper build critically upon both of these foundations.

Core Contributions: Introducing Non-Reversibility

The central innovation of this paper lies in two new algorithms:

Non-Reversible Anchored Langevin Dynamics (NALD)

NALD targets sampling from distributions in Euclidean space that may be non-differentiable and heavy-tailed. Its key design feature is adding a circulation drift term to the original dynamics — a drift generated by a potentially state-dependent, divergence-free skew-symmetric matrix field together with a stream potential.

Non-Reversible Reflected Anchored Langevin Dynamics (NRALD)

NRALD addresses sampling from non-differentiable target distributions on constrained spaces. It employs a normal reflection mechanism at the boundary, enabling the algorithm to operate effectively within bounded domains.

The most attractive property of both methods is that they preserve the target distribution without requiring any derivatives of the target density. This is especially important for non-smooth distributions, where derivatives may not even exist.

Technical Highlights: Breaking Reversibility for Faster Convergence

Most traditional sampling methods satisfy reversibility — that is, the detailed balance condition. The theoretical contribution of this paper lies in proving that by breaking reversibility, NALD and NRALD converge to the target distribution faster than their reversible counterparts.

The authors provide rigorous theoretical support from three angles:

• Finite-time non-asymptotic convergence analysis: quantifies how close the algorithm gets to the target distribution within a finite number of steps;
• Large deviations analysis: characterizes the exponential behavior of convergence rates;
• Asymptotic variance reduction: demonstrates that the non-reversible design lowers the asymptotic variance of estimators.

Furthermore, the proposed dynamics admit a random-time-change representation, which provides a flexible mathematical framework for both theoretical analysis and practical implementation. The approach applies to all of Euclidean space as well as to bounded domains with normal reflection.

Why Does Non-Reversibility Accelerate Sampling?

This is a particularly illuminating topic in sampling theory. While reversible Markov processes are simple to design and straightforward to verify for correctness, their convergence speed is often bottlenecked by the spectral gap. Introducing a non-reversible circulation term effectively injects a "rotational" component into the sampling trajectory, enabling the sampler to explore the state space more directionally and avoiding the slow, diffusive back-and-forth characteristic of reversible processes.

This paper cleverly applies this idea within a gradient-free anchored framework — a regime that prior work on non-reversible acceleration has rarely ventured into. Classical non-reversible acceleration typically relies on manipulating the gradient of a potential function, but in non-smooth and heavy-tailed settings, gradients may not exist or may be numerically unstable. NALD sidesteps this limitation through the construction of a divergence-free skew-symmetric matrix field.

Experimental Validation and Application Prospects

The paper validates the proposed algorithms through numerical experiments. Results demonstrate that the non-reversible variants exhibit faster sampling efficiency than their reversible counterparts when handling non-smooth and heavy-tailed target distributions.

From an applications perspective, these methods hold potential value in the following scenarios:

• Bayesian statistics: posterior sampling with sparse priors (e.g., Laplace priors that render the density non-differentiable);
• Constrained optimization and constrained sampling, such as settings where parameters must satisfy boundary conditions;
• Heavy-tailed noise modeling, commonly encountered in finance, risk analysis, and related fields;
• Scientific computing tasks where Monte Carlo methods demand high-precision, low-variance estimates.

Summary

This work makes substantial advances at both the theoretical and algorithmic levels of non-smooth sampling. It successfully transplants the classical idea of "non-reversible acceleration" into the gradient-free, anchored Langevin framework capable of handling heavy-tailed and constrained distributions, backed by comprehensive analyses of convergence, large deviations, and variance reduction. For researchers working on sampling algorithms, Markov chain Monte Carlo (MCMC), and related theory, this paper offers a compelling new tool worthy of close study.