Dummy-Point Differential Privacy: A New Privacy Amplification Mechanism Under Observable Batch Sizes

The privacy guarantees of Differentially Private Stochastic Gradient Descent (DP-SGD) typically rely on the privacy amplification effect brought by random sampling. But this effect has an often-overlooked precondition: the batch size must not be leaked. In reality, servers or observers of communication traffic can often see the batch size, and memory and computation constraints require truncating oversized batches—truncation itself significantly increases privacy loss. A recent study proposes using a random number of "dummy points" to solve both problems simultaneously, and provides an efficiently computable privacy analysis for distributed settings.

Problem: Observable Batch Sizes Erode Privacy Guarantees

In typical DP-SGD deployments, the server processes data in batches. Mainstream approaches such as Poisson sampling and random allocation both provide provable privacy amplification, but they share a consequence: the batch size is variable. The privacy guarantees of these schemes rest on an assumption—that the batch size is not publicly revealed.

This assumption is fragile in practice. In distributed settings, the server performing aggregation, or any party observing communication traffic, may learn the actual size of each batch. Even in centralized settings, memory and computation constraints usually require that a single batch not exceed some upper limit s, and must be truncated when exceeded. Prior work has shown that such truncation leads to a significant increase in privacy loss.

In other words, the gains from privacy amplification may be partially offset by observable batch sizes and truncation operations. The question the research aims to answer is: can the benefits of privacy amplification be preserved under conditions where batch sizes are publicly visible and truncation limits exist?

Privacy Amplification is one of the core mechanisms behind DP-SGD's privacy guarantees. The basic idea is: if each iteration randomly selects only a small subset of data for computation, then even if a single sample leaks information in a single iteration, the attacker has difficulty determining whether that sample was ever involved in training because the probability of it not being selected is high—thereby reducing privacy loss overall. Common sources of privacy amplification include Poisson subsampling (each sample is independently selected with a fixed probability) and random allocation (samples are randomly assigned to different batches). However, the mathematical derivation of this amplification effect typically assumes that the attacker can only observe the algorithm's output (such as gradient updates) and cannot observe intermediate quantities of the sampling process—batch size being one of them. Once the batch size is observable, the attacker gains additional information about which samples were selected, thereby weakening or even destroying the original privacy amplification guarantee.

Mechanism: Using Dummy Points to "Mask" the True Batch Size

The paper's approach is direct: in addition to real data points, inject a random number of dummy points into the batch, so that the batch size seen by external observers no longer directly corresponds to the number of real participating samples. The size information of the real batch is thus randomized, and the batch size release itself is "privatized."

The key advantage of this mechanism lies in its simplicity of implementation. In distributed settings, each participant only needs to generate dummy points as agreed and participate in aggregation alongside real data, without requiring additional secure channels or complex protocol modifications.

The analysis covers two sampling schemes: Poisson sampling and random allocation. The authors provide a unified general analytical framework for both, and particularly emphasize that the analysis supports efficient and relatively tight privacy accounting—which is important for scenarios requiring repeated computation of privacy budgets in practical deployments.

Dummy Points are a commonly used technique in differential privacy, referring to synthetic records injected into a dataset that carry no real information. In DP-SGD, dummy points participate in forward and backward computation, but their gradient contributions are discarded or canceled out, so they do not affect model updates. The role of dummy points is similar to "padding": by randomly varying the number of dummy points in a batch, the observed total batch size loses its deterministic correspondence with the number of real samples, thereby "privatizing" the batch size release process. From a privacy analysis perspective, the injection of dummy points is equivalent to applying a randomization to the real batch, which introduces additional randomness into count releases and can be modeled as a subsampling mechanism, thereby bringing privacy amplification. The cost of dummy points is additional computation overhead, since each dummy point requires a full forward and backward pass, so their number needs to be carefully controlled.

Privacy Accounting: Additional Amplification from Subsampling

The mechanism has another less intuitive effect: it creates a subsampling-style privacy amplification for the count releases themselves. Since dummy points are randomly injected, the relationship between the observed count and the true count carries randomness, and this randomness provides additional privacy protection for count publication.

The direct consequence of this amplification is that the expected number of dummy points required can be asymptotically smaller than what is needed to "directly privatize the batch size using non-negative noise (sensitivity 1)." In other words, protecting the batch size becomes "cheaper" than the naive approach thanks to the subsampling effect, and training efficiency is not dragged down by a large number of dummy points.

From an engineering perspective, this determines whether the mechanism is usable: dummy points participate in forward and backward computation, and their number directly corresponds to additional compute overhead. The smaller the expected number, the closer the mechanism is to zero cost.

Empirical Results: Better Than Direct Accounting Under Truncation

The paper conducts an empirical evaluation for Poisson sampling with truncated batches, comparing against the truncation direct accounting method recently proposed by Ganesh (2025). The results show that the mechanism performs better in the privacy-efficiency trade-off.

As a note, the comparison baseline is the latest method specifically targeting the truncation problem, rather than naive privacy accounting, so this advantage is more informative. However, the paper's empirical scope is concentrated on the combination of Poisson sampling plus truncation, and the empirical performance of the random allocation scheme still needs more data support.

Ganesh (2025)'s truncation direct accounting method is a privacy accounting scheme proposed for the batch truncation problem in DP-SGD. In standard DP-SGD, when the batch size exceeds a preset upper limit s, truncation (i.e., randomly discarding some samples) is necessary to ensure computational feasibility. Ganesh's method directly accounts for privacy loss on the truncated batch, taking into account the additional privacy leakage caused by the truncation operation. However, this method does not address the issue of the batch size itself being publicly observed, and its privacy guarantees still rest on the assumption that the batch size is not visible. The dummy-point mechanism proposed in this paper simultaneously addresses both batch size leakage and truncation loss, so in the Poisson sampling with truncation scenario, compared with Ganesh's method, it can achieve smaller dummy-point overhead or a better privacy-efficiency trade-off under the same privacy budget. This comparison baseline is well-targeted, highlighting the advantage of the dummy-point mechanism under realistic threat models.

Practical Significance and Boundaries

For teams building DP-SGD systems, this work points out an easily overlooked threat model: the batch size itself is a privacy attack surface. When system logs, network packet sizes, or the internal state of the aggregation service expose batch scale, the originally computed privacy budget may be overly optimistic. The dummy-point mechanism provides a low-intrusion fix, especially suitable for distributed training environments where modifying communication protocols is difficult.

The boundaries are equally clear: the mechanism addresses privacy loss caused by batch size leakage and truncation, and does not change the noise calibration logic of DP-SGD itself; although the number of dummy points is reduced through subsampling amplification, it still varies with privacy parameters and batch upper limits, so deployment requires a concrete trade-off between privacy budget and computation overhead. In addition, the paper's analysis targets Poisson sampling and random allocation, and the adaptability of other sampling schemes is outside the scope of discussion.

For researchers focused on the engineering deployment of privacy amplification, the value of this work lies in turning "batch size visibility" from a default-ignored assumption into a design variable that can be explicitly handled.

One-line Summary

Injecting random dummy points can privatize DP-SGD batch sizes, preserving privacy amplification and reducing overhead in truncation scenarios.

Paragraph Summary

DP-SGD's privacy amplification typically assumes batch sizes are not leaked, but distributed servers or traffic observers may learn batch sizes, and truncation due to memory constraints significantly increases privacy loss. A recent study proposes injecting a random number of dummy points into batches, decoupling the externally observed batch size from the true sample count and thereby privatizing batch size release. The mechanism is simple to implement, applies to Poisson sampling and random allocation, and leverages the subsampling effect to amplify the privacy of count releases, making the expected number of required dummy points asymptotically smaller than directly privatizing the batch size. Under Poisson sampling with truncation, empirical results show its privacy-efficiency trade-off outperforms Ganesh (2025)'s truncation direct accounting method. This work turns batch size visibility from an ignored assumption into an explicitly handleable design variable.