Causes and Solutions for Periodic Severe Fluctuations in Dice Evaluation Metrics During Training

Problem Background: Abnormal Evaluation Metrics in U-Net Segmentation Tasks

In the training practice of semantic segmentation tasks, a common yet perplexing phenomenon is that the model's evaluation metrics exhibit periodic severe fluctuations. A Reddit user shared their specific case — using a pre-trained U-Net for semantic segmentation with approximately 3 million parameters, a training set of 550 images, and a validation set of 150 images.

U-Net was originally proposed by Ronneberger et al. in 2015 for biomedical image segmentation. Its core innovation lies in the symmetric encoder-decoder structure and skip connections. The encoder extracts multi-scale features through progressive downsampling, the decoder restores spatial resolution through upsampling, and the skip connections directly pass high-resolution features from the encoder to the decoder, helping recover fine boundary information. A "pre-trained U-Net" typically means the encoder uses a backbone network (such as ResNet or EfficientNet) pre-trained on large-scale datasets like ImageNet, while only the decoder is trained from scratch. This can significantly accelerate convergence and improve generalization on small datasets. A model with 3 million parameters is considered lightweight, suitable for small to medium-scale datasets.

The user employed a combination of Cross-Entropy Loss and Dice Loss as the loss function, with Dice score as the evaluation metric. The key observation was: When using Cross-Entropy Loss alone, the model converged well; but once Dice Loss was introduced, the Dice score evaluation metric exhibited severe fluctuations.

This phenomenon involves several important mechanisms in deep learning training. Understanding them not only solves the current problem but also helps us better design training workflows for segmentation models.

Why Dice Loss Is Inherently Unstable

Gradient Characteristics of Dice Loss

Dice Loss has fundamentally different mathematical properties from Cross-Entropy Loss. Cross-Entropy Loss originates from information theory and measures the difference between the predicted probability distribution and the true distribution. In semantic segmentation, it computes the loss value independently for each pixel and then takes the average. This means each pixel contributes gradients that are mutually independent and bounded in magnitude, resulting in smooth and predictable overall gradient behavior. However, this pixel-wise independence also means it doesn't directly optimize region-level overlap — even if most pixels are correctly classified, Cross-Entropy Loss cannot capture the structured error pattern when a small number of boundary pixels are predicted incorrectly.

In stark contrast, Dice Loss is computed based on the overlap between the entire predicted region and the ground truth region. It is essentially a region-level metric.

The Dice coefficient (also known as the continuous version of the F1 Score) originates from the Sørensen-Dice similarity coefficient in statistics, initially used in ecology to compare the similarity between two sample sets. Its formula is:

Dice = 2 * |X ∩ Y| / (|X| + |Y|)

In deep learning, Dice Loss approximates set overlap computation in a differentiable way by treating the prediction probability map as a soft mask. Unlike Cross-Entropy, the gradient of each pixel in Dice Loss depends not only on that pixel's own predicted value but also on the sum of all other pixel predictions (since it appears in the denominator). This global coupling property means that a small change in a single pixel's prediction can propagate through the denominator, affecting the gradient magnitude of all pixels.

When the number of predicted foreground pixels is very small or nearly zero, the denominator approaches zero, causing both the loss value and gradients to oscillate severely. Especially during early training or in scenarios with severe class imbalance, a slight change in the model's predictions for a particular batch can trigger large jumps in the Dice score.

Amplification Effect of Class Imbalance

In semantic segmentation tasks, the pixel ratio between foreground (target regions) and background is often severely imbalanced. While Dice Loss is commonly used to mitigate class imbalance, it also makes predictions extremely sensitive to small targets or sparse foreground. When the foreground region accounts for a very small proportion (e.g., 1%-5%), the values of |X| and |Y| in the denominator are small, and any prediction fluctuation gets amplified into drastic gradient changes. When a validation batch happens to contain more difficult-to-segment samples, the Dice score drops noticeably, creating fluctuations.

Root Cause Analysis of Periodic Fluctuations

Data Ordering and Batch Sampling

The user specifically mentioned that the fluctuations were periodic, which is a key clue. Periodic fluctuations are often related to how training data is traversed. If the data order is fixed within each epoch, or certain difficult samples in the validation set always appear at fixed positions, this leads to regular metric oscillations across epochs.

For a small dataset of only 550 training images, this effect is particularly pronounced. From a statistical perspective, small datasets face several core challenges: First, high sample variance — the statistical properties of each batch may differ significantly, causing drastic fluctuations in batch-computed loss values and gradients; second, high overfitting risk — the model tends to memorize features of specific samples rather than learning generalized patterns; third, insufficient data diversity — certain difficult patterns may only appear in a few samples, and when these samples are sampled or not sampled, model performance exhibits obvious jumps. These factors, combined with Dice Loss's region-level sensitivity, further exacerbate metric fluctuations.

Learning Rate and Loss Weight Imbalance

Another common cause is a learning rate that's too high, or an improper weight ratio between Cross-Entropy Loss and Dice Loss. When the scales and gradient magnitudes of the two losses don't match, Dice Loss gradients may dominate the optimization direction in certain steps, pushing the model into unstable regions, after which Cross-Entropy Loss "pulls it back," creating back-and-forth oscillations.

Practical Solutions

Adjusting Loss Weights and Warmup Strategy

An effective approach is to use a progressive loss combination: rely primarily on Cross-Entropy Loss in the early stages of training to let the model quickly converge to a reasonable region, then gradually increase the weight of Dice Loss. This warmup strategy avoids the instability caused by Dice Loss dominating optimization in the early stages.

The idea behind progressive loss combination is rooted in Curriculum Learning theory — let the model learn simple tasks first, then gradually introduce difficult objectives. Cross-Entropy Loss provides dense pixel-wise supervision signals with stable gradients, suitable for the model to quickly establish basic pixel classification capability in early training. Introducing Dice Loss only makes sense once the model can roughly localize target regions — at this point, foreground predictions are no longer near zero, the denominator is stable, and Dice Loss can safely push the model to optimize region overlap and boundary precision. This strategy is widely adopted in medical image segmentation competitions (such as MICCAI challenges), with many top-performing solutions reporting significant performance improvements through loss scheduling.

# Example: Dynamically adjusting loss weights
dice_weight = min(1.0, current_epoch / warmup_epochs)
loss = ce_loss + dice_weight * dice_loss

Adding Smoothing Terms and Reducing Learning Rate

Adding a smooth constant (smooth term, typically set to 1.0) to both the numerator and denominator of Dice Loss can effectively prevent numerical instability when the denominator approaches zero:

Dice = (2 * |X ∩ Y| + smooth) / (|X| + |Y| + smooth)

At the same time, appropriately reducing the learning rate or using a learning rate scheduler (such as cosine annealing) can significantly reduce metric fluctuation amplitude. Cosine Annealing is a learning rate scheduling strategy proposed by Loshchilov and Hutter in their 2017 SGDR paper. It smoothly decays the learning rate from its initial value to near zero following a cosine function, with an optional restart. Compared to fixed learning rates or step-wise decay, cosine annealing offers advantages: the higher learning rate in early training helps the model escape local optima, while the gradually decreasing rate in later stages allows fine-grained convergence near the optimal solution. For scenarios with unstable Dice Loss, cosine annealing is particularly effective — it naturally reduces step size in later training, diminishing the impact of Dice gradient fluctuations on parameter updates. Other commonly used strategies include ReduceLROnPlateau (automatically reducing the learning rate by monitoring validation metrics) and Warmup+Linear Decay.

Checking Data Flow and Increasing Batch Size

To address periodic fluctuations, it is recommended to enable data shuffling at each epoch and verify that the validation set evaluation logic is stable. When GPU memory allows, appropriately increasing batch size can make Dice Loss estimation smoother, as it computes over a larger sample region.

It's worth noting that when increasing batch size, the computation method of Dice Loss also needs consideration: whether to compute a single Dice value jointly over the entire batch (batch-level Dice) or compute for each sample individually and then average (sample-level Dice). The former provides more stable gradient estimates because it computes overlap over a larger pixel set; the latter preserves the independence of each sample but remains sensitive to samples containing extremely small foreground regions. In practice, many frameworks default to sample-level computation, and switching to batch-level computation can sometimes significantly improve training stability.

Summary and Takeaways

Periodic fluctuations in evaluation metrics are not the model "going wrong" but rather the result of Dice Loss's inherent properties interacting with training configuration. For segmentation practitioners, it's important to understand:

Dice Loss is a region-level metric that is highly sensitive to small targets and class imbalance, with gradients that are inherently less stable than Cross-Entropy. Through proper loss weight scheduling, smoothing term settings, learning rate control, and data flow management, one can achieve smooth training curves while retaining the advantages of Dice Loss (better region overlap).

Ultimately, model quality should not be judged solely by peak metrics at a single epoch but by long-term trends and stability on the validation set. Moderate fluctuations are acceptable, but severe oscillations signal the need to re-examine training configuration. In actual projects, it is recommended to record both Cross-Entropy Loss and Dice score curves simultaneously, diagnosing the root cause by comparing the behavior patterns of both — if Cross-Entropy steadily decreases while Dice fluctuates severely, the problem can almost certainly be attributed to the Dice Loss configuration.

Key Takeaways