Phase-HDC

The arXiv paper "Phase-HDC: Replacing Optimizer History with Gradient Thresholds in Discrete Phase Learning" proposes a memory-free training method that replaces optimizer history with gradient thresholds.

The learnable parameters of hyperdimensional classifiers are low-bit angle values, which the paper refers to as phase memory.

Phase-HDC rotates each angle parameter by at most one step per update, in the direction opposite to the sign of the current gradient, and only modifies it when the gradient is sufficiently large.

The paper proves that the update rule of Phase-HDC is the exact solution to a first-order loss model in which each modified parameter incurs a fixed cost.

In experiments where all other conditions were held constant and only the update rule was replaced, Phase-HDC achieved the accuracy of 6-bit momentum Adam while reducing memory to one-third.

Across eleven datasets spanning images, tabular data, and text, Phase-HDC reduces memory by 16–23× compared to standard float32 Adam and by 4–6× compared to 8-bit Adam.

Phase-HDC loses approximately 5 accuracy points on average relative to float32 Adam.

On six of the eleven datasets Phase-HDC achieves higher accuracy than 8-bit Adam, and on the byte-level text prediction task 8-bit Adam collapses entirely.

The paper explicitly states that Phase-HDC's memory savings are at the logical-state level, not measured hardware memory.

Phase-HDC is better suited for edge and embedded scenarios where memory is extremely constrained and there is already a demand for hyperdimensional models.