IMGNet: A New Face Verification Approach Challenging Cosine Similarity with Sign Pattern Matching

Reflections from an Independent Researcher in Indonesia

In the field of face recognition, cosine similarity is almost the default standard for determining whether two feature vectors belong to the same person. This dominance is no accident — around 2015, deep learning methods like DeepFace and FaceNet ushered in the high-dimensional embedding paradigm for face recognition. Researchers found that in feature spaces of 128 to 512 dimensions, cosine similarity is more robust than Euclidean distance: Euclidean distance is sensitive to vector magnitude, which is often influenced by identity-irrelevant factors like image brightness and contrast. Cosine similarity computes the cosine of the angle between two vectors, completely ignoring their magnitudes and focusing solely on direction — a property that makes it naturally suited for high-dimensional embedding spaces. When feature vectors are L2-normalized, cosine similarity becomes equivalent to a dot product, and highly optimized BLAS libraries keep latency extremely low even at millions of comparisons. Mainstream loss functions like ArcFace and CosFace explicitly embed cosine similarity into their training objectives, forming a closed loop of "train with cosine, infer with cosine."

From an information geometry perspective, L2-normalized feature vectors are distributed on a unit hypersphere, and cosine similarity is equivalent to the geodesic distance metric on that hypersphere — fully consistent with the angular margin modeling in methods like ArcFace. However, when we collapse all discriminative information from a 512-dimensional space into a single scalar, information loss is inevitable — the Johnson-Lindenstrauss Lemma reveals the inherent cost of dimensionality compression. High-dimensional spaces also suffer from the "curse of dimensionality": as dimensions increase, pairwise distances tend to concentrate, reducing the discriminability of nearest neighbors — a challenge shared by all global similarity metrics. Cosine similarity is fundamentally a global metric, compressing the entire embedding vector into a single scalar and failing to capture local structural information.

An independent researcher from Indonesia noticed this potential limitation and asked an intriguing question: Are we over-relying on this default choice? The model he built, IMGNet, attempts to replace traditional cosine similarity with an entirely different approach: sliding window sign pattern matching.

The author's inspiration has a culturally resonant quality. He points out that in Javanese, "thank you" is "matur suwun," while in Sundanese, the same sentiment is expressed as "hatur nuhun." The surface forms are completely different, yet the meaning is identical — identity is preserved through relational structure, not absolute values. This is the core philosophy of IMGNet: rather than comparing the global angular direction of embedding vectors, find consistent sign patterns within locally overlapping windows of the embedding.

IMGNet's Core Technical Innovations

SW Block: Multi-Scale Relational Operations

At the base of the model, standard convolutions are replaced by "multi-scale relational operations." For each pixel, the model computes differences between that pixel and all its neighbors across prime window sizes {3, 5, 7}, then uses a small MLP to map the 240 differences per pixel to output channels. The choice of prime sizes is deliberate — prime numbers avoid divisibility relationships between different scales, reducing receptive field overlap and making the three scales capture as complementary information as possible. This design echoes neuroscience findings on multi-scale processing in the visual cortex: simple cells in V1 respond to edges at different spatial frequencies, which is essentially a form of multi-scale differential computation. From a feature engineering standpoint, difference operations are naturally translation-invariant and illumination-robust — computing differences between neighboring pixels automatically cancels global brightness offsets, a principle shared with successful image gradient features like SIFT and HOG. The distinction is that IMGNet embeds this operation end-to-end within a deep learning framework for learning. This design emphasizes "relative relationships" over "absolute values" from the ground up, consistent with the relational structure philosophy of the entire model.

IMG Sign MSE Loss: A Pure Sign Pattern Loss

The idea of sign pattern matching has deep information-theoretic roots. To understand this design, we need to look back at the field of Locality-Sensitive Hashing (LSH): Moses Charikar's Simhash algorithm (2004) revealed a profound fact — binarizing high-dimensional vectors via random hyperplanes (taking the sign bit) produces a collision probability exactly equal to 1 minus the angle between the two vectors divided by π. This directly establishes a mathematical equivalence between sign consistency and cosine similarity. This means sign information is a sufficient statistic for directional information: given enough random projections, a sign vector can losslessly recover cosine similarity.

The sign function is the most extreme form of quantization, compressing a continuous real number into a single bit — but this extreme compression preserves directional information while discarding magnitude. Localized sign comparison further introduces spatial context: patterns of sign consistency between adjacent dimensions form a structured local descriptor, analogous to the success of LBP (Local Binary Patterns) in texture description. Another advantage of localizing global comparisons is natural noise resistance: random noise within a local window does not propagate to other windows, whereas in a global metric, the influence of a single anomalous dimension is averaged and spread across the final scalar, making it hard to localize or eliminate.

IMGNet localizes this global sign comparison — rather than performing a global sign comparison across the entire vector, it compares local sign consistency within sliding windows, effectively replacing a single global decision boundary with multiple local "hyperplane sets." In theory, this can capture local discriminative structure that global cosine similarity might smooth over, while improving robustness to local noise.

The author claims this is, to their knowledge, the first face verification loss function based entirely on sign pattern consistency without relying on magnitude. The core formula is as follows:

score = mean(gate(tanh(β · E1 · E2)))  # sliding window, β=10
loss_same = ((1 - score) ** 2).mean()   # push toward 1.0
loss_diff = (score ** 2).mean()          # push toward 0.0

In terms of training stability, this sign-based loss exhibits a variance of only ±0.40% between epochs 29 and 50, compared to ±2.25% for a magnitude-based variant — sign pattern matching shows significantly greater stability during training.

Three Metrics, Shared Threshold, and Voting

The model defines three metrics, all in the [0,1] range: IMG Sign Score, AMP IMG Score, and Chain Score, sharing a threshold derived from scanning the IMG Sign. A voting system is built on top of this: 2/3 or 3/3 votes yield a MATCH, 1/3 yields UNCERTAIN, and 0/3 yields DIFFERENT. This echoes the ensemble learning idea of majority voting among weak classifiers, providing stronger robustness for the final decision.

Experimental Results: Respectable Performance from a 10MB Model

IMGNet was evaluated on several standard face verification benchmarks. The model weighs only 10.58 MB (FP32) and was trained on 490,000 images from CASIA-WebFace (a widely used academic training set containing approximately 10,000 identities — far smaller than industrial-scale datasets like MS-Celeb-1M with its millions of images). Below is a comparison of IMG Sign versus traditional Cosine:

Dataset	IMG Sign	Cosine
LFW	96.27%	95.53%
AgeDB-30	78.80%	77.22%
CALFW	78.73%	78.32%
CPLFW	76.85%	74.62%
Overall	81.02%	79.49%

Understanding these results requires some familiarity with the face recognition benchmark ecosystem. LFW (Labeled Faces in the Wild, 2007) was the first widely accepted benchmark for face verification under natural conditions, containing 13,000 images of 5,749 identities. However, since top models have nearly saturated it (exceeding 99.8%), its discriminative power has faded. IMGNet's 96.27% on LFW is mid-tier, indicating significant room for improvement. AgeDB-30 specifically collects photos of the same individuals spanning 30 years of age, reflecting the effect of aging on facial features. CPLFW (Cross-Pose) and CALFW (Cross-Age), proposed in 2018, are more challenging variants designed to target mainstream models' weaknesses in pose and age variation — better reflecting real-world deployment challenges. IMG Sign outperforms Cosine on all test sets, most notably on the cross-pose CPLFW dataset by over 2 percentage points, suggesting that sign patterns are more robust to pose variation. It is worth noting that industrial evaluations also rely on set-level benchmarks like IJB-B and IJB-C (which include video frames) and fairness test sets for specific demographics — dimensions that IMGNet has not yet addressed, and thresholds that must be crossed before moving from academic exploration to industrial application.

Direct Transfer to ArcFace Embeddings

Even more noteworthy is the experiment where the author applied the IMG Sign Score without any retraining directly to pretrained ArcFace (buffalo_l) embeddings, achieving 99.58% on LFW — only 0.24% below ArcFace + Cosine's 99.82%.

ArcFace is the benchmark algorithm in face recognition today. Its core idea is to add additive angular margin to class boundaries in cosine space, forcing the model to learn more compact feature representations. The deeper logic of this design lies in geometric consistency between the training objective and inference metric — the loss function directly imposes constraints in angular space, causing the trained embedding vectors to form well-defined clusters on the hypersphere, with small intra-class angles and large inter-class angles. buffalo_l is trained on the MS1MV3 dataset (approximately 3 million images) and achieves close to 99.8% on LFW, making it a commonly used industrial baseline. It is precisely this deep geometric property — compact clustering structure on the hypersphere — that allows effective discriminative information to be extracted even with a non-standard metric like sign patterns.

This result demonstrates that even in mature embeddings trained for cosine similarity, sign pattern consistency remains a valid discriminative feature. The author speculates that sign pattern consistency may be a fundamental property of well-trained face embeddings, independent of the training objective itself.

An Unexpected Finding Awaiting Validation

While building an ablation visualization tool with custom polygon masks, the author observed an interesting phenomenon: occluding the same facial region in photos of the same person produced delta peaks at similar embedding dimensions, whereas for photos of different people, the positions of these peaks differed significantly.

The author speculates that the overlapping sliding window loss may induce a kind of implicit spatial organization in the embedding space. The author honestly acknowledges that this finding has not been formally validated and remains a preliminary observation. If confirmed by future research, it could offer a new entry point for interpretability research and may connect interestingly with existing work on "feature localization" in neural networks.

Core Claim: Metrics and Training Objectives Should Be Co-Designed

The broader hypothesis underlying IMGNet is metric-loss alignment: similarity metrics should be co-designed with training objectives, rather than defaulting to cosine similarity without deliberation.

The field of metric learning has a clear developmental arc on this issue. Early contrastive loss (2006) and triplet loss (2015) both implicitly assumed Euclidean geometry, treating feature extraction and similarity measurement as two independent stages. ArcFace's (2019) breakthrough lay precisely in embedding angular margin directly into the Softmax classification head, making the training objective geometrically consistent with the cosine metric — one of the key reasons it surpassed prior methods. CircleLoss (2020) further unified the pairwise similarity learning framework, showing that different loss functions are essentially different strategies for assigning gradient weights to similarity scores; ProxyNCA++ replaced sample pairs with class proxies, reducing the computational complexity of metric learning. Together, these works reveal that the geometric structure of feature space is not a byproduct of training but is actively shaped by the form of the loss function.

It is worth noting that the metric-loss alignment principle has universal value in machine learning beyond face recognition. In NLP, contrastive learning frameworks like SimCSE face a similar issue: cosine similarity is used to measure sentence semantic similarity, yet the pretraining objective (e.g., MLM) does not explicitly align with this metric. In recommendation systems, the misalignment between the dot product of dual-tower model vectors and actual business objectives (click-through rate, conversion rate) is a long-standing engineering challenge. These cross-domain examples all point to the same deep issue: the geometric structure of feature space is shaped by the training objective, while the metric function used at inference time implicitly assumes something about that geometry — when the assumption doesn't match the actual geometry, performance loss follows.

IMGNet's attempt resonates with the evolution of metric learning, but goes further: it not only requires consistency between inference metric and training objective, but also demands that relational structure be encoded into bottom-level feature extraction, extending the metric-loss alignment principle to the inductive bias of the feature extractor. This reminds us that there may be a deeper coupling between feature extraction and similarity measurement, and that re-examining this coupling may yield unexpected performance gains.

As an early-stage exploration by an independent researcher, some of IMGNet's conclusions await more rigorous validation. But it offers a novel perspective and demonstrates that researchers with limited resources can still make valuable contributions at the level of foundational methodology. Interested readers can consult the paper published on Zenodo, the GitHub repository, and the model weights on HuggingFace.

Key Takeaways