5.3M-Parameter Clothing Classifier Pasma: Small-Model Upset or Scenario-Limited?

Title

5.3M-Parameter Clothing Classifier Pasma: Small-Model Upset or Scenario-Limited?

Meta Description

Vestes Archive open-sources Pasma, a 5.3M-parameter clothing classifier. Its combined-set Top-1 of 89.12% beats zero-shot SigLIP-Base, but rankings reverse on the home/catalog subset. This article analyzes its deployment trade-offs, category recall weaknesses, and reproducibility limits.

Article Content

When choosing between small models and zero-shot vision models, a practice from Vestes Archive offers an unusual answer: a task-fine-tuned small model leads substantially on the combined test set but loses in a specific scenario.

Their open-source Pasma-Small-320 has only 5.296M parameters, and its FP16 ONNX graph is just 10.32 MiB, yet it takes on large zero-shot models like SigLIP-Base. The "reversal" in evaluation results is more worthy of discussion than the win or loss itself.

Evaluation Setup: A Deployment Comparison, Not a Fair Ablation

The deployment problem Pasma solves is very specific: classify a single garment without uploading photos or running large vision-language models. The output includes 20 clothing types plus category and visual material scores.

According to the author's explanation in the original Reddit post, Pasma is the result of task fine-tuning and distillation, while SigLIP-Base remains in a zero-shot state. Therefore, this comparison is positioned as a "deployment comparison," not an ablation with equal training amounts, and it cannot be used to claim that small backbones are universally stronger.

This disclaimer itself is commendable. Many small-model comparison articles overlook whether the baseline was trained equally, leading to exaggerated conclusions.

Zero-shot means the model directly performs classification or recognition without any fine-tuning or examples for the downstream task. SigLIP-Base is a vision-language contrastive learning model trained on large-scale image-text pairs, mapping images and text into the same embedding space; at inference, text descriptions (such as "a photo of a blouse") are matched with images for classification. Its advantages are strong generalization and no need for task data, but its compute and parameter count are far larger than a dedicated small model.

Task fine-tuning and distillation involve continued training on a specific dataset, or having a small model mimic a large model's outputs, trading some generalization for target-domain accuracy and extremely small size. The two differ in training objectives, data exposure, and deployment cost, which is why the article emphasizes this is a "deployment comparison" rather than a controlled-variable ablation.

Data Reversal: Leading on the Combined Set, Lagging on the Home/Catalog Set

On a held-out combined set of 1,103 real photos, Pasma-Small's clothing-type Top-1 accuracy is 89.12%, versus 80.24% for zero-shot SigLIP-Base—a gap of nearly 9 percentage points.

But on a subset containing only 249 home/catalog images, the ranking completely reverses: Pasma scores 83.53% while SigLIP-Base scores 87.15%.

This shows that fine-tuned models are more sensitive to the training distribution—strong on similar styles, but once they encounter catalog-style white-background images or specific shooting styles, their generalization may lag behind large zero-shot models. The author also explicitly states that this is not meant to prove small models are better in all scenarios.

Domain shift refers to a mismatch between the test data distribution and the training data distribution—for example, the training set consists mostly of lifestyle photos while the test set contains white-background catalog images, different lighting, or shooting angles. Fine-tuned models tend to overfit the training distribution and degrade more noticeably on new styles; large zero-shot models are usually more robust to style changes because they have seen a wider range of image-text data. This is also a possible reason for the ranking reversal on the home/catalog subset.

Top-1 and Top-3 accuracy are common metrics in classification tasks: Top-1 requires the highest-probability class to be correct; Top-3 counts a hit as long as the correct class appears among the three highest-probability candidates. A higher Top-3 indicates that although the model may not directly give the correct answer, it can keep the correct option in the candidate list, which suits an interaction flow of "suggestion + user correction."

Top-3 Usability and the Weakest Category

For practical applications, Top-1 is often not the only metric. Pasma-Small achieves 97.10% Top-3 accuracy on the combined set, meaning it is quite workable as "suggested candidates."

But the author points out a key weakness: the Top-1 recall for blouse is only 47.8%. Looking only at overall accuracy easily overlooks such class-level imbalance.

Their design approach positions the output as user-correctable suggestions rather than authoritative labels. Material scores also only describe appearance and do not represent verified fiber composition.

Release Contents and Reproducibility Boundaries

This release provides v1.0 candidate weights and inference code, including ONNX, CoreML packages, safetensors, and a Python inference implementation; the code and fitted weights use the Apache-2.0 license. The author also provides a local browser demo.

Note the reproducibility limits: the training images are no longer distributed, so the training process cannot be fully independently reproduced from the public code. Mobile performance and some export alignment work remain unfinished.

ONNX is an open neural network exchange format that allows models to run across different frameworks and hardware; CoreML is Apple's on-device inference framework, convenient for local execution on iOS/macOS devices; safetensors is a safe and efficient weight storage format. Releasing all three indicates the author considered both cross-platform deployment and on-device inference needs.

Apache-2.0 is a permissive open-source license that allows commercial use, modification, and redistribution, provided copyright and license notices are retained. Note that the license covers the code and fitted weights, not the training images; the fact that training data is not distributed is precisely the key reproducibility limitation—without the original data, third parties cannot retrain an identical model.

Cross-Verification from Multiple Sources

Information from the Gitee repository mirror, the Hugging Face model page, and the original Reddit post corroborates one another: model size, evaluation numbers, and license terms are consistent, and the author openly invites criticism regarding the evaluation setup and cross-domain performance.

However, it should be noted that the conclusion "ranking reversal on the home/catalog subset" currently comes only from the author's own evaluation and has not yet been independently reproduced by a third party. If your application scenario leans toward e-commerce white-background images, it is advisable to validate on your own data before deciding whether to adopt it.

Implications for Deployment Selection

Pasma's value is not in proving that small models "beat" large models, but in demonstrating engineering trade-offs under specific deployment constraints: 5M-level parameters and 10MB-level model size, suitable for on-device or privacy-sensitive scenarios.

But imbalanced category recall, sensitivity to domain shift, and non-reproducible training data are all costs that must be faced when selecting a model. For developers, a more pragmatic approach is to keep the zero-shot model as a fallback and measure the small model's actual performance on business data.

The author also openly solicits reports from different image domains, especially which label confusions make the model unsuitable for specific applications. This openness is more valuable for reference than a pretty accuracy number.

One-line Summary

After fine-tuning, a 5M-parameter small model leads zero-shot large models on the combined set, but reverses on the home/catalog subset, revealing the cost of domain shift.

Paragraph Summary

Vestes Archive open-sourced Pasma-Small-320, with only 5.296M parameters and a 10.32MiB ONNX graph, for single-garment classification. On a combined set of 1,103 real photos, its task-fine-tuned Top-1 accuracy reaches 89.12%, higher than zero-shot SigLIP-Base's 80.24%; but on a 249-image home/catalog subset the ranking reverses (83.53% vs 87.15%), showing the fine-tuned model is more sensitive to the training distribution. The author explicitly positions it as a deployment comparison rather than a fair ablation and notes the class imbalance weakness that blouse Top-1 recall is only 47.8%. The model is released under Apache-2.0 with ONNX, CoreML, safetensors, and inference code, but training images are not distributed, so full reproduction is impossible. The conclusion: small models suit on-device privacy scenarios but must be validated on business data with a zero-shot fallback retained.