Unverified50% confidenceFactExact time
The experiment used the following hyperparameters: batch size 16, LoRA rank 16, LoRA alpha 32, dropout 0.05, learning rate 0.0002, 1 epoch, warm-up ratio 0.05, and weight decay 0.
1
Sources
50%
Confidence
Long-term
Relevance
11/18/2026
First Seen
Sources
Small Model Fine-Tuning Upset: 2B/4B Parameters Beat GPT-6 Luna
youtubeNeuralNine11/18/2026
Related Entities
Related Claims
UnverifiedThe fixed test conditions included: base model unsloth/Llama-3.2-3B-Instruct-bnb-4bit, 1000 samples, 2 epochs, 250 steps, batch size 2, gradient accumulation 4, effective batch size 8, and about 24.3M trainable parameters (0.75% of total parameters).65% similarUnverifiedThe test used the minimax_h3_fl2va_int8_convrot model.64% similarUnverifiedOn 300 practice exam papers labeled by Qwen3-32B, fine-tuned v3 of Laya raised the judgment macro-F1 across 11 questions from 0.33 to 0.57 versus the zero-shot version, cut calibration error from 0.15 to 0.07, and reduced danger-level MAE from 2.78 to 1.56.64% similarUnverifiedScores across 19 models ranged from 0.55 to 1.00, with Qwen 3 235B A22B Instruct ranking first at 1.00.63% similarUnverifiedActual testing of two quantization models: IQ2_XXS achieves 48 tokens/s for short Q&A and 63 tokens/s for long text; IQ3_XXS achieves 30 tokens/s for short Q&A and 39 tokens/s for long text, with VRAM usage of approximately 16.44 GB, activating 10,138 experts—IQ2 is about 60% faster than IQ3.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1185325API
curl https://kongchang.com/api/v1/knowledge/claims/1185325MCP
get_claim(id=1185325)