Unverified50% confidenceFactExact time
Shivacon-Energy's LoRA fine-tuning configuration is rank 16, alpha 32, with about 8.8 million trainable parameters (1.75% of the model), 1596 rows of training data, trained on a Google Colab T4 for 300 steps, 3 epochs, learning rate 2e-4, training loss 0.4018, validation loss 0.08377.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
UnverifiedLoRA训练中常用的学习率为1e-4到5e-4之间，常用的Network Rank值为8、16、32、64、12873% similarUnverifiedLoRA使可训练参数量通常仅为全参微调的0.1%-1%，可在单张RTX 3090/4090（显存24GB）上微调7B乃至13B级别的模型73% similarUnverified以 GPT-3 为例，r=4 时 LoRA 可将可训练参数从 175B 降至约 4.7M，减少约 37,000 倍71% similarUnverifiedTaking GPT-3's attention layers as an example, the original weight matrix is approximately 12,288×12,288 (about 150 million parameters), while a LoRA decomposition with rank r=8 requires training only about 200,000 parameters, a compression ratio exceeding 700×.69% similarUnverifiedThe author's publicly shared standard training configuration is: Krea 2 Standard model, Standard LoRA, Network rank 32, Alpha 32, Epochs 64 (saving every 4 epochs), Batch size 1, target resolution 0.25 MP, EMA 0.98.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012401API
curl https://kongchang.com/api/v1/knowledge/claims/1012401MCP
get_claim(id=1012401)