Shivacon-Energy
A 0.5B-parameter energy-domain adapter fine-tuned with LoRA on Qwen2.5-0.5B-Instruct, focused on behavior calibration, anti-hallucination, and JSON format reliability rather than knowledge injection.
Timeline (last 90 days)
Shivacon-Energy is a 0.5B-parameter energy-domain LoRA adapter based on Qwen2.5-0.5B-Instruct, whose core goal is calibration rather than knowledge expansion, emphasizing refusal to fabricate and outputting valid JSON.
Shivacon-Energy's LoRA fine-tuning configuration is rank 16, alpha 32, with about 8.8 million trainable parameters (1.75% of the model), 1596 rows of training data, trained on a Google Colab T4 for 300 steps, 3 epochs, learning rate 2e-4, training loss 0.4018, validation loss 0.08377.
The repository designed a four-gate sellability harness, and adapters from 9 different domains all ran the same set of tests, with ultimately only the energy-domain adapter passing and being released.
The energy adapter's specific scores in the four-gate evaluation are: G1 base model comparison 0 wins/0 losses/6 ties, G2 domain tasks 9/10, G3 unknown honesty 5/5, G3 refusal capability 5/5, G4 JSON/tool format 5/5, hallucination rate 0/10.
G1's 0 wins/0 losses/6 ties result shows that the energy adapter does not know more energy facts than the base model; its advantage is purely behavioral-level honest calibration and JSON reliability.
8 of the 9 adapters failed to pass the gate, with failure reasons including data contamination, over-refusal, and even one adapter denying a real court case.
The v1 version of EVAL_REPORT.md recorded 4/4 hallucination failures (fabricated countries, laws, drugs, and fake tool calls), physics errors (10 kW does not equal 10,000 kWh per hour), Chinese token leakage, and fabricated file paths.
The v6 dataset was rebuilt around v1's failure modes, including refusal rows, anti-hallucination pairs, calibration examples, and tool trajectories; v2 achieved 9/10 domain tasks, 5/5 honesty, 5/5 refusal capability, and 0/10 hallucination rate on the same evaluation harness, with data composition as the only variable.
All evaluations of Shivacon-Energy run on real CPU inference, using greedy decoding, max_new_tokens=100, the Qwen chat template, per-prompt hard timeouts, and every failure case is human-adjudicated with transcripts published.
Shivacon-Energy's known limitations include: no knowledge improvement over the base model, energy domain only, 0.5B parameters cannot replace engineering analysis or professional energy audits, one garbled answer on practical home electricity-saving advice, and strict JSON wording occasionally leaking `Thought:` lines.
All Facts (10)
Shivacon-Energy is a 0.5B-parameter energy-domain LoRA adapter based on Qwen2.5-0.5B-Instruct, whose core goal is calibration rather than knowledge expansion, emphasizing refusal to fabricate and outputting valid JSON.
50%UnverifiedShivacon-Energy's LoRA fine-tuning configuration is rank 16, alpha 32, with about 8.8 million trainable parameters (1.75% of the model), 1596 rows of training data, trained on a Google Colab T4 for 300 steps, 3 epochs, learning rate 2e-4, training loss 0.4018, validation loss 0.08377.
50%UnverifiedThe repository designed a four-gate sellability harness, and adapters from 9 different domains all ran the same set of tests, with ultimately only the energy-domain adapter passing and being released.
50%UnverifiedThe energy adapter's specific scores in the four-gate evaluation are: G1 base model comparison 0 wins/0 losses/6 ties, G2 domain tasks 9/10, G3 unknown honesty 5/5, G3 refusal capability 5/5, G4 JSON/tool format 5/5, hallucination rate 0/10.
50%UnverifiedG1's 0 wins/0 losses/6 ties result shows that the energy adapter does not know more energy facts than the base model; its advantage is purely behavioral-level honest calibration and JSON reliability.
50%Unverified8 of the 9 adapters failed to pass the gate, with failure reasons including data contamination, over-refusal, and even one adapter denying a real court case.
50%UnverifiedThe v1 version of EVAL_REPORT.md recorded 4/4 hallucination failures (fabricated countries, laws, drugs, and fake tool calls), physics errors (10 kW does not equal 10,000 kWh per hour), Chinese token leakage, and fabricated file paths.
50%UnverifiedThe v6 dataset was rebuilt around v1's failure modes, including refusal rows, anti-hallucination pairs, calibration examples, and tool trajectories; v2 achieved 9/10 domain tasks, 5/5 honesty, 5/5 refusal capability, and 0/10 hallucination rate on the same evaluation harness, with data composition as the only variable.
50%UnverifiedAll evaluations of Shivacon-Energy run on real CPU inference, using greedy decoding, max_new_tokens=100, the Qwen chat template, per-prompt hard timeouts, and every failure case is human-adjudicated with transcripts published.
50%UnverifiedShivacon-Energy's known limitations include: no knowledge improvement over the base model, energy domain only, 0.5B parameters cannot replace engineering analysis or professional energy audits, one garbled answer on practical home electricity-saving advice, and strict JSON wording occasionally leaking `Thought:` lines.
50%