Fine-Tuning Qwen3-4B in Practice: Fixing Role Confusion with Just 100 Data Samples

From Failure to a Successful Fine-Tuning Journey

For many developers looking to get started with LLM fine-tuning, smaller parameter models are often the best place to begin. Recently, a content creator on Bilibili shared their complete hands-on experience fine-tuning the Qwen3-4B model for the second time. Compared to models with tens or even hundreds of billions of parameters, a 4B (4 billion parameter) model can be fine-tuned on consumer-grade hardware, making it an ideal choice for validating methodologies and building experience.

Qwen3 is the third-generation large language model series from Alibaba Cloud's Tongyi Qianwen team, spanning multiple parameter sizes from 0.6B to 235B. The 4B version has approximately 4 billion parameters, uses a Transformer decoder architecture, and supports multilingual understanding and generation. Models of this scale can typically handle inference and even fine-tuning on a single consumer GPU (such as an RTX 4090 with 24GB VRAM), significantly lowering the barrier to entry for individual developers. This is precisely why 4B-class models have become an ideal starting point for learning fine-tuning techniques.

The core objective of this fine-tuning effort was clear — to solve the role confusion problem left over from the previous attempt. After some twists and turns, the creator finally announced a successful fine-tuning and conducted preliminary verification of the results.

Notably, this was the author's second attempt at fine-tuning a 4B-class model. The iterative process from failure to success is actually the most valuable part of fine-tuning practice — problem identification, data supplementation, and re-verification. This closed-loop methodology applies to the vast majority of fine-tuning scenarios.

Role Confusion: A Classic Pain Point in Small Model Fine-Tuning

In role-playing or identity-setting applications, model "role confusion" is an extremely common problem. Role confusion refers to the model's inability to stably maintain its assigned identity during multi-turn conversations or complex scenarios, resulting in self-perception drift, irrelevant responses, or identity mix-ups.

This type of problem is especially pronounced in smaller parameter models. Because a 4B model has limited representational capacity, if the training data lacks sufficient and consistent identity-anchoring samples, the model can easily "forget" who it is during inference. From a technical perspective, maintaining identity consistency across multi-turn conversations fundamentally relies on the attention mechanism's sustained focus on the System Prompt, as well as the model's internal parameterized representation of "self-identity." When model parameters are insufficient, this representation can easily be diluted or interfered with by other semantic signals in long contexts. This is why many developers discover after their first fine-tuning attempt that the model has learned knowledge but can't maintain its persona.

Targeted Fix: Identity Stability Data

To address the role confusion problem, the creator adopted a direct and effective strategy — adding identity stability data. According to their description, approximately 100 to 200 identity stability-related data samples were specifically added for this fine-tuning round.

The logic behind this approach is worth elaborating. In Supervised Fine-Tuning (SFT), data "distribution" is often more critical than data "quantity." SFT is a key stage in the LLM training pipeline — after a pre-trained model learns language patterns from massive unlabeled text, it needs to use high-quality human-annotated instruction-response pairs during the SFT stage to align with human intent. Common fine-tuning methods include full-parameter fine-tuning and Parameter-Efficient Fine-Tuning (such as LoRA, QLoRA, etc.), where the latter freezes most parameters and only trains a small number of low-rank adaptation matrices to achieve low-cost training. When the model frequently encounters consistent identity statements in training samples, it reinforces the anchoring of that identity in its parameters. The fact that just one to two hundred targeted data samples can produce visible improvements on a small model demonstrates that:

• Precise data supplementation outperforms blindly expanding the dataset. Rather than adding massive amounts of generalized data, it's better to design small batches of high-quality samples targeting specific deficiencies. This approach is known in the industry as "targeted data engineering" — customizing training data based on the model's specific weaknesses to achieve maximum improvement with minimal data.
• Small models are more sensitive to data distribution. The same amount of supplemental data often has a more pronounced marginal effect on a 4B model than on a larger model. This is because small models have a more compact parameter space, where even a small amount of data can produce observable shifts in parameter distribution, while large models' high parameter redundancy means the same data volume has proportionally less impact.

Based on the author's feedback, this batch of identity stability data worked "quite well," with the role confusion problem being noticeably alleviated.

Effect Verification and Catastrophic Forgetting Risk

It should be objectively noted that this fine-tuning has currently only been verified on the identity stability dimension. The creator acknowledged that other model capabilities (such as knowledge accuracy, multi-turn conversation coherence, instruction following, etc.) have not yet been systematically tested and can only be "tested gradually."

This is actually an easily overlooked aspect of fine-tuning practice. After solving a specific problem, new side effects may be introduced — for example, overfitting to reinforce identity stability might actually damage the model's general capabilities, commonly known as "Catastrophic Forgetting."

Catastrophic Forgetting is a classic problem in sequential learning for neural networks, first systematically identified by McCloskey and Cohen in 1989. When a model is fine-tuned on new data, parameter updates may overwrite previously learned knowledge representations, causing sharp performance degradation on old tasks. The industry has developed multiple mitigation strategies: Elastic Weight Consolidation (EWC) protects old knowledge by penalizing large modifications to important parameters; data mixing replay incorporates original pre-training data proportionally into the fine-tuning dataset; additionally, controlling learning rate, training steps, and using parameter-efficient methods like LoRA themselves have certain regularization effects.

Therefore, comprehensive evaluation is just as important as single-objective optimization. For serious fine-tuning projects, it's recommended to establish a multi-dimensional evaluation set and conduct regression testing after each iteration to ensure that gaining new capabilities doesn't come at the cost of losing old ones.

Next Steps: From Dense Architecture to MoE Architecture

After completing the 4B model fine-tuning, the author has already set their sights on a more advanced target — the 30B-A3B model in the Qwen3 series. This is a model using MoE (Mixture of Experts) architecture, with total parameters of approximately 30B, but only activating about 3B parameters per inference pass, balancing capability and efficiency.

The MoE (Mixture of Experts) architecture was originally proposed by Jacobs et al. in 1991 and has been brought back into the mainstream in recent years by works like Google's Switch Transformer and GShard. Its core idea is to split the model's feed-forward network layers into multiple parallel "expert" sub-networks, with a Gating Network activating only a few experts to process input tokens during each inference pass. This allows the model to have massive total parameters (improving knowledge capacity and expressiveness) while maintaining relatively low actual computational overhead (FLOPs). Qwen3-30B-A3B is a typical representative of this design: 30B total parameters provide ample knowledge storage space, while only about 3B parameters are activated for computation per token, achieving inference efficiency close to a 3B dense model.

Crossing from the Dense architecture 4B model to an MoE architecture significantly increases fine-tuning complexity. MoE models involve optimization of expert routing strategies, load balancing loss (ensuring experts are evenly utilized to avoid some becoming "idle"), and gradient allocation across different experts. In terms of VRAM, although only partial parameters are activated during inference, fine-tuning typically requires loading all parameters and their corresponding optimizer states, demanding far more hardware resources than a dense model with equivalent active parameters. Furthermore, MoE model fine-tuning requires special attention to data diversity — if training data is too homogeneous, the routing network may assign most inputs to only a few experts, causing "expert collapse." This will be a more challenging endeavor.

Additionally, the author mentioned struggling with version selection — newer version or older version. This also reflects the current state of the open-source LLM ecosystem: model iteration speed is extremely fast, and developers often need to balance between "chasing the new" and "seeking stability." Newer versions typically offer better performance and architecture, but ecosystem tools (such as inference framework adaptation for vLLM and SGLang, fine-tuning tool support for LLaMA-Factory and Axolotl), community experience accumulation, and known issue investigation all take time to mature. The general recommendation: for production deployments, choose stable versions thoroughly validated by the community; for learning and experimentation, chasing the new lets you engage with cutting-edge technical implementations.

Summary

Although this Qwen3-4B fine-tuning practice was modest in scale, it presents a complete and pragmatic fine-tuning loop: identify the problem → supplement data in a targeted manner → verify effects → plan next steps. For developers looking to get started with LLM fine-tuning, practicing with small models and validating methodologies at minimal cost is undoubtedly a robust learning path. This "small steps, fast iterations" approach helps developers build intuitive understanding of the entire fine-tuning workflow while accumulating valuable hands-on experience with limited resources. We look forward to the author's follow-up sharing on the 30B-A3B MoE model.