Fine-tuning

当代前沿模型的训练方法包括RLHF（基于人类反馈的强化学习）及其后继者RLAIF和过程奖励模型（PRM）

Fine-tuning involves taking an existing large model and further training it with a specially prepared proprietary dataset, thereby changing the model's weight values

A pre-trained model refers to a model that has been trained in advance on large-scale datasets, which users can directly fine-tune or call to complete specific tasks.

A well-aligned model will guide meta-prompts toward compliant creative expression, while an insufficiently aligned model may exhibit unexpected behavior.

Fine-tuning requires hundreds to thousands of high-quality training samples, hours of GPU compute time, and ongoing maintenance costs, and the entire process must be repeated whenever knowledge is updated.

Models can still experience stitching-type hallucinations when integrating multiple retrieved results, so RAG is not the ultimate solution for eliminating hallucinations.

Fine-tuning burns knowledge into model weights, making knowledge updates costly as they require retraining; RAG stores knowledge in an external database, so updates only require modifying the database without changing the model.

In AI coding, overly specific instructions 'lock down' the model's solution space, whereas moderately abstract high-level instructions can instead activate broader reasoning pathways in the model.

A study published on arXiv (arXiv:2609.21096) proposes distinguishing hallucinated responses from trustworthy responses by analyzing the topological structure of information flow patterns in attention graphs.

The cost of a single model fine-tuning ranges from hundreds to tens of thousands of dollars.

Researchers have found that fine-tuning only a small number of model parameters is sufficient to remove safety alignment (de-alignment attacks), and that certain jailbreak prompts can transfer across different models.

RAG naturally supports citation capability, with answers pointing to source passages; fine-tuning does not have built-in citations.

In production environments, RAG's ongoing costs are mainly retrieval and longer prompts per answer; fine-tuning's ongoing costs are mainly training runs and evaluation after each training.

RAG changes what the model can see, while fine-tuning changes how the model performs.

For most enterprises, the main engineering expense of RAG and fine-tuning is engineering time rather than API bills.

The data pricing methods currently being explored in the industry include: by data volume (per token/per record), tiered licensing by use case (separate pricing for pre-training/fine-tuning/evaluation), revenue sharing based on downstream model gains, and auction mechanisms.

Fine-tuning adjusts model weights and behavior (tone, format, reasoning style); it is not a conventional means of giving an application access to frequently changing factual information.

RAG and fine-tuning are not mutually exclusive; retrieval and a fine-tuned model can be used together: use fine-tuning to shape the style and structure of answers, and use RAG to provide specific facts and traceable sources.

Common paths in the industry for solving the cross-session memory problem of coding agents include fine-tuning (high cost, long cycle) and RAG (retrieval-augmented generation).

Architecture selection should be based on requirements: traditional machine learning for classification and prediction, foundation model APIs for text generation and summarization, RAG for answers based on private or changing documents, fine-tuning only for specialized behaviors that prompt engineering cannot satisfy, computer vision for image and video detection, and agent architecture only for multi-step workflows that require tool invocation