Fine-Tune an LLM or Build an App? Data Preparation Is the Architecture Core

Title

Fine-Tune an LLM or Build an App? Data Preparation Is the Architecture Core

Meta Description

Fine-tune an LLM or build an LLM app? A Reddit discussion argues that data preparation should be the architecture core, not an afterthought. This article analyzes three practical levers: schema-first design, replayable pipelines, and a shared foundation for training and retrieval—plus how data engineering's role in team division of labor is shifting.

Article Content

When many teams kick off an LLM project, they spend a lot of time agonizing over which base model to choose, whether to use LoRA or full-parameter fine-tuning, and whether to add RAG. A recent discussion thread in the Reddit community raised an underappreciated point: whether you take the fine-tuning route or build an LLM application, data preparation should not be a step you patch up after the fact—it must be designed as part of the system architecture.

This judgment seems plain, but in engineering practice it directly determines whether a project can iterate continuously or has to be fully reworked the first time the data format changes.

Why Data Preparation Is Always Treated as "Dirty Work"

In traditional machine learning projects, the data pipeline has clear upstream and downstream stages: collection, cleaning, labeling, feature engineering, and training. What's special about the LLM era is that data simultaneously plays three roles—training corpus, retrieval knowledge base, and runtime context. The three have inconsistent requirements for data.

Fine-tuning needs high-quality, uniformly formatted instruction-answer pairs with low tolerance for noise; RAG needs document chunks that can be split, retrieved, and carry metadata; and prompt engineering relies on structured context assembly. If a team organizes data for only one of these uses early in the project, switching approaches later will almost inevitably require rebuilding the data layer.

The core argument of the Reddit original post is exactly this: data preparation is not a preprocessing step before training, but an architectural decision on par with model selection and service orchestration.

Typical structure of instruction fine-tuning data

The five fields instruction, input, output, source, and version mentioned in the article correspond to a common sample in Supervised Fine-Tuning (SFT). instruction describes the task requirement, input provides optional task input, and output is the answer the model is expected to generate; source marks whether the sample comes from human labeling, a public dataset, or synthetic data, for auditing and deduplication; and version allows the dataset to evolve incrementally without breaking the reproducibility of historical training records. This structure is not the only standard—popular formats such as Alpaca and ShareGPT each have variants—but clear and stable field semantics matter more than specific naming.

Differences in data requirements between LoRA and full-parameter fine-tuning

LoRA (Low-Rank Adaptation) reduces memory and storage overhead by freezing the base model and training only low-rank adaptation matrices, so its data volume requirements are usually lower than full-parameter fine-tuning, but it is equally sensitive to data quality and format consistency—low-quality samples are more easily amplified into style drift in parameter-efficient fine-tuning. Full-parameter fine-tuning, by contrast, has higher requirements for data scale and diversity, and each iteration costs more, so the importance of data version management rises accordingly. The shared conclusion is: the stability of the data schema is independent of the choice of fine-tuning method.

Three Levers for Incorporating the Data Layer into Architecture Design

1. Schema-first, not model-first

Most teams' approach is to decide the model first, then work backward to the data format. A more reliable order is the reverse: first define the data schema—fields, types, versions, source markers—then evaluate which model or retrieval approach can consume this schema. The cost is a slower start; the benefit is that models are replaceable.

In practice, instruction fine-tuning data should contain at least five fields: instruction, input, output, source, and version, where source is used to trace data provenance for auditing and deduplication, and version is used to support incremental dataset updates without breaking existing training records.

2. Data pipelines must be idempotent and replayable

A typical failure in LLM projects is "which version of the data was used in the last training run?" If cleaning scripts are not replayable and intermediate artifacts have no snapshots, then when model performance regresses there is simply no way to locate the cause. Building the data pipeline as a production service—versioned artifact storage, replayable cleaning logic, and change logs—is a necessary investment at the architectural level.

3. Training data and retrieval data share a foundation

Fine-tuning and RAG are not mutually exclusive; many production systems use both. If the training corpus and the retrieval knowledge base come from two independent data sources and cleaning logic, knowledge inconsistency arises: what the model learns during fine-tuning contradicts what retrieval returns. Sharing the same raw data foundation and doing cleaning and branching at the same layer can significantly reduce such conflicts.

RAG's requirements for data splitting and metadata

The core of RAG (Retrieval-Augmented Generation) is to split external documents into retrievable chunks, recall them by relevance at inference time, and splice them into the context. This has structural differences from the requirements of fine-tuning corpora: retrieval chunks need length control to fit the embedding model's context window, need to retain metadata such as titles, sources, and timestamps to support filtering and provenance, and need to preserve semantic completeness when splitting, avoiding cutting a sentence in half. If the training corpus and retrieval knowledge base share the same set of raw documents, branching logic needs to be designed after the cleaning layer—the same source data produces instruction samples and retrieval chunks respectively, rather than maintaining an independent pipeline for each. This is the engineering meaning of "shared foundation, same-layer branching" mentioned in the article.

Practical Impact on Team Division of Labor

Incorporating data preparation into the architecture means data engineering is no longer a "support role." In project scheduling, data schema design and pipeline construction should start in parallel with model evaluation, rather than waiting until model selection is finalized. This also explains why many LLM projects perform well in the POC stage but struggle when entering production—POCs can assemble data by hand, but production environments cannot.

There is a judgment worth remembering in the original post discussion: models will change and frameworks will change, but a well-designed data layer can persist and be reused across these changes. Prioritizing budget and manpower toward the data layer yields higher long-term returns than chasing the latest model.

Summary

Fine-tune or build an app—the question itself may be pointed in the wrong direction. What truly determines whether an LLM project can scale is whether the data layer is designed as a first-class citizen. Schema-first, replayable pipelines, and a shared foundation for training and retrieval—these three are the minimum consensus that can be implemented today.

One-line Summary

Data preparation should be a first-class citizen of LLM system architecture, not a preprocessing step before training.

Paragraph Summary

Based on a discussion in the Reddit community, the article points out that data preparation has long been underestimated in LLM projects. Data simultaneously plays three roles—training corpus, retrieval knowledge base, and runtime context—and the three have inconsistent requirements. If data is organized for only a single purpose, switching approaches will almost inevitably require rework. The author proposes three implementable architectural levers: schema-first, defining fields, types, and versions before choosing a model to ensure model replaceability; data pipelines with idempotency and replayability to support failure localization when performance regresses; and when fine-tuning and RAG coexist, sharing the same data foundation to avoid knowledge inconsistency. The article further points out that this means data engineering is no longer a support role and should start in parallel with model evaluation—which is also the root cause of why many projects perform well in POC but struggle when entering production.