LoRA Training Showdown: We Tested 6 Base Models — Does Ideogram Beat Flux?

A Four-Day Model Benchmark Experiment

In the world of AI image generation, your choice of base model directly determines how accurately and generalizably a character LoRA can reproduce a subject. A base model is the foundational weight structure of a generative AI system, while LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique — it inserts low-rank decomposition matrices alongside the original weight matrices, allowing targeted behavioral adjustments to the base model with minimal additional parameters. The quality of the base model defines the "feature space" that LoRA has to work with: if the base model already has a weak internal representation of certain face types, no amount of fine-tuning can fully compensate for that fundamental limitation.

Recently, a developer running an AI avatar generation service shared a detailed benchmark on Reddit — a roughly four-day experiment testing six popular models through the same training pipeline to compare their real-world performance on character LoRA training.

What makes this test valuable is its production-oriented perspective. The developer had previously built his workflow around Flux 1 Dev, but noticed that whenever a new model launched, the number of character LoRA uploads on CivitAI dropped significantly compared to the peak days of Flux 1 Dev. To find a better base model for his production pipeline, he put two 4070 Ti GPUs in his basement to work — running continuously for several days.

One notable detail: he delegated nearly all training configuration tuning (rank, learning rate, text encoder learning rate) and error handling to Claude. This is a telling sign of a broader trend — with LLM assistance, the technical barrier to LoRA training is dropping fast.

Methodology: Two Subject Types and Three Rounds of Prompt Iteration

Deliberately Choosing "Hard-to-Reproduce" Subjects as a Stress Test

The choice of training subjects was deliberate and representative. The first subject was a generic white woman — a face type that most models handle well, serving as a useful baseline reference. The second subject intentionally featured South Asian and Black individuals.

There's a meaningful technical reason behind this choice. The pretraining datasets used by mainstream image generation models (including the Stable Diffusion family, Flux, etc.) — such as LAION-5B — contain far more samples of white Western faces than South Asian, African, or other underrepresented groups. This imbalanced data distribution causes models to develop weaker internal representations of minority faces. When LoRA fine-tuning is applied, there aren't enough "anchor points" to adjust from, and outputs tend to drift toward the statistical average of that demographic in the training data — producing stereotyped, caricature-like faces. According to the developer, mainstream models consistently struggle with these two demographics, frequently overfitting and degrading into "racialized cartoon stereotypes." This phenomenon directly exposes a fundamental gap in training data diversity across current mainstream image models.

Three Rounds of Iteration to Find True Generalization

The testing itself went through multiple iterations. The developer openly admitted that v1 and v2 prompt sets were too simple and masked a critical problem: overfitted models often just "replay" training images — delivering seemingly high fidelity while having essentially zero generalization ability.

Overfitting means a model performs well on training data but generalizes poorly to new, unseen scenarios. In LoRA training, the classic symptom of overfitting is that inference can only "reproduce" poses, lighting, or backgrounds seen during training — as soon as you change the scene in a prompt, the character's features collapse. Common culprits include excessively high rank values (the dimensionality of the low-rank matrices), poorly tuned learning rates, or too many training steps.

The v3 prompt set addressed this by using significantly more complex prompts specifically designed to test whether each model could reconstruct a character's facial features in entirely new scenarios — rather than simply memorizing the training images. This improvement made the final comparative results far more meaningful.

Detailed Results for All Six Models

The six models tested were: Krea, Ideogram, Flux 1 Dev, Flux 2 Dev, Flux 2 Klein, and Z Image.

One important technical detail: after Claude-assisted optimization, most models fit within 16GB of VRAM. The sole exception was Flux 2 Dev — and this ties directly to its architectural choices. The Flux model family, released by Black Forest Labs, uses a Diffusion Transformer (DiT) architecture, replacing the traditional U-Net with a dual-stream attention mechanism. Flux 2 Dev introduces Mistral as its text encoder — a relatively large language model whose embedding dimensions and sequence lengths far exceed those of lightweight encoders like CLIP, resulting in several times the VRAM overhead compared to its predecessor. This is precisely why it cannot run comfortably in a 16GB VRAM environment.

The developer resorted to renting a 5090 GPU via RunPod — a GPU cloud platform for AI developers offering on-demand access to everything from RTX 4090s to H100s and B200s, billed by the hour — and attempted training twice. The results were poor, and mounting cloud costs eventually led him to abandon deeper testing of this model.

Overall Rankings

After multiple rounds of training and comparison, the developer's final rankings were:

Ideogram > Flux 1 Dev > Z Image > Flux 2 Klein > Krea > Flux 2 Dev

Several results here are genuinely surprising:

• Flux 2 Dev landed at the bottom, while the lighter Flux 2 Klein ranked significantly higher
• Flux 1 Dev held steady in second place, validating the lasting value of a mature, well-tested model
• Ideogram took the top spot, emerging as the clear winner of this benchmark

Ideogram was founded by former Google Brain researchers and initially gained recognition for its strong text rendering capabilities — a long-standing challenge in image generation. The specifics of its underlying architecture and training data strategy haven't been fully disclosed, but based on these results, its overall performance in character facial fidelity and cross-scene generalization surpassed other leading models — with a particularly notable advantage in reproducing diverse ethnicities.

Switching from Flux 1 Dev to Ideogram: The Developer's Real Decision

Based on his findings, the developer made a clear call: migrate his primary production pipeline from Flux 1 Dev to Ideogram.

In his words: "Aside from a few weird outlier results, Ideogram is a massive improvement over Flux 1 in terms of output quality." For an AI avatar service that depends heavily on accurate character reproduction, that kind of qualitative gap is more than enough justification to migrate.

That said, he acknowledged one open variable: his current Flux 1 Dev LoRA production environment runs on H100s, and Ideogram's training time on H100 hardware hasn't been systematically evaluated yet. For a commercial service, training speed has direct implications for cost and throughput — and this will need to be assessed before making the final switch.

Practical Takeaways for LoRA Training Practitioners

While this was a personal experiment, the methodology offers genuinely useful guidance for the broader LoRA training community:

• Watch out for overfitting's "false high scores": High fidelity measured with simple prompts is deeply misleading. You must introduce novel, complex scenes to validate true generalization ability. Balancing rank settings, learning rate scheduling, and training step count is the core lever for preventing overfitting.
• Test diverse demographic groups: If your application serves a global user base, differences in reproduction quality across skin tones and ethnicities cannot be ignored. Mainstream models already have weaker pretraining representations for minority faces — and LoRA fine-tuning tends to amplify that gap further.
• Newer doesn't mean better: Flux 2 Dev's last-place finish demonstrates that a larger model with a more modern architecture doesn't automatically outperform a mature, established one on specific tasks like character LoRA training. Architectural innovations — like introducing the Mistral encoder — may bring gains in some dimensions while becoming a liability in others.
• Leverage AI-assisted hyperparameter tuning: Delegating configuration optimization and error handling to tools like Claude can dramatically reduce the manual effort of running batch experiments, freeing developers to focus on interpreting results.

It's worth noting that the developer closed with an open call for help: "If anyone knows better configuration techniques for training Flux 2 Dev, I'd love to hear them." This is a fair reminder that Flux 2 Dev's poor showing remains an open question — it's unclear whether the culprit is an inherent model limitation or simply the fact that the Mistral encoder hasn't yet found its optimal configuration for LoRA training scenarios.

Conclusion

In a landscape where new image generation models launch constantly, there's no universal answer to "which base model is best for character LoRA training." But a real-world benchmark run over multiple days, grounded in an actual production use case, carries far more weight than any marketing claim.

For developers planning to build AI portrait or avatar services, Ideogram's top ranking here may be a signal worth taking seriously. Consider running a validation round on your own dataset — paying close attention to generalization performance across diverse demographic samples and complex prompt scenarios — before committing to a final choice for your production pipeline.