Six AI Models Tested for Novel Writing: Which One Reigns Supreme?

Which AI model is best for writing novels? There's no single definitive answer — but a multi-dimensional benchmark organized by Bilibili creator Xingyue offers a highly useful reference. The evaluation placed six mainstream large language models — Gemini 3.8 Flash High, Claude Opus 5, GPT 5.6, DeepSeek V4, Qwen 3.8, and GLM 5.3 — on the same scoring sheet, rating each across three writing scenarios: short fiction, long-form male-oriented web novels (男频), and long-form female-oriented web novels (女频). Scores from multiple long-form writers were also incorporated, and the results were averaged into a final leaderboard. The outcome surprised many.

Short Fiction: Qwen Emerges as the Biggest Dark Horse

The short fiction evaluation focused on works around 10,000 characters, comparing performance across three key stages: opening hook, outline, and main body. In this scenario, Qwen scored 8 points and ranked near the top, with reviewers describing their first experience as "jaw-dropping."

Qwen's strength wasn't dominance in any single category but rather well-rounded consistency. Its hooks had genuine flair, its outlines were clear, and its main body text showed relatively little "AI flavor." From brainstorming to outlining to drafting, it could carry a short story all the way through. Reviewers stressed that there's no reason to lower expectations just because it's a domestic Chinese model.

By comparison, Gemini was characterized as "creatively strong with standout genre-fiction ideas" — toss in a premise and it delivers unexpected yet coherent twists. However, its body text showed heavier AI flavor, with characters prone to feeling flat and scripted. GPT, on the other hand, is the quintessential "executor" — not strong on creativity, with outlines leaning toward the generic, but its body text is logically coherent, callbacks land, and it follows instructions well. It's best suited for drafting when you already have a concept and outline, earning a short fiction score of 7.

Claude was rated "competent but unremarkable" in short fiction — less creative than Gemini, less stylistically distinct than some domestic models — scoring 5. DeepSeek^Ad similarly landed at 5: affordable and solid, but lacking a compelling reason to choose it over others. GLM showed solid creativity and a high rate of inspired ideas, but its body text had a "long-form feel" — each scene layered with atmosphere, interiority, and backstory — which actually works against the tight pacing short fiction demands. It's better deployed for brainstorming and outlining.

---

What is "AI flavor"? The term (AI味) is informal jargon common among web novel writers. It refers to a cluster of tell-tale machine-generated traits: highly symmetrical sentence structures (e.g., "He felt... She thought..." cycling repeatedly), emotional descriptions drawn from a fixed lexicon ("heart racing," "eyes glistening"), a habit of capping each paragraph with a reflection or takeaway, and character motivations that serve only the immediate scene without carrying forward. Any one of these might go unnoticed, but when they pile up across 10,000 characters, the overall texture degrades noticeably. For writing assistance, "light AI flavor" means generated content can move directly into revision rather than requiring a full manual rewrite — which is exactly why reviewers treated it as a key dimension.

---

Long-Form Male-Oriented Fiction: GPT Tops the Chart on Structural Strength

Switch to long-form male-oriented fiction and the rankings shuffle immediately. GPT 5.6 earned the highest score of 9, with strengths concentrated in logical architecture and manuscript review — when handling large-worldbuilding genres like xianxia and high-powered martial arts, it can map out power progression systems cleanly and catch contradictions in lore or character consistency. Its drawback: the longer the instructions, the more likely it is to "freelance," and classical Chinese settings can occasionally surface awkward phrasing.

Gemini was judged to have entered the top tier here (8.5), with three key strengths: fast generation, stable instruction-following, and enough creative divergence for brainstorming — particularly useful when grinding against a deadline. Its weaknesses remain in dialogue texture during scene continuations and emotional peak delivery, where it still trails GPT. Claude (8) delivers genuine prose quality, long-form stylistic consistency, and strong atmospheric buildup — but generation speed is noticeably slower and output quality shows more variance.

Qwen (8) excels at capturing domestic social context and everyday interpersonal dynamics. In contemporary, historical, and slice-of-life genres, character motivations feel grounded and immersive — though its ceiling on extreme prose craft and creative range is constrained by web fiction conventions. DeepSeek (7.5) scores highest for reliability — it rarely goes off the rails — but its ceiling is limited and it struggles to sustain more imagination-intensive premises. GLM (7) is balanced across the board with decent speed, suitable for lightweight tasks like summaries and setting organization, but lacks a standout advantage that makes it irreplaceable.

Long-Form Female-Oriented Fiction: Claude and GLM Stage a Comeback

In the female-oriented romance and emotional fiction category, the rankings flip again. Claude earned the highest score of the entire evaluation at 9.5 — the female-fiction reviewer said she generates roughly 80% of her scripts directly with Claude, and that this version is the closest to the official release. The core appeal: body text flows smoothly and requires minimal revision after generation.

GLM 5.3 also earned 9 points in this category, deemed a strong fit for contemporary female-oriented romance — light on AI flavor, logically coherent, and less prone to paragraph-level jumps. The reviewer offered one specific tip: when generating body text, try setting the imagination parameter to 0.8. Interestingly, the short fiction reviewer criticized GLM for "too much setup," while the female-fiction reviewer valued precisely that quality for body text — the same model receiving opposite assessments depending on the use case.

GPT (8.5) continues to win on planning in this genre, best used for generating a complete outline first before handing creative events and conflict to a more imagination-forward model. Gemini (7.5) performs well for scene breakdowns, manuscript review, and generating the romantic tension in "pampering" (宠文) stories. DeepSeek (7) runs smoothly enough to serve as a backup. Qwen scored only 6 in this category, mainly useful for generating cheat-code setups and character cards — functional but unremarkable.

---

A note on GLM's "imagination parameter" (suggested at 0.8): This corresponds to the model's Temperature or a similar creativity-divergence coefficient. Lower values produce more conservative output that stays close to existing context; higher values push the model toward introducing new vocabulary and associations. Contemporary female-oriented fiction demands strong plot continuity — logical jumps between paragraphs directly break immersion. Dialing this parameter down moderately keeps the model "well-behaved," extending established character states and scene rhythm rather than introducing new elements at the start of each paragraph. Different platforms label this parameter differently — "randomness," "temperature," or "creativity level" — so check your platform's interface to confirm the corresponding option.

---

Live Generation Test: Don't Just Judge the Prose — Check Whether It Honors the Rules

Beyond scores, the evaluation included live text demonstrations. The focus wasn't on how smooth the prose sounded, but whether the model could "hold" an already-established set of plot rules.

The setup specified "completing three food delivery orders to finish an F-rank quest" as a clear threshold. The O5.5 version, after delivering just one order, wrote "quest complete, review satisfactory" — leaving the other two orders unaddressed, effectively erasing the rule. The 6.1s version went further, rewriting "three orders to complete the quest" as "three consecutive orders for a bonus reward" — replacing the original constraint with a new one.

This example highlights something easy to overlook: output that reads smoothly doesn't mean it respected the plot constraints. An unexplained blank like "what the red dragon is used for" isn't inherently wrong for a model to fill in — what matters is whether the new rule is internally consistent and advances the current story. A single generation test can't determine a model's overall strength, but it can tell you exactly what to audit in your own manuscript: your rules, your characters, and your conflicts.

---

Why "rule tampering" is a systemic risk, not a one-off glitch: Large language models generating continuations are simultaneously influenced by narrative conventions in their training data ("quest complete" is a standard story-progression beat) and by the user's current prompt. When these create a hidden conflict, the model defaults to the "smoother" narrative trajectory rather than strictly enforcing a custom rule buried in a long document. This is why longer setting documents and more complex worldbuilding require placing core constraints in a visible position at the top of the prompt — or listing them separately as a checklist before each continuation — rather than relying on the model to "remember" every rule from context on its own.

---

Final Rankings and Selection Advice

Averaging all nine sets of scores, the comprehensive leaderboard reads:

• 1st Place: Gemini 3.8 Flash High — average 7.22
• 2nd Place: GPT 5.6 — average 7.17
• 3rd Place: Claude Opus 5 — average 6.56
• 4th Place: GLM 5.3 — average 5.67

It's worth emphasizing that these remain subjective writing evaluations, and the gaps between the top three are inconsistent across tasks. The more practical takeaway isn't to treat the rankings as gospel, but to match your genre and current workflow stage to the right tool:

• Short fiction hooks and overall completion: look at Qwen
• Outlines, architecture, and manuscript review: look at GPT 5.6
• Deadline drafting and idea divergence: look at Gemini
• Emotional atmosphere and climax refinement: look at Claude
• Contemporary female-oriented romance body text: try GLM 5.3

The reviewers' scores help you narrow the field — but the real test is generating a passage from your own manuscript, then checking whether the model honored your rules, characters, and conflicts. No model is universally superior. There's only the one that fits what you're writing.