Local Creative Writing Model Showdown: Fine-tuned Qwen3.8 27B Wins, DeepSeek v4.1 Underwhelms

Is it actually reliable to write web novels and copy with locally deployed open-source models? A Bilibili creator recently ran a head-to-head test, using three creative writing prompts to evaluate seven models based on Qwen3.8 27B—including popular community fine-tunes and the official base model—along with DeepSeek v4.1 for comparison. The results were somewhat surprising: the Hemingway model, purpose-built for literature, took first place, the chat-oriented HumanLike Chat came in second, while the much-anticipated DeepSeek v4.1 performed mediocrely and was even described as "plain boiled water."

Testing Method and Scoring Criteria

The test selected six community fine-tuned models, all based on Qwen3.8 27B: Hemingway, HumanLike Chat, Alexs WinX (emphasizing high EQ), DavidAU Brandings (Spring and Autumn King), Dongfang Shuo (specialized for Chinese web novels), and the official base Qwen3.8 27B as a benchmark. The seventh was DeepSeek^Ad v4.1 (official 1017 version).

The three prompts each tested different abilities:

• Prompt 1: Scene narrative—two painter friends, one holding a solo exhibition, the other helping hang paintings. The task required conveying the complex emotion of "happy yet slightly jealous" through actions, dialogue, and details, without stating it directly.
• Prompt 2: Dialogue creation—roommates of three years are about to part ways, writing 12 lines of dialogue around who gets the rice cooker. No narration or inner monologue allowed; readers must feel that neither wants the other to leave.
• Prompt 3: A complete 600-word short story—given a title or scene, requiring a complete narrative structure, effective emotional rendering, and no errors in character relationships.

In terms of scoring weights, Prompt 3 accounted for 50%, and the first two prompts 25% each. The creator explained that Prompt 3 demands the most in narrative closure, emotional rendering, and character consistency, and best exposes the gaps between models.

Fine-tuning refers to continuing to train an already pre-trained base model on data from a specific domain or style, so that the model acquires the expression habits and knowledge of that domain. In this article, all six community models were fine-tuned on the Qwen3.8 27B base, so their underlying language capabilities are similar; the differences mainly come from the style and quality of the fine-tuning data. This also explains why stylized models like "Hemingway" and "HumanLike Chat" can outperform the official base model on literary tasks—fine-tuning is like giving the model "extra tutoring," reinforcing output preferences for specific scenarios.

27B parameter scale means the model has about 27 billion learnable parameters, placing it in the medium-to-large range of open-source models. This scale is sufficient to support fairly fluent Chinese generation, but limited by context window and VRAM, it cannot output extremely long text (like a million-word novel) in one go; it usually requires segmented generation, manual stitching, and proofreading. Parameter scale also directly affects the hardware threshold for local deployment; a 27B model typically requires 24GB or more of VRAM, or running after quantization.

Prompt 1: Hemingway Wins, Showing Skill in Detailed Narrative

Prompt 1 required writing about "happiness and jealousy while hanging paintings," without directly stating the emotions. The Hemingway model scored 90 points, ranking first among the seven models.

The creator showed Hemingway's passage: Lao Zhou helps his friend A Zhe hang paintings. An audience member asks A Zhe, "Did you study painting since childhood?" Lao Zhou jumps in, "We both started as kids," then grins and straightens a chair that wasn't even crooked. At the end, the crowd surges toward A Zhe, and Lao Zhou stands still without moving. These actions and details convey both "happy for his friend" and "faint jealousy" without a single direct statement.

HumanLike Chat ranked second, using a similar flashback technique: the friend says with a smile, "You've been staring at my painting for three days," using a teasing line to reveal the guilt, but the ending isn't as tight as Hemingway's. The official base model ranked third, balanced but lacking highlights. DeepSeek v4.1 ranked fifth, described as "decent writing, but an average ending." DavidAU Brandings (Spring and Autumn King) performed acceptably on this prompt but didn't make the top three overall.

Creator's comment: Hemingway stands out in local character description and dialogue, but the story's completeness is slightly lacking.

Prompt 2: HumanLike Chat Shows EQ, Natural Dialogue

Prompt 2 tested the ability to write dialogue that "doesn't say it outright." Hemingway was still first, but HumanLike Chat's high-EQ performance was impressive.

In HumanLike Chat's dialogue, Person A says, "Let me have the pot—I need to make porridge tomorrow morning," and Person B replies, "Then your inner pot is big; cooking a whole pot for one person leaves half left over." A says, "Your kitchen stove is small, it won't fit," and B responds, "We'll squeeze it in. We've lived together three years—what haven't we squeezed through?" A few everyday lines, both trying to hold on, yet neither says it outright. The creator commented, "The EQ is indeed good, the dialogue natural, the writing restrained."

The official base model ranked third—dialogue reasonable but lacking emotional tension. DeepSeek v4.1 was unremarkable on this prompt: no obvious errors, but it didn't move the reader. Dongfang Shuo had confused character relationships on this prompt and ranked low.

Prompt 3: HumanLike Chat Takes a Surprise First, but Character Errors Are Widespread

Prompt 3 was the 600-word complete short story, carrying the highest weight and also generating the most mistakes. HumanLike Chat took first place with a story about an old man, Shen Yan, who, after his wife's death, misses the 8 o'clock bus for the first time. The text uses "waiting for the bus" to link three layers of waiting: his wife waiting for him while alive, him waiting for his wife, and his daughter waiting for him. Both emotional rendering and narrative structure were praised.

But the creator also pointed out that the piece had a character reference error at the end—"before mother died" should be "before wife died," and the daughter standing at the door looking out was described as "exactly like how mother used to stand by the window waiting for him." This error wasn't unique to HumanLike Chat; the other 27B models all showed similar problems to varying degrees.

Hemingway ranked second on this prompt, also writing about waiting for the bus; the narrative was complete but the emotional impact was slightly weaker. The official base model ranked third. DeepSeek v4.1's story was described as "too plain, like a running account"—the daughter takes the bus, arrives at the stop, picks up the cake—nothing that moved the reader, resulting in a low score. Dongfang Shuo had the most errors, with confused characters and jumbled structure, ranking last among the seven models.

Character reference errors are a typical problem in long-text generation, referring to the model confusing character identities, titles, or relationships in the narrative—for example, writing "wife" as "mother." Such errors are common in 27B-level models, rooted in the model's lack of explicit memory and consistency-checking mechanisms for character relationships across the entire text. During segmented generation, each segment is reasoned about independently, and the model easily "hallucinates" titles based on local context, leading to contradictions before and after. Mitigation methods include: fixing a character relationship table in the prompt, manually checking titles after segmentation, and using models that support long contexts or external memory tools as aids.

Overall Rankings and Usage Recommendations

Combining the three prompts, the final ranking is as follows:

1. Hemingway—first overall, outstanding character description and dialogue
2. HumanLike Chat—second overall, best emotional rendering and dialogue naturalness
3. Official base Qwen3.8 27B—balanced but unsurprising
4. DavidAU Brandings (Spring and Autumn King)—acceptable on Prompt 1, overall not as good as the base
5. Alexs WinX—high-EQ label, but average creative writing performance
6. DeepSeek v4.1—no obvious errors, but lacking highlights
7. Dongfang Shuo—specialized for Chinese web novels, but insufficient basic literary competence, with severe character confusion

The creator's practical advice: if you want to use local models to assist creative writing, Hemingway and HumanLike Chat are worth trying first, with balanced chat and writing performance. DavidAU and Alexs WinX may be good at reasoning and agents, but creative writing isn't their strength. Although Dongfang Shuo emphasizes web novel optimization, its basic narrative ability falls short and character relationships easily become confused, making long web novels risky.

Another important reminder: 27B-level models cannot generate a million-word novel in one go; they can only generate in segments, and multi-character relationships require manual control. The creator emphasized: "You can't leave it entirely to the model—after writing, you may need to make certain revisions."

A Brief Observation

The sample size of this test is small, and three prompts can hardly cover all dimensions of creative writing, but it reveals a noteworthy phenomenon: models with strong base capabilities (like the official base) aren't necessarily more impressive at creative writing than specialized fine-tunes, and chat-oriented models actually have an advantage in emotional expression. DeepSeek v4.1 has always had a good reputation for Chinese ability, but in tasks requiring emotional tension and narrative skill, its "safe play" style becomes a shortcoming.

For users who want to deploy locally for web novels and copywriting, model selection is only the first step; prompt design, segmentation strategy, and manual polishing are equally critical. The creator put the download links for Hemingway and HumanLike Chat in the pinned comment—those interested can try them out.