Is the Gemini Model Too Wordy? NeuralYap Benchmark Reveals AI "Chatterbox" Rankings

A YouTuber launched the NeuralYap benchmark, using four dimensions to quantify how verbose large models are.
A YouTuber launched a custom benchmark called NeuralYap to specifically measure how verbose large models are. The benchmark relies on no external frameworks or system prompts and is based solely on raw API responses to capture models' most authentic chatterbox tendencies. The verbosity index consists of four dimensions—phrasing, typography, formatting, and length—turning vague subjective impressions into comparable metrics. Test results show the Gemini series ranks high, with Claude Opus 5 taking the "number one chatterbox" title. The article notes that this phenomenon reflects the interaction cost brought about by alignment training that encourages politeness and thoroughness, reminding developers that evaluating a model can't just look at the capability ceiling but must also consider the communication burden in actual use.
A Benchmark Designed to Measure "Annoyingness" Is Born
While most evaluations are still competing over scores and capabilities, one YouTuber took the opposite approach, launching a custom benchmark called NeuralYap that specifically measures how "annoying" a model is—in other words, how verbose it gets when you talk to it. According to the video, this test uses no external frameworks, injects no system prompts, and only looks at responses from raw API calls, attempting to capture the model's most authentic "chatterbox" tendencies.

This approach hits on a real pain point many developers face: no matter how capable a model is, if it responds with a whole essay every time, the user experience gets dragged down. NeuralYap breaks this subjective feeling into quantifiable dimensions, giving "annoyingness" a leaderboard for the first time.
Raw API calls refer to developers sending requests directly to a model provider's interface, without any intermediary layer (such as orchestration frameworks like LangChain) and without system prompts or preset conversation templates. Under these conditions, the model's replies are determined entirely by the user message and the model's own default behavior, reflecting the "factory style" shaped by the vendor during training and alignment.
By contrast, calls constrained by system prompts can make a model behave concisely and restrained—but that is "corrected by instruction" behavior, not the model's default tendency. Using raw API calls as a benchmark therefore more authentically exposes how a model talks when unsupervised, and gets closer to what ordinary users experience when using chat products directly.
Breaking Down "Yap" Across Four Dimensions
NeuralYap doesn't simply count words; instead, its "verbosity index" is composed of multiple factors working together. According to the video's creator, it mainly includes the following:
- Phrases: How annoying the model's phrasing is—for example, templated expressions like "You're absolutely right," "You have every reason to feel frustrated," "You have every reason to push back," "The honest answer is," and so on.
- Typography: The density of elements like em dashes and emoji.
- Formatting: Whether the response looks like a long Markdown article.
- Length: How long the response itself is.

The value of this breakdown lies in turning "verbosity" from a vague intuition into a comparable metric. Different models may fail on different dimensions: some are oily in their phrasing, some are flashy in formatting, and some are simply too talkative.
System prompts are high-priority instructions injected before a conversation begins, used to set the role, tone, and behavioral boundaries—for example, "keep answers as short as possible" or "don't use emoji." They are independent of user messages and, in most APIs, carry higher weight than ordinary dialogue.
System prompts are the most direct means developers have to control a model's verbosity, but they can only constrain behavior—they can't change the model's own default tendencies. When system prompts are missing or weakened, the model falls back to the style it learned during training, which is exactly why NeuralYap deliberately strips system prompts away: it's not measuring "can you make it shut up," but "how much does it love to talk in the first place."
Gemini Models Rank High, Claude Opus 5 Takes the Top Spot
The test results aren't surprising, yet they're worth pondering. According to the video's creator, the Gemini series models rank high in NeuralYap, and a higher rank in this benchmark means more verbose and more annoying. The creator bluntly called this result "pretty bad."

More surprising is the top spot: Claude Opus 5 took the title of "number one chatterbox." The creator believes most people would agree with this judgment. Gemini models also rank high, showing that "chatterbox" is not the exclusive domain of any one model—it's a general tendency in the conversational style of current large models.

This ranking has practical reference value for developers: if you're building a conversational product sensitive to response conciseness, NeuralYap's dimension breakdown can serve as a basis for selecting models and designing post-processing logic.
Alignment training refers to using methods such as reinforcement learning from human feedback (RLHF) and Constitutional AI to make model outputs better match human preferences, including goals like being helpful, harmless, and honest. During this process, annotators typically give higher scores to answers that are "well explained, polite, and thoughtful," and models therefore acquire verbose expression habits.
The problem is that there is tension between alignment goals and interaction efficiency: an answer that scores higher in evaluations is not necessarily an answer users want to read in a continuous conversation. Verbosity is essentially a side effect of alignment training, which also explains why models from different vendors collectively fail on NeuralYap.
The Product Thinking Behind "Chatterbox"
Although NeuralYap is a personal benchmark with a teasing tone, it points to a serious issue: model alignment training often encourages being "polite, thorough, and well explained," and when these traits stack up, they become verbosity that exhausts users.
From an engineering perspective, this benchmark also reminds us that evaluating a model can't just look at the capability ceiling—it must also consider interaction cost. Performance under raw API calls deserves particular attention because it strips away the constraints of system prompts and exposes the true face of a model's default style. For products pursuing concise interaction, this may be closer to user experience than any benchmark score.
Related articles

GPT-6.2 Astra and Sol Released: Dual-Model Architecture and Community Buzz
OpenAI releases GPT-6.2 with Astra and Sol models. This article summarizes the official announcement and Hacker News discussion, analyzing the potential impact of the dual-model architecture on developers and offering selection advice.

Gmod TTT Egyptian-Themed Map: Space Capitalism vs. the Pharaoh
Explore the fun of a Gmod TTT Egyptian-themed map: players pick roles like pharaoh, teleport to space via a cube, and engage in a humorous capitalism vs. communism showdown with chaotic prop fighting.

GraphRAG: Beyond Search, Toward Global Knowledge Discovery
GraphRAG is a knowledge-graph-enhanced retrieval technology from Microsoft Research that goes beyond local search in traditional RAG to enable global knowledge discovery. This article introduces its core ideas, generational evolution, and healthcare use cases.