Are Machine Learning Papers Getting Too Wordy? A Discussion on Academic Writing

When Papers Run 40+ Pages: The 'Bloat' Anxiety in ML Research

A researcher sparked a widely resonant discussion on Reddit with a pointed question: has machine learning research become too verbose? They admitted finding modern ML papers increasingly hard to digest — primarily because of dramatic page count inflation. Papers running 20, 30, or even 40+ pages of pure text are no longer unusual.

The poster cited Meta's LLaMA paper (arXiv:2302.13971) as an example, acknowledging that LLMs inherently require extensive coverage of training details, datasets, and evaluation results — but arguing that this tendency toward "text padding" has spread well beyond the LLM domain, even infecting traditionally math-heavy fields like optimization.

The discussion touched a real nerve in the academic community: as paper counts and page lengths keep climbing, the cost of extracting useful information is going up, not down.

Conversational Language Invading Scientific Writing

The poster raised a second concern: the shifting style of scientific writing itself. They singled out Anthropic's influential technical blog post A Mathematical Framework for Transformer Circuits (transformer-circuits.pub) as a case in point. While the piece does incorporate mathematical notation, its writing style struck the poster as somehow off.

The issue: the authors liberally mixed casual, conversational phrasing into a serious technical publication — expressions like "a lot of...", "we don't feel...", and "it feels like..." appear frequently. In this reader's view, such language undermines both the rigor and the readability of the work. Scientific conclusions should be grounded in verifiable arguments, not subjective "feelings."

This reflects a deeper tension: as research increasingly relies on empirical findings and intuition-driven exploration — especially in interpretability and large model behavior analysis — authors often cannot provide rigorous mathematical proofs and must resort to descriptive language to convey their observations. This is partly an honest reflection of where the field stands, and partly a symptom of gaps in academic writing training.

A Mathematical Framework for Transformer Circuits (2021) is a foundational work by Anthropic's interpretability team that attempts to analyze the internal attention mechanisms of Transformers using linear algebra, introducing conceptual frameworks such as the "residual stream" and "attention head composition." Notably, it was published as a technical blog post rather than a traditional journal paper — a choice that itself reflects the shifting publication culture in ML, where an increasing volume of important work bypasses peer review in favor of preprints (arXiv) or institutional blogs. The blog format naturally permits a more narrative writing style, allowing authors to freely express intuitions and uncertainties encountered during exploration. But this blurs genre boundaries: is the reader looking at a scientific claim or a reflective log? When this style bleeds back into formal academic papers, it produces exactly the discomfort the original post described.

Math Disappearing, Block Diagrams Proliferating

The most technically substantive critique in the thread concerned a regression in how ideas are expressed. The poster argued that in certain research areas, rigorous mathematical exposition is becoming increasingly rare, replaced by an over-reliance on block diagrams.

The problem: while block diagrams are intuitive, they inherently omit a large number of implementation details. Translating a method accurately into mathematical formulas or runnable code from an architecture diagram alone is nearly impossible — it typically requires a great deal of guesswork and reverse engineering. This poses a real barrier to reproducing research and verifying results.

This "diagram over equation" trend is partly a byproduct of deep learning's engineering culture — when models are assembled from modular components, block diagrams are the most natural communication language. But for researchers who prize precision and reproducibility, papers that lack mathematical specifications and pseudocode simply offload the burden of reconstruction onto the reader.

Ablation studies are the standard method in modern deep learning papers for validating the contribution of individual components: by removing or replacing a module one at a time and observing performance changes, authors quantify each component's necessity. However, the credibility of ablation studies depends heavily on precise mathematical descriptions. If a paper provides only block diagrams without formal definitions of each module, readers often cannot determine whether a "same-named" component in two different papers refers to the same operation. This is the hidden cost of diagram dependency: when researchers try to reproduce an ablation setup in their own work, vague visualizations lead to implementation drift, rendering cross-paper comparisons meaningless. The reproducibility crisis in deep learning is closely tied to this issue — top venues like NeurIPS and ICML have responded institutionally by introducing mechanisms like the "Reproducibility Checklist," requiring authors to explicitly report hyperparameters, random seeds, and code implementations.

Several Structural Reasons Behind Page Bloat

Setting aside personal preferences about writing style, there are objective reasons why ML papers have gotten longer — and these deserve a fair hearing:

Heavier Experimental Burden

Modern ML papers are expected to compare against a large number of benchmarks, with accompanying ablations, hyperparameter analyses, and multiple baselines. Simply reporting and analyzing these experiments can easily consume a dozen pages.

The Rise of Appendix Culture

Many conferences impose page limits on the main body, so authors pack extensive details into appendices. A "40-page paper" may have more than half its content in appendices containing proofs, additional experiments, and implementation details. In a sense, this actually reflects improved transparency.

Page limits at major ML venues have a long history: NeurIPS and ICML typically cap the main body at 8–9 pages (excluding references), with ICLR following similar conventions. These rules were originally meant to reduce reviewer burden and keep papers focused — but in practice they have spawned substantial appendices, with some papers' appendices running two to three times the length of the main body. More notably, appendix content typically falls outside reviewers' mandatory reading scope, meaning a large amount of technical detail lives outside the peer review process. From an information-completeness standpoint, an appendix is certainly better than "details omitted" — but when core proof assumptions and critical data processing steps are squeezed into the appendix, readers must invest far more time than an "8-page paper" would suggest to form a complete judgment, defeating the original purpose of the page limit.

Expanding Cross-Domain Scope

Large model research frequently spans data engineering, training infrastructure, evaluation methodology, and other subfields — each requiring its own treatment, which naturally drives up page count.

Progress or Regression?

This discussion has no clean answer, but it illuminates a real divide forming within the ML research community. On one hand, longer papers with more experimental detail and fuller appendices genuinely improve the completeness and reproducibility of research. On the other hand, conversational writing and diagram dependency have eroded rigor and information density in meaningful ways.

For readers, a pragmatic approach is: start with the abstract and conclusion, then examine the key figures and equations, treating lengthy prose as supplementary rather than primary. For writers, the value of this thread is as a reminder — page count does not equal contribution, and clear mathematical exposition with well-specified pseudocode is almost always more valuable than thousands of descriptive words.

At its core, the goal of academic writing should be to transmit knowledge efficiently, not to demonstrate effort. When papers increasingly read like narratives rather than arguments, perhaps the entire community should ask itself: are we advancing science, or adding noise?