Carta: Pandoc Rewritten in Rust — 45x Faster, 20x Smaller

A Performance-Oriented Rewrite of a Document Converter

Document conversion is a fundamental need in technical writing, academic publishing, and content engineering, and Pandoc has long been the de facto standard in this space. Created in 2006 by John MacFarlane (jgm), a philosophy professor at UC Berkeley, it embodies decades of expertise in document processing and functional programming, capable of freely converting between dozens of formats including Markdown, DOCX, LaTeX, and HTML. However, power comes at the cost of binary size and runtime overhead.

Recently, a developer named mfkrause posted his new project Carta on Reddit — a reimplementation of Pandoc in Rust. Its goal is crystal clear: retain Pandoc's core workflow while being smaller and faster. According to the author, Carta's compiled binary is approximately 1/20th the size of Pandoc's, with conversion speeds up to 45x faster.

Why Rewrite Pandoc in Rust

The author openly states that he holds Pandoc in very high regard, but two specific issues have bothered him for a long time.

Bloated Binary Size

Pandoc's build artifact on Apple Silicon is approximately 180 MB. For a command-line document conversion tool, this is quite substantial — especially when you need to package it into container images, CI/CD pipelines, or embed it in other applications, where it creates significant distribution and storage burden. Carta compresses this to roughly one-twentieth, bringing it down to approximately 9 MB.

The core reason for the size difference lies in the underlying language runtime. Pandoc is written in Haskell, and its compiler GHC (Glasgow Haskell Compiler) statically links the runtime system (RTS) into the binary when generating executables. This runtime includes a garbage collector, lightweight thread scheduler, exception handling mechanisms, and other components — even if the program doesn't need concurrency features, this infrastructure still takes up considerable space. Additionally, Haskell's rich standard library and Pandoc's numerous third-party dependencies further inflate the final artifact under static linking. In contrast, Rust's compiled output includes no GC runtime, has a lean standard library, and the LLVM backend's Link-Time Optimization (LTO) can effectively strip unused code.

Document Conversion Speed

Pandoc's execution speed is acceptable for single-document scenarios, but when you need to batch-convert large numbers of documents, the performance bottleneck becomes apparent. Carta leverages Rust's zero-cost abstractions and efficient memory management to achieve up to 45x speed improvements. This is especially critical for automated scenarios processing thousands of documents, such as static site generation and document archival batch processing.

Rust's "Zero-Cost Abstractions" mean that high-level abstractions used by developers — generics, iterator combinators, pattern matching, etc. — generate no more runtime overhead after compilation than hand-written low-level code. The Rust compiler uses monomorphization to expand generic code into specialized versions for concrete types, eliminating the cost of dynamic dispatch. Meanwhile, Rust's ownership system and borrow checker complete memory safety verification at compile time, eliminating the need for a garbage collector to track object lifetimes at runtime. This means the program enters its working state immediately upon startup, with no GC pauses (stop-the-world pauses), and the timing of memory allocation and deallocation is entirely deterministic. For document conversion scenarios that frequently create and destroy AST nodes, the performance advantage of this deterministic memory management is particularly significant.

Interestingly, Pandoc is written in Haskell, and its runtime and GC mechanisms affect startup speed and memory usage to some degree. GHC's runtime employs a generational garbage collection strategy that performs excellently for long-running services, but for short-lived command-line tools with a "start-process-exit" lifecycle, the overhead of runtime initialization and GC becomes a larger proportion of total execution time. When batch-converting thousands of files, each invocation repeatedly pays this startup cost, and the cumulative effect is considerable. Rust's GC-free, compile-time deterministic approach gives Carta a natural advantage in both dimensions. This is yet another example of the recent trend of rewriting foundational tools in Rust.

Carta's Architecture: Following Pandoc's Philosophy

Carta's underlying working principle is highly similar to Pandoc's, which is the key to maintaining compatibility.

The core workflow is: a reader parses the input document into an internal abstract syntax tree (AST), then a writer outputs this AST in the target format. This decoupled design enables combinatorial conversion capability — theoretically the total number of supported conversions equals reader_formats × writer_formats, meaning any supported input format can be converted to any supported output format.

This "Intermediate Representation" (IR) design philosophy originates from compiler theory. In classic compiler architecture, the frontend parses source code into IR, and the backend translates IR into target code. The core advantage of this layered design is solving the "M×N problem": if there are M input formats and N output formats, direct pairwise implementation requires M×N converters; but with an intermediate representation, you only need M readers and N writers, reducing total effort to M+N. The LLVM project is the most successful industrial-scale implementation of this philosophy — it defines LLVM IR as an intermediate language, so any new language only needs to implement a frontend to gain support for all backend platforms. Pandoc brought this compiler design philosophy into the document conversion domain, defining a rich set of document AST types (including paragraph, heading, list, table, code block, quote, metadata, and other node types) to preserve semantic information between formats to the greatest extent possible.

Carta follows the same philosophy, meaning that when extending to new formats in the future, it only needs to add the corresponding reader or writer without implementing conversion logic separately for each pair of formats.

Implemented Features and Pandoc Compatibility

Carta currently covers the most commonly used formats and features:

• Markdown family: Reading and writing multiple Markdown dialects
• DOCX: Word document conversion
• TeX / LaTeX: Formats commonly used in academic writing
• Syntax highlighting: Code block colorization
• JSON filters: Fully compatible with Pandoc — the existing Pandoc filter ecosystem can be directly reused
• Standalone mode: Generating complete standalone documents rather than fragments

Among these, full JSON filter compatibility is a particularly important design decision. Pandoc's filter mechanism is one of the most powerful extension capabilities in its ecosystem. Its working principle embodies the Unix pipe philosophy: after Pandoc parses a document into an AST, it serializes it in JSON format and passes it to an external filter program via standard input (stdin). The filter modifies the JSON-form AST and returns it via standard output (stdout), after which Pandoc renders the modified AST into the target format. Since filters are independent processes, they can be implemented in any programming language — the Python community's panflute and pandocfilters libraries are the most commonly used filter development frameworks. Typical filter use cases include: automatic numbering of figures and equations, rendering LaTeX formulas as images, replacing custom macros, and inserting cross-references.

Carta maintaining this compatibility means the entire existing filter toolchain can migrate seamlessly, significantly lowering the barrier to switching from Pandoc.

What's Not Yet Supported

As a project still in its early stages, Carta also clearly lists its current missing features:

• PDF output: Pandoc itself doesn't directly generate PDFs — it first generates LaTeX source files, then invokes TeX engines like pdflatex, xelatex, or lualatex for typesetting and compilation. An alternative path goes through HTML intermediate format using tools like wkhtmltopdf or Prince for rendering. Either path involves complex external dependency management, template systems, font handling, and typesetting engine integration, making implementation quite complex.
• Lua filters: Lua filters were an important improvement introduced in Pandoc 2.0. Unlike JSON filters that require launching external processes, Lua filters run inside the Pandoc process through an embedded Lua 5.4 interpreter, directly manipulating AST objects and avoiding the overhead of JSON serialization/deserialization and inter-process communication — their performance is significantly better than JSON filters. Many modern Pandoc workflows (such as academic paper template systems and technical documentation generation pipelines) deeply depend on Lua filters. For Carta, implementing Lua filters means integrating a Lua runtime and implementing a complete AST binding API, which is no small engineering effort.

These two features happen to be heavily relied upon by advanced Pandoc users. Therefore, at this stage, Carta is better suited as a replacement for high-frequency, lightweight, batch conversion scenarios rather than a complete Pandoc replacement. The author also provides a comprehensive feature comparison table in the project for users to judge whether it meets their needs.

Open Source License and Ecosystem Positioning

Carta is released under a MIT or Apache-2.0 dual license, which is standard practice in the Rust community and extremely friendly to both commercial and open-source use. The project is hosted on GitHub (mfkrause/carta), and the author is actively soliciting community feedback.

In terms of positioning, Carta is unlikely to replace the fully-featured Pandoc in the short term, but it fills a genuine need: running document conversions in environments sensitive to binary size and speed. CI pipelines, edge computing, desktop application embedding, and content engineering requiring batch processing are all potential use cases.

Conclusion

Carta is yet another典型 case of the "rewrite classic tools in Rust" trend. In recent years, the Rust community has produced numerous such projects: ripgrep (replacing grep, typically 2-10x faster), fd (replacing find), bat (replacing cat with built-in syntax highlighting), exa/eza (replacing ls), zoxide (replacing cd), delta (replacing diff), and more. At the larger infrastructure level, this trend extends into the JavaScript toolchain space — SWC (replacing Babel), Turbopack (replacing Webpack), Biome (replacing ESLint + Prettier) — as well as uv in the Python package management space (replacing pip/poetry). All are successful cases of rewriting traditional toolchains in Rust to achieve order-of-magnitude performance improvements. These projects share several common characteristics: compiled into dependency-free static binaries for easy distribution, extremely fast startup, controlled memory usage, and good cross-platform support.

Carta doesn't try to surpass Pandoc in features. Instead, it chose a clear differentiation path — through a more modern language implementation, delivering order-of-magnitude improvements in performance and distribution cost while maintaining compatibility with the existing ecosystem as much as possible.

For developers pursuing maximum efficiency whose current needs fall within the supported format range, Carta is worth trying. And as features like PDF output and Lua filters are gradually filled in, it has the potential to become a truly competitive lightweight option in the document conversion space.