Synthadoc

Synthadoc is an open-source AI-native wiki engine whose job is to synthesize raw documents such as PDFs, web pages, and transcripts into structured Markdown wiki pages via an LLM.

Synthadoc's content health scanner deliberately uses pure Python heuristic rules, does not rely on an LLM, and identifies structural problems such as duplicate sections, empty sections, thin sections, and unclosed code blocks through deterministic regular expressions.

In Synthadoc's content health workflow, the LLM reads the page, understands the structural problems, produces revision suggestions, and ultimately affects persisted state (writing to the wiki).

The Synthadoc team designed a five-step workflow: load the page and the issue list, send to the LLM for processing, show a before-and-after comparison, wait for user approval, and take a snapshot before writing.

Synthadoc added the `_is_draft()` regex to detect linguistic markers of incomplete thinking (such as TODO, FIXME, Expand to, Hmm,, Wait,, I'll, Let me, etc.); if there is a match, the proposed content is marked and the page is automatically skipped.

Synthadoc set a hard skip at six issues for the CLI provider path: pages with more than six structural issues are skipped for repair, while the API provider takes a separate code path with no such limit.

Every proposed fix in Synthadoc must pass a before-and-after comparison manual gate review, where the confirmation card displays the page slug, the issue summary, the first 2000 characters of the original content, the first 2000 characters of the proposed content, as well as approve/skip buttons and a five-minute timeout that automatically rejects.

The Synthadoc team divides failures of Agentic systems into two categories: category A drift (the model understands the task but applies wrong judgment) and category B hallucination (the model substitutes its own knowledge for the task itself).

Synthadoc introduced the `_extract_md_from_response()` preprocessing function, which strips everything before the first `#` H1 heading in the response and recovers the Markdown content using the first H1 as a clean boundary.

In failure one observed in Synthadoc, the 'fixed' content the LLM returned for a page about the open-source movement had no structural similarity to the original text; the model rewrote a wiki page using its own pretrained knowledge.

In failure three observed in Synthadoc, when the content health workflow was routed through CLI-based agent tools (such as opencode or Claude Code), the response text contained shell command output and traces of internal tool calls, and the model bypassed the constructed context to read the actual files on disk directly.

In failure five observed in Synthadoc, the model silently 'corrected' the year in the original title '## The Cathedral and the Bazaar (1999)', deleting the year because it believed the article was actually published in 1997.