No More AI Hallucinations: How One Developer Built a Cover Letter Generator That Can't Lie

When AI Cover Letters Start "Lying for You"

A developer shared their experience on Reddit: last month, they tried three different AI cover letter generators — and every single one fabricated things out of thin air. One told recruiters they had "5+ years of Kubernetes production experience," when they had never touched Kubernetes in a production environment. Another claimed they had "led a team of 40 engineers," when they had never managed anyone.

This kind of hallucination might make a résumé look more impressive at first glance, but it's a ticking time bomb — any follow-up question in an interview could expose the lie immediately. So the developer decided to build their own tool: CoverCraft, built around one core principle: the AI cannot claim anything that cannot be traced back to the actual text of your résumé. If it's not in your résumé, it won't appear in your cover letter.

Four Engineering Decisions That Make the AI "Afraid to Lie"

What's genuinely interesting here isn't the idea of building a cover letter generator — it's the four key engineering trade-offs the author made to combat the LLM's tendency toward hallucination.

1. Match Scoring Stays Out of the LLM's Hands

Most tools let the model spit out a random "95% match" score — a number that's neither reproducible nor trustworthy. The author separated the task: the LLM is only responsible for classification (STRONG_MATCH / PARTIAL / MISSING), while the final score is calculated by regular application code using a fixed formula:

score = (strong*1.0 + partial*0.6 + transferable*0.4) / total

This means that for the same résumé paired with the same job description (JD), the score will be identical every time — no randomness whatsoever. Leaving "judgment" to the model and "calculation" to the code is a separation-of-concerns approach worth borrowing.

This "judgment vs. calculation" separation maps directly to the Separation of Concerns principle in software engineering. LLMs are fundamentally probabilistic samplers — their outputs are influenced by the temperature parameter, and the same input can produce different results each time. Delegating numerical computation to a model is like using dice instead of an abacus: even if the result looks precise, there's no auditable logical chain behind it. By contrast, letting the model handle qualitative classification while handing off quantification to deterministic code, you get the best of both worlds: the model's semantic understanding + the reproducibility of code. This is also why in high-compliance domains like financial risk management or medical diagnosis, AI is typically designed to "assist human decision-making" rather than "directly output conclusions" — clearly defined responsibility boundaries are themselves a source of reliability.

2. GitHub Commit History as Proof of Skills

Descriptions like "architected a multi-node agent system" on a résumé sound impressive, but they're hard to verify. CoverCraft uses the GitHub MCP tool to pull real commit history from a user's public repositories, anchoring capability claims to specific commit hashes. If you claim expertise in Kubernetes but your commit history shows no trace of it, the system flags that discrepancy directly — rather than letting the LLM smooth it over.

GitHub MCP (Model Context Protocol) is an open protocol introduced by Anthropic in late 2024, allowing LLMs to call external tools and data sources through a standardized interface without requiring custom integration logic for each tool. In CoverCraft's architecture, a Python MCP server acts as a middleware layer that communicates with the GitHub API, pulls repository metadata and commit history, and passes structured results to the language model. The key advantage of this architecture is "fact offloading": the model no longer relies on knowledge memorized during training, but instead queries the user's real, existing digital footprint in real time. Commit hashes serve as immutable, timestamped records with built-in auditability — anyone can access the corresponding public repository to verify them. This is the key mechanism that elevates "capability claims" from subjective descriptions to verifiable evidence.

3. Adversarial "Red Team" Review to Catch Exaggerated Language

After generation, an adversarial review module kicks in and "downgrades" inflated verbs: for example, "spearheaded" becomes "led implementation of," and "pioneered" becomes "developed" — provided the original résumé doesn't support that level of claim. This acts as a "remove the fluff" gate on the output.

The term "Red Team" originates from Cold War military war-gaming exercises, where a dedicated team would play the attacker to probe defensive vulnerabilities. In AI safety, red-teaming has become a standard step before releasing large models — a group specifically tries to coax the model into producing harmful or incorrect outputs, thereby evaluating the model's safety boundaries. CoverCraft internalizes this concept into an automated process: one model instance generates the cover letter, while a second model instance is explicitly instructed to act as a "critical reviewer," specifically looking for exaggerations between the output and the original résumé. The two model instances are adversarial rather than cooperative — a classic application of Adversarial Collaboration in multi-agent architecture, and an effective engineering approach for improving the trustworthiness of generated content.

4. Human Approval Checkpoint

The tool conducts company research via real-time web search, but that information is not automatically injected into the prompt. It's presented to the user as "cards" first, and you approve or reject each item individually — eliminating any possibility of information quietly slipping into the generated content.

Tech Stack: Runs Entirely on Free Tiers

The author published the full technology stack: Next.js + Python MCP server + Gemini (with fallback and circuit breaker logic for 429 rate limiting) + Tavily search + Langfuse for tracing. The entire system runs on each provider's free tier, with an operating cost of $0.

Worth noting: this project, registered as an independent MCP server, was discovered and listed by M8ven's MCP Trust Index with a "Verified Publisher" designation, including real-time monitoring — the author says they never submitted it proactively; M8ven's crawler found and audited it automatically. This provides a degree of third-party validation for the project's authenticity.

Langfuse is an open-source LLM observability platform designed specifically for tracing, debugging, and evaluating the complete call chain of language model applications. In an AI application, a single user request often triggers multiple model calls, tool calls, and data queries — without dedicated tracing tools, diagnosing issues or optimizing performance is nearly impossible. Langfuse records the inputs and outputs, latency, token usage, and cost of each call, and supports organizing multiple calls into a visualized "Trace" tree structure. For a multi-step agent system like CoverCraft — involving résumé parsing, GitHub data retrieval, match scoring, generation, and red-team review — call tracing is not only a debugging tool but also the data foundation for future iteration. Langfuse offers a free hosted tier, consistent with the zero-cost design philosophy of the entire stack.

A Product Philosophy Question With No Right Answer

At the end of the post, the author raises two thought-provoking questions that cut to the heart of the project's value proposition.

The first: Do recruiters actually read cover letters, or has this format become a hollow formality? This touches on the fundamental purpose of cover letters — if recruiters never read them, even the most honest generator is solving the wrong problem.

The second is sharper: when a résumé has skill gaps, the author chose to "honestly acknowledge the gap" rather than simply skip over it. They're even questioning whether this was a bad decision. From a game-theory perspective, admitting a shortcoming might come across as refreshingly genuine — or it might get you filtered out in the first screening pass. There's no right answer; it depends on whether you're facing a recruiter who values integrity or an ATS system doing pure keyword matching.

Implications for AI Product Design

Beyond the job-hunting context, CoverCraft's greatest value as a reference lies in how it demonstrates a systematic approach to constraining generative AI hallucinations: separating deterministic calculations from the model, anchoring facts to verifiable external data sources (GitHub commits), filtering exaggeration through adversarial secondary review, and maintaining human approval as a final checkpoint.

This design paradigm of "don't trust the model, leave evidence everywhere" applies to any AI application where factual accuracy matters — whether it's legal documents, medical advice, or financial analysis. While the industry is still caught up in the "bigger model = better" frenzy, engineering-discipline-driven approaches like this one for combating hallucinations may be exactly what production AI applications actually need.