LedgerMind AI: Turning Voice Notes into a Trustworthy Ledger — AI Only Drafts, Never Records

In one sentence: let AI be the bookkeeping assistant, not the bookkeeper

In Nigeria, many small shop owners, contractors, and farmers don't keep proper books. They rely on self-addressed notes, a page in an exercise book, or a voice memo to track their business: "Paid the plumber 40,000," "Customer owes me 60,000." These fragments carry the true history of the business, but no one can ask them questions — how much did I earn this week? Who still owes me money?

LedgerMind AI tries to solve this. It's a configurable business-records and intelligence platform: a shop owner types or speaks a note, a hosted Gemma model reads it and proposes a draft record, the owner reviews and confirms, the backend then computes all figures from saved records, and finally the AI summarizes the business in plain language.

The project makes an architectural trade-off that every AI application should study: numbers come from code, narrative comes from the model. The model doesn't touch the database, doesn't write SQL, can't save or confirm any payment — and when it errors out or is unavailable, the ledger numbers are still correct.

Why not build generic accounting software

The author's judgment is clear: restaurants, supermarkets, and contractors record completely different things, and one set of fields can't serve them all. LedgerMind uses a template-driven records engine, where different business types have their own fields, record types, and reports while sharing the same backend.

The target paying users are equally well-defined — shop and restaurant owners, small contractors, and farmers who want to know where their money goes but don't want to learn accounting software and can't afford a bookkeeper.

Four design principles that amount to an AI trust manual

Templates, not one-size-fits-all. Differences between business types are absorbed by the template layer, avoiding rewriting a system for each industry.

AI drafts, humans confirm. The model's output is only a draft and never goes straight into the database. Even vendor invoices must be written to a separate data table, distinct from ordinary purchase records, only after the user clicks "Confirm and save."

Numbers are computed by code, text is generated by the model. Every total is calculated from saved records; the model only writes the analytical narrative. This is the hardest guardrail in the entire project.

Honestly label capability boundaries. Reports are computed from saved records; reconciliation discrepancies are only signals for review, not evidence of missing funds or misconduct. The author explicitly states: this is not audit-grade accounting.

In an era where AI applications routinely overstate their capabilities, this self-limitation actually becomes part of the product's credibility.

Current state: half works, half openly acknowledged as unfinished

The author chose to publish an incomplete version before the deadline and lays out completion status item by item.

What works and has been self-tested includes: registration and login, revocable sessions, password and profile management (passwords hashed with PBKDF2-HMAC-SHA256, storing only token hashes); ownership isolated by business with server-side enforcement of membership checks; end-to-end usability for restaurant and supermarket scenarios, covering record entry, reports, the AI note parser, and the AI business analyst; browser-based voice input via the Web Speech API with text input as a fallback; deployment on Render with optional Sentry tracing.

What remains unfinished is equally candid: business types like contractors, pharmacies, farms, and logistics are only partially usable — fields, labels, totals, and reports are still shaped around the shop scenario, some records aren't counted and totals show ₦0, and the overview and report pages can contradict each other; outside the shop scenario the AI extraction makes mistakes, such as misjudging the direction of funds (receipt vs. payment), storing placeholder words as customer names, or failing to ask follow-up questions when amounts are unclear; the hosted Gemma service occasionally returns 429, 500, or 503, and the retry logic is too weak to cope; some business-analyst requests respond slowly; and "profit" isn't yet computed — the author plans to answer profit questions with net cash flow (inflows minus outflows) and clearly label it as not audited profit.

The author has 25 specific fixes in hand and promises that post-deadline commits will be listed in the repository README.

PBKDF2-HMAC-SHA256 is a password-based key-derivation function: it takes the user's password plus a random salt and runs many iterations of HMAC-SHA256, deliberately slowing computation so that offline brute-force attacks become far more expensive. The system stores only the derived result rather than the plaintext password, and verification recomputes and compares. The original text also mentions "storing only token hashes," meaning session tokens are likewise stored in the database only as hashes, so even if the database leaks, attackers can't directly impersonate a valid session.

Web Speech API is a built-in browser speech-recognition interface that converts microphone input to text without any additional server-side component, making it suitable for quickly launching voice bookkeeping; however, its recognition quality depends on the browser and operating system, and support varies across browsers, so keeping text input as a fallback is a necessary design.

Four real problems exposed by Sentry

This section is the most valuable engineering retrospective in the entire project. After grouping AI endpoint failures in Sentry, the author read out four things:

First, most failures are the model provider being busy, and they hit the most-traveled path. On /api/v1/ai/interpret-record there were 7 HTTP 503s, 3 HTTP 429s, and 1 inference timeout — and note parsing is the very first thing a user does on entering. The original retry waited only 1 and 2 seconds, and the logs show all three attempts failing within 3 seconds.

Second, the business analyst's output limit was set too low. One analysis was truncated at 2,500 tokens; the author plans to raise it to 5,000.

Third, one analyst request took about 45 seconds. That's utterly unacceptable for a shop owner waiting on a phone.

Fourth, the issue title tells me only the status code, not the cause. Even when the logs contain Google's stated reason, it's just a generic "Internal error." The author needs to record the attempt number, model, and latency of each failure.

The corresponding fixes are already clear: add exponential backoff with jitter, respect Retry-After, record each attempt as a separate span, and give the model smaller data snapshots.

Exponential backoff with jitter is the standard retry strategy for transient service unavailability: after each failure the wait time grows multiplicatively (e.g., 1s, 2s, 4s, 8s), plus a random perturbation to prevent many clients from retrying at the same instant and re-overwhelming a service that just recovered. In the original text, waiting only 1 and 2 seconds and having all three attempts fail within 3 seconds is a textbook symptom of missing backoff.

Retry-After is an HTTP response header by which the server explicitly tells the client "try again after this long." Respecting it is more reliable than guessing intervals yourself and significantly reduces wasted requests. Recording each retry as a separate Sentry span (trace segment) makes the multiple attempts of one request individually visible, thereby distinguishing "failed on the first try" from "failed after three retries" — two completely different failure patterns.

Why choose an open-weight model

The author doesn't shy away from the hosted-inference trade-off: users don't need to own a GPU to use the app, but the customer's notes do leave the application and reach the provider. LedgerMind AI does not claim that, when using hosted inference, data stays on infrastructure it controls.

Open weights provide a way out. When the hosted endpoint returns errors that can't be weathered, you can switch providers, or self-host when the business can afford it, instead of waiting for a single vendor to recover. The model can also be kept on a very short leash — it doesn't write SQL or store records, the backend computes all numbers, and when a cheaper or better model comes along in the future, only the "narrator" needs replacing without touching the numbers.

The author considers fine-tuning the most critical next step for open weights: adapting to how these shop owners actually talk — informal notes, currency amounts, and phrasings like "the customer owes me" where a change of direction changes everything. General-purpose APIs perform poorly at this, as the project's own extraction errors have already demonstrated.

Open-weight models are models whose parameter weights are publicly downloadable and can be deployed independently, as opposed to closed-source models offered only via API. The key distinction isn't whether the license permits commercial use, but whether users can move the model onto their own hardware or another cloud. For an application like LedgerMind, this means that when the hosting provider keeps returning 429/503, the team can switch to another inference provider, or even build its own inference when volume justifies the cost, rather than passively waiting for a single vendor to recover.

Conversely, open weights don't automatically mean data stays local: as long as third-party hosted inference is still being called, the user's notes still leave the application. What really determines the data boundary is the deployment location, not whether the weights are public. The author states this accurately, without packaging "open weights" as "data never leaves."

Deployment architecture

The frontend — React + TypeScript + Vite — is deployed on Render (ledgermind-web.onrender.com); the backend — FastAPI + SQLAlchemy + Alembic + PostgreSQL — is also on Render (ledgermind-api-4guj.onrender.com); AI inference goes through Google-hosted Gemma; and Sentry handles monitoring and tracing of AI calls. The author specifically notes: there is no claim that Gemma itself runs on Render.

Render's free tier may take up to a minute to wake up after idling on first load, which is why a friendlier "server is waking up" prompt is needed.

What's worth learning from

The most valuable thing about this project isn't the feature list, but that it writes "what AI can do" and "what AI cannot do" into the architecture rather than the documentation. The two-step draft-confirm flow, the separation of computation from narrative, and the public statement of audit boundaries together form a reusable trust design pattern. For any team integrating LLMs into finance, healthcare, or compliance scenarios, these four principles are more worth copying than any prompt-engineering trick.

Code and demo: GitHub repository IamClassified9/LedgerMind-Ai, live app at ledgermind-web.onrender.com (free tier; first load may require waiting for wake-up).