Don't Let the LLM Make Decisions: Engineering a Predictable AI Tutor

Title

Don't Let the LLM Make Decisions: Engineering a Predictable AI Tutor

Meta Description

A deep dive into engineering a predictable AI tutor: the LLM only writes hint copy, while whether to help and how much to help are decided by deterministic code. Covers policy pure functions, attempt-count thresholds, anti-leak guards, and unit testing practices.

Article Content

Feed the LLM a problem, a learner's attempt log, and the reference answer, then tell it "help the learner, but don't give away the answer too early"—that's the most obvious approach to building an AI tutor. It works in a demo, fails in an experiment, and breaks silently in production.

One developer ran a controlled experiment with a SQL tutor: Group A was forced to try first, received a hint only when stuck, and got the answer only as a last resort; Group B, like most AI tools, answered every request. After 6 people per group used it, they were tested a week later without the tutor. This article breaks down his engineering implementation—the core rule is just one sentence: the LLM never decides whether to help or how much; ordinary code decides that. The LLM only writes the wording of the hint after the decision is made.

Why You Can't Let the LLM Make Decisions

Consider a prompt version that says: "If the learner seems stuck, give a hint; if they've genuinely made an effort, give the answer."

Then try to answer one question: what exactly did Group A receive?

You can't. "Seems stuck" is a judgment the model makes fresh every time. Two learners in the same situation might get different help, and the same learner retrying might get something different too. The very thing you want to test becomes a moving target.

The other problem is sample size. In a study of 6 people, there were only about 23 moments when the tutor needed to decide "what help to give." You can't judge the model's decision quality with 23 examples. But you can fully test a pure function—he wrote 123 unit tests.

So the system was split into three kinds of parts:

Part type	What it does	Why it belongs here
Human	Writes and runs SQL	The skill being measured belongs to the learner
Deterministic code	Try-first rules, hint/answer rules, answer checker, anti-leak guard	The thing being tested must be fixed and inspectable
Probabilistic LLM	Only writes hint copy	The narrative aimed at "this specific error" is something code can't produce

The model is just a small component, not the product itself.

The boundary between deterministic code and the probabilistic LLM is the core of this article's architecture. Deterministic code refers to programs that always produce the same output for the same input, exhaustively verifiable with unit tests; the sampling mechanism of a probabilistic LLM makes its output inherently random, so the same prompt may generate different results across runs. In experimental design, this non-determinism directly contaminates the independent variable—if "whether to give a hint" is decided by the model, then the timing of the help a learner receives is itself unreproducible. So the author separated "decision" from "expression": decision logic goes into testable pure functions, and the model only handles the natural-language generation task that can't be hard-coded with rules. This separation doesn't mean weakening the model's capability; it limits the model's scope to the part where it is truly irreplaceable—generating personalized narratives for specific errors.

The Policy Core: A Pure Function

The core of policy.js is a pure function with no DOM, no network, and no touching the LLM. It takes a problem's log and returns an action: NONE (already solved), REFUSE_GATE (Group A, no valid attempt yet), REFUSE_RETRY (Group A, hint already given, keep trying), HINT (LLM writes the copy), REVEAL (give the reference query).

const N = item.N ?? 2; // after a hint, try N more times before giving the answer
const counted = countableAttempts(item.attempts);
const helpServed = item.helpServed || [];

if (item.attempts.some(a => a.outcome === 'correct')) return { action: ACTION.NONE };
// the entire difference between Group A and Group B is this one line
if (item.arm === 'B') return { action: ACTION.REVEAL };
// Group A: no help at all until an attempt is counted
if (counted.length === 0) return { action: ACTION.REFUSE_GATE };
if (helpServed.some(h => h.action === ACTION.REVEAL)) return { action: ACTION.REVEAL };
const hint = helpServed.find(h => h.action === ACTION.HINT);
if (!hint) return { action: ACTION.HINT };
const sinceHint = counted.length - hint.afterCountedAttempts;
return sinceHint >= N ? { action: ACTION.REVEAL } : { action: ACTION.REFUSE_RETRY };

The only thing in the whole block worth a second look is that one if between Group A and Group B. Anyone can read exactly what each group received. Every decision records counted and sinceHint, so every tap of the help button can be proven from the log why the tutor responded the way it did.

N is fixed at 2 for everyone. If it varied per person, you'd be testing two things at once, and with 3 people per group you couldn't separate them anyway.

What Counts as an Attempt

Group A's rule is "try first." The code needs to determine what counts as a real attempt, and that became the hardest part of the whole policy.

The first version's logic was: if the result set differs from the previous one, it counts as an attempt. Simple, but wrong.

The bug: a broken query has no result set. For a learner whose queries never execute, no attempt is ever counted, and the gate never opens. In early test rounds, 101 of 122 attempts had syntax errors and wouldn't run at all. This isn't an edge case—it's the main case.

And a query that fails to parse can still carry a complete but wrong mental model:

SELECT customer, COUNT(*) FROM orders WHERE COUNT(*) > 3

It won't run, yet it happens to be exactly the typical error that teaching is meant to correct (filtering groups with WHERE instead of HAVING). It should count.

But asdf also fails to parse. If garbage input counted too, a learner could bang out three lines of nonsense and get a hint without thinking about anything. Education research calls this behavior "gaming the system," and learners find this shortcut quickly.

So he added a substantive threshold: an attempt must contain an SQL keyword plus a name from the problem's schema. Matching deliberately uses substrings rather than strict tokenization—real learners have written GroupBy and havingcount>=3, misspelled but with a genuine mental model, and strict token matching would drop them.

There's also a fail open detail: if the schema list fails to load, the threshold only checks keywords. A misconfigured page must never silently reject all learners, or Group A would turn into "no help at all" without anyone noticing.

Verified against 128 attempts from a screening round: 121 passed, and the 7 rejected were 6 empty submissions and 1 bare SELECT. No real attempt was lost.

fail open versus fail closed are two common failure strategies for configuration systems. Fail closed means denying service by default when a dependency fails to load; fail open means allowing it by default. When the schema list fails to load, the threshold degrades to checking keywords only—this is fail open: better to let some garbage input through than to have a config error trap every learner outside the gate. In an educational setting this trade-off matters especially—wrongly rejecting real learners directly breaks the experimental group's design, and it typically happens silently and goes unnoticed. By contrast, the anti-leak guard falls back to a hand-written hint when hint generation fails, which reflects the same design philosophy: the system's default failure mode must point toward "the learner can still keep learning," not "the system is safe but unusable."

The Anti-Leak Guard

Here comes the uncomfortable part: to write a useful hint about a wrong query, the LLM has to see the correct query. He calls this the reference query. The model holds the answer in its hands while writing the hint, and sometimes it just prints it out.

If that text reaches the learner's eyes, Group A silently becomes Group B on that problem. The experiment no longer tests anything, and the log shows nothing unusual.

So every hint passes through hint-guard.js before reaching the learner. It's deterministic, doesn't call the model, and just looks at the text and says yes or no. Four kinds of checks: a query that looks runnable (SELECT plus a real table name after FROM/JOIN); a clause that distinguishes the reference query, unless the learner has already written it; the reference query reproduced verbatim too much; and a passage of "spoken answer"—mentioning four or more SQL keywords.

If a hint is rejected, the model is asked again; if it still fails, a hand-written fallback hint for that problem is used. Every attempt and every rejection is logged. Result: zero false positives on 101 real hints; 22 of 24 deliberately written leak samples were caught.

The reference query in this article refers to the official standard-answer SQL for each exercise. For the LLM to write an apt hint about a learner's error, it must understand what the correct solution is, so the reference query is included in context when generating the hint. The risk is that the model may simply restate that query, letting a Group A learner get the answer without trying. The guard's checking logic is essentially detecting "the degree of information overlap between the hint text and the reference query": whether a complete runnable query appears, whether it reproduces clauses the learner hasn't written yet, whether it copies the reference query verbatim, and whether it narrates the answer structure in natural language. The four checks go from coarse to fine, corresponding to leak forms at different granularities, ultimately achieving zero false positives and high recall.

Every Rule Was Forced by an Earlier Rule

Every rule in the guard corresponds to the failure of a simpler rule.

"Contains SELECT and FROM means a leak" was the first version of the check, but those two words are normal English. "You wrote SELECT, but where does the data FROM come from?" is a good hint, yet it got blocked. The fix: a runnable query must name a real table name, and the reference query tells you which tables exist.

The semicolon rule was likewise deleted—"Nothing is processed; you need to say where the data comes from" got caught by mistake, and the table-name rule already covered real queries.

Quoting the learner's own words back caused 7 of 8 rejections in one test run to be false positives. A hint would say "your MAX(price) is fine," and since MAX(price) is part of the answer, it got blocked. But the learner had already written it; it's right there in the editor. Now only clauses the learner hasn't yet written count as a leak.

Leaks without SQL are subtler: "The FROM keyword selects the products table, then GROUP BY groups by category..." Not a single clause appears verbatim, yet you could type out the answer by following it. The signal is shape, not content: a real hint talks about only one or two points, while a spoken answer names everything. The threshold wasn't set by feel—among 101 real hints, 66 mentioned no SQL keyword at all, 29 mentioned one, 5 mentioned two, 1 mentioned three, and none mentioned four; the two narrative leaks mentioned six or seven each. The boundary of four has margin on both sides.

Silent fallback was the hardest to find: the LLM sits behind a Supabase edge function, so the API key never reaches the browser. A free Supabase project pauses after about a week of no traffic, and when it pauses there's no error—Group A just quietly serves hand-written fallback hints. The app looks fine but is actually running a different tutor. He found it once, fixed it, and added a script that checks before every session.

This SQL tutor is on GitHub (withholding-pilot): Group A and Group B share the same page, the same 16 exercises, and the same grader—the only difference is one config flag.

Lessons for Putting LLMs into Products

Ask this of every decision: does it need judgment, or does it need to be the same every time?

Whatever needs to be explainable, testable, or consistent goes into code. Whatever needs language aimed at "this specific case" goes to the model. Between the model and the user, insert a deterministic check at every point where the model getting it wrong would be costly.

This tutor can state clearly what each group received precisely because of that dividing line. And that's also why the experiment could fail honestly—when two checks failed, he knew the problem was with the learner, not with the model being in a different mood on some Tuesday.

Next in the series: the pitfalls of running your first controlled experiment as a developer, including a problem a Demo Day judge caught on the spot.

One-line Summary

The success of an AI tutor isn't in the prompt—it's in giving decisions to code and letting the LLM only write the copy.

Paragraph Summary

The article documents a developer's engineering work building a SQL tutor. The core principle: the LLM never decides whether to help or how much—a deterministic pure function does—and the model only writes the hint wording. The policy is implemented as a side-effect-free policy.js, the entire difference between Group A and Group B is one if, every decision is logged and traceable, and 123 unit tests cover it. The hardest part was determining what counts as a real attempt—broken queries have no result set, so a substantive threshold is needed to separate genuine mental models from garbage input. The anti-leak guard runs four deterministic checks before a hint reaches the learner, with zero false positives on 101 real hints and 22 of 24 leak samples caught. The author concludes: put what needs consistency into code, give language aimed at specific cases to the model, and insert deterministic checks at costly stages.