The Free Model Returned Valid JSON, and My Schema Said Nothing

Two days ago, I launched a small overnight extraction job, watched it exit with code 0, and went to bed satisfied. The next morning, downstream reported nothing at all, the logs were clean, and every row in the database was records: []. The model had returned perfectly valid JSON, and my schema had never told it that an empty array was the one answer I couldn't accept.

These are field notes from the next 48 hours: what I tried, what actually went wrong, and which parts I'd repeat without hesitation. If you're pointing batch jobs at free model tiers and free server instances, this is the shape of the pit you're about to fall into.

What I Was Actually Asking For

The job itself was deliberately boring. It reads a batch of small text chunks, asks the model to extract line items (id, amount, currency), validates the result, and writes one row per chunk. The worker runs on whatever host I specify, and this run's host was a free server option rather than a machine I rent monthly.

I used MonkeyCode's free model access for the extraction calls, and the worker itself ran on the same product's free server option. Disclosure: this article was prepared as part of a MonkeyCode product promotion. I deliberately avoided quoting quota, token limits, or latency numbers, because anything I measured on one instance last night doesn't represent what you'll get.

That's the first lesson, and it cost me about six hours: on a free tier, the numbers you measure are not the numbers you're guaranteed. Measure anyway, but treat your measurements as a floor you've verified, not a ceiling you can rely on.

The Order in Which the Problems Surfaced

Hour 3: The Empty Success

My JSON Schema said "records": {"type": "array"}. An empty array is a valid array. When the model hit a chunk it couldn't parse confidently, it made the reasonable choice: return nothing rather than invent line items. I hadn't written any rule saying "empty is failure," so nothing failed.

The one-keyword fix is "minItems": 1, the highest-value line in this entire article. Every schema I write now contains at least one business constraint, because a structural schema proves shape, not meaning.

JSON Schema is a declarative language for describing and validating the structure of JSON documents; by itself it constrains only the shape of data, not its semantics. type: array merely requires the value to be an array, and the empty array [] satisfies it completely; items constrains only the form of elements inside the array and imposes no requirement on length. To make a schema express business meaning, you must use constraint keywords such as minItems (minimum number of elements), minLength (minimum string length), required (mandatory fields), enum (enumerated values), and pattern (regex matching). Understanding this is crucial: passing the validator does not mean the data is usable. A schema's default behavior is permissive, and every business invariant must be written in explicitly.

Hour 11: The Repair Loop That Fixed Nothing

My first instinct was to retry on validation failure. I wrote a three-attempt loop with backoff, ran it over 400 chunks, and spent a large chunk of time watching the same malformed payload come back three times. Retrying a content failure isn't a retry; it's a coin flip with extra steps.

Backoff retry is a common pattern in distributed systems: after a failure, wait a while and try again, with the wait time typically growing exponentially and including random jitter to avoid multiple clients retrying simultaneously and causing a thundering herd. But it only works when the failure is transient, such as network jitter or brief upstream overload. For content-level failures—where the model returned well-formed but semantically wrong output, or the JSON was truncated—requesting again with the same prompt will most likely produce the same wrong result, because neither the input nor the model has changed. Identifying whether a failure is retryable is the most easily overlooked step in retry strategy design.

Hour 19: The Config Typo That Would Have Run All Night

A misspelled model name caused every request to return 4xx. My retry wrapper treated every non-200 as a transient error, so the job happily backed off and retried for hours. No backoff strategy can fix a wrong environment variable, and there's no reason to discover it at 7 a.m.

HTTP 4xx status codes belong to the client-error category, where 401 (unauthorized), 403 (forbidden), and 404 (not found) usually point to credential or configuration problems rather than transient server-side failures; 5xx is the server-error category and is worth retrying. Many retry libraries default to "retry on anything that isn't 2xx," which turns a misspelled model name or an expired API key into a whole night of spinning. The right approach is to classify status codes before the retry logic, treating 401/403/404 as fatal errors that immediately terminate the entire batch and raise an alert, rather than quietly decaying into a pile of retry records that look like progress.

Hour 27: Not My Machine

Mid-run, the worker vanished, and the logs I'd been appending to vanished with it. The extraction calls were fine. The host was ephemeral, which is normal for a free server option, and I'd been treating someone else's machine as my own disk.

An ephemeral instance is compute whose lifecycle the user doesn't control, common in free tiers, serverless functions, and preemptible instances. Such instances can be terminated at any time due to idle timeouts, resource reclamation, or platform scheduling, and data on their local disk is lost with them. So anything that needs to persist—logs, intermediate results, checkpoint state—must be written to external storage (object storage, a database, or a machine you control). A practical principle: treat the worker as stateless, and treat persistence as an explicit network write, so that when the instance disappears your progress doesn't disappear with it.

The Failure Taxonomy I Use Now

Before I touch a retry strategy, I write down which kind of failure I'm facing. This table is the whole design:

Failure class	Symptom	Does retry help?	Action
transport	Timeout, connection reset, 502/503	Sometimes	Bounded backoff, up to 3 times
rate_limited	429 with a retry hint	Yes, later	Honor the hint, then stop
truncated	JSON ends mid-token, no closing brace	No	Shrink the input, then retry once
schema	Parses fine, violates constraints	No	One repair attempt, then fail
semantic	Passes schema, fails reconciliation	No	Quarantine for human review
config	Model name returns 401/403/404	Never	Stop the entire run

The rule I care most about is the last row. A config error should kill the run in the first minute, not quietly decay into a thousand logged retries that look like progress on your dashboard.

The Preflight Probe: 40 Minutes Before the First Real Run

Here's the test plan I run before pointing any overnight job at a free model tier. It's a plan, not a benchmark, meant to be run on your own account.

1. Send ten identical requests and record whether all ten satisfy the schema.
2. Send one deliberately ambiguous chunk and check whether the model invents numbers or returns an empty list.
3. Send one oversized chunk and note whether you get truncation, a refusal, or a 4xx.
4. Send one request with a deliberately wrong model name and confirm your code stops instead of retrying.
5. Sleep through any idle window you suspect, then send another request.
6. Kill the worker mid-run and confirm the logs survive somewhere you control.

The probe is small enough to read in one sitting:

# probe.py - run before trusting a free tier with an overnight job.
import os, json, time, requests

ENDPOINT = os.environ["MODEL_ENDPOINT"]   # documented chat endpoint from the provider
MODEL    = os.environ["MODEL_NAME"]       # the name listed today, not a year ago
HEADERS  = {"Authorization": f"Bearer {os.environ['MODEL_KEY']}"}

SCHEMA = {
    "type": "object",
    "required": ["records"],
    "properties": {
        "records": {
            "type": "array",
            "minItems": 1,               # <-- the keyword that saved me
            "items": {
                "type": "object",
                "required": ["id", "amount", "currency"],
                "properties": {
                    "id":       {"type": "string", "minLength": 1},
                    "amount":   {"type": "number"},
                    "currency": {"type": "string", "enum": ["USD", "EUR", "JPY"]},
                },
            },
        }
    },
}

PROMPT = (
    "Extract every line item from TEXT into JSON matching SCHEMA. "
    "Never invent an amount. If a line has no amount, skip it. "
    "If the text has no line items, return an empty records array.\n\n"
    "SCHEMA: {schema}\n\nTEXT:\n{text}\n"
)

def ask(text, timeout=60):
    body = {
        "model": MODEL,
        "messages": [{"role": "user", "content": PROMPT.format(
            schema=json.dumps(SCHEMA), text=text)}],
    }
    started = time.monotonic()
    r = requests.post(ENDPOINT, headers=HEADERS, json=body, timeout=timeout)
    elapsed = round(time.monotonic() - started, 2)
    return r.status_code, elapsed, r.text

if __name__ == "__main__":
    cases = {
        "clean":    "2 x widget @ 19.00 USD\n1 x cable @ 4.50 USD",
        "ambiguous": "some widgets, price on invoice",
        "empty":    "no line items in this document",
    }
    out = open("probe_results.jsonl", "a")   # keep this off the free machine
    for name, text in cases.items():
        status, elapsed, raw = ask(text)
        out.write(json.dumps({"case": name, "status": status,
                              "elapsed_s": elapsed,
                              "bytes": len(raw)}) + "\n")
        out.flush()
        print(name, status, elapsed, len(raw))

Run it, fill in what you see, and let the numbers argue with your assumptions. My numbers argued with several of mine.

The Worker Skeleton I'll Reuse

The second artifact is the loop I now reuse whenever model output has to satisfy a contract. It classifies before retrying, and it keeps state somewhere the machine can't delete.

import json, time

RETRYABLE = {"transport", "rate_limited"}
CONTENT   = {"schema", "semantic", "truncated"}
FATAL     = {"config"}

def classify(status, parsed, violations):
    if status in (401, 403, 404):      return "config", status
    if status == 429:                  return "rate_limited", status
    if status >= 500:                  return "transport", status
    if parsed is None:                 return "truncated", status
    if violations:                     return "schema", violations[:3]
    return "ok", None

def handle(status, parsed, violations, repair_used):
    kind, detail = classify(status, parsed, violations)
    if kind == "ok":            return "accept", None
    if kind in FATAL:           return "stop_run", detail
    if kind == "rate_limited":  return "sleep_then_retry", detail
    if kind == "transport":     return "backoff_retry", detail
    if kind == "truncated":     return "shrink_input_retry", detail
    if kind == "schema" and not repair_used:
        return "repair_once", detail   # send validator errors back, once
    if kind == "schema":        return "fail_row", detail
    return "quarantine", detail        # semantic: don't auto-repair

The repair step is one extra call containing the validator's error message and the offending payload, attempted only once. If that attempt fails, the row fails loudly into a quarantine file I can read over coffee.

The Check That Isn't a Schema Check

Structure isn't truth. After validation I reconcile: sum the extracted amounts and compare against totals I know from the source document. If the discrepancy exceeds my per-currency tolerance, the row is quarantined rather than retried. That single comparison has caught more bad rows than all the schema keywords combined, and it requires no model access at all.

The Results Template

I won't paste my latency numbers as if they were yours. Instead, here's the table I fill in after every probe run:

Probe case	Pass?	Failure class	Action taken
Ten identical clean requests
Ambiguous chunk
Oversized chunk
Wrong model name
Request after idle
Kill worker mid-run

If two or more rows return schema or semantic, don't scale the batch. Fix the contract first, because the free tier will happily give you a thousand well-formed wrong answers for free.

Limitations, and Who Shouldn't Do This

• You can't treat the free server option as persistent storage. Keep logs and results off the machine, or accept that a restart will eat your progress.
• This workflow needs a validator and a reconciliation source. If you have no ground truth to compare against, quarantine has nowhere to go.
• If you need a hard latency SLO or guaranteed concurrency, a free tier is the wrong tool, and no amount of retry logic fixes that.
• Regulated data that can't leave your network shouldn't be sent to any hosted model, free or paid.
• If your output must be byte-for-byte deterministic, a model plus validator is still a probabilistic system with a gate in front of it.

What I'll Repeat

Write business constraints into the schema before the first run. Classify failures before retrying. Stop the run on config errors. Keep logs where the host can't delete them. And run the probe every time you change the model name, because the cheapest bug is the one that fails in the first minute rather than the nineteenth hour.

If you want to try the probe yourself, it runs directly against MonkeyCode's free model access, and the worker can sit on the free server option while the logs stay on your own disk. Just fill in the results table honestly before you trust it with 400 chunks.

What was the last "valid" response that quietly ruined your morning?