Free Token Grants Don't Equal Real Compute: Five Common Misconceptions

Model leaderboards are everywhere, and everyone's tired of them. Instead of arguing over which ranking is more authoritative, try a different approach: use a reconcilable ledger to take apart the idea of "free credit."

MonkeyCode is an open-source project offering free model access and a free server option, and its operator states it provides 10 million free tokens (disclosure: this article is sponsored content for MonkeyCode). We won't talk about undisclosed model names, hardware specs, or throughput numbers here—if there's no source, we don't make it up. What we can give you is a probe script, and the five misconceptions it can puncture.

Why a Ledger Beats a Dashboard

A dashboard gives you a single number, and a number is not evidence. What you actually need is the raw requests and responses behind that number: status codes, token counts, retry attempts, hashes. Those are the real receipts.

The Probe: One File, Three Prompts, Nine Calls

ledger_probe.py is an illustrative scaffold, and this article reports no measured results. It does three things: sends three prompts to an OpenAI-compatible endpoint, retries up to three times per prompt (exponential backoff), fixes temperature at 0, and writes each record to ledger.jsonl.

**An OpenAI-compatible endpoint** refers to an HTTP interface that follows the OpenAI Chat Completions API request/response format. It typically lives at the `/chat/completions` path, accepts fields like `model`, `messages`, and `temperature`, and on success returns a `choices` array plus a `usage` count (`prompt_tokens`, `completion_tokens`). Many self-hosted inference frameworks (such as vLLM, Ollama, LM Studio) and cloud vendors provide this compatibility layer so existing clients can switch without modification. For this article's methodology, that compatibility layer matters a lot: as long as the endpoint behaves consistently, the same probe script can be reused across any provider for reconciliation, without being locked in by a proprietary SDK or private protocol.

**Temperature** is a sampling parameter that controls the randomness of the output distribution. The lower the value, the more the model tends to pick the highest-probability token; setting it to 0 usually means greedy decoding, i.e., taking the maximum-probability option at every step. But "temperature 0" is not equivalent to "a unique result": batch scheduling, floating-point non-determinism, mixture-of-experts routing, quantization error, and provider-side redeployments can all cause the same input to produce different outputs. This is why the probe fixing temperature at 0 only removes one variable—actually judging determinism requires hashing completion outputs for comparison, not relying on the parameter itself.
# ledger_probe.py — illustrative scaffold, no measured results reported here
import json, os, time, uuid, hashlib
from pathlib import Path
import httpx

BASE  = os.environ["LLM_BASE_URL"]   # any OpenAI-compatible endpoint
KEY   = os.environ["LLM_API_KEY"]
MODEL = os.environ["MODEL_ID"]       # taken from the project's own list

LEDGER = Path("ledger.jsonl")

PROMPTS = [
    "Reply with the single word: ok",
    "Count the letters in 'strawberry'. Show only the number.",
    "Return strict JSON: {\"n\": 1, \"unit\": \"token\"}",
]

def call(prompt, attempt):
    body = {"model": MODEL, "temperature": 0,
            "messages": [{"role": "user", "content": prompt}]}
    started = time.perf_counter()
    r = httpx.post(f"{BASE}/chat/completions", json=body, timeout=60,
                   headers={"Authorization": f"Bearer {KEY}"})
    elapsed = (time.perf_counter() - started) * 1000

    rec = {
        "probe_id": str(uuid.uuid4()),
        "attempt": attempt,
        "status": r.status_code,
        "latency_ms": round(elapsed, 1),
        "prompt_sha256": hashlib.sha256(prompt.encode()).hexdigest()[:12],
    }

    if r.status_code == 200:
        data = r.json()
        usage = data.get("usage") or {}
        rec["prompt_tokens"] = usage.get("prompt_tokens")
        rec["completion_tokens"] = usage.get("completion_tokens")
        text = data["choices"][0]["message"]["content"]
        rec["completion_sha256"] = hashlib.sha256(text.encode()).hexdigest()[:12]

    with LEDGER.open("a") as fh:
        fh.write(json.dumps(rec) + "\n")
    return rec

for prompt in PROMPTS:
    for attempt in range(1, 4):
        rec = call(prompt, attempt)
        if rec["status"] == 200:
            break
        time.sleep(2 ** attempt)   # the free tier rate-limits, so plan ahead

Why hash? Because "it works" is not a metric. A hash either repeats or it doesn't.

Reconciliation Steps

reconcile.py reads your own ledger, not the dashboard rented to you by the vendor:

# reconcile.py — read your own ledger, not a rented dashboard
import json, collections

rows = [json.loads(line) for line in open("ledger.jsonl")]
ok   = [r for r in rows if r["status"] == 200]

print("requests:", len(rows), "ok:", len(ok), "retries:", len(rows) - len(ok))
print("prompt_tokens:    ", sum(r.get("prompt_tokens", 0) for r in ok))
print("completion_tokens:", sum(r.get("completion_tokens", 0) for r in ok))

hashes = collections.Counter(r["completion_sha256"] for r in ok)
print("distinct completions:", len(hashes))
print("most common:", hashes.most_common(3))

Run it, and you're holding a receipt.

Misconception #1: 10 Million Free Tokens Means 10 Million Usable Tokens

No. Your prompt overhead eats into that credit first. Every retry resends the input—system prompt, tool schema, retrieved context, all billed as input. Three retries on a bloated prompt can triple your input cost. Under a fixed quota, retries are not free.

The correct model: effective output is only part of the grant. Measure your retry rate before planning capacity.

Misconception #2: If It Answered Once, It's Deterministic

A single sample says almost nothing about variance. Ask the same question five times, hash each completion, and count how many distinct hashes you get.

Even with temperature at 0, you may see drift—from redeployments, quantized builds, or routing changes.

The correct model: pin a model version, record hashes, and rerun the probe after every provider change.

Misconception #3: Free Means Unlimited Concurrency

Free tiers rate-limit. That's normal, and it's fine. What's actually dangerous is silent failure: a 429 gets logged as an "error" and then retried forever in a loop.

Log your status codes, stop when you hit a rate limit, and don't keep banging on a door that just closed.

• 200: log token counts and hashes
• 429: log, back off, cap attempts
• 5xx: log, back off, cap attempts
• Other 4xx: fail loudly, don't retry

The correct model: free access is a budget with a governor, not an infinite pipe.

Exponential backoff is a retry cadence where the wait time after each failure grows by a multiplier—say 2, 4, 8 seconds—often with random jitter to avoid multiple clients retrying simultaneously and causing congestion. It's well-suited for 429 (rate limiting) and 5xx (temporary server faults), since these errors tend to heal over time. But backoff is not a panacea: retrying 4xx errors like authentication failures or malformed requests just wastes quota and hides the problem, so the probe script caps retry attempts and writes every attempt to the ledger, making "retries" a measurable cost rather than a hidden one.

Misconception #4: Running on a Free Server Equals Running Locally

It's a different machine, a different clock, and possibly a different image. Environment drift is the quietest bug. Your laptop runs Python 3.12; that box might not.

So pin down what you can control: lockfiles, container digests, and an exported environment block. Write that environment block into the same ledger, right next to the tokens.

The correct model: a free server is a place to reproduce results, not a place to assume they hold.

Misconception #5: The Vendor's Counter Is a Receipt

The vendor counts what its gateway sees; you count what your client actually sends. When a proxy retries, a stream breaks, or a request never lands, those two numbers diverge.

Reconcile both sides. If the gap exceeds a few percentage points, go find out why.

The correct model: two independent counters, one deliberately boring script, and a written note about the delta.

Two-counter reconciliation is necessary because billing and client-side observation have systematic differences. The vendor counts tokens after a request reaches its gateway, whereas the client counts what it itself sent. Proxies in between, SDK retries, interrupted streaming responses, connection timeouts, and load-balancer replays can all make the two sides diverge: a request may be recorded once by the client but counted multiple times at the gateway; conversely, when streaming is interrupted, the vendor may have already billed for partial output while the client only sees an error. Regularly comparing the total token counts on both sides and recording a note about the delta is the most direct way to catch this kind of silent discrepancy.

A Decision Table

Question	Free tier is enough	Move off the free tier
Prototypes, low volume	Yes	—
Nightly CI on a fixed prompt set	Yes, cap the attempts	If retries exceed the cap
Customer-facing latency commitments	No	Yes
Data residency requirements	No	Yes
Long-context batch jobs	Test first, then measure	If input cost dominates
Anything you can't rerun	No	Yes

Who Shouldn't Use This Approach

Be honest with yourself before you start.

• People who need a latency SLA today. A free tier is not an SLA.
• Teams bound by data residency or retention rules. Read those rules first.
• People who won't read the provider's limits page. You'll guess wrong.
• People who treat a free grant as permanent capacity. It's just a starting point.

A free credit is a place to learn the shape of your costs, not a capacity plan.

Limitations I Won't Hide

This article does not verify current quotas, model lists, or server specs; go by the numbers in the project's own documentation on the day you read it. Quotas change, and a ledger written last month doesn't know today's limits. The scaffold above is illustrative, and this article reports no benchmark results, because the author has not run any published benchmarks.

Test First, Trust Later

Write the ledger before you argue about models.

If you want to try this scaffold without tying up a card, free model access and the free server option are the relevant entry points. Before you plan anything, read the current limits on the project's own pages.

Then run three prompts, count the hashes, and reconcile the tokens. Which misconception will fall first in front of you?