90-Minute Hands-On Probe Workshop for Free Model Endpoints: From Doubt to Decision

Moving a summarization step onto a free model endpoint, a refactoring job starts failing at 2 a.m. a week later—nobody changed the code; only the response structure and the latency tail changed. This scenario plays out repeatedly across teams. Free access answers the question "can a developer try the model," not "can an automation loop depend on it."

Why Measure Before Adopting

Most of the fatigue around AI tools isn't fatigue with the models themselves—it's fatigue with unverified claims. A team adopts an endpoint because the demo looked fast, then discovers in production that timeouts come with no status code, usage counters are sometimes missing, and rate limits hide in response headers nobody parses.

The probe below doesn't evaluate vendors. It measures one account, one model, one network path, on one afternoon. That's a deliberately narrow conclusion—and the only one this workshop is willing to defend.

The Ninety-Minute Schedule

The first ten minutes set the frame: the facilitator draws the story of that failing job on the whiteboard and asks the room—which property would you want to have known in advance? Latency, error taxonomy, and token accounting usually win.

Minutes 10 through 30 are for inventory: participants list every place their code calls a model and mark which ones sit inside automation loops rather than behind human interaction. Only automated calls matter for this workshop.

Minutes 30 through 55 are hands-on building—two files, about ninety lines, run from the terminal. Minutes 55 through 70 are for analysis: percentiles, error counts, and the awkward question of what to do when the usage object is missing. Minutes 70 through 85 turn numbers into decisions using the decision table below. The final five minutes are for writing the stop rules into the repository—a decision nobody wrote down is just a preference.

Exercise One: The Probe Itself

The probe sends the same fixed prompt twelve times in series, logging one line of JSON per call. Serial matters: concurrency looks pretty but destroys the signal, because rate limiters fold the latency distribution into a step function.

// probe.mjs — node probe.mjs > runs.jsonl
import { performance } from 'node:perf_hooks';

const BASE  = process.env.PROBE_BASE_URL;
const PATH  = process.env.PROBE_PATH ?? '/chat/completions';
const KEY   = process.env.PROBE_API_KEY;
const MODEL = process.env.PROBE_MODEL;
const N     = Number(process.env.PROBE_N ?? 12);

The path above is a common OpenAI-compatible shape, not a guarantee. Participants should first verify the base URL and path against the vendor's current documentation, and write the link into the file header.

Each call carries a 60-second AbortController timeout; on 429 or 5xx, read the retry-after header, and if it's absent, retry with exponential backoff plus jitter, up to four times. On success, record the usage object, the response text length, and a 120-character sample. Credentials are exported via environment variables rather than pasted into files—the facilitator checks this habit before the first run.

PROBE_BASE_URL="$MC_BASE_URL" PROBE_PATH="/chat/completions" \
PROBE_API_KEY="$MC_API_KEY" PROBE_MODEL="$MC_MODEL" PROBE_N=12 \
node probe.mjs > runs.jsonl

AbortController is a cancellation primitive built into both browsers and Node.js (v15+); it doesn't initiate requests itself. The pattern is to create a controller, bind it to fetch's signal option, then call controller.abort() elsewhere (usually in a setTimeout), and the pending request rejects immediately with an AbortError. For the probe, this is more reliable than wrapping a Promise.race at the call site, because Promise.race only makes the outer layer return early—the underlying connection still hangs, continuing to consume quota or connection pool slots on subsequent requests. Note that AbortError is not an HTTP error—the response never came back, the status code is missing, so the probe must count it separately from 4xx/5xx, otherwise the error taxonomy gets polluted.

retry-after is an HTTP response header telling the client how long to wait before retrying; the value can be seconds or an HTTP date. 429 (Too Many Requests) and some 503s carry it, but many free-tier implementations simply omit it—which is exactly why the article says "if it's absent, use exponential backoff plus jitter." Exponential backoff grows the wait time by powers of 2 (e.g., 1s, 2s, 4s, 8s), and jitter adds a random amount to each wait to prevent multiple clients from retrying in lockstep and creating a secondary surge. The four-retry cap keeps a single probe run from dragging on unacceptably long due to persistent rate limiting.

Exercise Two: Reading the Numbers

The aggregation script groups by status and outputs p50, p95, max, and the ratio of usage_reported. The percentiles from twelve samples are descriptive, not inferential, and the script should say so when it prints them.

The column most worth watching is rarely the median. If usage_reported doesn't equal the success count, the accounting layer is untrustworthy, and any per-request cost model the team builds is a guess. Whoever spots this should stop tweaking prompts and go read the vendor's terms.

p50 is the median: the value between the 6th and 7th of twelve sorted samples—half the requests are faster, half slower. p95 means 95% of requests are faster than that value, only about 5% slower—with twelve samples, p95 effectively falls between the 11th and 12th samples, almost "the second-slowest run," so it's highly unstable. The max is literally the slowest run, most sensitive to long tails but also most easily skewed by a single network hiccup. The point of reading all three together: p50 describes the norm, p95 describes how often you'll hit an anomaly in automation, and max tells you how bad the worst case gets. The article emphasizes "descriptive, not inferential" because you can't infer the shape of the population distribution from twelve observations—there's no reason to assume latency follows a normal or any specific distribution, and the tail usually doesn't.

There's a rough rule of thumb for the relationship between N and percentiles: talking about p95 needs at least a few dozen samples, and p99 needs hundreds. Twelve runs give you existence evidence—whether the usage field is missing, whether there are 429s, whether the tail is two seconds or twenty—not precise percentile estimates. This is also why the aggregation script should warn when asked to output p99: that number is interpolated from two or three samples, looks precise, and is actually meaningless.

Exercise Three: A Collector That Stays Put

A laptop on office Wi-Fi is a terrible measurement instrument; on hotel Wi-Fi it's worse. To make twelve runs comparable across weeks, the collector needs to sit in a stable environment.

MonkeyCode offers free model access and a free server option, making it a suitable place to run this test bench for the first round: the probe runs from a fixed machine, writes runs.jsonl, and the team reads the file. Endpoints, current limits, and terms should be verified against documentation before the session—not looked up during it. No quota, hardware spec, or duration is assumed here, because those details change and a workshop shouldn't teach stale numbers.

(Disclosure: This article is part of MonkeyCode product promotion content.)

The same caveat applies to any free tier: a free server is a place to run measurements, not a substitute for an availability guarantee. Teams that need guarantees should talk to the vendor, not read a workshop outline.

Exercise Four: Three Common Pitfalls in the Room

Someone always wants to add concurrency. Let them—in a separate file—then compare. The step function that appears is the lesson. Someone else wants to tweak prompts until the output quality looks good; that's a different workshop, and mixing it in pollutes the latency data.

A third person asks whether twelve runs is enough. It's enough to find missing usage fields, 429 storms, or two-second tails; it's not enough to estimate p99, and the aggregation script should print a warning when someone asks for it.

The Decision Table

Fill in this table together as a group, then commit it alongside the probe script.

Run Signal	Interpretation	Action
p95 well below loop timeout, no errors, usage present	Predictability sufficient for a controlled trial	Integrate after the timeout and kill switch, log every response
p50 normal, p95 near or above timeout	The tail will wake someone at night	Add a timeout below the observed p95 and a fallback path first
Most successful responses missing usage	Cost and token budget unmeasurable	Don't enforce budgets on this endpoint yet
429s appear at concurrency of one	Limits are tighter than the loop assumes	Batch, cache, or move calls out of the loop

The "kill switch" in the table refers to a code path that can immediately stop all model calls at runtime—usually an environment variable or config flag that, when triggered, exits the loop rather than continuing to retry. It and the timeout are two different layers of defense: the timeout limits how long a single call can drag on, and the kill switch limits how long an entire automation chain can burn. A loop without a kill switch amplifies errors when the vendor side has problems—each failure triggers a retry, retries trigger rate limiting, rate limiting triggers more retries.

"Fallback path" refers to the alternative behavior when the primary endpoint is unavailable or exceeds a threshold—it can be degrading to a slower but more stable endpoint, queuing for a later retry, or skipping the step outright and marking it incomplete in an identifiable way. The decision table's second row requires "add a timeout below the observed p95 first," and the logic is: if the timeout is set above p95, then at least 5% of calls will spend far more time than expected before the timeout kicks in, and the tail latency passes straight through into your loop. Pressing the timeout below the observed p95 means deliberately giving up that 5% in exchange for a bounded loop duration. Whether that trade-off is acceptable depends on whether the task can tolerate skipping—it's not a blanket answer.

Limitations and Who Should Skip This

Twelve serial calls measure one machine, one path, one moment. Vendor behavior changes, free-tier conditions change, and one good afternoon proves nothing about next month. The probe also doesn't cover output quality, data handling, or licensing—those are separate reviews with their own owners.

Teams with service-level obligations, teams handling regulated data without a review agreement, and teams planning production traffic should not treat this process as a substitute for procurement. It's a first filter, not a contract.

For teams that want to re-run weekly without maintaining a machine, putting the collector on a MonkeyCode free server pointed at free model access is the shortest path the organizer knows—just remember to bring the current terms to the session and read them aloud.

One-line Summary

Use a ninety-line serial probe to measure free model endpoint latency, errors, and accounting—then decide whether automation can depend on it.

Paragraph Summary

The article proposes a ninety-minute workshop flow: use about ninety lines of Node.js code to serially call a free model endpoint twelve times, recording latency, status, error taxonomy, and usage fields, separating the verification of "can free access be tried" from "can an automation loop depend on it." The core measurement principle is serial over concurrent, because rate limiters fold the concurrent latency distribution into a step function that destroys the signal. When reading the numbers, the most important things to watch aren't the median but whether the usage field is missing (meaning the cost model is unreliable), whether p95 approaches the timeout (meaning the tail will wake someone at night), and whether 429s appear at concurrency of one. The article provides a decision table mapping signals to actions, and emphasizes that twelve samples only suffice for existence questions—not for estimating p99—and that the whole process is a first filter, not a procurement substitute.