Free Compute Cluster Admission: Local Drill with Free-Path Dwell

During the nightly batch window, your free Worker shows idle memory and idle CPU cores, yet the queue age has climbed to 41 seconds. There are still 90 seconds left before the original enqueue deadline, and two unused retries remain in the job policy. Raising concurrency looks like the gentle choice—and that reading is exactly wrong.

The Core Tension: Idle Resources Are Not a Service-Time Promise

The host utilization chart says the machine still has headroom, while the queue age says the batch is already sliding toward a timeout. You need a metric that reconciles both views—this article calls it free-path dwell.

The calculation is simple: dwell = queue_age + expected_service + retry_tax. Compare it against the remaining deadline slack, and only allow the free path when dwell still fits inside slack. If dwell exceeds slack, free capacity is the wrong bet, and the job should be deferred or handed to a reserved Worker.

The core principle: don't buy extra retries on slots you don't own. Cheap token pricing can't repay a missed deadline—what actually expires is your wall-clock budget.

Declarative Inputs for the Local Drill

Before touching any host, write the conditions down. This drill runs as a local process, not on your cluster. The inputs below are examples—adjust them to your deadlines and record the changes, and don't tune parameters only after you see a passing result:

• Batch size of 40 jobs, one first attempt each
• Estimated 1200 tokens per job to complete
• Expected service time on the free path: 8 seconds
• Deadline slack at enqueue: 90 seconds
• Retry tax equal to one full service interval
• Maximum tolerable queue age: 30 seconds

Note that the token count is your estimate—it is not a vendor quota, nor any provider's billing statement. What is being tested here is admission math, not model quality.

Keep the Topology to One Page

Keep the components small enough that failures stay localized. One gate sits between enqueue and all Worker selection, and it can admit, defer, or shift to reserved:

enqueue -> dwell_gate -> free_path
                 |-> defer_queue
                 |-> reserved_path

The gate reads four fields and writes one decision. Record these fields on every attempt, including the rejections:

• queue_age_s: seconds since the job was enqueued
• expected_service_s: the declared service time
• retry_tax_s: the wall-clock price of one unused attempt, in seconds
• deadline_slack_s: deadline minus elapsed wall-clock time
• decision: admit_free, defer, or reserved

Utilization stays on the dashboard, out of the gate's internals—it explains whether the host is hot, not whether the queue is late. Slack is the threshold, because a deadline is an absolute fact. Queue age has veto power, not full decision power: a young queue can still lose if the retry tax is fat.

Running the Gate on a Laptop

Save the following script as dwell_gate.py, run it with Python 3, and don't put it on the cluster. It calls no models and opens no ports.

#!/usr/bin/env python3
# Local admission drill. Expected lines are comments, not your metrics.
from dataclasses import dataclass

@dataclass
class Job:
    queue_age_s: float
    expected_service_s: float
    retry_tax_s: float
    deadline_slack_s: float
    max_queue_age_s: float = 30.0

def dwell(job: Job) -> float:
    return job.queue_age_s + job.expected_service_s + job.retry_tax_s

def decide(job: Job) -> str:
    if job.queue_age_s > job.max_queue_age_s:
        return 'defer'
    if dwell(job) > job.deadline_slack_s:
        return 'reserved'
    return 'admit_free'

def decide_safe(job: Job) -> str:
    if job.expected_service_s <= 0 or job.deadline_slack_s <= 0:
        return 'defer'
    return decide(job)

samples = [
    Job(12, 8, 8, 90),   # expected admit_free
    Job(41, 8, 8, 90),   # expected defer, age cap
    Job(20, 8, 8, 30),   # expected reserved, dwell over slack
    Job(20, 8, 0, 30),   # expected admit_free, retry tax removed
    Job(10, 0, 8, 90),   # expected defer, non-positive service
    Job(10, 8, 8, -1),   # expected defer, negative slack
]

if __name__ == '__main__':
    for i, job in enumerate(samples, start=1):
        choice = decide_safe(job)
        print(
            'job=%s dwell=%.0f slack=%.0f decision=%s'
            % (i, dwell(job), job.deadline_slack_s, choice)
        )

Run the file with a single local Python command, staying in the directory where you saved the script. A missing file is a drill error, not a capacity signal.

python3 dwell_gate.py

Expected Output

Treat the following as expected output, not measurements:

job=1 dwell=28 slack=90 decision=admit_free
job=2 dwell=57 slack=90 decision=defer
job=3 dwell=36 slack=30 decision=reserved
job=4 dwell=28 slack=30 decision=admit_free
job=5 dwell=18 slack=90 decision=defer
job=6 dwell=26 slack=-1 decision=defer

Line by line: Job 1 fits, so the free path is an acceptable bet; Job 2 is young on slack but old on queue age—defer it rather than hoping a shared slot clears up; Job 3 is still under the age cap, but dwell exceeds slack, so it leaves the free path; Job 4 fits again once the retry tax is removed—this is the trap in retry policies: a free retry is only actually cheaper when the extra seconds still fit; Job 5 has zero service time, so the guard defers; Job 6 has negative slack, so it is not admitted. Both guards fail closed, which is the whole point of the drill.

queue_age_s	dwell_s	slack_s	decision	why
12	28	90	admit_free	dwell fits and age is young
41	57	90	defer	age cap trips before slack
20	36	30	reserved	age is fine, dwell is not
20	28	30	admit_free	dwell fits once retry tax is removed
10	18	90	defer	service-time guess is non-positive
10	26	-1	defer	deadline has passed

Injecting a Boring Failure

Before trusting the printed gate, inject a stuck attempt: set queue age to 50, slack to 40, and force the retry tax to a full service interval of 8. Dwell becomes 66 against 40 seconds of slack, the age cap trips first, and the printed decision should be defer. If the script still prints admit_free, stop the rollout.

A second injection sets expected service time to zero. Dwell will be undercounted and the gate will admit too early—reject non-positive service-time guesses before comparing. Negative slack means the deadline has already passed: defer, alert, and don't hide the miss inside a retry.

Cleanup is deleting the local script and stopping the process:

rm -f ./dwell_gate.py

If a unit or cron copied this drill, disable that unit first, then confirm the process table no longer shows the script. Leave production flags untouched until the canary section below.

Pricing Tokens in Occupied Seconds

A free completion still costs you wall-clock seconds you can't get back. Multiply the estimated token count by dwell seconds to get a pressure gauge—that product is not a bill and not a benchmark.

1200 tokens over 66 seconds is a heavy wait; the same tokens on an 8-second reserved turn weigh far less. What you are buying is clock time, not just completion text. Count retries as extra token copies rather than a free do-over—two attempts can double both the token load and the dwell. If the model jump has its own queue, add that age in too; a zero price does not make the second queue instantaneous.

Wall-clock time and CPU time are two different measurement units. CPU time only accumulates the seconds a process actually occupies the processor executing instructions; a process blocked waiting on I/O, queuing, or rate limiting consumes almost none of it. Wall-clock time is the real physical seconds from start to finish, ticking whether the process is running, ready, or blocked. For deadline-sensitive batch work, the only meaningful one is wall-clock time: every second a job waits in queue eats into deadline slack, but host utilization metrics can't see that wait. This is the root of how "idle memory, idle CPU, yet queue age has climbed to 41 seconds" can all be true at once—utilization measures resource occupancy, not how far your job is from its deadline. Therefore admission decisions must be built on wall-clock quantities (queue age, service time, retry tax), relegating utilization to an explanatory dashboard role.

Where the Free Path Is Allowed to Help

Disclosure: this article was prepared as part of a MonkeyCode product promotion. The operator notes for this draft mention two availability claims: free model access and a free server option.

Neither claim states quota, chip, or expiry. Confirm these facts in the current product documentation before planning. The free path can still be the wrong bet for this batch.

Only use it when the dwell gate returns admit_free. The free server suits this kind of local work—use it to replay the gate, not to absorb a late queue. If that option disappears tomorrow, your SLO must still hold, so don't put the fallback on the same free host. Reserved means a Worker you can drain and roll back. Shared free capacity can't be a rollback target. Put longer experiments on jobs that are safe to defer, and point only jobs with abundant slack at the free model path. Keep deadline work on the reserved path you already trust—free access can carry a drill, not a commitment.

Thresholds Worth Arguing About in Review

Choose deadline slack as the primary admission threshold. A deadline is a wall-clock fact, not a utilization metric. Queue age is a leading indicator you watch on the side. Utilization comes third, and it should have no vote—a cold GPU can still sit behind someone else's queue, and a hot GPU can still serve you as long as your dwell fits.

Starting caps for the declared workload above: defer when queue age exceeds 30 seconds; leave the free path when dwell exceeds remaining slack; price each unused retry at a full service interval; and reject zero or negative service-time guesses. Revisit the 30-second cap after a week of real logs. If defers cluster near 20 seconds, tighten the cap; if reserved switches dominate, your service-time guesses are too timid.

Deadline slack is the seconds remaining between a job's absolute deadline and the current wall-clock time—a hard budget that decreases monotonically and fails the moment it hits zero. Setting it as the primary admission threshold, rather than using utilization or queue age, is because only slack directly encodes the fact of "is there still time?" In practice slack comes in two layers: the initial slack granted at enqueue (90 seconds in the example) and the remaining slack at decision time (30 seconds in the example). Dwell must be compared against remaining slack to be meaningful—judging with initial slack systematically overestimates the margin. When slack is negative, the deadline has passed, and any retry just hides the failure in a later attempt; the correct action is to defer and alert. Likewise, queue age is only a leading indicator: young doesn't mean safe (the retry tax may be fat), and old isn't necessarily fatal (if service is very fast and slack is abundant)—which is why the age cap vetoes in the gate while the remaining decisions go to the dwell-versus-slack comparison.

Alert on Misses, Not on Idle Hosts

Wake people up when an admit_free decision later records a slack miss. Don't alert just because memory looks full or empty—a miss is the user-facing failure you set out to avoid.

decision=admit_free AND slack_miss=true

Route that alert to the same channel as the queue-age page, and attach dwell, slack, and retry count to the incident record. Without those three fields, the on-call engineer will end up chasing utilization again. If misses exceed 1% of free admissions, freeze the flag. That 1% is just a starting line for this drill—replace it with your error budget before production.

How to Roll Back Without Draining the Wrong Queue

Ship the gate behind a flag, off by default. Canary only a 5% slice of batch jobs, and watch slack misses, defer rate, and reserved-switch rate. If the miss rate rises, turn off the feature flag, drain the jobs already admitted on the prior stable path, and don't flush the deferred queue into free Workers.

Paste the following steps into the runbook before the canary starts: disable the feature flag named dwell_gate_enabled; confirm new jobs skip the gate within a minute; let in-flight reserved jobs finish, then stop the canary hosts; and delete the local drill files so cron can't resurrect them.

Rollback is complete when the new decisions stop appearing in logs. Keep the reserved pool warm until that log line goes quiet. A half-disabled gate is worse than no gate.

Who Should Skip This Method

• Skip it when work can neither wait nor move: a pager path with no reserved fallback needs a different design; this gate only chooses between admit, defer, and reserved.
• Skip it for interactive leases that need sub-second slack: that failure mode needs different hold-time controls—don't mix it into this batch-deadline drill.
• Skip it when you need a model-quality benchmark: the script never sends a prompt and never scores an answer, so it won't tell you which model is cheaper per token.
• Skip it when service time is still an unknown distribution: a single guessed interval will lie under a long tail—measure p95 service time before automating the switch.

What to Record Before the Next Review

Keep one line per decision, not one per host. Store queue age, dwell, slack, decision, and retry count, and drop raw prompts from the logs to keep the cost file small.

At the end of a week of logs, review three ratios, keeping the window fixed for comparability: the count of free admissions that later missed their slack budget; the count of defers that could have fit on a reserved Worker; and the count of reserved switches that dwell would have fit for free.

The first ratio means your service-time guesses are too generous; the second means you deferred work that a reserved turn could have saved; the third means the age cap is stricter than the deadline. Adjust one cap at a time, then rerun the local samples—changing both the age and slack rules at once hides the cause. Write the new cap next to the old expected-output line.

Free capacity is a tool, not a standing reservation. Use the free model path when dwell fits inside your slack; when it doesn't, pay for the Worker you can actually drain. If these options are available in your account, replay the samples and run that replay on the free server before attaching any batch—check the current docs first, because availability is not an SLO.

One-line Summary

Use dwell-versus-deadline-slack admission math to decide whether the free path is still the right bet.

Paragraph Summary

The article notes that "idle resources" and "climbing queue age" can both be true during nightly batch work, because utilization is not a service-time promise. The author proposes free-path dwell = queue age + expected service time + retry tax, and compares it against remaining deadline slack: only when dwell fits inside slack is the free path allowed, otherwise defer or shift to a reserved Worker. The core trap is the free retry—it's only actually cheaper when the extra seconds still fit inside slack. The article provides a local Python drill script demonstrating six samples across the three decisions admit_free / defer / reserved, and injects two fail-closed guards for non-positive service time and negative slack. It closes with engineering practice: price tokens in wall-clock seconds, alert on slack misses rather than idle hosts, use a feature flag for a 5% canary with a drainable rollback path, and the boundaries of where this method applies.