Scaling Actors and Workers in Agent Substrate with HPA

Scaling Agent workloads is getting trickier by the day—when multiple Agents need to run simultaneously, is the memory of the underlying Workers sufficient? How do you dynamically schedule Actors without wasting resources? Agent Substrate's answer is to bring HPA (Horizontal Pod Autoscaler) into the WorkerPool layer, letting Workers scale on demand.

Agent Substrate's Scheduling Model

Agent Substrate abstracts an Agent's runtime environment as an Actor. Actors run on top of Workers, and Workers are managed collectively by a WorkerPool. The core design intent of this hierarchy is: when an Actor isn't scheduled, it sits in a "suspended" state—existing only as a record, consuming neither CPU nor memory. Only when a session request arrives is the Actor awakened and an attempt made to schedule it onto a Worker with available memory.

This "warm state" mechanism avoids the multi-second latency of a cold start. Once an Actor is awakened, it doesn't go through a lengthy recovery process—requests get a fast response. For scenarios that need to call Agents frequently, this design directly determines the quality of user experience.

The key constraint is memory. In the WorkerPool configuration, each Worker has a memory ceiling of 1Gi, while each Actor's template requires 768Mi. This means one Worker can hold only one Actor—a second Actor would need 1536Mi, exceeding the limit and getting rejected by the scheduler.

The Actor model originates from the concurrent computation theory proposed by Carl Hewitt in 1973. Its core idea is: the fundamental unit of computation is the Actor, each Actor has its own private state, and Actors communicate only through asynchronous message passing without sharing memory. This model naturally avoids the complexity introduced by lock contention and shared state. In Agent Substrate, an Actor abstracts an Agent's runtime environment—it contains the code, dependencies, and runtime state the Agent needs. The reason for choosing the Actor abstraction rather than processes or containers is that Agent invocation patterns are clearly intermittent: fast response is needed when a session arrives, followed by long idle periods after the session ends. An Actor's suspend mechanism matches this pattern perfectly—when suspended, an Actor retains only a state snapshot without consuming compute resources; when awakened, it restores from the snapshot, eliminating the overhead of initializing from scratch.

Deployment and Suspension: The Actor Lifecycle

Deploying Actors is straightforward. The following command creates six Actors in bulk:

for i in 1 2 3 4 5 6; do
  kubectl ate create actor "a$i" -a ate-lab-burst --template burst
done

After creation, all Actors start in the ACTOR_STATE_SUSPENDED state—they're merely a record plus a snapshot in GCS. Six Actors map to two Workers, forming a 3:1 ratio—two Actors can be scheduled, and the remaining four must wait for Workers to release memory before they can be awakened.

After using curl to request Actors a1 and a2, both Workers are fully occupied, each showing 768Mi/1Gi of memory usage. Requesting a3 at this point returns a 504 timeout—there isn't enough free memory to schedule a third Actor. After manually suspending a1 to free memory, a3 can be scheduled successfully. This manual process validates the strong dependency of Actor scheduling on memory.

How HPA Solves the Worker Scaling Problem

Manually suspending Actors is only a stopgap. The real question is: when multiple Actors are queued waiting for scheduling, how do you automatically increase the number of Workers?

The HPA configuration allows the WorkerPool to scale up to a maximum of 10 replicas. When six Actors are awakened simultaneously (via ./scripts/burst.sh -m hold -d 120), the first two schedule successfully, while the remaining four receive 504s due to insufficient memory. At this point, scale the WorkerPool to 6 replicas:

kubectl -n ate-lab-burst scale workerpool burst --replicas=6

The Workers scale from 2/2 to 6/6, all six Actors get scheduled, and requests return 200. The entire process requires no manual intervention to suspend or resume Actors—HPA senses the scheduling pressure and automatically increases Worker replicas, and the queued waiting Actors are immediately allocated.

The clever part of this mechanism is the separation of responsibilities: the Actor layer only cares about "is there enough Worker memory available for scheduling," while the Worker layer uses HPA to respond to load signals and decide "do we need more replicas." The two don't interfere with each other, yet they work in concert.

HPA (Horizontal Pod Autoscaler) is a workload scaling mechanism natively provided by Kubernetes. It periodically collects metrics from target resources (such as CPU utilization, memory usage, or custom metrics), compares them against user-defined thresholds, and then proportionally adjusts the number of Pod replicas managed by controllers like Deployment, ReplicaSet, or StatefulSet, bringing the actual load closer to the desired value. HPA supports setting minReplicas and maxReplicas to bound scaling, preventing unbounded growth or scaling down to zero. In the Agent Substrate scenario, the WorkerPool is configured as HPA's scaling target, and scheduling pressure (i.e., the number of Actors queued waiting for memory) becomes the trigger signal for scale-out. It's important to distinguish: HPA handles changes in horizontal replica count, whereas VPA (Vertical Pod Autoscaler) adjusts the resource quota of a single Pod. This article chooses HPA over VPA because a single Worker's memory ceiling determines that it can only hold a fixed number of Actors—increasing a single Worker's memory can't solve the scheduling bottleneck; increasing the number of Workers is the correct scaling direction.

Why Agent Scenarios Need This Kind of Scaling

The article closes with a thought-provoking point: the scope of an Agent's availability impact far exceeds that of traditional applications. A typical application going down might affect only one department or one function, but if an Agent becomes unavailable, multiple teams across the entire organization (engineering, marketing, HR) could be affected.

Agents are being embedded into more and more daily workflows, and their scalability is no longer a concern for the technical team alone. When an organization runs dozens or even hundreds of Agents simultaneously, the scheduling efficiency of the underlying Workers directly determines whether these Agents can reliably respond to requests. Through its suspend-wake mechanism and HPA-driven Worker scaling, Agent Substrate seeks to strike a balance between the two—neither letting idle Actors waste resources nor letting queued Actors wait indefinitely.

From a practical standpoint, the core value of this approach is shifting the scaling strategy from "pre-allocating large amounts of resources" to "dynamic scaling based on actual scheduling demand." For Agent-intensive deployment environments, this model can significantly reduce resource waste while ensuring response speed.

One-line Summary

Agent Substrate uses an Actor suspend mechanism and HPA-driven on-demand Worker scaling to solve memory bottlenecks in concurrent Agent scheduling.

Paragraph Summary

Agent Substrate abstracts Agent runtime environments as Actors, with WorkerPool managing Worker nodes collectively. Actors are suspended by default and only awakened when session requests arrive, at which point they attempt to schedule onto Workers with sufficient memory—this warm-state mechanism avoids cold-start latency. Since each Worker has a 1Gi memory ceiling while each Actor needs 768Mi, a single Worker can hold only one Actor, and excess Actors must queue. Manually suspending already-scheduled Actors can temporarily free memory, but bulk scenarios still require an automated solution. The article demonstrates that scaling the WorkerPool from 2 to 6 replicas via HPA lets all six queued Actors schedule successfully and return 200, with no manual intervention. This approach achieves a separation of responsibilities between Actor-level memory scheduling and Worker-level replica scaling, shifting the scaling strategy from pre-allocating large amounts of resources to on-demand dynamic scaling, significantly reducing resource waste while ensuring Agent responsiveness.