PARC: Parallelism-Aligned Pruning and Round-Aware Context Reuse for On-Device Multimodal Agents

Deploying on-device multimodal agents has long faced two structural tensions: the dimensions of pretrained attention structures may not divide evenly into the number of compute cores on edge accelerators, resulting in irregular partitioning; and standard prefix caching retains historical visual tokens in persistent KV caches, whose capacity grows linearly with the number of interaction rounds, quickly exhausting limited memory budgets. A study from rss proposes PARC (Parallelism-Aligned Pruning and Round-Aware Context Reuse), attempting to resolve both types of mismatch simultaneously at the levels of model structure and runtime caching.

Two Deployment Mismatches: From Core Alignment to Cache Bloat

PARC's starting point is a re-examination of on-device deployment constraints. At the model level, the attention head dimension often has no integer-multiple relationship with the target core count, forcing hardware schedulers into irregular partitioning and thereby reducing compute utilization. At the runtime level, conventional prefix caching stuffs system prompts, historical text, and multi-round visual observations indiscriminately into the KV cache; as dialogue rounds increase, visual tokens continuously occupy persistent space, directly shrinking the number of supported interaction rounds.

These two problems appear independent but actually point to the same design flaw: neither model structure nor runtime resource management is aware of the actual form of the deployment hardware. PARC's core idea is to align pruning strategies with parallelism and cache organization with token lifecycles.

Prefix Caching is a key optimization technique in LLM inference serving: when multiple requests share the same prefix (such as a system prompt or few-shot examples), it is computed once and the corresponding KV cache is persisted, allowing subsequent requests to reuse it directly and avoid recomputation. In standard implementations, the KV cache is appended linearly by token position, and all historical tokens reside in memory on equal footing. For multimodal agents, the visual tokens added each round (a single image may correspond to hundreds to thousands of tokens) continuously accumulate in the cache, causing memory usage to grow linearly with the number of rounds. This "append in chronological order" cache model fundamentally conflicts with the heterogeneity of token lifecycles in agent scenarios (system prompts remain valid long-term, while visual observations are only valid for the current round).

Parallelism-Aligned Structured Pruning

At the model level, PARC employs first-order Taylor importance evaluation combined with a Grouped-Query Attention (GQA)-aware selection strategy to perform structured pruning on attention dimensions. Unlike general-purpose pruning, its constraint explicitly targets hardware: the pruned attention dimension must be divisible by the target core count while preserving the query-to-key/value ratio and the per-head dimension.

The significance of this constraint is that pruning is no longer merely a reduction in parameter count but simultaneously changes the regularity of hardware partitioning. The study's parameter-matched control group further verifies this: with the same parameter count, only the core-aligned pruned version achieves speed gains, indicating that performance improvement mainly comes from core alignment rather than simple parameter reduction. This is a valuable reminder for on-device model compression practice—fewer parameters do not equal faster execution; whether the structure fits hardware parallelism is equally critical.

Grouped-Query Attention (GQA) is an attention variant between Multi-Head Attention (MHA) and Multi-Query Attention (MQA). In GQA, query heads are divided into groups, each sharing the same pair of key/value heads, thereby significantly reducing KV cache size and memory bandwidth requirements while maintaining model quality. GQA's group structure imposes additional constraints on pruning strategies: pruning must preserve the grouping ratio between query heads and key/value heads; otherwise, the structural integrity of the attention mechanism is broken, causing sharp accuracy degradation. PARC's GQA-aware selection strategy evaluates the importance of each attention dimension under this constraint, ensuring the pruned structure still conforms to GQA's grouping rules while satisfying the hardware alignment requirement of core-count divisibility.

KV Cache Reorganized by Lifecycle

On the runtime side, PARC splits cache content into three categories by lifecycle: system prompts, persistent text history, and transient visual observations. System prompts remain stable across rounds, text history updates progressively with tool execution, and visual observations are only valid for the current round. By synchronously updating text history during tool calls, historical images are removed from the persistent cache, increasing the number of interaction rounds that fit within a fixed memory budget.

This lifecycle-layered approach essentially changes the KV cache from "append in chronological order" to "manage by validity period." For multimodal agents typically centered on GUI operations, visual input is the main source of cache bloat; marking it as transient data frees up considerable persistent space.

First-order Taylor importance evaluation is a commonly used channel/dimension importance metric in structured pruning. Its core idea is to perform a first-order Taylor expansion of the loss function with respect to the output of a given attention dimension, approximating the impact on the loss of removing that dimension by the product of "gradient × activation value." The smaller the absolute value of the product, the weaker the dimension's contribution to the model output and the smaller the accuracy loss after pruning. Compared with heuristic methods based on weight norms, Taylor evaluation leverages gradient information on specific input data and more accurately reflects dimension importance under the actual task distribution. PARC combines it with GQA structural constraints, prioritizing removal of dimensions with the lowest Taylor importance while satisfying hardware alignment conditions, thereby balancing structural constraints and accuracy preservation.

Measured Performance: Lossless Accuracy and Efficiency Gains

The study evaluated incremental interface performance on 2,062 GUI actions and observed no accuracy regression. Task-aware recovery training restored the action-type accuracy gap to 98.7% and coordinate hit rate to original levels. Under seven-core deployment conditions, hardware-aligned pruning reduced visual encoder latency by 11.4%, increased decoding throughput by 23.8%, and lowered per-stack high-water memory by 12.7%.

Two points are worth noting behind these numbers. First, recovery training can compensate for most of the accuracy loss caused by pruning, indicating that structured pruning combined with targeted fine-tuning is a viable on-device deployment path. Second, latency and throughput improvements are accompanied by reduced memory usage, meaning PARC offers multi-dimensional practical value on resource-constrained devices rather than a simple trade-off of memory for speed.

Implications for On-Device Agent Engineering

PARC's value goes beyond a set of experimental numbers; it suggests that optimizing on-device agents requires co-design of model and system. If pruning strategies are divorced from the target hardware's core topology, gains will be offset by partitioning overhead; if cache management does not layer lifecycles, no amount of memory can fill the gap of multi-round visual interaction.

For teams pushing multimodal agents to edge devices, this approach offers a actionable checklist: first confirm the divisibility relationship between attention dimensions and core count, then examine which content in the KV cache truly needs cross-round persistence. Only by combining both can more interaction rounds be supported within a fixed budget.

One-line Summary

PARC resolves deployment mismatches for on-device multimodal agents through parallelism-aligned pruning and lifecycle-aware cache reuse.

Paragraph Summary

PARC addresses two structural tensions in on-device multimodal agent deployment—attention dimensions not dividing evenly into hardware core counts, causing irregular partitioning, and linear bloat of historical visual tokens in standard prefix caches exhausting memory—by proposing coordinated optimization of model structure and runtime caching. At the model level, it uses first-order Taylor importance and GQA-aware selection for structured pruning, making attention dimensions divisible by core count while preserving the query-to-key/value ratio; at the runtime level, it splits the KV cache by lifecycle into system prompts, persistent text history, and transient visual observations, updating text history during tool execution so historical images exit the persistent cache. On 2,062 GUI actions, there is no accuracy regression, and recovery training closes 98.7% of the action-type accuracy gap; under seven-core deployment, latency drops 11.4%, throughput rises 23.8%, and memory drops 12.7%. Parameter-matched controls confirm that speedup mainly comes from core alignment rather than parameter reduction.