On-Device Agent

When choosing a local inference solution, besides parameter count and benchmarks, one also needs to pay attention to activated parameters, memory usage, and whether format and latency optimizations have been made for tool calling.

On-device agents refer to AI systems that run locally on phones, laptops or edge devices and can autonomously plan and call tools to complete multi-step tasks, rather than merely doing single-turn question answering.

On-device agents place noticeably different demands on models than cloud agents, and low latency and stable structured output are often more critical than the absolute level of intelligence.

Regarding the issue of agent identity verification and human authorization, no widely adopted open standard has yet emerged in the industry.

Modern smartphones are already capable of running large MoE (Mixture of Experts) models locally and maintaining personal knowledge graphs.

The article recommends retaining a human confirmation step for high-risk actions that is independent of the model's reasoning chain, to avoid delegating authorization decisions entirely to a reasoning chain that may have been poisoned.

The article recommends not allowing agents to use user session tokens directly, and instead assigning them independent credentials with limited scope.

In most existing implementations, on-device agents directly borrow the user's session token, making agent requests indistinguishable from the user's own requests from the server's perspective.

Assigning independent keys to agents allows the server side to distinguish between "direct user operations" and "agent-proxied operations" and apply different permission boundaries accordingly.