Agent A1 Open Source: 35B MoE Local Inference Speed Test & Deployment Guide

What is Agent A1?

A homegrown open-source model from Shanghai is generating buzz in the local AI community. Agent A1 is released by Intern Science (书生·科学), a team under the Shanghai Artificial Intelligence Laboratory, and is officially positioned as "the open-source hub for AI for Science." The model quickly attracted attention after launch, with a focus on agentic tasks and full support for free local deployment.

Architecturally, Agent A1 is a 35 billion-parameter MoE (Mixture of Experts) model that activates only about 3 billion parameters per inference pass. MoE is a design that splits a large neural network into multiple "expert sub-networks" — during inference, a gating mechanism dynamically selects a small number of experts based on the input, rather than activating all parameters. This architecture was first proposed in 1991 and has been adopted at scale by top-tier labs like Mistral and Google in recent years (e.g., Mixtral 8x7B, Gemini 1.5). Its core advantage is the decoupling of model "capacity" (total learnable knowledge) from "inference cost" — 35B total parameters can encode knowledge comparable to a dense large model, while each forward pass only requires the compute of ~3B active parameters, dramatically reducing VRAM usage and inference latency. This "large capacity, small activation" design is precisely why it can achieve high-speed inference on consumer hardware.

It also supports a 256K ultra-long context window. The context window refers to the maximum number of tokens a model can "see" in a single inference pass — 256K means it can process roughly 200,000 Chinese characters simultaneously, equivalent to a mid-length novel. Achieving such long contexts typically relies on positional encoding improvements (such as RoPE extrapolation) and attention mechanism optimizations like FlashAttention. For agentic tasks, a long context is especially critical, as agents need to maintain coherent "memory" across multiple tool calls, long reasoning chains, and large amounts of code context — virtually no real-world task scenario will hit the context limit.

Intern Science has previously released models including Intern Agent, and Agent A1 is their latest iteration in the agentic direction, deeply optimized for long-horizon tasks. Long-horizon tasks are one of the core challenges in AI agent research — they refer to complex goals that require multiple steps and rounds of decision-making to complete, such as autonomously developing a complete software feature (requirements analysis → code writing → debugging → testing → documentation), or conducting systematic literature research and writing a review. These tasks demand robust planning capabilities, tool-calling ability, and cross-step state maintenance.

Performance Benchmark: 95 Tokens/Second Local Inference

In local model testing, what stands out most about Agent A1 is its inference speed. Running on a Mac Studio with an M4 chip, it achieved 95 tokens/second — fully local, no internet required.

Tokens per second (tok/s) is the core metric for measuring language model inference speed. A single Chinese character typically corresponds to 1–2 tokens, and an English word to about 1–1.5 tokens. For practical usability, 30 tok/s or above is generally considered "smooth and usable," while 95 tok/s approaches human fast-reading speed — the wait is virtually imperceptible. The key factor driving local inference speed is hardware memory bandwidth — the M4 chip's unified memory architecture delivers over 120 GB/s of memory bandwidth, and combined with the MoE architecture activating only ~3B parameters per pass, this produces the impressive result.

Horizontal comparison with similar models:

• Agent A1: 95 tokens/second — fastest
• Gemma 4 Coder: noticeably slower than A1
• Qwen series: even slower

While maintaining solid generation quality, A1's speed places it firmly in the top tier of local models. Note that actual speeds will vary depending on hardware configuration and background load.

On comprehensive capability benchmarks, the picture is more grounded. According to the blogger's custom local model evaluation (GoldieBench), Agent A1 scored 4.8/10, ranking second — outperforming Gemma 4, but clearly behind the Qwen series (around 7.14).

To be objective, A1 is not a flagship model chasing top overall scores, nor is it competing head-on with models like Fable 5. Its value lies in specialized capabilities for specific scenarios.

Capability Boundaries: Strong at Agentic Tasks, Weak at Visual Rendering

In real-world testing, after connecting A1 to a local coding environment, the blogger successfully built several small projects: an open-world game called Dragon Realm, a virtual keyboard with volume and oscillator controls, a neon-style Snake game, and a 3D solar system visualization. The quality of the 3D solar system output was notably impressive for a small local model.

Agent A1 has a very clear positioning — this is a model for research and agentic tasks, not a visual/UI design model. The officially stated optimization targets include:

• Long-horizon search
• Engineering and development tasks
• Scientific research assistance
• Instruction following
• Tool calling

Across multiple agentic benchmarks, A1 performs strongly. The team claims state-of-the-art performance on SEAL-0, IFBench (instruction following), and BrowseComp for long-horizon search and tool use; on the HLE science benchmark, real-world test data shows it outperforms Qwen 3.6 and Step 3.5 Flash.

Strengths and Weaknesses at a Glance

Core strengths:

• Agent-tuned specifically for agentic scenarios — strong at tool calling and scientific research tasks
• Fully free local deployment via Ollama
• Ideal for building local agent loops and offline coding workflows

Notable weaknesses:

• Weak at visual rendering, UI polish, and frontend-type tasks
• Overall capability scores still trail the Qwen series

Local Deployment Tutorial: Quick Start with Ollama

The barrier to deploying Agent A1 is low — it natively supports running via Ollama, and a few commands are all you need to get the model up and running. Ollama is one of the most popular local LLM runtime frameworks available today. It wraps complex components like model quantization, hardware acceleration, and the inference engine (based on llama.cpp) into a clean CLI tool, allowing users to complete the full pipeline from model download to API service launch with a single command. Ollama natively supports Apple Silicon Metal acceleration, NVIDIA CUDA, and AMD ROCm, and provides a locally-hosted REST API compatible with the OpenAI format — enabling seamless integration with higher-level agent frameworks like Hermes and Open WebUI. It has become the de facto standard entry point for open-source local deployment.

To set up a local coding agent, you can connect it to a local agent framework like Hermes:

1. Open a terminal and navigate to the Hermes directory
2. Switch the model backend to Ollama
3. Confirm that Agent A1 has been downloaded in Ollama
4. Create a new local model profile for A1 under Profiles

Once these steps are complete, you'll have a fully offline, privacy-preserving, zero-cost coding agent — capable of assisting with coding work even on a plane or in an environment without internet access.

Summary: A Pragmatic Choice for Local Agentic Workflows

Agent A1's value is not in "topping the leaderboards" — it lies in finding a pragmatic balance across three dimensions: inference speed, free availability, and full offline operation. The 95 tok/s local inference speed, 256K ultra-long context, and MoE architecture optimized specifically for tool calling and long-horizon tasks together form a reliable foundation for building local agentic workflows.

For developers who prioritize data privacy, need to run coding agents offline, or have scientific research assistance needs, this open-source, locally deployable 35B MoE model is a new entry in the Chinese model ecosystem worth taking seriously. That said, if your primary need is frontend visual effects or maximum overall capability, the Qwen series remains the stronger choice for now.

The most practical advice remains the same as always: with local models, always run them yourself — leaderboards can only point you in a direction; only real-world testing can tell you whether a model is truly right for your workflow.

Key Takeaways