Orca

In-flight Batching technology was first proposed in the Orca paper and was subsequently widely adopted by inference frameworks such as vLLM and TensorRT-LLM.

The author ultimately chose to keep using herdr rather than Orca, and the decisive question was whether they wanted a new app or a better terminal, and the author chose a better terminal.

Orca is a GUI-first ADE that can display Claude and Codex usage, rate limit reset times, supports worktree functionality, can fan out one prompt to multiple agents, and annotates AI-generated diffs line by line.

The core result of the vLLM paper is that, compared with FasterTransformer and Orca, vLLM improves the throughput of popular LLMs by 2-4x at the same latency level, with greater improvements for longer sequences, larger models, and more complex decoding.

Orca is an agent development environment that distributes the same prompt to multiple agents running in independent git work trees, compares and merges the best answers, supports more than 40 command-line agents, and provides iOS and Android apps.

In Orca, each agent is assigned an independent worktree, allowing different agents to work in parallel without file conflicts, which is fundamentally different from opening the same project in multiple windows.

Orca is a desktop tool positioned as an 'Agent cockpit' that allows multiple command-line programming agents such as Codex, Claude Code, and OpenCode to run side by side, with each agent having its own independent Git worktree, and supporting cross-platform and remote execution.

The Orca project was created in March and has accumulated over 70,000 Stars to date, and is in a state of high-frequency active development.

Orca supports all three major desktop platforms — macOS, Windows, and Linux — and also provides iOS and Android applications.

Orca can drive programming agents on remote hosts via SSH worktrees, allowing code execution to take place on the server side while the desktop or mobile device serves as the control console.

Orca does not sell models nor cover users' subscription costs; users can directly reuse model subscriptions they have already purchased, with no additional model call fees stacked on top.

Orca's pricing is benchmarked against Devin's Pro plan, which is $240 per year.

The correspondence between Orca and Devin is loose — Devin itself is an agent, whereas Orca is a cockpit for driving agents the user already has, offering an alternative path rather than directly competing on features.

According to Orca's README, the development team releases updates every day, the feature list is always behind reality, and the accurate feature inventory is the changelog rather than written documentation.

To understand what Orca can actually do at any given moment, checking the changelog is more reliable than reading the documentation.

Orca (a paper published in 2022) was the first to propose the concept of Continuous Batching.

The actual stability of Orca and the maturity of its multi-agent coordination still need to be further validated in real-world projects.

Orca uses TypeScript as its primary development language.

Orca supports connecting any coding agent and uses the user's own subscription account (BYO Subscription model) to drive operation, without being locked into any single model or service provider.

Orca provides three modes of operation simultaneously: desktop, mobile, and VPS.