TomasuLLM
A runtime acceleration system for LLM-based agents, inspired by the out-of-order execution concept of the Tomasulo algorithm. It employs speculative execution, copy-on-write sandbox isolation, and in-order commit mechanisms to reduce tool-call waiting latency and improve overall agent throughput.
Timeline (last 90 days)
TomasuLLM is a runtime system that enables an agent's tool calls to execute out of trajectory order while ensuring correctness of task execution through out-of-order execution with in-order commitment.
TomasuLLM works by drafting actions the agent may execute in the future and running them ahead of time in isolated copy-on-write sandboxes.
In TomasuLLM, speculative results are formally committed only after they have been validated against the committed state in trajectory order.
TomasuLLM achieves a 1.31× speedup on 100 tasks from SWE-bench Verified.
TomasuLLM achieves a 1.35× speedup on 28 tasks from Terminal-Bench 2.0.
TomasuLLM achieves a 1.27× matched-progress improvement across 18 sessions on SWE-Marathon.
TomasuLLM's speedup ratio increases as tool latency grows.
TomasuLLM produces zero false accepts across 4,010 audited commit-validation records.
The TomasuLLM paper is a newly submitted arXiv preprint and is currently still at an early stage.
All Facts (8)
TomasuLLM is a runtime system that enables an agent's tool calls to execute out of trajectory order while ensuring correctness of task execution through out-of-order execution with in-order commitment.
50%UnverifiedTomasuLLM works by drafting actions the agent may execute in the future and running them ahead of time in isolated copy-on-write sandboxes.
50%UnverifiedIn TomasuLLM, speculative results are formally committed only after they have been validated against the committed state in trajectory order.
50%UnverifiedTomasuLLM achieves a 1.31× speedup on 100 tasks from SWE-bench Verified.
50%UnverifiedTomasuLLM achieves a 1.35× speedup on 28 tasks from Terminal-Bench 2.0.
50%UnverifiedTomasuLLM achieves a 1.27× matched-progress improvement across 18 sessions on SWE-Marathon.
50%UnverifiedTomasuLLM's speedup ratio increases as tool latency grows.
50%UnverifiedTomasuLLM produces zero false accepts across 4,010 audited commit-validation records.
50%