Hands-On Review of an Open-Source Multi-Agent Business Simulation Project

Multi-agent systems are moving from papers into interactive engineering practice. A Bilibili creator has open-sourced a multi-agent sandbox business project in which 2–4 coffee/oat/energy-bar shops make autonomous decisions under constraints of customer flow, bidding, inventory, and supplier negotiations. Based on the creator's demo and measured data, this article breaks down the architecture design, decision mechanisms, and empirical performance, and offers reusable development insights.

Project Overview: A Multi-Agent Commercial Showdown

The core of the project is a turn-based business simulator. Each round has 2–4 shops (configurable), each corresponding to one Agent, selling three product types: coffee, oats, and energy bars. Every round, Agents must decide: pricing, restock quantity, marketing spend, and up to three rounds of price negotiation with supplier Agents. Customer purchasing behavior is simulated by fixed random rules; after settlement, the system reports profit and cash changes, from which Agents absorb experience and adjust subsequent strategies.

The project encodes classic business trade-offs—low price for volume vs. profit priority, stockpiling to avoid stockouts vs. cash flow occupation, inventory spoilage vs. lost sales from stockouts—all as environmental constraints, letting different Agents compete under identical rules. This design makes results checkable, reproducible, and easy for side-by-side comparison of different models or strategies.

Technical Architecture: Core Agent Development Topics in One Place

In the demo, the creator lists the Agent tech stack covered by the project, which is almost a "multi-agent system development checklist":

• Planning and re-planning: A profit-goal-driven outer loop; Agents re-evaluate strategy after each settlement.
• Tool calling: Pricing calculation, inventory queries, supplier communication, etc. are all exposed as tools.
• Multi-agent communication: Collaboration and competition between shops, and between shops and suppliers, involving message passing and protocol design.
• Short-term and long-term memory: Distinguishing the current turn's context from cross-turn historical experience, with retrieval mechanisms deciding "what is worth remembering."
• Error experience filtering: Avoiding crystallizing failed decisions into memory—the creator stresses this is an easily overlooked engineering detail.
• HITL (Human-in-the-Loop): Supports a manual approval switch that can be toggled to model auto-execution.
• Database and idempotency: Turn-state persistence and duplicate-request handling—topics on the web-engineering side.
• Agent evaluation: Supports creating controlled experiments—fixing rule variables and only adjusting models or module switches—to compare profit and Token consumption.

The creator admits that, compared with pure coding-agent projects, this one is closer to "agent-leaning + web-leaning" hybrid job requirements, involving databases, Redis, idempotency, and other engineering skills, giving it a differentiating edge on a resume.

Memory mechanisms: In multi-agent systems, short-term memory typically refers to the current turn's dialogue context and temporary state (e.g., this round's quotes, inventory snapshot), while long-term memory refers to experience accumulated across turns (e.g., the effects of historical pricing strategies, suppliers' concession patterns). Retrieval mechanisms determine which historical fragments an Agent "recalls" when deciding—common approaches include similarity-based vector retrieval, time-decay weighting, or filtering by task type. Error experience filtering means assessing quality before writing to long-term memory, to avoid misjudging an accidental failure (e.g., a loss caused by random customer-flow fluctuation) as a strategic error and crystallizing it.

Idempotency: In turn-based systems, network retries or repeated user clicks can cause the same turn's decision request to be submitted multiple times. Idempotent design ensures repeated calls with the same request ID return the same result without deducting inventory twice or generating multiple orders. Common implementations include database unique constraints, Redis deduplication keys (e.g., SETNX), and request-fingerprint comparison. For Agent systems, idempotency also involves side-effect control of tool calls—for example, the "place order with supplier" tool must have idempotent semantics, otherwise retries could cause duplicate procurement.

Decision Modes: From Rule Engine to Autonomous Planning

The project offers three shop decision modes, corresponding to different levels of Agent autonomy:

1. Standard decision (workflow): A fixed rule engine executing preset logic, suitable as a baseline.
2. Trial calculation and review: The Agent proposes two plans, calculates their respective returns, then integrates a decision. The creator measured that this mode ranked first in all three matches, but Token consumption was nearly three times the baseline, and time nearly three times as well.
3. Autonomous planning and dynamic review: Fully driven by the large model, independent of hard-coded rules—the highest-autonomy mode.

This layered design lets developers clearly measure the benefits and costs of "increasing Agent autonomy." In the demo, the creator runs two shops simultaneously—one using large-model autonomous planning, one using a fixed rule engine—forming a direct comparison.

HITL (Human-in-the-Loop): Refers to retaining human approval nodes in an Agent's decision flow, so humans can intervene before critical operations (e.g., confirming quotes, approving restocks). There are usually two forms: blocking approval, where the Agent pauses execution awaiting human confirmation; and asynchronous review, where the Agent executes first but marks the action "pending review," allowing humans to veto or correct afterward. HITL's core value lies in risk control and trust building, especially for scenarios with low error tolerance such as finance and supply chains. The cost is reduced automation and increased response latency, so approval granularity should be configured dynamically according to task risk level.

Empirical Performance: Large Models Don't Always Win

In the default scenario, the creator pitted a large-model Agent against a fixed strategy; after five rounds the large model narrowly lost, earning only 996 coins versus the opponent's 1600. The creator attributes this to "quoting too high," but notes the large model showed stronger offensive awareness and a tendency to overtake later.

Switching to the "Supply Chain Storm" scenario (intense resource competition, ten rounds), the tables turned. The large-model Agent clearly outperformed the fixed strategy in resource-scarce conditions, with the profit gap widening as rounds progressed. The profit curves shown by the creator indicate the fixed strategy had almost no chance of a comeback in the mid-to-late game. This comparison shows: scenario complexity determines the marginal value of Agent autonomy. In simple scenarios with few variables, the stability of a fixed strategy may prevail; in scenarios with intense resource competition requiring dynamic trade-offs, the large model's planning ability can show its advantage.

Another noteworthy detail is the HITL trade-off. When the creator set manual approval to auto-execute, the Agent could fully autonomously complete negotiation and restocking decisions; retaining approval is similar to the confirmation mechanism in Codex, where each step requires a human click to agree. The two modes correspond to different risk preferences and degrees of automation.

Controlled experiment design: In Agent evaluation, controlled experiments require strict variable control—apart from the factor under test (e.g., model type, whether the memory module is enabled), the environment's initial state, random seed, opponent strategy, and number of rounds should all be kept consistent. Because multi-agent systems are stochastic (e.g., customer purchasing behavior simulated by random rules), a single match's result is insufficient to support conclusions; typically multiple repeated experiments are needed, reporting mean and variance. The three experiments and five-round/ten-turn matches in the creator's demo are small-sample exploratory tests, and their conclusions should be treated as directional references rather than statistically significant evidence.

Implications for Developers

The greatest value of this project isn't the gimmick of "letting Agents run shops," but that it provides an observable, comparable, reproducible multi-agent experimental ground. Developers can:

• Fix rule variables and change only models or module switches to quantify profit differences across configurations;
• Observe the trade-off between Token consumption and decision quality, providing a basis for production environment selection;- Trace which historical experiences the Agent referenced via turn details (in the creator's demo you can see experiences from rounds 5, 6, and 7 being invoked), debugging memory retrieval logic.

The creator says the project will be open-sourced on GitHub, with a free source-code access channel on their personal homepage. For engineers wanting to move from single-Agent applications to multi-Agent system development, this is a practice project with a moderate barrier and dense technical points.

A caveat: some conclusions in the demo are based on limited matches (e.g., three experiments, five rounds/ten turns), with limited statistical significance; performance may vary considerably across different models and prompts. Readers are advised to run multiple controlled experiments themselves before drawing conclusions.

One-line Summary

A Bilibili creator open-sources a multi-agent sandbox business project; tests show large models win in complex scenarios but consume more Tokens.

Paragraph Summary

This article breaks down a multi-agent sandbox business project open-sourced by a Bilibili creator, in which 2–4 shop Agents autonomously decide pricing, restocking, marketing, and supplier negotiations. The project covers core topics including planning, tool calling, multi-agent communication, short/long-term memory, HITL, and database idempotency, and offers three decision modes: rule engine, trial-calculation review, and autonomous planning. Tests show the large-model Agent narrowly loses to a fixed strategy in simple scenarios but has a clear advantage in the resource-competitive "Supply Chain Storm" scenario; the trial-calculation review mode yields the best profit but consumes about three times the baseline Tokens. The project supports controlled experiments and suits practice for hybrid Agent development roles, though the demo's conclusions are based on limited matches with limited statistical significance.