Ditching the Vector Database: SQLite BM25 and a 3.6ns Rust Sentinel for Multi-Agent Refactoring

When a multi-agent system runs in real trading scenarios, which parts should be handed to the LLM and which must be backed by deterministic systems? An open-source hedge fund experiment team, driven by community feedback, completely refactored its architecture: abandoning the vector database, switching to SQLite's BM25 full-text search, and rewriting the real-time risk-control sentinel in Rust. Their benchmark results and lessons learned are valuable to anyone building an agent pipeline.

Fixing Three Key Failure Modes

1. Letting the LLM directly interpret raw metrics inevitably causes polarity hallucinations

A community commenter pointed out: don't let the LLM interpret raw quantitative factors without ontology constraints. After verification, the team admitted the criticism was entirely correct. When they fed bare numbers like PER = 85.0 and 20-day historical volatility = 65% directly to the model, the agent frequently fabricated bullish justifications. For example, the model would argue that "a sharp jump in volatility indicates expanding business activity and strong buying momentum"—which is exactly contrary to basic risk management logic.

The fix was to introduce a strict Factor Ontology Schema: define 11 core quantitative factors and bind an explicit polarity contract to each factor, divided into three categories: HIGHER_IS_BULLISH (e.g., ROE, order flow imbalance), LOWER_IS_BULLISH (e.g., PER, debt ratio), and EXTREME_IS_RISK (e.g., volatility). Before any agent sees the data, the schema layer makes a deterministic pre-judgment of the metric and attaches an explicit signal (BULLISH, BEARISH, or RISK) to the state. The team reports that after implementation, polarity hallucinations in the investment committee dropped to 0%.

The universal significance of this approach: LLMs are good at synthesis and debate, but not at making directional judgments on numerical values. Stripping directional judgment out of model reasoning and hardening it with a schema both eliminates hallucinations and makes every decision auditable.

The essential difference between BM25 and vector similarity retrieval

BM25 is a classic probabilistic ranking function in information retrieval. It scores each document based on term frequency, inverse document frequency, and document length normalization, with the core assumption that the more often and the more rarely a query term appears in a document, the more relevant it is. In contrast, embedding retrieval compresses text into a dense vector and measures semantic closeness with cosine similarity. BM25's advantage is exact matching: when a query contains terms like "trailing stop sweep," only documents that actually contain that term will score high; dense vectors may bring "stop-loss" and "risk control" closer, but can also rank semantically related yet contextually irrelevant fragments higher. BM25 requires no training and no external API, and a single query often completes in microseconds, so in scenarios with short texts and strong terminology like structured logs and retrospective notes, it is often more reliable than high-dimensional embeddings.

2. Using a vector database for reflective memory is severe over-engineering

When implementing the Reflexion pattern (retrieving "lessons learned" from past failed trades), the team initially considered an embedding API plus a local vector store. Another commenter suggested simply abandoning embeddings and using SQLite's WAL mode with FTS5's BM25 full-text search instead. The team benchmarked the two approaches on a Linux server:

• Embedding API + vector similarity: round-trip latency 200ms to 450ms, external API costs, and occasional rate limiting.
• SQLite FTS5 BM25: latency 0.35ms, zero external dependencies, zero API cost.

More critical is the difference in retrieval quality. Trade retrospectives rely on highly specific contextual keywords, such as "before semiconductor earnings," "trailing stop sweep," "volume spike," "bearish divergence." The team found that BM25's keyword relevance hit exact-match historical lessons more reliably than fuzzy cosine similarity in high-dimensional embedding space.

The implication for agent memory design: when your retrieval need is essentially "find historical fragments containing these specific terms," keyword search is often superior to semantic search, and its latency and cost are lower by more than an order of magnitude.

3. Python cannot serve as a safety line: the 3.6ns Rust sentinel

The team believes Python is suitable for high-level agent orchestration (LangGraph, state graphs, API calls), but using Python loops for real-time risk monitoring during a market flash crash is dangerous: GIL contention and L1/L2 cache misses caused by scattered object pointers will drag down response.

They rewrote the execution safety sentinel in pure Rust, using a Struct-of-Arrays (SoA) memory layout, and added #![forbid(unsafe_code)] to disable unsafe code. Prices, stop-loss trigger lines, and trailing stop thresholds are all placed in contiguous flat slices, so the CPU doesn't need to chase pointers and data stays within cache lines. The team's real benchmarks on an Ubuntu server:

• Single-position stop-loss/take-profit evaluation: 3.63 nanoseconds
• Batch evaluation of 1000 positions: 3.62 microseconds
• Volume clock conversion for 1000 K-lines: 3.90 microseconds
• Python-to-Rust persistent pipeline IPC round trip: 2.88 microseconds

The division of labor thus becomes clear: Python handles "slow thinking"—a single agent committee meeting lasts up to 15 minutes; the Rust sentinel monitors the entire portfolio at nanosecond granularity.

Struct-of-Arrays (SoA) memory layout

In the traditional Array-of-Structs (AoS) layout, each position object packs fields such as price, stop-loss line, and trigger threshold together; when iterating, the CPU jumps across memory following object pointers, easily causing L1/L2 cache misses, and a single cache miss can cost hundreds of clock cycles. SoA instead arranges the values of the same field across all positions into a contiguous flat slice, so during iteration the data is strictly adjacent in memory, can be loaded as full cache lines, and is easier for the compiler to auto-vectorize. In nanosecond-level risk-control loops, this layout often brings throughput improvements of several times or more, and it is language-agnostic—an underlying technique worth borrowing for any latency-sensitive system.

Universal Lessons from This Architecture

The team distilled the lessons into one sentence in their conclusion: the LLM should only handle what it is good at—context synthesis and adversarial debate; all deterministic parts, including hard stop-losses, factor polarity, and fast memory retrieval, should be handed to strict schemas, fast local databases, and compiled systems languages.

Three conclusions transfer to almost any agent system:

1. Don't hand directional judgment of numerical values to the model; use a schema for deterministic pre-judgment, and let the model only explain and debate.
2. Reflective memory doesn't have to be vectorized; keyword search is fast and accurate in specific contexts, at nearly zero cost.
3. Real-time safety lines must break away from the interpreter; memory layout and language choice directly determine the lower bound of response.

It is worth noting that at the end of the post, the team stated that English is not their native language and that the original notes were formatted and polished with the help of an LLM; the code and benchmarks are open-sourced on GitHub (lumina-ai-hedge-fund), available for verifying the concrete implementation of the Rust SoA sentinel and the ontology schema.

One-line Summary

An open-source hedge fund team refactored its multi-agent trading system with deterministic schemas, SQLite BM25, and a Rust sentinel.

Paragraph Summary

An open-source hedge fund experiment team refactored its multi-agent trading architecture based on community feedback, fixing three key failure modes. First, feeding bare metrics directly to the LLM induced polarity hallucinations, so the team introduced a factor ontology schema binding BULLISH/BEARISH/RISK polarity contracts to 11 quantitative factors, reducing investment committee polarity hallucinations to zero. Second, reflective memory abandoned embeddings plus a vector store in favor of SQLite FTS5 BM25, cutting round-trip latency from 200-450ms to 0.35ms, with more precise keyword retrieval. Third, real-time risk control was rewritten in pure Rust with a Struct-of-Arrays layout, disabling unsafe code, with single-position evaluation at just 3.63 nanoseconds. The final division of labor: Python handles the slow-thinking agent committee, while the Rust sentinel handles nanosecond-level portfolio monitoring, connected via persistent pipeline IPC.