Strata
An open-source semantic middleware layer for large language models that inserts a controllable semantic layer between data and models. It intercepts non-compliant, unsafe, or unauthorized LLM requests through rule-based constraints, and supports auditable allow/deny policy expressions to govern model interactions.
Core Facts
该项目已在GitHub开源
Strata uses low-bit quantization; the minimum configuration requires approximately 38 GB of combined memory and VRAM, the highest-quality configuration requires approximately 55 GB, and 64 GB of system memory is the ideal target.
In an RTX 5070 (12GB VRAM) configuration, the local Qwen3 9B model's output speed reached approximately 70–76 tokens/second.
Timeline (last 90 days)
Unsloth v0.1.907-beta officially claims inference speed improvements of up to about 5x, mainly from a cache hit rate of about 95% and Strata-style fixes for MoE architectures.
The locally running version uses 2-bit quantization, while the cloud may be running a larger model version, so this comparison should be viewed more as a speed test than a quality comparison
Speculative decoding technology makes the speed approximately 1.6 to 1.8 times faster, and in actual tests the large model accepted the small assistant's guesses approximately 79% of the time
According to the README, the 12 GB RTX 5070 achieves approximately 94 tokens/second, while the 32 GB RTX 5090 can exceed 250 tokens/second
In a Python programming problem test, Strata completed it in approximately 4 seconds while the cloud took nearly 19 seconds; in a creative writing test, Strata took approximately 3 seconds and the cloud approximately 10 seconds
Strata is a free open-source project that enables AI models with 125 billion parameters to run on ordinary desktop computers
Strata uses a 'doorbell' mechanism: the GPU identifies the 10 experts needed for each layer and writes them to shared memory, the CPU detects the signal and allocates tasks, and the GPU and CPU process the same layer simultaneously
In almost every test, the local machine was 3 to 5 times faster than the cloud
According to the paper, VRAM is more important than raw GPU compute power, and each additional 1 GB of VRAM can accommodate approximately 700 more experts
When the developer ran the project's UPDATE script, the Git operation failed with the error 'no common ancestor'.
40 more timeline events
All Facts (20)
该项目已在GitHub开源
80%VerifiedStrata uses low-bit quantization; the minimum configuration requires approximately 38 GB of combined memory and VRAM, the highest-quality configuration requires approximately 55 GB, and 64 GB of system memory is the ideal target.
70%VerifiedIn an RTX 5070 (12GB VRAM) configuration, the local Qwen3 9B model's output speed reached approximately 70–76 tokens/second.
70%VerifiedIn decode speed tests, speeds of approximately 45 tokens/second are achievable within 20,000 tokens, dropping sharply to approximately 20 tokens/second beyond 30,000 tokens.
70%VerifiedDeveloper Primitive reported a generation speed of 142.6 tokens/s for Qwen3 Flash Next running locally.
65%Partially VerifiedStrata's core approach is to have the CPU compute directly in memory in-place when an expert is not on the GPU, while the GPU processes cached experts in parallel, making memory a second compute domain rather than slow virtual VRAM.
65%UnverifiedOfficial estimates suggest a 24GB GPU can run approximately 100–140 tokens/second.
60%UnverifiedUnsloth v0.1.907-beta officially claims inference speed improvements of up to about 5x, mainly from a cache hit rate of about 95% and Strata-style fixes for MoE architectures.
50%UnverifiedStrata uses a 'doorbell' mechanism: the GPU identifies the 10 experts needed for each layer and writes them to shared memory, the CPU detects the signal and allocates tasks, and the GPU and CPU process the same layer simultaneously
50%UnverifiedAccording to the README, the 12 GB RTX 5070 achieves approximately 94 tokens/second, while the 32 GB RTX 5090 can exceed 250 tokens/second
50%UnverifiedIn a Python programming problem test, Strata completed it in approximately 4 seconds while the cloud took nearly 19 seconds; in a creative writing test, Strata took approximately 3 seconds and the cloud approximately 10 seconds
50%UnverifiedSpeculative decoding technology makes the speed approximately 1.6 to 1.8 times faster, and in actual tests the large model accepted the small assistant's guesses approximately 79% of the time
50%UnverifiedAccording to the paper, VRAM is more important than raw GPU compute power, and each additional 1 GB of VRAM can accommodate approximately 700 more experts
50%UnverifiedThe locally running version uses 2-bit quantization, while the cloud may be running a larger model version, so this comparison should be viewed more as a speed test than a quality comparison
50%UnverifiedIn almost every test, the local machine was 3 to 5 times faster than the cloud
50%UnverifiedStrata is a free open-source project that enables AI models with 125 billion parameters to run on ordinary desktop computers
50%UnverifiedThis incident currently stems primarily from a single whistleblower's observations, lacks a response from the project maintainers and independent corroboration, and the specific motivation has not been confirmed.
50%UnverifiedThe Strata engine is partially written in C++ and invoked via Python scripts for configuration.
50%UnverifiedThe default service port for Strata is local port 8080, which can be modified in the startup script.
50%UnverifiedConcurrency capability and extreme speed are limitations of Strata; opening multiple concurrent conversations will be very laggy, making it suitable for single-user, single-conversation scenarios.
50%