Strata
An open-source semantic middleware layer for large language models that inserts a controllable semantic layer between data and models. It intercepts non-compliant, unsafe, or unauthorized LLM requests through rule-based constraints, and supports auditable allow/deny policy expressions to govern model interactions.
Core Facts
该项目已在GitHub开源
Strata uses low-bit quantization; the minimum configuration requires approximately 38 GB of combined memory and VRAM, the highest-quality configuration requires approximately 55 GB, and 64 GB of system memory is the ideal target.
In an RTX 5070 (12GB VRAM) configuration, the local Qwen3 9B model's output speed reached approximately 70–76 tokens/second.
Timeline (last 90 days)
A user on Reddit shared their change of attitude toward Qwen 3.8 Flash Next, going from观望 to being genuinely impressed by its actual performance after configuring the Strata inference framework.
Under the Strata framework, Flash Next runs with IQ4 xs quantization, and its performance left the user convinced, in their view approaching the level of the models they used professionally last year.
Strata appears to be an inference framework or toolchain that can simplify the model deployment process and lower the barrier to trying it out.
The community has custom inference engine projects such as gufo, halogen, strixite, and Strata.
The user pointed out that the new version adds unnecessary dependency requirements for service deployment.
Currently, the information about the Strata upgrade issues comes mainly from a single Reddit user, lacking official confirmation or cross-verification from more independent sources.
A user on Reddit posted a warning that after upgrading Strata they encountered a large number of breaking changes.
The user believes that Strata's Web UI seems to be shifting substantially toward a desktop application direction.
The problems reported by this user involve both the Windows and Linux platforms.
Berth supports native switching among multiple engines such as Ninfer-4090 (llama-swap), Strata, ComfyUI, and OpenWebUI.
40 more timeline events
All Facts (20)
该项目已在GitHub开源
80%VerifiedStrata uses low-bit quantization; the minimum configuration requires approximately 38 GB of combined memory and VRAM, the highest-quality configuration requires approximately 55 GB, and 64 GB of system memory is the ideal target.
70%VerifiedIn an RTX 5070 (12GB VRAM) configuration, the local Qwen3 9B model's output speed reached approximately 70–76 tokens/second.
70%VerifiedStrata is an open-source project that serves as both an inference engine and a one-click packaged installer, providing a pre-selected quantized version by default along with an OpenAI-compatible local API.
65%VerifiedDeveloper Primitive reported a generation speed of 142.6 tokens/s for Qwen3 Flash Next running locally.
65%Partially VerifiedStrata's core approach is to have the CPU compute directly in memory in-place when an expert is not on the GPU, while the GPU processes cached experts in parallel, making memory a second compute domain rather than slow virtual VRAM.
65%UnverifiedOfficial estimates suggest a 24GB GPU can run approximately 100–140 tokens/second.
60%UnverifiedIf the quantized model weights are 150 GB, reading the weights once at a bandwidth of 273 GB/s takes approximately 0.55 seconds, giving a theoretical single-token generation speed of approximately 1.8 tokens/s (excluding computation latency and cross-node communication overhead).
60%UnverifiedStrata appears to be an inference framework or toolchain that can simplify the model deployment process and lower the barrier to trying it out.
50%UnverifiedUnder the Strata framework, Flash Next runs with IQ4 xs quantization, and its performance left the user convinced, in their view approaching the level of the models they used professionally last year.
50%UnverifiedThe community has custom inference engine projects such as gufo, halogen, strixite, and Strata.
50%UnverifiedThe problems reported by this user involve both the Windows and Linux platforms.
50%UnverifiedCurrently, the information about the Strata upgrade issues comes mainly from a single Reddit user, lacking official confirmation or cross-verification from more independent sources.
50%UnverifiedThe user pointed out that the new version adds unnecessary dependency requirements for service deployment.
50%UnverifiedThe user believes that Strata's Web UI seems to be shifting substantially toward a desktop application direction.
50%UnverifiedBerth supports native switching among multiple engines such as Ninfer-4090 (llama-swap), Strata, ComfyUI, and OpenWebUI.
50%UnverifiedBerth provides API service capability and can serve as an API endpoint to load Strata, Polystrata, and any custom engine.
50%UnverifiedWhen deploying the Strata model, at least 100GB of disk space must be reserved, otherwise the model weights may fail to download.
50%UnverifiedWhen enabling LAN access, be sure to also enable the API Key, otherwise anyone on the same network can access the service.
50%UnverifiedStrata's source code is hosted on GitHub, which some domestic network environments cannot access directly.
50%