Berth
A system-level GPU memory scheduler that automatically switches among multiple AI inference engines and manages memory allocation and release, with support for API serving and protection of gaming scenarios.
Timeline (last 90 days)
Berth is a system-level VRAM scheduler that monitors GPU status and automatically unloads the current engine and loads the target engine when needed, without manual intervention.
Berth supports native switching among multiple engines such as Ninfer-4090 (llama-swap), Strata, ComfyUI, and OpenWebUI.
Berth provides API service capability and can serve as an API endpoint to load Strata, Polystrata, and any custom engine.
Berth maintains a complete operation log, allowing you to see when, which engine, and what operation was performed.
One of Berth's design goals is to prevent background tasks (such as cron jobs or API requests) from seizing the GPU while the user is gaming or running custom workflows.
Each engine running under Berth's scheduling can exclusively use all of the VRAM.
The code for the Berth project is hosted on GitHub, in the repository T-Crypt/berth.
Berth's round-robin scheduling approach cannot handle scenarios that require multiple engines to be online simultaneously responding to requests.
As a system-level scheduling layer, Berth must replace the loading logic built into each engine and take unified control of VRAM allocation and release.
All Facts (9)
Berth is a system-level VRAM scheduler that monitors GPU status and automatically unloads the current engine and loads the target engine when needed, without manual intervention.
50%UnverifiedBerth supports native switching among multiple engines such as Ninfer-4090 (llama-swap), Strata, ComfyUI, and OpenWebUI.
50%UnverifiedBerth provides API service capability and can serve as an API endpoint to load Strata, Polystrata, and any custom engine.
50%UnverifiedBerth maintains a complete operation log, allowing you to see when, which engine, and what operation was performed.
50%UnverifiedOne of Berth's design goals is to prevent background tasks (such as cron jobs or API requests) from seizing the GPU while the user is gaming or running custom workflows.
50%UnverifiedEach engine running under Berth's scheduling can exclusively use all of the VRAM.
50%UnverifiedThe code for the Berth project is hosted on GitHub, in the repository T-Crypt/berth.
50%UnverifiedBerth's round-robin scheduling approach cannot handle scenarios that require multiple engines to be online simultaneously responding to requests.
50%UnverifiedAs a system-level scheduling layer, Berth must replace the loading logic built into each engine and take unified control of VRAM allocation and release.
50%