Unverified50% confidenceFactExact time
In llama.cpp, --n-gpu-layers N means putting the N layers of the model closest to the input side on the GPU for computation, with the remaining layers falling back to the CPU.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
Unverifiedllama.cpp supports hybrid CPU+GPU inference, controls the number of layers offloaded to the GPU via the --n-gpu-layers parameter, and includes a built-in HTTP server compatible with the OpenAI API format.76% similarVerifiedllama.cpp supports hybrid CPU and GPU inference (layer offloading), allowing partial model layers to be loaded into GPU VRAM while the remaining layers are kept in memory75% similarUnverifiedIn llama.cpp, the -ngl (number of GPU layers) parameter controls how many layers are placed on the GPU, used to balance speed and model size.72% similarVerifiedCPU+GPU hybrid inference uses a layer-level offloading mechanism, controlling the number of offloaded layers via the n_gpu_layers parameter; the main bottleneck is PCIe bandwidth latency, which typically reduces generation speed to 1/3 to 1/2 of pure GPU inference.71% similarUnverifiedllama.cpp pursues low-level control, with everything from quantization types to the number of GPU layers adjustable by hand, making it the first choice for embedded and edge deployment.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1028883API
curl https://kongchang.com/api/v1/knowledge/claims/1028883MCP
get_claim(id=1028883)