halogen-flash-server

An open-source inference server derived from llama.cpp, developed by the local LLM community and optimized for specific hardware platforms to improve inference performance.