LiteLLM
LiteLLM is an open-source unified invocation library and proxy middleware for large language models. It provides a standardized API interface compatible with the OpenAI format and supports routing requests to more than 100 LLM providers, including OpenAI, Anthropic, Azure, and locally deployed models. Core features include multi-model load balancing, request format conversion, cost tracking, and access control, enabling developers to integrate and manage LLM services from different sources in a unified manner.
Core Facts
LiteLLM is an open-source unified LLM access layer project whose Proxy mode can uniformly convert API requests in different formats into OpenAI-compatible format.
Open-source projects such as LiteLLM and OneAPI also provide similar unified interface layers.
LiteLLM是一个开源的LLM API统一代理层，将数十家模型提供商的API统一封装为OpenAI兼容格式
Timeline (last 90 days)
The model_prices_and_context_window_backup.json file maintained by LiteLLM is a community-collaborated model metadata repository recording parameters such as each model's input/output pricing, maximum context window, and supported features, updated manually by maintainers based on official API documentation and actual call results.
Data related to Fable 5.5 has appeared in LiteLLM's model_prices_and_context_window_backup.json file, indicating that the model already has callable conditions at the API level, with pricing and context window parameters determined.
Open-source projects such as Portkey and LiteLLM provide multi-model routing and observability capabilities.
The litellm CLI's --host parameter defaults to 0.0.0.0, i.e. it listens on all network interfaces by default.
LiteLLM's stdio MCP server is essentially a command executed by the proxy on its host, and is disabled by default in the current version unless the process environment sets LITELLM_ENABLE_MCP_STDIO=true.
Unless EXPERIMENTAL_UI_LOGIN is explicitly set to false, the LiteLLM proxy will attempt to decrypt any incoming token not starting with sk- as a UI session token; the GHSA-7hp6 advisory states that versions 1.87.0 to 1.90.x are likewise exploitable when it is set to true.
When rotating the master key in LiteLLM, if there is no salt key, use POST /key/regenerate with new_master_key (back up the database first); if there is a salt key, do not call that endpoint, just replace the key and restart.
In the LiteLLM source code, get_salt_key() reads the LITELLM_SALT_KEY environment variable, and when it is not set falls back to the master key as the salt value, used to encrypt provider keys stored in the database.
The correct order for disabling the master key as the UI password is: first create a proxy_admin user with a separate password in the UI and verify that it can log in, then set master_key in general_settings and disable_env_credential_login: true.
LiteLLM's /ui admin panel logs in using UI_USERNAME (default admin) and UI_PASSWORD; if UI_PASSWORD is not set, LiteLLM accepts the master key itself as the password, and anyone who logs in this way is a proxy administrator.
40 more timeline events
All Facts (20)
LiteLLM is an open-source unified LLM access layer project whose Proxy mode can uniformly convert API requests in different formats into OpenAI-compatible format.
80%VerifiedOpen-source projects such as LiteLLM and OneAPI also provide similar unified interface layers.
75%VerifiedLiteLLM是一个开源的LLM API统一代理层，将数十家模型提供商的API统一封装为OpenAI兼容格式
75%VerifiedSimilar open-source LLM gateway solutions in the industry include LiteLLM, OneAPI, and PortKey.
75%VerifiedProjects such as OpenRouter and LiteLLM are also exploring multi-model orchestration solutions.
75%VerifiedRouteLLM and LiteLLM are open-source frameworks for implementing LLM intelligent routing functionality.
70%VerifiedOpenRouter is an AI model aggregation and routing platform that allows developers to access hundreds of models from multiple vendors through a unified OpenAI-compatible API interface.
70%VerifiedThe LiteLLM project is developed in Python.
65%VerifiedLiteLLM provides two usage modes: a Python SDK and a Proxy Server (AI Gateway).
65%VerifiedLiteLLM is an open-source project developed by the BerriAI team.
65%VerifiedLLM routing is an inference optimization strategy that dynamically selects models based on task complexity, cost budget, and response speed requirements; open-source frameworks such as RouteLLM and LiteLLM are already exploring this direction.
65%UnverifiedLiteLLM的日志系统支持与Langfuse、Helicone、Datadog和Prometheus等可观测性平台集成
90%UnverifiedLiteLLM的护栏系统支持集成第三方审核服务如Lakera和Presidio
90%Unverified该方案的所有数据和推理都在本地完成，不会外传到第三方服务器，适合对数据隐私有严格要求的企业场景
90%Unverified通过设置环境变量ANTHROPIC_BASE_URL指向LiteLLM Proxy的本地端口，可以将Claude Agent SDK的请求重定向到本地模型，无需修改SDK源码
90%UnverifiedLiteLLM和OneAPI等开源项目将不兼容OpenAI格式的模型（如Claude的原生API、Gemini的Vertex AI接口）统一适配为OpenAI格式
85%Unverified业界知名的合规LLM网关方案包括LiteLLM、OneAPI以及企业级的Portkey
85%UnverifiedLiteLLM的负载均衡策略支持加权轮询和最少连接数等算法
85%UnverifiedLiteLLM是Python生态中的多模型统一代理，聚焦于API兼容性转换
80%UnverifiedPortkey和LiteLLM是商业化的AI网关产品，提供统一的API代理层
80%