llm-free-cascade
An open-source Node.js tool that integrates the free-tier APIs of multiple LLM providers in a cascading manner, enabling automatic switching and key rotation, and transparently presenting a unified interface to callers above it.
Timeline (last 90 days)
llm-free-cascade is an open-source Node tool that automatically switches among multiple free LLM APIs in a cascade manner, making multiple free tiers work like a single service.
llm-free-cascade's local server by default only listens on 127.0.0.1, because it needs to hold all API keys in memory.
llm-free-cascade can integrate with Ollama, so that when all free tiers are busy at the same time, the local Ollama answers questions as the last backup.
llm-free-cascade uses the MIT license, has zero dependencies, and runs on Node 18 and above.
When a timeout or 5xx error is encountered, after 2 consecutive occurrences the provider should be skipped for one minute, to avoid every request waiting the full timeout duration due to provider downtime.
In the free tier scenario, sending the same request to two providers simultaneously to take the faster response consumes double the quota, so this tool abandoned that strategy.
Many LLM providers do not allow bypassing limits through multiple free accounts, and using multiple free accounts may violate the provider's terms.
All Facts (7)
llm-free-cascade is an open-source Node tool that automatically switches among multiple free LLM APIs in a cascade manner, making multiple free tiers work like a single service.
50%Unverifiedllm-free-cascade's local server by default only listens on 127.0.0.1, because it needs to hold all API keys in memory.
50%Unverifiedllm-free-cascade can integrate with Ollama, so that when all free tiers are busy at the same time, the local Ollama answers questions as the last backup.
50%Unverifiedllm-free-cascade uses the MIT license, has zero dependencies, and runs on Node 18 and above.
50%UnverifiedWhen a timeout or 5xx error is encountered, after 2 consecutive occurrences the provider should be skipped for one minute, to avoid every request waiting the full timeout duration due to provider downtime.
50%UnverifiedIn the free tier scenario, sending the same request to two providers simultaneously to take the faster response consumes double the quota, so this tool abandoned that strategy.
50%UnverifiedMany LLM providers do not allow bypassing limits through multiple free accounts, and using multiple free accounts may violate the provider's terms.
50%