GreenRoute
A local-first LLM routing service. It is a lightweight Python service based on FastAPI that determines whether a request should be handled by a local model or in the cloud, and estimates the resulting resource savings.
Timeline (last 90 days)
GreenRoute is a lightweight Python service based on FastAPI, with a Web chat interface on the front end, used to determine whether an AI request should be processed locally, explain the basis for the decision, and estimate resource savings.
GreenRoute performs four steps for each prompt: classify the request, check device status, route to Llama 3.2 1B running locally on Ollama or to Google Gemini in the cloud, and finally estimate the resource footprint relative to an all-cloud baseline.
GreenRoute uses rules rather than a trained classifier, and decisions must be explainable in one sentence and completed in well under 1 millisecond.
The second step of GreenRoute's routing consists of seven checks (task type, device mode, battery, input length, confidence, complexity, memory), and the first failed check sends the request to the cloud and becomes the user-visible reason.
GreenRoute's complexity ceiling depends on the local model size: about 0.35 for a ~1B model, 0.5 for ~3B, and 0.6 for ~7B.
When estimating the additional power consumption of local inference, GreenRoute assumes a laptop CPU of 25W by default, and carbon emissions are based on the 2025 world average grid intensity of 458 gCO₂/kWh (from Ember, published via Our World in Data).
In a test of 60 everyday prompts, GreenRoute kept 35 (58%) on the device, agreeing with manual annotation on 57/60, with all three disagreements leaning toward safely sending the request to the cloud.
On the 35 prompts that the router kept local, Llama 3.2 1B's answers were judged acceptable 30 times (86%, average 4.4/5); on the 23 text prompts sent to the cloud, the same model was acceptable only 4 times (17%, average 2.0/5).
Compared with sending all 60 prompts to the cloud, GreenRoute's estimated footprint is 48% lower in energy (7.5 vs 14.4 Wh), 58% lower in water (6.5 vs 15.6 mL), and 19% lower in CO₂e (1.46 vs 1.80 g).
Local routing stops saving energy at an additional generation power of about 136W; a laptop CPU (about 25W) or Apple Silicon Mac (about 10W) is far below this, while a gaming desktop GPU pushed to 150W would use about 6% more energy than the all-cloud baseline.
2 more timeline events
All Facts (12)
GreenRoute is a lightweight Python service based on FastAPI, with a Web chat interface on the front end, used to determine whether an AI request should be processed locally, explain the basis for the decision, and estimate resource savings.
50%UnverifiedGreenRoute performs four steps for each prompt: classify the request, check device status, route to Llama 3.2 1B running locally on Ollama or to Google Gemini in the cloud, and finally estimate the resource footprint relative to an all-cloud baseline.
50%UnverifiedGreenRoute uses rules rather than a trained classifier, and decisions must be explainable in one sentence and completed in well under 1 millisecond.
50%UnverifiedThe second step of GreenRoute's routing consists of seven checks (task type, device mode, battery, input length, confidence, complexity, memory), and the first failed check sends the request to the cloud and becomes the user-visible reason.
50%UnverifiedGreenRoute's complexity ceiling depends on the local model size: about 0.35 for a ~1B model, 0.5 for ~3B, and 0.6 for ~7B.
50%UnverifiedWhen estimating the additional power consumption of local inference, GreenRoute assumes a laptop CPU of 25W by default, and carbon emissions are based on the 2025 world average grid intensity of 458 gCO₂/kWh (from Ember, published via Our World in Data).
50%UnverifiedIn a test of 60 everyday prompts, GreenRoute kept 35 (58%) on the device, agreeing with manual annotation on 57/60, with all three disagreements leaning toward safely sending the request to the cloud.
50%UnverifiedOn the 35 prompts that the router kept local, Llama 3.2 1B's answers were judged acceptable 30 times (86%, average 4.4/5); on the 23 text prompts sent to the cloud, the same model was acceptable only 4 times (17%, average 2.0/5).
50%UnverifiedCompared with sending all 60 prompts to the cloud, GreenRoute's estimated footprint is 48% lower in energy (7.5 vs 14.4 Wh), 58% lower in water (6.5 vs 15.6 mL), and 19% lower in CO₂e (1.46 vs 1.80 g).
50%UnverifiedLocal routing stops saving energy at an additional generation power of about 136W; a laptop CPU (about 25W) or Apple Silicon Mac (about 10W) is far below this, while a gaming desktop GPU pushed to 150W would use about 6% more energy than the all-cloud baseline.
50%UnverifiedKeyword semantic counting should count only in the instructions and not in pasted material, which is GreenRoute's first build lesson.
50%UnverifiedWhen multiple rules trigger at the same time, the most complex task wins, biasing errors toward the cloud (the expensive direction) rather than toward wrong answers.
50%