Palimpsest Gateway
An open-source, self-hosted LLM API key pool gateway compatible with the OpenAI interface. It supports multi-vendor API key pool management, automatic failover, cross-vendor context continuation, and encrypted key storage. The project is built with Python and FastAPI.
Timeline (last 90 days)
Palimpsest Gateway is a self-hosted, OpenAI-compatible LLM key-pool gateway that provides unified key pooling and automatic failover while maintaining conversation continuity during switching.
Palimpsest Gateway is open-sourced under the MIT license, written in Python/FastAPI, and can be run via docker compose up.
The core of Palimpsest Gateway consists of the gateway itself plus Redis, where Redis handles the key registry, session history, usage counters, and audit logs.
Palimpsest Gateway supports forwarding requests to OpenAI, Anthropic, Google Gemini, or OpenRouter.
Applications make calls through a gateway key (pgw_live_...) and never touch the actual vendor keys.
To achieve streaming failover without breaking the client, the gateway fully buffers the entire vendor stream in memory before sending anything to the client, with failover completing before any bytes leave the gateway.
The failover loop categorizes exceptions: transient errors trigger a cooldown before switching to the next key, exhaustion triggers a switch to the next key, rejection suspends that key, and only fatal errors are actually thrown.
For cross-vendor context continuation, the gateway stores each conversation turn in Redis by session; during failover it reconstructs a sliding window, defaulting to the most recent 4 turns plus the system prompt and current message.
Vendor keys are encrypted at rest using a Fernet master key; the master key is generated at first startup, persisted to a Docker volume, and never leaves the container.
Rotating the master key is a destructive operation (make rotate-secrets), by intentional design.
5 more timeline events
All Facts (15)
The Palimpsest Gateway project is located at github.com/hammrouni/palimpsest-gateway, and the image can be pulled via docker pull ghcr.io/hammrouni/palimpsest-gateway:latest.
50%UnverifiedPalimpsest Gateway is open-sourced under the MIT license, written in Python/FastAPI, and can be run via docker compose up.
50%UnverifiedThe core of Palimpsest Gateway consists of the gateway itself plus Redis, where Redis handles the key registry, session history, usage counters, and audit logs.
50%UnverifiedPalimpsest Gateway supports forwarding requests to OpenAI, Anthropic, Google Gemini, or OpenRouter.
50%UnverifiedApplications make calls through a gateway key (pgw_live_...) and never touch the actual vendor keys.
50%UnverifiedTo achieve streaming failover without breaking the client, the gateway fully buffers the entire vendor stream in memory before sending anything to the client, with failover completing before any bytes leave the gateway.
50%UnverifiedThe failover loop categorizes exceptions: transient errors trigger a cooldown before switching to the next key, exhaustion triggers a switch to the next key, rejection suspends that key, and only fatal errors are actually thrown.
50%UnverifiedFor cross-vendor context continuation, the gateway stores each conversation turn in Redis by session; during failover it reconstructs a sliding window, defaulting to the most recent 4 turns plus the system prompt and current message.
50%UnverifiedVendor keys are encrypted at rest using a Fernet master key; the master key is generated at first startup, persisted to a Docker volume, and never leaves the container.
50%UnverifiedRotating the master key is a destructive operation (make rotate-secrets), by intentional design.
50%UnverifiedThe key selection algorithm first prioritizes keys with a request cap that still have remaining quota (with more remaining quota preferred), then keys with no cap, in order to use free keys first with paid keys as a fallback.
50%UnverifiedPalimpsest Gateway's API starts at http://localhost:8000 by default; including the X-Session-ID header in requests enables server-side conversation history management.
50%UnverifiedIn Palimpsest Gateway's dashboard, model pricing is entered by the user themselves, with no pre-configured presets bundled in.
50%UnverifiedPalimpsest Gateway is a self-hosted, OpenAI-compatible LLM key-pool gateway that provides unified key pooling and automatic failover while maintaining conversation continuity during switching.
50%UnverifiedPalimpsest Gateway's project roadmap includes multi-gateway clustering, vendor quota synchronization, and request transformation, with OpenRouter already supporting real quota reading.
50%