LGOS: A Self-Hosted Deployment Solution That Disguises LangGraph Workflows as OpenAI Models

The Deployment Dilemma of Self-Hosting LangGraph

For developers who heavily use LangChain and LangGraph, a persistent pain point isn't building workflows themselves, but rather how to deploy these workflows. Recently, a senior software engineer shared an open-source project on Reddit that he's been developing for nearly a year and a half—langgraph-openai-serve (LGOS for short)—attempting to fundamentally solve this challenge.

The author candidly admits that he's been using LangChain and LangGraph since their early versions, and has also tried frameworks like OpenAI Agents and Haystack, but ultimately returned to LangGraph. The reason lies in LangGraph's granular control over workflows—whether simple graphs or extremely complex ones—which strikes just the right balance.

It's worth understanding that LangGraph is a framework launched by the LangChain team for building stateful, multi-step AI agent workflows. Built on the abstraction of directed graphs, it allows developers to orchestrate LLM calls, tool usage, conditional branching, and loop logic into complex workflows. Unlike traditional chain-based invocations, LangGraph supports loops and conditional jumps, making it particularly suitable for building applications that require multi-turn reasoning, self-correction, or multi-agent collaboration. LangChain is the broader LLM application development framework, providing foundational capabilities like model abstraction, prompt templates, and document loading, while LangGraph can be viewed as its advanced extension at the workflow orchestration layer.

However, deployment has always been troublesome. As a self-hosting enthusiast, he wanted the entire tech stack to be open-source and easy to run on his own infrastructure. But reality is: LangServe has been deprecated and archived, with official recommendations shifting toward LangGraph Platform; while there's Aegra as a fully self-hostable LangGraph Platform API implementation, for personal projects the author wanted something simpler—a mature API contract already supported by numerous clients.

LGOS's Core Idea: Disguise LangGraph Graphs as OpenAI Models

LGOS's design philosophy is quite clever: it allows you to register LangGraph graphs as OpenAI "model values" and serve them through a documented, OpenAI-compatible interface subset. Currently, it supports two key endpoints:

• /v1/responses
• /v1/chat/completions

It's important to understand the ecosystem significance behind these two endpoints. OpenAI's Chat Completions API (/v1/chat/completions) has become the de facto standard interface in the LLM space, with almost all major LLM providers (such as Anthropic, Google, Mistral) offering adaptation layers compatible with this interface. The Responses API (/v1/responses) is a next-generation interface introduced by OpenAI in 2025, supporting richer tool invocation and multimodal interaction. Around this API contract, a vast client ecosystem has formed—from chat frontends like Open WebUI, to multi-model routing gateways like LiteLLM, to various monitoring and evaluation tools. Choosing compatibility with this interface means zero-cost access to the entire ecosystem, without having to write adaptation code for each new protocol.

The biggest advantage of this design is zero-learning-curve ecosystem compatibility. Without needing to learn any LGOS-specific APIs, you can directly use the standard OpenAI SDK to connect your graphs to frontend clients like Open WebUI and Chainlit. Furthermore, you can place these graphs behind OpenAI-compatible gateways like Bifrost or LiteLLM for unified management.

In other words, the LangGraph agent you've painstakingly built appears to the outside world as a regular "OpenAI model," and any tool supporting the OpenAI API can plug and play. This "protocol adaptation" strategy essentially converges compatibility costs once at the service layer.

Horizontal Scaling Enabled by Stateless Design

A notable architectural decision is: From LGOS's perspective, regular conversations are stateless. LGOS doesn't store user chat history; transcripts are owned by the UI or client and resent with history when needed.

Stateless architecture is a core principle in microservice design. In a stateless service, each request contains all the information needed to process it, with the server not relying on any context left by previous requests. This means any service instance can handle any request, and load balancers can freely distribute requests to any node. In contrast, stateful services require state synchronization between instances or routing the same user's requests to the same instance (session affinity), which significantly increases operational complexity. In containerized and Kubernetes environments, stateless services can respond to traffic growth by simply increasing Pod replicas, without considering state migration issues.

The direct benefit of this choice is simpler horizontal scaling—stateless services are naturally easy to scale out. For scenarios that truly require state, LGOS still provides support: such as persistent human-in-the-loop interrupts, LangGraph checkpoints, and application data stored through LangGraph Store. This "stateless by default, stateful on demand" layered design is a pragmatic engineering tradeoff.

LGOS Feature Overview: From Streaming Responses to Cross-Process Coordination

The author lists several features he's particularly proud of, with quite comprehensive coverage:

• Native streaming and non-streaming responses
• Client-executed function tools and graph-hosted tools
• Human-in-the-loop (HITL) based on LangGraph interrupts, exposed as function calls in the Responses API
• Citations and state updates authored by graph creators
• LangGraph subgraphs support
• Typed, discoverable runtime settings
• Custom graph input, runtime context, and output adapters
• File input via OpenAI Files API IDs
• PostgreSQL checkpoints, Store support, and cross-worker interrupt coordination
• Optional Langfuse tracing and OpenTelemetry support

Among these features, several concepts are worth deeper understanding.

About Human-in-the-Loop (HITL) Mechanism: Human-in-the-Loop is an important design pattern in AI agent systems, referring to introducing human review or decision-making at critical nodes in automated workflows. Typical scenarios include: requesting human confirmation before agents execute high-risk operations (like sending emails, modifying databases), or requesting additional information from humans when agents encounter uncertainty. LangGraph natively supports HITL through its "Interrupt" mechanism: workflows pause at specified nodes, transfer control to humans, and resume execution after receiving human input. LGOS cleverly maps this mechanism to function calls in the OpenAI Responses API—clients receive a "function call" request, which is actually requesting human intervention, and the human's reply is sent back as a "function return value," driving the workflow to continue.

About Checkpoint and Persistence Mechanism: LangGraph's checkpoint mechanism is the key infrastructure for implementing reliable stateful workflows. Whenever a node in the workflow completes execution, the current complete graph state—including all node outputs, message history, custom state variables, etc.—is serialized and persistently stored. This enables workflows to resume precisely after interruption at any node, whether the interruption is due to human-in-the-loop waiting, system failure, or active pause. PostgreSQL as a checkpoint storage backend provides transactional consistency and durability guarantees. LangGraph Store is a more general key-value storage layer that allows graphs to read and write application-level data during execution, which can persist across multiple runs.

About Observability Toolchain: Langfuse is an open-source observability platform focused on LLM applications, providing tracing, evaluation, prompt management, and cost monitoring capabilities. It allows developers to record input/output, latency, token consumption, and cost for each LLM call, presenting the entire workflow's execution chain in a visualized manner. OpenTelemetry is a general observability standard under the Cloud Native Computing Foundation (CNCF), covering the three pillars of distributed tracing, metrics, and logs, widely integrated in various infrastructure and application frameworks. LGOS's support for both solutions means developers can use Langfuse optimized for LLMs for in-depth analysis, or integrate tracing data into enterprise existing OpenTelemetry-compatible monitoring systems.

Among these, mapping LangGraph's interrupt mechanism to Responses API function calls is an ingenious design—it allows complex human-in-the-loop processes to be naturally expressed within the standard protocol framework. Cross-worker interrupt coordination indicates the author has given serious engineering consideration to multi-instance deployment scenarios.

Ready-to-Use Docker Compose Demo Stack

To help newcomers understand how these components work together, the author built a self-contained demo stack with quite rich content:

• 14 documented example graphs
• Two frontends: Chainlit and Open WebUI
• PostgreSQL database
• S3-based Files API
• Optional Bifrost or LiteLLM routing

Users only need to configure the .env file and can launch the entire stack with a single Docker Compose command. This "configured environment, one-click run" experience is crucial for lowering the entry barrier for open-source projects, and reflects the author's pragmatism as a self-hoster.

Transparent Statement About AI-Assisted Coding

Worth mentioning is that the author made a candid transparency statement in the post: yes, he used coding agents as tools during development.

However, he emphasizes that as a senior software engineer, he reviews every output from the agent, rewrites any parts he disagrees with, and takes full responsibility for architecture, code quality, and releases. He explicitly states: "This is not an unreviewed vibe-coded project."

In the current context where AI-assisted programming is increasingly prevalent but "AI-generated code quality" is controversial, such a statement is both responsible to the community and reflects an engineering attitude worth emulating—tools are accelerators, not a transfer of responsibility. So-called "vibe coding" is a popular term in the developer community recently, referring to developers letting AI generate code based solely on intuition or simple natural language descriptions, without rigorous review, testing, and architectural control. While this approach may be effective in rapid prototyping, it often introduces quality issues and technical debt that are difficult to detect in production-level projects.

LGOS Version Evolution and Open Source Commitment

The author explains why he chose to make the project public now: initially LGOS was built only for himself, until the recent v0.16.0 version added Responses API support, when he felt the project was mature enough to accept broader feedback.

LGOS uses the MIT license, and the author commits that it will always remain open-source and free. He also extends an invitation at the end of the post: hoping the community will share how they currently deploy LangGraph applications, and what they want to build with LGOS.

Summary: A Pragmatic Deployment Alternative After LangServe Deprecation

With LangServe deprecated and LangGraph Platform moving toward platformization, LGOS provides a middle path between "fully managed platform" and "building from scratch": servicing your LangGraph graphs with a mature, universal API contract.

For developers who want to retain LangGraph's control granularity while reusing the vast toolchain of the OpenAI ecosystem, LGOS is an option worth attention. Its value lies not only in technical implementation, but more in its choice of a "minimize ecosystem friction" deployment philosophy.

Key Takeaways

• LGOS allows LangGraph workflows to be registered as OpenAI models, compatible with OpenAI's Chat Completions and Responses APIs
• Stateless architecture enables simple horizontal scaling while supporting persistent state for HITL, checkpoints, and application data when needed
• Native integration with existing OpenAI ecosystem tools like Open WebUI, Chainlit, LiteLLM, and Bifrost
• Provides a ready-to-use Docker Compose stack with 14 example graphs, multiple frontends, and complete infrastructure
• Open-source under MIT license as a pragmatic self-hosting alternative after LangServe deprecation