LangChain LCEL Expression Language Deep Dive: Pipe Operator, RunnableLambda, and Parallel Execution in Practice

Introduction

LangChain Expression Language (LCEL) is the modern approach to building chain invocations in the LangChain framework. It replaces the traditional LLMChain method, offering more flexible and intuitive chain composition capabilities. LCEL's design philosophy draws heavily from the concept of "Function Composition" in functional programming — in mathematics and functional programming, function composition means chaining multiple functions together so that the output of the previous function becomes the input of the next, in the form of f(g(x)). Pipe operators (such as |> or .) in functional languages like Haskell and Scala are manifestations of this idea. LCEL brings this paradigm into LLM application development, making each processing step (prompt formatting, model inference, output parsing) a pure data transformation function — decoupled and independently testable. This article provides an in-depth analysis of LCEL's core concepts, including the underlying principles of the pipe operator, practical applications of RunnableLambda, RunnableParallel, and RunnablePassThrough, helping you quickly get started with LCEL and build complex LLM application chains.

From Traditional LLMChain to LCEL

Limitations of the Traditional Approach

In earlier versions of LangChain, building chain invocations required using the LLMChain class. The typical usage was to define a Prompt Template, LLM, and Output Parser separately, then pass them as parameters to LLMChain:

chain = LLMChain(prompt=prompt, llm=llm, output_parser=output_parser)
chain.invoke({"topic": "RAG"})

This approach had two main issues: first, parameters were constrained by predefined interfaces, limiting flexibility; second, this API has been officially marked as deprecated and will no longer be maintained in future versions.

LCEL's Concise Expression

In contrast, LCEL uses the pipe operator (|) to chain components together with extremely concise syntax:

chain = prompt | llm | output_parser
chain.invoke({"topic": "RAG"})

The semantics of the pipe operator are highly intuitive: pass the output of the left component as the input to the right component. This declarative style makes the data flow throughout the chain immediately clear and aligns well with functional programming thinking.

Underlying Principles of the Pipe Operator

Python Magic Methods: The Secret of Operator Overloading

Python's Magic Methods (also called Dunder Methods) are a core mechanism of Python's data model, allowing developers to overload the behavior of built-in operators for custom classes. __or__ corresponds to the bitwise OR operator | — when the Python interpreter encounters a | b, it actually executes a.__or__(b). LangChain leverages this mechanism so that all components inheriting from the Runnable base class can be naturally chained using the | symbol. Similarly, __add__ corresponds to +, __mul__ corresponds to *, and __getitem__ corresponds to []. This operator overloading mechanism makes building Domain-Specific Languages (DSLs) possible — LCEL is essentially a chain invocation DSL embedded in Python, enabling developers to describe data processing workflows in a way that approaches natural language.

Manually Implementing a Runnable Class

To understand how it works, we can implement a simplified version of the Runnable class ourselves:

class Runnable:
    def __init__(self, func):
        self.func = func
    
    def invoke(self, input):
        return self.func(input)
    
    def __or__(self, other):
        def chained(input):
            return other.invoke(self.invoke(input))
        return Runnable(chained)

The core logic lies in the __or__ method: it creates a new Runnable that first executes the current function, then passes the result to the next Runnable. This is the essence of LCEL's chain invocation — achieving step-by-step data transformation through function composition.

Chain Execution Example

Define three simple functions and link them with the pipe operator:

add_five = Runnable(lambda x: x + 5)
sub_five = Runnable(lambda x: x - 5)
mul_five = Runnable(lambda x: x * 5)

chain = add_five | sub_five | mul_five
chain.invoke(3)  # 3+5=8, 8-5=3, 3*5=15 → outputs 15

Input 3, add 5 to get 8, subtract 5 to get 3, multiply by 5 to get 15. This simple example clearly demonstrates how the pipe operator chains multiple operations into a data processing pipeline.

RunnableLambda: Turning Any Function into a Composable Component

LangChain's RunnableLambda is essentially the official version of the Runnable class we manually implemented above. It wraps any Python function into an LCEL-compatible component, enabling seamless integration into chain invocations.

Practical Application: Text Post-Processing Chain

The following example demonstrates how to append custom text processing logic after LLM generation:

from langchain_core.runnables import RunnableLambda

def replace_word(text):
    return text.replace("AI", "Skynet")

chain = prompt | llm | output_parser | RunnableLambda(replace_word)

This chain first has the LLM generate a report about AI, then uses RunnableLambda to replace all occurrences of "AI" with "Skynet". While it's a fun example, it demonstrates an important capability: any Python function can be seamlessly integrated into an LCEL chain through RunnableLambda, greatly extending the chain's processing capabilities.

Tips for Handling Multi-Parameter Functions

It's important to note that the input to each Runnable in LCEL is a single object. If your function requires multiple parameters, the recommended approach is to have the function accept a dictionary and unpack it internally:

def multi_param_func(inputs):
    text = inputs["text"]
    word = inputs["word"]
    return text.replace(word, "replaced")

This pattern is very common in real projects, especially in scenarios where you need to pass both context and user input simultaneously.

RunnableParallel and PassThrough: The Power of Parallel Execution

Use Case: Multi-Source RAG Retrieval

Retrieval-Augmented Generation (RAG) is one of the most mainstream architecture patterns in current LLM applications. Its core idea is to retrieve document fragments related to the question from an external knowledge base before the LLM generates an answer, injecting them as context into the Prompt. This compensates for the LLM's knowledge cutoff limitations and reduces hallucinations. Vector databases (such as Chroma, Pinecone, Weaviate) are critical infrastructure for RAG — they convert text into high-dimensional vectors and support semantic similarity retrieval.

In real-world RAG applications, we often need to retrieve information from multiple knowledge bases simultaneously. This is exactly where RunnableParallel and RunnablePassThrough shine.

Assume we have two vector databases:

• Vector Store A: stores release date information for Deep Seek R3
• Vector Store B: stores architecture parameter information for Deep Seek V3

Building a Parallel Retrieval Chain

from langchain_core.runnables import RunnableParallel, RunnablePassThrough

retrieval = RunnableParallel(
    context_a=retriever_a,
    context_b=retriever_b,
    question=RunnablePassThrough()
)

chain = retrieval | prompt | llm | output_parser

This code has two key components:

1. RunnableParallel: Executes two retrievers simultaneously, fetching context information from different data sources and merging the results into a single dictionary output
2. RunnablePassThrough: Passes the user's original question through to the next step unchanged, without any modification

When we ask "What architecture does the model Deep Seek released in December use?", the LLM needs to synthesize information from both data sources to provide a complete answer. It's worth noting that DeepSeek V3 uses a Mixture of Experts (MoE) architecture — unlike traditional dense models (which activate all parameters during each inference), MoE models contain multiple "expert" sub-networks and a Router. During each inference, the router activates only a few experts to process the current input. This allows the model to have an extremely large total parameter count (671B) while requiring far less actual computation than a dense model of equivalent size, significantly reducing inference costs while maintaining high performance.

Final output: "Deep Seek V3 model released in December 2024 is a Mixture of Experts model with 671 billion parameters."