Getting Started with LLM App Development: From Token Fundamentals to LangChain and LangGraph

Why can models like ChatGPT, Claude, Gemini, and DeepSeek understand and respond to you like a human? The core of this entire article can be summed up in one sentence: LLMs only predict the next token. So every useful application is scaffolding built around it—what context it can see, which tools it can call, what it can remember.

Once you understand this, you'll see why the same model performs so differently across products: the difference isn't in the model itself, but in how well the scaffolding is built.

Overview of LLM app development

What Exactly Is an LLM

A Large Language Model (LLM) is an AI system for processing and generating human-like text. Under the hood it's deep learning—specifically, the Transformer architecture introduced in the famous paper "Attention Is All You Need." The model is trained on massive amounts of text (such as the fineweb dataset), from which it learns language patterns, contextual relationships, and semantics. This makes them good at tasks like text generation, translation, and summarization.

Traditional rule-based systems execute instructions written by humans, whereas LLMs are probabilistic—they predict the most likely next word based on what they've seen during training. That's exactly why they can handle a wide variety of questions while giving contextually appropriate answers.

Five Keywords You Can't Avoid

Tokenization: Breaking text into smaller units called tokens. A token can be a word, part of a word, or a single character. The model processes tokens, not raw text.

Embeddings: Representing words or phrases as numeric vectors that capture their relationships. This is how the model knows that two different words have similar meanings.

Context window: The maximum amount of text the model can process at once. The larger the window, the more it can remember and the more coherent its responses. Beyond the window, earlier parts of the conversation fall out.

Temperature: The randomness knob. Turn it up for more creative, varied output; turn it down for more predictable, reproducible results.

Fine-tuning: Continuing to train a model on a specific dataset so it performs better in specialized domains like medicine, law, or finance.

The Transformer architecture is a neural network structure proposed by a Google team in the 2017 paper "Attention Is All You Need." Its core innovation is the Self-Attention mechanism. Unlike recurrent neural networks (RNNs), which process text sequentially, character by character, self-attention allows the model to "see" all other words in the sequence at once when processing each word, dynamically computing relevance weights between them. This makes training highly parallelizable and enables the model to capture long-range semantic dependencies. Mainstream large models such as GPT, BERT, Claude, and Gemini are all built on the Transformer, with differences mainly in layer count, attention variants (such as multi-head attention, sparse attention), and training approach (autoregressive vs. masked language modeling). Understanding the Transformer helps clarify why the context window and attention budget become hard constraints for the model.

What LLMs Are Good and Bad At

LLMs, and AI in general, are not a panacea. Recognizing their boundaries matters more than blind optimism.

Where They Truly Shine

Writing coherent, human-like text that fits the context
Quickly extracting and summarizing large amounts of information
Working across multiple languages
Generating structured output

Hallucination refers to the model generating content that sounds plausible and fluent but is factually wrong or fabricated—such as nonexistent paper citations, invented legal provisions, or incorrect names and dates. It's not a program bug, but a byproduct of the probabilistic generation mechanism: the model predicts the most likely next token based on the training distribution, and when reliable knowledge or context is lacking, it "fills in" a statistically plausible answer rather than admitting it doesn't know. Higher temperatures, noisy training data, and questions outside the training scope all exacerbate hallucination. Common mitigation approaches include retrieval-augmented generation (RAG), having the model cite sources, requiring it to express uncertainty, and adding human or rule-based verification in critical scenarios.

Where They Easily Fail

They hallucinate—and with great confidence
Short memory, limited context window
Possible bias depending on training data
Logical reasoning isn't a strength
High operating cost and energy consumption

We've covered the model's own characteristics. Now comes the key part: how do you actually use it to build things?

How to Build Apps with LLMs

A bare model can't do much. If you want to solve real problems rather than build yet another chatbot, you have to give it resources: internet access, task tools (calling APIs, querying databases), and so on. That's where LangChain comes in.

LangChain

If you're serious about building LLM apps, LangChain deserves a spot in your toolbox. It provides a structured way to converse with models and makes it much easier to connect models to external data sources, databases, and APIs. It comes with built-in components like memory, chains, and external tools (such as search APIs). Whether you're building a chatbot, a research assistant, or an AI knowledge base, it saves you a ton of plumbing work.

A few reasons to choose LangChain:

Integrations: Easily connect to different models, databases, and external APIs
Tools: Models can call search APIs, calculators, and other tools to give better answers
Chains: String multiple LLM calls into a conversation flow
Memory: Preserve conversation context across multiple turns

Keeping AI Conversations on Track

LLM attention is limited (again, the context window problem), and long conversations tend to go off the rails. Three ways to keep things under control:

Trimming: Cut out unnecessary parts of the conversation
Filtering: Keep only the most relevant messages
Summarising: Compress lengthy conversations into brief recaps

There's a lot you can build with LangChain: chatbots, smart search, writing assistants, automated research tools. It's suitable for any scenario that needs to connect to databases, external APIs, and knowledge retrieval systems.

But it has limitations. LangChain structures your AI interactions, but you're still bound by the model's own constraints, like the context window and lack of long-term memory. Using it alone also doesn't give you advanced decision-making or complex multi-step orchestration. That's the problem LangGraph solves.

Chains are the fundamental composition unit in LangChain, referring to connecting multiple components in a predetermined order to form a reusable processing pipeline. The simplest chain might just be "Prompt template → LLM → Output parser," while more complex ones can string together multiple LLM calls, external tools, and conditional logic. The value of chains lies in encapsulating scattered call logic into a declarative structure, centralizing management of prompt templates, model selection, and input/output formats for easier testing and reuse. However, chains are static, linear or lightly branched, and cannot express loops or dynamic decisions—which is exactly why LangGraph later replaced chain-based orchestration with a graph structure.

LangGraph

LangGraph is a flexible framework for building dynamic, multi-step AI applications. If a simple app is "input prompt, output answer," LangGraph lets you build workflows with multiple decision points, multiple agents communicating with each other, and structured automation. It's especially useful when your AI needs to interact back and forth with users, handle different workflows, or remember across sessions.

You can use it for query routing, breaking large tasks into smaller steps, and parallel execution. Automated customer service flows, step-by-step user guidance, AI-assisted content generation—it provides the components to orchestrate all of this.

Where to Store Conversation Data

Want your AI to "remember" things? LangGraph gives you several options:

Memory Saver: Local storage, fast access
Postgres Saver: External storage using PostgreSQL
MongoDB: Store conversations in a NoSQL database
Redis Saver: Fast, efficient AI memory storage

Prep Before Going to Production

Use a different thread ID for each session
Use persistent checkpoints to save conversation history
Avoid oversized user prompts that overwhelm the model
Stream responses for a smoother experience
Handle errors (rate limiting, content moderation, etc.)

Agent, in the LLM context, refers to a model-driven system that can autonomously decide its next action: after receiving a goal, it doesn't output an answer all at once, but loops through "think—choose tool—execute—observe results" until the task is complete. Unlike fixed chains, an agent's path is dynamically generated at runtime. LangGraph uses a graph (nodes representing processing steps, edges representing transition conditions) to explicitly describe this kind of loop, making message passing, state sharing, and interrupt recovery between multiple agents controllable and debuggable. Common patterns include ReAct (reasoning plus acting), planner-executor division of labor, and a supervisor agent dispatching multiple specialized agents.

Prompt Engineering

To get accurate, relevant answers from an LLM, prompts are crucial. Prompt engineering is half art, half science—how you frame the question directly changes the quality of the answer.

Role division in a conversation:

SystemMessage: Sets the AI's basic rules, behavior, and personality—defines the role, tone, or overall guidelines
HumanMessage: What the user sends, usually typed by a person
AIMessage: What the model returns—text, or a request to call a tool
ToolMessage: Brings the result of a tool call back to the model, usually after fetching or processing external data

Where This Article's Boundaries Lie

You might not need a framework at all. The most powerful counterargument comes from Anthropic's "Building effective agents": start by calling the API directly, because frameworks add layers, hide prompts, and make debugging harder. If your app is just one or two model calls, a plain SDK means less to learn and less to break.

This is a map with an expiration date. Framework APIs in this field change every few months, and the author's list of LLM "weaknesses" is also shrinking. Before copying patterns from this article, check the current documentation first.

This is an overview, not a build log. No code, no measured data. If you want a complete case study, check out the Finchat project.

One Last Thing to Remember

With frameworks like LangChain and LangGraph, building apps with LLMs is easier than ever. Chatbots, smart assistants, knowledge search tools—it all comes down to three things: prompt engineering, context management, and conversation workflows.

If you only remember one thing: first decide what the model needs to see, then decide what it's allowed to do, and finally decide what it should remember. Follow that order. The right framework is the one that makes these three decisions easiest to read clearly.

One-line Summary

An LLM's essence is just predicting the next token; all application value comes from the scaffolding built around it.

Paragraph Summary

The article points out that models like ChatGPT and Claude have only one underlying capability—predicting the next token—and all practical applications are scaffolding built on top of this: what context it can see, which tools it can call, what it can remember. Around this core, the article introduces five key concepts under the Transformer architecture (tokenization, embeddings, context window, temperature, fine-tuning), outlines LLMs' strengths and weaknesses (such as hallucination, short memory, weak reasoning), and lays out a path for building apps: use LangChain to connect models with external tools and manage memory and chains, use LangGraph for multi-step orchestration and multi-agent collaboration, and improve output quality through role division of System/Human/AI/Tool messages in prompt engineering. The article ends by reminding readers that frameworks aren't necessary—Anthropic recommends starting with direct API calls; this map has an expiration date, and contains no code or measured data. The core decision order is: define context first, then permissions, and finally memory.