Write Functions First, Then Use Frameworks: Best Practices for AI Agent Development

Most AI Agent tutorials start by telling you to bring in a framework, but if you assemble your agent from plain functions from the start, you'll have clearer control over the system. Based on the views of LlamaIndex co-founder Jerry Liu on the Chain of Thought podcast, this article explores when to hand-write functions and when to bring in a framework, and offers actionable criteria for deciding.

Why write plain functions first?

Functions give you a clear map of the workflow: which steps run in what order, and how data passes to the next step. Branching logic stays explicit, and you can log inputs and outputs at boundaries you control. When something fails, the error surfaces directly in the code you wrote and can read, rather than hiding behind a framework's abstraction layer.

Structured output is the key hinge here. Models output free text by default, which is friendly to humans but fragile for programs. When the next step is code, you should require the model to output JSON that conforms to a schema—for example, extracting fields, classifying a ticket, or choosing a tool name and parameters. When you need the same fields every time, enforcing a schema is more reliable than loose JSON mode. Of course, you still need to validate field values and handle responses where the model refuses or gets truncated.

Below is illustrative code showing a planning step with a schema plus deterministic tool execution:

from typing import Literal, Optional
from pydantic import BaseModel

class Plan(BaseModel):
    action: Literal["search_kb", "reply", "escalate"]
    query: Optional[str] = None
    reply_text: Optional[str] = None

def plan_next_step(user_message: str, kb_snippet: str) -> Plan:
    # Call the model API here, passing schema=Plan
    ...

def search_kb(query: str) -> str:
    return "..."

def run_turn(user_message: str) -> str:
    snippet = search_kb(user_message[:200])
    plan = plan_next_step(user_message, snippet)
    if plan.action == "search_kb" and plan.query:
        return search_kb(plan.query)
    if plan.action == "reply" and plan.reply_text:
        return plan.reply_text
    return "escalated"

This loop doesn't need a framework. As requirements grow, you can add retrieval, additional model calls, and validation logic as more functions, directly assembling a compound AI system.

Structured Output refers to the model directly generating a response that conforms to a predefined schema (such as a JSON Schema or Pydantic model) rather than free text. Common implementations include providing a function calling interface, constrained decoding, or post-processing parsing plus validation. Constrained decoding masks out tokens that violate the grammar during the sampling stage, giving the highest reliability; function calling depends on the provider's implementation, and behavior may differ across platforms. Either way, you need to handle cases where the model refuses, output is truncated, or fields are missing or of the wrong type. Production environments usually add a layer of schema validation and retry logic.

What signals indicate it's time to adopt a framework?

The value of a framework lies in using it to solve problems you have already encountered. Before you hit those problems, dependencies only hide the parts you still need to test—especially the data you feed the model. Four signals are worth watching:

1. Debuggability under failure

When the agent fails, can you inspect tool calls and application state without digging into the framework's internals? If failures happen frequently and your logs and state tracking are becoming harder to maintain, then run tracing and persistent step records may be worth the dependency.

2. Recovery after interruption

Can you resume a long task without redoing side effects (such as sending emails, charging payments, or writing database records)? If users or workflows need durable checkpoints, you're really looking for a framework with state management and replay semantics.

3. Branching and coordination complexity

When control flow is no longer a straight line but involves parallel branches, human approvals, or handoffs between sub-agents, orchestration primitives can replace the ad-hoc state machine you're tired of maintaining.

4. Efficiency of retrieval iteration

Can you change how context is fetched and evaluate answers without fighting the framework's abstractions? If experimentation is slow, either simplify the stack or choose a tool that treats retrieval as a pluggable component.

LlamaIndex co-founder Jerry Liu noted on episode 61 of the Chain of Thought podcast that when the shape of the application is still open-ended, the framework layer still has value:

"When a domain is still being explored and the final application has a lot of flexibility, that's when the framework layer adds value, because the framework provides core abstractions that make it easier to build certain things."

He also connected this shift to context work: "Because the abstractions are solidifying, but the need for high-quality production context is still evolving."

Checkpoint and Replay semantics are core capabilities of durable workflow engines: save the complete run state (including completed steps, intermediate results, and the pending queue) after key steps, and recover from the most recent checkpoint after a failure rather than rerunning from scratch. During replay, you must ensure that already-executed side effects are not triggered again, which typically requires tool calls to be idempotent or requires patterns such as a transactional outbox to bind side effects to state changes. When implementing by hand, the maintenance cost of task IDs, idempotency keys, and step tables rises quickly as branches multiply—which is exactly what a framework can replace.

What is each framework feature replacing?

Only by mapping framework features to your actual pain points can you judge whether they're worth adopting.

• Interruption and recovery: A hand-written approach needs custom task IDs, idempotent tools, and a self-maintained step table; a framework provides centralized run state, checkpoints, and replay.
• Complex branching: A hand-written approach means nested conditions and hand-written state machines; a framework provides graph or workflow primitives.
• Tool pipeline: A hand-written approach needs per-provider wrappers and argument parsing; a framework provides a registry and calling conventions.
• Observability gaps: A hand-written approach means scattered logs; a framework provides tracing tied to tool calls.

If the hand-written column is still cheap for your team, the framework column may not pay off. Simple workflows need almost no orchestration, while complex workflows may benefit from a framework. Either way, you can't escape the responsibility of testing the model's input context.

Compound AI Systems, a term proposed by teams such as Databricks, refer to systems that don't rely on a single model to complete everything end to end, but instead combine multiple model calls, retrievers, tools, validators, and glue code. Their quality is determined jointly by each component and how they connect, not just by the underlying model's capabilities. This perspective means: when evaluating, you should design test sets for the entire pipeline, and when locating bottlenecks you need to distinguish whether the problem is insufficient retrieval recall, model reasoning errors, or missing validation logic. Treating the agent as a compound system also explains why hand-written function assembly has an advantage early on—each component's inputs and outputs are explicitly visible, making it easy to test and replace individually.

How to compare the two approaches fairly?

A pragmatic approach is to compare both construction methods on the same workflow. Use four check questions to judge: inspectability of tool calls and state under failure, whether recovery duplicates side effects, whether branching becomes simpler, and whether retrieval changes are easy to evaluate.

Specifically: build a minimal version with functions plus structured output, then build the same version with the framework you're considering. Same tools, same retrieval, same evaluation set, then run the four questions above against both paths. Only adopt the dependency when it not only saves implementation time but also improves inspectability, recovery behavior, or coordination.

Don't adopt a framework just because an agent loop is improved at the abstraction level. The time to bring it in is when your existing codebase is already trying to become that framework.

Action checklist

• Before adding orchestration dependencies, start with functions, explicit control flow, and model outputs bound to a schema.
• Treat the agent as a compound pipeline: model steps, retrieval, tools, validation, and glue code together determine quality.
• Consider a framework when you need inspectable failures, recoverable runs, heavy branching, or faster retrieval experimentation.
• Map each framework feature to code you're currently maintaining, not to the feature matrix on a landing page.
• Before committing, compare the direct implementation with the framework implementation on the same workflow and the same evaluation data.