CrewAI + FastAPI in Practice: Building a Multi-Agent Collaborative Application

What is CrewAI

As large language models mature, single-model conversations can no longer meet the demands of complex business scenarios. "Multi-agent collaboration" is rapidly becoming the new paradigm for AI application development. The rise of Multi-Agent Systems is fundamentally rooted in the inherent limitations of individual large models — even top-tier models like GPT-4 can still suffer from hallucinations, forgotten intermediate steps, or insufficient reasoning depth when faced with tasks requiring long-chain reasoning, cross-domain knowledge integration, or extremely long context handling. Multi-agent architectures tackle this by decomposing tasks and enabling specialized division of labor: complex problems are broken down into manageable sub-tasks, with each Agent focusing on its area of expertise, significantly improving overall output quality.

CrewAI is a representative framework in this trend — designed specifically for building multi-Agent systems, enabling multiple Agents with different roles and goals to collaborate on complex tasks that would be difficult to handle independently. Notably, CrewAI is built on top of the LangChain ecosystem, but focuses on solving LangChain's complexity around multi-Agent orchestration: LangChain provides the underlying LLM calls, tool integrations, and chain-of-thought reasoning capabilities, while CrewAI wraps a higher-level "role-playing" abstraction on top, allowing developers to quickly build collaborative systems without needing to deeply understand the underlying Agent loop mechanics.

The core idea is: decompose a complex task and assign the pieces to different Agents, each completing their responsibilities using their specific skills and tools, then aggregating the results toward the overall goal. This design closely mirrors a real-world project team — everyone plays their part to deliver a complete outcome.

This article covers the core concepts of CrewAI and demonstrates how to wrap a multi-Agent application as an API service using FastAPI.

The Five Core Concepts of CrewAI

Understanding CrewAI comes down to truly grasping a few key abstractions.

Agent

An Agent is an autonomous, controllable execution unit that can perform tasks, make decisions, and communicate with other Agents — analogous to a member of a team. Its key attributes include:

• role: defines the function the Agent serves within the team
• goal: the specific objective the Agent aims to achieve
• backstory: provides context for the Agent and shapes its behavioral style

The backstory is far from decorative text — it is essentially a role definition written into the System Prompt, directly influencing the LLM's output style and decision-making tendencies. A well-crafted backstory can effectively activate the model's knowledge in a specific domain, making it perform more like a domain expert.

Task

A Task is a concrete unit of work assigned to an Agent, requiring all the details necessary for execution. Core attributes include the task description, the assigned Agent, expected output, and the list of available tools. One often-overlooked feature is that Tasks support a context attribute — the output of a previous task can be passed as input to the next, enabling chained task execution. Output can be in structured format or written directly to a file in a specified format (e.g., txt, md).

Process

A Process coordinates how Agents execute tasks, similar to a project manager within a team. Two mechanisms are currently available:

• Sequential: Tasks are executed in a predefined order from the task list; the output of one task becomes the context for the next
• Hierarchical: A designated "manager Agent" oversees the planning, delegation, and validation of tasks; tasks are not pre-assigned but dynamically dispatched based on each Agent's capabilities

The hierarchical process implements a reasoning loop similar to ReAct (Reasoning + Acting) under the hood — the manager Agent continuously observes sub-Agent execution results and dynamically decides the next task assignment strategy. This is why hierarchical processes require a separately specified, more capable manager_llm.

Crew

A Crew represents a group of Agents collaborating to complete tasks. It defines the execution strategy, collaboration approach, and overall workflow. Key attributes include the task list (tasks), Agent list (agents), and process. For hierarchical processes, manager_llm must also be specified to designate the LLM used by the manager Agent.

Pipeline

A Pipeline is a higher-level orchestration structure that allows multiple Crews to execute sequentially or in parallel. It involves concepts like Stages, Runs, and Traces, and is suited for building more complex multi-stage workflows. Multiple Crews running in parallel within a Pipeline are independent of each other, each with their own Agents and Tasks; the Pipeline aggregates results from all stages at the end. This makes it ideal for scenarios requiring simultaneous analysis of the same problem from multiple dimensions.

Walkthrough of the Official Starter Example

To aid understanding, this tutorial uses the official starter example, which automatically generates a research report on a given topic.

The example defines two Agents:

• Researcher: A senior data researcher whose goal is to explore cutting-edge developments on a topic, with a backstory of being "a seasoned researcher skilled at uncovering the latest advances and presenting information in a clear and concise manner"
• Reporting Analyst: Creates detailed reports based on data analysis results, with a backstory of being "a meticulous analyst skilled at transforming complex data into clear and concise reports"

Two corresponding Tasks are defined: a research task (conduct in-depth research on a topic, output a list of ten bullet points, assigned to the Researcher) and a report task (expand the ten points into a full report, output as a Markdown file, assigned to the Reporting Analyst). The two tasks are linked via context, forming a "research first, then write" pipeline.

Environment Setup and Multi-Model Integration

Preparation before the hands-on work covers several areas, with flexibility in LLM integration being a key highlight of this tutorial — supporting three options: GPT, non-GPT cloud-based models, and local open-source models.

Development Environment

Anaconda provides the Python virtual environment, PyCharm serves as the IDE, and Python 3.11 is used.

Three LLM Integration Options

• GPT models: Accessed via a proxy
• Non-GPT cloud models: Managed uniformly through One-API (an OpenAI interface management and distribution system). One-API is an open-source API middleware whose core value is adapting different vendors' LLM APIs into the OpenAI format — since OpenAI's API specification has become the de facto industry standard, major domestic model providers (Alibaba, iFlytek, Zhipu, etc.) all offer compatible interfaces. One-API acts as a middleware layer to centrally manage multiple API keys, enable load balancing, and monitor usage and costs. Using Alibaba's Qwen as an example: create a channel in One-API, bind the Bailian platform API key, generate a token to get a unified API key, and the code can seamlessly switch between Qwen, iFlytek Spark, Zhipu, and other domestic models
• Local open-source models: Deployed using Ollama. Ollama is currently one of the most mainstream local LLM runtime frameworks, implementing efficient CPU/GPU hybrid inference based on llama.cpp. It supports pulling and running dozens of open-source models like Llama, Mistral, Gemma, and Qwen with a single command, significantly lowering the hardware barrier for local deployment. For enterprise scenarios sensitive to data privacy, Ollama provides fully offline inference — all data stays within the local network. Commonly pre-installed models include Qwen2, Llama 3.1 (8B), and Gemma2

This "one codebase, multiple models" architecture makes it very convenient for developers to switch between testing and production environments.

Building the CrewAI + FastAPI Application

The overall approach is two steps: first get the official template running, then wrap it as an API service.

Step 1: Run the Official Template

Use the CrewAI scaffold command to create a project template. The generated project structure is clean: the config directory contains two configuration files, agents.yaml and tasks.yaml, along with main.py (entry point) and crew.py (the core script for creating the Crew).

After configuring the API URL, API key, and model name in .env, run crewai install to install dependencies, then crewai run to start the application. Upon completion, a report.md file is generated in the project root directory.

The collaboration process is clearly visible in the execution logs: the Researcher Agent first fetches the latest information on the topic, and CrewAI creates an execution chain that guides it through the work step by step. Once the Researcher finishes, the Reporting Analyst Agent takes over, writes the report based on the provided information, and outputs it to a Markdown file.

Step 2: Wrap as a FastAPI Service

FastAPI is a modern asynchronous Python web framework built on Starlette and Pydantic, with native support for async IO (async/await). In AI service scenarios, LLM inference often takes several seconds or even tens of seconds; FastAPI's async nature allows it to continue handling other requests while waiting for model responses, significantly improving service concurrency. Its auto-generated OpenAPI documentation also makes it easy for frontend teams to integrate quickly.

FastAPI is introduced on top of the official template, enabling CrewAI to provide standardized API services externally. The core logic includes:

1. Model configuration: Three configurations are pre-defined in code — Qwen Max, GPT-4o mini, and Llama 3.1 — switchable via a single flag
2. Application initialization: FastAPI's Lifespan hook is used to load the corresponding LLM environment variables on startup based on the flag, and perform cleanup on shutdown
3. POST endpoint: Accepts a user-provided topic, calls the run function to trigger the Crew's kickoff method, and returns the report result to the requester via either streaming or non-streaming response

A separate api_test.py script sends a POST request to localhost:8012 to test the complete service. The returned result is identical to local execution, confirming the wrapper is working correctly.

Comparing All Three Models in Practice

Testing the same prompt across all three models yields some insightful conclusions:

• GPT-4o mini: Best performance — strictly outputs exactly 10 items as required, with faster speed
• Qwen Max: Good results, but output 15 items (did not strictly follow the 10-item requirement), slightly slower than GPT-4o mini
• Llama 3.1 (7B local): Weakest performance — the research phase only found 3 items, the final report was notably thin, and it struggled under the constraints of local hardware resources

There's a clear technical reason behind these results: a model's instruction-following capability is highly correlated with its parameter count. 7B-parameter local models generally underperform larger or closed-source commercial models in terms of instruction following, format control, and complex reasoning — larger models learn finer-grained instruction boundaries during the RLHF (Reinforcement Learning from Human Feedback) stage. In Agent systems, instruction following is especially critical: Agents must act precisely according to the roles and output formats defined in the system prompt, and any deviation can break the integrity of the task chain, causing downstream Agents to receive substandard context input.

This comparison highlights a key engineering insight: model selection must be thoroughly evaluated when building Agent applications. Small local models are free and privacy-preserving, but often fall short on complex tasks. When hardware allows, it is recommended to prioritize models with larger parameter counts (e.g., 32B+ local models, or cloud-based commercial models).

Summary

CrewAI provides a clear abstraction system for multi-Agent application development — from Agent and Task to Process, Crew, and Pipeline, offering layered support for orchestration needs ranging from simple to complex. Combined with FastAPI, developers can quickly wrap Agent capabilities into standard APIs for external consumption and integrate them into existing systems.

What truly determines application quality, beyond a well-designed framework, is the choice of underlying LLM. GPT, domestic cloud models, and local open-source models each involve trade-offs, requiring a balance between cost, privacy, and performance — cloud-based commercial models have strong instruction-following capabilities but carry data egress risks, while local open-source models offer privacy control but demand higher hardware requirements and have performance ceilings. For developers looking to get started with multi-Agent development, CrewAI's low barrier to entry and high flexibility make it an ideal starting point.

Key Takeaways