LLM 0.32a0 Released: Message Sequences and Typed Streaming Refactor Explained

Simon Willison has released version 0.32a0 alpha of his open-source project LLM, a major backward-compatible refactor. Simon Willison is the co-creator of the Django web framework and the author of the data exploration tool Datasette. In recent years, he has become one of the most active open-source developers in the AI tooling space. The LLM project is both a command-line tool and a Python library designed to provide a unified interface for accessing various large language models (including OpenAI, Anthropic, local models, and more). Through its plugin ecosystem, it supports dozens of different model backends and has garnered over 10,000 stars on GitHub, making it a key piece of infrastructure for interacting with LLMs in the Python ecosystem. This release introduces two core changes: modeling model inputs as message sequences, and modeling model responses as typed streaming parts. These changes reflect the rapid evolution of the LLM landscape over the past two years.

From Simple Prompts to Message Sequences: The Evolution of LLM Input Models

The LLM project started in April 2023, when the world was simple: send a text prompt, get a text response. But as ChatGPT demonstrated the value of conversational interaction, model inputs have fundamentally become sequences of conversation turns.

This shift has a clear industry trajectory. When OpenAI launched the Chat Completions API in March 2023, it changed the input format from a single prompt string to a messages array (containing roles like system, user, and assistant). This design quickly became the de facto industry standard. Major model providers including Anthropic, Google Gemini, and Mistral all adopted similar message sequence formats. The core advantage of this design is that it can precisely express the contextual structure of multi-turn conversations, support few-shot example injection, and provide clear semantic separation between system prompts and user inputs.

Before version 0.32, LLM handled multi-turn conversations through a conversation object, but this approach had an obvious limitation — it was impossible to inject an existing conversation history from the start. This made tasks like building OpenAI-compatible APIs extremely difficult.

The new version introduces the llm.user() and llm.assistant() builder functions, allowing developers to pass in message arrays directly:

import llm
from llm import user, assistant

model = llm.get_model("gpt-5.5")
response = model.prompt(messages=[
    user("Capital of France?"),
    assistant("Paris"),
    user("Germany?"),
])
print(response.text())

The original prompt= parameter still works — LLM upgrades it under the hood to a single-element message array. Additionally, the new response.reply() method provides a more natural way to continue conversations.

Typed Streaming Parts: A New Approach to Multimodal Output

This is the more forward-looking change in this refactor. Today's frontier models return far more than plain text — Claude returns reasoning processes plus text plus tool calls, OpenAI's code interpreter returns execution results, and multimodal models can even interleave images and audio clips within streaming responses.

To understand the technical context of this change, you first need to understand the concept of streaming responses: models return results token by token during generation rather than waiting for complete generation before returning everything at once. This is critical for user experience — users can see the first character within milliseconds. As model capabilities have expanded, the content types in streaming responses have become extremely complex: Anthropic Claude's extended thinking feature outputs a complete reasoning chain before the formal answer (similar to an externalized Chain-of-Thought display); OpenAI's function calling and tool use mechanisms interleave structured tool call instructions within the stream; multimodal models may mix text, images, and audio within the same response stream. Traditional plain-text stream processing — simply concatenating each chunk into a string — is completely inadequate for this complexity.

The new stream_events() API models responses as a series of typed event streams:

for event in response.stream_events():
    if event.type == "text":
        print(event.chunk, end="", flush=True)
    elif event.type == "tool_call_name":
        print(f"\nTool call: {event.chunk}(", end="", flush=True)
    elif event.type == "tool_call_args":
        print(event.chunk, end="", flush=True)

The practical benefits of this design are immediate: the CLI tool can now display "thinking" text and final response text in different colors, with thinking text output to stderr so it doesn't interfere with pipe operations. Combined with the updated llm-anthropic plugin, Claude's reasoning process can be elegantly displayed and controlled.

Serialization and Deserialization: Breaking Free from SQLite Dependency

Previously, LLM's code for persisting conversations to SQLite was quite rigid. Version 0.32a0 adds response.to_dict() and Response.from_dict() methods that return JSON-style dictionaries defined with TypedDict, allowing developers to freely choose their storage backend.

TypedDict is an important feature in Python's typing module that allows developers to define precise key-value type constraints for dictionaries. Unlike regular dictionaries, TypedDict provides full type inference and error detection in static type checking tools (such as mypy and pyright). Using TypedDict instead of custom classes in serialization scenarios means the serialized result is inherently a JSON-compatible dictionary structure, requiring no additional encoding/decoding logic while retaining type safety — a design choice that balances practicality with engineering rigor.

serializable = response.to_dict()
# Store it wherever you like
response = Response.from_dict(serializable)

This opens the door to using the LLM Python library without SQLite dependency, giving developers more flexibility to integrate the tool across different projects.

Design Philosophy: Backward-Compatible Incremental Evolution

Interestingly, despite being a "major refactor," it maintains backward compatibility. The existing model.prompt("text") usage is completely unaffected, and the only CLI-level addition is the -R/--no-reasoning flag. This reflects a mature open-source project evolution strategy: paving the way for new capabilities without breaking existing user workflows. This strategy has deep roots in the Python ecosystem — Python's own painful migration from 2.x to 3.x made the entire community highly vigilant about breaking changes, and excellent library maintainers go to great lengths to smooth transitions through gradual deprecation and compatibility layers.

Future Plans and Industry Significance

Simon says the next priority is redesigning the SQLite logging system so it can capture the details returned by the new abstractions with greater granularity. He's leaning toward modeling it as a graph structure to support scenarios like OpenAI-style conversations that keep extending without creating duplicate storage in the database.

This design approach deserves deeper examination. Traditional conversation logs typically use linear storage — each conversation is saved as a complete record. But in practice, conversations often take on tree-like or graph-like structures: users might branch off into multiple different follow-up conversations from the same conversation node, or use a conversation history as a prefix for a new conversation. With linear storage, every branch requires copying the complete message history, creating significant data redundancy. Graph-structured storage allows multiple conversation branches to share common history nodes, similar to Git's commit graph — saving storage space while precisely tracking the evolution path of conversations. This design is particularly important for scenarios requiring frequent A/B testing of different prompt strategies or building complex agent workflows.

The significance of this refactor extends beyond the LLM project itself — it reflects the entire industry's ongoing thinking about "what is the right abstraction for interacting with large models." From plain text to multimodal, from single calls to tool orchestration, the abstraction layer must keep pace with the evolution of model capabilities. Similar evolution is happening simultaneously in other projects: LangChain evolved from simple chain-based calls to LangGraph with support for complex agent graphs, and Vercel's AI SDK is continually refactoring its streaming protocol to accommodate new model capabilities. For Python developers building AI applications, LLM 0.32a0 offers a design reference worth studying — it demonstrates how to provide sufficient expressive power for increasingly complex model interaction patterns while maintaining a clean API.

Key Takeaways

• LLM 0.32a0 upgrades model input from simple text prompts to message sequences, supporting direct injection of conversation history
• A new typed streaming event mechanism (stream_events) distinguishes between different output types such as text, tool calls, and reasoning processes
• A to_dict/from_dict serialization mechanism lets developers freely choose storage solutions independent of SQLite
• Full backward compatibility is maintained — existing API usage is unaffected
• Future plans include redesigning the SQLite logging system as a graph structure to avoid duplicate storage of conversation history