GPT-5.6 Hands-On: Farm Automation, Mathematical Breakthroughs, and End-to-End AI Execution

When AI Steps Off the Screen and Into the Physical World

Over the past few weeks, users around the world have begun testing the GPT-5.6 model in the all-new ChatGPT. The biggest difference from previous versions is that this generation's capabilities are no longer confined to the screen—they've begun to make an impact in the real, physical world. From farms in Hokkaido, to home businesses in New York, to research breakthroughs by Polish mathematicians, GPT-5.6 is demonstrating an entirely new paradigm for AI applications.

According to the source video, this release comprises a model family of three members—Sol, Terra, and Luna—each covering different application scenarios. The three names evoke the cosmic imagery of "Sun, Earth, and Moon," hinting at an ecosystem that spans everything from high-powered reasoning to everyday assistance. This model-family strategy has become the mainstream approach among leading AI labs: Google (Gemini Ultra/Pro/Flash) and Anthropic (Claude Opus/Sonnet/Haiku) both adopt similar tiered architectures, matching different parameter scales and inference costs to different scenario needs, thereby avoiding the dilemma of trading off performance against deployment cost within a single model.

Industry Context: Why Are All Leading Labs Moving Toward Model Families? The inference cost of large language models scales roughly linearly with parameter count, and the API call cost of a single ultra-large model in high-concurrency scenarios is often prohibitive for small and medium-sized businesses. By building model families, labs can use lightweight models (such as Flash/Haiku/Luna) to handle high-frequency, low-complexity everyday tasks, while reserving heavyweight models (such as Ultra/Opus/Sol) for professional scenarios requiring deep reasoning—striking an optimal balance between overall inference cost and user experience. The essence of this strategy is "allocating compute on demand," rather than a technical compromise.

The Deeper Logic of Compute Economics: From the perspective of inference cost structure, the strategic significance of the GPT-5.6 three-model family goes far beyond mere product-line segmentation. The Transformer architecture was introduced in 2017 by the Google Brain team in the paper Attention Is All You Need. Its core mechanism—Multi-Head Self-Attention—has a computational complexity of O(n²) with respect to sequence length, which directly causes inference costs to climb sharply as the context window expands. As a typical example, the cost of a single call to a flagship model with hundreds of billions of parameters may be 10–20 times that of calling a lightweight model with tens of billions of parameters. Using the A100 GPU as a reference, a 100-billion-parameter model requires about 80GB of VRAM to handle a single inference request, whereas a 10-billion-parameter-class model needs only about 20GB. Benefiting from KV Cache optimization and INT8/INT4 quantization techniques, the actual throughput of lightweight models can often reach 5–10 times that of flagship models. When daily API call volume reaches the scale of hundreds of millions, this cost difference directly determines the sustainability of the business model. It is precisely for this reason that the three-tier division of Sol (flagship reasoning), Terra (professional tasks), and Luna (everyday assistance) serves as both a technical roadmap and a business-model design—giving users of different willingness-to-pay what they each need, while maximizing overall compute utilization.

The core thread running through all these cases is the leap in the model's ability to "complete tasks end-to-end in a single pass."

Farm Automation: IoT in Practice in Hokkaido

The most striking case in the video comes from Hiroki, a farmer in Hokkaido, Japan. With the help of GPT-5.6, he achieved automatic control of the opening and closing of his greenhouse door—which is by no means a simple software task.

GPT-5.6 served throughout as an engineering consultant spanning both software and hardware: first advising Hiroki on which hardware to purchase, then guiding him through wiring the Raspberry Pi, and finally helping him step by step to complete the motor installation.

About the Raspberry Pi and the Barriers to IoT: The Raspberry Pi is a single-board computer developed by the UK-based Raspberry Pi Foundation. Since its release in 2012, over 50 million units have been sold. Priced at around $35, it can connect directly to physical devices such as sensors and motors via its GPIO (General Purpose Input/Output) pins. The GPIO interface has 40 pins and supports multiple communication protocols including digital input/output, PWM (Pulse Width Modulation), I2C, SPI, and UART. Following the BCM (Broadcom) numbering convention of ARM processors and operating at 3.3V logic levels, it is the core bridge connecting the software world to the physical world.

It's worth noting that the GPIO pins themselves have a maximum output current of only about 16mA, which is insufficient to drive a motor directly. The Raspberry Pi's GPIO ecosystem has already formed a complete knowledge graph in the field of IoT education: relay modules, as high-voltage isolation devices, convert the GPIO's low-voltage 3.3V control signal into switching control for a 220V AC or 12V DC motor via electromagnetic induction—essentially a safe isolation scheme of "controlling high power with a small signal." H-bridge driver chips (with the L298N as a representative) provide finer PWM speed control, achieving linear control of motor speed by changing the pulse duty cycle, while also supporting forward/reverse switching. Both schemes have extensive open-source library support in the DIY community (such as RPi.GPIO and gpiozero), but details like precise wiring order, pull-up/pull-down resistor configuration, and the GPIO numbering system (distinguishing BCM numbering vs. BOARD numbering) remain the pitfalls that beginners most often stumble on. The layering of these two levels of hardware knowledge is precisely the root cause of the greatest barrier facing ordinary users: a single wiring error at any step could destroy the Raspberry Pi or even cause a safety accident.

The Hidden Costs of IoT Hardware Debugging: Take a typical greenhouse automation project as an example. The GPIO wiring debugging stage alone involves at least three common failure modes: first, logic-level mismatch (connecting a 5V device directly to a 3.3V GPIO pin, destroying the chip); second, floating pins (unconfigured pull-up/pull-down resistors causing random level fluctuations, leading to erratic motor jitter); and third, broken ground loops (improper common grounding among multiple devices causing signal interference or device non-response). To professional embedded engineers, these three issues are basic common sense, but for agricultural practitioners lacking an electrical background, each one requires expert guidance to identify and resolve. AI, serving as an ever-available "electrical engineering consultant," is especially valuable in this scenario.

The Deeper Value of AI as a Cross-Domain Knowledge Integrator: The knowledge stack involved in a complete greenhouse automation project includes: electrical engineering (motor drives, relay selection, current/voltage matching), embedded systems (GPIO programming, real-time scheduling), Linux system administration (permission configuration, process daemons), Python/C programming, and the communication protocols of specific sensors (I2C/SPI/UART). This knowledge was originally scattered across dozens of technical manuals and required years of engineering practice to master. IoT projects typically require developers to simultaneously grasp electrical principles, embedded programming, Linux system configuration, and specific hardware specifications—work that in the past could only be handled by embedded engineers. AI's value in this scenario lies in instantly contextualizing this knowledge—not giving generic answers, but providing executable, precise guidance tailored to the specific hardware models Hiroki purchased and the specific wiring state at hand. The value of AI acting as a "cross-software-hardware engineering consultant" is essentially aggregating knowledge—once scattered across multiple specialized books and years of practical experience—into a single conversational interface, on demand and in real time. An IoT project that once required specialized expertise thus becomes an operation that even an ordinary farmer can complete independently.

Hiroki summed up the difference between the old and new models precisely: the old model required repeated multi-turn interactions and continuous error correction, whereas the new model supports "one-shot prompting"—Codex reads the database on its own, proactively invokes various tools, understands the ultimate goal, and achieves it independently. He called it an "unbelievable change."

Smart Assistant for Small Businesses: From Chaotic Input to Polished Output

A Five-Minute "Brain Dump" Yields a Complete Business Dashboard

The second case comes from a couple in New York running a breakfast cereal business, along with Jake from the Three Wishes team. Jake demonstrated the instruction he gave the model:

"I always start by saying, please set yourself a goal of completing the task end-to-end. Please use the historical release data to guide the dashboard itself. Use Three Wishes' brand elements, especially the new branding..."

This input, described on the spot as "the most disorganized thought stream I've ever seen," precisely highlights GPT-5.6's core strength—it can automatically transform a long, fragmented verbal description into a complete execution plan, proactively pull relevant data and brand assets, and ultimately output polished, usable dashboards, presentations, or spreadsheets.

It's worth noting that this ability to "tolerate ambiguous input and automatically complete intent" relies technically on the model's deep inference of contextual intent, rather than simple instruction matching. This fundamentally differs from the logic of traditional RPA (Robotic Process Automation) tools that require strictly structured input. Traditional RPA tools such as UiPath and Automation Anywhere rely on predefined rules and precise UI element positioning—essentially "record-and-replay" automation scripts: once an interface element's position changes or the input format doesn't match expectations, the process breaks and throws an error.

From RPA to AI Agent: A Generational Leap in the Automation Paradigm: Traditional RPA emerged in the 2000s, with its technical foundation being screen scraping and rule engines—simulating manual operations by identifying the absolute coordinates or XPath paths of UI elements. The fundamental flaw of this "brittle automation" is its zero tolerance for environmental changes: a single software interface update could cause the entire automation process to collapse, resulting in extremely high maintenance costs. RPA vendors' typical response was to introduce an "image recognition" fault-tolerance layer, but this was merely a stopgap that couldn't understand task semantics at all.

In contrast, LLM-based AI Agents introduce a semantic understanding layer as a buffer: even if the input format changes or interface elements shift, the model can still locate intent through semantic inference and dynamically adjust its execution path. This fundamentally solves the brittleness problem of traditional RPA—and explains why Gartner ranked "AI-augmented automation" first among its top ten strategic technology trends in 2024, and why many RPA vendors (including UiPath and Blue Prism) began integrating LLM capabilities into their product lines after 2024. From an underlying architecture perspective, the essence of this evolution is replacing the "rule-matching layer" with a "semantic-reasoning layer": the former relies on manually enumerating all possible input variants, while the latter learns a generalized understanding of intent through training, thereby naturally possessing fault tolerance for unseen inputs. LLM-based AI Agents achieve the leap from "rigid automation" to "elastic autonomy": the model can not only tolerate format changes but also proactively infer user intent and dynamically select tool chains when input is ambiguous.

The Technical Mechanism of Intent Inference: Prompt Engineering and Semantic Compression The model's ability to handle "chaotic input" stems technically from its statistical modeling of vast amounts of human natural language during pretraining—the model learns to map superficially messy language signals to underlying structured intent. This process is analogous to "Semantic Compression" in information theory: extracting the minimal sufficient statistic (i.e., the core intent and constraints of the task) from redundant natural-language expression, then expanding it into an executable sequence of actions. Jake's "most disorganized thought stream" is, to the model, essentially a piece of structured information containing the task goal (dashboard), data source (historical release data), style constraints (brand elements), and execution mode (end-to-end)—just expressed in the informal way humans are accustomed to. This is precisely the core capability that distinguishes LLMs from traditional NLP systems: the latter relies on explicit grammatical structures, while the former can understand the speaker's true intent at the pragmatic level.

The user's exclamation was quite representative: "This feels like something only big companies worth billions of dollars can usually build, and we can get it now? I have my own custom software system." This statement pinpoints the true value of AI productivity tools for small and medium-sized businesses—thoroughly democratizing custom-software capabilities that were once out of reach.

Cracking a Mathematical Problem Unsolved for Three Years

The weightiest case comes from Polish mathematician Bartosz. He had studied the same math problem for a full three years without ever breaking through. After trying Codex 5.6, the situation changed fundamentally.

Bartosz said that Codex proposed a "completely new idea" that helped him disprove the conjecture he had been trying to prove for those three years. The technical key lies in this: GPT-5.6 can decompose a complex problem into parallel workflows, with multiple Agents solving different sub-problems separately.

Notably, "disproving a conjecture that had been studied for three years" is itself of significant value—it saves the researcher from continuing to invest time down an erroneous path. The role AI plays here is akin to that of an "external reviewer": because it carries none of the researcher's path dependence or confirmation bias, it is actually more likely to identify fundamental flaws in the line of reasoning.

Cognitive Science Background: Why Can AI Break Through the Fixed Thinking of Human Researchers? Confirmation bias is one of the most thoroughly documented systematic biases in cognitive psychology, referring to people's tendency to seek out, interpret, and remember information consistent with their existing beliefs while unconsciously ignoring counterevidence. Daniel Kahneman, in Thinking, Fast and Slow, classifies it as a typical flaw of "System 1 thinking"—intuition-driven rapid judgments that are easily anchored by existing frameworks. In long-term scientific research, researchers often develop an emotional attachment to hypotheses in which they have invested substantial time, leading to a marked decrease in sensitivity to negative evidence—a phenomenon also described psychologically as a cognitive extension of the "sunk cost fallacy." Deeper neuroscience research shows that when the prefrontal cortex focuses on a single hypothesis over the long term, it forms an "attentional tunneling" effect, actively suppressing the processing of information that contradicts the current mental model—an evolutionary optimization for the brain to conserve cognitive resources, yet an obstacle in creative work requiring paradigm shifts. AI's advantage in this scenario lies not only in compute power but also in its "path-independence" characteristic—the model does not assign extra weight to a conjecture simply because it has been "studied for three years." This aligns closely with the objective-prior principle of Bayesian inference and is highly complementary to the mechanism of human peer review, rather than a mere replacement for it.

The Technical Foundation of Multi-Agent Parallel Architecture: Multi-Agent architecture is a key technical path for deploying large models on complex tasks. Its core idea originates from distributed computing—decomposing a complex problem into multiple sub-tasks, processing them in parallel via mutually independent "agents," and then aggregating the results through a coordination layer.

Search-Space Coverage in Mathematical Proof Scenarios: In computational complexity theory, mathematical proof is a typical "wide search space, high verification cost" problem: the number of possible proof paths is often exponential, while verifying the correctness of a single path requires rigorous logical deduction. Human researchers, constrained by linear thinking and working memory capacity (psychological research shows that the average capacity of human working memory is about "7±2" chunks of information, first quantified by George Miller in his 1956 paper The Magical Number Seven, Plus or Minus Two), can typically only explore 2–3 main paths serially. The core breakthrough of the multi-agent framework lies in parallelizing the search process: different agents can simultaneously explore entirely different proof paths—algebraic transformations, geometric interpretations, constructive proofs—while a coordination layer (Orchestrator Agent) is responsible for evaluating the progress of each path and dynamically allocating computational resources to the most promising directions.

In form, this mechanism closely resembles Monte Carlo Tree Search (MCTS)—an extended application of the very core algorithmic idea that AlphaGo relied on to defeat human Go champions. MCTS was formally proposed by Rémi Coulom in 2006 and later became widely known through DeepMind's engineering applications in AlphaGo (2016) and AlphaZero (2017). Its core four-step cycle—Selection, Expansion, Simulation, and Backpropagation—constitutes an effective approximate sampling scheme for exponential search spaces. MCTS simulates future states through random sampling, dynamically balancing between "exploring new paths" and "exploiting known advantageous paths" (the classic Exploration-Exploitation tradeoff), thereby efficiently locating optimal moves within the exponential branches of the Go game tree. Notably, the "Chain-of-Thought Scaling" introduced by OpenAI in its o1/o3 series models, as well as DeepMind's application in AlphaProof for mathematical proofs, both reflect an extension of this search idea: exploring a broader solution space by investing more computational resources at inference time, rather than relying solely on increasing parameter scale.

Formal Verification and the Technical Boundaries of AI Mathematical Reasoning: DeepMind's AlphaProof converts mathematical proof problems into formal proof tasks, relying on theorem provers such as Lean and Isabelle as the verification backend—every step of derivation the model generates is mechanically verified by the prover, eliminating the risk of "hallucinated proofs." This complements GPT-5.6's multi-agent framework: the former guarantees rigor through formal verification, while the latter enhances coverage breadth through semantic understanding and parallel exploration. In Bartosz's case, the "completely new idea" provided by the AI still needed rigorous verification by the mathematician himself—which is precisely the rational division of labor in human-AI collaboration at the current stage: the AI is responsible for rapidly scanning the solution space and proposing hypothesis directions, while the human is responsible for formal verification and final judgment.

The solution space of mathematical proofs shares profound structural similarities with Go games: both face an exponentially exploding branching factor, and the verification cost of a single path is extremely high. The multi-agent framework borrows this idea, replacing serial search with parallelization—essentially trading compute for broader coverage of the solution space. In mathematical proof scenarios, multi-agent parallel processing significantly improves solving efficiency and the breadth of solution-space coverage. Leading institutions such as OpenAI, Anthropic, and Google DeepMind have all listed multi-agent frameworks as core research directions since 2024, and this architecture is also the underlying infrastructure for AI's evolution from a "single-turn Q&A tool" into an "autonomous execution system."

The Core Leap: True End-to-End Autonomous Execution

Taken together, the most essential progress of GPT-5.6 across these three cases can be summarized as "autonomous end-to-end execution capability."

Understanding the Technical Significance of "End-to-End": The concept of "End-to-End" originates from the field of machine learning, and its history has clear milestones: in 2012, AlexNet defeated hand-crafted feature-engineering methods in the ImageNet competition with an end-to-end convolutional network, marking the formal establishment of this paradigm; in 2016, WaveNet generated high-quality speech in an end-to-end manner, phasing out the segmented pipeline of traditional TTS; and in 2017, the Transformer extended the end-to-end paradigm to sequence modeling. First systematically expounded by Yann LeCun and others in convolutional neural network research on image recognition, the end-to-end approach refers to a model mapping directly from raw input to final output, without the need for manually designed step-by-step feature engineering—traditional machine learning pipelines require manually extracting features (such as edge detection, color histograms) before feeding them into a classifier, whereas an end-to-end model fuses feature extraction and classification decision-making into one, with gradient descent automatically optimizing the entire mapping function.

When applied to AI Agents, this concept has undergone an important extension from "perception end-to-end" to "action end-to-end": the model's output boundary expands from token sequences to state changes in the real world. This also echoes, at the engineering level, an early implementation of the "World Model" concept advocated by Yann LeCun—the Agent needs to build an internal representation of the environmental state and, on that basis, plan a sequence of actions. The model can not only answer "how to do it" but also autonomously invoke external tools (code executors, databases, APIs), perceive the results of execution, and dynamically adjust its strategy until the task is complete. The essential difference from a traditional "chatbot" is this: the latter merely conveys information, while the former is a true task executor.

This leap relies technically on the maturation of Function Calling and Tool Use capabilities. OpenAI officially introduced Function Calling in June 2023, allowing the model to output "function-call intent" in structured JSON format, which an external system then actually executes before returning the result to the model. The core breakthrough of this mechanism lies in transforming the model from a "text generator" into an "intent parser": the model is responsible for understanding the goal, planning steps, and selecting tools, while the execution layer is handled by deterministic code—each performing its own role.

The Technical Evolution of Function Calling: After Function Calling was introduced in June 2023, it underwent two key upgrades: first, Parallel Function Calling, which allows the model to initiate multiple tool calls simultaneously within a single response, significantly reducing the total latency of multi-step tasks—for example, querying a weather API and a calendar API at the same time rather than waiting serially; second, the Tool Choice Force Mode, which allows the caller to force the model to use a specific tool, improving the controllability and predictability of the system—especially critical for enterprise scenarios requiring strict auditing. On this foundation, the Structured Outputs feature introduced in 2024 further guarantees the JSON Schema compliance of function-call parameters, raising the reliability of tool calls from "best effort" to "guaranteed." This technical accumulation, working in concert with tools such as the code interpreter, browser, and file system, together constitutes the engineering infrastructure that enables GPT-5.6 to "complete tasks end-to-end in a single pass."

The ReAct Framework: A Unified Paradigm for Agent Reasoning and Action The engineering implementation of Function Calling corresponds academically to the ReAct (Reasoning + Acting) framework proposed by Google Research in 2022. ReAct interweaves the LLM's reasoning process (Thought) with external tool invocation (Action) into a unified sequence: the model first generates an internal reasoning trace (similar to "I need to query the database first, then compute the summary"), then outputs the corresponding action command, and the execution result is fed back to the model as an Observation, driving the next round of reasoning. This "Thought-Action-Observation" loop is precisely the core architectural prototype of all mainstream AI Agent frameworks today (LangChain, LlamaIndex, AutoGen). Compared with pure Chain-of-Thought reasoning, ReAct's introduction of external knowledge sources enables the model to break through the limitations of parametric memory and dynamically access the latest information; compared with pure tool calling, the explicit reasoning trace greatly enhances the interpretability and debuggability of Agent behavior—these two points together explain why it has become the standard architecture for production-grade AI Agents.

The industry refers to this capability leap as a shift from LLM (Large Language Model) to LAM (Large Action Model)—companies like Rabbit were among the first to popularize this term, emphasizing that the model can directly operate GUI interfaces and invoke OS-level APIs, achieving a complete closed loop of perception-planning-execution. The fundamental difference between LAM and LLM is this: the output of an LLM terminates in text, whereas the output of a LAM terminates in a state change in the real world; the former is an information conveyor, the latter an action executor. The further extension of this capability boundary is precisely the most fundamental generational difference between GPT-5.6 and earlier versions—and a key milestone in AI's evolution from an "assistive tool" into an "autonomous agent."

Whether it's proactively reading databases, independently invoking tools, or understanding the ultimate goal and planning the path on its own, the model demonstrates a shift from "passive response" to "proactive completion." Users are thus freed from tedious multi-turn debugging—what once required constant error correction, supplementation, and iteration can now reach the result directly with a single complete statement.

As the video summarizes: "With 5.6, you can do far more than you imagine."

From Tool to Productivity Partner

From the greenhouses of Hokkaido, to the cereal business on New York dining tables, to the frontiers of mathematical research in Poland, what GPT-5.6 demonstrates is not just an improvement in model performance, but a profound evolution in the form of AI applications. When AI can autonomously complete end-to-end tasks and begins to influence the physical world, its role upgrades from a mere "tool" to a true "productivity partner."

That said, all the above cases come from official promotional videos. The actual boundaries of the model's capabilities, its stability, and its performance in broader scenarios still await third-party testing and long-term validation by more users. Regardless, the direction GPT-5.6 points to—enabling AI to truly land and create visible value in the real world—represents an important trend in the current development of large-model technology.

Key Takeaways

Key Takeaways