IBM Think 2025 Deep Dive: Rising Hallucination Rates in Reasoning Models and OpenAI's Acquisition of Windsurf

This episode of IBM's podcast Mixture of Experts brings together IBM researchers and product leaders for an in-depth discussion on three major topics: the key announcements from IBM Think 2025, the resurgence of hallucination problems in reasoning models, and the strategic intent behind OpenAI's $3 billion acquisition of Windsurf. Taken together, these threads trace a critical inflection point in the AI industry — a shift from model competition toward engineering execution, and from technical showmanship toward real value delivery.

IBM Think 2025: Generative Computing, 150+ Enterprise AI Agents, and Granite 4

Generative Computing: A New Paradigm for AI Development

At this year's Think conference, IBM introduced a compelling new concept — Generative Computing. Kate Soule, Director of Technical Product Management for Granite, described it as the "third wave of computing," following traditional computing and quantum computing.

The core idea is straightforward: today's LLM-based application development is still largely stuck in the "prompt engineering" phase — developers write lengthy, brittle prompts that are over-optimized for specific models. This approach is neither maintainable nor secure. Generative Computing advocates bringing software engineering and computer science best practices into AI development: using abstraction layers, programmatic control flow, and modular design so that models only handle tasks that genuinely require natural language capabilities, rather than "handing everything off to the model."

Kate emphasized that this structured approach also unlocks the potential of Inference Time Compute — breaking a large prompt into smaller tasks, running multiple generations, and synthesizing richer responses while reducing computational cost and latency. Inference Time Compute has been a major technical direction in the LLM space over the past two years. Rather than scaling up training, it invests more compute at inference time to achieve higher-quality outputs through techniques like multi-sampling, self-verification, and chain-of-thought decomposition. OpenAI's o-series models and DeepSeek-R1 are prime examples. IBM's Generative Computing concept essentially combines this inference-time approach with modular software design — instead of relying on a single massive prompt, complex tasks are broken into multiple controllable subtasks via programmatic orchestration, each independently invoking the model with result validation, improving output quality while reducing per-inference compute overhead.

150+ Pre-Built AI Agents and the Granite 4 Model Launch

IBM announced over 150 enterprise-grade pre-built AI Agents through the Watson X Orchestrate platform, with seamless integrations for tools like Salesforce, Workday, and Adobe. Chief Research Scientist Kauthar emphasized that these agents work out of the box while also supporting flexible customization on top of models like Granite and Mistral — reflecting a modular, multi-model design philosophy.

On the model side, IBM quietly released a Granite 4 model preview during the conference. It uses a Mixture of Experts (MoE) architecture with parameter counts ranging from 3B to 20B, and the smallest version can run a 120K token context window with just 15GB of memory. MoE is a conditional computation technique that sets up multiple parallel "expert" sub-networks inside the model, with a gating network dynamically deciding which experts handle each input token. This means that while the total parameter count may be large, only a fraction of parameters are activated during any given inference pass — preserving model capacity while dramatically reducing actual compute. Google's Switch Transformer was an early success story for MoE in large models, while Mistral's Mixtral 8x7B and DeepSeek-V3 brought the architecture into the mainstream. Granite 4's MoE design means its active parameter count during inference is far smaller than its total, which is precisely what enables it to run a 120K token context window in just 15GB of memory. Rather than chasing trillion-parameter scale, Granite 4 takes a "small but precise" approach — optimized for specific industry scenarios, emphasizing efficiency and cost-effectiveness as a high-performance complement to larger models.

The Hallucination Problem in Reasoning Models: Smarter but Less Reliable?

The Data Confirms It: AI Hallucination Rates Are Rising

A recent New York Times report revealed a troubling trend: as reasoning models grow more capable, hallucination rates are going up, not down. OpenAI's own model card data shows that o4-mini hallucinates more than o3, and o3 more than o1 — a clear upward curve.

Skylar admitted to finding this puzzling: "Accuracy is improving, math skills are getting stronger, but hallucinations are also increasing." Kate offered a deeper explanation: this is fundamentally a misalignment of incentives.

Current reasoning models are trained with extensive reinforcement learning, with a core objective of making models more "talkative" — generating more detailed, persuasive chains of thought. To understand the underlying mechanism: today's reasoning models are typically aligned using RLHF and its variants (such as GRPO and DPO). During this process, models learn to generate outputs preferred by human evaluators. The problem is that when faced with complex reasoning tasks, human evaluators tend to rate answers with thorough explanations and complete logical chains more highly — even when those answers contain subtle factual errors. This creates what's known as "reward hacking" — models learn to generate lengthy, convincing-sounding reasoning chains that aren't necessarily accurate. Humans naturally favor detailed, well-reasoned arguments but aren't good at fact-checking them. This creates a fundamental conflict with the objective function of earlier models, which aimed to "give precise answers." Longer chains of thought also mean more generation steps, and from a probabilistic standpoint, if each reasoning step has an error probability p, then a chain of n steps has an overall accuracy of roughly (1-p)^n — the more steps, the higher the compounding error rate grows exponentially. This mathematically explains why more capable reasoning models can exhibit higher hallucination rates.

Kauthar further noted that these models use probabilistic prediction rather than logical deduction to generate responses. Each step in multi-step reasoning introduces compounding error effects, and existing attribution and explanation tools simply can't keep up with model complexity. "Even the explanations shown to users sometimes don't reflect the model's actual internal process."

Hallucination Is an Inherent Limitation, Not a Bug to Be Fixed

A few years ago, some researchers optimistically predicted that hallucinations would be "completely solved within 18 months." Reality has proven that far too optimistic. The consensus emerging from the discussion is that hallucination may be an inherent limitation of current LLM architectures, not a simple engineering defect.

But that doesn't mean nothing can be done. Kate pointed out that the Generative Computing methodology is well-suited to address this challenge: rather than letting the model "handle everything" and hoping for the best, you build programmatic control flows that validate outputs at each critical node. IBM's Granite Guardian model can detect hallucinations in any fact-based response or function call.

Skylar proposed a pragmatic evaluation framework: the key is the error tolerance of the downstream use case. Hallucinations in a search engine are annoying but not catastrophic; hallucinations in legal arguments or medical diagnoses can be disastrous. When deploying AI, enterprises must calibrate the strictness of their safeguards based on the specific use case's tolerance for error.

Kauthar concluded that the future may require hybrid architectures — combining LLMs with verifiable databases or symbolic logic engines, rather than relying purely on reinforcement learning to "patch" hallucinations. This neuro-symbolic approach combines the natural language understanding of large language models with the precise reasoning of traditional knowledge graphs and rule engines, potentially delivering significant improvements in factual accuracy while preserving model flexibility.

OpenAI's $3 Billion Acquisition of Windsurf: The Anxiety of a Model Company

Why Pay $3 Billion for a "Code Editor"?

News of OpenAI's impending $3 billion acquisition of the coding environment Windsurf has sparked widespread debate. Some have questioned: if AGI is truly just around the corner, why spend a fortune on an "AI-powered text editor"?

Kate's analysis cuts to the heart of it: OpenAI became the giant it is today because the ChatGPT chat interface brought models to mass consumers, generating enormous volumes of user data to continuously improve the models. But in 2025's biggest killer use case — AI coding assistance — OpenAI lacks its own UI entry point and developer reach. AI coding assistance has become one of the fastest-growing AI application segments of 2024–2025, with tens of millions of developers worldwide using such tools daily. The competitive landscape in this market is clearly stratified: at the bottom are model providers (OpenAI, Anthropic, Google, etc.); in the middle are IDEs and editors (VS Code, JetBrains, etc.); at the top are products that deeply integrate model capabilities with development workflows. Windsurf (formerly Codeium) is a major player in this space, competing directly with GitHub Copilot, Cursor, and Tabnine. Cursor's rapid rise — by forking VS Code and deeply integrating AI features — proved the enormous value of the "integration layer." Acquiring Windsurf gives OpenAI a mature IDE product, an active developer community, and invaluable code-writing behavioral data — data that is critical for training the next generation of coding models. In essence, it's closing a strategic gap.

Skylar offered a sharp reframing: rather than calling Windsurf a "thin wrapper around GPT," it's better understood as an Integrator. OpenAI excels at building models but has a clear weakness in integrating those models into real development workflows. The gap between "wrapper" and "integrator" is precisely where that $3 billion valuation lives.

The Era of AI Vertical Integration Has Arrived

Kauthar placed this acquisition within a broader industry trend: as the model layer becomes increasingly commoditized, vertical AI copilots (for finance, law, science, medicine, and other domains) are becoming the new competitive battleground. Owning the UX layer is a strategic move — when model performance converges, ecosystems and developer tools become the real moat.

Kate further noted that this trend could lead to an Apple-style vertical integration model in AI: end-to-end control from hardware to model to application. Vertical integration has a deep history in tech — Apple built enormous user stickiness and margins by controlling the complete chain from chips (M-series) to OS (iOS/macOS) to the App Store. Similar trends are emerging in AI: NVIDIA is extending from GPU hardware to the CUDA software ecosystem and inference frameworks; Google controls the full stack from TPU chips to Gemini models to Search and Android applications; and OpenAI is now extending into the application layer through the Windsurf acquisition. The risk of this vertical integration is the potential for closed ecosystems and vendor lock-in. This is precisely why maintaining a robust open-source AI ecosystem is so critical — open-source models (including Meta's Llama series, the Hugging Face platform, and IBM's own Granite models) ensure enterprise customers have alternatives, avoiding lock-in to any single vendor's tech stack, while also driving technical transparency and innovation velocity across the industry.

Skylar closed with a historical analogy: thirty years ago, operating systems defined the computing experience; then it was browsers; now it's LLMs. But these differentiators ultimately converge. "Five years from now, looking back, you'll find that most of the options were pretty much the same."

Conclusion: The Paradigm Shift in AI in 2025

From IBM's Generative Computing to the hallucination struggles of reasoning models, to OpenAI's vertical integration ambitions, the AI industry in 2025 is undergoing a profound paradigm shift: from "whose model is bigger and stronger" to "who can deliver AI into real business scenarios more reliably and efficiently." The ceiling on model capabilities may keep rising, but the engineering challenges on the ground — reliability, maintainability, safety — are the real battleground that will determine whether AI can create genuine commercial value.

Key Takeaways

• IBM introduced the concept of "Generative Computing," advocating for software engineering best practices to restructure AI application development and replace fragile prompt engineering
• Hallucination rates in reasoning models are rising as capabilities improve (o4-mini > o3 > o1), rooted in a reinforcement learning incentive misalignment between verbose chain-of-thought and factual accuracy
• OpenAI's $3 billion acquisition of Windsurf is fundamentally about closing the gap in UI entry points and developer reach in the coding assistance space — upgrading from "wrapper" to "integrator"
• The AI industry is shifting from model competition toward vertical integration; as the model layer commoditizes, ecosystems and UX layers become the new moat
• IBM released the Granite 4 Mixture of Experts model (3B–20B parameters), taking a "small but precise" approach emphasizing efficiency and enterprise deployment effectiveness