OpenAI Health AI Lead: Can ChatGPT Replace Doctors?

Hundreds of millions of people consult ChatGPT about health questions every week. Karan Singhal, head of health AI research at OpenAI, laid out the company's health AI strategy in detail on the a16z podcast—from how a general-purpose model becomes a medical expert, to the boundaries of trust, safety, and medical responsibility.

Why a general model is the right foundation for medical AI

The industry has long debated "general models vs. specialty models." Singhal, who trained medical specialty models in a previous role, has firsthand experience on both sides. His verdict: OpenAI is taking the route of "specialized training within a general model"—folding professional medical data into every stage of ChatGPT's training so the model enjoys both the scale benefits of a frontier model and domain-specific medical capability.

The underlying logic is a "bet on scale." A smaller medical specialty model trained on its own would quickly fall behind the frontier. Embedding medical evaluations and training data into a general model means the model rises with overall capability gains. Singhal says this is effectively "the best of both worlds."

This architectural decision also explains why the same model can help you write code, plan a trip, and interpret lab results. According to Singhal, over the past year, medical data has been continuously added to all ChatGPT models at every training stage.

HealthBench is a public benchmark released by OpenAI in May 2025 to measure how large language models perform in health scenarios. Rather than a traditional multiple-choice medical exam, it is a multi-turn conversational evaluation set written by practicing physicians from multiple countries for real clinical situations, containing roughly 5,000 conversations, each paired with a physician-written rubric. The scoring dimensions cover accuracy, safety, communication quality, instruction following, expression of uncertainty (i.e., "hedging"), and whether terminology depth is adjusted to the user's identity. An important design feature of HealthBench is "consensus scoring": each case has criteria independently written by multiple physicians, ultimately aggregated into a unified scoring framework to mitigate the bias of any single physician's judgment. The benchmark is openly available and has become an important yardstick for comparing medical AI capabilities across the industry.

The real demand behind 300 million weekly active users

OpenAI has publicly disclosed that more than 300 million users per week use ChatGPT for health and wellness questions, and that number is still growing rapidly. Singhal emphasizes that the vast majority are free users, and most are outside the United States—"making trustworthy, reliable, abundant health intelligence accessible to everyone for free or nearly free" is one of the core goals of OpenAI's health strategy.

These users are not asking about rare and complex conditions. They are mostly seeking help between visits: how to understand what their doctor said, how to prepare for the next appointment, how to follow up on next steps. The reality of the U.S. healthcare system is that more than 100 million people lack a regular primary care physician, and even those who have one may wait over 40 days for an appointment, with the actual visit often lasting only 15 minutes. AI fills precisely this gap.

Singhal shared the case that struck him most: Nathan Labenz's son developed severe fatigue and irritability, and AI helped them assess during a vacation that the situation was serious and seek care promptly, ultimately leading to a diagnosis of an aggressive leukemia. During subsequent treatment, AI also helped interpret lab reports, providing reference before specialists offered their explanations.

Trust mechanism: don't let the model say only what you want to hear

Podcast host Jay Rughani raised a sharp question: when patients pour out their symptoms to a large model, they often hope to hear "everything is fine," but medically they may need to hear something completely different. How does the model learn to tell the difference?

Singhal's answer points to a key design principle: safety first. In evaluation and training, OpenAI specifically sets up evaluation items that track these difficult scenarios—when the model should clearly state "this is not safe, you should see a doctor," it must hold firm even if the user doesn't want to hear it or is seeking alternative therapies. He says performance on these evaluations has shown "quite significant improvement" over time.

Another pillar of trust is layered adaptation of answers. According to Singhal, as early as the release of the HealthBench evaluation, there was already a dimension specifically examining whether the model could adjust terminology and technical depth based on whether the user is a layperson or a clinician. The model has since improved in both measurement and training on this front.

In areas where medical consensus remains contested, OpenAI requires the model to recognize its own uncertainty and speak candidly when evidence is insufficient—this is also what the "hedging" dimension of the public HealthBench evaluation examines.

A "consensus engine" of hundreds of physicians worldwide

An easily overlooked engineering challenge: medical experts themselves often disagree. Singhal says plainly that ask five doctors the same question and you may get eight different answers.

OpenAI's approach is to build a global physician network—hundreds of experts spanning dozens of specialties, languages, and countries of practice—using aggregated expert wisdom to build evaluation and training data rather than relying on the judgment of two or three doctors. The same methodology extends to mental health: OpenAI works with licensed mental health experts worldwide to define model behavior for tricky scenarios and reach consensus, and has built a mental health benchmark on that basis.

Interestingly, Singhal admits that disagreement in mental health is greater than in physical health, making an "industry-shared yardstick" all the more important.

AI medical responsibility and the regulatory boundary is key context for understanding OpenAI's positioning. In the United States, medical practice is governed by state licensure laws; diagnosis, prescribing, and administering tests must be performed by licensed clinicians, and AI itself is not qualified to practice medicine. The U.S. Food and Drug Administration (FDA) has a dedicated approval pathway for "medical device software," but general-purpose consumer health information tools typically fall outside it, with oversight relying more on Federal Trade Commission enforcement against false advertising and state consumer protection laws. OpenAI explicitly defines ChatGPT as "health information" rather than "medical care," precisely to avoid the risks of practicing medicine and product liability. Meanwhile, industry self-regulatory frameworks are taking shape—for example, the WHO's ethical guidance on medical AI emphasizes transparency, accountability, and human oversight. This "information vs. care" distinction also explains why OpenAI emphasizes integration with electronic health record vendors like Epic and with health systems—the last mile of clinical decision-making and responsibility still rests with licensed professionals.

Closing the 5-to-17-year knowledge gap

Singhal raises an underappreciated problem in the healthcare system: from the publication of biomedical papers, to updates in clinical guidelines, to ordinary doctors actually changing their practice behavior, this diffusion process takes 5 to 17 years depending on the study. He asks in return: "You don't want to practice medicine 5 to 17 years behind the medical frontier, do you?"

OpenAI's ambition is to compress this cycle to weeks, days, or even hours. The product direction for clinicians is to bring the latest biomedical literature into the 15-minute consultation in real time.

In the face of genuine medical uncertainty, OpenAI's strategy is twofold: let the model recognize and acknowledge uncertainty, while also accelerating scientific discovery itself. Singhal mentions the concept of "Move 37"—borrowing from AlphaGo's move that stunned human experts, he hopes AI in health can also offer judgments that human experts hadn't thought of but that later prove highly insightful.

The 5–17 year lag from paper to clinical practice is not a new finding but a long-standing conclusion of research in evidence-based medicine. In the early 2000s, the U.S. Institute of Medicine (IOM) noted in its report Crossing the Quality Chasm that it takes an average of about 17 years for known-effective treatments to be routinely applied in practice. Subsequent research (such as the review by Morris et al., 2011, in the Journal of the Royal Society of Medicine) revised this range to 5 to 17 years, depending on the type of intervention, the frequency of guideline updates, and the difficulty of changing physician behavior. The lag occurs at multiple stages: from paper publication to systematic review, from review to clinical guideline, from guideline to physician awareness, and from awareness to actual prescribing behavior. The potential value of AI lies in compressing the first three stages—real-time retrieval of the latest literature, dynamically updated guideline summaries, and decision support embedded in the consultation—but the final stage (behavior change) still depends on workflow integration and trust-building, which is why OpenAI emphasizes integration with systems like Epic.

The sharp drop in the cost of intelligence: lessons from GPT-6 Luna

Cost is the key variable determining whether this transformation can be broadly shared. According to Singhal, GPT-6 Luna, the cheapest model currently on OpenAI's API, outperforms 5.6 Sol—the strongest model from just one month earlier—on HealthBench Professional, at 34 times lower cost.

"For a given level of health intelligence, cost will plummet" is one of OpenAI's core bets. Singhal hopes this leads to the "commoditization" of health intelligence—not hoarding intelligence, but letting everyone use it. But he also acknowledges that health intelligence has not yet had a macro-level impact on the $5.3 trillion in total U.S. healthcare spending. Real cost compression requires integration, workflow redesign, and scaled deployment, with "more care, less paperwork" as the direction.

As for recent news about "payer AI and provider AI fighting each other and driving up spending," Singhal sees it as short-term pain: ideally, providers and payers could shake hands based on the real record of a visit, with AI assisting adjudication, rather than getting locked into an AI arms race.

HealthBench Professional is a higher-difficulty subset of the HealthBench system aimed at professional clinical scenarios, with conversations closer to real consultations between doctors or specialty referrals, requiring deeper medical reasoning and terminological precision from the model. Alongside it is a layered evaluation dimension that examines whether the model can adjust its expression to the user's identity. Comparing a new model's performance on this professional subset with the previous generation's strongest model is a direct way to measure "cost decline at equivalent intelligence level." Singhal's statement that "GPT-6 Luna surpasses the strongest model from a month ago and is 34 times cheaper" is precisely using this benchmark to quantify the cost-performance curve, supporting his argument for the "commoditization of health intelligence"—that is, pushing the price per unit of intelligence down to a level where it can be distributed free at scale without sacrificing professional quality.

Ecosystem partnerships and the "quarterback" positioning

OpenAI is clear that it will not take on all medical services itself. Singhal repeatedly emphasizes the concept of the "last mile": OpenAI's role is to make it easier for partners to use AI to support the patient's entire care journey.

Concrete moves include integration with Epic—doctors can call Epic within ChatGPT, and ChatGPT can be embedded in Epic workflows. UCSF is an early adopter, using it for chart review; Advent Health has deployed agents across its workspace and reports roughly an 80% reduction in administrative workflow costs.

For the personal assistant product Dots, OpenAI has set a safety baseline consistent with its other products, while exploring the move from "safety floor" to "raising the ceiling"—for example, proactively reminding users in chronic disease management to review medications and monitor changes in A1C indicators.

Singhal makes clear that OpenAI does health information, not medical care itself: it does not provide prescriptions, does not perform tests, and does not bear legal responsibility. But ChatGPT can serve as the "quarterback" in the patient's care journey, helping to navigate, coordinate, and smooth the last mile across the ecosystem.

The next five years: cheaper drugs, earlier intervention

Asked about the breakthroughs most likely to change human health in the next 5 to 10 years, Singhal offers two directions: first, AI-guided generation of new targets and new molecules; second, AI-guided prioritization and validation of clinical trials—there are currently far more drug candidates than trial slots, and AI can help screen them more efficiently.

At the clinical level, he expects the biggest macro variable in the next year or two to be proactive prevention: AI-assisted health management that keeps people from reaching the expensive surgery stage. As for the role of robots in care, he believes both technology and reimbursement integration will take longer.

The podcast host closes with a change already underway: when ChatGPT appears on a doctor's screen, should patients be glad or worried? Singhal observes that the "Overton window" is shifting in real time—more and more patients and doctors are finding that AI, as a "second check" for each side, is actually reassuring.

One-line Summary

OpenAI embeds medical training in a general model, serves 300 million health-query users weekly, and bets on plummeting costs and ecosystem collaboration.

Paragraph Summary

Karan Singhal, head of health AI research at OpenAI, explained the company's health AI strategy on the a16z podcast. The core approach is to embed medical data into every training stage of a general model rather than training a separate specialty model, balancing scale benefits with domain expertise. More than 300 million users per week consult ChatGPT about health questions, mostly free users outside the U.S., mainly filling the information gap between visits. The trust mechanism prioritizes safety: the model must insist on recommending medical care even when users want reassurance, and is trained on consensus evaluations built by hundreds of physicians worldwide. On cost, GPT-6 Luna surpasses the strongest model from a month earlier on HealthBench Professional while being 34 times cheaper. OpenAI positions itself as a health information provider and the "quarterback" of the care journey, not medical care itself, relying on ecosystem partners like Epic to complete the last mile.