Serving 650,000 Doctors: OpenEvidence's Architecture and Cost Optimization Journey

When a piece of software directly influences physicians' clinical decisions, the reliability and efficiency of its technical architecture is no longer just an engineering concern — it's a matter of medical safety. OpenEvidence provides doctors with a medical evidence search service grounded in peer-reviewed literature. Its engineering team achieved rapid deployment, elastic scaling, and substantial cost optimization on the Vercel platform, supporting a vast user base spanning more than 10,000 hospitals and medical centers across the United States.

Technical Challenges in a Medical Context: Unacceptable Errors and Downtime

OpenEvidence is positioned to help doctors quickly find medical evidence with cited sources at the point of care. Whether it's verifying dosages during an ICU round or handling a cross-specialty case without a specialist present, doctors need a stable, fast, and trustworthy tool.

Engineering lead Kush Khosla put it bluntly: "Honestly, life-or-death is not an exaggeration. We're dealing with emergency room scenarios where they really need to know what's happening right now. Uptime is critical for us."

Fellow engineering lead Andy Yoon added that many doctors are used to outdated software, but a more modern experience must meet their expectations for speed, stability, and information credibility. Although the team is small, it needs a mechanism to test updates before they reach production and to roll back quickly when something goes wrong.

This forms the underlying logic behind OpenEvidence's technology choices: the deployment process must be simple enough, rollbacks must be fast enough, and performance must be reliable enough.

Small Team, Massive Traffic: The Next.js + Vercel Deployment Workflow

OpenEvidence's physician-facing web app is built on the Next.js framework. The team's workflow is deeply integrated with Vercel's preview deployment capabilities: after engineers push changes to a development branch, Vercel automatically builds and deploys a preview version, generating a link for live demos and colleague testing; pushing to the production branch triggers a production build and deployment; and if a release needs to be reverted, the team can roll back to a previous deployment version.

Andy explained how well this toolchain fits the team's composition: "When I joined, I was pretty much the only person on the team with a real frontend background. Most people did Python and machine learning." It's precisely this staffing structure that made Vercel's managed deployment capabilities a key enabler for the team's growth.

Kush described the psychological impact of this workflow from a responsibility standpoint: "If I push a frontend change, I know that within about five minutes, 650,000 doctors will see that change. Vercel picks it up and deploys it to everyone directly. Every time I push code, that's a daily reminder."

Zero Extra Configuration Under Traffic Surges

OpenEvidence unexpectedly went viral on TikTok, racking up roughly two million views in a week. When Andy saw the message on Slack, traffic was already climbing. He checked the logs and found that response times remained fast, error rates were near zero, and no additional capacity configuration was needed — he then went right back to his normal work.

Andy said: "Vercel completely kept up with the growth in usage. We never went down due to capacity issues, and we never needed to configure anything extra. Being able to trust that it's just there — to the point where we barely have to think about it — is remarkable."

This elasticity not only handles sudden spikes but also supports sustained growth. Kush noted that when the number of doctors grew fivefold, he never had to think about what happened to page load or first paint times.

Next.js is an open-source frontend framework based on React that supports server-side rendering (SSR), static site generation (SSG), and incremental static regeneration (ISR), with built-in capabilities such as file-system routing, API routes, and image optimization. For a web app like OpenEvidence that needs to present medical evidence quickly while ensuring fast first-screen loading, Next.js's SSR can complete data fetching and page rendering on the server, reducing client-side wait times; its routing and build conventions also make it easier for team members with limited frontend backgrounds to get up to speed. Vercel is the creator of Next.js, and the platform has deep integration with Next.js for builds, preview deployments, and edge networking — an important backdrop for the team's choice of this combination.

Preview Deployment is a core capability of modern frontend hosting platforms: every time code is pushed to a non-production branch, the platform automatically builds and generates a temporary deployment environment with its own URL. It upgrades code review from "looking at diffs" to "looking at running results," allowing product, design, and clinical advisors to validate interactions on a real link without having to set up a local environment. For a team like OpenEvidence with limited frontend manpower that also needs doctors to participate in trial feedback, preview deployments bring the cost of testing and demos down to nearly zero. The production branch corresponds to the official deployment, and historical deployment versions can be rolled back at any time, forming a closed loop of "preview — validate — release — rollback."

Cost Optimization: Fluid Compute and a 90% Reduction in Server-Side Costs

As usage grew, VP of Engineering Micah Smith began focusing on compute costs. The team enabled Fluid compute for functions on Vercel, with remarkable results.

According to Micah: "We reduced our server-side costs by 90% while maintaining the same performance. Even at 1,000x scale, Vercel accounts for less than 5% of our overall infrastructure spend."

The core mechanism of Fluid compute is to let multiple requests share a single running function instance, so that while one request is waiting for a database or AI backend response, the instance can handle other work. Combined with the Active CPU pricing model, this waiting time incurs no CPU charges, though memory is still billed while the request is in progress.

For a service like OpenEvidence that frequently calls AI backends and databases, this billing model aligns closely with its workload characteristics.

Fluid compute is a function compute model introduced by Vercel, addressing the cold-start and resource-idling problems of traditional serverless functions where "one instance handles only one request at a time." It allows the same running instance to be reused across multiple concurrent requests; when a request is in I/O wait (such as calling a database or an AI inference backend), the instance can move on to handle other requests, thereby improving per-instance utilization and reducing cold starts. The accompanying Active CPU pricing bills only for the time a request actually occupies CPU — no CPU charges during I/O waits — but memory is billed for the duration of the request's existence. This model is particularly well suited to server-side workloads like OpenEvidence's, which are dominated by waiting for external AI and database responses.

Extending to Patients: Isolated Deployment of Independent Projects

OpenEvidence's first patient-facing app, Take Home, allows doctors to translate clinical answers into plain language and share them with patients, who can log in to view and save the evidence. Clinical trial matching is also among the app's planned directions.

The team followed the pattern established by dozens of early proof-of-concept projects: each app is deployed as an independent Vercel project with a custom domain. Take Home likewise has its own project and domain, so engineers can update the patient experience without deploying other apps.

Kush explained the logic behind continuing this pattern: "We're very familiar with Next and Vercel, and we get independent deployments that run seamlessly and are isolated from each other. There are many areas we still want to enter, so we just make it a new Vercel project."

Through Take Home, evidence that originally helped doctors during consultations can become material patients continue to consult after leaving the exam room.

A Vercel Project is the deployment and configuration unit on the platform; each project is bound to a code repository and has its own environment variables, build configuration, domain, and deployment history. Splitting different apps into independent projects means a single release only affects the corresponding app, with the blast radius and rollback scope confined to a single product; at the same time, each project can scale independently based on its own traffic, preventing patient-side traffic fluctuations from affecting critical physician-side services. For a small team whose product line is still expanding, this "a new direction is a new project" pattern reduces the coupling and operational burden of iterating on multiple apps in parallel.

The Medical Value Behind Engineering Efficiency

The OpenEvidence case demonstrates a clear path: for engineering teams with limited resources but heavy responsibilities, choosing a managed platform that matches their tech stack enables rapid iteration and cost control without sacrificing reliability. According to its official information, the platform supported more than 20 million clinical consultations in January 2026, and in 2025 more than 100 million U.S. patients received care from doctors using OpenEvidence.

When deployment takes minutes, when traffic surges require no manual intervention, and when infrastructure costs drop to single-digit percentages, a small team can refocus on what truly matters — helping doctors find trustworthy medical evidence at critical moments.

One-line Summary

OpenEvidence uses Next.js and Vercel to achieve minute-level deployments, elastic surge handling, and a 90% reduction in server-side costs.

Paragraph Summary

OpenEvidence provides doctors with medical evidence search grounded in peer-reviewed literature, serving more than 10,000 hospitals across the U.S., where downtime and errors are unacceptable. The team built its physician-facing app on Next.js and deeply integrated Vercel's preview and production deployment workflows: changes go live in about five minutes, rollbacks are fast, and the setup suits a small team with limited frontend manpower. An unexpected TikTok viral moment brought a traffic surge of roughly two million views in a week, yet the platform maintained fast responses and near-zero error rates without any extra capacity configuration. After enabling Fluid compute and Active CPU pricing, server-side costs dropped by 90%, and even at 1,000x scale, Vercel still accounts for less than 5% of infrastructure spend. The patient-facing app Take Home is deployed as an independent Vercel project, enabling isolated updates and independent iteration.