AI Makes Work Better, But Not Necessarily Workers: Lessons from a Three-Month Patent Lawyer Experiment

When AI can consistently improve work quality, does it also cultivate better professionals? A research team including David Autor and Tanya Rodchenko from Google DeepMind conducted a three-month randomized controlled trial that offers a thought-provoking answer: AI's empowering effects vary by seniority — senior patent lawyers used AI to sharpen their judgment, while junior lawyers showed a troubling divergence in skill development.

Published by the National Bureau of Economic Research (NBER), the study is titled "Does AI Assistance Enhance or Erode Professional Competence? Evidence from a Three-Month Field Experiment in Patent Drafting." It touches on a question central to the future of the entire knowledge workforce: could the tools that make work better today prevent the formation of tomorrow's experts?

Experimental Design: Embedding AI into Real Professional Workflows

Understanding the true impact of AI on professional competence requires three elements that are rarely present simultaneously: sustained AI use over months rather than hours of daily work, credible assessment of independent judgment when AI is unavailable, and blind scoring by domain experts. This study was designed precisely to fill that gap.

The research team recruited 133 lawyers from 11 intellectual property firms with regular business relationships with Google, and randomly assigned a then-unreleased Google Labs patent drafting assistant (since integrated into Gemini Notebook). Two-thirds of lawyers at each firm were placed in the "treatment group" and given early access; the rest formed the "control group," receiving AI-use training but being prohibited from using the tool during the three-month study period.

Assessments were conducted twice. Ten days after the experiment began, all participants received inventor materials for a fictional invention and were asked to draft a patent; 90 days later, they repeated the exercise with a different set of simulated materials. All submissions were scored by third-party patent experts across five dimensions: enforceability, accuracy, strategic ambiguity (tactical claim scope definition), completeness, and clarity.

---

Randomized controlled trials (RCTs) are the gold standard for evaluating intervention effects, but are extremely difficult to implement in professional services — law firms are generally unwilling to let outside researchers randomly assign tools, restrict lawyers from using specific technologies, or maintain long-term control groups under billing pressure. Most prior AI effectiveness research has relied on laboratory tasks (completed within hours) or observational data, making it difficult to separate "tool effects" from "user selection bias." By partnering with 11 firms and framing the AI tool as an "early access privilege" rather than a restriction, this study minimized control group resistance. It stands as one of the longest-running randomized controlled experiments on AI in real knowledge-work professional settings to date.

"Patent profanity" is an industry term in patent practice referring to language in patent text that opposing counsel or courts can cite to narrow the scope of patent protection — for example, using limiting words like "only," "solely," or "must," or overclaiming the novelty of an invention. Identifying and removing such language is a hallmark of senior patent lawyers' core judgment, and was a primary focus of this study's redlining task. This type of judgment relies heavily on accumulated experience in patent litigation and is difficult for AI to directly replicate.

AI Does Improve Work — But the Gains Come from Raising the Floor

The results were as expected: AI brought significant improvements in drafting quality. At day 10, lawyers with AI tools scored 0.34 standard deviations higher — equivalent to moving up roughly 10 percentile points in the control group's ranking. By day 90, this improvement grew to 0.38 standard deviations (about 11 percentile points). The gains were remarkably consistent across all quality dimensions.

But an intriguing pattern emerged: better performance came from a decline in low scores and an increase in middling scores, with no change in the proportion of excellent scores. In other words, AI lifted results from the "worst" tier to the "below-average" and "middle" tiers, without producing more "top-tier" work. This "floor-raising without ceiling-breaking" effect was especially pronounced among junior lawyers — their quality improvements came alongside significant time savings, for instance completing the day-10 task an average of 18 minutes faster than the control group (which averaged 124 minutes).

When AI Is Taken Away: The Truth About Judgment Emerges

Patent lawyers do more than draft — they also review peers' work and flag potentially fatal flaws that could invalidate a patent in court (known in the industry as "patent profanity"), such as overclaiming an invention's novelty or scope. So the research team added a "redlining" task at day 90: participants were asked to manually mark up and correct an existing patent filled with substantive and stylistic errors.

Critically, no one was allowed to use AI for this task. It directly measured how much lawyers had developed their own skills — the kind of professional judgment that senior lawyers rely on daily and that often cannot be delegated to AI.

The results were sobering. Once AI was removed, the floor-raising effect from the drafting task disappeared. Lawyers with AI access still outperformed the control group on the unaided redlining task by 0.32 standard deviations (about 9 percentile points), but this advantage was entirely driven by senior lawyers — who led the control group by 0.45 standard deviations (about 13 percentile points), while junior lawyers showed no discernible improvement.

More alarming was the "polarization" in junior lawyers' scores: more extremely low scores, fewer mediocre scores, more good scores — but still no more excellent ones. This means that while AI can help newcomers deliver "acceptable" work in some scenarios, that machine-assisted performance has not translated into the accelerated accumulation of the professional competence that truly makes senior practitioners valuable.

The Senior–Junior Divide: Who Is Actually Learning?

Qualitative analysis revealed deeper differences in how lawyers of different seniority levels interact with AI.

Regardless of whether they were in the treatment or control group, junior lawyers' submissions followed rigid formalism: they worked sequentially from start to finish, often spending most of their time polishing low-stakes introductory language before ever reaching the core claims. They focused on surface-level synonym substitution rather than substantive scope optimization. Even when they identified serious flaws, they often left only diagnostic comments without executing the corrections. Since this "diagnose but don't fix" posture was equally common among junior lawyers in the control group who didn't use AI, it represents a baseline competence gap among junior practitioners — three months of relying on AI to execute rewrites did not close that gap.

Senior lawyers, by contrast, consistently benefited from AI exposure. Senior lawyers in the study spent more time on the redlining task; they skipped low-risk language, rebuilt claims from scratch, stripped out phrasing that might inadvertently narrow legal rights or limit protection scope, and paired many edits with explicit legal rationale. In follow-up interviews, senior lawyers described AI output as a "logic auditor" rather than a finished product: it loosened their attachment to existing text, forced them to clearly articulate the "why" and "how" of structural changes, and in doing so activated and honed their foundational professional competence.

---

The cognitive science concept of "offloading" may explain junior lawyers' difficulties: when an external tool absorbs the cognitive load, the individual's neural circuits for that task lack the opportunity to be activated and reinforced. This aligns with the "desirable difficulty" theory of learning — appropriate struggle and error feedback are necessary conditions for internalizing skills. Senior lawyers can extract "logic auditor"-style learning gains from AI precisely because they already have sufficient knowledge frameworks to evaluate whether AI output is right or wrong, allowing them to convert AI-generated challenges into active reflection. Junior lawyers have not yet built that framework, so AI's floor-raising effect actually allows them to bypass the most critical error–reflection–correction cycles.

Implications: Don't Let Today's Tools Block Tomorrow's Experts

The study's core insight is this: improving current performance and building lasting professional judgment are two different goals — and for junior professionals, they may even be in conflict.

Foundational professional competence may be the critical missing link: only by possessing it can repeated practice with AI assistance translate into mature professional judgment over a career. And that judgment remains important even as model capabilities continue to advance — lawyers must still exercise judgment in interactions with judges, clients, and colleagues, in contexts where they cannot rely on AI tools.

The researchers acknowledged the experiment's limitations: the sample of patent lawyers is limited (reflecting the high billing rates of top professionals), and the three-month observation window is quite short relative to the years needed to develop lasting professional competence. Moreover, the rapid evolution of model capabilities, AI proficiency, and AI penetration in the legal field over the past year means that if the experiment were repeated today, results might differ.

But one clear conclusion stands: to understand the true impact of AI assistance on professional skills, we need testing methods that can distinguish between "effects of using the tool" and "user performance without assistance." A key challenge in the current AI wave is ensuring that tools designed to produce better work don't prevent junior professionals from growing into the experts we'll need in the future.

This finding creates a subtle tension with the long-term "skill premium" trend in economics. Decades of research — including Autor's own earlier work — suggests that automation tends to replace routine cognitive tasks and enhance the relative productivity of high-skilled workers, thereby expanding the skill premium. But this study implies that if AI simultaneously hinders newcomers' path toward becoming high-skilled practitioners, the long-term consequences may extend beyond widening income gaps to a shrinking reservoir of judgment across entire professions. When today's senior experts retire, their successors may lack the foundational competence needed to handle complex situations without AI assistance. This is particularly worth watching in high-stakes professional fields like medicine, law, and engineering.