AI + Python Reimagines Empirical Research: From Stata to Fully Automated Paper Generation

From Stata to Python: A Paradigm Shift in Empirical Research

In the field of econometrics and social science empirical research, tooling has historically lagged behind broader technological progress. A presenter from a Peking University–Stanford joint training program put it bluntly: the tool stacks used by top empirical researchers are undergoing a fundamental transformation.

This shift has deep historical roots. The generational succession from SPSS to Stata to Python mirrors a profound evolution in econometric methodology. SPSS, launched in 1968, long dominated statistical analysis in the social sciences. Stata, introduced in 1985, established its stronghold in economics thanks to its concise command syntax and powerful panel data capabilities. Each tool transition reflected a leap in methodological complexity — as quasi-experimental methods like Difference-in-Differences (DID), Regression Discontinuity Design (RDD), and Instrumental Variables (IV) became standard, the ceiling of legacy tools grew ever more visible. Python's rise coincided with the critical inflection point where causal inference and machine learning began to converge, giving birth to methods like Synthetic Control and Double Machine Learning (Double ML) — approaches architecturally designed to integrate deeply with the Python ecosystem.

The presenter described the typical setup at top U.S. universities: VS Code paired with Claude Code and Codex, or RStudio, alongside the emerging open-source tool Posit (an R-ecosystem tool with Python compatibility). These three configurations have become near-essential for contemporary empirical practitioners.

Equally notable is the generational divide. The presenter offered a vivid analogy: "It's like when you started school, your professor was using SPSS, you were using Stata, and now your students are already using Python. What young people adopt represents the future." This isn't simply a tool swap — it's a complete paradigm shift in how research is done.

Why Python Has Become the Empirical Researcher's Tool of Choice

The presenter's advocacy for Python is grounded in solid technical reasoning. He argued that Python's combined strengths in data processing, algorithms, APIs, modeling, and computational efficiency have "far surpassed Stata, R, and SPSS" across the board.

The core logic is this — Python synthesizes the best capabilities of every major tool. It absorbs SQL's database strengths, and the widely respected Pandas library was actually developed by someone with an economics background. Pandas was created by Wes McKinney in 2008 while he was at the hedge fund AQR Capital Management, motivated directly by the pain points of financial and economic data analysis — handling heterogeneous tabular data with time-based indexes. McKinney's background in mathematics and statistics at MIT meant that Pandas' DataFrame design philosophy aligned naturally with econometric thinking: variables as columns, observations as rows, with built-in support for missing values and grouped aggregation. The presenter emphasized this point: "Someone with an economics background truly understands what data people need — computer scientists don't treat data as their main focus."

Even more critical is Python's dominance in machine learning. The presenter stated plainly that Python "crushes R" in this domain: "Machine learning algorithms are written in Python by default, including neural networks. On the day ChatGPT and tools like it were born, the code behind them was all Python." Many cutting-edge causal inference models — such as Causal Forest — are essentially unimplementable in Stata. Causal Forest, formally proposed by Susan Athey and Stefan Wager in 2019, identifies individual-level Conditional Average Treatment Effects (CATE) under high-dimensional covariates, relying on specialized Python libraries like Microsoft Research's EconML. Double ML, also from Athey's team, eliminates regularization bias via cross-fitting and represents the frontier of the "machine learning + causal inference" fusion paradigm. Python is the natural home for these ML-driven causal inference methods, which are increasingly the dominant identification strategies in top-journal publications.

An Open-Source Toolkit: 1,100+ Functions, Free Forever

The presenter introduced a Python package developed by his team, clearly inspired by two landmark projects: Pandas (data processing) and scikit-learn (the design philosophy of completing modeling tasks in just a few lines of code).

The package now includes over 1,100 functions — more than 200 of them causal inference methods — and is committed to being "permanently free and open-source." Theoretically, the team's design is built around the potential outcomes (counterfactual) framework. The core idea of the Rubin Causal Model is that for individual i, there exists a potential outcome Y_i(1) under treatment and Y_i(0) under control; the causal effect is defined as their difference, but they can never both be observed — this is the "fundamental problem of causal inference." The team drew on Angrist's classic textbook Mostly Harmless Econometrics, which systematically frames this approach and elevates IV, DID, RDD, and other identification strategies from statistical correlation to causal interpretation — work that earned Angrist the 2021 Nobel Prize in Economics.

In terms of design philosophy, the team intentionally preserved researchers' existing habits: regression equation syntax mimics R's style to reduce migration costs, while machine learning modules borrow scikit-learn's minimalism — analyses that would traditionally require extensive coding can now be done in just a few lines.

The Skill Mechanism: Crystallizing Research Experience into Reusable Automation Modules

One of the most thought-provoking concepts introduced was Skills. The team encapsulates core models as Skills, enabling them to automatically complete an entire empirical analysis pipeline end-to-end.

The value of Skills lies in preserving uniquely human experience and workflows. The presenter gave an example: if you're a university professor and your students need to follow specific departmental traditions and formatting requirements for their theses, you can encapsulate all of that into a Skill for students to invoke. "Skills are best used to capture human-specific experience and processes — and they can massively accelerate your research at the same time."

At its core, this is an approach to knowledge asset creation — transforming tacit domain expertise into explicit, executable automation modules that allow accumulated research intelligence to be reused at scale.

The Stata Code Plugin: AI Directly Drives Stata with Automatic Error Correction

The team's VS Code plugin, Stata Code, includes a key capability: connecting AI to Stata via MCP (Model Context Protocol).

MCP (Model Context Protocol) is an open standard protocol released by Anthropic in November 2024, designed to solve interoperability between AI models and external tools. Before MCP, every AI application required separate integration work for each tool, creating an "M×N" complexity problem. MCP standardizes this into an "M+N" architecture: AI models act as clients, various tools act as MCP servers, and they communicate through a unified protocol.

In the Stata Code plugin, this mechanism works as follows: the user writes code in VS Code, the AI sends the code to a background Stata instance, Stata returns the result to the AI — all without manually opening Stata. The presenter noted that some competing tools require launching Stata first and then a server separately, calling it "too cumbersome." Stata Code condenses these advantages into a single tool.

The more critical design principle is automatic iterative error correction: if code throws an error, the AI automatically revises it and resends it to Stata until it runs successfully. "What I want is a 100% runnable result — if there's an error, you fix it yourself." This hands the entire debugging iteration loop to AI, dramatically improving research efficiency.

Notably, the solution supports swapping in domestic Chinese large models (such as MiniMax, Zhipu, Kimi, etc.), with API keys and configuration details encapsulated and hidden — making it accessible even for complete beginners.

Paper Workflow: A Fully Automated Pipeline from Raw Data to Draft Paper

The most striking demonstration was the Paper Workflow Skill. The presenter provided only a fictional CFPS panel dataset (studying the effect of internet usage on personal income) and a single Prompt — and the AI produced a structurally complete economics paper.

The generated paper followed rigorous academic structure, covering data and variable descriptions, identification strategy, baseline regression, heterogeneity analysis, robustness checks, and mechanism analysis — with the final output in LaTeX format, convertible to PDF. Deliverables included properly formatted three-line tables, forest plots, and coefficient display figures. "All the formulas were done by AI — nothing was manually changed."

The presenter was particularly enthusiastic about LaTeX over Word: "You're best off writing your paper in LaTeX, because AI can edit it very conveniently. Word's formatting is still quite problematic." This is also the broadly accepted approach among top U.S. empirical researchers. LaTeX, developed by Leslie Lamport in 1984 on top of Donald Knuth's TeX system, is essentially a "what you think is what you get" document typesetting language, and is the de facto submission standard for journals like the American Economic Review and Econometrica. For AI-assisted writing, LaTeX's plain-text nature is crucial — AI can precisely edit any section, table, or formula without the risk of format corruption that comes with manipulating Word's .docx format. This makes LaTeX the natural infrastructure for a "human-AI collaborative paper writing" workflow.

AI's Limitations and the Irreplaceable Core Value of the Researcher

The presenter maintained a rare clarity of judgment. He candidly acknowledged that Agents do fail — in literature citation, "around 20% of references may still come out wrong," requiring manual verification of citation formats. Domestically in China, Claude Code cannot access the internet to retrieve literature and can only reference PDFs from a local folder — a real-world limitation for now.

More importantly, his take on human value was clear: "AI only surpasses humans in aggregate — there are still things it can't fully surpass." He emphasized that the paper he demonstrated required two rounds of Prompt refinement before reaching a satisfactory result. "If you don't have the expertise, and AI keeps running, how do you judge whether the output is good? The role and value of humans is still there — you just don't need to do the repetitive work anymore. You can focus on oversight, creativity, and innovation."

This may be the most thought-provoking takeaway from the entire presentation: AI liberates researchers from mechanical labor, but a researcher's professional judgment, domain insight, and academic sensibility remain irreplaceable core competencies.

Key Takeaways