Vibe Coding in Practice: AI Agents Complete Full Research Experiments in Minutes

From Writing Code to "Describing Requirements": A Paradigm Shift in Research

In traditional research workflows, a data analysis experiment — from data cleaning and model selection to code writing and result visualization — can take days or even weeks. With the maturation of AI coding agents like Codex and Claude Code, a new mode called Vibe Coding is emerging. Researchers no longer need to write code themselves; they simply describe their needs in natural language, and the AI automatically handles everything from data exploration to experimental modeling.

What is Vibe Coding? The concept was introduced by OpenAI co-founder Andrej Karpathy in early 2025 to describe a new programming paradigm driven by intuition and executed by AI. Unlike traditional programming, which requires developers to master syntax and debug logic precisely, Vibe Coding is centered on expressing intent in natural language and fully delegating code implementation to AI. In research contexts, this approach bridges the gap between domain expertise and software engineering skills — researchers typically excel at the former but are weaker in the latter.

A hands-on demo by Bilibili creator Qian Yu shows that this model is already capable of handling standardized research tasks. Using a heart disease prediction CSV dataset as an example, he demonstrated how an AI Agent can run a rigorous machine learning experiment end-to-end in just a few minutes.

The Test: How a Single CSV Automatically Becomes a Complete Experiment

The AI First "Understands" Your Data

The starting point is remarkably simple — just give Codex or Claude Code a single instruction: "Help me analyze the data in the current folder and see what research tasks we can do."

The AI Agent automatically runs a data inspection: identifying variable types, category distributions, and missing values. In this case, the dataset contained 303 samples and 13 features, along with a binary label for "disease present or not." Notably, the AI didn't just provide descriptive statistics — it also offered expert-level judgment: the sample size is small (303 records), which makes it difficult to support a high-impact publication, but it's suitable for building an interpretable clinical prediction model.

This "research-aware" feedback stems from the ReAct (Reasoning + Acting) architecture underlying tools like Codex and Claude Code — the model can not only generate code but also access the file system, run interpreters, catch errors, and self-correct, forming a complete loop of "perceive → plan → execute → feedback." This context awareness and autonomous error-correction capability is what sets AI Agents apart from ordinary code completion tools, enabling them to act like experienced collaborators who can provide input at the level of research design.

Rigorous Experimental Design and Automated Execution

Equally noteworthy is the AI's adherence to experimental standards. When asked to perform the heart disease classification task, the AI proactively adopted a standard research pipeline:

An 80/20 split, with 20% held out as a completely isolated test set
5-fold cross-validation on the 80% training set to effectively prevent data leakage
Automatic removal of duplicate records to ensure data quality
Comparison across multiple algorithms (Random Forest, SVM, Logistic Regression, Gradient Boosting, etc.)

Why strictly isolate the test set? This relates to the principle of preventing Data Leakage in machine learning — if information from the training set improperly bleeds into the validation or test phase, model performance will be systematically overestimated. This is one of the most common forms of experimental misconduct in academia. Best practice requires that the test set be "locked" before the experiment begins and used only for final evaluation. Preprocessing steps like standardization must also be fit only on the training fold and then transformed onto the validation fold; doing otherwise also constitutes leakage. The AI's automatic compliance with these standards reflects that its training data has internalized a wide body of machine learning best practices.

The AI then automatically wrote and ran code to complete model training. Final results showed that the Logistic Regression model achieved an AUC of 0.89 — a strong result.

How to interpret AUC 0.89? AUC (Area Under the ROC Curve) is a core metric for evaluating the overall performance of binary classifiers — 0.5 represents random guessing, while 1.0 is perfect classification. In clinical settings like heart disease prediction, AUC is more informative than simple accuracy, because datasets often have class imbalances and the costs of false negatives and false positives are not equal. An AUC of 0.89 with only 303 samples is a strong result, but it also confirms the AI's earlier caution: small-sample results should not be generalized rashly, and validation on a larger independent cohort is recommended before formal publication.

Experiment results are all here

Throughout the process, most of the waiting time came from the training phases of more computationally intensive algorithms like Random Forest and SVM.

Outputs: A Complete Research Experiment Folder

After the experiment, the folder that originally contained only the dataset now held a complete experiment directory, fully viewable in VSCode.

A new experiment folder has appeared

The outputs cover all the elements of a well-structured experiment:

Visualizations: label distribution plots, ROC curve comparisons across models, confusion matrices, and feature importance charts
Model weight files: saved weights for each trained algorithm
Experiment records: full logs, metric tables (AUC, sensitivity, specificity, etc.), and data split records

From code and models to charts and logs, a reproducible, open-sourceable experiment is completely organized. The engineering structure that researchers used to build manually is now generated by AI in one shot.

Flexible Extension: Add Analyses with a Single Line

The most disruptive aspect of Vibe Coding is the extremely low cost of iteration. Want to add a new algorithm (like XGBoost) or include more complex analyses? Just give the Agent a natural language instruction.

Adding interpretability analysis

For example, adding SHAP interpretability analysis, introducing a stronger model for comparison, or supplementing with an external validation set — all require just a single descriptive sentence.

What is SHAP? SHAP (SHapley Additive exPlanations) is currently the most widely used framework for explaining machine learning models, grounded in Shapley values from game theory. The core idea is: for each prediction, quantify how much each feature "contributed," with mathematical guarantees of consistency and fairness. In high-stakes domains like healthcare and finance, model interpretability is just as important as accuracy — regulators and clinicians need to understand "why the model made this decision," not just accept black-box outputs. SHAP makes originally opaque ensemble models like Random Forest and XGBoost auditable and is a common requirement in medical AI publications.

"One line of text, one experimental iteration" represents a quantum leap in efficiency compared to development cycles that previously took days or weeks.

This methodology isn't limited to tabular data — for more complex data types like images and text, AI Agents can also handle the full pipeline of analysis, modeling, and visualization.

Environment Setup: Key Points for Installing Codex and Claude Code

Configuring the environment for AI coding agents is straightforward. Here are the core steps:

General Prerequisites

Install Node.js: Both Codex and Claude Code depend on Node.js — just follow the prompts to complete installation
Open a terminal (PowerShell on Windows, the built-in terminal on Mac) and verify that Node.js is installed successfully by running the appropriate command

Installing the Agents

Codex: Run the corresponding command in your terminal to install, then verify the version number
Claude Code: Same process — install Node.js first, then install Claude Code in the terminal, log in, and run claude to start submitting requests in the conversation interface

Working in the terminal

Model Access

When logging in, you can use the official API or connect through a proxy, including mainstream models like the GPT-5 series. Once the code is generated, open it in VSCode or PyCharm to view and edit.

Conclusion: AI Is Reshaping the Technical Barriers to Research

This hands-on test clearly illustrates a trend: a large portion of standardized, engineering-heavy experimental work in research is being rapidly taken over by AI Agents. Researchers no longer need to wrestle with code syntax and engineering scaffolding — they can focus their energy on problem definition, result interpretation, and innovative design.

That said, Vibe Coding is not a silver bullet. It excels at executing standardized processes, but the scientific value and novelty of an experiment still require human judgment. For researchers who are not strong programmers, however, these tools dramatically lower the technical barrier to "doing experiments with data," enabling more people to focus on the science itself.