Using AI Agents to Write Research Experiment Logs: A Complete Methodology and Workflow

In scientific research, experiment logging is often the most overlooked yet most critical component. Based on the content from Section 4 of Chapter 0 in the Bilibili series "AI-Accelerated Research Workflow," this article systematically outlines how to use AI Agents to establish a Research Log that spans an entire project, turning experiment records into a powerful tool for scientific reflection and decision-making.

Why You Need a Unified Research Log

Information in a research project typically falls into three categories: experiment records, research ideas, and changes to methods and code. Splitting these into three separate documents creates two obvious problems. First, it leads to information loss and fragmentation—experiments, ideas, and code changes should be interconnected, and separating them breaks this context. Second, when using an AI Agent, feeding three separate documents results in excessive token consumption and low efficiency.

Tokens here refer to the basic unit of text processing for large language models. One Chinese character corresponds to roughly 1.5–2 tokens, and one English word corresponds to about 1–1.5 tokens. Current mainstream models like GPT-4o and Claude 3.5 offer context windows of 128K or even larger, but token consumption directly affects API costs and inference speed. Feeding three separate documents to the AI means redundantly loading background information with every interaction. Merging everything into a single document allows the AI to load the full context once for analysis and reasoning, potentially saving several times the API costs over a long-term project.

The better approach is to consolidate all three types of information into a single Markdown document, keeping it within a reasonable context length. Markdown is a lightweight markup language written in plain text that uses simple symbols (such as # for headings, - for lists, and ** for bold) to generate structured documents. It's particularly well-suited for research records because: plain text is natively compatible with version control systems like Git, making every change traceable; virtually all AI models can perfectly parse Markdown structure, helping Agents understand document hierarchy; and files can be opened in any text editor without depending on specific software, ensuring long-term readability.

The core value of this approach is: when you need the AI to reflect on experimental issues, it has access to the complete context. Based on coherent historical records, the AI can compare daily experiment results, help you analyze problems, and even design future experiment plans and daily workflows.

This document is essentially a "research diary," maintained continuously from project inception to completion, recording the full information flow of the entire research process.

The Skeleton Structure of an Experiment Log

A complete experiment log should have a clear hierarchical structure:

Project Starting Point: Define the Research Position

The document should begin by clearly defining the project's overall position, including the current research question, research scope, and objectives to be achieved. This section serves as the anchor for the entire Research Log, providing background context for all subsequent experiments.

Daily Records Organized by Date

Each day's records are organized by date, containing one or more experiments. Daily records follow a fixed framework:

• Today's Goals and Plan: Set the day's objectives before starting experiments in the morning
• Experiment Execution and Changes: Multiple experiments are typically conducted throughout the day, appended sequentially as experiment001, experiment002, etc.
• End-of-Day Summary: Wrap up at the end of the day or before leaving work

Project Wrap-Up: Overall Retrospective

When all experiments are completed, conduct an overall project summary. There's an important principle here: if certain sections have missing information, leave them blank—never let the AI fabricate content to fill the gaps.

Operational Workflow During Project Progression

Using a typical experiment day as an example, the progression logic of the Research Log becomes clear. Suppose you start experiments at 9 AM—first write down the day's plan in the document, then run the first experiment (experiment001) and check whether the output is anomalous.

If an anomaly is discovered in the afternoon, the correct approach is to create experiment002 and reference experiment001, rather than deleting or overwriting experiment001. This is a core principle repeatedly emphasized throughout the methodology—failed experiment records are equally valuable. In the evening, write the day's interpretation and summary, explain why new evidence changed your judgment, and outline what to do tomorrow.

Four Components of the Daily Wrap-Up Summary

Before the end of each day, divide the summary into four distinct sections:

1. What Happened: What actually occurred in today's experiments, and what was accomplished
2. Interpretation: Explain the results (this part requires the researcher's personal verification)
3. Issues and Corrections: What problems exist in today's experiments, and what needs to be fixed
4. Next Steps: What to do tomorrow, what conditions need to be met, and where to start

These four sections combine to form the closing portion of a day's record, completing the day's documentation.

Structured Thinking Framework Before Experiments

Before conducting an experiment, a structured thinking process can significantly improve experiment quality. Ask yourself several key questions before starting:

• What question is this experiment trying to answer?
• Controlled Variables: What conditions are being changed in this experiment, and are all other conditions held constant?
• Success Criteria: What results are expected, and what constitutes a valid answer?
• Failure Handling: What happens if the experiment fails—should we stop?

The concept of "controlled variables" follows the most fundamental principle in experimental science—the Controlled Variable Method. Its core idea is to change only one variable (the independent variable) in a single experiment while keeping all other conditions constant, thereby establishing the causal relationship between that variable and the experimental outcome. In machine learning and deep learning experiments, this means adjusting only one hyperparameter at a time (such as learning rate, batch size, or a specific layer of the model architecture) to accurately attribute the source of performance changes. If multiple variables are changed simultaneously, even if results improve, it's impossible to determine which change was responsible—a problem known in research as "confounding variables." Making this classic scientific methodology explicit as an actionable checklist is precisely the value of the structured thinking framework.

Regarding failure handling, there's a clear rule: if an experiment fails, record it faithfully as-is—do not attempt to overwrite this experiment with a new approach. If experiment one fails, proceed to experiment two, syncing experiment one's conditions but without overwriting experiment one's records and outputs. This "append-only, never overwrite" principle runs throughout the entire methodology.

How to Write Records After Completing an Experiment

After an experiment is complete, changes in experimental conditions and code modifications should be written in the same record entry, explaining why changes were made and what was actually changed.

For computer science experiments, here's an efficient technique: if code files have been modified, first make a git commit, then have the Agent read the git change log so it can automatically summarize what code was changed and what effects were achieved.

Git is currently the most widely used distributed version control system, originally developed by Linus Torvalds to manage the Linux kernel codebase. In research settings, Git can manage not only code but also data processing scripts, configuration files, and even every change to paper LaTeX source files. Each git commit generates a unique hash value recording who modified what content and when. git diff shows exact differences between two versions, while git log displays the complete modification history. Having an Agent read git records to automatically generate code change summaries essentially leverages Git's native change-tracking capability, transforming machine-readable diff information into human-understandable natural language descriptions. This ensures record accuracy while reducing the burden of manual documentation.

For experimental disciplines like biology, the Agent will proactively ask about detailed circumstances, which the researcher fills in personally. The complete closed loop for daily experiment records is: formulate experiment plan and objectives → conduct the experiment → verify facts → record code changes with git (or manually log experimental details) → human judgment of results → proceed to the next experiment. The next experiment must follow from the original records, which significantly reduces overhead in terms of computational resources and costs.

Division of Labor Between AI Agents and Researchers

The most commendable aspect of this methodology is its clear delineation of responsibilities between AI Agents and human researchers.

An AI Agent refers to an AI system with autonomous planning, tool invocation, and environmental interaction capabilities, distinguishing it from simple conversational large models. A typical Agent can decompose complex tasks into sub-steps, invoke external tools (such as file read/write, code execution, and web search), and adjust subsequent behavior based on execution feedback. Current mainstream Agent frameworks include AutoGPT, LangChain Agent, Claude's Computer Use, and others. In research settings, Agents can automatically read experiment logs, analyze data files, and propose hypothetical questions, but their core limitation is the lack of real-world physical perception—they cannot judge whether experimental data is authentic or whether conclusions are reliable. This is precisely why this article emphasizes human-machine division of labor.

What the Agent Is Responsible For

The Agent can collect background information on problems, organize file structures, flag missing information, and audit contradictions. Its core responsibility is "asking the right questions"—guiding researchers through questions to supplement factual information.

What the Researcher Is Responsible For

The researcher is responsible for summarizing each experiment record, planning next steps, and approving plans. It is strongly recommended not to use fully automated mode—every experimental step should be confirmed by a human, which is safer and more reliable.

Risks to Watch Out for When Using Agents

When using Agents, watch out for several key issues:

• Whether the Agent has generated sensitive information
• Whether it has overwritten historical records
• Whether the factual sources in its suggestions are accurate
• Whether evidence from different experiments conflicts
• Whether it has executed tasks beyond its authorized scope

The most critical warning is: never let the Agent make its own fabricated content look real. This warning points directly to a core technical challenge in AI—AI Hallucination. AI hallucination refers to large language models generating content that appears plausible but is actually incorrect or unfounded. Its root cause lies in the generation mechanism of language models: they predict the next most probable token based on statistical probability, rather than based on understanding and verifying facts. In the context of research records, the risk of AI hallucination is particularly severe—an Agent might fabricate nonexistent experimental data, invent citations, or present speculative reasoning as definitive conclusions. This can not only mislead researchers' judgment but may also constitute academic misconduct. In recent years, multiple paper retractions have been directly linked to AI-generated false citations.

Therefore, never write "possible causes" as conclusions, and never equate "code was modified" with "results were produced"—code changes alone do not constitute verified success.

The correct approach is: have the Agent flag items pending confirmation, pose verification questions, and after it has asked all the necessary questions, have the researcher input the real information. This is the proper way to do human-AI collaboration.

Conclusion: Start with a research_log.md

The core deliverable of this lesson is actually quite simple—create a research_log.md, write your first experiment record, and establish both daily log conventions and per-experiment log conventions for all future experiments.

But behind this seemingly simple document lies a rigorous philosophy of research documentation: append-only, never overwrite; failures are valuable experiences; clear human-machine division of labor; never let AI fabricate conclusions. These principles ensure the authenticity and traceability of experiment records, making AI a true asset for research reflection rather than a liability. According to the content creator's preview, the next section will cover "extremely important" prompt engineering—well worth following.