UnFlow: Replacing Flat Lists with Graph-Based ML Experiment Tracking

The Persistent Problem with ML Experiment Tracking

Anyone who has done machine learning research has encountered this scenario: to tune a model, you repeatedly modify learning rates, adjust training epochs, swap model architectures, optimize preprocessing code... After a few weeks, your experiment logs contain hundreds of runs.

But traditional experiment tracking tools (like common experiment management platforms) typically present these runs as a flat list:

run_001
run_002
run_003
run_004
...

The ML experiment tracking space has evolved over nearly a decade, producing a tool ecosystem led by MLflow, Weights & Biases (W&B), Neptune.ai, Comet ML, and others. MLflow, open-sourced by Databricks, is currently the most widely used experiment management framework, recording parameters, metrics, and model artifacts with the "run" as its basic unit. W&B is known for its excellent visualization and team collaboration features. These tools already do a remarkable job at "recording results from a single run," but their modeling of "how experiments relate to and evolve from each other" remains relatively weak — they are essentially flat databases with powerful filtering and grouping capabilities.

Recently, a developer shared his open-source project UnFlow (GitHub: UnFlow-Labs/mlunflow) on Reddit, attempting to fundamentally change this tracking paradigm. He highlighted a pain point overlooked by most researchers: experiments are not independent of each other — they have evolutionary relationships.

Why Flat Lists Fall Short for Experiment Tracking

The author argues that storing experiments as a pile of independent records makes certain critical questions extremely difficult to answer:

• What exactly changed between two experiments?
• Which experiments are essentially identical computations?
• Have I already run this experiment before?
• How did I evolve from experiment A to experiment B?
• Can I navigate the entire experiment lineage like browsing history, instead of searching through a heap of runs?

What these questions have in common is that they don't concern the isolated results of a single run — they concern the relationships and lineage between runs. When experiment counts balloon to hundreds, repeated experiments, redundant computations, and hard-to-trace changes become invisible killers of research efficiency.

Notably, the concept of "lineage" is already deeply rooted in the data engineering domain. Data lineage tracks the complete transformation path of data from raw sources to final reports and serves as foundational infrastructure for data governance and compliance auditing. But during the exploratory phase of ML experiments — where researchers frequently modify code and parameters in rapid iteration — lineage tracking has long lacked convenient tooling.

UnFlow's Core Idea: Modeling Experiments as a Graph

UnFlow's proposed solution is a fairly intuitive abstraction — model experiments as a graph.

This idea doesn't come from nowhere. Modeling change history as a graph has successful precedent in software engineering — Git version control is based on a directed acyclic graph (DAG) to manage code commit history. Each commit is a node, parent-child relationships form edges, and branching and merging operations are naturally expressed on the graph. Developers can visualize a project's evolution path via git log --graph and examine precise differences between any two versions with git diff. UnFlow's design philosophy can be understood as "bringing Git's version graph thinking into experiment management," but it faces more complex challenges: code changes in Git are text-level diffs, while experiment changes involve semantic judgments across multiple dimensions including parameters, data, and compute environments.

The Meaning of Nodes and Edges

In this experiment graph:

• Nodes represent a "state" — the specific configuration and computed results of a given experiment;
• Edges represent "transformations" — what changed between one state and another.

In other words, the graph structure itself encodes the evolutionary path of experiments. You no longer need to manually record "this time I changed the learning rate from 0.01 to 0.001" — that change naturally becomes an edge connecting two nodes.

Automatic Detection of Code and Parameter Changes

UnFlow's implementation mechanism works by monitoring code changes within a Python function and parameter changes passed to that function, using these to construct the state graph. A key design decision is that a new state node is only created and computation only executed when a real transformation exists.

This means if an experiment is completely equivalent to an existing node (neither code nor parameters have changed), UnFlow can identify it as a duplicate computation and avoid redundant execution. This directly addresses researchers' concern of "have I already run this experiment?"

The deduplication mechanism draws on content-addressable thinking — a storage strategy that uses data content hash values as unique identifiers. This pattern is widely used in Git object storage (where every file, directory, and commit is uniquely identified by its content's SHA-1 hash), IPFS distributed file systems, and Docker image layer management. The core logic is: if two pieces of data have identical content, their hash values must be identical, so you only need to store and execute once. UnFlow applies this concept to experiment states — by hashing the combination of function code and parameters, it determines whether two experiment invocations are essentially equivalent, achieving automatic deduplication and saving researchers precious GPU compute and time.

The author also candidly acknowledges the current limitations: it currently only supports tracking changes to a single function, not an entire codebase. This is a reasonable trade-off for an early-stage project.

Experiment Lineage Tracking: An Abstraction Worth Discussing

The most valuable aspect of UnFlow may not be the code implementation itself, but the mental framework it proposes. The author explicitly states in his post that the project is still in its early stages, and he values feedback more than packaging it as a finished product. He poses three open questions:

1. Does the "experiments as a graph" abstraction make sense? For researchers accustomed to linear run lists, can a graph structure better map the real experimental process?
2. Are you currently troubled by duplicate or redundant experiments? This is key to validating whether the need is real.
3. If you could see complete experiment lineage, what would you most want to query or visualize?

These questions touch on a long-standing but not yet elegantly solved issue in the MLOps space: the traceability of experiment lineage. Existing tools are mostly good at recording "results" but not at expressing "process" and "causality."

From a broader industry context, data lineage and model lineage are advanced capabilities in MLOps maturity models. In enterprise ML systems, regulatory compliance requirements (such as the EU's upcoming AI Act, and financial industry model risk management standards like SR 11-7) often require organizations to fully trace a model's training data sources, preprocessing steps, and hyperparameter decision chains. Tools like Apache Airflow, Kubeflow Pipelines, and DVC (Data Version Control) provide some lineage recording capabilities at the pipeline level, but they primarily focus on "auditability of production pipelines" — the execution history of an already-defined workflow. The defining characteristic of the research exploration phase is that the workflow itself is constantly changing, and researchers don't know what the final pipeline will look like. Tracking the "experimental evolution path during the exploration phase" is precisely the gap UnFlow aims to fill.

How UnFlow Differs from Existing Experiment Tracking Tools

From an industry perspective, ML experiment tracking is far from a new concept — mature commercial and open-source solutions exist. But their core models are mostly run-centric: each run is a record with attached metrics, parameters, and artifacts.

UnFlow attempts to shift toward a lineage-centric model. The potential value of this shift includes:

• Deduplication capability: Through content-addressable state determination, it naturally avoids redundant computation and saves compute resources;
• Navigability: It transforms the exploration process into a visualizable path, making it easy to trace back "why was this decision made";
• Reproducibility: The graph structure clearly records every transformation from initial state to current result.

Of course, challenges are equally apparent. The single-function limitation means it currently struggles to cover complex multi-stage pipelines; accurately detecting "semantic equivalence" rather than "literal sameness" in code without intruding on researchers' workflows is also a technical challenge.

Determining whether two pieces of code are "semantically equivalent" is a problem with theoretical depth in computer science. Strictly speaking, deciding whether two programs compute the same function is an undecidable problem (formally proven by Rice's theorem). But in engineering practice, multiple approximate approaches exist: abstract syntax tree (AST)-based comparison can ignore variable renaming, whitespace, and comment differences, focusing on code structure; execution trace-based dynamic equivalence detection judges by observing whether programs produce the same output given the same input; for pure functions (functions without side effects), deterministic input-output comparison is the most reliable method. UnFlow likely employs AST-level code normalization combined with parameter hashing — this is practical enough for the typical scenario of "a researcher modified a hyperparameter," but still carries risk of misjudgment for functions involving random seeds, global state, file I/O, or network calls. Finding the balance between practicality and accuracy will be one of the core technical challenges as the project evolves.

Conclusion: Early Project, Real Problem

UnFlow is currently a very early-stage open-source exploration, far from polished. But the problem it raises is real and universal — almost every researcher who has done extensive experiment tuning has gotten lost among hundreds of runs.

Re-imagining experiments from "a flat table" to "a directed evolution graph" — this perspective alone is worth discussion across the community. For developers concerned with ML engineering efficiency and experiment reproducibility, UnFlow may be a project worth following and providing feedback on.

Project repository: github.com/UnFlow-Labs/mlunflow