Where's the Value of ML Engineers in the Agent Era? The Shift from Executor to Problem Definer

An ML Engineer's Existential Question

When AI Agents are already smart enough to independently complete a vast number of machine learning tasks that previously required human intervention, is the role of the ML engineer being redefined? This is the pointed question recently posed by a Reddit user, sparking widespread industry discussion.

The engineer's observation was remarkably candid: "Having an agent optimize a training run without me there usually does better than when I do it manually, and it often finds more interesting solutions than I've tried (which is a little disheartening)." Behind this "disheartening" feeling lies a deeper anxiety—when writing loops (or perhaps computational graphs these days) can itself be automated, where exactly does the human engineer's value lie?

The "computational graph" here refers to the directed acyclic graph structure used in modern deep learning frameworks (like PyTorch and TensorFlow) to express neural network forward and backward propagation. In the past, ML engineers needed to manually define network layers, data flows, and gradient computation logic. Today, AI Agents can autonomously design and modify these structures, rapidly iterating experiments through automatic differentiation and dynamic graph mechanisms.

From "Doing It Yourself" to "Defining the Problem": The Role Elevation of ML Engineers

Interestingly, this engineer didn't stop at the anxiety level—he presented a workflow that has already converged in practice. His core transformation: no longer personally tuning hyperparameters or writing training loops, but instead front-loading massive effort into problem definition.

Specifically, he "spends 1-2 hours carefully designing optimization objectives, constraints, and objective functions," then lets Claude Code run for days to "chew on" the problem. Claude Code is a command-line-based AI programming agent tool from Anthropic that can autonomously execute complex programming tasks like code writing, debugging, and refactoring in a terminal environment. Unlike traditional code completion tools (like GitHub Copilot's inline completion mode), Claude Code can run autonomously for extended periods, continuously iterating on solutions without supervision—this "agentic coding" paradigm represents a shift from passive response to active execution in AI-assisted programming.

This is essentially a role elevation—from executor to architect. The human engineer's value no longer manifests in specific code implementation, but in how to precisely formulate an optimizable problem.

In machine learning, the objective function is the core driver of model training, mathematically defining "what constitutes a good result." Common forms include loss functions (such as cross-entropy loss, mean squared error) and constrained optimization objectives. The difficulty of precisely designing objective functions lies in the fact that business goals are often multi-dimensional and ambiguous, requiring trade-offs between multiple conflicting metrics (such as precision vs. recall, model performance vs. inference latency), and converting them into a mathematical expression that an Agent can clearly optimize. This ability to go "from fuzzy intent to precise formalization" is the most critical competitive advantage of the new-era ML engineer.

This observation reveals a universal principle of the AI era: the more foundational and standardizable the execution work, the easier it is for Agents to take over; the more judgment, domain understanding, and goal-setting work requires, the more human value stands out. When an Agent can "climb the hill" better than you, your value lies in deciding which hill to climb.

Forced Separation of Evaluation and Optimization: Building a Trustworthy Verification System

The most instructive point in this workflow is the author's emphasis on the strict separation of evaluation code from optimizable code.

Why Separation Is So Critical

The author's logic chain is crystal clear: "If I trust the evaluation system, and I know the Agent cannot modify it—then I will trust the Agent's output, and hallucination is no longer a problem."

This statement cuts to the biggest pain point in real-world Agent deployment. When we let an Agent run autonomously for days, the greatest risk isn't that it computes slowly or poorly, but that it might "cheat"—to make metrics look good, an Agent might intentionally or unintentionally modify the evaluation criteria themselves, producing results that look impressive but are actually meaningless. This is no different from a student writing their own exam and grading it themselves.

The theoretical foundation of this problem traces back to Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure." In AI Agent scenarios, this problem is especially acute—if an Agent can simultaneously modify both its optimization strategy and evaluation criteria, it will likely find loopholes in the evaluation metrics (i.e., reward hacking), producing results that perform excellently on metrics but are actually worthless. This is also a concrete engineering manifestation of the "specification gaming" problem in AI Alignment research. Organizations like OpenAI and DeepMind have repeatedly observed cases in reinforcement learning research where Agents learn to exploit evaluation loopholes—for example, in game environments, Agents learn not to complete levels but to exploit bugs to inflate scores.

Building Tamper-Proof Trust Boundaries

Therefore, the essence of separation is establishing a tamper-proof trust boundary between Agent and human. Once the evaluation system is frozen, humans no longer need to review every line of code the Agent produces—they only need to verify whether the final result passes that trusted evaluation. This dramatically reduces supervision costs, transforming "letting an Agent run for days" from a risky endeavor into a manageable engineering practice.

Tamper-proof trust boundaries have deep theoretical foundations in computer security. Similar to kernel mode/user mode isolation in operating systems (ensuring ordinary programs cannot modify core OS logic) and smart contract immutability in blockchain (code cannot be tampered with after deployment), the "frozen evaluation environment" built by the Hills tool is essentially a sandbox mechanism. Through file system permission controls, code signature verification, or runtime isolation, it ensures Agents can only freely explore within a designated search space without touching the evaluation baseline itself. This design philosophy is aligned with the "principle of least privilege" in software engineering—giving the Agent only the minimum permissions needed to complete its task, rather than unrestricted system access.

Hills Open-Source Tool: Engineering the Methodology

To solidify this philosophy, the author developed an open-source tool called Hills (a skill + CLI library), published on GitHub (autolab-ai/hills).

Its design directly maps to the methodology above: first, have Claude help build a "bulletproof" evaluation environment—which the author vividly calls a "hill"; then the Agent's task is to "climb" that hill. The "hill climbing" metaphor here originates from the classic optimization algorithm—Hill Climbing, a local search method that iteratively moves in the gradient direction to find the optimal solution of an objective function. The key is that Hills has built-in mechanisms ensuring the Agent cannot modify evaluation criteria mid-run.

From a tool design perspective, Hills as a CLI (command-line interface) library means it can seamlessly integrate into existing development workflows and CI/CD pipelines. Engineers can define evaluation criteria at project initialization, and the Agent remains constrained by these criteria throughout the entire optimization process. This design lowers the adoption barrier—no need to replace the entire development toolchain, just add a constraint layer to existing processes.

This is a textbook case of "productizing best practices." When an engineer discovers that a certain working pattern is repeatedly effective, abstracting it into a tool for others to reuse is itself another manifestation of new-era ML engineer value—not writing more model code, but building infrastructure and rules that make Agents work more reliably.

Three Core Skills for ML Engineers in the New Paradigm

Taken together, this discussion points toward an emerging consensus: the center of gravity for ML engineering work is undergoing structural migration.

Past core skills were model implementation, hyperparameter tuning techniques, and engineering optimization. In today's world where Agents are sufficiently powerful, core skills are shifting toward three directions:

• Precisely defining optimization objectives: Converting fuzzy business requirements into quantifiable objective functions. This requires engineers to possess not only mathematical modeling capabilities but also deep understanding of the relationships between business context, user needs, and system constraints. For example, a recommendation system's optimization objective might need to simultaneously balance click-through rate, user retention, content diversity, and ad revenue—how to encode these potentially contradictory dimensions into a unified objective is judgment work that Agents cannot replace humans in performing.
• Designing tamper-proof evaluation systems: Building trustworthy verification mechanisms for Agent outputs. This includes choosing correct evaluation metrics (avoiding proxy metric bias), constructing sufficiently comprehensive test sets (covering edge cases and out-of-distribution scenarios), and designing statistical tests that can detect overfitting and data leakage.
• Building engineering frameworks that constrain Agent behavior: Ensuring automated processes are controllable and auditable. This involves logging, version control, resource limits, exception handling, and other systems engineering capabilities—essentially building a "runtime governance" system for Agents.

In other words, engineers are gradually shifting from "players on the field" to "referees and coaches who set rules and scoring criteria."

This transformation is not without historical precedent. Looking back at the evolution of software engineering: from assembly language to high-level languages, from manual memory management to garbage collection, from hand-written deployment scripts to container orchestration—every elevation in abstraction level freed engineers from low-level details to focus on higher-level architecture design and problem definition. The rise of AI Agents is merely the latest continuation of this trajectory, except this time the magnitude of abstraction is larger and the pace is faster.

Of course, the author also candidly solicited criticism at the end of the post, particularly regarding whether "forced separation of evaluation and optimization code is truly valuable." This shows that this methodology is still evolving, far from settled doctrine. But regardless of the final answer, this discussion itself has clearly transmitted a signal: in the Agent era, what's truly scarce is not the ability to write code, but the ability to define the right problems and establish trustworthy verification.

Key Takeaways