How a Hugging Face Engineer Automated His Team's Entire Workflow with AI Agents

In the AI boom, hundreds of research papers emerge daily, yet countless model weights remain scattered across Google Drive, Dropbox, and other third-party platforms with terrible discoverability. In a recent tech talk, Niels, a machine learning engineer at Hugging Face, shared in detail how he used AI Agents to fully automate the core work of his Community Science Team. This isn't just an engineering case study—it reflects the architectural evolution from "Workflow to Agent."

From Manual Outreach to Automation: The Starting Point

Niels has been at Hugging Face for five years. His Community Science Team has a nickname—"the team that moves Google Drives onto the Hub." Behind this slightly tongue-in-cheek name lies a real pain point.

Researchers habitually store model weights on GitHub Releases, Dropbox, Zenodo, and similar platforms, making these artifacts hard to search and difficult to reproduce. Hugging Face offers a complete solution: paper pages automatically pulled from arXiv, with associated models and datasets linked on the side; the platform provides metadata tags and filters so users can quickly search by language, task type, compatible library, and more.

Hugging Face Hub plays a role in the machine learning ecosystem similar to GitHub for code. As of 2024, it hosts over 700,000 models, 150,000 datasets, and 250,000 Spaces applications. Its core value lies not just in storage but in standardization: through unified Model Card specifications, researchers must declare training data, evaluation metrics, known limitations, and ethical considerations, dramatically improving model reproducibility and auditability. The Hub also deeply integrates with mainstream open-source libraries like Transformers, Diffusers, and PEFT, allowing users to load any model with a single line of code.

For researchers, moving their work to the Hub is a win-win: it boosts visibility while improving documentation quality (through Model Cards and Dataset Cards). So Niels's daily routine used to involve constantly opening Issues on GitHub using nearly identical templates, suggesting authors upload their work and submitting PRs to complete documentation.

But the problem was obvious: manually opening Issues and submitting PRs one by one simply doesn't scale. Facing the flood of papers appearing on arXiv daily, human effort couldn't keep up.

Phase One: Replicating the Manual Process with Deterministic Workflows

Facing the scaling challenge, Niels first had to answer an architectural question: should he build a more deterministic Workflow, or a fully autonomous Agent?

He referenced the classic Anthropic blog post Building Effective Agents: Workflows invoke LLMs along a predefined pipeline—more predictable and controllable but less flexible; fully autonomous Agents have the LLM calling tools in a loop until the task is complete—flexible but unpredictable.

Anthropic's late-2024 blog post drew a clear distinction between LLM application architectures: in Workflows, control belongs to the code—developers define what to do at each step, how to branch, and when to terminate; in Agents, control belongs to the model—the model decides what tool to call next, whether more information is needed, and when to declare the task complete. The tradeoff is fundamentally one of determinism versus flexibility. Anthropic explicitly recommended at the time: if a problem can be solved with a Workflow, don't use an Agent, because determinism is stronger, debugging is easier, and costs are more controllable.

Since the project started during the period when Anthropic advised "don't build an Agent unless necessary, start simple, avoid frameworks where possible," Niels chose a deterministic Workflow. He fully replicated his manual outreach process:

• Find the paper's GitHub link
• Read the repository README to identify artifacts shareable on the Hub
• Check whether Model/Dataset Cards are complete and metadata tags are present
• Submit PRs for incomplete cards, open Issues for missing uploads
• Follow up on author replies

On the technical implementation side, he used no Agent framework—each step directly called the LLM API, ensuring determinism and controllability. The deployment was extremely simple—just a Cron Job running for free via GitHub Actions, executing once every night.

Cron Job is the classic scheduled task mechanism in Unix systems, allowing scripts to run on preset schedules (e.g., "every day at 2 AM"). GitHub Actions is GitHub's CI/CD service, offering 2,000 free minutes per month for public repositories. Combining the two, developers can achieve zero-cost scheduled automation: by configuring a schedule trigger in the workflow YAML, GitHub spins up a temporary container at the designated time to execute the specified script. The advantages are no server maintenance and zero cost; the downsides are a 6-hour maximum runtime per execution and no persistent state support.

As Niels put it: "While I sleep, an Agent is doing my work—but strictly speaking, it's just a Python script running LLM API calls."

For observability, he uses Langfuse for tracing, monitoring LLM inputs/outputs, costs, and latency. Langfuse is an open-source LLM application observability platform, similar to traditional APM (Application Performance Monitoring) tools but designed specifically for large language model scenarios. It tracks the complete chain of each LLM call: the input prompt, output response, token consumption, response latency, and intermediate states in multi-step reasoning. For a system automatically processing hundreds of Issues daily, observability is crucial—it helps identify when the model produces inappropriate outputs and which types of repositories tend to cause misjudgments, enabling continuous optimization of prompts and process logic.

Today, hundreds of GitHub Issues are automatically created every night.

Phase Two: Shifting to Autonomous Agents for Handling Replies

After automating Issue creation, a new problem emerged: author replies piled up into a sea of unread notifications, and responding to each one became its own burden—"kind of like clearing your inbox every day."

A few months ago, Niels automated the "follow-up replies" step as well. This time he made a different architectural choice—a fully autonomous Agent.

The reason for the shift is worth noting. At a Workshop at the AI Engineer conference, Anthropic's stance showed a subtle change: they began stating that "models are now good enough to use fully autonomous Agents, rather than being limited to Workflows." The Cursor team also shared that they replaced 12,000 lines of complex custom Workflow code with skills of only 200 lines each. This shift reflects the leap in LLM capabilities between 2024 and 2025—once model reasoning ability and tool-calling reliability reach a certain threshold, hard-coding complex logic becomes a burden rather than an advantage, and it's better to let the model decide autonomously.

The tech stack for this autonomous Agent is remarkably lean:

• Core: Claude Agent SDK (recently switched to open-source models like GLM-4.6 via Hugging Face Inference Providers)
• Tools: Primarily Bash, combined with Hugging Face CLI skills for executing relevant commands
• Deployment: Modal, leveraging its batch processing capabilities to spin up many containers in parallel, each container being an Agent loop dedicated to processing a single Issue
• Output: Follow-up comments on GitHub, with results posted to a Slack channel

Claude Agent SDK is Anthropic's lightweight Agent development tool, with the core philosophy of letting the LLM autonomously call tools in a loop until the task is complete—developers only need to define available tools and stop conditions. Modal is a Serverless compute platform optimized for batch processing and parallel computation—developers define functions with Python decorators, and Modal automatically packages them into containers with elastic scaling. The combined power is that each GitHub Issue's processing is encapsulated as an independent Agent instance, and Modal can spin up dozens or even hundreds of containers simultaneously for parallel processing, achieving high-throughput automated outreach while only charging for actual compute time.

Niels specifically mentioned that after switching to the open-source GLM series, its performance on benchmarks like Post-Training Bench even surpassed Claude Opus at a lower price—"you really can't find a reason not to use it." The GLM (General Language Model) series is developed by Zhipu AI, with GLM-4.6 being its latest generation. Post-Training Bench focuses on evaluating models' practical performance after alignment techniques like RLHF and DPO, including instruction following, tool calling, and multi-turn dialogue. Between 2024 and 2025, the competitive landscape for open-source models underwent fundamental change: DeepSeek^Ad, Qwen, GLM, Llama, and other series matched or surpassed closed-source models like GPT-4 and Claude on multiple benchmarks, typically at an order of magnitude lower inference cost. Through Hugging Face Inference Providers, developers can use these open-source models as easily as calling an API while retaining the flexibility to switch at any time.

This autonomous Agent only needs a CLI, a skill, and a shell environment to complete all its work.

Real Results: Win-Win and the "Dead Internet" Phenomenon

The system has delivered impressive results. Niels shared several representative cases:

• An Apple researcher proactively reached out in response to the Agent's suggestion to publish artifacts from an Apple paper
• The Agent successfully prompted Google DeepMind to release a mathematics dataset
• The Chinese company PaddleOCR migrated all their OCR models to Hugging Face
• The most viral case was the Tiny Recursive Models paper—the Issue received 60+ upvotes, and the models were ultimately published

Out of the thousands of Issues created, Niels has received only two negative comments. The overwhelming majority of authors react with: "Oh right, putting the weights on Hugging Face makes total sense—why didn't I think of that?"

An interesting detail: Niels hasn't disclosed that these Issues are sent by an Agent. His concern is practical—if authors know they're interacting with a bot, they might simply close the Issue. And since the Agent's content is identical to what he would write manually, he sees no reason to explicitly state it. Even more interesting, he frequently sees people using their own Agents to reply to his Agent—a scenario that indeed has a "Dead Internet" flavor. "Dead Internet Theory" was originally a conspiracy theory claiming most internet content is already AI-generated. As Agent systems proliferate, this once-fringe theory is partially becoming reality—when AI Agents begin interacting with each other, humans gradually recede into the background of certain online conversations.

How to Prevent Agents from Creating "Slop"

Facing the criticism that "an Agent mass-creating Issues across the web amounts to generating spam," Niels strongly recommends Hamel Husain's blog post LLM Evals FAQ, arguing that evaluation is the key to preventing Agents from becoming slop machines.

Hamel Husain is a well-known practitioner in AI engineering who previously worked on the GitHub Copilot team and later focused on evaluation methodology for LLM applications. His core argument in LLM Evals FAQ is that evaluation isn't a one-time task but should be a continuous iterative loop. Specific methods include: building evaluation sets from real user feedback, using LLM-as-Judge for automated scoring, and setting up regression tests to prevent prompt modifications from degrading quality. For Agent systems, evaluation is especially critical—since Agent outputs have higher uncertainty, without systematic evaluation, it's easy to produce large volumes of low-quality or even harmful output at scale. "Slop" in the AI community specifically refers to valueless AI-generated content garbage, similar to early internet SEO spam but at far greater scale and speed.

"Don't forget to do evaluations" is the conclusion Niels repeatedly emphasized.

Beyond the core automation system, the Community Science Team has other experiments. Niels runs a Twitter account called "daily papers" using the same Workflow to automatically post interesting papers—it's grown to over 90,000 followers with zero manual involvement, even using Gemini to select accompanying images. He's also currently attempting to revive the Papers with Code website that Meta shut down, hoping to make research artifacts and the latest SOTA progress more accessible.

Engineering Philosophy for the Agent Era

Niels's talk offers several highly practical conclusions:

First, open-source models are now strong enough. The GLM series, DeepSeek, and other open-source models are fully viable replacements for closed-source models in real production scenarios, with clear cost-performance advantages.

Second, Agents are gradually surpassing Workflows. As model capabilities continue to improve, simple autonomous Agents equipped with a CLI and skills can replace thousands of lines of custom code. The underlying logic of this trend is: when model reasoning reliability exceeds a certain threshold, encoding business logic in prompts and tool definitions is more maintainable and extensible than hard-coding it in conditional branches.

Third, start simple and don't forget evaluation. Regardless of which architecture you choose, observability and evaluation are the foundation for keeping systems running reliably over time.

The core takeaway from this case is that truly effective AI automation doesn't necessarily require complex frameworks or flashy architectures. A Cron Job, a CLI, an evaluation mechanism, combined with increasingly powerful model capabilities—that's enough to scale a team's core work to internet-wide levels.